2024-04-24T08:40:10.930 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T08:40:10.930 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} --name rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219 -- /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219/orig.config.yaml 2024-04-24T08:40:10.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T08:40:11.087 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219 branch: rishabh-main-apr17-a654945 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7671219' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-main-apr17-a654945 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-main-apr17-a654945 sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7954 sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: rishabh-main-apr17-a654945 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi069.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMTekBTAJqh8wjIKHR5OeiLPqHY0gpHPKpuaKOns8SdwT/9OKNKbSaAlp+a2yYZff1hW/3YAqpIbFrL2RxJEsDA= smithi081.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHPYstxFn2yzqm8aijm4VdNHZ2Zm1zGkPkXcj8JTJUD+UotBcYC47p9JDaP4+eW0AsbJRVUlc24TUeqCXe5PYuY= smithi130.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLDmN90dI9z7y6K6jl3Pa8evR6qPIeIb6Qvf2Zf8iOAQQOdC+qA0X8neB1mDK31WocyDvMaoXme0yEGVs5cUL5w= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_07:32:23 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T08:40:11.087 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa; will attempt to use it 2024-04-24T08:40:11.088 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks 2024-04-24T08:40:11.088 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T08:40:11.090 INFO:teuthology.task.internal:Checking packages... 2024-04-24T08:40:11.114 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1f81fb397ae98da7563d451a78c61574c8f4e6e0' 2024-04-24T08:40:11.114 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T08:40:11.114 INFO:teuthology.packaging:ref: None 2024-04-24T08:40:11.114 INFO:teuthology.packaging:tag: None 2024-04-24T08:40:11.114 INFO:teuthology.packaging:branch: rishabh-main-apr17-a654945 2024-04-24T08:40:11.114 INFO:teuthology.packaging:sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:40:11.115 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-main-apr17-a654945 2024-04-24T08:40:11.392 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3116.g1f81fb39 2024-04-24T08:40:11.393 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T08:40:11.400 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T08:40:11.401 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T08:40:11.408 INFO:teuthology.task.internal:Saving configuration 2024-04-24T08:40:11.427 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T08:40:11.433 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T08:40:11.457 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:35:17.219894', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMTekBTAJqh8wjIKHR5OeiLPqHY0gpHPKpuaKOns8SdwT/9OKNKbSaAlp+a2yYZff1hW/3YAqpIbFrL2RxJEsDA='} 2024-04-24T08:40:11.479 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:35:17.221574', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHPYstxFn2yzqm8aijm4VdNHZ2Zm1zGkPkXcj8JTJUD+UotBcYC47p9JDaP4+eW0AsbJRVUlc24TUeqCXe5PYuY='} 2024-04-24T08:40:11.501 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:35:17.223487', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLDmN90dI9z7y6K6jl3Pa8evR6qPIeIb6Qvf2Zf8iOAQQOdC+qA0X8neB1mDK31WocyDvMaoXme0yEGVs5cUL5w='} 2024-04-24T08:40:11.501 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T08:40:11.507 INFO:teuthology.task.internal:roles: ubuntu@smithi069.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-24T08:40:11.507 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-24T08:40:11.507 INFO:teuthology.task.internal:roles: ubuntu@smithi130.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-24T08:40:11.507 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T08:40:11.602 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ffa8efb9160>, signals=[15]) 2024-04-24T08:40:11.603 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T08:40:11.608 INFO:teuthology.task.internal:Opening connections... 2024-04-24T08:40:11.608 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi069.front.sepia.ceph.com 2024-04-24T08:40:11.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:40:11.688 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi081.front.sepia.ceph.com 2024-04-24T08:40:11.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:40:11.760 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi130.front.sepia.ceph.com 2024-04-24T08:40:11.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:40:11.832 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T08:40:11.839 DEBUG:teuthology.orchestra.run.smithi069:> uname -m 2024-04-24T08:40:11.857 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2024-04-24T08:40:11.857 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2024-04-24T08:40:11.917 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream" 2024-04-24T08:40:11.917 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="9" 2024-04-24T08:40:11.917 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2024-04-24T08:40:11.917 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2024-04-24T08:40:11.917 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="9" 2024-04-24T08:40:11.918 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el9" 2024-04-24T08:40:11.918 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T08:40:11.918 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2024-04-24T08:40:11.918 INFO:teuthology.orchestra.run.smithi069.stdout:LOGO="fedora-logo-icon" 2024-04-24T08:40:11.918 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T08:40:11.918 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/" 2024-04-24T08:40:11.918 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T08:40:11.918 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T08:40:11.918 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T08:40:11.919 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2024-04-24T08:40:11.952 DEBUG:teuthology.orchestra.run.smithi081:> uname -m 2024-04-24T08:40:11.970 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2024-04-24T08:40:11.970 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release 2024-04-24T08:40:12.026 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="CentOS Stream" 2024-04-24T08:40:12.026 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="9" 2024-04-24T08:40:12.027 INFO:teuthology.orchestra.run.smithi081.stdout:ID="centos" 2024-04-24T08:40:12.027 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="rhel fedora" 2024-04-24T08:40:12.027 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="9" 2024-04-24T08:40:12.027 INFO:teuthology.orchestra.run.smithi081.stdout:PLATFORM_ID="platform:el9" 2024-04-24T08:40:12.027 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T08:40:12.027 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2024-04-24T08:40:12.027 INFO:teuthology.orchestra.run.smithi081.stdout:LOGO="fedora-logo-icon" 2024-04-24T08:40:12.027 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T08:40:12.027 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://centos.org/" 2024-04-24T08:40:12.027 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T08:40:12.027 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T08:40:12.027 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T08:40:12.028 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2024-04-24T08:40:12.052 DEBUG:teuthology.orchestra.run.smithi130:> uname -m 2024-04-24T08:40:12.070 INFO:teuthology.orchestra.run.smithi130.stdout:x86_64 2024-04-24T08:40:12.070 DEBUG:teuthology.orchestra.run.smithi130:> cat /etc/os-release 2024-04-24T08:40:12.127 INFO:teuthology.orchestra.run.smithi130.stdout:NAME="CentOS Stream" 2024-04-24T08:40:12.128 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION="9" 2024-04-24T08:40:12.128 INFO:teuthology.orchestra.run.smithi130.stdout:ID="centos" 2024-04-24T08:40:12.128 INFO:teuthology.orchestra.run.smithi130.stdout:ID_LIKE="rhel fedora" 2024-04-24T08:40:12.128 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION_ID="9" 2024-04-24T08:40:12.128 INFO:teuthology.orchestra.run.smithi130.stdout:PLATFORM_ID="platform:el9" 2024-04-24T08:40:12.128 INFO:teuthology.orchestra.run.smithi130.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T08:40:12.128 INFO:teuthology.orchestra.run.smithi130.stdout:ANSI_COLOR="0;31" 2024-04-24T08:40:12.128 INFO:teuthology.orchestra.run.smithi130.stdout:LOGO="fedora-logo-icon" 2024-04-24T08:40:12.128 INFO:teuthology.orchestra.run.smithi130.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T08:40:12.128 INFO:teuthology.orchestra.run.smithi130.stdout:HOME_URL="https://centos.org/" 2024-04-24T08:40:12.128 INFO:teuthology.orchestra.run.smithi130.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T08:40:12.128 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T08:40:12.128 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T08:40:12.129 INFO:teuthology.lock.ops:Updating smithi130.front.sepia.ceph.com on lock server 2024-04-24T08:40:12.152 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T08:40:12.162 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T08:40:12.167 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T08:40:12.168 DEBUG:teuthology.orchestra.run.smithi069:> test '!' -e /home/ubuntu/cephtest 2024-04-24T08:40:12.170 DEBUG:teuthology.orchestra.run.smithi081:> test '!' -e /home/ubuntu/cephtest 2024-04-24T08:40:12.173 DEBUG:teuthology.orchestra.run.smithi130:> test '!' -e /home/ubuntu/cephtest 2024-04-24T08:40:12.189 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T08:40:12.195 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T08:40:12.195 DEBUG:teuthology.orchestra.run.smithi069:> test -z $(ls -A /var/lib/ceph) 2024-04-24T08:40:12.231 DEBUG:teuthology.orchestra.run.smithi081:> test -z $(ls -A /var/lib/ceph) 2024-04-24T08:40:12.234 DEBUG:teuthology.orchestra.run.smithi130:> test -z $(ls -A /var/lib/ceph) 2024-04-24T08:40:12.346 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T08:40:12.419 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T08:40:12.434 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-24T08:40:12.434 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-24T08:40:12.434 DEBUG:teuthology.orchestra.run.smithi069:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T08:40:12.435 DEBUG:teuthology.orchestra.run.smithi081:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T08:40:12.435 DEBUG:teuthology.orchestra.run.smithi130:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T08:40:12.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:40:12.452 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:40:12.452 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T08:40:12.452 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-04-24T08:40:12.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:40:12.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:40:12.454 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:40:12.454 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T08:40:12.454 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2024-04-24T08:40:12.454 DEBUG:teuthology.task.kernel:sha1 for mon.c is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:40:12.455 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T08:40:12.455 DEBUG:teuthology.orchestra.run.smithi130:> uname -r 2024-04-24T08:40:12.509 INFO:teuthology.orchestra.run.smithi081.stdout:5.14.0-437.el9.x86_64 2024-04-24T08:40:12.509 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:40:12.509 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T08:40:12.509 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:40:12.514 INFO:teuthology.orchestra.run.smithi130.stdout:5.14.0-437.el9.x86_64 2024-04-24T08:40:12.514 INFO:teuthology.orchestra.run.smithi069.stdout:5.14.0-437.el9.x86_64 2024-04-24T08:40:12.514 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:40:12.514 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T08:40:12.514 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:40:12.515 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:40:12.515 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T08:40:12.516 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:40:12.647 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum upgrade -y linux-firmware 2024-04-24T08:40:12.670 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum upgrade -y linux-firmware 2024-04-24T08:40:12.792 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum upgrade -y linux-firmware 2024-04-24T08:40:15.157 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-24T08:40:15.393 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 7.9 MB/s | 8.0 MB 00:01 2024-04-24T08:40:15.849 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.0 MB 00:01 2024-04-24T08:40:17.485 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-24T08:40:17.516 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-24T08:40:17.575 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-04-24T08:40:21.731 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-24T08:40:22.293 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 5.4 MB/s | 6.2 MB 00:01 2024-04-24T08:40:23.944 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 16 kB 00:00 2024-04-24T08:40:24.320 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-04-24T08:40:24.886 INFO:teuthology.orchestra.run.smithi130.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-04-24T08:40:25.898 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-24T08:40:29.935 INFO:teuthology.orchestra.run.smithi130.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-24T08:40:30.125 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:00:01 ago on Wed 24 Apr 2024 08:40:29 AM UTC. 2024-04-24T08:40:30.920 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-24T08:40:31.930 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-24T08:40:31.931 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-04-24T08:40:31.931 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-24T08:40:31.976 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 584 kB/s | 6.2 MB 00:10 2024-04-24T08:40:32.030 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.c... 2024-04-24T08:40:32.030 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:40:32.178 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T08:40:32.178 DEBUG:teuthology.orchestra.run.smithi130:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T08:40:32.180 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi130.front.sepia.ceph.com to complete... 2024-04-24T08:40:32.902 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-04-24T08:40:32.903 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-04-24T08:40:32.903 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-04-24T08:40:33.025 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-24T08:40:33.025 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:40:33.242 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T08:40:33.243 DEBUG:teuthology.orchestra.run.smithi081:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T08:40:33.245 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi081.front.sepia.ceph.com to complete... 2024-04-24T08:40:35.287 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 8.5 kB/s | 16 kB 00:01 2024-04-24T08:40:37.423 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-24T08:40:42.487 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-04-24T08:40:44.469 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-04-24T08:40:44.470 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do. 2024-04-24T08:40:44.470 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-04-24T08:40:44.565 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-24T08:40:44.565 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:40:44.783 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T08:40:44.783 DEBUG:teuthology.orchestra.run.smithi069:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T08:40:44.785 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi069.front.sepia.ceph.com to complete... 2024-04-24T08:40:55.548 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24 08:40:55 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T08:40:55.548 INFO:teuthology.orchestra.run.smithi130.stderr:FINISHED --2024-04-24 08:40:55-- 2024-04-24T08:40:55.548 INFO:teuthology.orchestra.run.smithi130.stderr:Total wall clock time: 23s 2024-04-24T08:40:55.548 INFO:teuthology.orchestra.run.smithi130.stderr:Downloaded: 1 files, 341M in 23s (14.8 MB/s) 2024-04-24T08:40:55.551 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.c... 2024-04-24T08:40:55.551 DEBUG:teuthology.orchestra.run.smithi130:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T08:40:57.683 INFO:teuthology.orchestra.run.smithi130.stdout:Verifying... ######################################## 2024-04-24T08:40:57.818 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing... ######################################## 2024-04-24T08:40:57.941 INFO:teuthology.orchestra.run.smithi130.stdout:Updating / installing... 2024-04-24T08:41:05.918 INFO:teuthology.orchestra.run.smithi130.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T08:41:06.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24 08:41:06 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T08:41:06.150 INFO:teuthology.orchestra.run.smithi069.stderr:FINISHED --2024-04-24 08:41:06-- 2024-04-24T08:41:06.150 INFO:teuthology.orchestra.run.smithi069.stderr:Total wall clock time: 21s 2024-04-24T08:41:06.150 INFO:teuthology.orchestra.run.smithi069.stderr:Downloaded: 1 files, 341M in 21s (16.3 MB/s) 2024-04-24T08:41:06.152 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-24T08:41:06.153 DEBUG:teuthology.orchestra.run.smithi069:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T08:41:07.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24 08:41:07 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T08:41:07.931 INFO:teuthology.orchestra.run.smithi081.stderr:FINISHED --2024-04-24 08:41:07-- 2024-04-24T08:41:07.931 INFO:teuthology.orchestra.run.smithi081.stderr:Total wall clock time: 35s 2024-04-24T08:41:07.931 INFO:teuthology.orchestra.run.smithi081.stderr:Downloaded: 1 files, 341M in 34s (9.97 MB/s) 2024-04-24T08:41:07.955 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-24T08:41:07.955 DEBUG:teuthology.orchestra.run.smithi081:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T08:41:08.263 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying... ######################################## 2024-04-24T08:41:08.416 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing... ######################################## 2024-04-24T08:41:08.543 INFO:teuthology.orchestra.run.smithi069.stdout:Updating / installing... 2024-04-24T08:41:10.587 INFO:teuthology.orchestra.run.smithi081.stdout:Verifying... ######################################## 2024-04-24T08:41:10.728 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing... ######################################## 2024-04-24T08:41:10.841 INFO:teuthology.orchestra.run.smithi081.stdout:Updating / installing... 2024-04-24T08:41:16.498 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T08:41:18.782 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T08:41:42.868 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi130.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T08:41:42.868 DEBUG:teuthology.orchestra.run.smithi130:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T08:41:42.953 INFO:teuthology.orchestra.run.smithi130.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:42.953 INFO:teuthology.orchestra.run.smithi130.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:42.954 INFO:teuthology.orchestra.run.smithi130.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:42.954 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T08:41:42.954 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T08:41:42.954 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T08:41:42.954 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T08:41:42.954 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T08:41:42.954 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T08:41:42.954 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T08:41:42.954 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T08:41:42.954 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T08:41:42.954 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T08:41:42.954 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T08:41:42.954 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T08:41:42.955 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T08:41:42.955 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T08:41:42.955 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T08:41:42.955 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T08:41:42.955 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T08:41:42.955 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T08:41:42.955 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T08:41:42.955 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T08:41:42.955 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T08:41:42.955 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T08:41:42.955 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T08:41:42.955 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T08:41:42.955 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T08:41:42.955 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T08:41:42.956 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T08:41:42.956 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T08:41:42.956 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T08:41:42.956 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T08:41:42.956 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T08:41:42.956 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T08:41:42.956 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T08:41:42.956 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T08:41:42.956 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T08:41:42.956 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T08:41:42.956 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T08:41:42.956 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T08:41:42.956 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T08:41:42.956 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T08:41:42.956 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T08:41:42.957 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T08:41:42.957 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T08:41:42.957 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T08:41:42.957 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T08:41:42.957 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T08:41:42.957 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T08:41:42.957 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T08:41:42.957 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T08:41:42.957 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T08:41:42.957 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T08:41:42.957 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T08:41:42.957 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T08:41:42.957 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T08:41:42.957 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T08:41:42.958 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T08:41:42.958 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T08:41:42.958 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T08:41:42.958 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T08:41:42.958 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T08:41:42.958 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T08:41:42.958 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T08:41:42.958 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T08:41:42.958 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T08:41:42.958 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T08:41:42.958 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T08:41:42.958 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T08:41:42.958 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T08:41:42.958 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T08:41:42.959 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T08:41:42.959 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T08:41:42.959 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T08:41:42.959 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T08:41:42.959 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T08:41:42.959 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T08:41:42.959 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T08:41:42.959 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T08:41:42.959 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T08:41:42.959 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T08:41:42.959 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T08:41:42.959 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T08:41:42.959 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T08:41:42.959 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T08:41:42.959 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T08:41:42.960 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T08:41:42.960 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T08:41:42.960 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T08:41:42.960 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T08:41:42.960 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T08:41:42.960 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T08:41:42.960 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T08:41:42.960 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T08:41:42.960 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T08:41:42.960 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T08:41:42.960 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T08:41:42.960 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T08:41:42.960 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T08:41:42.960 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T08:41:42.960 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T08:41:42.961 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T08:41:42.961 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T08:41:42.961 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T08:41:42.961 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T08:41:42.961 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T08:41:42.961 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T08:41:42.961 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T08:41:42.961 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T08:41:42.961 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T08:41:42.961 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T08:41:42.961 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T08:41:42.961 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T08:41:42.961 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T08:41:42.961 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T08:41:42.962 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T08:41:42.962 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T08:41:42.962 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T08:41:42.962 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T08:41:42.962 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T08:41:42.962 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T08:41:42.962 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T08:41:42.962 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T08:41:42.962 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T08:41:42.962 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T08:41:42.962 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T08:41:42.962 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T08:41:42.962 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T08:41:42.962 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T08:41:42.963 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T08:41:42.963 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T08:41:42.963 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T08:41:42.963 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T08:41:42.963 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T08:41:42.963 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T08:41:42.963 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T08:41:42.963 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T08:41:42.963 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T08:41:42.963 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T08:41:42.963 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T08:41:42.963 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T08:41:42.963 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T08:41:42.963 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T08:41:42.964 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T08:41:42.964 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T08:41:42.964 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T08:41:42.964 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T08:41:42.964 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T08:41:42.964 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T08:41:42.964 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T08:41:42.964 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T08:41:42.964 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T08:41:42.964 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T08:41:42.964 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T08:41:42.964 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T08:41:42.964 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T08:41:42.964 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T08:41:42.964 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T08:41:42.965 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T08:41:42.965 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T08:41:42.965 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T08:41:42.965 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T08:41:42.965 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T08:41:42.965 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T08:41:42.965 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T08:41:42.965 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T08:41:42.965 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T08:41:42.965 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T08:41:42.965 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T08:41:42.965 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T08:41:42.965 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T08:41:42.965 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T08:41:42.965 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T08:41:42.965 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T08:41:42.966 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T08:41:42.966 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T08:41:42.966 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T08:41:42.966 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T08:41:42.966 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T08:41:42.966 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T08:41:42.966 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T08:41:42.966 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T08:41:42.966 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T08:41:42.966 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T08:41:42.966 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T08:41:42.966 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T08:41:42.966 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T08:41:42.967 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T08:41:42.967 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T08:41:42.967 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T08:41:42.967 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T08:41:42.967 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T08:41:42.967 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T08:41:42.967 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T08:41:42.967 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T08:41:42.967 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T08:41:42.967 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T08:41:42.967 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T08:41:42.967 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T08:41:42.967 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T08:41:42.967 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T08:41:42.967 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T08:41:42.968 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T08:41:42.968 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T08:41:42.968 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T08:41:42.968 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T08:41:42.968 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T08:41:42.968 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T08:41:42.968 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T08:41:42.968 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T08:41:42.968 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T08:41:42.968 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T08:41:42.968 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T08:41:42.968 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T08:41:42.968 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T08:41:42.968 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T08:41:42.969 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T08:41:42.969 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T08:41:42.969 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T08:41:42.969 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T08:41:42.969 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T08:41:42.969 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T08:41:42.969 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T08:41:42.969 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T08:41:42.969 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T08:41:42.969 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T08:41:42.969 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T08:41:42.969 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T08:41:42.969 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T08:41:42.969 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T08:41:42.969 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T08:41:42.970 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T08:41:42.970 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T08:41:42.970 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T08:41:42.970 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T08:41:42.970 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T08:41:42.970 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T08:41:42.970 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T08:41:42.970 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T08:41:42.970 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T08:41:42.970 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T08:41:42.970 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T08:41:42.970 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T08:41:42.970 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T08:41:42.970 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T08:41:42.970 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T08:41:42.971 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T08:41:42.971 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T08:41:42.971 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T08:41:42.971 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T08:41:42.971 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T08:41:42.971 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T08:41:42.971 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T08:41:42.971 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T08:41:42.971 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T08:41:42.971 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T08:41:42.971 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T08:41:42.971 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T08:41:42.971 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T08:41:42.971 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T08:41:42.972 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T08:41:42.972 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T08:41:42.972 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T08:41:42.972 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T08:41:42.972 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T08:41:42.972 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T08:41:42.972 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T08:41:42.972 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T08:41:42.972 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T08:41:42.972 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T08:41:42.972 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T08:41:42.972 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T08:41:42.972 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T08:41:42.972 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T08:41:42.973 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T08:41:42.973 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T08:41:42.973 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T08:41:42.973 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T08:41:42.973 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T08:41:42.973 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T08:41:42.973 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T08:41:42.973 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T08:41:42.973 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T08:41:42.973 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T08:41:42.973 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T08:41:42.973 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T08:41:42.973 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T08:41:42.973 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T08:41:42.974 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T08:41:42.974 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T08:41:42.974 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T08:41:42.974 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T08:41:42.974 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T08:41:42.974 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T08:41:42.974 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T08:41:42.974 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T08:41:42.974 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T08:41:42.974 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T08:41:42.974 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T08:41:42.974 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T08:41:42.974 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T08:41:42.974 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T08:41:42.975 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T08:41:42.975 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T08:41:42.975 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T08:41:42.975 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T08:41:42.975 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T08:41:42.975 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T08:41:42.975 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T08:41:42.975 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T08:41:42.975 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T08:41:42.975 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T08:41:42.975 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T08:41:42.976 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T08:41:42.976 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T08:41:42.976 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T08:41:42.976 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T08:41:42.976 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T08:41:42.976 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T08:41:42.976 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T08:41:42.976 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T08:41:42.976 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T08:41:42.976 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T08:41:42.976 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T08:41:42.976 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T08:41:42.976 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T08:41:42.976 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T08:41:42.977 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T08:41:42.977 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T08:41:42.977 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T08:41:42.977 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T08:41:42.977 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T08:41:42.977 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T08:41:42.977 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T08:41:42.977 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T08:41:42.977 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T08:41:42.977 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T08:41:42.977 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T08:41:42.977 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T08:41:42.977 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T08:41:42.977 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T08:41:42.978 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T08:41:42.978 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T08:41:42.978 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T08:41:42.978 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T08:41:42.978 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T08:41:42.978 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T08:41:42.978 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T08:41:42.978 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T08:41:42.978 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T08:41:42.978 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T08:41:42.978 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T08:41:42.978 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T08:41:42.978 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T08:41:42.978 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T08:41:42.979 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T08:41:42.979 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T08:41:42.979 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T08:41:42.979 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T08:41:42.979 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T08:41:42.979 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T08:41:42.979 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T08:41:42.979 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T08:41:42.979 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T08:41:42.979 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T08:41:42.979 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T08:41:42.979 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T08:41:42.979 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T08:41:42.979 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T08:41:42.979 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T08:41:42.980 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T08:41:42.980 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T08:41:42.980 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T08:41:42.980 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T08:41:42.980 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T08:41:42.980 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T08:41:42.980 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T08:41:42.980 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T08:41:42.980 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T08:41:42.980 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T08:41:42.980 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T08:41:42.980 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T08:41:42.980 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T08:41:42.980 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T08:41:42.981 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T08:41:42.981 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T08:41:42.981 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T08:41:42.981 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T08:41:42.981 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T08:41:42.981 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T08:41:42.981 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T08:41:42.981 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T08:41:42.981 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T08:41:42.981 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T08:41:42.981 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T08:41:42.981 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T08:41:42.981 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T08:41:42.981 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T08:41:42.981 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T08:41:42.982 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T08:41:42.982 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T08:41:42.982 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T08:41:42.982 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T08:41:42.982 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T08:41:42.982 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T08:41:42.982 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T08:41:42.982 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T08:41:42.983 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T08:41:42.983 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T08:41:42.983 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T08:41:42.983 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T08:41:42.983 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T08:41:42.983 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T08:41:42.983 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T08:41:42.983 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T08:41:42.983 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T08:41:42.983 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T08:41:42.983 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T08:41:42.983 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T08:41:42.983 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T08:41:42.983 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T08:41:42.983 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T08:41:42.984 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T08:41:42.984 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T08:41:42.984 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T08:41:42.984 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T08:41:42.984 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T08:41:42.984 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T08:41:42.984 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T08:41:42.984 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T08:41:42.984 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T08:41:42.984 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T08:41:42.984 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T08:41:42.984 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T08:41:42.984 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T08:41:42.984 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T08:41:42.984 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T08:41:42.985 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T08:41:42.985 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T08:41:42.985 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T08:41:42.985 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T08:41:42.985 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T08:41:42.985 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T08:41:42.985 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T08:41:42.985 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T08:41:42.985 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T08:41:42.985 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T08:41:42.985 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T08:41:42.985 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T08:41:42.985 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T08:41:42.985 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T08:41:42.985 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T08:41:42.986 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T08:41:42.986 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T08:41:42.986 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T08:41:42.986 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T08:41:42.986 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T08:41:42.986 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T08:41:42.986 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T08:41:42.986 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T08:41:42.986 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T08:41:42.986 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T08:41:42.986 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T08:41:42.986 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T08:41:42.986 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T08:41:42.986 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T08:41:42.986 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T08:41:42.987 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T08:41:42.987 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T08:41:42.987 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T08:41:42.987 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T08:41:42.987 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T08:41:42.987 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T08:41:42.987 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T08:41:42.987 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T08:41:42.987 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T08:41:42.987 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T08:41:42.987 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T08:41:42.987 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T08:41:42.987 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T08:41:42.987 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T08:41:42.988 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T08:41:42.988 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T08:41:42.988 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T08:41:42.988 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T08:41:42.988 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T08:41:42.988 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T08:41:42.988 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T08:41:42.988 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T08:41:42.988 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T08:41:42.988 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T08:41:42.988 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T08:41:42.988 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T08:41:42.988 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T08:41:42.988 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T08:41:42.989 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T08:41:42.989 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T08:41:42.989 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T08:41:42.989 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T08:41:42.989 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T08:41:42.989 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T08:41:42.989 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T08:41:42.989 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T08:41:42.989 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T08:41:42.989 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T08:41:42.989 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T08:41:42.989 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T08:41:42.989 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T08:41:42.989 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T08:41:42.990 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T08:41:42.990 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T08:41:42.990 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T08:41:42.990 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T08:41:42.990 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T08:41:42.990 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T08:41:42.990 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T08:41:42.990 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T08:41:42.990 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T08:41:42.990 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T08:41:42.990 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T08:41:42.990 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T08:41:42.990 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T08:41:42.990 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T08:41:42.991 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T08:41:42.991 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T08:41:42.991 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T08:41:42.991 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T08:41:42.991 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T08:41:42.991 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T08:41:42.991 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T08:41:42.991 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T08:41:42.991 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T08:41:42.991 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T08:41:42.991 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T08:41:42.991 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T08:41:42.991 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T08:41:42.991 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T08:41:42.991 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T08:41:42.991 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T08:41:42.992 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T08:41:42.992 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T08:41:42.992 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T08:41:42.992 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T08:41:42.992 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T08:41:42.992 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T08:41:42.992 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T08:41:42.992 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T08:41:42.992 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T08:41:42.992 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T08:41:42.992 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T08:41:42.992 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T08:41:42.992 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T08:41:42.992 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T08:41:42.992 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T08:41:42.993 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T08:41:42.993 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T08:41:42.993 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T08:41:42.993 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T08:41:42.993 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T08:41:42.993 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T08:41:42.993 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T08:41:42.993 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T08:41:42.993 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T08:41:42.993 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T08:41:42.993 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T08:41:42.993 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T08:41:42.993 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T08:41:42.993 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T08:41:42.993 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T08:41:42.994 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T08:41:42.994 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T08:41:42.994 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T08:41:42.994 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T08:41:42.994 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T08:41:42.994 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T08:41:42.994 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T08:41:42.994 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T08:41:42.994 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T08:41:42.994 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T08:41:42.994 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T08:41:42.994 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T08:41:42.994 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T08:41:42.994 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T08:41:42.995 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T08:41:42.995 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T08:41:42.995 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T08:41:42.995 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T08:41:42.995 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T08:41:42.995 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T08:41:42.995 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T08:41:42.995 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T08:41:42.995 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T08:41:42.995 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T08:41:42.995 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T08:41:42.995 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T08:41:42.995 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T08:41:42.995 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T08:41:42.995 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T08:41:42.996 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T08:41:42.996 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T08:41:42.996 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T08:41:42.996 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T08:41:42.996 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T08:41:42.996 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T08:41:42.996 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T08:41:42.996 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T08:41:42.996 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T08:41:42.996 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T08:41:42.996 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T08:41:42.996 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T08:41:42.996 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T08:41:42.996 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T08:41:42.996 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T08:41:42.997 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T08:41:42.997 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T08:41:42.997 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T08:41:42.997 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T08:41:42.997 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T08:41:42.997 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T08:41:42.997 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T08:41:42.997 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T08:41:42.997 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T08:41:42.997 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T08:41:42.997 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T08:41:42.997 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T08:41:42.997 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T08:41:42.997 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T08:41:42.997 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T08:41:42.997 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T08:41:42.998 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T08:41:42.998 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T08:41:42.998 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T08:41:42.998 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T08:41:42.998 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T08:41:42.998 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T08:41:42.998 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T08:41:42.998 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T08:41:42.998 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T08:41:42.998 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T08:41:42.998 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T08:41:42.998 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T08:41:42.998 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T08:41:42.998 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T08:41:42.999 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T08:41:42.999 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T08:41:42.999 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T08:41:42.999 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T08:41:42.999 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T08:41:42.999 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T08:41:42.999 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T08:41:42.999 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T08:41:42.999 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T08:41:42.999 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T08:41:42.999 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T08:41:42.999 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T08:41:42.999 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T08:41:43.000 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T08:41:43.000 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T08:41:43.000 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T08:41:43.000 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T08:41:43.000 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T08:41:43.000 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T08:41:43.000 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T08:41:43.000 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T08:41:43.000 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T08:41:43.000 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T08:41:43.000 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T08:41:43.000 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T08:41:43.000 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T08:41:43.000 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T08:41:43.000 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T08:41:43.001 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T08:41:43.001 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T08:41:43.001 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T08:41:43.001 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T08:41:43.001 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T08:41:43.001 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T08:41:43.001 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T08:41:43.001 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T08:41:43.001 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T08:41:43.001 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T08:41:43.001 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T08:41:43.001 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T08:41:43.001 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T08:41:43.001 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T08:41:43.001 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T08:41:43.001 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T08:41:43.002 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T08:41:43.002 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T08:41:43.002 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T08:41:43.002 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T08:41:43.002 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T08:41:43.002 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T08:41:43.002 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T08:41:43.002 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T08:41:43.002 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T08:41:43.002 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T08:41:43.002 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T08:41:43.002 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T08:41:43.002 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T08:41:43.002 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T08:41:43.003 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T08:41:43.003 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T08:41:43.003 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T08:41:43.003 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T08:41:43.003 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T08:41:43.003 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T08:41:43.003 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T08:41:43.003 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T08:41:43.003 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T08:41:43.003 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T08:41:43.003 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T08:41:43.003 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T08:41:43.003 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T08:41:43.003 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T08:41:43.003 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T08:41:43.004 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T08:41:43.004 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T08:41:43.004 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T08:41:43.004 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T08:41:43.004 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T08:41:43.004 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T08:41:43.004 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T08:41:43.004 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T08:41:43.004 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T08:41:43.004 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T08:41:43.004 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T08:41:43.004 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T08:41:43.004 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T08:41:43.004 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T08:41:43.004 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T08:41:43.005 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T08:41:43.005 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T08:41:43.005 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T08:41:43.005 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T08:41:43.005 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T08:41:43.005 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T08:41:43.005 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T08:41:43.005 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T08:41:43.005 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T08:41:43.005 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T08:41:43.005 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T08:41:43.005 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T08:41:43.005 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T08:41:43.005 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T08:41:43.006 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T08:41:43.006 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T08:41:43.006 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T08:41:43.006 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T08:41:43.006 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T08:41:43.006 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T08:41:43.006 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T08:41:43.006 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T08:41:43.006 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T08:41:43.006 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T08:41:43.006 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T08:41:43.006 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T08:41:43.006 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T08:41:43.006 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T08:41:43.007 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T08:41:43.007 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T08:41:43.007 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T08:41:43.007 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T08:41:43.007 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T08:41:43.007 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T08:41:43.007 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T08:41:43.007 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T08:41:43.007 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T08:41:43.007 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T08:41:43.007 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T08:41:43.007 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T08:41:43.007 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T08:41:43.007 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T08:41:43.007 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T08:41:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T08:41:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T08:41:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T08:41:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T08:41:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T08:41:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T08:41:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T08:41:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T08:41:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T08:41:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T08:41:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T08:41:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T08:41:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T08:41:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T08:41:43.008 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T08:41:43.009 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T08:41:43.009 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T08:41:43.009 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T08:41:43.009 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T08:41:43.009 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T08:41:43.009 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T08:41:43.009 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T08:41:43.009 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T08:41:43.009 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T08:41:43.009 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T08:41:43.009 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T08:41:43.009 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T08:41:43.009 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T08:41:43.009 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T08:41:43.010 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T08:41:43.010 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T08:41:43.010 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T08:41:43.010 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T08:41:43.010 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T08:41:43.010 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T08:41:43.010 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T08:41:43.010 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T08:41:43.010 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T08:41:43.010 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T08:41:43.010 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T08:41:43.010 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T08:41:43.010 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T08:41:43.010 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T08:41:43.011 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T08:41:43.011 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T08:41:43.011 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T08:41:43.011 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T08:41:43.011 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T08:41:43.011 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T08:41:43.011 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T08:41:43.011 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T08:41:43.011 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T08:41:43.011 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T08:41:43.011 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T08:41:43.011 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T08:41:43.011 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T08:41:43.011 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T08:41:43.011 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T08:41:43.012 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T08:41:43.012 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T08:41:43.012 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T08:41:43.012 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T08:41:43.012 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T08:41:43.012 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T08:41:43.012 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T08:41:43.012 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T08:41:43.012 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T08:41:43.012 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T08:41:43.012 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T08:41:43.012 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T08:41:43.012 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T08:41:43.012 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T08:41:43.012 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T08:41:43.013 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T08:41:43.013 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T08:41:43.013 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T08:41:43.013 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T08:41:43.013 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T08:41:43.013 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T08:41:43.013 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T08:41:43.013 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T08:41:43.013 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T08:41:43.013 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T08:41:43.013 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T08:41:43.013 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T08:41:43.013 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T08:41:43.013 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T08:41:43.013 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T08:41:43.014 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T08:41:43.014 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T08:41:43.014 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T08:41:43.014 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T08:41:43.014 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T08:41:43.014 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T08:41:43.014 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T08:41:43.014 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T08:41:43.014 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T08:41:43.014 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T08:41:43.014 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T08:41:43.014 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T08:41:43.014 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T08:41:43.014 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T08:41:43.014 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T08:41:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T08:41:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T08:41:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T08:41:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T08:41:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T08:41:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T08:41:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T08:41:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T08:41:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T08:41:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T08:41:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T08:41:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T08:41:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T08:41:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T08:41:43.015 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T08:41:43.016 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T08:41:43.016 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T08:41:43.016 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T08:41:43.016 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T08:41:43.016 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T08:41:43.016 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T08:41:43.016 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T08:41:43.016 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T08:41:43.016 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T08:41:43.016 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T08:41:43.016 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T08:41:43.016 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T08:41:43.016 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T08:41:43.016 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T08:41:43.017 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T08:41:43.017 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T08:41:43.017 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T08:41:43.017 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T08:41:43.017 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T08:41:43.017 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T08:41:43.017 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T08:41:43.017 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T08:41:43.017 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T08:41:43.017 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T08:41:43.017 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T08:41:43.017 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T08:41:43.017 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T08:41:43.017 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T08:41:43.017 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T08:41:43.018 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T08:41:43.018 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T08:41:43.018 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T08:41:43.018 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T08:41:43.018 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T08:41:43.018 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T08:41:43.018 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T08:41:43.018 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T08:41:43.018 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T08:41:43.018 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T08:41:43.018 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T08:41:43.018 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T08:41:43.018 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T08:41:43.018 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T08:41:43.018 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T08:41:43.019 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T08:41:43.019 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T08:41:43.019 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T08:41:43.019 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T08:41:43.019 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T08:41:43.019 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T08:41:43.019 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T08:41:43.019 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T08:41:43.019 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T08:41:43.019 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T08:41:43.019 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T08:41:43.019 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T08:41:43.019 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T08:41:43.019 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T08:41:43.019 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T08:41:43.020 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T08:41:43.020 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T08:41:43.020 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T08:41:43.020 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T08:41:43.020 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T08:41:43.020 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T08:41:43.020 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T08:41:43.020 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T08:41:43.020 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T08:41:43.020 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T08:41:43.020 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T08:41:43.020 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T08:41:43.020 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T08:41:43.020 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T08:41:43.021 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T08:41:43.021 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T08:41:43.021 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T08:41:43.021 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T08:41:43.021 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T08:41:43.021 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T08:41:43.021 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T08:41:43.021 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T08:41:43.021 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T08:41:43.021 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T08:41:43.021 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T08:41:43.021 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T08:41:43.021 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T08:41:43.021 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T08:41:43.021 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T08:41:43.022 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T08:41:43.022 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T08:41:43.022 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T08:41:43.022 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T08:41:43.022 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T08:41:43.022 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T08:41:43.022 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T08:41:43.022 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T08:41:43.022 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T08:41:43.022 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T08:41:43.022 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T08:41:43.022 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T08:41:43.022 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T08:41:43.022 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T08:41:43.023 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T08:41:43.023 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T08:41:43.023 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T08:41:43.023 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T08:41:43.023 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T08:41:43.023 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T08:41:43.023 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T08:41:43.023 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T08:41:43.023 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T08:41:43.023 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T08:41:43.023 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T08:41:43.023 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T08:41:43.023 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T08:41:43.023 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T08:41:43.023 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T08:41:43.023 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T08:41:43.024 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T08:41:43.024 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T08:41:43.024 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T08:41:43.024 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T08:41:43.024 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T08:41:43.024 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T08:41:43.024 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T08:41:43.024 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T08:41:43.024 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T08:41:43.024 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T08:41:43.024 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T08:41:43.024 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T08:41:43.024 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T08:41:43.024 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T08:41:43.024 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T08:41:43.025 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T08:41:43.025 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T08:41:43.025 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T08:41:43.025 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T08:41:43.025 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T08:41:43.025 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T08:41:43.025 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T08:41:43.025 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T08:41:43.025 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T08:41:43.025 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T08:41:43.025 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T08:41:43.025 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T08:41:43.025 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T08:41:43.025 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T08:41:43.025 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T08:41:43.026 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T08:41:43.026 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T08:41:43.026 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T08:41:43.026 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T08:41:43.026 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T08:41:43.026 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T08:41:43.026 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T08:41:43.026 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T08:41:43.026 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T08:41:43.026 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T08:41:43.026 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T08:41:43.026 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T08:41:43.026 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T08:41:43.026 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T08:41:43.027 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T08:41:43.027 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T08:41:43.027 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T08:41:43.027 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T08:41:43.027 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T08:41:43.027 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T08:41:43.027 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T08:41:43.027 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T08:41:43.027 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T08:41:43.027 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T08:41:43.027 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T08:41:43.027 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T08:41:43.027 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T08:41:43.027 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T08:41:43.027 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T08:41:43.028 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T08:41:43.028 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T08:41:43.028 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T08:41:43.028 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T08:41:43.028 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T08:41:43.028 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T08:41:43.028 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T08:41:43.028 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T08:41:43.028 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T08:41:43.028 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T08:41:43.028 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T08:41:43.028 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T08:41:43.028 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T08:41:43.028 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T08:41:43.028 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T08:41:43.028 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T08:41:43.029 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T08:41:43.029 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T08:41:43.029 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T08:41:43.029 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T08:41:43.029 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T08:41:43.029 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T08:41:43.029 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T08:41:43.029 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T08:41:43.029 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T08:41:43.029 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T08:41:43.029 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T08:41:43.029 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T08:41:43.029 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T08:41:43.029 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T08:41:43.029 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T08:41:43.030 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T08:41:43.030 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T08:41:43.030 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T08:41:43.030 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T08:41:43.030 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T08:41:43.030 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T08:41:43.030 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T08:41:43.030 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T08:41:43.030 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T08:41:43.030 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T08:41:43.030 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T08:41:43.030 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T08:41:43.030 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T08:41:43.030 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T08:41:43.030 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T08:41:43.031 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T08:41:43.031 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T08:41:43.031 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T08:41:43.031 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T08:41:43.031 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T08:41:43.031 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T08:41:43.031 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T08:41:43.031 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T08:41:43.031 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T08:41:43.031 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T08:41:43.031 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T08:41:43.031 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T08:41:43.031 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T08:41:43.031 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T08:41:43.031 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T08:41:43.032 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T08:41:43.032 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T08:41:43.032 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T08:41:43.032 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T08:41:43.032 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T08:41:43.032 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T08:41:43.032 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T08:41:43.032 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T08:41:43.032 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T08:41:43.032 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T08:41:43.032 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T08:41:43.032 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T08:41:43.032 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T08:41:43.032 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T08:41:43.032 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T08:41:43.033 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T08:41:43.033 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T08:41:43.033 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T08:41:43.033 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T08:41:43.033 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T08:41:43.033 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T08:41:43.033 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T08:41:43.033 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T08:41:43.033 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T08:41:43.033 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T08:41:43.033 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T08:41:43.033 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T08:41:43.033 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T08:41:43.033 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T08:41:43.034 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T08:41:43.034 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T08:41:43.034 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T08:41:43.034 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T08:41:43.034 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T08:41:43.034 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T08:41:43.034 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T08:41:43.034 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T08:41:43.034 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T08:41:43.034 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T08:41:43.034 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T08:41:43.034 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T08:41:43.034 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T08:41:43.034 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T08:41:43.034 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T08:41:43.035 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T08:41:43.035 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T08:41:43.035 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T08:41:43.035 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T08:41:43.035 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T08:41:43.035 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T08:41:43.035 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T08:41:43.035 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T08:41:43.035 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T08:41:43.035 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T08:41:43.035 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T08:41:43.035 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T08:41:43.035 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T08:41:43.035 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T08:41:43.036 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T08:41:43.036 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T08:41:43.036 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T08:41:43.036 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T08:41:43.036 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T08:41:43.036 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T08:41:43.036 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T08:41:43.036 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T08:41:43.036 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T08:41:43.036 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T08:41:43.036 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T08:41:43.036 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T08:41:43.036 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T08:41:43.036 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T08:41:43.036 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T08:41:43.037 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T08:41:43.037 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T08:41:43.037 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T08:41:43.037 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T08:41:43.037 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T08:41:43.037 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T08:41:43.037 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T08:41:43.037 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T08:41:43.037 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T08:41:43.037 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T08:41:43.037 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T08:41:43.037 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T08:41:43.037 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T08:41:43.037 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T08:41:43.037 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T08:41:43.038 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T08:41:43.038 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T08:41:43.038 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T08:41:43.038 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T08:41:43.038 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T08:41:43.038 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T08:41:43.038 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T08:41:43.038 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T08:41:43.038 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T08:41:43.038 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T08:41:43.038 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T08:41:43.038 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T08:41:43.038 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T08:41:43.038 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T08:41:43.039 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T08:41:43.039 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T08:41:43.039 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T08:41:43.039 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T08:41:43.039 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T08:41:43.039 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T08:41:43.039 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T08:41:43.039 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T08:41:43.039 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T08:41:43.039 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T08:41:43.039 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T08:41:43.039 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T08:41:43.039 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T08:41:43.039 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T08:41:43.039 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T08:41:43.039 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T08:41:43.040 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T08:41:43.040 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T08:41:43.040 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T08:41:43.040 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T08:41:43.040 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T08:41:43.040 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T08:41:43.040 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T08:41:43.040 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T08:41:43.040 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T08:41:43.040 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T08:41:43.040 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T08:41:43.040 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T08:41:43.040 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T08:41:43.040 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T08:41:43.041 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T08:41:43.041 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T08:41:43.041 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T08:41:43.041 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T08:41:43.041 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T08:41:43.041 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T08:41:43.041 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T08:41:43.041 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T08:41:43.041 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T08:41:43.041 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T08:41:43.041 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T08:41:43.041 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T08:41:43.041 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T08:41:43.041 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T08:41:43.041 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T08:41:43.042 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T08:41:43.042 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T08:41:43.042 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T08:41:43.042 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T08:41:43.042 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T08:41:43.042 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T08:41:43.042 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T08:41:43.042 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T08:41:43.042 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T08:41:43.042 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T08:41:43.042 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T08:41:43.042 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T08:41:43.042 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T08:41:43.043 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T08:41:43.043 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T08:41:43.043 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T08:41:43.043 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T08:41:43.043 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T08:41:43.043 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T08:41:43.043 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T08:41:43.043 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T08:41:43.043 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T08:41:43.043 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T08:41:43.043 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T08:41:43.043 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T08:41:43.043 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T08:41:43.043 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T08:41:43.043 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T08:41:43.044 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T08:41:43.044 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T08:41:43.044 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T08:41:43.044 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T08:41:43.044 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T08:41:43.044 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T08:41:43.044 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T08:41:43.044 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T08:41:43.044 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T08:41:43.044 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T08:41:43.044 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T08:41:43.044 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T08:41:43.044 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T08:41:43.044 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T08:41:43.044 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T08:41:43.045 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T08:41:43.045 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T08:41:43.045 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T08:41:43.045 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T08:41:43.045 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T08:41:43.045 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T08:41:43.045 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T08:41:43.045 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T08:41:43.045 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T08:41:43.045 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T08:41:43.045 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T08:41:43.045 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T08:41:43.045 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T08:41:43.045 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T08:41:43.045 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T08:41:43.046 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T08:41:43.046 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T08:41:43.046 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T08:41:43.046 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T08:41:43.046 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T08:41:43.046 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T08:41:43.046 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T08:41:43.046 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T08:41:43.046 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T08:41:43.046 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T08:41:43.046 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T08:41:43.046 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T08:41:43.046 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T08:41:43.046 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T08:41:43.047 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T08:41:43.047 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T08:41:43.047 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T08:41:43.047 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T08:41:43.047 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T08:41:43.047 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T08:41:43.047 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T08:41:43.047 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T08:41:43.047 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T08:41:43.047 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T08:41:43.047 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T08:41:43.047 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T08:41:43.047 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T08:41:43.047 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T08:41:43.047 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T08:41:43.048 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T08:41:43.048 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T08:41:43.048 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T08:41:43.048 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T08:41:43.048 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T08:41:43.048 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T08:41:43.048 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T08:41:43.048 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T08:41:43.048 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T08:41:43.048 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T08:41:43.048 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T08:41:43.048 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T08:41:43.048 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T08:41:43.048 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T08:41:43.049 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T08:41:43.049 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T08:41:43.049 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T08:41:43.049 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T08:41:43.049 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T08:41:43.049 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T08:41:43.049 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T08:41:43.049 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T08:41:43.049 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T08:41:43.049 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T08:41:43.049 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T08:41:43.049 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T08:41:43.049 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T08:41:43.049 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T08:41:43.049 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T08:41:43.050 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T08:41:43.050 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T08:41:43.050 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T08:41:43.050 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T08:41:43.050 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T08:41:43.050 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T08:41:43.050 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T08:41:43.050 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T08:41:43.050 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T08:41:43.050 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T08:41:43.050 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T08:41:43.050 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T08:41:43.050 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T08:41:43.050 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T08:41:43.050 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T08:41:43.051 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T08:41:43.051 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T08:41:43.051 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T08:41:43.051 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T08:41:43.051 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T08:41:43.051 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T08:41:43.051 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T08:41:43.051 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T08:41:43.051 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T08:41:43.051 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T08:41:43.051 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T08:41:43.051 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T08:41:43.051 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T08:41:43.051 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T08:41:43.052 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T08:41:43.052 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T08:41:43.052 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T08:41:43.052 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T08:41:43.052 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T08:41:43.052 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T08:41:43.052 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T08:41:43.052 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T08:41:43.052 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T08:41:43.052 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T08:41:43.052 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T08:41:43.052 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T08:41:43.052 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T08:41:43.052 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T08:41:43.052 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T08:41:43.052 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T08:41:43.053 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T08:41:43.053 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T08:41:43.053 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T08:41:43.053 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T08:41:43.053 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T08:41:43.053 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T08:41:43.053 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T08:41:43.053 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T08:41:43.053 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T08:41:43.053 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T08:41:43.053 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T08:41:43.053 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T08:41:43.053 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T08:41:43.053 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T08:41:43.054 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T08:41:43.054 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T08:41:43.054 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T08:41:43.054 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T08:41:43.054 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T08:41:43.054 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T08:41:43.054 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T08:41:43.054 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T08:41:43.054 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T08:41:43.054 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T08:41:43.054 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T08:41:43.054 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T08:41:43.054 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T08:41:43.054 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T08:41:43.054 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T08:41:43.055 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T08:41:43.055 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T08:41:43.055 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T08:41:43.055 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T08:41:43.055 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T08:41:43.055 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T08:41:43.055 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T08:41:43.055 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T08:41:43.055 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T08:41:43.055 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T08:41:43.055 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T08:41:43.055 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T08:41:43.055 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T08:41:43.055 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T08:41:43.055 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T08:41:43.056 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T08:41:43.056 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T08:41:43.056 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T08:41:43.056 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T08:41:43.056 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T08:41:43.056 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T08:41:43.056 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T08:41:43.056 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T08:41:43.056 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T08:41:43.056 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T08:41:43.056 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T08:41:43.056 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T08:41:43.056 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T08:41:43.056 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T08:41:43.057 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T08:41:43.057 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T08:41:43.057 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T08:41:43.057 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T08:41:43.057 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T08:41:43.057 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T08:41:43.057 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T08:41:43.057 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T08:41:43.057 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T08:41:43.057 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T08:41:43.057 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T08:41:43.057 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T08:41:43.057 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T08:41:43.057 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T08:41:43.057 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T08:41:43.058 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T08:41:43.058 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T08:41:43.058 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T08:41:43.058 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T08:41:43.058 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T08:41:43.058 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T08:41:43.058 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T08:41:43.058 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T08:41:43.058 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T08:41:43.058 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T08:41:43.058 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T08:41:43.058 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T08:41:43.058 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T08:41:43.058 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T08:41:43.059 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T08:41:43.059 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T08:41:43.059 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T08:41:43.059 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T08:41:43.059 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T08:41:43.059 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T08:41:43.059 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T08:41:43.059 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T08:41:43.059 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T08:41:43.059 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T08:41:43.059 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T08:41:43.059 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T08:41:43.059 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T08:41:43.059 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T08:41:43.060 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T08:41:43.060 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T08:41:43.060 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T08:41:43.060 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T08:41:43.060 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T08:41:43.060 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T08:41:43.060 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T08:41:43.060 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T08:41:43.060 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T08:41:43.060 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T08:41:43.060 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T08:41:43.060 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T08:41:43.060 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T08:41:43.060 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T08:41:43.061 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T08:41:43.061 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T08:41:43.061 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T08:41:43.061 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T08:41:43.061 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T08:41:43.061 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T08:41:43.061 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T08:41:43.061 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T08:41:43.061 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T08:41:43.061 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T08:41:43.061 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T08:41:43.061 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T08:41:43.061 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T08:41:43.061 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T08:41:43.061 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T08:41:43.061 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T08:41:43.062 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T08:41:43.062 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T08:41:43.062 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T08:41:43.062 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T08:41:43.062 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T08:41:43.062 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T08:41:43.062 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T08:41:43.062 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T08:41:43.062 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T08:41:43.062 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T08:41:43.062 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T08:41:43.062 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T08:41:43.062 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T08:41:43.062 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T08:41:43.063 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T08:41:43.063 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T08:41:43.063 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T08:41:43.063 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T08:41:43.063 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T08:41:43.063 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T08:41:43.063 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T08:41:43.063 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T08:41:43.063 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T08:41:43.063 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T08:41:43.063 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T08:41:43.063 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T08:41:43.063 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T08:41:43.063 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T08:41:43.063 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T08:41:43.064 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T08:41:43.064 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T08:41:43.064 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T08:41:43.064 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T08:41:43.064 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T08:41:43.064 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T08:41:43.064 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T08:41:43.064 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T08:41:43.064 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T08:41:43.064 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T08:41:43.064 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T08:41:43.064 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T08:41:43.064 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T08:41:43.064 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T08:41:43.064 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T08:41:43.065 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T08:41:43.065 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T08:41:43.065 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T08:41:43.065 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T08:41:43.065 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T08:41:43.065 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T08:41:43.065 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T08:41:43.065 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T08:41:43.065 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T08:41:43.065 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T08:41:43.065 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T08:41:43.065 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T08:41:43.065 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T08:41:43.065 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T08:41:43.065 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T08:41:43.065 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T08:41:43.066 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T08:41:43.066 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T08:41:43.066 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T08:41:43.066 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T08:41:43.066 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T08:41:43.066 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T08:41:43.066 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T08:41:43.066 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T08:41:43.066 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T08:41:43.066 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T08:41:43.066 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T08:41:43.066 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T08:41:43.066 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T08:41:43.066 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T08:41:43.067 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T08:41:43.067 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T08:41:43.067 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T08:41:43.067 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T08:41:43.067 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T08:41:43.067 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T08:41:43.067 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T08:41:43.067 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T08:41:43.067 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T08:41:43.067 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T08:41:43.067 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T08:41:43.067 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T08:41:43.067 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T08:41:43.067 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T08:41:43.067 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T08:41:43.067 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T08:41:43.068 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T08:41:43.068 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T08:41:43.068 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T08:41:43.068 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T08:41:43.068 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T08:41:43.068 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T08:41:43.068 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T08:41:43.068 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T08:41:43.068 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T08:41:43.068 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T08:41:43.068 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T08:41:43.068 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T08:41:43.068 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T08:41:43.068 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T08:41:43.068 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T08:41:43.068 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T08:41:43.069 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T08:41:43.069 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T08:41:43.069 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T08:41:43.069 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T08:41:43.069 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T08:41:43.069 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T08:41:43.069 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T08:41:43.069 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T08:41:43.069 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T08:41:43.069 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T08:41:43.069 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T08:41:43.069 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T08:41:43.069 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T08:41:43.069 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T08:41:43.069 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T08:41:43.070 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T08:41:43.070 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T08:41:43.070 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T08:41:43.070 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T08:41:43.070 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T08:41:43.070 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T08:41:43.070 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T08:41:43.070 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T08:41:43.070 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T08:41:43.070 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T08:41:43.070 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T08:41:43.070 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T08:41:43.070 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T08:41:43.070 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T08:41:43.071 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T08:41:43.071 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T08:41:43.071 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T08:41:43.071 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T08:41:43.071 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T08:41:43.071 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T08:41:43.071 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T08:41:43.071 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T08:41:43.071 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T08:41:43.071 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T08:41:43.071 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T08:41:43.071 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T08:41:43.071 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T08:41:43.071 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T08:41:43.071 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T08:41:43.072 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T08:41:43.072 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T08:41:43.072 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T08:41:43.072 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T08:41:43.072 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T08:41:43.072 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T08:41:43.072 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T08:41:43.072 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T08:41:43.072 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T08:41:43.072 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T08:41:43.072 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T08:41:43.072 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T08:41:43.072 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T08:41:43.072 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T08:41:43.072 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T08:41:43.073 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T08:41:43.073 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T08:41:43.073 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T08:41:43.073 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T08:41:43.073 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T08:41:43.073 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T08:41:43.073 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T08:41:43.073 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T08:41:43.073 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T08:41:43.073 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T08:41:43.073 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T08:41:43.073 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T08:41:43.073 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T08:41:43.073 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T08:41:43.074 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T08:41:43.074 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T08:41:43.074 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T08:41:43.074 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T08:41:43.074 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T08:41:43.074 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T08:41:43.074 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T08:41:43.074 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T08:41:43.074 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T08:41:43.074 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T08:41:43.074 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T08:41:43.074 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T08:41:43.074 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T08:41:43.074 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T08:41:43.075 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T08:41:43.075 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T08:41:43.075 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T08:41:43.075 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T08:41:43.075 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T08:41:43.075 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T08:41:43.075 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T08:41:43.075 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T08:41:43.075 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T08:41:43.075 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T08:41:43.075 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T08:41:43.075 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T08:41:43.075 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T08:41:43.075 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T08:41:43.075 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T08:41:43.076 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T08:41:43.076 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T08:41:43.076 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T08:41:43.076 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T08:41:43.076 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T08:41:43.076 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T08:41:43.076 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T08:41:43.076 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T08:41:43.076 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T08:41:43.076 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T08:41:43.076 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T08:41:43.076 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T08:41:43.076 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T08:41:43.076 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T08:41:43.077 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T08:41:43.077 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T08:41:43.077 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T08:41:43.077 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T08:41:43.077 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T08:41:43.077 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T08:41:43.077 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T08:41:43.077 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T08:41:43.077 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T08:41:43.077 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T08:41:43.077 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T08:41:43.077 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T08:41:43.077 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T08:41:43.077 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T08:41:43.077 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T08:41:43.078 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T08:41:43.078 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T08:41:43.078 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T08:41:43.078 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T08:41:43.078 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T08:41:43.078 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T08:41:43.078 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T08:41:43.078 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T08:41:43.078 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T08:41:43.078 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T08:41:43.078 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T08:41:43.078 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T08:41:43.078 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T08:41:43.078 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T08:41:43.078 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T08:41:43.079 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T08:41:43.079 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T08:41:43.079 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T08:41:43.079 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T08:41:43.079 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T08:41:43.079 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T08:41:43.079 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T08:41:43.079 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T08:41:43.079 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T08:41:43.079 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T08:41:43.079 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T08:41:43.079 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T08:41:43.079 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T08:41:43.079 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T08:41:43.080 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T08:41:43.080 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T08:41:43.080 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T08:41:43.080 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T08:41:43.080 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T08:41:43.080 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T08:41:43.080 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T08:41:43.080 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T08:41:43.080 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T08:41:43.080 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T08:41:43.080 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T08:41:43.080 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T08:41:43.080 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T08:41:43.080 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T08:41:43.081 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T08:41:43.081 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T08:41:43.081 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T08:41:43.081 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T08:41:43.081 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T08:41:43.081 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T08:41:43.081 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T08:41:43.081 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T08:41:43.081 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T08:41:43.081 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T08:41:43.081 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T08:41:43.081 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T08:41:43.081 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T08:41:43.081 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T08:41:43.081 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T08:41:43.082 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T08:41:43.082 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T08:41:43.082 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T08:41:43.082 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T08:41:43.082 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T08:41:43.082 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T08:41:43.082 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T08:41:43.082 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T08:41:43.082 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T08:41:43.082 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T08:41:43.082 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T08:41:43.082 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T08:41:43.082 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T08:41:43.082 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T08:41:43.083 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T08:41:43.083 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T08:41:43.083 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T08:41:43.083 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T08:41:43.083 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T08:41:43.083 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T08:41:43.083 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T08:41:43.083 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T08:41:43.083 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T08:41:43.083 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T08:41:43.083 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T08:41:43.083 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T08:41:43.083 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T08:41:43.083 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T08:41:43.083 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T08:41:43.084 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T08:41:43.084 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T08:41:43.084 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T08:41:43.084 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T08:41:43.084 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T08:41:43.084 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T08:41:43.084 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T08:41:43.084 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T08:41:43.084 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T08:41:43.084 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T08:41:43.084 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T08:41:43.084 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T08:41:43.084 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T08:41:43.084 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T08:41:43.084 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T08:41:43.085 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T08:41:43.085 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T08:41:43.085 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T08:41:43.085 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T08:41:43.085 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T08:41:43.085 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T08:41:43.085 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T08:41:43.085 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T08:41:43.085 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T08:41:43.085 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T08:41:43.085 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T08:41:43.085 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T08:41:43.085 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T08:41:43.085 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T08:41:43.085 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T08:41:43.086 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T08:41:43.086 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T08:41:43.086 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T08:41:43.086 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T08:41:43.086 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T08:41:43.086 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T08:41:43.086 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T08:41:43.086 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T08:41:43.086 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T08:41:43.086 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T08:41:43.086 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T08:41:43.086 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T08:41:43.086 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T08:41:43.086 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T08:41:43.087 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T08:41:43.087 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T08:41:43.087 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T08:41:43.087 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T08:41:43.087 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T08:41:43.087 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T08:41:43.087 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T08:41:43.087 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T08:41:43.087 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T08:41:43.091 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:43.091 DEBUG:teuthology.orchestra.run.smithi130:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T08:41:43.114 INFO:teuthology.orchestra.run.smithi130.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T08:41:43.115 INFO:teuthology.orchestra.run.smithi130.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T08:41:43.115 INFO:teuthology.orchestra.run.smithi130.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T08:41:43.115 INFO:teuthology.orchestra.run.smithi130.stdout:fi 2024-04-24T08:41:43.115 INFO:teuthology.orchestra.run.smithi130.stdout:for file in vmlinuz System.map config; do 2024-04-24T08:41:43.115 INFO:teuthology.orchestra.run.smithi130.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T08:41:43.115 INFO:teuthology.orchestra.run.smithi130.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:41:43.115 INFO:teuthology.orchestra.run.smithi130.stdout: fi 2024-04-24T08:41:43.115 INFO:teuthology.orchestra.run.smithi130.stdout:done 2024-04-24T08:41:43.115 INFO:teuthology.orchestra.run.smithi130.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T08:41:43.115 INFO:teuthology.orchestra.run.smithi130.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T08:41:43.115 INFO:teuthology.orchestra.run.smithi130.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:41:43.115 INFO:teuthology.orchestra.run.smithi130.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:41:43.115 INFO:teuthology.orchestra.run.smithi130.stdout:fi 2024-04-24T08:41:43.116 INFO:teuthology.orchestra.run.smithi130.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T08:41:43.116 INFO:teuthology.orchestra.run.smithi130.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:43.116 INFO:teuthology.orchestra.run.smithi130.stdout:fi 2024-04-24T08:41:43.116 INFO:teuthology.orchestra.run.smithi130.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T08:41:43.116 INFO:teuthology.orchestra.run.smithi130.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T08:41:43.116 INFO:teuthology.orchestra.run.smithi130.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:43.116 INFO:teuthology.orchestra.run.smithi130.stdout:fi 2024-04-24T08:41:43.117 DEBUG:teuthology.orchestra.run.smithi130:> sudo rpm -qi grub2-tools 2024-04-24T08:41:43.198 INFO:teuthology.orchestra.run.smithi130.stdout:Name : grub2-tools 2024-04-24T08:41:43.198 INFO:teuthology.orchestra.run.smithi130.stdout:Epoch : 1 2024-04-24T08:41:43.198 INFO:teuthology.orchestra.run.smithi130.stdout:Version : 2.06 2024-04-24T08:41:43.198 INFO:teuthology.orchestra.run.smithi130.stdout:Release : 68.el9 2024-04-24T08:41:43.198 INFO:teuthology.orchestra.run.smithi130.stdout:Architecture: x86_64 2024-04-24T08:41:43.198 INFO:teuthology.orchestra.run.smithi130.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T08:41:43.198 INFO:teuthology.orchestra.run.smithi130.stdout:Group : Unspecified 2024-04-24T08:41:43.198 INFO:teuthology.orchestra.run.smithi130.stdout:Size : 8264254 2024-04-24T08:41:43.198 INFO:teuthology.orchestra.run.smithi130.stdout:License : GPLv3+ 2024-04-24T08:41:43.199 INFO:teuthology.orchestra.run.smithi130.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T08:41:43.199 INFO:teuthology.orchestra.run.smithi130.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T08:41:43.199 INFO:teuthology.orchestra.run.smithi130.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T08:41:43.199 INFO:teuthology.orchestra.run.smithi130.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T08:41:43.199 INFO:teuthology.orchestra.run.smithi130.stdout:Packager : builder@centos.org 2024-04-24T08:41:43.199 INFO:teuthology.orchestra.run.smithi130.stdout:Vendor : CentOS 2024-04-24T08:41:43.199 INFO:teuthology.orchestra.run.smithi130.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T08:41:43.199 INFO:teuthology.orchestra.run.smithi130.stdout:Summary : Support tools for GRUB. 2024-04-24T08:41:43.199 INFO:teuthology.orchestra.run.smithi130.stdout:Description : 2024-04-24T08:41:43.199 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:41:43.199 INFO:teuthology.orchestra.run.smithi130.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T08:41:43.199 INFO:teuthology.orchestra.run.smithi130.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T08:41:43.199 INFO:teuthology.orchestra.run.smithi130.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T08:41:43.199 INFO:teuthology.orchestra.run.smithi130.stdout:hardware devices. 2024-04-24T08:41:43.199 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:41:43.200 INFO:teuthology.orchestra.run.smithi130.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T08:41:43.200 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T08:41:43.200 INFO:teuthology.task.kernel:Updating grub on smithi130 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:43.201 DEBUG:teuthology.orchestra.run.smithi130:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T08:41:43.748 INFO:teuthology.orchestra.run.smithi130.stderr:Generating grub configuration file ... 2024-04-24T08:41:45.186 INFO:teuthology.orchestra.run.smithi130.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T08:41:45.217 INFO:teuthology.orchestra.run.smithi130.stderr:done 2024-04-24T08:41:45.219 DEBUG:teuthology.orchestra.run.smithi130:> mktemp 2024-04-24T08:41:45.237 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/tmp.2y215AIm2c 2024-04-24T08:41:45.237 DEBUG:teuthology.orchestra.run.smithi130:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2y215AIm2c 2024-04-24T08:41:45.317 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 0666 /tmp/tmp.2y215AIm2c 2024-04-24T08:41:45.451 DEBUG:teuthology.orchestra.remote:smithi130:/tmp/tmp.2y215AIm2c is 6KB 2024-04-24T08:41:45.500 DEBUG:teuthology.orchestra.run.smithi130:> rm -fr /tmp/tmp.2y215AIm2c 2024-04-24T08:41:45.515 DEBUG:teuthology.orchestra.run.smithi130:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T08:41:45.576 INFO:teuthology.orchestra.run.smithi130.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T08:41:45.576 INFO:teuthology.orchestra.run.smithi130.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T08:41:45.577 INFO:teuthology.orchestra.run.smithi130.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T08:41:45.577 INFO:teuthology.orchestra.run.smithi130.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T08:41:45.578 DEBUG:teuthology.orchestra.run.smithi130:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:45.735 DEBUG:teuthology.orchestra.run.smithi130:> sudo shutdown -r now 2024-04-24T08:41:55.439 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi069.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T08:41:55.439 DEBUG:teuthology.orchestra.run.smithi069:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T08:41:55.524 INFO:teuthology.orchestra.run.smithi069.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:55.524 INFO:teuthology.orchestra.run.smithi069.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:55.524 INFO:teuthology.orchestra.run.smithi069.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:55.525 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T08:41:55.525 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T08:41:55.525 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T08:41:55.525 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T08:41:55.525 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T08:41:55.525 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T08:41:55.525 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T08:41:55.525 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T08:41:55.525 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T08:41:55.525 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T08:41:55.526 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T08:41:55.526 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T08:41:55.526 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T08:41:55.526 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T08:41:55.526 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T08:41:55.526 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T08:41:55.526 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T08:41:55.526 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T08:41:55.527 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T08:41:55.527 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T08:41:55.527 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T08:41:55.527 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T08:41:55.527 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T08:41:55.527 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T08:41:55.527 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T08:41:55.527 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T08:41:55.527 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T08:41:55.527 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T08:41:55.527 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T08:41:55.527 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T08:41:55.527 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T08:41:55.527 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T08:41:55.527 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T08:41:55.528 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T08:41:55.528 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T08:41:55.528 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T08:41:55.528 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T08:41:55.528 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T08:41:55.528 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T08:41:55.528 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T08:41:55.528 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T08:41:55.528 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T08:41:55.528 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T08:41:55.528 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T08:41:55.528 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T08:41:55.528 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T08:41:55.528 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T08:41:55.529 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T08:41:55.529 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T08:41:55.529 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T08:41:55.529 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T08:41:55.529 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T08:41:55.529 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T08:41:55.529 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T08:41:55.529 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T08:41:55.529 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T08:41:55.529 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T08:41:55.529 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T08:41:55.529 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T08:41:55.529 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T08:41:55.529 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T08:41:55.529 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T08:41:55.530 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T08:41:55.530 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T08:41:55.530 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T08:41:55.530 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T08:41:55.530 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T08:41:55.530 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T08:41:55.530 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T08:41:55.530 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T08:41:55.530 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T08:41:55.530 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T08:41:55.530 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T08:41:55.530 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T08:41:55.530 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T08:41:55.531 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T08:41:55.531 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T08:41:55.531 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T08:41:55.531 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T08:41:55.531 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T08:41:55.531 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T08:41:55.531 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T08:41:55.531 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T08:41:55.531 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T08:41:55.531 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T08:41:55.531 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T08:41:55.531 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T08:41:55.531 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T08:41:55.531 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T08:41:55.531 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T08:41:55.532 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T08:41:55.532 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T08:41:55.532 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T08:41:55.532 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T08:41:55.532 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T08:41:55.532 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T08:41:55.532 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T08:41:55.532 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T08:41:55.532 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T08:41:55.532 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T08:41:55.532 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T08:41:55.532 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T08:41:55.532 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T08:41:55.532 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T08:41:55.532 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T08:41:55.533 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T08:41:55.533 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T08:41:55.533 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T08:41:55.533 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T08:41:55.533 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T08:41:55.533 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T08:41:55.533 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T08:41:55.533 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T08:41:55.533 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T08:41:55.533 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T08:41:55.533 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T08:41:55.533 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T08:41:55.533 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T08:41:55.533 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T08:41:55.533 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T08:41:55.534 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T08:41:55.534 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T08:41:55.534 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T08:41:55.534 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T08:41:55.534 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T08:41:55.534 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T08:41:55.534 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T08:41:55.534 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T08:41:55.534 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T08:41:55.534 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T08:41:55.534 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T08:41:55.534 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T08:41:55.534 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T08:41:55.534 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T08:41:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T08:41:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T08:41:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T08:41:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T08:41:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T08:41:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T08:41:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T08:41:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T08:41:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T08:41:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T08:41:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T08:41:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T08:41:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T08:41:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T08:41:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T08:41:55.536 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T08:41:55.536 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T08:41:55.536 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T08:41:55.536 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T08:41:55.536 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T08:41:55.536 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T08:41:55.536 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T08:41:55.536 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T08:41:55.536 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T08:41:55.537 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T08:41:55.537 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T08:41:55.537 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T08:41:55.537 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T08:41:55.537 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T08:41:55.537 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T08:41:55.538 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T08:41:55.538 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T08:41:55.538 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T08:41:55.538 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T08:41:55.538 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T08:41:55.538 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T08:41:55.539 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T08:41:55.539 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T08:41:55.539 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T08:41:55.539 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T08:41:55.539 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T08:41:55.539 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T08:41:55.539 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T08:41:55.540 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T08:41:55.540 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T08:41:55.540 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T08:41:55.540 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T08:41:55.540 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T08:41:55.540 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T08:41:55.540 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T08:41:55.540 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T08:41:55.540 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T08:41:55.540 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T08:41:55.540 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T08:41:55.540 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T08:41:55.541 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T08:41:55.541 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T08:41:55.541 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T08:41:55.541 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T08:41:55.541 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T08:41:55.541 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T08:41:55.541 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T08:41:55.542 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T08:41:55.542 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T08:41:55.542 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T08:41:55.542 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T08:41:55.542 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T08:41:55.542 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T08:41:55.542 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T08:41:55.542 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T08:41:55.542 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T08:41:55.542 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T08:41:55.542 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T08:41:55.542 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T08:41:55.542 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T08:41:55.543 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T08:41:55.543 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T08:41:55.543 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T08:41:55.543 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T08:41:55.543 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T08:41:55.543 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T08:41:55.543 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T08:41:55.543 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T08:41:55.543 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T08:41:55.543 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T08:41:55.543 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T08:41:55.543 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T08:41:55.543 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T08:41:55.543 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T08:41:55.543 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T08:41:55.543 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T08:41:55.544 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T08:41:55.544 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T08:41:55.544 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T08:41:55.544 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T08:41:55.544 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T08:41:55.544 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T08:41:55.544 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T08:41:55.544 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T08:41:55.544 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T08:41:55.544 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T08:41:55.544 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T08:41:55.544 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T08:41:55.544 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T08:41:55.544 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T08:41:55.545 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T08:41:55.545 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T08:41:55.545 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T08:41:55.545 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T08:41:55.545 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T08:41:55.545 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T08:41:55.545 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T08:41:55.545 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T08:41:55.545 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T08:41:55.545 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T08:41:55.545 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T08:41:55.545 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T08:41:55.545 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T08:41:55.545 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T08:41:55.545 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T08:41:55.546 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T08:41:55.546 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T08:41:55.546 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T08:41:55.546 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T08:41:55.546 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T08:41:55.546 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T08:41:55.546 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T08:41:55.546 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T08:41:55.546 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T08:41:55.546 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T08:41:55.546 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T08:41:55.546 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T08:41:55.546 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T08:41:55.546 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T08:41:55.547 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T08:41:55.547 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T08:41:55.547 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T08:41:55.547 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T08:41:55.547 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T08:41:55.547 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T08:41:55.547 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T08:41:55.547 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T08:41:55.547 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T08:41:55.547 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T08:41:55.547 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T08:41:55.547 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T08:41:55.547 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T08:41:55.547 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T08:41:55.547 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T08:41:55.548 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T08:41:55.548 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T08:41:55.548 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T08:41:55.548 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T08:41:55.548 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T08:41:55.548 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T08:41:55.548 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T08:41:55.548 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T08:41:55.548 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T08:41:55.548 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T08:41:55.548 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T08:41:55.548 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T08:41:55.548 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T08:41:55.548 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T08:41:55.548 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T08:41:55.549 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T08:41:55.549 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T08:41:55.549 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T08:41:55.549 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T08:41:55.549 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T08:41:55.549 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T08:41:55.549 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T08:41:55.549 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T08:41:55.549 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T08:41:55.549 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T08:41:55.549 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T08:41:55.549 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T08:41:55.549 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T08:41:55.549 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T08:41:55.549 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T08:41:55.550 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T08:41:55.550 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T08:41:55.550 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T08:41:55.550 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T08:41:55.550 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T08:41:55.550 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T08:41:55.550 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T08:41:55.550 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T08:41:55.550 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T08:41:55.550 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T08:41:55.550 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T08:41:55.550 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T08:41:55.550 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T08:41:55.550 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T08:41:55.550 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T08:41:55.551 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T08:41:55.551 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T08:41:55.551 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T08:41:55.551 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T08:41:55.551 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T08:41:55.551 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T08:41:55.551 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T08:41:55.551 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T08:41:55.551 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T08:41:55.551 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T08:41:55.551 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T08:41:55.551 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T08:41:55.551 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T08:41:55.551 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T08:41:55.551 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T08:41:55.552 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T08:41:55.552 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T08:41:55.552 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T08:41:55.552 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T08:41:55.552 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T08:41:55.552 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T08:41:55.552 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T08:41:55.552 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T08:41:55.552 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T08:41:55.552 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T08:41:55.552 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T08:41:55.552 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T08:41:55.552 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T08:41:55.552 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T08:41:55.553 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T08:41:55.553 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T08:41:55.553 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T08:41:55.553 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T08:41:55.553 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T08:41:55.553 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T08:41:55.553 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T08:41:55.553 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T08:41:55.553 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T08:41:55.553 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T08:41:55.553 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T08:41:55.553 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T08:41:55.553 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T08:41:55.553 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T08:41:55.554 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T08:41:55.554 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T08:41:55.554 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T08:41:55.554 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T08:41:55.554 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T08:41:55.555 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T08:41:55.555 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T08:41:55.555 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T08:41:55.555 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T08:41:55.555 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T08:41:55.555 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T08:41:55.555 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T08:41:55.555 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T08:41:55.555 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T08:41:55.555 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T08:41:55.555 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T08:41:55.555 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T08:41:55.555 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T08:41:55.556 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T08:41:55.556 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T08:41:55.556 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T08:41:55.556 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T08:41:55.556 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T08:41:55.556 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T08:41:55.556 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T08:41:55.556 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T08:41:55.556 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T08:41:55.556 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T08:41:55.557 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T08:41:55.557 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T08:41:55.557 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T08:41:55.557 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T08:41:55.557 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T08:41:55.557 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T08:41:55.557 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T08:41:55.558 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T08:41:55.558 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T08:41:55.558 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T08:41:55.558 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T08:41:55.558 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T08:41:55.558 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T08:41:55.558 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T08:41:55.558 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T08:41:55.558 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T08:41:55.558 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T08:41:55.558 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T08:41:55.558 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T08:41:55.558 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T08:41:55.559 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T08:41:55.559 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T08:41:55.559 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T08:41:55.559 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T08:41:55.559 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T08:41:55.559 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T08:41:55.559 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T08:41:55.559 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T08:41:55.559 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T08:41:55.559 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T08:41:55.559 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T08:41:55.559 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T08:41:55.559 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T08:41:55.559 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T08:41:55.559 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T08:41:55.560 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T08:41:55.560 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T08:41:55.560 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T08:41:55.560 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T08:41:55.560 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T08:41:55.560 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T08:41:55.560 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T08:41:55.560 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T08:41:55.560 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T08:41:55.561 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T08:41:55.561 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T08:41:55.561 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T08:41:55.561 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T08:41:55.561 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T08:41:55.561 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T08:41:55.562 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T08:41:55.562 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T08:41:55.562 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T08:41:55.562 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T08:41:55.562 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T08:41:55.562 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T08:41:55.562 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T08:41:55.562 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T08:41:55.562 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T08:41:55.562 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T08:41:55.562 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T08:41:55.562 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T08:41:55.563 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T08:41:55.563 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T08:41:55.563 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T08:41:55.563 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T08:41:55.563 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T08:41:55.563 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T08:41:55.563 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T08:41:55.564 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T08:41:55.564 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T08:41:55.564 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T08:41:55.564 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T08:41:55.564 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T08:41:55.564 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T08:41:55.564 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T08:41:55.564 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T08:41:55.564 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T08:41:55.564 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T08:41:55.564 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T08:41:55.565 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T08:41:55.565 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T08:41:55.565 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T08:41:55.565 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T08:41:55.565 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T08:41:55.565 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T08:41:55.565 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T08:41:55.565 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T08:41:55.565 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T08:41:55.565 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T08:41:55.565 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T08:41:55.565 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T08:41:55.565 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T08:41:55.565 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T08:41:55.565 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T08:41:55.566 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T08:41:55.566 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T08:41:55.566 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T08:41:55.566 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T08:41:55.566 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T08:41:55.566 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T08:41:55.566 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T08:41:55.566 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T08:41:55.566 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T08:41:55.566 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T08:41:55.566 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T08:41:55.567 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T08:41:55.567 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T08:41:55.567 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T08:41:55.567 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T08:41:55.567 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T08:41:55.567 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T08:41:55.567 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T08:41:55.567 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T08:41:55.567 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T08:41:55.567 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T08:41:55.567 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T08:41:55.567 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T08:41:55.567 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T08:41:55.567 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T08:41:55.568 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T08:41:55.568 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T08:41:55.568 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T08:41:55.568 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T08:41:55.568 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T08:41:55.568 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T08:41:55.568 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T08:41:55.568 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T08:41:55.568 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T08:41:55.568 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T08:41:55.568 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T08:41:55.568 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T08:41:55.568 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T08:41:55.568 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T08:41:55.569 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T08:41:55.569 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T08:41:55.569 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T08:41:55.569 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T08:41:55.569 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T08:41:55.569 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T08:41:55.569 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T08:41:55.569 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T08:41:55.569 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T08:41:55.569 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T08:41:55.569 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T08:41:55.569 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T08:41:55.569 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T08:41:55.569 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T08:41:55.569 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T08:41:55.570 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T08:41:55.570 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T08:41:55.570 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T08:41:55.570 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T08:41:55.570 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T08:41:55.570 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T08:41:55.570 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T08:41:55.570 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T08:41:55.570 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T08:41:55.570 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T08:41:55.570 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T08:41:55.570 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T08:41:55.570 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T08:41:55.570 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T08:41:55.571 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T08:41:55.571 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T08:41:55.571 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T08:41:55.571 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T08:41:55.571 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T08:41:55.571 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T08:41:55.571 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T08:41:55.571 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T08:41:55.571 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T08:41:55.571 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T08:41:55.571 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T08:41:55.571 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T08:41:55.571 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T08:41:55.571 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T08:41:55.572 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T08:41:55.572 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T08:41:55.572 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T08:41:55.572 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T08:41:55.572 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T08:41:55.572 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T08:41:55.572 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T08:41:55.572 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T08:41:55.572 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T08:41:55.572 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T08:41:55.572 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T08:41:55.572 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T08:41:55.572 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T08:41:55.572 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T08:41:55.572 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T08:41:55.573 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T08:41:55.573 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T08:41:55.573 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T08:41:55.573 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T08:41:55.573 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T08:41:55.573 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T08:41:55.573 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T08:41:55.573 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T08:41:55.573 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T08:41:55.573 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T08:41:55.574 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T08:41:55.574 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T08:41:55.574 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T08:41:55.574 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T08:41:55.574 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T08:41:55.574 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T08:41:55.574 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T08:41:55.574 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T08:41:55.574 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T08:41:55.574 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T08:41:55.574 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T08:41:55.574 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T08:41:55.575 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T08:41:55.575 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T08:41:55.575 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T08:41:55.575 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T08:41:55.575 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T08:41:55.575 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T08:41:55.576 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T08:41:55.576 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T08:41:55.576 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T08:41:55.576 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T08:41:55.576 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T08:41:55.576 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T08:41:55.577 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T08:41:55.577 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T08:41:55.577 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T08:41:55.577 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T08:41:55.577 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T08:41:55.577 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T08:41:55.577 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T08:41:55.577 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T08:41:55.577 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T08:41:55.577 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T08:41:55.577 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T08:41:55.578 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T08:41:55.578 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T08:41:55.578 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T08:41:55.578 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T08:41:55.578 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T08:41:55.578 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T08:41:55.578 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T08:41:55.578 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T08:41:55.578 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T08:41:55.578 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T08:41:55.578 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T08:41:55.578 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T08:41:55.579 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T08:41:55.579 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T08:41:55.579 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T08:41:55.579 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T08:41:55.579 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T08:41:55.579 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T08:41:55.579 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T08:41:55.579 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T08:41:55.579 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T08:41:55.579 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T08:41:55.579 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T08:41:55.579 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T08:41:55.580 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T08:41:55.580 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T08:41:55.580 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T08:41:55.580 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T08:41:55.580 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T08:41:55.580 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T08:41:55.580 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T08:41:55.580 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T08:41:55.580 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T08:41:55.580 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T08:41:55.580 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T08:41:55.580 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T08:41:55.580 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T08:41:55.580 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T08:41:55.580 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T08:41:55.581 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T08:41:55.581 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T08:41:55.581 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T08:41:55.581 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T08:41:55.581 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T08:41:55.581 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T08:41:55.581 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T08:41:55.581 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T08:41:55.581 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T08:41:55.581 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T08:41:55.581 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T08:41:55.582 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T08:41:55.582 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T08:41:55.582 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T08:41:55.582 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T08:41:55.582 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T08:41:55.582 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T08:41:55.582 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T08:41:55.582 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T08:41:55.582 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T08:41:55.582 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T08:41:55.582 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T08:41:55.582 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T08:41:55.583 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T08:41:55.583 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T08:41:55.583 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T08:41:55.583 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T08:41:55.583 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T08:41:55.583 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T08:41:55.583 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T08:41:55.583 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T08:41:55.583 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T08:41:55.583 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T08:41:55.583 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T08:41:55.583 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T08:41:55.583 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T08:41:55.584 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T08:41:55.584 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T08:41:55.584 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T08:41:55.584 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T08:41:55.584 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T08:41:55.584 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T08:41:55.584 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T08:41:55.584 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T08:41:55.584 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T08:41:55.584 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T08:41:55.584 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T08:41:55.584 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T08:41:55.584 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T08:41:55.584 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T08:41:55.585 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T08:41:55.585 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T08:41:55.585 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T08:41:55.585 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T08:41:55.585 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T08:41:55.585 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T08:41:55.585 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T08:41:55.585 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T08:41:55.585 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T08:41:55.585 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T08:41:55.585 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T08:41:55.585 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T08:41:55.585 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T08:41:55.585 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T08:41:55.586 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T08:41:55.586 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T08:41:55.586 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T08:41:55.586 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T08:41:55.586 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T08:41:55.586 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T08:41:55.586 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T08:41:55.586 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T08:41:55.586 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T08:41:55.586 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T08:41:55.586 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T08:41:55.586 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T08:41:55.586 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T08:41:55.586 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T08:41:55.586 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T08:41:55.587 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T08:41:55.587 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T08:41:55.587 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T08:41:55.587 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T08:41:55.587 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T08:41:55.587 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T08:41:55.587 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T08:41:55.587 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T08:41:55.587 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T08:41:55.587 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T08:41:55.587 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T08:41:55.587 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T08:41:55.587 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T08:41:55.587 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T08:41:55.588 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T08:41:55.588 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T08:41:55.588 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T08:41:55.588 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T08:41:55.588 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T08:41:55.588 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T08:41:55.588 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T08:41:55.588 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T08:41:55.588 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T08:41:55.588 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T08:41:55.588 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T08:41:55.588 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T08:41:55.588 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T08:41:55.589 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T08:41:55.589 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T08:41:55.589 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T08:41:55.589 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T08:41:55.589 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T08:41:55.589 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T08:41:55.589 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T08:41:55.589 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T08:41:55.589 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T08:41:55.589 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T08:41:55.589 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T08:41:55.589 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T08:41:55.589 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T08:41:55.589 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T08:41:55.590 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T08:41:55.590 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T08:41:55.590 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T08:41:55.590 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T08:41:55.590 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T08:41:55.590 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T08:41:55.590 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T08:41:55.590 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T08:41:55.590 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T08:41:55.590 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T08:41:55.590 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T08:41:55.590 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T08:41:55.590 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T08:41:55.590 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T08:41:55.591 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T08:41:55.591 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T08:41:55.591 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T08:41:55.591 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T08:41:55.591 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T08:41:55.591 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T08:41:55.591 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T08:41:55.591 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T08:41:55.591 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T08:41:55.591 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T08:41:55.591 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T08:41:55.591 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T08:41:55.591 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T08:41:55.591 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T08:41:55.591 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T08:41:55.592 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T08:41:55.592 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T08:41:55.592 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T08:41:55.592 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T08:41:55.592 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T08:41:55.592 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T08:41:55.592 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T08:41:55.592 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T08:41:55.592 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T08:41:55.592 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T08:41:55.592 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T08:41:55.592 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T08:41:55.593 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T08:41:55.593 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T08:41:55.593 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T08:41:55.593 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T08:41:55.593 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T08:41:55.593 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T08:41:55.593 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T08:41:55.593 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T08:41:55.593 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T08:41:55.593 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T08:41:55.593 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T08:41:55.593 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T08:41:55.593 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T08:41:55.593 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T08:41:55.593 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T08:41:55.594 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T08:41:55.594 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T08:41:55.594 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T08:41:55.594 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T08:41:55.594 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T08:41:55.594 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T08:41:55.594 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T08:41:55.594 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T08:41:55.594 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T08:41:55.594 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T08:41:55.594 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T08:41:55.594 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T08:41:55.594 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T08:41:55.594 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T08:41:55.594 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T08:41:55.595 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T08:41:55.595 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T08:41:55.595 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T08:41:55.595 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T08:41:55.595 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T08:41:55.595 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T08:41:55.595 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T08:41:55.595 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T08:41:55.595 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T08:41:55.595 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T08:41:55.595 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T08:41:55.595 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T08:41:55.596 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T08:41:55.596 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T08:41:55.596 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T08:41:55.596 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T08:41:55.596 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T08:41:55.596 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T08:41:55.596 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T08:41:55.596 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T08:41:55.596 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T08:41:55.596 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T08:41:55.596 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T08:41:55.596 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T08:41:55.596 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T08:41:55.597 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T08:41:55.597 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T08:41:55.597 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T08:41:55.597 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T08:41:55.597 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T08:41:55.597 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T08:41:55.597 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T08:41:55.597 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T08:41:55.597 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T08:41:55.597 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T08:41:55.597 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T08:41:55.597 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T08:41:55.597 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T08:41:55.597 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T08:41:55.597 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T08:41:55.598 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T08:41:55.598 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T08:41:55.598 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T08:41:55.598 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T08:41:55.598 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T08:41:55.598 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T08:41:55.598 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T08:41:55.598 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T08:41:55.598 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T08:41:55.598 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T08:41:55.598 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T08:41:55.599 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T08:41:55.599 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T08:41:55.599 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T08:41:55.599 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T08:41:55.599 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T08:41:55.599 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T08:41:55.599 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T08:41:55.599 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T08:41:55.599 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T08:41:55.599 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T08:41:55.599 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T08:41:55.599 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T08:41:55.599 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T08:41:55.599 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T08:41:55.599 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T08:41:55.600 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T08:41:55.600 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T08:41:55.600 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T08:41:55.600 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T08:41:55.600 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T08:41:55.600 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T08:41:55.600 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T08:41:55.600 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T08:41:55.600 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T08:41:55.600 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T08:41:55.600 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T08:41:55.600 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T08:41:55.600 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T08:41:55.600 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T08:41:55.601 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T08:41:55.601 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T08:41:55.601 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T08:41:55.601 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T08:41:55.601 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T08:41:55.601 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T08:41:55.601 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T08:41:55.601 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T08:41:55.601 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T08:41:55.601 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T08:41:55.601 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T08:41:55.601 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T08:41:55.601 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T08:41:55.601 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T08:41:55.601 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T08:41:55.602 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T08:41:55.602 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T08:41:55.602 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T08:41:55.602 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T08:41:55.602 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T08:41:55.602 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T08:41:55.602 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T08:41:55.602 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T08:41:55.602 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T08:41:55.603 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T08:41:55.603 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T08:41:55.603 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T08:41:55.603 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T08:41:55.603 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T08:41:55.603 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T08:41:55.603 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T08:41:55.603 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T08:41:55.603 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T08:41:55.603 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T08:41:55.603 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T08:41:55.604 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T08:41:55.604 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T08:41:55.604 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T08:41:55.604 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T08:41:55.604 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T08:41:55.604 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T08:41:55.604 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T08:41:55.604 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T08:41:55.604 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T08:41:55.604 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T08:41:55.604 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T08:41:55.604 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T08:41:55.604 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T08:41:55.604 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T08:41:55.604 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T08:41:55.605 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T08:41:55.605 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T08:41:55.605 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T08:41:55.605 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T08:41:55.605 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T08:41:55.605 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T08:41:55.606 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T08:41:55.606 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T08:41:55.606 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T08:41:55.606 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T08:41:55.606 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T08:41:55.606 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T08:41:55.606 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T08:41:55.607 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T08:41:55.607 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T08:41:55.607 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T08:41:55.607 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T08:41:55.607 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T08:41:55.607 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T08:41:55.607 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T08:41:55.607 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T08:41:55.607 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T08:41:55.607 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T08:41:55.607 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T08:41:55.607 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T08:41:55.607 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T08:41:55.607 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T08:41:55.608 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T08:41:55.608 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T08:41:55.608 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T08:41:55.608 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T08:41:55.608 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T08:41:55.608 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T08:41:55.608 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T08:41:55.608 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T08:41:55.608 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T08:41:55.608 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T08:41:55.608 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T08:41:55.608 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T08:41:55.608 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T08:41:55.608 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T08:41:55.609 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T08:41:55.609 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T08:41:55.609 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T08:41:55.609 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T08:41:55.609 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T08:41:55.609 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T08:41:55.609 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T08:41:55.609 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T08:41:55.609 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T08:41:55.609 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T08:41:55.609 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T08:41:55.609 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T08:41:55.609 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T08:41:55.609 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T08:41:55.609 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T08:41:55.609 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T08:41:55.610 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T08:41:55.610 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T08:41:55.610 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T08:41:55.610 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T08:41:55.610 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T08:41:55.610 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T08:41:55.610 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T08:41:55.610 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T08:41:55.610 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T08:41:55.610 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T08:41:55.610 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T08:41:55.610 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T08:41:55.610 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T08:41:55.611 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T08:41:55.611 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T08:41:55.611 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T08:41:55.611 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T08:41:55.611 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T08:41:55.611 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T08:41:55.611 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T08:41:55.611 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T08:41:55.611 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T08:41:55.611 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T08:41:55.611 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T08:41:55.611 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T08:41:55.611 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T08:41:55.611 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T08:41:55.612 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T08:41:55.612 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T08:41:55.612 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T08:41:55.612 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T08:41:55.612 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T08:41:55.612 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T08:41:55.612 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T08:41:55.612 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T08:41:55.612 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T08:41:55.612 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T08:41:55.612 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T08:41:55.612 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T08:41:55.612 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T08:41:55.612 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T08:41:55.613 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T08:41:55.613 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T08:41:55.613 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T08:41:55.613 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T08:41:55.613 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T08:41:55.613 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T08:41:55.613 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T08:41:55.613 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T08:41:55.613 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T08:41:55.613 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T08:41:55.613 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T08:41:55.613 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T08:41:55.613 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T08:41:55.613 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T08:41:55.613 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T08:41:55.613 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T08:41:55.614 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T08:41:55.614 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T08:41:55.614 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T08:41:55.614 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T08:41:55.614 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T08:41:55.614 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T08:41:55.614 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T08:41:55.614 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T08:41:55.614 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T08:41:55.614 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T08:41:55.615 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T08:41:55.615 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T08:41:55.615 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T08:41:55.615 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T08:41:55.615 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T08:41:55.615 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T08:41:55.616 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T08:41:55.616 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T08:41:55.616 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T08:41:55.616 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T08:41:55.616 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T08:41:55.616 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T08:41:55.617 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T08:41:55.617 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T08:41:55.617 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T08:41:55.617 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T08:41:55.617 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T08:41:55.617 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T08:41:55.618 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T08:41:55.618 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T08:41:55.618 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T08:41:55.618 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T08:41:55.618 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T08:41:55.618 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T08:41:55.619 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T08:41:55.619 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T08:41:55.619 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T08:41:55.619 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T08:41:55.619 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T08:41:55.619 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T08:41:55.619 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T08:41:55.619 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T08:41:55.619 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T08:41:55.619 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T08:41:55.619 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T08:41:55.619 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T08:41:55.619 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T08:41:55.619 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T08:41:55.620 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T08:41:55.620 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T08:41:55.620 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T08:41:55.620 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T08:41:55.620 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T08:41:55.620 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T08:41:55.620 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T08:41:55.620 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T08:41:55.620 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T08:41:55.620 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T08:41:55.620 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T08:41:55.620 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T08:41:55.620 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T08:41:55.620 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T08:41:55.620 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T08:41:55.621 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T08:41:55.621 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T08:41:55.621 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T08:41:55.621 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T08:41:55.621 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T08:41:55.621 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T08:41:55.621 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T08:41:55.621 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T08:41:55.621 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T08:41:55.621 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T08:41:55.621 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T08:41:55.621 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T08:41:55.621 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T08:41:55.621 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T08:41:55.621 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T08:41:55.622 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T08:41:55.622 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T08:41:55.622 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T08:41:55.622 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T08:41:55.622 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T08:41:55.622 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T08:41:55.622 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T08:41:55.622 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T08:41:55.622 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T08:41:55.622 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T08:41:55.622 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T08:41:55.622 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T08:41:55.622 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T08:41:55.622 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T08:41:55.622 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T08:41:55.623 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T08:41:55.623 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T08:41:55.623 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T08:41:55.623 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T08:41:55.623 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T08:41:55.623 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T08:41:55.623 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T08:41:55.623 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T08:41:55.623 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T08:41:55.623 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T08:41:55.623 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T08:41:55.623 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T08:41:55.623 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T08:41:55.623 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T08:41:55.624 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T08:41:55.624 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T08:41:55.624 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T08:41:55.624 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T08:41:55.624 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T08:41:55.624 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T08:41:55.624 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T08:41:55.624 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T08:41:55.624 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T08:41:55.624 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T08:41:55.624 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T08:41:55.624 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T08:41:55.624 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T08:41:55.624 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T08:41:55.624 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T08:41:55.625 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T08:41:55.625 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T08:41:55.625 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T08:41:55.625 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T08:41:55.625 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T08:41:55.625 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T08:41:55.625 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T08:41:55.625 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T08:41:55.625 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T08:41:55.625 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T08:41:55.625 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T08:41:55.625 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T08:41:55.625 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T08:41:55.625 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T08:41:55.626 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T08:41:55.626 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T08:41:55.626 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T08:41:55.626 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T08:41:55.626 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T08:41:55.626 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T08:41:55.626 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T08:41:55.626 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T08:41:55.626 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T08:41:55.626 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T08:41:55.626 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T08:41:55.626 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T08:41:55.626 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T08:41:55.626 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T08:41:55.627 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T08:41:55.627 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T08:41:55.627 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T08:41:55.627 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T08:41:55.627 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T08:41:55.627 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T08:41:55.627 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T08:41:55.627 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T08:41:55.627 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T08:41:55.627 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T08:41:55.627 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T08:41:55.627 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T08:41:55.627 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T08:41:55.627 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T08:41:55.627 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T08:41:55.628 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T08:41:55.628 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T08:41:55.628 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T08:41:55.628 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T08:41:55.628 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T08:41:55.628 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T08:41:55.628 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T08:41:55.628 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T08:41:55.628 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T08:41:55.628 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T08:41:55.628 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T08:41:55.628 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T08:41:55.628 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T08:41:55.629 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T08:41:55.629 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T08:41:55.629 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T08:41:55.629 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T08:41:55.629 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T08:41:55.629 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T08:41:55.629 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T08:41:55.629 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T08:41:55.629 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T08:41:55.629 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T08:41:55.629 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T08:41:55.629 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T08:41:55.629 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T08:41:55.630 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T08:41:55.630 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T08:41:55.630 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T08:41:55.630 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T08:41:55.630 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T08:41:55.630 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T08:41:55.630 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T08:41:55.630 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T08:41:55.630 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T08:41:55.630 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T08:41:55.630 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T08:41:55.630 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T08:41:55.630 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T08:41:55.630 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T08:41:55.631 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T08:41:55.631 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T08:41:55.631 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T08:41:55.631 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T08:41:55.631 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T08:41:55.631 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T08:41:55.631 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T08:41:55.631 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T08:41:55.631 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T08:41:55.631 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T08:41:55.631 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T08:41:55.631 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T08:41:55.631 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T08:41:55.631 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T08:41:55.631 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T08:41:55.632 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T08:41:55.632 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T08:41:55.632 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T08:41:55.632 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T08:41:55.632 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T08:41:55.632 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T08:41:55.632 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T08:41:55.632 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T08:41:55.632 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T08:41:55.632 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T08:41:55.632 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T08:41:55.632 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T08:41:55.632 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T08:41:55.632 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T08:41:55.632 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T08:41:55.633 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T08:41:55.633 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T08:41:55.633 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T08:41:55.633 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T08:41:55.633 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T08:41:55.633 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T08:41:55.633 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T08:41:55.633 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T08:41:55.633 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T08:41:55.633 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T08:41:55.633 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T08:41:55.633 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T08:41:55.633 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T08:41:55.633 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T08:41:55.634 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T08:41:55.634 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T08:41:55.634 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T08:41:55.634 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T08:41:55.634 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T08:41:55.634 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T08:41:55.634 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T08:41:55.634 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T08:41:55.634 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T08:41:55.634 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T08:41:55.634 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T08:41:55.634 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T08:41:55.634 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T08:41:55.634 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T08:41:55.635 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T08:41:55.635 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T08:41:55.635 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T08:41:55.635 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T08:41:55.635 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T08:41:55.635 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T08:41:55.635 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T08:41:55.635 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T08:41:55.635 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T08:41:55.635 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T08:41:55.635 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T08:41:55.635 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T08:41:55.635 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T08:41:55.636 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T08:41:55.636 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T08:41:55.636 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T08:41:55.636 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T08:41:55.636 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T08:41:55.636 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T08:41:55.636 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T08:41:55.636 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T08:41:55.636 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T08:41:55.636 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T08:41:55.636 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T08:41:55.636 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T08:41:55.636 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T08:41:55.636 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T08:41:55.636 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T08:41:55.637 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T08:41:55.637 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T08:41:55.637 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T08:41:55.637 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T08:41:55.637 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T08:41:55.637 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T08:41:55.637 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T08:41:55.637 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T08:41:55.637 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T08:41:55.637 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T08:41:55.637 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T08:41:55.637 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T08:41:55.637 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T08:41:55.637 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T08:41:55.637 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T08:41:55.638 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T08:41:55.638 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T08:41:55.638 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T08:41:55.638 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T08:41:55.638 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T08:41:55.638 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T08:41:55.638 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T08:41:55.638 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T08:41:55.639 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T08:41:55.639 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T08:41:55.639 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T08:41:55.639 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T08:41:55.639 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T08:41:55.639 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T08:41:55.639 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T08:41:55.639 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T08:41:55.639 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T08:41:55.639 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T08:41:55.639 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T08:41:55.640 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T08:41:55.640 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T08:41:55.640 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T08:41:55.640 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T08:41:55.640 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T08:41:55.640 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T08:41:55.640 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T08:41:55.640 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T08:41:55.640 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T08:41:55.640 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T08:41:55.640 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T08:41:55.640 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T08:41:55.640 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T08:41:55.640 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T08:41:55.640 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T08:41:55.641 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T08:41:55.641 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T08:41:55.641 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T08:41:55.641 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T08:41:55.641 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T08:41:55.641 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T08:41:55.641 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T08:41:55.641 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T08:41:55.641 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T08:41:55.641 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T08:41:55.641 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T08:41:55.641 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T08:41:55.641 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T08:41:55.641 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T08:41:55.642 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T08:41:55.642 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T08:41:55.642 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T08:41:55.642 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T08:41:55.642 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T08:41:55.642 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T08:41:55.642 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T08:41:55.642 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T08:41:55.642 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T08:41:55.642 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T08:41:55.643 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T08:41:55.643 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T08:41:55.643 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T08:41:55.643 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T08:41:55.643 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T08:41:55.643 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T08:41:55.643 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T08:41:55.643 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T08:41:55.643 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T08:41:55.643 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T08:41:55.643 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T08:41:55.643 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T08:41:55.643 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T08:41:55.643 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T08:41:55.643 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T08:41:55.644 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T08:41:55.644 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T08:41:55.644 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T08:41:55.644 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T08:41:55.644 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T08:41:55.644 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T08:41:55.644 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T08:41:55.644 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T08:41:55.644 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T08:41:55.644 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T08:41:55.644 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T08:41:55.644 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T08:41:55.644 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T08:41:55.644 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T08:41:55.645 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T08:41:55.645 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T08:41:55.645 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T08:41:55.645 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T08:41:55.645 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T08:41:55.645 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T08:41:55.645 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T08:41:55.645 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T08:41:55.645 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T08:41:55.645 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T08:41:55.645 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T08:41:55.645 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T08:41:55.645 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T08:41:55.645 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T08:41:55.645 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T08:41:55.646 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T08:41:55.646 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T08:41:55.646 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T08:41:55.646 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T08:41:55.646 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T08:41:55.646 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T08:41:55.646 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T08:41:55.646 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T08:41:55.646 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T08:41:55.646 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T08:41:55.646 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T08:41:55.647 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T08:41:55.647 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T08:41:55.647 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T08:41:55.647 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T08:41:55.647 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T08:41:55.647 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T08:41:55.647 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T08:41:55.647 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T08:41:55.647 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T08:41:55.647 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T08:41:55.647 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T08:41:55.647 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T08:41:55.647 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T08:41:55.647 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T08:41:55.647 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T08:41:55.648 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T08:41:55.648 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T08:41:55.648 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T08:41:55.648 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T08:41:55.648 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T08:41:55.648 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T08:41:55.648 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T08:41:55.648 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T08:41:55.648 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T08:41:55.648 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T08:41:55.648 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T08:41:55.648 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T08:41:55.648 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T08:41:55.648 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T08:41:55.648 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T08:41:55.649 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T08:41:55.649 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T08:41:55.649 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T08:41:55.649 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T08:41:55.649 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T08:41:55.649 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T08:41:55.649 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T08:41:55.649 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T08:41:55.649 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T08:41:55.649 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T08:41:55.649 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T08:41:55.649 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T08:41:55.649 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T08:41:55.649 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T08:41:55.650 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T08:41:55.650 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T08:41:55.650 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T08:41:55.650 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T08:41:55.650 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T08:41:55.650 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T08:41:55.650 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T08:41:55.650 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T08:41:55.650 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T08:41:55.650 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T08:41:55.650 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T08:41:55.650 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T08:41:55.650 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T08:41:55.650 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T08:41:55.651 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T08:41:55.651 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T08:41:55.651 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T08:41:55.651 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T08:41:55.651 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T08:41:55.651 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T08:41:55.651 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T08:41:55.651 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T08:41:55.651 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T08:41:55.651 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T08:41:55.651 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T08:41:55.651 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T08:41:55.651 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T08:41:55.651 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T08:41:55.651 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T08:41:55.652 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T08:41:55.652 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T08:41:55.652 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T08:41:55.652 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T08:41:55.652 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T08:41:55.652 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T08:41:55.652 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T08:41:55.652 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T08:41:55.652 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T08:41:55.652 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T08:41:55.652 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T08:41:55.652 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T08:41:55.652 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T08:41:55.652 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T08:41:55.652 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T08:41:55.653 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T08:41:55.653 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T08:41:55.653 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T08:41:55.653 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T08:41:55.653 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T08:41:55.653 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T08:41:55.653 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T08:41:55.653 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T08:41:55.653 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T08:41:55.653 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T08:41:55.653 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T08:41:55.653 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T08:41:55.653 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T08:41:55.653 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T08:41:55.653 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T08:41:55.654 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T08:41:55.654 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T08:41:55.654 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T08:41:55.654 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T08:41:55.654 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T08:41:55.654 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T08:41:55.654 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T08:41:55.654 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T08:41:55.654 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T08:41:55.654 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T08:41:55.654 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T08:41:55.654 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T08:41:55.654 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T08:41:55.655 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T08:41:55.655 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T08:41:55.655 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T08:41:55.655 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T08:41:55.655 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T08:41:55.655 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T08:41:55.655 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T08:41:55.655 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T08:41:55.655 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T08:41:55.655 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T08:41:55.655 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T08:41:55.655 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T08:41:55.655 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T08:41:55.655 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T08:41:55.655 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T08:41:55.655 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T08:41:55.656 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T08:41:55.656 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T08:41:55.656 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T08:41:55.656 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T08:41:55.656 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T08:41:55.656 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T08:41:55.656 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T08:41:55.656 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T08:41:55.656 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T08:41:55.656 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T08:41:55.656 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T08:41:55.656 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T08:41:55.656 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T08:41:55.656 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T08:41:55.656 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T08:41:55.656 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T08:41:55.657 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T08:41:55.657 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T08:41:55.657 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T08:41:55.657 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T08:41:55.657 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T08:41:55.657 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T08:41:55.657 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T08:41:55.657 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T08:41:55.657 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T08:41:55.657 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T08:41:55.657 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T08:41:55.657 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T08:41:55.657 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T08:41:55.658 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T08:41:55.658 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T08:41:55.658 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T08:41:55.658 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T08:41:55.658 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T08:41:55.658 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T08:41:55.658 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T08:41:55.658 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T08:41:55.658 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T08:41:55.658 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T08:41:55.658 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T08:41:55.658 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T08:41:55.658 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T08:41:55.658 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T08:41:55.658 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T08:41:55.659 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T08:41:55.659 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T08:41:55.659 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T08:41:55.659 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T08:41:55.659 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T08:41:55.659 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T08:41:55.659 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T08:41:55.659 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T08:41:55.659 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T08:41:55.659 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T08:41:55.659 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T08:41:55.659 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T08:41:55.659 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T08:41:55.660 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T08:41:55.660 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T08:41:55.660 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T08:41:55.660 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T08:41:55.660 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T08:41:55.660 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T08:41:55.660 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T08:41:55.660 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T08:41:55.660 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T08:41:55.660 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T08:41:55.660 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T08:41:55.661 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T08:41:55.661 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T08:41:55.661 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T08:41:55.661 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T08:41:55.661 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T08:41:55.661 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T08:41:55.662 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T08:41:55.662 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T08:41:55.662 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T08:41:55.662 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T08:41:55.662 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T08:41:55.662 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T08:41:55.663 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T08:41:55.663 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T08:41:55.663 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T08:41:55.663 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T08:41:55.663 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T08:41:55.663 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T08:41:55.663 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T08:41:55.663 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T08:41:55.663 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T08:41:55.664 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T08:41:55.664 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T08:41:55.664 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T08:41:55.664 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T08:41:55.664 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T08:41:55.664 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T08:41:55.664 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T08:41:55.664 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T08:41:55.664 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T08:41:55.664 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T08:41:55.664 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T08:41:55.664 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T08:41:55.664 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T08:41:55.664 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T08:41:55.665 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T08:41:55.665 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T08:41:55.665 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T08:41:55.665 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T08:41:55.665 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T08:41:55.665 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T08:41:55.665 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T08:41:55.665 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T08:41:55.665 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T08:41:55.665 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T08:41:55.665 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T08:41:55.665 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T08:41:55.666 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T08:41:55.666 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T08:41:55.666 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T08:41:55.666 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T08:41:55.666 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T08:41:55.666 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T08:41:55.666 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T08:41:55.667 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T08:41:55.667 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T08:41:55.667 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T08:41:55.667 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T08:41:55.667 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T08:41:55.667 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T08:41:55.667 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T08:41:55.667 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T08:41:55.667 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T08:41:55.667 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T08:41:55.667 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T08:41:55.667 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T08:41:55.667 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T08:41:55.667 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T08:41:55.667 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T08:41:55.668 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T08:41:55.668 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T08:41:55.668 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T08:41:55.668 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T08:41:55.668 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T08:41:55.668 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T08:41:55.668 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T08:41:55.668 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T08:41:55.668 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T08:41:55.668 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T08:41:55.668 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T08:41:55.668 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T08:41:55.668 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T08:41:55.668 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T08:41:55.669 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T08:41:55.669 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T08:41:55.669 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T08:41:55.669 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T08:41:55.669 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T08:41:55.669 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T08:41:55.669 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T08:41:55.669 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T08:41:55.669 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T08:41:55.669 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T08:41:55.669 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T08:41:55.669 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T08:41:55.669 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T08:41:55.669 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T08:41:55.670 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T08:41:55.670 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T08:41:55.670 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T08:41:55.670 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T08:41:55.670 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T08:41:55.670 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T08:41:55.670 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T08:41:55.670 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T08:41:55.670 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T08:41:55.670 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T08:41:55.670 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T08:41:55.670 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T08:41:55.670 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T08:41:55.671 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T08:41:55.671 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T08:41:55.671 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T08:41:55.671 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T08:41:55.671 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T08:41:55.671 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T08:41:55.671 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T08:41:55.671 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T08:41:55.671 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T08:41:55.671 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T08:41:55.671 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T08:41:55.671 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T08:41:55.671 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T08:41:55.671 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T08:41:55.672 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T08:41:55.672 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T08:41:55.672 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T08:41:55.672 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T08:41:55.672 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T08:41:55.672 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T08:41:55.672 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T08:41:55.672 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T08:41:55.672 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T08:41:55.672 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T08:41:55.672 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T08:41:55.672 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T08:41:55.672 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T08:41:55.672 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T08:41:55.673 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T08:41:55.673 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T08:41:55.673 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T08:41:55.673 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T08:41:55.673 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T08:41:55.673 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T08:41:55.673 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T08:41:55.673 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T08:41:55.673 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T08:41:55.673 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T08:41:55.673 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T08:41:55.673 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T08:41:55.673 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T08:41:55.673 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T08:41:55.674 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T08:41:55.674 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T08:41:55.674 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T08:41:55.674 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T08:41:55.674 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T08:41:55.674 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T08:41:55.674 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T08:41:55.674 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T08:41:55.674 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T08:41:55.674 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T08:41:55.674 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T08:41:55.674 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T08:41:55.675 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T08:41:55.675 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T08:41:55.675 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T08:41:55.675 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T08:41:55.675 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T08:41:55.675 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T08:41:55.675 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T08:41:55.675 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T08:41:55.675 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T08:41:55.675 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T08:41:55.675 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T08:41:55.675 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T08:41:55.675 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T08:41:55.675 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T08:41:55.675 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T08:41:55.675 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T08:41:55.676 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T08:41:55.676 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T08:41:55.676 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T08:41:55.676 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T08:41:55.676 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T08:41:55.676 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T08:41:55.676 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T08:41:55.676 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T08:41:55.676 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T08:41:55.676 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T08:41:55.676 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T08:41:55.676 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T08:41:55.676 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T08:41:55.676 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T08:41:55.676 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T08:41:55.677 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T08:41:55.677 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T08:41:55.677 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T08:41:55.677 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T08:41:55.677 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T08:41:55.677 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T08:41:55.677 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T08:41:55.677 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T08:41:55.677 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T08:41:55.677 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T08:41:55.677 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T08:41:55.677 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T08:41:55.677 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T08:41:55.677 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T08:41:55.677 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T08:41:55.678 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T08:41:55.678 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T08:41:55.678 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T08:41:55.678 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T08:41:55.678 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T08:41:55.681 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:55.682 DEBUG:teuthology.orchestra.run.smithi069:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T08:41:55.704 INFO:teuthology.orchestra.run.smithi069.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T08:41:55.704 INFO:teuthology.orchestra.run.smithi069.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T08:41:55.704 INFO:teuthology.orchestra.run.smithi069.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T08:41:55.705 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2024-04-24T08:41:55.705 INFO:teuthology.orchestra.run.smithi069.stdout:for file in vmlinuz System.map config; do 2024-04-24T08:41:55.705 INFO:teuthology.orchestra.run.smithi069.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T08:41:55.705 INFO:teuthology.orchestra.run.smithi069.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:41:55.705 INFO:teuthology.orchestra.run.smithi069.stdout: fi 2024-04-24T08:41:55.705 INFO:teuthology.orchestra.run.smithi069.stdout:done 2024-04-24T08:41:55.706 INFO:teuthology.orchestra.run.smithi069.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T08:41:55.706 INFO:teuthology.orchestra.run.smithi069.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T08:41:55.706 INFO:teuthology.orchestra.run.smithi069.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:41:55.706 INFO:teuthology.orchestra.run.smithi069.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:41:55.706 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2024-04-24T08:41:55.706 INFO:teuthology.orchestra.run.smithi069.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T08:41:55.706 INFO:teuthology.orchestra.run.smithi069.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:55.706 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2024-04-24T08:41:55.706 INFO:teuthology.orchestra.run.smithi069.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T08:41:55.706 INFO:teuthology.orchestra.run.smithi069.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T08:41:55.706 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:55.707 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2024-04-24T08:41:55.708 DEBUG:teuthology.orchestra.run.smithi069:> sudo rpm -qi grub2-tools 2024-04-24T08:41:55.780 INFO:teuthology.orchestra.run.smithi069.stdout:Name : grub2-tools 2024-04-24T08:41:55.780 INFO:teuthology.orchestra.run.smithi069.stdout:Epoch : 1 2024-04-24T08:41:55.781 INFO:teuthology.orchestra.run.smithi069.stdout:Version : 2.06 2024-04-24T08:41:55.781 INFO:teuthology.orchestra.run.smithi069.stdout:Release : 68.el9 2024-04-24T08:41:55.781 INFO:teuthology.orchestra.run.smithi069.stdout:Architecture: x86_64 2024-04-24T08:41:55.781 INFO:teuthology.orchestra.run.smithi069.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T08:41:55.781 INFO:teuthology.orchestra.run.smithi069.stdout:Group : Unspecified 2024-04-24T08:41:55.781 INFO:teuthology.orchestra.run.smithi069.stdout:Size : 8264254 2024-04-24T08:41:55.781 INFO:teuthology.orchestra.run.smithi069.stdout:License : GPLv3+ 2024-04-24T08:41:55.781 INFO:teuthology.orchestra.run.smithi069.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T08:41:55.781 INFO:teuthology.orchestra.run.smithi069.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T08:41:55.781 INFO:teuthology.orchestra.run.smithi069.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T08:41:55.781 INFO:teuthology.orchestra.run.smithi069.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T08:41:55.781 INFO:teuthology.orchestra.run.smithi069.stdout:Packager : builder@centos.org 2024-04-24T08:41:55.781 INFO:teuthology.orchestra.run.smithi069.stdout:Vendor : CentOS 2024-04-24T08:41:55.781 INFO:teuthology.orchestra.run.smithi069.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T08:41:55.781 INFO:teuthology.orchestra.run.smithi069.stdout:Summary : Support tools for GRUB. 2024-04-24T08:41:55.782 INFO:teuthology.orchestra.run.smithi069.stdout:Description : 2024-04-24T08:41:55.782 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:41:55.782 INFO:teuthology.orchestra.run.smithi069.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T08:41:55.782 INFO:teuthology.orchestra.run.smithi069.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T08:41:55.782 INFO:teuthology.orchestra.run.smithi069.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T08:41:55.782 INFO:teuthology.orchestra.run.smithi069.stdout:hardware devices. 2024-04-24T08:41:55.782 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:41:55.782 INFO:teuthology.orchestra.run.smithi069.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T08:41:55.783 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T08:41:55.783 INFO:teuthology.task.kernel:Updating grub on smithi069 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:55.783 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T08:41:56.399 INFO:teuthology.orchestra.run.smithi069.stderr:Generating grub configuration file ... 2024-04-24T08:41:56.412 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi081.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T08:41:56.412 DEBUG:teuthology.orchestra.run.smithi081:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T08:41:56.503 INFO:teuthology.orchestra.run.smithi081.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:56.503 INFO:teuthology.orchestra.run.smithi081.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:56.503 INFO:teuthology.orchestra.run.smithi081.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:56.503 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T08:41:56.503 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T08:41:56.503 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T08:41:56.503 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T08:41:56.503 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T08:41:56.503 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T08:41:56.503 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T08:41:56.503 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T08:41:56.503 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T08:41:56.504 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T08:41:56.504 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T08:41:56.504 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T08:41:56.504 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T08:41:56.504 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T08:41:56.504 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T08:41:56.504 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T08:41:56.504 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T08:41:56.504 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T08:41:56.504 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T08:41:56.504 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T08:41:56.504 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T08:41:56.504 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T08:41:56.504 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T08:41:56.504 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T08:41:56.505 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T08:41:56.505 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T08:41:56.505 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T08:41:56.505 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T08:41:56.505 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T08:41:56.505 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T08:41:56.505 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T08:41:56.505 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T08:41:56.505 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T08:41:56.505 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T08:41:56.505 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T08:41:56.505 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T08:41:56.505 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T08:41:56.506 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T08:41:56.506 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T08:41:56.506 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T08:41:56.506 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T08:41:56.506 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T08:41:56.506 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T08:41:56.506 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T08:41:56.506 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T08:41:56.506 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T08:41:56.506 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T08:41:56.506 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T08:41:56.506 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T08:41:56.506 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T08:41:56.506 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T08:41:56.507 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T08:41:56.507 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T08:41:56.507 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T08:41:56.507 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T08:41:56.507 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T08:41:56.507 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T08:41:56.507 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T08:41:56.507 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T08:41:56.507 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T08:41:56.507 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T08:41:56.507 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T08:41:56.508 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T08:41:56.508 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T08:41:56.508 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T08:41:56.508 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T08:41:56.508 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T08:41:56.508 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T08:41:56.508 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T08:41:56.508 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T08:41:56.508 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T08:41:56.508 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T08:41:56.508 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T08:41:56.508 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T08:41:56.508 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T08:41:56.508 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T08:41:56.508 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T08:41:56.509 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T08:41:56.509 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T08:41:56.509 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T08:41:56.509 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T08:41:56.509 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T08:41:56.509 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T08:41:56.509 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T08:41:56.509 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T08:41:56.509 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T08:41:56.509 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T08:41:56.509 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T08:41:56.509 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T08:41:56.509 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T08:41:56.509 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T08:41:56.510 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T08:41:56.510 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T08:41:56.510 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T08:41:56.510 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T08:41:56.510 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T08:41:56.510 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T08:41:56.510 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T08:41:56.510 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T08:41:56.510 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T08:41:56.510 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T08:41:56.510 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T08:41:56.510 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T08:41:56.510 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T08:41:56.510 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T08:41:56.511 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T08:41:56.511 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T08:41:56.511 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T08:41:56.511 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T08:41:56.511 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T08:41:56.511 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T08:41:56.511 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T08:41:56.511 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T08:41:56.511 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T08:41:56.511 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T08:41:56.511 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T08:41:56.511 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T08:41:56.512 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T08:41:56.512 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T08:41:56.512 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T08:41:56.512 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T08:41:56.512 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T08:41:56.512 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T08:41:56.512 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T08:41:56.512 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T08:41:56.512 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T08:41:56.512 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T08:41:56.512 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T08:41:56.512 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T08:41:56.512 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T08:41:56.513 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T08:41:56.513 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T08:41:56.513 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T08:41:56.513 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T08:41:56.513 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T08:41:56.513 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T08:41:56.513 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T08:41:56.513 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T08:41:56.513 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T08:41:56.513 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T08:41:56.513 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T08:41:56.513 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T08:41:56.514 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T08:41:56.514 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T08:41:56.514 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T08:41:56.514 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T08:41:56.514 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T08:41:56.514 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T08:41:56.514 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T08:41:56.514 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T08:41:56.514 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T08:41:56.514 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T08:41:56.514 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T08:41:56.514 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T08:41:56.514 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T08:41:56.515 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T08:41:56.515 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T08:41:56.515 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T08:41:56.515 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T08:41:56.515 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T08:41:56.515 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T08:41:56.515 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T08:41:56.515 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T08:41:56.515 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T08:41:56.515 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T08:41:56.515 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T08:41:56.515 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T08:41:56.515 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T08:41:56.515 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T08:41:56.515 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T08:41:56.516 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T08:41:56.516 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T08:41:56.516 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T08:41:56.516 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T08:41:56.516 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T08:41:56.516 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T08:41:56.516 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T08:41:56.516 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T08:41:56.516 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T08:41:56.516 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T08:41:56.516 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T08:41:56.516 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T08:41:56.516 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T08:41:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T08:41:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T08:41:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T08:41:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T08:41:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T08:41:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T08:41:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T08:41:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T08:41:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T08:41:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T08:41:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T08:41:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T08:41:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T08:41:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T08:41:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T08:41:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T08:41:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T08:41:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T08:41:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T08:41:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T08:41:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T08:41:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T08:41:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T08:41:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T08:41:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T08:41:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T08:41:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T08:41:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T08:41:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T08:41:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T08:41:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T08:41:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T08:41:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T08:41:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T08:41:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T08:41:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T08:41:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T08:41:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T08:41:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T08:41:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T08:41:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T08:41:56.520 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T08:41:56.520 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T08:41:56.520 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T08:41:56.520 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T08:41:56.520 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T08:41:56.520 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T08:41:56.520 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T08:41:56.520 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T08:41:56.520 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T08:41:56.520 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T08:41:56.520 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T08:41:56.520 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T08:41:56.520 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T08:41:56.520 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T08:41:56.521 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T08:41:56.521 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T08:41:56.521 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T08:41:56.521 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T08:41:56.521 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T08:41:56.521 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T08:41:56.521 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T08:41:56.521 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T08:41:56.521 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T08:41:56.521 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T08:41:56.521 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T08:41:56.521 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T08:41:56.521 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T08:41:56.521 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T08:41:56.521 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T08:41:56.522 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T08:41:56.522 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T08:41:56.522 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T08:41:56.522 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T08:41:56.522 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T08:41:56.522 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T08:41:56.522 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T08:41:56.522 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T08:41:56.522 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T08:41:56.522 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T08:41:56.522 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T08:41:56.522 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T08:41:56.523 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T08:41:56.523 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T08:41:56.523 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T08:41:56.523 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T08:41:56.523 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T08:41:56.523 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T08:41:56.523 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T08:41:56.523 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T08:41:56.523 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T08:41:56.523 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T08:41:56.523 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T08:41:56.523 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T08:41:56.523 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T08:41:56.523 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T08:41:56.524 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T08:41:56.524 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T08:41:56.524 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T08:41:56.524 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T08:41:56.524 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T08:41:56.524 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T08:41:56.524 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T08:41:56.524 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T08:41:56.524 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T08:41:56.524 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T08:41:56.524 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T08:41:56.524 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T08:41:56.524 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T08:41:56.524 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T08:41:56.525 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T08:41:56.525 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T08:41:56.525 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T08:41:56.525 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T08:41:56.525 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T08:41:56.525 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T08:41:56.525 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T08:41:56.525 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T08:41:56.525 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T08:41:56.525 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T08:41:56.525 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T08:41:56.525 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T08:41:56.525 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T08:41:56.525 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T08:41:56.526 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T08:41:56.526 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T08:41:56.526 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T08:41:56.526 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T08:41:56.526 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T08:41:56.526 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T08:41:56.526 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T08:41:56.526 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T08:41:56.526 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T08:41:56.526 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T08:41:56.526 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T08:41:56.526 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T08:41:56.526 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T08:41:56.526 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T08:41:56.527 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T08:41:56.527 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T08:41:56.527 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T08:41:56.527 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T08:41:56.527 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T08:41:56.527 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T08:41:56.527 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T08:41:56.527 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T08:41:56.527 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T08:41:56.527 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T08:41:56.527 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T08:41:56.527 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T08:41:56.527 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T08:41:56.527 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T08:41:56.527 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T08:41:56.528 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T08:41:56.528 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T08:41:56.528 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T08:41:56.528 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T08:41:56.528 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T08:41:56.528 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T08:41:56.528 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T08:41:56.528 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T08:41:56.528 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T08:41:56.528 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T08:41:56.528 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T08:41:56.528 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T08:41:56.528 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T08:41:56.528 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T08:41:56.528 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T08:41:56.529 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T08:41:56.529 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T08:41:56.529 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T08:41:56.529 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T08:41:56.529 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T08:41:56.529 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T08:41:56.529 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T08:41:56.529 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T08:41:56.529 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T08:41:56.529 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T08:41:56.529 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T08:41:56.529 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T08:41:56.529 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T08:41:56.529 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T08:41:56.530 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T08:41:56.530 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T08:41:56.530 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T08:41:56.530 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T08:41:56.530 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T08:41:56.530 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T08:41:56.530 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T08:41:56.530 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T08:41:56.530 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T08:41:56.530 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T08:41:56.530 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T08:41:56.530 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T08:41:56.530 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T08:41:56.531 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T08:41:56.531 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T08:41:56.531 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T08:41:56.531 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T08:41:56.531 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T08:41:56.531 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T08:41:56.531 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T08:41:56.531 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T08:41:56.531 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T08:41:56.531 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T08:41:56.531 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T08:41:56.531 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T08:41:56.531 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T08:41:56.531 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T08:41:56.531 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T08:41:56.532 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T08:41:56.532 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T08:41:56.532 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T08:41:56.532 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T08:41:56.532 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T08:41:56.532 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T08:41:56.532 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T08:41:56.532 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T08:41:56.532 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T08:41:56.532 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T08:41:56.532 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T08:41:56.532 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T08:41:56.532 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T08:41:56.532 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T08:41:56.532 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T08:41:56.533 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T08:41:56.533 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T08:41:56.533 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T08:41:56.533 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T08:41:56.533 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T08:41:56.533 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T08:41:56.533 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T08:41:56.533 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T08:41:56.533 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T08:41:56.533 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T08:41:56.533 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T08:41:56.533 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T08:41:56.533 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T08:41:56.533 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T08:41:56.534 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T08:41:56.534 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T08:41:56.534 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T08:41:56.534 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T08:41:56.534 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T08:41:56.534 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T08:41:56.534 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T08:41:56.534 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T08:41:56.534 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T08:41:56.534 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T08:41:56.534 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T08:41:56.534 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T08:41:56.534 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T08:41:56.535 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T08:41:56.535 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T08:41:56.535 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T08:41:56.535 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T08:41:56.535 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T08:41:56.535 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T08:41:56.535 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T08:41:56.535 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T08:41:56.535 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T08:41:56.535 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T08:41:56.535 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T08:41:56.535 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T08:41:56.535 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T08:41:56.535 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T08:41:56.535 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T08:41:56.536 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T08:41:56.536 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T08:41:56.536 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T08:41:56.536 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T08:41:56.536 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T08:41:56.536 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T08:41:56.536 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T08:41:56.536 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T08:41:56.536 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T08:41:56.536 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T08:41:56.536 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T08:41:56.536 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T08:41:56.536 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T08:41:56.536 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T08:41:56.536 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T08:41:56.537 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T08:41:56.537 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T08:41:56.537 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T08:41:56.537 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T08:41:56.537 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T08:41:56.537 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T08:41:56.537 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T08:41:56.537 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T08:41:56.537 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T08:41:56.537 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T08:41:56.537 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T08:41:56.537 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T08:41:56.538 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T08:41:56.538 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T08:41:56.538 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T08:41:56.538 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T08:41:56.538 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T08:41:56.538 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T08:41:56.538 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T08:41:56.538 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T08:41:56.538 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T08:41:56.538 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T08:41:56.538 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T08:41:56.538 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T08:41:56.538 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T08:41:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T08:41:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T08:41:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T08:41:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T08:41:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T08:41:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T08:41:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T08:41:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T08:41:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T08:41:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T08:41:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T08:41:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T08:41:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T08:41:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T08:41:56.539 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T08:41:56.540 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T08:41:56.540 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T08:41:56.540 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T08:41:56.540 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T08:41:56.540 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T08:41:56.540 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T08:41:56.540 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T08:41:56.540 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T08:41:56.540 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T08:41:56.540 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T08:41:56.540 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T08:41:56.540 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T08:41:56.540 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T08:41:56.540 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T08:41:56.540 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T08:41:56.541 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T08:41:56.541 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T08:41:56.541 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T08:41:56.541 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T08:41:56.541 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T08:41:56.541 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T08:41:56.541 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T08:41:56.541 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T08:41:56.541 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T08:41:56.541 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T08:41:56.541 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T08:41:56.541 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T08:41:56.541 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T08:41:56.541 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T08:41:56.542 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T08:41:56.542 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T08:41:56.542 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T08:41:56.542 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T08:41:56.542 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T08:41:56.542 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T08:41:56.542 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T08:41:56.542 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T08:41:56.542 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T08:41:56.542 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T08:41:56.542 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T08:41:56.542 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T08:41:56.543 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T08:41:56.543 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T08:41:56.543 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T08:41:56.543 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T08:41:56.543 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T08:41:56.543 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T08:41:56.543 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T08:41:56.543 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T08:41:56.543 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T08:41:56.543 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T08:41:56.543 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T08:41:56.543 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T08:41:56.543 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T08:41:56.543 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T08:41:56.544 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T08:41:56.544 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T08:41:56.544 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T08:41:56.544 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T08:41:56.544 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T08:41:56.544 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T08:41:56.544 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T08:41:56.544 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T08:41:56.544 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T08:41:56.544 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T08:41:56.544 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T08:41:56.544 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T08:41:56.544 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T08:41:56.544 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T08:41:56.544 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T08:41:56.545 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T08:41:56.545 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T08:41:56.545 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T08:41:56.545 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T08:41:56.545 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T08:41:56.545 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T08:41:56.545 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T08:41:56.545 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T08:41:56.545 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T08:41:56.545 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T08:41:56.545 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T08:41:56.545 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T08:41:56.545 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T08:41:56.546 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T08:41:56.546 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T08:41:56.546 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T08:41:56.546 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T08:41:56.546 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T08:41:56.546 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T08:41:56.546 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T08:41:56.546 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T08:41:56.546 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T08:41:56.546 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T08:41:56.546 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T08:41:56.546 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T08:41:56.546 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T08:41:56.546 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T08:41:56.546 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T08:41:56.547 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T08:41:56.547 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T08:41:56.547 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T08:41:56.547 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T08:41:56.547 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T08:41:56.547 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T08:41:56.547 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T08:41:56.547 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T08:41:56.547 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T08:41:56.547 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T08:41:56.547 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T08:41:56.547 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T08:41:56.547 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T08:41:56.547 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T08:41:56.547 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T08:41:56.547 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T08:41:56.548 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T08:41:56.548 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T08:41:56.548 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T08:41:56.548 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T08:41:56.548 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T08:41:56.548 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T08:41:56.548 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T08:41:56.548 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T08:41:56.548 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T08:41:56.548 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T08:41:56.548 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T08:41:56.548 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T08:41:56.548 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T08:41:56.548 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T08:41:56.548 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T08:41:56.548 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T08:41:56.549 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T08:41:56.549 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T08:41:56.549 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T08:41:56.549 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T08:41:56.549 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T08:41:56.549 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T08:41:56.549 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T08:41:56.549 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T08:41:56.549 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T08:41:56.549 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T08:41:56.549 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T08:41:56.549 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T08:41:56.549 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T08:41:56.549 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T08:41:56.549 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T08:41:56.550 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T08:41:56.550 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T08:41:56.550 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T08:41:56.550 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T08:41:56.550 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T08:41:56.550 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T08:41:56.550 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T08:41:56.550 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T08:41:56.550 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T08:41:56.550 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T08:41:56.550 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T08:41:56.550 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T08:41:56.550 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T08:41:56.550 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T08:41:56.550 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T08:41:56.551 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T08:41:56.551 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T08:41:56.551 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T08:41:56.551 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T08:41:56.551 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T08:41:56.551 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T08:41:56.551 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T08:41:56.551 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T08:41:56.551 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T08:41:56.551 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T08:41:56.551 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T08:41:56.551 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T08:41:56.551 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T08:41:56.551 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T08:41:56.551 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T08:41:56.552 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T08:41:56.552 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T08:41:56.552 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T08:41:56.552 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T08:41:56.552 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T08:41:56.552 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T08:41:56.552 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T08:41:56.552 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T08:41:56.552 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T08:41:56.552 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T08:41:56.552 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T08:41:56.552 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T08:41:56.552 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T08:41:56.552 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T08:41:56.552 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T08:41:56.553 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T08:41:56.553 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T08:41:56.553 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T08:41:56.553 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T08:41:56.553 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T08:41:56.553 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T08:41:56.553 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T08:41:56.553 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T08:41:56.553 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T08:41:56.553 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T08:41:56.553 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T08:41:56.553 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T08:41:56.553 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T08:41:56.553 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T08:41:56.553 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T08:41:56.553 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T08:41:56.554 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T08:41:56.554 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T08:41:56.554 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T08:41:56.554 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T08:41:56.554 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T08:41:56.554 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T08:41:56.554 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T08:41:56.554 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T08:41:56.554 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T08:41:56.554 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T08:41:56.554 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T08:41:56.554 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T08:41:56.554 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T08:41:56.554 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T08:41:56.554 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T08:41:56.555 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T08:41:56.555 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T08:41:56.555 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T08:41:56.555 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T08:41:56.555 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T08:41:56.555 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T08:41:56.555 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T08:41:56.555 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T08:41:56.555 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T08:41:56.555 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T08:41:56.555 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T08:41:56.555 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T08:41:56.555 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T08:41:56.555 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T08:41:56.555 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T08:41:56.556 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T08:41:56.556 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T08:41:56.556 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T08:41:56.556 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T08:41:56.556 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T08:41:56.556 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T08:41:56.556 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T08:41:56.556 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T08:41:56.556 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T08:41:56.556 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T08:41:56.556 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T08:41:56.556 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T08:41:56.556 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T08:41:56.556 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T08:41:56.556 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T08:41:56.557 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T08:41:56.557 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T08:41:56.557 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T08:41:56.557 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T08:41:56.557 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T08:41:56.557 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T08:41:56.557 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T08:41:56.557 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T08:41:56.557 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T08:41:56.557 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T08:41:56.557 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T08:41:56.557 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T08:41:56.557 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T08:41:56.557 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T08:41:56.557 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T08:41:56.558 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T08:41:56.558 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T08:41:56.558 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T08:41:56.558 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T08:41:56.558 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T08:41:56.558 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T08:41:56.558 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T08:41:56.558 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T08:41:56.558 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T08:41:56.558 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T08:41:56.558 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T08:41:56.558 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T08:41:56.558 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T08:41:56.558 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T08:41:56.558 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T08:41:56.559 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T08:41:56.559 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T08:41:56.559 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T08:41:56.559 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T08:41:56.559 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T08:41:56.559 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T08:41:56.559 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T08:41:56.559 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T08:41:56.559 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T08:41:56.559 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T08:41:56.559 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T08:41:56.559 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T08:41:56.559 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T08:41:56.559 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T08:41:56.559 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T08:41:56.559 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T08:41:56.560 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T08:41:56.560 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T08:41:56.560 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T08:41:56.560 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T08:41:56.560 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T08:41:56.560 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T08:41:56.560 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T08:41:56.560 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T08:41:56.560 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T08:41:56.560 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T08:41:56.560 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T08:41:56.560 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T08:41:56.560 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T08:41:56.560 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T08:41:56.560 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T08:41:56.561 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T08:41:56.561 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T08:41:56.561 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T08:41:56.561 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T08:41:56.561 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T08:41:56.561 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T08:41:56.561 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T08:41:56.561 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T08:41:56.561 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T08:41:56.561 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T08:41:56.561 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T08:41:56.561 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T08:41:56.561 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T08:41:56.561 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T08:41:56.561 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T08:41:56.562 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T08:41:56.562 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T08:41:56.562 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T08:41:56.562 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T08:41:56.562 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T08:41:56.562 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T08:41:56.562 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T08:41:56.562 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T08:41:56.562 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T08:41:56.562 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T08:41:56.562 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T08:41:56.562 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T08:41:56.562 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T08:41:56.563 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T08:41:56.563 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T08:41:56.563 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T08:41:56.563 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T08:41:56.563 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T08:41:56.563 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T08:41:56.563 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T08:41:56.563 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T08:41:56.563 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T08:41:56.563 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T08:41:56.563 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T08:41:56.563 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T08:41:56.563 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T08:41:56.563 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T08:41:56.563 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T08:41:56.564 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T08:41:56.564 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T08:41:56.564 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T08:41:56.564 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T08:41:56.564 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T08:41:56.564 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T08:41:56.564 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T08:41:56.564 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T08:41:56.564 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T08:41:56.564 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T08:41:56.564 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T08:41:56.564 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T08:41:56.564 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T08:41:56.564 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T08:41:56.564 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T08:41:56.565 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T08:41:56.565 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T08:41:56.565 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T08:41:56.565 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T08:41:56.565 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T08:41:56.565 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T08:41:56.565 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T08:41:56.565 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T08:41:56.565 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T08:41:56.565 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T08:41:56.565 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T08:41:56.565 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T08:41:56.565 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T08:41:56.565 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T08:41:56.566 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T08:41:56.566 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T08:41:56.566 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T08:41:56.566 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T08:41:56.566 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T08:41:56.566 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T08:41:56.566 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T08:41:56.566 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T08:41:56.566 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T08:41:56.566 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T08:41:56.566 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T08:41:56.566 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T08:41:56.566 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T08:41:56.566 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T08:41:56.567 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T08:41:56.567 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T08:41:56.567 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T08:41:56.567 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T08:41:56.567 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T08:41:56.567 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T08:41:56.567 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T08:41:56.567 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T08:41:56.567 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T08:41:56.567 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T08:41:56.567 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T08:41:56.567 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T08:41:56.567 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T08:41:56.567 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T08:41:56.568 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T08:41:56.568 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T08:41:56.568 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T08:41:56.568 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T08:41:56.568 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T08:41:56.568 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T08:41:56.568 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T08:41:56.568 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T08:41:56.568 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T08:41:56.568 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T08:41:56.568 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T08:41:56.568 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T08:41:56.568 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T08:41:56.569 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T08:41:56.569 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T08:41:56.569 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T08:41:56.569 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T08:41:56.569 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T08:41:56.569 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T08:41:56.569 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T08:41:56.569 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T08:41:56.569 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T08:41:56.569 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T08:41:56.569 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T08:41:56.569 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T08:41:56.569 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T08:41:56.569 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T08:41:56.570 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T08:41:56.570 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T08:41:56.570 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T08:41:56.570 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T08:41:56.570 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T08:41:56.570 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T08:41:56.570 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T08:41:56.570 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T08:41:56.570 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T08:41:56.570 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T08:41:56.570 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T08:41:56.570 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T08:41:56.570 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T08:41:56.570 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T08:41:56.571 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T08:41:56.571 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T08:41:56.571 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T08:41:56.571 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T08:41:56.571 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T08:41:56.571 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T08:41:56.571 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T08:41:56.571 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T08:41:56.571 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T08:41:56.571 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T08:41:56.571 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T08:41:56.571 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T08:41:56.571 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T08:41:56.571 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T08:41:56.571 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T08:41:56.572 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T08:41:56.572 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T08:41:56.572 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T08:41:56.572 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T08:41:56.572 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T08:41:56.572 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T08:41:56.572 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T08:41:56.572 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T08:41:56.572 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T08:41:56.572 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T08:41:56.572 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T08:41:56.572 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T08:41:56.572 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T08:41:56.572 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T08:41:56.573 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T08:41:56.573 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T08:41:56.573 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T08:41:56.573 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T08:41:56.573 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T08:41:56.573 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T08:41:56.573 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T08:41:56.573 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T08:41:56.573 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T08:41:56.573 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T08:41:56.573 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T08:41:56.573 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T08:41:56.573 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T08:41:56.573 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T08:41:56.573 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T08:41:56.574 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T08:41:56.574 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T08:41:56.574 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T08:41:56.574 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T08:41:56.574 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T08:41:56.574 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T08:41:56.574 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T08:41:56.574 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T08:41:56.574 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T08:41:56.574 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T08:41:56.574 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T08:41:56.574 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T08:41:56.574 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T08:41:56.575 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T08:41:56.575 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T08:41:56.575 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T08:41:56.575 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T08:41:56.575 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T08:41:56.575 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T08:41:56.575 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T08:41:56.575 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T08:41:56.575 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T08:41:56.575 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T08:41:56.575 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T08:41:56.575 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T08:41:56.575 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T08:41:56.575 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T08:41:56.576 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T08:41:56.576 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T08:41:56.576 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T08:41:56.576 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T08:41:56.576 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T08:41:56.576 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T08:41:56.576 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T08:41:56.576 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T08:41:56.576 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T08:41:56.576 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T08:41:56.576 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T08:41:56.576 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T08:41:56.576 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T08:41:56.576 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T08:41:56.576 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T08:41:56.577 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T08:41:56.577 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T08:41:56.577 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T08:41:56.577 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T08:41:56.577 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T08:41:56.577 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T08:41:56.577 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T08:41:56.577 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T08:41:56.577 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T08:41:56.577 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T08:41:56.577 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T08:41:56.577 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T08:41:56.577 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T08:41:56.577 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T08:41:56.578 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T08:41:56.578 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T08:41:56.578 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T08:41:56.578 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T08:41:56.578 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T08:41:56.578 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T08:41:56.578 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T08:41:56.578 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T08:41:56.578 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T08:41:56.578 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T08:41:56.579 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T08:41:56.579 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T08:41:56.579 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T08:41:56.579 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T08:41:56.579 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T08:41:56.580 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T08:41:56.580 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T08:41:56.580 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T08:41:56.580 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T08:41:56.580 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T08:41:56.580 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T08:41:56.580 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T08:41:56.581 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T08:41:56.581 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T08:41:56.581 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T08:41:56.581 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T08:41:56.581 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T08:41:56.581 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T08:41:56.582 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T08:41:56.582 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T08:41:56.582 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T08:41:56.582 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T08:41:56.582 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T08:41:56.582 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T08:41:56.583 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T08:41:56.583 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T08:41:56.583 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T08:41:56.583 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T08:41:56.583 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T08:41:56.583 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T08:41:56.584 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T08:41:56.584 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T08:41:56.584 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T08:41:56.584 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T08:41:56.584 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T08:41:56.584 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T08:41:56.584 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T08:41:56.584 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T08:41:56.584 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T08:41:56.585 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T08:41:56.585 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T08:41:56.585 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T08:41:56.585 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T08:41:56.585 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T08:41:56.585 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T08:41:56.585 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T08:41:56.585 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T08:41:56.585 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T08:41:56.585 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T08:41:56.585 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T08:41:56.585 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T08:41:56.585 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T08:41:56.585 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T08:41:56.585 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T08:41:56.586 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T08:41:56.586 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T08:41:56.586 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T08:41:56.586 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T08:41:56.586 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T08:41:56.586 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T08:41:56.586 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T08:41:56.586 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T08:41:56.586 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T08:41:56.586 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T08:41:56.587 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T08:41:56.588 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T08:41:56.588 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T08:41:56.588 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T08:41:56.588 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T08:41:56.588 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T08:41:56.588 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T08:41:56.588 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T08:41:56.588 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T08:41:56.588 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T08:41:56.588 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T08:41:56.588 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T08:41:56.588 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T08:41:56.588 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T08:41:56.588 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T08:41:56.589 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T08:41:56.589 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T08:41:56.589 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T08:41:56.589 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T08:41:56.589 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T08:41:56.589 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T08:41:56.589 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T08:41:56.589 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T08:41:56.589 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T08:41:56.589 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T08:41:56.589 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T08:41:56.589 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T08:41:56.589 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T08:41:56.589 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T08:41:56.589 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T08:41:56.589 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T08:41:56.590 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T08:41:56.590 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T08:41:56.590 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T08:41:56.590 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T08:41:56.590 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T08:41:56.590 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T08:41:56.590 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T08:41:56.590 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T08:41:56.590 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T08:41:56.590 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T08:41:56.590 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T08:41:56.590 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T08:41:56.590 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T08:41:56.590 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T08:41:56.591 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T08:41:56.591 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T08:41:56.591 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T08:41:56.591 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T08:41:56.591 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T08:41:56.591 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T08:41:56.591 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T08:41:56.591 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T08:41:56.591 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T08:41:56.591 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T08:41:56.591 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T08:41:56.591 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T08:41:56.591 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T08:41:56.591 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T08:41:56.591 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T08:41:56.592 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T08:41:56.592 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T08:41:56.592 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T08:41:56.592 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T08:41:56.592 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T08:41:56.592 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T08:41:56.592 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T08:41:56.592 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T08:41:56.592 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T08:41:56.592 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T08:41:56.592 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T08:41:56.592 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T08:41:56.592 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T08:41:56.592 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T08:41:56.592 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T08:41:56.593 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T08:41:56.593 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T08:41:56.593 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T08:41:56.593 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T08:41:56.593 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T08:41:56.593 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T08:41:56.593 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T08:41:56.593 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T08:41:56.593 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T08:41:56.593 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T08:41:56.593 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T08:41:56.593 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T08:41:56.593 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T08:41:56.593 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T08:41:56.593 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T08:41:56.594 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T08:41:56.594 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T08:41:56.594 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T08:41:56.594 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T08:41:56.594 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T08:41:56.594 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T08:41:56.594 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T08:41:56.594 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T08:41:56.594 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T08:41:56.594 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T08:41:56.594 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T08:41:56.594 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T08:41:56.594 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T08:41:56.595 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T08:41:56.595 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T08:41:56.595 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T08:41:56.595 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T08:41:56.595 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T08:41:56.595 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T08:41:56.595 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T08:41:56.595 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T08:41:56.595 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T08:41:56.595 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T08:41:56.595 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T08:41:56.595 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T08:41:56.595 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T08:41:56.595 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T08:41:56.596 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T08:41:56.596 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T08:41:56.596 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T08:41:56.596 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T08:41:56.596 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T08:41:56.596 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T08:41:56.596 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T08:41:56.596 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T08:41:56.596 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T08:41:56.597 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T08:41:56.597 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T08:41:56.597 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T08:41:56.597 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T08:41:56.597 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T08:41:56.597 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T08:41:56.597 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T08:41:56.597 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T08:41:56.597 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T08:41:56.597 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T08:41:56.597 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T08:41:56.597 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T08:41:56.597 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T08:41:56.597 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T08:41:56.597 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T08:41:56.597 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T08:41:56.598 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T08:41:56.598 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T08:41:56.598 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T08:41:56.598 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T08:41:56.598 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T08:41:56.598 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T08:41:56.598 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T08:41:56.598 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T08:41:56.598 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T08:41:56.598 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T08:41:56.598 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T08:41:56.598 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T08:41:56.598 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T08:41:56.598 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T08:41:56.599 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T08:41:56.599 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T08:41:56.599 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T08:41:56.599 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T08:41:56.599 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T08:41:56.599 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T08:41:56.599 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T08:41:56.599 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T08:41:56.599 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T08:41:56.599 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T08:41:56.599 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T08:41:56.599 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T08:41:56.600 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T08:41:56.600 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T08:41:56.600 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T08:41:56.600 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T08:41:56.600 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T08:41:56.600 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T08:41:56.601 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T08:41:56.601 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T08:41:56.601 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T08:41:56.601 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T08:41:56.601 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T08:41:56.601 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T08:41:56.602 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T08:41:56.602 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T08:41:56.602 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T08:41:56.602 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T08:41:56.602 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T08:41:56.602 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T08:41:56.602 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T08:41:56.602 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T08:41:56.603 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T08:41:56.603 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T08:41:56.603 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T08:41:56.603 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T08:41:56.603 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T08:41:56.603 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T08:41:56.603 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T08:41:56.603 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T08:41:56.603 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T08:41:56.603 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T08:41:56.603 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T08:41:56.603 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T08:41:56.603 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T08:41:56.603 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T08:41:56.603 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T08:41:56.603 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T08:41:56.604 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T08:41:56.604 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T08:41:56.604 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T08:41:56.604 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T08:41:56.604 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T08:41:56.604 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T08:41:56.604 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T08:41:56.604 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T08:41:56.605 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T08:41:56.605 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T08:41:56.605 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T08:41:56.605 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T08:41:56.605 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T08:41:56.605 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T08:41:56.605 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T08:41:56.605 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T08:41:56.605 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T08:41:56.605 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T08:41:56.605 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T08:41:56.606 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T08:41:56.606 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T08:41:56.606 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T08:41:56.606 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T08:41:56.606 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T08:41:56.606 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T08:41:56.606 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T08:41:56.606 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T08:41:56.606 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T08:41:56.606 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T08:41:56.606 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T08:41:56.606 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T08:41:56.607 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T08:41:56.607 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T08:41:56.607 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T08:41:56.607 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T08:41:56.607 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T08:41:56.607 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T08:41:56.607 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T08:41:56.607 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T08:41:56.607 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T08:41:56.607 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T08:41:56.607 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T08:41:56.607 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T08:41:56.607 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T08:41:56.607 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T08:41:56.608 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T08:41:56.608 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T08:41:56.608 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T08:41:56.608 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T08:41:56.608 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T08:41:56.608 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T08:41:56.608 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T08:41:56.608 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T08:41:56.608 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T08:41:56.608 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T08:41:56.608 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T08:41:56.608 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T08:41:56.608 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T08:41:56.608 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T08:41:56.609 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T08:41:56.609 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T08:41:56.609 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T08:41:56.609 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T08:41:56.609 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T08:41:56.609 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T08:41:56.609 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T08:41:56.609 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T08:41:56.609 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T08:41:56.609 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T08:41:56.609 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T08:41:56.610 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T08:41:56.610 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T08:41:56.610 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T08:41:56.610 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T08:41:56.610 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T08:41:56.610 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T08:41:56.610 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T08:41:56.610 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T08:41:56.610 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T08:41:56.610 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T08:41:56.610 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T08:41:56.610 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T08:41:56.610 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T08:41:56.610 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T08:41:56.611 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T08:41:56.611 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T08:41:56.611 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T08:41:56.611 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T08:41:56.611 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T08:41:56.611 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T08:41:56.611 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T08:41:56.611 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T08:41:56.611 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T08:41:56.611 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T08:41:56.611 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T08:41:56.611 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T08:41:56.611 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T08:41:56.611 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T08:41:56.612 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T08:41:56.612 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T08:41:56.612 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T08:41:56.612 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T08:41:56.612 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T08:41:56.612 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T08:41:56.612 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T08:41:56.612 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T08:41:56.612 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T08:41:56.612 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T08:41:56.612 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T08:41:56.612 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T08:41:56.612 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T08:41:56.612 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T08:41:56.613 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T08:41:56.613 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T08:41:56.613 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T08:41:56.613 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T08:41:56.613 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T08:41:56.613 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T08:41:56.613 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T08:41:56.613 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T08:41:56.613 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T08:41:56.613 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T08:41:56.613 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T08:41:56.613 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T08:41:56.613 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T08:41:56.613 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T08:41:56.614 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T08:41:56.614 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T08:41:56.614 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T08:41:56.614 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T08:41:56.614 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T08:41:56.614 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T08:41:56.614 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T08:41:56.614 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T08:41:56.614 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T08:41:56.614 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T08:41:56.614 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T08:41:56.614 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T08:41:56.614 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T08:41:56.614 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T08:41:56.615 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T08:41:56.615 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T08:41:56.615 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T08:41:56.615 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T08:41:56.615 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T08:41:56.615 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T08:41:56.615 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T08:41:56.615 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T08:41:56.615 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T08:41:56.615 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T08:41:56.615 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T08:41:56.615 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T08:41:56.615 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T08:41:56.615 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T08:41:56.615 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T08:41:56.616 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T08:41:56.616 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T08:41:56.616 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T08:41:56.616 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T08:41:56.616 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T08:41:56.616 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T08:41:56.616 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T08:41:56.616 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T08:41:56.616 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T08:41:56.616 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T08:41:56.616 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T08:41:56.616 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T08:41:56.616 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T08:41:56.616 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T08:41:56.617 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T08:41:56.617 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T08:41:56.617 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T08:41:56.617 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T08:41:56.617 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T08:41:56.617 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T08:41:56.617 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T08:41:56.617 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T08:41:56.617 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T08:41:56.617 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T08:41:56.617 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T08:41:56.617 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T08:41:56.617 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T08:41:56.617 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T08:41:56.617 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T08:41:56.618 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T08:41:56.618 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T08:41:56.618 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T08:41:56.618 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T08:41:56.618 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T08:41:56.618 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T08:41:56.618 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T08:41:56.618 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T08:41:56.618 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T08:41:56.618 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T08:41:56.618 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T08:41:56.618 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T08:41:56.618 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T08:41:56.618 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T08:41:56.619 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T08:41:56.619 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T08:41:56.619 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T08:41:56.619 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T08:41:56.619 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T08:41:56.619 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T08:41:56.619 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T08:41:56.619 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T08:41:56.621 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T08:41:56.621 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T08:41:56.621 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T08:41:56.621 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T08:41:56.621 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T08:41:56.621 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T08:41:56.621 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T08:41:56.622 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T08:41:56.622 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T08:41:56.622 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T08:41:56.622 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T08:41:56.622 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T08:41:56.622 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T08:41:56.622 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T08:41:56.622 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T08:41:56.622 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T08:41:56.622 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T08:41:56.623 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T08:41:56.623 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T08:41:56.623 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T08:41:56.623 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T08:41:56.623 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T08:41:56.623 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T08:41:56.623 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T08:41:56.623 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T08:41:56.623 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T08:41:56.623 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T08:41:56.623 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T08:41:56.623 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T08:41:56.624 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T08:41:56.624 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T08:41:56.624 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T08:41:56.624 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T08:41:56.624 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T08:41:56.624 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T08:41:56.624 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T08:41:56.624 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T08:41:56.624 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T08:41:56.624 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T08:41:56.624 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T08:41:56.624 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T08:41:56.624 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T08:41:56.625 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T08:41:56.625 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T08:41:56.625 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T08:41:56.625 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T08:41:56.625 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T08:41:56.625 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T08:41:56.625 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T08:41:56.625 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T08:41:56.625 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T08:41:56.625 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T08:41:56.625 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T08:41:56.625 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T08:41:56.625 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T08:41:56.625 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T08:41:56.625 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T08:41:56.626 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T08:41:56.626 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T08:41:56.626 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T08:41:56.626 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T08:41:56.626 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T08:41:56.626 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T08:41:56.626 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T08:41:56.626 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T08:41:56.626 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T08:41:56.626 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T08:41:56.626 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T08:41:56.626 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T08:41:56.626 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T08:41:56.626 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T08:41:56.627 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T08:41:56.627 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T08:41:56.627 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T08:41:56.627 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T08:41:56.627 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T08:41:56.627 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T08:41:56.627 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T08:41:56.627 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T08:41:56.627 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T08:41:56.627 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T08:41:56.627 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T08:41:56.627 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T08:41:56.627 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T08:41:56.627 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T08:41:56.627 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T08:41:56.627 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T08:41:56.628 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T08:41:56.628 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T08:41:56.628 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T08:41:56.628 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T08:41:56.628 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T08:41:56.628 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T08:41:56.628 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T08:41:56.628 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T08:41:56.628 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T08:41:56.628 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T08:41:56.628 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T08:41:56.628 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T08:41:56.628 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T08:41:56.628 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T08:41:56.628 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T08:41:56.629 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T08:41:56.629 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T08:41:56.629 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T08:41:56.629 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T08:41:56.629 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T08:41:56.629 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T08:41:56.629 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T08:41:56.629 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T08:41:56.629 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T08:41:56.629 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T08:41:56.629 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T08:41:56.629 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T08:41:56.629 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T08:41:56.629 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T08:41:56.630 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T08:41:56.630 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T08:41:56.630 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T08:41:56.630 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T08:41:56.630 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T08:41:56.630 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T08:41:56.630 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T08:41:56.630 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T08:41:56.630 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T08:41:56.630 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T08:41:56.630 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T08:41:56.630 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T08:41:56.630 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T08:41:56.630 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T08:41:56.631 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T08:41:56.631 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T08:41:56.631 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T08:41:56.631 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T08:41:56.631 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T08:41:56.631 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T08:41:56.631 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T08:41:56.631 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T08:41:56.631 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T08:41:56.631 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T08:41:56.631 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T08:41:56.631 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T08:41:56.631 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T08:41:56.631 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T08:41:56.631 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T08:41:56.632 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T08:41:56.632 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T08:41:56.632 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T08:41:56.632 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T08:41:56.632 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T08:41:56.632 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T08:41:56.632 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T08:41:56.632 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T08:41:56.632 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T08:41:56.632 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T08:41:56.632 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T08:41:56.632 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T08:41:56.632 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T08:41:56.632 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T08:41:56.633 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T08:41:56.633 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T08:41:56.633 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T08:41:56.633 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T08:41:56.633 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T08:41:56.633 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T08:41:56.633 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T08:41:56.633 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T08:41:56.633 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T08:41:56.633 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T08:41:56.633 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T08:41:56.633 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T08:41:56.633 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T08:41:56.634 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T08:41:56.634 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T08:41:56.634 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T08:41:56.634 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T08:41:56.634 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T08:41:56.634 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T08:41:56.634 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T08:41:56.634 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T08:41:56.634 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T08:41:56.634 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T08:41:56.634 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T08:41:56.634 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T08:41:56.635 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T08:41:56.635 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T08:41:56.635 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T08:41:56.635 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T08:41:56.635 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T08:41:56.635 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T08:41:56.635 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T08:41:56.636 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T08:41:56.636 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T08:41:56.636 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T08:41:56.636 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T08:41:56.636 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T08:41:56.636 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T08:41:56.637 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T08:41:56.637 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T08:41:56.637 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T08:41:56.637 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T08:41:56.637 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T08:41:56.637 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T08:41:56.637 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T08:41:56.637 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T08:41:56.637 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T08:41:56.637 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T08:41:56.638 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T08:41:56.638 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T08:41:56.638 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T08:41:56.638 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T08:41:56.638 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T08:41:56.638 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T08:41:56.638 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T08:41:56.638 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T08:41:56.638 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T08:41:56.638 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T08:41:56.638 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T08:41:56.638 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T08:41:56.638 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T08:41:56.639 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T08:41:56.639 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T08:41:56.639 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T08:41:56.639 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T08:41:56.639 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T08:41:56.639 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T08:41:56.639 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T08:41:56.639 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T08:41:56.639 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T08:41:56.639 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T08:41:56.639 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T08:41:56.639 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T08:41:56.639 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T08:41:56.639 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T08:41:56.640 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T08:41:56.640 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T08:41:56.640 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T08:41:56.640 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T08:41:56.640 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T08:41:56.640 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T08:41:56.640 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T08:41:56.640 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T08:41:56.640 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T08:41:56.640 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T08:41:56.640 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T08:41:56.640 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T08:41:56.640 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T08:41:56.640 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T08:41:56.641 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T08:41:56.641 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T08:41:56.641 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T08:41:56.641 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T08:41:56.641 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T08:41:56.641 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T08:41:56.641 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T08:41:56.642 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T08:41:56.642 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T08:41:56.642 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T08:41:56.642 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T08:41:56.642 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T08:41:56.642 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T08:41:56.642 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T08:41:56.642 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T08:41:56.642 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T08:41:56.642 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T08:41:56.643 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T08:41:56.643 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T08:41:56.643 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T08:41:56.643 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T08:41:56.643 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T08:41:56.643 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T08:41:56.643 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T08:41:56.643 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T08:41:56.643 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T08:41:56.643 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T08:41:56.643 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T08:41:56.643 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T08:41:56.643 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T08:41:56.643 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T08:41:56.644 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T08:41:56.644 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T08:41:56.644 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T08:41:56.644 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T08:41:56.644 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T08:41:56.644 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T08:41:56.644 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T08:41:56.644 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T08:41:56.644 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T08:41:56.644 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T08:41:56.644 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T08:41:56.644 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T08:41:56.644 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T08:41:56.644 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T08:41:56.645 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T08:41:56.645 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T08:41:56.645 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T08:41:56.645 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T08:41:56.645 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T08:41:56.645 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T08:41:56.645 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T08:41:56.645 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T08:41:56.645 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T08:41:56.645 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T08:41:56.645 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T08:41:56.645 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T08:41:56.645 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T08:41:56.645 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T08:41:56.646 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T08:41:56.646 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T08:41:56.646 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T08:41:56.646 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T08:41:56.646 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T08:41:56.646 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T08:41:56.646 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T08:41:56.646 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T08:41:56.646 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T08:41:56.646 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T08:41:56.646 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T08:41:56.646 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T08:41:56.646 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T08:41:56.646 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T08:41:56.647 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T08:41:56.647 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T08:41:56.647 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T08:41:56.647 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T08:41:56.647 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T08:41:56.647 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T08:41:56.647 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T08:41:56.647 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T08:41:56.647 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T08:41:56.647 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T08:41:56.647 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T08:41:56.648 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T08:41:56.648 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T08:41:56.648 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T08:41:56.648 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T08:41:56.648 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T08:41:56.648 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T08:41:56.648 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T08:41:56.648 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T08:41:56.648 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T08:41:56.648 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T08:41:56.648 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T08:41:56.648 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T08:41:56.648 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T08:41:56.648 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T08:41:56.648 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T08:41:56.648 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T08:41:56.649 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T08:41:56.649 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T08:41:56.649 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T08:41:56.649 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T08:41:56.649 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T08:41:56.649 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T08:41:56.649 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T08:41:56.649 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T08:41:56.649 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T08:41:56.649 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T08:41:56.649 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T08:41:56.649 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T08:41:56.649 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T08:41:56.649 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T08:41:56.649 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T08:41:56.650 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T08:41:56.650 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T08:41:56.650 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T08:41:56.650 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T08:41:56.650 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T08:41:56.650 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T08:41:56.650 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T08:41:56.650 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T08:41:56.650 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T08:41:56.650 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T08:41:56.650 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T08:41:56.650 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T08:41:56.650 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T08:41:56.650 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T08:41:56.651 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T08:41:56.651 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T08:41:56.651 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T08:41:56.651 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T08:41:56.651 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T08:41:56.651 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T08:41:56.651 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T08:41:56.651 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T08:41:56.651 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T08:41:56.655 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:56.655 DEBUG:teuthology.orchestra.run.smithi081:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T08:41:56.678 INFO:teuthology.orchestra.run.smithi081.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T08:41:56.679 INFO:teuthology.orchestra.run.smithi081.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T08:41:56.679 INFO:teuthology.orchestra.run.smithi081.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T08:41:56.679 INFO:teuthology.orchestra.run.smithi081.stdout:fi 2024-04-24T08:41:56.679 INFO:teuthology.orchestra.run.smithi081.stdout:for file in vmlinuz System.map config; do 2024-04-24T08:41:56.679 INFO:teuthology.orchestra.run.smithi081.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T08:41:56.679 INFO:teuthology.orchestra.run.smithi081.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:41:56.679 INFO:teuthology.orchestra.run.smithi081.stdout: fi 2024-04-24T08:41:56.679 INFO:teuthology.orchestra.run.smithi081.stdout:done 2024-04-24T08:41:56.680 INFO:teuthology.orchestra.run.smithi081.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T08:41:56.680 INFO:teuthology.orchestra.run.smithi081.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T08:41:56.680 INFO:teuthology.orchestra.run.smithi081.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:41:56.680 INFO:teuthology.orchestra.run.smithi081.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:41:56.680 INFO:teuthology.orchestra.run.smithi081.stdout:fi 2024-04-24T08:41:56.680 INFO:teuthology.orchestra.run.smithi081.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T08:41:56.680 INFO:teuthology.orchestra.run.smithi081.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:56.680 INFO:teuthology.orchestra.run.smithi081.stdout:fi 2024-04-24T08:41:56.680 INFO:teuthology.orchestra.run.smithi081.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T08:41:56.680 INFO:teuthology.orchestra.run.smithi081.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T08:41:56.680 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:56.680 INFO:teuthology.orchestra.run.smithi081.stdout:fi 2024-04-24T08:41:56.682 DEBUG:teuthology.orchestra.run.smithi081:> sudo rpm -qi grub2-tools 2024-04-24T08:41:56.761 INFO:teuthology.orchestra.run.smithi081.stdout:Name : grub2-tools 2024-04-24T08:41:56.761 INFO:teuthology.orchestra.run.smithi081.stdout:Epoch : 1 2024-04-24T08:41:56.761 INFO:teuthology.orchestra.run.smithi081.stdout:Version : 2.06 2024-04-24T08:41:56.761 INFO:teuthology.orchestra.run.smithi081.stdout:Release : 68.el9 2024-04-24T08:41:56.761 INFO:teuthology.orchestra.run.smithi081.stdout:Architecture: x86_64 2024-04-24T08:41:56.761 INFO:teuthology.orchestra.run.smithi081.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T08:41:56.762 INFO:teuthology.orchestra.run.smithi081.stdout:Group : Unspecified 2024-04-24T08:41:56.762 INFO:teuthology.orchestra.run.smithi081.stdout:Size : 8264254 2024-04-24T08:41:56.762 INFO:teuthology.orchestra.run.smithi081.stdout:License : GPLv3+ 2024-04-24T08:41:56.762 INFO:teuthology.orchestra.run.smithi081.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T08:41:56.762 INFO:teuthology.orchestra.run.smithi081.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T08:41:56.762 INFO:teuthology.orchestra.run.smithi081.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T08:41:56.762 INFO:teuthology.orchestra.run.smithi081.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T08:41:56.762 INFO:teuthology.orchestra.run.smithi081.stdout:Packager : builder@centos.org 2024-04-24T08:41:56.762 INFO:teuthology.orchestra.run.smithi081.stdout:Vendor : CentOS 2024-04-24T08:41:56.762 INFO:teuthology.orchestra.run.smithi081.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T08:41:56.762 INFO:teuthology.orchestra.run.smithi081.stdout:Summary : Support tools for GRUB. 2024-04-24T08:41:56.762 INFO:teuthology.orchestra.run.smithi081.stdout:Description : 2024-04-24T08:41:56.762 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:41:56.762 INFO:teuthology.orchestra.run.smithi081.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T08:41:56.763 INFO:teuthology.orchestra.run.smithi081.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T08:41:56.763 INFO:teuthology.orchestra.run.smithi081.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T08:41:56.763 INFO:teuthology.orchestra.run.smithi081.stdout:hardware devices. 2024-04-24T08:41:56.763 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:41:56.763 INFO:teuthology.orchestra.run.smithi081.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T08:41:56.764 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T08:41:56.764 INFO:teuthology.task.kernel:Updating grub on smithi081 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:56.764 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T08:41:57.326 INFO:teuthology.orchestra.run.smithi081.stderr:Generating grub configuration file ... 2024-04-24T08:41:57.796 INFO:teuthology.orchestra.run.smithi069.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T08:41:57.836 INFO:teuthology.orchestra.run.smithi069.stderr:done 2024-04-24T08:41:57.838 DEBUG:teuthology.orchestra.run.smithi069:> mktemp 2024-04-24T08:41:57.853 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.wZtbeBLYBj 2024-04-24T08:41:57.853 DEBUG:teuthology.orchestra.run.smithi069:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wZtbeBLYBj 2024-04-24T08:41:57.919 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.wZtbeBLYBj 2024-04-24T08:41:58.051 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.wZtbeBLYBj is 6KB 2024-04-24T08:41:58.098 DEBUG:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.wZtbeBLYBj 2024-04-24T08:41:58.113 DEBUG:teuthology.orchestra.run.smithi069:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T08:41:58.174 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T08:41:58.174 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T08:41:58.174 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T08:41:58.174 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T08:41:58.175 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:58.339 DEBUG:teuthology.orchestra.run.smithi069:> sudo shutdown -r now 2024-04-24T08:41:58.793 INFO:teuthology.orchestra.run.smithi081.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T08:41:58.812 INFO:teuthology.orchestra.run.smithi081.stderr:done 2024-04-24T08:41:58.819 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-04-24T08:41:58.835 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.UqvDESC1Ax 2024-04-24T08:41:58.835 DEBUG:teuthology.orchestra.run.smithi081:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UqvDESC1Ax 2024-04-24T08:41:58.902 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.UqvDESC1Ax 2024-04-24T08:41:59.038 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.UqvDESC1Ax is 6KB 2024-04-24T08:41:59.087 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.UqvDESC1Ax 2024-04-24T08:41:59.101 DEBUG:teuthology.orchestra.run.smithi081:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T08:41:59.162 INFO:teuthology.orchestra.run.smithi081.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T08:41:59.163 INFO:teuthology.orchestra.run.smithi081.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T08:41:59.163 INFO:teuthology.orchestra.run.smithi081.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T08:41:59.163 INFO:teuthology.orchestra.run.smithi081.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T08:41:59.164 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T08:41:59.330 DEBUG:teuthology.orchestra.run.smithi081:> sudo shutdown -r now 2024-04-24T08:42:15.754 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-04-24T08:42:15.755 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-04-24T08:42:15.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:42:28.355 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-24T08:42:28.355 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-04-24T08:42:28.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:42:29.333 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-24T08:42:29.334 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-04-24T08:42:29.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:43:15.797 DEBUG:teuthology.orchestra.remote:timed out 2024-04-24T08:43:24.806 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-04-24T08:43:24.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:43:28.362 DEBUG:teuthology.orchestra.remote:timed out 2024-04-24T08:43:29.338 DEBUG:teuthology.orchestra.remote:timed out 2024-04-24T08:43:32.499 DEBUG:teuthology.orchestra.run.smithi130:> true 2024-04-24T08:43:33.184 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-04-24T08:43:33.184 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T08:43:33.185 DEBUG:teuthology.orchestra.run.smithi130:> uname -r 2024-04-24T08:43:33.224 INFO:teuthology.orchestra.run.smithi130.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T08:43:33.225 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T08:43:33.225 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T08:43:33.225 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T08:43:34.226 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-04-24T08:43:34.226 DEBUG:teuthology.orchestra.run.smithi130:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T08:43:34.364 INFO:teuthology.orchestra.run.smithi130.stdout:ttyS1 2024-04-24T08:43:34.428 DEBUG:teuthology.parallel:result is None 2024-04-24T08:43:37.363 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-04-24T08:43:37.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:43:38.338 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-04-24T08:43:38.339 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:43:42.008 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-04-24T08:43:42.739 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-04-24T08:43:42.740 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T08:43:42.740 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-04-24T08:43:42.783 INFO:teuthology.orchestra.run.smithi081.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T08:43:42.784 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T08:43:42.784 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T08:43:42.784 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T08:43:43.784 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-24T08:43:43.785 DEBUG:teuthology.orchestra.run.smithi081:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T08:43:44.009 INFO:teuthology.orchestra.run.smithi081.stdout:ttyS1 2024-04-24T08:43:44.069 DEBUG:teuthology.parallel:result is None 2024-04-24T08:43:44.985 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-04-24T08:43:45.649 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-04-24T08:43:45.649 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T08:43:45.649 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2024-04-24T08:43:45.689 INFO:teuthology.orchestra.run.smithi069.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T08:43:45.689 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T08:43:45.689 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T08:43:45.689 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T08:43:46.689 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-24T08:43:46.690 DEBUG:teuthology.orchestra.run.smithi069:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T08:43:46.828 INFO:teuthology.orchestra.run.smithi069.stdout:ttyS1 2024-04-24T08:43:46.866 DEBUG:teuthology.parallel:result is None 2024-04-24T08:43:46.867 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T08:43:46.874 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T08:43:46.875 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T08:43:46.879 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T08:43:46.884 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T08:43:46.940 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T08:43:46.947 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T08:43:46.963 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T08:43:46.963 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T08:43:46.970 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T08:43:46.976 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T08:43:47.037 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T08:43:47.049 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T08:43:47.049 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T08:43:47.054 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T08:43:47.067 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T08:43:47.158 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T08:43:47.159 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T08:43:47.164 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T08:43:47.184 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T08:43:47.188 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T08:43:47.196 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T08:43:47.201 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T08:43:47.210 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T08:43:47.210 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T08:43:47.215 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T08:43:47.219 DEBUG:teuthology.orchestra.run.smithi130:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T08:43:47.289 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T08:43:47.301 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T08:43:47.301 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T08:43:47.309 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T08:43:47.313 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T08:43:47.357 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T08:43:47.459 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T08:43:47.547 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T08:43:47.619 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T08:43:47.701 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T08:43:47.702 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T08:43:47.767 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T08:43:47.840 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T08:43:47.940 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T08:43:48.010 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T08:43:48.098 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-04-24T08:43:48.098 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T08:43:48.170 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T08:43:48.249 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T08:43:48.331 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T08:43:48.402 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T08:43:48.467 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-24T08:43:48.468 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T08:43:48.534 DEBUG:teuthology.orchestra.run.smithi069:> sudo service rsyslog restart 2024-04-24T08:43:48.539 DEBUG:teuthology.orchestra.run.smithi081:> sudo service rsyslog restart 2024-04-24T08:43:48.546 DEBUG:teuthology.orchestra.run.smithi130:> sudo service rsyslog restart 2024-04-24T08:43:48.647 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T08:43:48.653 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T08:43:48.725 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T08:43:49.124 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T08:43:49.132 INFO:teuthology.task.internal:Starting timer... 2024-04-24T08:43:49.133 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T08:43:49.148 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T08:43:49.157 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T08:43:49.158 DEBUG:teuthology.orchestra.run.smithi069:> sudo service auditd rotate 2024-04-24T08:43:49.275 INFO:teuthology.orchestra.run.smithi069.stdout:Rotating logs: 2024-04-24T08:43:49.279 DEBUG:teuthology.orchestra.run.smithi081:> sudo service auditd rotate 2024-04-24T08:43:49.409 INFO:teuthology.orchestra.run.smithi081.stdout:Rotating logs: 2024-04-24T08:43:49.414 DEBUG:teuthology.orchestra.run.smithi130:> sudo service auditd rotate 2024-04-24T08:43:49.546 INFO:teuthology.orchestra.run.smithi130.stdout:Rotating logs: 2024-04-24T08:43:49.552 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T08:43:49.552 DEBUG:teuthology.orchestra.run.smithi069:> /usr/sbin/getenforce 2024-04-24T08:43:49.603 INFO:teuthology.orchestra.run.smithi069.stdout:Permissive 2024-04-24T08:43:49.604 DEBUG:teuthology.orchestra.run.smithi081:> /usr/sbin/getenforce 2024-04-24T08:43:49.657 INFO:teuthology.orchestra.run.smithi081.stdout:Permissive 2024-04-24T08:43:49.657 DEBUG:teuthology.orchestra.run.smithi130:> /usr/sbin/getenforce 2024-04-24T08:43:49.720 INFO:teuthology.orchestra.run.smithi130.stdout:Permissive 2024-04-24T08:43:49.720 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi069.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi081.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi130.front.sepia.ceph.com': 'permissive'} 2024-04-24T08:43:49.720 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T08:43:49.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:43:49.784 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T08:43:49.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:43:49.851 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T08:43:49.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:43:49.939 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T08:43:49.939 DEBUG:teuthology.orchestra.run.smithi069:> sudo /usr/sbin/setenforce permissive 2024-04-24T08:43:50.028 DEBUG:teuthology.orchestra.run.smithi081:> sudo /usr/sbin/setenforce permissive 2024-04-24T08:43:50.090 DEBUG:teuthology.orchestra.run.smithi130:> sudo /usr/sbin/setenforce permissive 2024-04-24T08:43:50.179 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T08:43:50.190 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-24T08:43:50.196 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-24T08:43:50.325 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T08:43:50.344 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T08:43:50.345 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi069.front.sepia.ceph.com,smithi081.front.sepia.ceph.com,smithi130.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T08:48:57.474 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi069.front.sepia.ceph.com'), Remote(name='ubuntu@smithi081.front.sepia.ceph.com'), Remote(name='ubuntu@smithi130.front.sepia.ceph.com')] 2024-04-24T08:48:57.475 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-04-24T08:48:57.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:48:57.588 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-04-24T08:48:57.745 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-04-24T08:48:57.746 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-04-24T08:48:57.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:48:57.854 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-04-24T08:48:58.006 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-04-24T08:48:58.006 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-04-24T08:48:58.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:48:58.113 DEBUG:teuthology.orchestra.run.smithi130:> true 2024-04-24T08:48:58.267 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-04-24T08:48:58.268 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T08:48:58.279 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T08:48:58.279 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T08:48:58.280 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T08:48:58.287 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T08:48:58.287 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T08:48:58.295 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T08:48:58.295 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T08:48:58.373 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T08:48:58.384 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T08:48:58.387 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T08:48:58.416 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T08:48:58.427 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T08:48:58.434 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T08:48:58.484 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: ntpd: command not found 2024-04-24T08:48:58.501 INFO:teuthology.orchestra.run.smithi081.stderr:sudo: ntpd: command not found 2024-04-24T08:48:58.507 INFO:teuthology.orchestra.run.smithi130.stderr:sudo: ntpd: command not found 2024-04-24T08:48:58.517 INFO:teuthology.orchestra.run.smithi069.stdout:506 Cannot talk to daemon 2024-04-24T08:48:58.531 INFO:teuthology.orchestra.run.smithi081.stdout:506 Cannot talk to daemon 2024-04-24T08:48:58.537 INFO:teuthology.orchestra.run.smithi130.stdout:506 Cannot talk to daemon 2024-04-24T08:48:58.559 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T08:48:58.571 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T08:48:58.576 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T08:48:58.600 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T08:48:58.611 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T08:48:58.615 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T08:48:58.730 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-04-24T08:48:58.739 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T08:48:58.739 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-04-24T08:48:58.739 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:48:58.739 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:48:58.739 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:48:58.739 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:48:58.742 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-04-24T08:48:58.750 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T08:48:58.750 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-04-24T08:48:58.750 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:48:58.750 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:48:58.750 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:48:58.750 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:48:58.757 INFO:teuthology.orchestra.run.smithi130.stderr:bash: line 1: ntpq: command not found 2024-04-24T08:48:58.765 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T08:48:58.765 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2024-04-24T08:48:58.765 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:48:58.765 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:48:58.765 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:48:58.765 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:48:58.766 INFO:teuthology.run_tasks:Running task install... 2024-04-24T08:48:58.775 DEBUG:teuthology.task.install:project ceph 2024-04-24T08:48:58.775 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0'}} 2024-04-24T08:48:58.775 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0'} 2024-04-24T08:48:58.775 INFO:teuthology.task.install:Using flavor: default 2024-04-24T08:48:58.781 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T08:48:58.781 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T08:48:58.781 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'tag': None, 'wait_for_package': False} 2024-04-24T08:48:58.782 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:48:58.784 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'tag': None, 'wait_for_package': False} 2024-04-24T08:48:58.784 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:48:58.785 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'tag': None, 'wait_for_package': False} 2024-04-24T08:48:58.785 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:48:59.002 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/ 2024-04-24T08:48:59.003 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3116.g1f81fb39 2024-04-24T08:48:59.009 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/ 2024-04-24T08:48:59.009 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3116.g1f81fb39 2024-04-24T08:48:59.016 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/ 2024-04-24T08:48:59.016 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3116.g1f81fb39 2024-04-24T08:48:59.387 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T08:48:59.387 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T08:48:59.388 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T08:48:59.479 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T08:48:59.480 DEBUG:teuthology.orchestra.run.smithi069:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1f81fb397ae98da7563d451a78c61574c8f4e6e0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T08:48:59.609 DEBUG:teuthology.orchestra.run.smithi069:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T08:48:59.702 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T08:48:59.702 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-04-24T08:48:59.702 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T08:48:59.710 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T08:48:59.710 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-24T08:48:59.710 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T08:48:59.713 DEBUG:teuthology.orchestra.run.smithi069:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T08:48:59.791 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T08:48:59.792 DEBUG:teuthology.orchestra.run.smithi081:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1f81fb397ae98da7563d451a78c61574c8f4e6e0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T08:48:59.792 INFO:teuthology.orchestra.run.smithi069.stdout:check_obsoletes = 1 2024-04-24T08:48:59.796 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T08:48:59.796 DEBUG:teuthology.orchestra.run.smithi130:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1f81fb397ae98da7563d451a78c61574c8f4e6e0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T08:48:59.797 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum clean all 2024-04-24T08:48:59.917 DEBUG:teuthology.orchestra.run.smithi081:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T08:48:59.923 DEBUG:teuthology.orchestra.run.smithi130:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T08:49:00.028 DEBUG:teuthology.orchestra.run.smithi081:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T08:49:00.031 DEBUG:teuthology.orchestra.run.smithi130:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T08:49:00.104 INFO:teuthology.orchestra.run.smithi130.stdout:check_obsoletes = 1 2024-04-24T08:49:00.105 INFO:teuthology.orchestra.run.smithi081.stdout:check_obsoletes = 1 2024-04-24T08:49:00.108 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum clean all 2024-04-24T08:49:00.109 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum clean all 2024-04-24T08:49:00.116 INFO:teuthology.orchestra.run.smithi069.stdout:48 files removed 2024-04-24T08:49:00.155 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T08:49:00.417 INFO:teuthology.orchestra.run.smithi130.stdout:48 files removed 2024-04-24T08:49:00.432 INFO:teuthology.orchestra.run.smithi081.stdout:48 files removed 2024-04-24T08:49:00.455 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T08:49:00.471 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T08:49:01.707 INFO:teuthology.orchestra.run.smithi069.stdout:ceph packages for x86_64 75 kB/s | 83 kB 00:01 2024-04-24T08:49:02.039 INFO:teuthology.orchestra.run.smithi081.stdout:ceph packages for x86_64 76 kB/s | 83 kB 00:01 2024-04-24T08:49:02.512 INFO:teuthology.orchestra.run.smithi130.stdout:ceph packages for x86_64 51 kB/s | 83 kB 00:01 2024-04-24T08:49:02.740 INFO:teuthology.orchestra.run.smithi069.stdout:ceph noarch packages 12 kB/s | 12 kB 00:00 2024-04-24T08:49:03.256 INFO:teuthology.orchestra.run.smithi081.stdout:ceph noarch packages 11 kB/s | 12 kB 00:01 2024-04-24T08:49:03.437 INFO:teuthology.orchestra.run.smithi130.stdout:ceph noarch packages 14 kB/s | 12 kB 00:00 2024-04-24T08:49:03.674 INFO:teuthology.orchestra.run.smithi069.stdout:ceph source packages 2.2 kB/s | 1.9 kB 00:00 2024-04-24T08:49:04.139 INFO:teuthology.orchestra.run.smithi081.stdout:ceph source packages 2.4 kB/s | 1.9 kB 00:00 2024-04-24T08:49:04.271 INFO:teuthology.orchestra.run.smithi130.stdout:ceph source packages 2.5 kB/s | 1.9 kB 00:00 2024-04-24T08:49:04.747 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 8.1 MB/s | 8.0 MB 00:00 2024-04-24T08:49:05.062 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 9.6 MB/s | 8.0 MB 00:00 2024-04-24T08:49:07.056 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-24T08:49:08.002 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.0 MB 00:03 2024-04-24T08:49:10.070 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-24T08:49:11.498 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-24T08:49:13.707 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 16 kB 00:00 2024-04-24T08:49:15.031 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-24T08:49:17.421 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 6.2 MB 00:03 2024-04-24T08:49:19.500 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-04-24T08:49:20.917 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-24T08:49:21.299 INFO:teuthology.orchestra.run.smithi130.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-24T08:49:23.293 INFO:teuthology.orchestra.run.smithi081.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T08:49:23.295 INFO:teuthology.orchestra.run.smithi081.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T08:49:23.315 INFO:teuthology.orchestra.run.smithi081.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T08:49:23.317 INFO:teuthology.orchestra.run.smithi081.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T08:49:23.318 INFO:teuthology.orchestra.run.smithi081.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T08:49:23.319 INFO:teuthology.orchestra.run.smithi081.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T08:49:23.321 INFO:teuthology.orchestra.run.smithi081.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T08:49:23.325 INFO:teuthology.orchestra.run.smithi081.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T08:49:23.326 INFO:teuthology.orchestra.run.smithi081.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T08:49:23.396 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-04-24T08:49:23.415 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================= 2024-04-24T08:49:23.415 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-04-24T08:49:23.415 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================= 2024-04-24T08:49:23.415 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-04-24T08:49:23.415 INFO:teuthology.orchestra.run.smithi081.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T08:49:23.415 INFO:teuthology.orchestra.run.smithi081.stdout: ceph x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 6.6 k 2024-04-24T08:49:23.415 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T08:49:23.415 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 890 k 2024-04-24T08:49:23.415 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 145 k 2024-04-24T08:49:23.415 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 1.5 M 2024-04-24T08:49:23.415 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 142 k 2024-04-24T08:49:23.415 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 3.5 M 2024-04-24T08:49:23.416 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 7.4 M 2024-04-24T08:49:23.416 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 49 k 2024-04-24T08:49:23.416 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 11 M 2024-04-24T08:49:23.416 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 49 M 2024-04-24T08:49:23.416 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 273 k 2024-04-24T08:49:23.416 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 762 k 2024-04-24T08:49:23.416 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 225 k 2024-04-24T08:49:23.416 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-shell x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 27 k 2024-04-24T08:49:23.416 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-top noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 18 k 2024-04-24T08:49:23.416 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T08:49:23.416 INFO:teuthology.orchestra.run.smithi081.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T08:49:23.417 INFO:teuthology.orchestra.run.smithi081.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T08:49:23.417 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 32 k 2024-04-24T08:49:23.417 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 746 k 2024-04-24T08:49:23.417 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 127 k 2024-04-24T08:49:23.417 INFO:teuthology.orchestra.run.smithi081.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T08:49:23.417 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T08:49:23.417 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T08:49:23.417 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T08:49:23.417 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T08:49:23.417 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T08:49:23.417 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 161 k 2024-04-24T08:49:23.417 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 320 k 2024-04-24T08:49:23.417 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 298 k 2024-04-24T08:49:23.418 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 99 k 2024-04-24T08:49:23.418 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 85 k 2024-04-24T08:49:23.418 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T08:49:23.418 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 170 k 2024-04-24T08:49:23.418 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T08:49:23.418 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrading: 2024-04-24T08:49:23.418 INFO:teuthology.orchestra.run.smithi081.stdout: librados2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.4 M 2024-04-24T08:49:23.418 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T08:49:23.418 INFO:teuthology.orchestra.run.smithi081.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T08:49:23.418 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T08:49:23.418 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-04-24T08:49:23.418 INFO:teuthology.orchestra.run.smithi081.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T08:49:23.418 INFO:teuthology.orchestra.run.smithi081.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T08:49:23.418 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T08:49:23.419 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 21 M 2024-04-24T08:49:23.419 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 34 k 2024-04-24T08:49:23.419 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 2.4 M 2024-04-24T08:49:23.419 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 249 k 2024-04-24T08:49:23.419 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 4.7 M 2024-04-24T08:49:23.419 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 17 M 2024-04-24T08:49:23.419 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 16 k 2024-04-24T08:49:23.419 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 25 k 2024-04-24T08:49:23.419 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T08:49:23.419 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T08:49:23.419 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T08:49:23.419 INFO:teuthology.orchestra.run.smithi081.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T08:49:23.419 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T08:49:23.420 INFO:teuthology.orchestra.run.smithi081.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T08:49:23.420 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T08:49:23.420 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T08:49:23.420 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 163 k 2024-04-24T08:49:23.420 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T08:49:23.420 INFO:teuthology.orchestra.run.smithi081.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T08:49:23.420 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T08:49:23.420 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T08:49:23.420 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T08:49:23.420 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 501 k 2024-04-24T08:49:23.420 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T08:49:23.420 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T08:49:23.420 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T08:49:23.420 INFO:teuthology.orchestra.run.smithi081.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T08:49:23.420 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T08:49:23.421 INFO:teuthology.orchestra.run.smithi081.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T08:49:23.421 INFO:teuthology.orchestra.run.smithi081.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T08:49:23.421 INFO:teuthology.orchestra.run.smithi081.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T08:49:23.421 INFO:teuthology.orchestra.run.smithi081.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T08:49:23.421 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T08:49:23.421 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T08:49:23.421 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T08:49:23.421 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T08:49:23.421 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T08:49:23.421 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T08:49:23.421 INFO:teuthology.orchestra.run.smithi081.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T08:49:23.421 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T08:49:23.422 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T08:49:23.422 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T08:49:23.422 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 45 k 2024-04-24T08:49:23.422 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 135 k 2024-04-24T08:49:23.422 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T08:49:23.422 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T08:49:23.422 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T08:49:23.422 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T08:49:23.422 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T08:49:23.422 INFO:teuthology.orchestra.run.smithi081.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T08:49:23.422 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T08:49:23.422 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T08:49:23.422 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T08:49:23.423 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T08:49:23.423 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T08:49:23.423 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T08:49:23.423 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T08:49:23.423 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T08:49:23.423 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T08:49:23.423 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T08:49:23.423 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T08:49:23.423 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T08:49:23.423 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T08:49:23.423 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T08:49:23.423 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T08:49:23.423 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T08:49:23.423 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T08:49:23.424 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T08:49:23.424 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T08:49:23.424 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T08:49:23.424 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T08:49:23.424 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T08:49:23.424 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T08:49:23.424 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T08:49:23.424 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T08:49:23.424 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T08:49:23.424 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T08:49:23.424 INFO:teuthology.orchestra.run.smithi081.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T08:49:23.424 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T08:49:23.424 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T08:49:23.425 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T08:49:23.425 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T08:49:23.425 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T08:49:23.425 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T08:49:23.425 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T08:49:23.425 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T08:49:23.425 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T08:49:23.425 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T08:49:23.425 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T08:49:23.425 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T08:49:23.425 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T08:49:23.425 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T08:49:23.426 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T08:49:23.426 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T08:49:23.426 INFO:teuthology.orchestra.run.smithi081.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T08:49:23.426 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T08:49:23.426 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T08:49:23.426 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T08:49:23.426 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T08:49:23.426 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T08:49:23.426 INFO:teuthology.orchestra.run.smithi081.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T08:49:23.426 INFO:teuthology.orchestra.run.smithi081.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T08:49:23.426 INFO:teuthology.orchestra.run.smithi081.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T08:49:23.426 INFO:teuthology.orchestra.run.smithi081.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T08:49:23.426 INFO:teuthology.orchestra.run.smithi081.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T08:49:23.426 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T08:49:23.427 INFO:teuthology.orchestra.run.smithi081.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T08:49:23.427 INFO:teuthology.orchestra.run.smithi081.stdout:Installing weak dependencies: 2024-04-24T08:49:23.427 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T08:49:23.427 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:49:23.427 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-04-24T08:49:23.427 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================= 2024-04-24T08:49:23.427 INFO:teuthology.orchestra.run.smithi081.stdout:Install 148 Packages 2024-04-24T08:49:23.427 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrade 4 Packages 2024-04-24T08:49:23.427 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:49:23.428 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 222 M 2024-04-24T08:49:23.428 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-04-24T08:49:24.837 INFO:teuthology.orchestra.run.smithi081.stdout:(1/152): ceph-19.0.0-3116.g1f81fb39.el9.x86_64. 12 kB/s | 6.6 kB 00:00 2024-04-24T08:49:25.503 INFO:teuthology.orchestra.run.smithi081.stdout:(2/152): ceph-fuse-19.0.0-3116.g1f81fb39.el9.x8 1.3 MB/s | 890 kB 00:00 2024-04-24T08:49:25.611 INFO:teuthology.orchestra.run.smithi081.stdout:(3/152): ceph-immutable-object-cache-19.0.0-311 1.3 MB/s | 145 kB 00:00 2024-04-24T08:49:26.020 INFO:teuthology.orchestra.run.smithi081.stdout:(4/152): ceph-mds-19.0.0-3116.g1f81fb39.el9.x86 5.8 MB/s | 2.4 MB 00:00 2024-04-24T08:49:26.287 INFO:teuthology.orchestra.run.smithi081.stdout:(5/152): ceph-mgr-19.0.0-3116.g1f81fb39.el9.x86 5.6 MB/s | 1.5 MB 00:00 2024-04-24T08:49:26.404 INFO:teuthology.orchestra.run.smithi081.stdout:(6/152): ceph-base-19.0.0-3116.g1f81fb39.el9.x8 2.6 MB/s | 5.4 MB 00:02 2024-04-24T08:49:26.996 INFO:teuthology.orchestra.run.smithi081.stdout:(7/152): ceph-mon-19.0.0-3116.g1f81fb39.el9.x86 6.6 MB/s | 4.7 MB 00:00 2024-04-24T08:49:27.053 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 924 kB/s | 19 MB 00:21 2024-04-24T08:49:27.290 INFO:teuthology.orchestra.run.smithi130.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-24T08:49:28.412 INFO:teuthology.orchestra.run.smithi081.stdout:(8/152): ceph-radosgw-19.0.0-3116.g1f81fb39.el9 7.5 MB/s | 11 MB 00:01 2024-04-24T08:49:28.496 INFO:teuthology.orchestra.run.smithi081.stdout:(9/152): ceph-selinux-19.0.0-3116.g1f81fb39.el9 306 kB/s | 25 kB 00:00 2024-04-24T08:49:29.690 INFO:teuthology.orchestra.run.smithi130.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T08:49:29.692 INFO:teuthology.orchestra.run.smithi130.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T08:49:29.712 INFO:teuthology.orchestra.run.smithi130.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T08:49:29.713 INFO:teuthology.orchestra.run.smithi130.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T08:49:29.714 INFO:teuthology.orchestra.run.smithi130.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T08:49:29.716 INFO:teuthology.orchestra.run.smithi130.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T08:49:29.718 INFO:teuthology.orchestra.run.smithi130.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T08:49:29.722 INFO:teuthology.orchestra.run.smithi130.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T08:49:29.723 INFO:teuthology.orchestra.run.smithi130.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T08:49:29.795 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-24T08:49:29.831 INFO:teuthology.orchestra.run.smithi130.stdout:======================================================================================= 2024-04-24T08:49:29.831 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-04-24T08:49:29.831 INFO:teuthology.orchestra.run.smithi130.stdout:======================================================================================= 2024-04-24T08:49:29.831 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-24T08:49:29.831 INFO:teuthology.orchestra.run.smithi130.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T08:49:29.831 INFO:teuthology.orchestra.run.smithi130.stdout: ceph x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 6.6 k 2024-04-24T08:49:29.832 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T08:49:29.832 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 890 k 2024-04-24T08:49:29.832 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 145 k 2024-04-24T08:49:29.832 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 1.5 M 2024-04-24T08:49:29.832 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 142 k 2024-04-24T08:49:29.832 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 3.5 M 2024-04-24T08:49:29.832 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 7.4 M 2024-04-24T08:49:29.832 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 49 k 2024-04-24T08:49:29.832 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 11 M 2024-04-24T08:49:29.832 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 49 M 2024-04-24T08:49:29.832 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-volume noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 273 k 2024-04-24T08:49:29.832 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 762 k 2024-04-24T08:49:29.832 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 225 k 2024-04-24T08:49:29.832 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-shell x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 27 k 2024-04-24T08:49:29.832 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-top noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 18 k 2024-04-24T08:49:29.833 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T08:49:29.833 INFO:teuthology.orchestra.run.smithi130.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T08:49:29.833 INFO:teuthology.orchestra.run.smithi130.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T08:49:29.833 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 32 k 2024-04-24T08:49:29.833 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 746 k 2024-04-24T08:49:29.833 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 127 k 2024-04-24T08:49:29.833 INFO:teuthology.orchestra.run.smithi130.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T08:49:29.833 INFO:teuthology.orchestra.run.smithi130.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T08:49:29.833 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T08:49:29.833 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T08:49:29.833 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T08:49:29.833 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T08:49:29.833 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 161 k 2024-04-24T08:49:29.833 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 320 k 2024-04-24T08:49:29.834 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 298 k 2024-04-24T08:49:29.834 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 99 k 2024-04-24T08:49:29.834 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 85 k 2024-04-24T08:49:29.834 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T08:49:29.834 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 170 k 2024-04-24T08:49:29.834 INFO:teuthology.orchestra.run.smithi130.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T08:49:29.834 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrading: 2024-04-24T08:49:29.834 INFO:teuthology.orchestra.run.smithi130.stdout: librados2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.4 M 2024-04-24T08:49:29.834 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T08:49:29.834 INFO:teuthology.orchestra.run.smithi130.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T08:49:29.834 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T08:49:29.834 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-04-24T08:49:29.834 INFO:teuthology.orchestra.run.smithi130.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T08:49:29.834 INFO:teuthology.orchestra.run.smithi130.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T08:49:29.834 INFO:teuthology.orchestra.run.smithi130.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T08:49:29.835 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 21 M 2024-04-24T08:49:29.835 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 34 k 2024-04-24T08:49:29.835 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 2.4 M 2024-04-24T08:49:29.835 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 249 k 2024-04-24T08:49:29.835 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 4.7 M 2024-04-24T08:49:29.835 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 17 M 2024-04-24T08:49:29.835 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 16 k 2024-04-24T08:49:29.835 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 25 k 2024-04-24T08:49:29.835 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T08:49:29.835 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T08:49:29.835 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T08:49:29.835 INFO:teuthology.orchestra.run.smithi130.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T08:49:29.835 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T08:49:29.836 INFO:teuthology.orchestra.run.smithi130.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T08:49:29.836 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T08:49:29.836 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T08:49:29.836 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 163 k 2024-04-24T08:49:29.836 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T08:49:29.836 INFO:teuthology.orchestra.run.smithi130.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T08:49:29.836 INFO:teuthology.orchestra.run.smithi130.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T08:49:29.836 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T08:49:29.836 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T08:49:29.836 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 501 k 2024-04-24T08:49:29.836 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T08:49:29.836 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T08:49:29.836 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T08:49:29.836 INFO:teuthology.orchestra.run.smithi130.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T08:49:29.837 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T08:49:29.837 INFO:teuthology.orchestra.run.smithi130.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T08:49:29.837 INFO:teuthology.orchestra.run.smithi130.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T08:49:29.837 INFO:teuthology.orchestra.run.smithi130.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T08:49:29.837 INFO:teuthology.orchestra.run.smithi130.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T08:49:29.837 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T08:49:29.837 INFO:teuthology.orchestra.run.smithi130.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T08:49:29.837 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T08:49:29.837 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T08:49:29.837 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T08:49:29.837 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T08:49:29.837 INFO:teuthology.orchestra.run.smithi130.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T08:49:29.837 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T08:49:29.838 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T08:49:29.838 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T08:49:29.838 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 45 k 2024-04-24T08:49:29.838 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 135 k 2024-04-24T08:49:29.838 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T08:49:29.838 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T08:49:29.838 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T08:49:29.838 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T08:49:29.838 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T08:49:29.838 INFO:teuthology.orchestra.run.smithi130.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T08:49:29.838 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T08:49:29.838 INFO:teuthology.orchestra.run.smithi130.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T08:49:29.839 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T08:49:29.839 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T08:49:29.839 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T08:49:29.839 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T08:49:29.839 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T08:49:29.839 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T08:49:29.839 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T08:49:29.839 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T08:49:29.839 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T08:49:29.839 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T08:49:29.839 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T08:49:29.839 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T08:49:29.840 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T08:49:29.840 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T08:49:29.840 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T08:49:29.840 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T08:49:29.840 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T08:49:29.840 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T08:49:29.840 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T08:49:29.840 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T08:49:29.840 INFO:teuthology.orchestra.run.smithi130.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T08:49:29.840 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T08:49:29.840 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T08:49:29.840 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T08:49:29.840 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T08:49:29.840 INFO:teuthology.orchestra.run.smithi130.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T08:49:29.841 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T08:49:29.841 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T08:49:29.841 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T08:49:29.841 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T08:49:29.841 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T08:49:29.841 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T08:49:29.841 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T08:49:29.841 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T08:49:29.841 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T08:49:29.841 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T08:49:29.841 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T08:49:29.841 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T08:49:29.842 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T08:49:29.842 INFO:teuthology.orchestra.run.smithi130.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T08:49:29.842 INFO:teuthology.orchestra.run.smithi130.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T08:49:29.842 INFO:teuthology.orchestra.run.smithi130.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T08:49:29.842 INFO:teuthology.orchestra.run.smithi130.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T08:49:29.842 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T08:49:29.842 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T08:49:29.842 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T08:49:29.842 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T08:49:29.842 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T08:49:29.842 INFO:teuthology.orchestra.run.smithi130.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T08:49:29.842 INFO:teuthology.orchestra.run.smithi130.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T08:49:29.842 INFO:teuthology.orchestra.run.smithi130.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T08:49:29.842 INFO:teuthology.orchestra.run.smithi130.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T08:49:29.843 INFO:teuthology.orchestra.run.smithi130.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T08:49:29.843 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T08:49:29.843 INFO:teuthology.orchestra.run.smithi130.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T08:49:29.843 INFO:teuthology.orchestra.run.smithi130.stdout:Installing weak dependencies: 2024-04-24T08:49:29.843 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T08:49:29.843 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:49:29.843 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-24T08:49:29.843 INFO:teuthology.orchestra.run.smithi130.stdout:======================================================================================= 2024-04-24T08:49:29.843 INFO:teuthology.orchestra.run.smithi130.stdout:Install 148 Packages 2024-04-24T08:49:29.843 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrade 4 Packages 2024-04-24T08:49:29.843 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:49:29.844 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 222 M 2024-04-24T08:49:29.844 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-24T08:49:30.396 INFO:teuthology.orchestra.run.smithi081.stdout:(10/152): ceph-osd-19.0.0-3116.g1f81fb39.el9.x8 4.2 MB/s | 17 MB 00:03 2024-04-24T08:49:30.596 INFO:teuthology.orchestra.run.smithi081.stdout:(11/152): cephfs-mirror-19.0.0-3116.g1f81fb39.e 1.1 MB/s | 225 kB 00:00 2024-04-24T08:49:30.705 INFO:teuthology.orchestra.run.smithi081.stdout:(12/152): cephfs-shell-19.0.0-3116.g1f81fb39.el 252 kB/s | 27 kB 00:00 2024-04-24T08:49:30.822 INFO:teuthology.orchestra.run.smithi081.stdout:(13/152): libcephfs-devel-19.0.0-3116.g1f81fb39 274 kB/s | 32 kB 00:00 2024-04-24T08:49:31.114 INFO:teuthology.orchestra.run.smithi081.stdout:(14/152): ceph-common-19.0.0-3116.g1f81fb39.el9 3.1 MB/s | 21 MB 00:06 2024-04-24T08:49:31.117 INFO:teuthology.orchestra.run.smithi130.stdout:(1/152): ceph-19.0.0-3116.g1f81fb39.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-24T08:49:31.406 INFO:teuthology.orchestra.run.smithi081.stdout:(15/152): libcephfs2-19.0.0-3116.g1f81fb39.el9. 1.3 MB/s | 746 kB 00:00 2024-04-24T08:49:31.523 INFO:teuthology.orchestra.run.smithi081.stdout:(16/152): librados-devel-19.0.0-3116.g1f81fb39. 1.1 MB/s | 127 kB 00:00 2024-04-24T08:49:31.639 INFO:teuthology.orchestra.run.smithi081.stdout:(17/152): libcephsqlite-19.0.0-3116.g1f81fb39.e 312 kB/s | 163 kB 00:00 2024-04-24T08:49:31.823 INFO:teuthology.orchestra.run.smithi081.stdout:(18/152): libradosstriper1-19.0.0-3116.g1f81fb3 1.6 MB/s | 501 kB 00:00 2024-04-24T08:49:31.923 INFO:teuthology.orchestra.run.smithi081.stdout:(19/152): python3-ceph-argparse-19.0.0-3116.g1f 456 kB/s | 45 kB 00:00 2024-04-24T08:49:32.032 INFO:teuthology.orchestra.run.smithi081.stdout:(20/152): python3-ceph-common-19.0.0-3116.g1f81 1.2 MB/s | 135 kB 00:00 2024-04-24T08:49:32.141 INFO:teuthology.orchestra.run.smithi081.stdout:(21/152): python3-cephfs-19.0.0-3116.g1f81fb39. 1.5 MB/s | 161 kB 00:00 2024-04-24T08:49:32.349 INFO:teuthology.orchestra.run.smithi081.stdout:(22/152): python3-rados-19.0.0-3116.g1f81fb39.e 1.5 MB/s | 320 kB 00:00 2024-04-24T08:49:32.416 INFO:teuthology.orchestra.run.smithi130.stdout:(2/152): ceph-fuse-19.0.0-3116.g1f81fb39.el9.x8 685 kB/s | 890 kB 00:01 2024-04-24T08:49:32.541 INFO:teuthology.orchestra.run.smithi081.stdout:(23/152): python3-rbd-19.0.0-3116.g1f81fb39.el9 1.5 MB/s | 298 kB 00:00 2024-04-24T08:49:32.658 INFO:teuthology.orchestra.run.smithi081.stdout:(24/152): python3-rgw-19.0.0-3116.g1f81fb39.el9 858 kB/s | 99 kB 00:00 2024-04-24T08:49:32.692 INFO:teuthology.orchestra.run.smithi130.stdout:(3/152): ceph-immutable-object-cache-19.0.0-311 527 kB/s | 145 kB 00:00 2024-04-24T08:49:32.767 INFO:teuthology.orchestra.run.smithi081.stdout:(25/152): rbd-fuse-19.0.0-3116.g1f81fb39.el9.x8 793 kB/s | 85 kB 00:00 2024-04-24T08:49:33.259 INFO:teuthology.orchestra.run.smithi081.stdout:(26/152): librgw2-19.0.0-3116.g1f81fb39.el9.x86 3.3 MB/s | 5.4 MB 00:01 2024-04-24T08:49:33.350 INFO:teuthology.orchestra.run.smithi081.stdout:(27/152): rbd-nbd-19.0.0-3116.g1f81fb39.el9.x86 1.8 MB/s | 170 kB 00:00 2024-04-24T08:49:33.400 INFO:teuthology.orchestra.run.smithi130.stdout:(4/152): ceph-base-19.0.0-3116.g1f81fb39.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-04-24T08:49:33.451 INFO:teuthology.orchestra.run.smithi081.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3116.g 344 kB/s | 34 kB 00:00 2024-04-24T08:49:33.568 INFO:teuthology.orchestra.run.smithi081.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3116.g1f81fb3 1.2 MB/s | 142 kB 00:00 2024-04-24T08:49:33.768 INFO:teuthology.orchestra.run.smithi130.stdout:(5/152): ceph-mgr-19.0.0-3116.g1f81fb39.el9.x86 4.1 MB/s | 1.5 MB 00:00 2024-04-24T08:49:33.802 INFO:teuthology.orchestra.run.smithi081.stdout:(30/152): rbd-mirror-19.0.0-3116.g1f81fb39.el9. 3.0 MB/s | 3.1 MB 00:01 2024-04-24T08:49:34.518 INFO:teuthology.orchestra.run.smithi130.stdout:(6/152): ceph-mon-19.0.0-3116.g1f81fb39.el9.x86 6.3 MB/s | 4.7 MB 00:00 2024-04-24T08:49:34.652 INFO:teuthology.orchestra.run.smithi081.stdout:(31/152): ceph-test-19.0.0-3116.g1f81fb39.el9.x 8.0 MB/s | 49 MB 00:06 2024-04-24T08:49:34.777 INFO:teuthology.orchestra.run.smithi081.stdout:(32/152): ceph-mgr-dashboard-19.0.0-3116.g1f81f 2.9 MB/s | 3.5 MB 00:01 2024-04-24T08:49:34.894 INFO:teuthology.orchestra.run.smithi081.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3116.g1f 1.0 MB/s | 249 kB 00:00 2024-04-24T08:49:34.919 INFO:teuthology.orchestra.run.smithi081.stdout:(34/152): ceph-mgr-rook-19.0.0-3116.g1f81fb39.e 350 kB/s | 49 kB 00:00 2024-04-24T08:49:35.003 INFO:teuthology.orchestra.run.smithi081.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3116.g1 152 kB/s | 16 kB 00:00 2024-04-24T08:49:35.303 INFO:teuthology.orchestra.run.smithi081.stdout:(36/152): ceph-volume-19.0.0-3116.g1f81fb39.el9 713 kB/s | 273 kB 00:00 2024-04-24T08:49:35.404 INFO:teuthology.orchestra.run.smithi081.stdout:(37/152): cephfs-top-19.0.0-3116.g1f81fb39.el9. 183 kB/s | 18 kB 00:00 2024-04-24T08:49:35.434 INFO:teuthology.orchestra.run.smithi130.stdout:(7/152): ceph-mds-19.0.0-3116.g1f81fb39.el9.x86 882 kB/s | 2.4 MB 00:02 2024-04-24T08:49:35.445 INFO:teuthology.orchestra.run.smithi081.stdout:(38/152): cephadm-19.0.0-3116.g1f81fb39.el9.noa 1.7 MB/s | 762 kB 00:00 2024-04-24T08:49:36.187 INFO:teuthology.orchestra.run.smithi081.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 3.1 MB/s | 7.4 MB 00:02 2024-04-24T08:49:36.221 INFO:teuthology.orchestra.run.smithi081.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 98 kB/s | 80 kB 00:00 2024-04-24T08:49:36.393 INFO:teuthology.orchestra.run.smithi130.stdout:(8/152): ceph-osd-19.0.0-3116.g1f81fb39.el9.x86 9.0 MB/s | 17 MB 00:01 2024-04-24T08:49:36.468 INFO:teuthology.orchestra.run.smithi130.stdout:(9/152): ceph-selinux-19.0.0-3116.g1f81fb39.el9 340 kB/s | 25 kB 00:00 2024-04-24T08:49:36.727 INFO:teuthology.orchestra.run.smithi130.stdout:(10/152): ceph-common-19.0.0-3116.g1f81fb39.el9 3.5 MB/s | 21 MB 00:06 2024-04-24T08:49:36.802 INFO:teuthology.orchestra.run.smithi130.stdout:(11/152): cephfs-mirror-19.0.0-3116.g1f81fb39.e 3.0 MB/s | 225 kB 00:00 2024-04-24T08:49:36.886 INFO:teuthology.orchestra.run.smithi130.stdout:(12/152): cephfs-shell-19.0.0-3116.g1f81fb39.el 327 kB/s | 27 kB 00:00 2024-04-24T08:49:36.969 INFO:teuthology.orchestra.run.smithi130.stdout:(13/152): libcephfs-devel-19.0.0-3116.g1f81fb39 386 kB/s | 32 kB 00:00 2024-04-24T08:49:37.145 INFO:teuthology.orchestra.run.smithi130.stdout:(14/152): libcephfs2-19.0.0-3116.g1f81fb39.el9. 4.2 MB/s | 746 kB 00:00 2024-04-24T08:49:37.148 INFO:teuthology.orchestra.run.smithi081.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 204 kB/s | 195 kB 00:00 2024-04-24T08:49:37.237 INFO:teuthology.orchestra.run.smithi130.stdout:(15/152): libcephsqlite-19.0.0-3116.g1f81fb39.e 1.8 MB/s | 163 kB 00:00 2024-04-24T08:49:37.246 INFO:teuthology.orchestra.run.smithi081.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 247 kB/s | 253 kB 00:01 2024-04-24T08:49:37.288 INFO:teuthology.orchestra.run.smithi081.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 436 kB/s | 803 kB 00:01 2024-04-24T08:49:37.336 INFO:teuthology.orchestra.run.smithi130.stdout:(16/152): librados-devel-19.0.0-3116.g1f81fb39. 1.3 MB/s | 127 kB 00:00 2024-04-24T08:49:37.454 INFO:teuthology.orchestra.run.smithi130.stdout:(17/152): libradosstriper1-19.0.0-3116.g1f81fb3 4.2 MB/s | 501 kB 00:00 2024-04-24T08:49:38.022 INFO:teuthology.orchestra.run.smithi081.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 138 kB/s | 106 kB 00:00 2024-04-24T08:49:38.147 INFO:teuthology.orchestra.run.smithi081.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 158 kB/s | 135 kB 00:00 2024-04-24T08:49:38.346 INFO:teuthology.orchestra.run.smithi130.stdout:(18/152): librgw2-19.0.0-3116.g1f81fb39.el9.x86 6.0 MB/s | 5.4 MB 00:00 2024-04-24T08:49:38.429 INFO:teuthology.orchestra.run.smithi130.stdout:(19/152): python3-ceph-argparse-19.0.0-3116.g1f 549 kB/s | 45 kB 00:00 2024-04-24T08:49:38.513 INFO:teuthology.orchestra.run.smithi130.stdout:(20/152): python3-ceph-common-19.0.0-3116.g1f81 1.6 MB/s | 135 kB 00:00 2024-04-24T08:49:38.613 INFO:teuthology.orchestra.run.smithi130.stdout:(21/152): python3-cephfs-19.0.0-3116.g1f81fb39. 1.6 MB/s | 161 kB 00:00 2024-04-24T08:49:38.812 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 805 kB/s | 6.2 MB 00:07 2024-04-24T08:49:38.856 INFO:teuthology.orchestra.run.smithi081.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 150 kB/s | 125 kB 00:00 2024-04-24T08:49:39.039 INFO:teuthology.orchestra.run.smithi130.stdout:(22/152): python3-rados-19.0.0-3116.g1f81fb39.e 755 kB/s | 320 kB 00:00 2024-04-24T08:49:39.072 INFO:teuthology.orchestra.run.smithi081.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 232 kB/s | 215 kB 00:00 2024-04-24T08:49:39.147 INFO:teuthology.orchestra.run.smithi130.stdout:(23/152): python3-rbd-19.0.0-3116.g1f81fb39.el9 2.7 MB/s | 298 kB 00:00 2024-04-24T08:49:39.148 INFO:teuthology.orchestra.run.smithi081.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-04-24T08:49:39.207 INFO:teuthology.orchestra.run.smithi081.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-24T08:49:39.231 INFO:teuthology.orchestra.run.smithi130.stdout:(24/152): python3-rgw-19.0.0-3116.g1f81fb39.el9 1.2 MB/s | 99 kB 00:00 2024-04-24T08:49:39.265 INFO:teuthology.orchestra.run.smithi081.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 374 kB/s | 21 kB 00:00 2024-04-24T08:49:39.314 INFO:teuthology.orchestra.run.smithi130.stdout:(25/152): rbd-fuse-19.0.0-3116.g1f81fb39.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-24T08:49:39.357 INFO:teuthology.orchestra.run.smithi081.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 3.4 MB/s | 313 kB 00:00 2024-04-24T08:49:39.416 INFO:teuthology.orchestra.run.smithi081.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-04-24T08:49:39.441 INFO:teuthology.orchestra.run.smithi081.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 391 kB/s | 33 kB 00:00 2024-04-24T08:49:39.483 INFO:teuthology.orchestra.run.smithi081.stdout:(54/152): python3-cryptography-36.0.1-4.el9.x86 538 kB/s | 1.2 MB 00:02 2024-04-24T08:49:39.508 INFO:teuthology.orchestra.run.smithi081.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 275 kB/s | 18 kB 00:00 2024-04-24T08:49:39.567 INFO:teuthology.orchestra.run.smithi081.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-24T08:49:39.592 INFO:teuthology.orchestra.run.smithi081.stdout:(57/152): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-04-24T08:49:39.648 INFO:teuthology.orchestra.run.smithi081.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 565 kB/s | 45 kB 00:00 2024-04-24T08:49:39.776 INFO:teuthology.orchestra.run.smithi081.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 4.5 MB/s | 582 kB 00:00 2024-04-24T08:49:39.827 INFO:teuthology.orchestra.run.smithi081.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 518 kB/s | 25 kB 00:00 2024-04-24T08:49:39.860 INFO:teuthology.orchestra.run.smithi081.stdout:(61/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-04-24T08:49:39.877 INFO:teuthology.orchestra.run.smithi081.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 956 kB/s | 47 kB 00:00 2024-04-24T08:49:39.881 INFO:teuthology.orchestra.run.smithi130.stdout:(26/152): rbd-mirror-19.0.0-3116.g1f81fb39.el9. 5.5 MB/s | 3.1 MB 00:00 2024-04-24T08:49:39.961 INFO:teuthology.orchestra.run.smithi081.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.5 MB/s | 3.0 MB 00:00 2024-04-24T08:49:39.981 INFO:teuthology.orchestra.run.smithi130.stdout:(27/152): rbd-nbd-19.0.0-3116.g1f81fb39.el9.x86 1.7 MB/s | 170 kB 00:00 2024-04-24T08:49:39.994 INFO:teuthology.orchestra.run.smithi081.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-04-24T08:49:40.028 INFO:teuthology.orchestra.run.smithi081.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-04-24T08:49:40.053 INFO:teuthology.orchestra.run.smithi081.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 386 kB/s | 35 kB 00:00 2024-04-24T08:49:40.073 INFO:teuthology.orchestra.run.smithi130.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3116.g 378 kB/s | 34 kB 00:00 2024-04-24T08:49:40.165 INFO:teuthology.orchestra.run.smithi130.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3116.g1f81fb3 1.5 MB/s | 142 kB 00:00 2024-04-24T08:49:40.312 INFO:teuthology.orchestra.run.smithi081.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 6.2 MB/s | 1.6 MB 00:00 2024-04-24T08:49:40.512 INFO:teuthology.orchestra.run.smithi081.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 4.5 MB/s | 911 kB 00:00 2024-04-24T08:49:40.570 INFO:teuthology.orchestra.run.smithi081.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-24T08:49:40.763 INFO:teuthology.orchestra.run.smithi081.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.2 MB/s | 4.4 MB 00:00 2024-04-24T08:49:40.766 INFO:teuthology.orchestra.run.smithi130.stdout:(30/152): ceph-test-19.0.0-3116.g1f81fb39.el9.x 11 MB/s | 49 MB 00:04 2024-04-24T08:49:40.863 INFO:teuthology.orchestra.run.smithi081.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 5.5 MB/s | 4.7 MB 00:00 2024-04-24T08:49:40.922 INFO:teuthology.orchestra.run.smithi081.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-24T08:49:40.972 INFO:teuthology.orchestra.run.smithi081.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-04-24T08:49:40.974 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 16 kB 00:00 2024-04-24T08:49:40.975 INFO:teuthology.orchestra.run.smithi130.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3116.g1f81f 4.4 MB/s | 3.5 MB 00:00 2024-04-24T08:49:41.075 INFO:teuthology.orchestra.run.smithi130.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3116.g1f 2.5 MB/s | 249 kB 00:00 2024-04-24T08:49:41.150 INFO:teuthology.orchestra.run.smithi081.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.3 MB/s | 244 kB 00:00 2024-04-24T08:49:41.183 INFO:teuthology.orchestra.run.smithi130.stdout:(33/152): ceph-mgr-rook-19.0.0-3116.g1f81fb39.e 459 kB/s | 49 kB 00:00 2024-04-24T08:49:41.267 INFO:teuthology.orchestra.run.smithi130.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3116.g1 200 kB/s | 16 kB 00:00 2024-04-24T08:49:41.375 INFO:teuthology.orchestra.run.smithi130.stdout:(35/152): ceph-volume-19.0.0-3116.g1f81fb39.el9 2.5 MB/s | 273 kB 00:00 2024-04-24T08:49:41.381 INFO:teuthology.orchestra.run.smithi081.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.1 MB/s | 248 kB 00:00 2024-04-24T08:49:41.439 INFO:teuthology.orchestra.run.smithi081.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 453 kB/s | 26 kB 00:00 2024-04-24T08:49:41.498 INFO:teuthology.orchestra.run.smithi081.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 334 kB/s | 19 kB 00:00 2024-04-24T08:49:41.551 INFO:teuthology.orchestra.run.smithi130.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 9.5 MB/s | 7.4 MB 00:00 2024-04-24T08:49:41.607 INFO:teuthology.orchestra.run.smithi081.stdout:(78/152): postgresql-server-13.14-1.el9.x86_64. 5.6 MB/s | 5.8 MB 00:01 2024-04-24T08:49:41.609 INFO:teuthology.orchestra.run.smithi130.stdout:(37/152): cephadm-19.0.0-3116.g1f81fb39.el9.noa 3.2 MB/s | 762 kB 00:00 2024-04-24T08:49:41.626 INFO:teuthology.orchestra.run.smithi130.stdout:(38/152): cephfs-top-19.0.0-3116.g1f81fb39.el9. 245 kB/s | 18 kB 00:00 2024-04-24T08:49:41.640 INFO:teuthology.orchestra.run.smithi081.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-04-24T08:49:41.666 INFO:teuthology.orchestra.run.smithi081.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 606 kB/s | 35 kB 00:00 2024-04-24T08:49:41.724 INFO:teuthology.orchestra.run.smithi081.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 250 kB 00:00 2024-04-24T08:49:41.735 INFO:teuthology.orchestra.run.smithi130.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 646 kB/s | 80 kB 00:00 2024-04-24T08:49:41.791 INFO:teuthology.orchestra.run.smithi081.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-24T08:49:41.811 INFO:teuthology.orchestra.run.smithi130.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-04-24T08:49:41.850 INFO:teuthology.orchestra.run.smithi081.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-24T08:49:41.861 INFO:teuthology.orchestra.run.smithi130.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-04-24T08:49:41.895 INFO:teuthology.orchestra.run.smithi130.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-04-24T08:49:41.908 INFO:teuthology.orchestra.run.smithi081.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 735 kB/s | 42 kB 00:00 2024-04-24T08:49:41.961 INFO:teuthology.orchestra.run.smithi130.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-04-24T08:49:42.012 INFO:teuthology.orchestra.run.smithi130.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 917 kB/s | 106 kB 00:00 2024-04-24T08:49:42.025 INFO:teuthology.orchestra.run.smithi081.stdout:(85/152): python3-babel-2.9.1-2.el9.noarch.rpm 5.4 MB/s | 6.0 MB 00:01 2024-04-24T08:49:42.039 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-04-24T08:49:42.045 INFO:teuthology.orchestra.run.smithi130.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-24T08:49:42.061 INFO:teuthology.orchestra.run.smithi081.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-04-24T08:49:42.071 INFO:teuthology.orchestra.run.smithi130.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-24T08:49:42.094 INFO:teuthology.orchestra.run.smithi081.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-24T08:49:42.121 INFO:teuthology.orchestra.run.smithi130.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-24T08:49:42.128 INFO:teuthology.orchestra.run.smithi081.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-24T08:49:42.187 INFO:teuthology.orchestra.run.smithi081.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 933 kB/s | 54 kB 00:00 2024-04-24T08:49:42.237 INFO:teuthology.orchestra.run.smithi081.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 360 kB/s | 51 kB 00:00 2024-04-24T08:49:42.338 INFO:teuthology.orchestra.run.smithi130.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-04-24T08:49:42.371 INFO:teuthology.orchestra.run.smithi081.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.0 MB/s | 5.1 MB 00:00 2024-04-24T08:49:42.396 INFO:teuthology.orchestra.run.smithi081.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 264 kB/s | 42 kB 00:00 2024-04-24T08:49:42.415 INFO:teuthology.orchestra.run.smithi130.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-04-24T08:49:42.454 INFO:teuthology.orchestra.run.smithi081.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 774 kB/s | 44 kB 00:00 2024-04-24T08:49:42.472 INFO:teuthology.orchestra.run.smithi130.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-04-24T08:49:42.488 INFO:teuthology.orchestra.run.smithi081.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-04-24T08:49:42.497 INFO:teuthology.orchestra.run.smithi130.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 267 kB/s | 21 kB 00:00 2024-04-24T08:49:42.513 INFO:teuthology.orchestra.run.smithi081.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 621 kB/s | 36 kB 00:00 2024-04-24T08:49:42.539 INFO:teuthology.orchestra.run.smithi130.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-04-24T08:49:42.547 INFO:teuthology.orchestra.run.smithi081.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 822 kB/s | 47 kB 00:00 2024-04-24T08:49:42.564 INFO:teuthology.orchestra.run.smithi130.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 494 kB/s | 33 kB 00:00 2024-04-24T08:49:42.606 INFO:teuthology.orchestra.run.smithi081.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 997 kB/s | 58 kB 00:00 2024-04-24T08:49:42.623 INFO:teuthology.orchestra.run.smithi130.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 316 kB/s | 18 kB 00:00 2024-04-24T08:49:42.639 INFO:teuthology.orchestra.run.smithi081.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.4 MB/s | 305 kB 00:00 2024-04-24T08:49:42.664 INFO:teuthology.orchestra.run.smithi081.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-24T08:49:42.699 INFO:teuthology.orchestra.run.smithi081.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-24T08:49:42.723 INFO:teuthology.orchestra.run.smithi081.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 776 kB/s | 45 kB 00:00 2024-04-24T08:49:42.782 INFO:teuthology.orchestra.run.smithi130.stdout:(55/152): ceph-radosgw-19.0.0-3116.g1f81fb39.el 1.4 MB/s | 11 MB 00:07 2024-04-24T08:49:42.807 INFO:teuthology.orchestra.run.smithi130.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 61 kB/s | 11 kB 00:00 2024-04-24T08:49:42.857 INFO:teuthology.orchestra.run.smithi130.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-24T08:49:42.865 INFO:teuthology.orchestra.run.smithi081.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-04-24T08:49:42.941 INFO:teuthology.orchestra.run.smithi130.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.5 MB/s | 3.0 MB 00:00 2024-04-24T08:49:42.974 INFO:teuthology.orchestra.run.smithi130.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 272 kB/s | 45 kB 00:00 2024-04-24T08:49:43.016 INFO:teuthology.orchestra.run.smithi130.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-04-24T08:49:43.042 INFO:teuthology.orchestra.run.smithi130.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 389 kB/s | 25 kB 00:00 2024-04-24T08:49:43.083 INFO:teuthology.orchestra.run.smithi130.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-04-24T08:49:43.109 INFO:teuthology.orchestra.run.smithi130.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 519 kB/s | 47 kB 00:00 2024-04-24T08:49:43.142 INFO:teuthology.orchestra.run.smithi130.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-24T08:49:43.168 INFO:teuthology.orchestra.run.smithi130.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 609 kB/s | 35 kB 00:00 2024-04-24T08:49:43.182 INFO:teuthology.orchestra.run.smithi081.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-04-24T08:49:43.201 INFO:teuthology.orchestra.run.smithi130.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2024-04-24T08:49:43.249 INFO:teuthology.orchestra.run.smithi081.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 4.6 MB/s | 309 kB 00:00 2024-04-24T08:49:43.300 INFO:teuthology.orchestra.run.smithi081.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 394 kB/s | 19 kB 00:00 2024-04-24T08:49:43.383 INFO:teuthology.orchestra.run.smithi081.stdout:(106/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-04-24T08:49:43.394 INFO:teuthology.orchestra.run.smithi130.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-04-24T08:49:43.544 INFO:teuthology.orchestra.run.smithi130.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-04-24T08:49:43.584 INFO:teuthology.orchestra.run.smithi081.stdout:(107/152): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-24T08:49:43.602 INFO:teuthology.orchestra.run.smithi130.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 4.3 MB/s | 911 kB 00:00 2024-04-24T08:49:43.609 INFO:teuthology.orchestra.run.smithi081.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 116 kB/s | 26 kB 00:00 2024-04-24T08:49:43.634 INFO:teuthology.orchestra.run.smithi081.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 982 kB/s | 49 kB 00:00 2024-04-24T08:49:43.636 INFO:teuthology.orchestra.run.smithi130.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-24T08:49:43.693 INFO:teuthology.orchestra.run.smithi081.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-04-24T08:49:43.718 INFO:teuthology.orchestra.run.smithi081.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 623 kB/s | 67 kB 00:00 2024-04-24T08:49:43.736 INFO:teuthology.orchestra.run.smithi130.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.8 MB/s | 4.4 MB 00:00 2024-04-24T08:49:43.777 INFO:teuthology.orchestra.run.smithi081.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.9 MB/s | 838 kB 00:00 2024-04-24T08:49:43.803 INFO:teuthology.orchestra.run.smithi130.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-04-24T08:49:43.829 INFO:teuthology.orchestra.run.smithi130.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 963 kB/s | 87 kB 00:00 2024-04-24T08:49:43.836 INFO:teuthology.orchestra.run.smithi081.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 754 kB/s | 43 kB 00:00 2024-04-24T08:49:43.877 INFO:teuthology.orchestra.run.smithi081.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 790 kB/s | 32 kB 00:00 2024-04-24T08:49:43.895 INFO:teuthology.orchestra.run.smithi130.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-24T08:49:43.938 INFO:teuthology.orchestra.run.smithi081.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-04-24T08:49:43.962 INFO:teuthology.orchestra.run.smithi081.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 181 kB/s | 15 kB 00:00 2024-04-24T08:49:43.962 INFO:teuthology.orchestra.run.smithi130.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-04-24T08:49:43.995 INFO:teuthology.orchestra.run.smithi081.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.0 MB/s | 172 kB 00:00 2024-04-24T08:49:44.037 INFO:teuthology.orchestra.run.smithi081.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-24T08:49:44.071 INFO:teuthology.orchestra.run.smithi081.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-04-24T08:49:44.096 INFO:teuthology.orchestra.run.smithi081.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 606 kB/s | 35 kB 00:00 2024-04-24T08:49:44.130 INFO:teuthology.orchestra.run.smithi130.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-04-24T08:49:44.130 INFO:teuthology.orchestra.run.smithi081.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.5 MB/s | 206 kB 00:00 2024-04-24T08:49:44.155 INFO:teuthology.orchestra.run.smithi130.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 134 kB/s | 26 kB 00:00 2024-04-24T08:49:44.188 INFO:teuthology.orchestra.run.smithi130.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 332 kB/s | 19 kB 00:00 2024-04-24T08:49:44.222 INFO:teuthology.orchestra.run.smithi130.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.6 MB/s | 172 kB 00:00 2024-04-24T08:49:44.247 INFO:teuthology.orchestra.run.smithi130.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 605 kB/s | 35 kB 00:00 2024-04-24T08:49:44.364 INFO:teuthology.orchestra.run.smithi081.stdout:(122/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.5 MB/s | 16 MB 00:02 2024-04-24T08:49:44.381 INFO:teuthology.orchestra.run.smithi130.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-04-24T08:49:44.397 INFO:teuthology.orchestra.run.smithi081.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 544 kB/s | 144 kB 00:00 2024-04-24T08:49:44.415 INFO:teuthology.orchestra.run.smithi130.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-04-24T08:49:44.448 INFO:teuthology.orchestra.run.smithi130.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-24T08:49:44.473 INFO:teuthology.orchestra.run.smithi130.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-24T08:49:44.525 INFO:teuthology.orchestra.run.smithi130.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 560 kB/s | 42 kB 00:00 2024-04-24T08:49:44.539 INFO:teuthology.orchestra.run.smithi081.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 4.6 MB/s | 2.0 MB 00:00 2024-04-24T08:49:44.564 INFO:teuthology.orchestra.run.smithi081.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 182 kB/s | 18 kB 00:00 2024-04-24T08:49:44.565 INFO:teuthology.orchestra.run.smithi130.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-04-24T08:49:44.590 INFO:teuthology.orchestra.run.smithi081.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 489 kB/s | 23 kB 00:00 2024-04-24T08:49:44.608 INFO:teuthology.orchestra.run.smithi130.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-24T08:49:44.615 INFO:teuthology.orchestra.run.smithi081.stdout:(127/152): python3-jaraco-8.2.1-3.el9.noarch.rp 43 kB/s | 11 kB 00:00 2024-04-24T08:49:44.640 INFO:teuthology.orchestra.run.smithi081.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-04-24T08:49:44.650 INFO:teuthology.orchestra.run.smithi130.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-04-24T08:49:44.665 INFO:teuthology.orchestra.run.smithi081.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 264 kB/s | 20 kB 00:00 2024-04-24T08:49:44.675 INFO:teuthology.orchestra.run.smithi130.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 772 kB/s | 51 kB 00:00 2024-04-24T08:49:44.691 INFO:teuthology.orchestra.run.smithi081.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 938 kB/s | 46 kB 00:00 2024-04-24T08:49:44.709 INFO:teuthology.orchestra.run.smithi130.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 930 kB/s | 54 kB 00:00 2024-04-24T08:49:44.741 INFO:teuthology.orchestra.run.smithi081.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-04-24T08:49:44.766 INFO:teuthology.orchestra.run.smithi081.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 775 kB/s | 58 kB 00:00 2024-04-24T08:49:44.776 INFO:teuthology.orchestra.run.smithi130.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 633 kB/s | 42 kB 00:00 2024-04-24T08:49:44.783 INFO:teuthology.orchestra.run.smithi081.stdout:(133/152): python3-more-itertools-8.12.0-2.el9. 678 kB/s | 79 kB 00:00 2024-04-24T08:49:44.842 INFO:teuthology.orchestra.run.smithi081.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-24T08:49:44.867 INFO:teuthology.orchestra.run.smithi081.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 165 kB/s | 16 kB 00:00 2024-04-24T08:49:44.876 INFO:teuthology.orchestra.run.smithi130.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.8 MB/s | 5.1 MB 00:00 2024-04-24T08:49:44.892 INFO:teuthology.orchestra.run.smithi081.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 569 kB/s | 28 kB 00:00 2024-04-24T08:49:44.901 INFO:teuthology.orchestra.run.smithi130.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 355 kB/s | 44 kB 00:00 2024-04-24T08:49:44.917 INFO:teuthology.orchestra.run.smithi081.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 674 kB/s | 90 kB 00:00 2024-04-24T08:49:44.943 INFO:teuthology.orchestra.run.smithi081.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 413 kB/s | 31 kB 00:00 2024-04-24T08:49:44.960 INFO:teuthology.orchestra.run.smithi130.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-04-24T08:49:44.976 INFO:teuthology.orchestra.run.smithi081.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-24T08:49:44.985 INFO:teuthology.orchestra.run.smithi130.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 431 kB/s | 36 kB 00:00 2024-04-24T08:49:45.001 INFO:teuthology.orchestra.run.smithi081.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 625 kB/s | 36 kB 00:00 2024-04-24T08:49:45.019 INFO:teuthology.orchestra.run.smithi130.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 817 kB/s | 47 kB 00:00 2024-04-24T08:49:45.027 INFO:teuthology.orchestra.run.smithi081.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 714 kB/s | 59 kB 00:00 2024-04-24T08:49:45.052 INFO:teuthology.orchestra.run.smithi081.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 682 kB/s | 51 kB 00:00 2024-04-24T08:49:45.069 INFO:teuthology.orchestra.run.smithi130.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-04-24T08:49:45.086 INFO:teuthology.orchestra.run.smithi081.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-24T08:49:45.095 INFO:teuthology.orchestra.run.smithi130.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 781 kB/s | 58 kB 00:00 2024-04-24T08:49:45.103 INFO:teuthology.orchestra.run.smithi081.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-24T08:49:45.128 INFO:teuthology.orchestra.run.smithi130.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-24T08:49:45.136 INFO:teuthology.orchestra.run.smithi081.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-24T08:49:45.154 INFO:teuthology.orchestra.run.smithi130.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-24T08:49:45.162 INFO:teuthology.orchestra.run.smithi081.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 268 kB/s | 20 kB 00:00 2024-04-24T08:49:45.187 INFO:teuthology.orchestra.run.smithi130.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 774 kB/s | 45 kB 00:00 2024-04-24T08:49:45.195 INFO:teuthology.orchestra.run.smithi081.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-24T08:49:45.246 INFO:teuthology.orchestra.run.smithi130.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 245 kB/s | 22 kB 00:00 2024-04-24T08:49:45.271 INFO:teuthology.orchestra.run.smithi081.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-24T08:49:45.514 INFO:teuthology.orchestra.run.smithi130.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-04-24T08:49:45.721 INFO:teuthology.orchestra.run.smithi130.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-04-24T08:49:45.747 INFO:teuthology.orchestra.run.smithi130.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-04-24T08:49:45.792 INFO:teuthology.orchestra.run.smithi130.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 281 kB/s | 19 kB 00:00 2024-04-24T08:49:45.847 INFO:teuthology.orchestra.run.smithi130.stdout:(107/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-04-24T08:49:46.054 INFO:teuthology.orchestra.run.smithi081.stdout:(149/152): librbd1-19.0.0-3116.g1f81fb39.el9.x8 3.7 MB/s | 3.1 MB 00:00 2024-04-24T08:49:46.100 INFO:teuthology.orchestra.run.smithi130.stdout:(108/152): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-24T08:49:46.123 INFO:teuthology.orchestra.run.smithi130.stdout:(109/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 81 kB/s | 26 kB 00:00 2024-04-24T08:49:46.148 INFO:teuthology.orchestra.run.smithi130.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 170 kB/s | 49 kB 00:00 2024-04-24T08:49:46.220 INFO:teuthology.orchestra.run.smithi130.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 958 kB/s | 67 kB 00:00 2024-04-24T08:49:46.257 INFO:teuthology.orchestra.run.smithi130.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-24T08:49:46.290 INFO:teuthology.orchestra.run.smithi130.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.8 MB/s | 838 kB 00:00 2024-04-24T08:49:46.332 INFO:teuthology.orchestra.run.smithi130.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 4.8 MB/s | 548 kB 00:00 2024-04-24T08:49:46.357 INFO:teuthology.orchestra.run.smithi130.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 711 kB/s | 43 kB 00:00 2024-04-24T08:49:46.383 INFO:teuthology.orchestra.run.smithi130.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 374 kB/s | 32 kB 00:00 2024-04-24T08:49:46.408 INFO:teuthology.orchestra.run.smithi130.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 202 kB/s | 15 kB 00:00 2024-04-24T08:49:46.442 INFO:teuthology.orchestra.run.smithi130.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-24T08:49:46.467 INFO:teuthology.orchestra.run.smithi130.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.3 MB/s | 358 kB 00:00 2024-04-24T08:49:46.500 INFO:teuthology.orchestra.run.smithi130.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-24T08:49:46.514 INFO:teuthology.orchestra.run.smithi081.stdout:(150/152): librados2-19.0.0-3116.g1f81fb39.el9. 2.5 MB/s | 3.4 MB 00:01 2024-04-24T08:49:46.526 INFO:teuthology.orchestra.run.smithi130.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 641 kB/s | 35 kB 00:00 2024-04-24T08:49:46.551 INFO:teuthology.orchestra.run.smithi130.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.5 MB/s | 206 kB 00:00 2024-04-24T08:49:46.656 INFO:teuthology.orchestra.run.smithi130.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-04-24T08:49:46.693 INFO:teuthology.orchestra.run.smithi130.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-24T08:49:46.718 INFO:teuthology.orchestra.run.smithi130.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 70 kB/s | 11 kB 00:00 2024-04-24T08:49:46.752 INFO:teuthology.orchestra.run.smithi130.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 188 kB/s | 18 kB 00:00 2024-04-24T08:49:46.777 INFO:teuthology.orchestra.run.smithi130.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 402 kB/s | 23 kB 00:00 2024-04-24T08:49:46.803 INFO:teuthology.orchestra.run.smithi130.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 235 kB/s | 19 kB 00:00 2024-04-24T08:49:46.828 INFO:teuthology.orchestra.run.smithi130.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 265 kB/s | 20 kB 00:00 2024-04-24T08:49:46.886 INFO:teuthology.orchestra.run.smithi130.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-24T08:49:46.903 INFO:teuthology.orchestra.run.smithi130.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 469 kB/s | 46 kB 00:00 2024-04-24T08:49:46.937 INFO:teuthology.orchestra.run.smithi130.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 830 kB/s | 79 kB 00:00 2024-04-24T08:49:46.962 INFO:teuthology.orchestra.run.smithi130.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 1.0 MB/s | 58 kB 00:00 2024-04-24T08:49:47.087 INFO:teuthology.orchestra.run.smithi130.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-04-24T08:49:47.171 INFO:teuthology.orchestra.run.smithi130.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 71 kB/s | 16 kB 00:00 2024-04-24T08:49:47.197 INFO:teuthology.orchestra.run.smithi130.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 836 kB/s | 90 kB 00:00 2024-04-24T08:49:47.213 INFO:teuthology.orchestra.run.smithi130.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 227 kB/s | 28 kB 00:00 2024-04-24T08:49:47.230 INFO:teuthology.orchestra.run.smithi130.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 540 kB/s | 31 kB 00:00 2024-04-24T08:49:47.255 INFO:teuthology.orchestra.run.smithi130.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-04-24T08:49:47.289 INFO:teuthology.orchestra.run.smithi130.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-24T08:49:47.306 INFO:teuthology.orchestra.run.smithi130.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 478 kB/s | 36 kB 00:00 2024-04-24T08:49:47.331 INFO:teuthology.orchestra.run.smithi130.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 702 kB/s | 51 kB 00:00 2024-04-24T08:49:47.365 INFO:teuthology.orchestra.run.smithi130.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2024-04-24T08:49:47.390 INFO:teuthology.orchestra.run.smithi130.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-04-24T08:49:47.423 INFO:teuthology.orchestra.run.smithi130.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.7 MB/s | 427 kB 00:00 2024-04-24T08:49:47.449 INFO:teuthology.orchestra.run.smithi130.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 373 kB/s | 20 kB 00:00 2024-04-24T08:49:47.482 INFO:teuthology.orchestra.run.smithi130.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-24T08:49:47.566 INFO:teuthology.orchestra.run.smithi130.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-24T08:49:47.663 INFO:teuthology.orchestra.run.smithi081.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 578 kB/s | 1.3 MB 00:02 2024-04-24T08:49:47.758 INFO:teuthology.orchestra.run.smithi130.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 7.1 MB/s | 1.3 MB 00:00 2024-04-24T08:49:47.858 INFO:teuthology.orchestra.run.smithi130.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-04-24T08:49:47.931 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-24T08:49:48.175 INFO:teuthology.orchestra.run.smithi130.stdout:(151/152): librbd1-19.0.0-3116.g1f81fb39.el9.x8 4.6 MB/s | 3.1 MB 00:00 2024-04-24T08:49:49.058 INFO:teuthology.orchestra.run.smithi130.stdout:(152/152): librados2-19.0.0-3116.g1f81fb39.el9. 2.2 MB/s | 3.4 MB 00:01 2024-04-24T08:49:49.066 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:49:49.067 INFO:teuthology.orchestra.run.smithi130.stdout:Total 12 MB/s | 222 MB 00:19 2024-04-24T08:49:49.921 INFO:teuthology.orchestra.run.smithi081.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 634 kB/s | 2.4 MB 00:03 2024-04-24T08:49:49.934 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:49:49.934 INFO:teuthology.orchestra.run.smithi081.stdout:Total 8.4 MB/s | 222 MB 00:26 2024-04-24T08:49:50.308 INFO:teuthology.orchestra.run.smithi069.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T08:49:50.309 INFO:teuthology.orchestra.run.smithi069.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T08:49:50.330 INFO:teuthology.orchestra.run.smithi069.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T08:49:50.331 INFO:teuthology.orchestra.run.smithi069.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T08:49:50.333 INFO:teuthology.orchestra.run.smithi069.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T08:49:50.334 INFO:teuthology.orchestra.run.smithi069.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T08:49:50.337 INFO:teuthology.orchestra.run.smithi069.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T08:49:50.340 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T08:49:50.341 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T08:49:50.411 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-04-24T08:49:50.432 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-04-24T08:49:50.432 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-04-24T08:49:50.432 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-04-24T08:49:50.433 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-04-24T08:49:50.433 INFO:teuthology.orchestra.run.smithi069.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T08:49:50.433 INFO:teuthology.orchestra.run.smithi069.stdout: ceph x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 6.6 k 2024-04-24T08:49:50.433 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T08:49:50.433 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 890 k 2024-04-24T08:49:50.433 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 145 k 2024-04-24T08:49:50.433 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 1.5 M 2024-04-24T08:49:50.433 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 142 k 2024-04-24T08:49:50.433 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 3.5 M 2024-04-24T08:49:50.433 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 7.4 M 2024-04-24T08:49:50.434 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 49 k 2024-04-24T08:49:50.434 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 11 M 2024-04-24T08:49:50.434 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 49 M 2024-04-24T08:49:50.434 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 273 k 2024-04-24T08:49:50.434 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 762 k 2024-04-24T08:49:50.434 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 225 k 2024-04-24T08:49:50.434 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 27 k 2024-04-24T08:49:50.434 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 18 k 2024-04-24T08:49:50.434 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T08:49:50.434 INFO:teuthology.orchestra.run.smithi069.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T08:49:50.434 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T08:49:50.434 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 32 k 2024-04-24T08:49:50.435 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 746 k 2024-04-24T08:49:50.435 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 127 k 2024-04-24T08:49:50.435 INFO:teuthology.orchestra.run.smithi069.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T08:49:50.435 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T08:49:50.435 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T08:49:50.435 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T08:49:50.435 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T08:49:50.435 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T08:49:50.435 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 161 k 2024-04-24T08:49:50.435 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 320 k 2024-04-24T08:49:50.435 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 298 k 2024-04-24T08:49:50.435 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 99 k 2024-04-24T08:49:50.435 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 85 k 2024-04-24T08:49:50.436 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T08:49:50.436 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 170 k 2024-04-24T08:49:50.436 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T08:49:50.436 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrading: 2024-04-24T08:49:50.436 INFO:teuthology.orchestra.run.smithi069.stdout: librados2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.4 M 2024-04-24T08:49:50.436 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T08:49:50.436 INFO:teuthology.orchestra.run.smithi069.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T08:49:50.436 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T08:49:50.436 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-04-24T08:49:50.436 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T08:49:50.436 INFO:teuthology.orchestra.run.smithi069.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T08:49:50.436 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T08:49:50.436 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 21 M 2024-04-24T08:49:50.437 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 34 k 2024-04-24T08:49:50.437 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 2.4 M 2024-04-24T08:49:50.437 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 249 k 2024-04-24T08:49:50.437 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 4.7 M 2024-04-24T08:49:50.437 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 17 M 2024-04-24T08:49:50.437 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 16 k 2024-04-24T08:49:50.437 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 25 k 2024-04-24T08:49:50.437 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T08:49:50.437 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T08:49:50.437 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T08:49:50.437 INFO:teuthology.orchestra.run.smithi069.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T08:49:50.437 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T08:49:50.437 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T08:49:50.437 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T08:49:50.438 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T08:49:50.438 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 163 k 2024-04-24T08:49:50.438 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T08:49:50.438 INFO:teuthology.orchestra.run.smithi069.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T08:49:50.438 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T08:49:50.438 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T08:49:50.438 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T08:49:50.438 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 501 k 2024-04-24T08:49:50.438 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T08:49:50.438 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T08:49:50.438 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T08:49:50.438 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T08:49:50.438 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T08:49:50.438 INFO:teuthology.orchestra.run.smithi069.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T08:49:50.439 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T08:49:50.439 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T08:49:50.439 INFO:teuthology.orchestra.run.smithi069.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T08:49:50.439 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T08:49:50.439 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T08:49:50.439 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T08:49:50.439 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T08:49:50.439 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T08:49:50.439 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T08:49:50.439 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T08:49:50.439 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T08:49:50.439 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T08:49:50.439 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T08:49:50.439 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 45 k 2024-04-24T08:49:50.439 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 135 k 2024-04-24T08:49:50.440 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T08:49:50.440 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T08:49:50.440 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T08:49:50.440 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T08:49:50.440 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T08:49:50.440 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T08:49:50.440 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T08:49:50.440 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T08:49:50.440 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T08:49:50.440 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T08:49:50.440 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T08:49:50.440 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T08:49:50.440 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T08:49:50.440 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T08:49:50.441 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T08:49:50.441 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T08:49:50.441 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T08:49:50.441 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T08:49:50.441 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T08:49:50.441 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T08:49:50.441 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T08:49:50.441 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T08:49:50.441 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T08:49:50.441 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T08:49:50.441 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T08:49:50.441 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T08:49:50.441 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T08:49:50.441 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T08:49:50.441 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T08:49:50.441 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T08:49:50.442 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T08:49:50.442 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T08:49:50.442 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T08:49:50.442 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T08:49:50.442 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T08:49:50.442 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T08:49:50.442 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T08:49:50.442 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T08:49:50.442 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T08:49:50.442 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T08:49:50.442 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T08:49:50.442 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T08:49:50.442 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T08:49:50.443 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T08:49:50.443 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T08:49:50.443 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T08:49:50.443 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T08:49:50.443 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T08:49:50.443 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T08:49:50.443 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T08:49:50.443 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T08:49:50.443 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T08:49:50.443 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T08:49:50.443 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T08:49:50.443 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T08:49:50.443 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T08:49:50.444 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T08:49:50.444 INFO:teuthology.orchestra.run.smithi069.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T08:49:50.444 INFO:teuthology.orchestra.run.smithi069.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T08:49:50.444 INFO:teuthology.orchestra.run.smithi069.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T08:49:50.444 INFO:teuthology.orchestra.run.smithi069.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T08:49:50.444 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T08:49:50.444 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T08:49:50.444 INFO:teuthology.orchestra.run.smithi069.stdout:Installing weak dependencies: 2024-04-24T08:49:50.444 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T08:49:50.444 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:49:50.444 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-04-24T08:49:50.444 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-04-24T08:49:50.444 INFO:teuthology.orchestra.run.smithi069.stdout:Install 148 Packages 2024-04-24T08:49:50.444 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrade 4 Packages 2024-04-24T08:49:50.445 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:49:50.445 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 222 M 2024-04-24T08:49:50.445 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-04-24T08:49:51.324 INFO:teuthology.orchestra.run.smithi069.stdout:(1/152): ceph-19.0.0-3116.g1f81fb39.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-24T08:49:51.516 INFO:teuthology.orchestra.run.smithi069.stdout:(2/152): ceph-fuse-19.0.0-3116.g1f81fb39.el9.x8 4.6 MB/s | 890 kB 00:00 2024-04-24T08:49:51.566 INFO:teuthology.orchestra.run.smithi069.stdout:(3/152): ceph-immutable-object-cache-19.0.0-311 2.9 MB/s | 145 kB 00:00 2024-04-24T08:49:51.767 INFO:teuthology.orchestra.run.smithi069.stdout:(4/152): ceph-mds-19.0.0-3116.g1f81fb39.el9.x86 12 MB/s | 2.4 MB 00:00 2024-04-24T08:49:51.884 INFO:teuthology.orchestra.run.smithi069.stdout:(5/152): ceph-mgr-19.0.0-3116.g1f81fb39.el9.x86 13 MB/s | 1.5 MB 00:00 2024-04-24T08:49:51.916 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-24T08:49:52.155 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-24T08:49:52.155 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-24T08:49:52.284 INFO:teuthology.orchestra.run.smithi069.stdout:(6/152): ceph-mon-19.0.0-3116.g1f81fb39.el9.x86 12 MB/s | 4.7 MB 00:00 2024-04-24T08:49:52.451 INFO:teuthology.orchestra.run.smithi069.stdout:(7/152): ceph-base-19.0.0-3116.g1f81fb39.el9.x8 3.9 MB/s | 5.4 MB 00:01 2024-04-24T08:49:52.797 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-04-24T08:49:53.035 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-04-24T08:49:53.035 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-04-24T08:49:53.501 INFO:teuthology.orchestra.run.smithi069.stdout:(8/152): ceph-radosgw-19.0.0-3116.g1f81fb39.el9 10 MB/s | 11 MB 00:01 2024-04-24T08:49:53.726 INFO:teuthology.orchestra.run.smithi069.stdout:(9/152): ceph-osd-19.0.0-3116.g1f81fb39.el9.x86 12 MB/s | 17 MB 00:01 2024-04-24T08:49:53.793 INFO:teuthology.orchestra.run.smithi069.stdout:(10/152): ceph-selinux-19.0.0-3116.g1f81fb39.el 86 kB/s | 25 kB 00:00 2024-04-24T08:49:53.877 INFO:teuthology.orchestra.run.smithi069.stdout:(11/152): cephfs-mirror-19.0.0-3116.g1f81fb39.e 2.7 MB/s | 225 kB 00:00 2024-04-24T08:49:53.985 INFO:teuthology.orchestra.run.smithi069.stdout:(12/152): cephfs-shell-19.0.0-3116.g1f81fb39.el 251 kB/s | 27 kB 00:00 2024-04-24T08:49:54.069 INFO:teuthology.orchestra.run.smithi069.stdout:(13/152): libcephfs-devel-19.0.0-3116.g1f81fb39 385 kB/s | 32 kB 00:00 2024-04-24T08:49:54.219 INFO:teuthology.orchestra.run.smithi069.stdout:(14/152): libcephfs2-19.0.0-3116.g1f81fb39.el9. 4.9 MB/s | 746 kB 00:00 2024-04-24T08:49:54.303 INFO:teuthology.orchestra.run.smithi069.stdout:(15/152): libcephsqlite-19.0.0-3116.g1f81fb39.e 1.9 MB/s | 163 kB 00:00 2024-04-24T08:49:54.386 INFO:teuthology.orchestra.run.smithi069.stdout:(16/152): librados-devel-19.0.0-3116.g1f81fb39. 1.5 MB/s | 127 kB 00:00 2024-04-24T08:49:54.495 INFO:teuthology.orchestra.run.smithi069.stdout:(17/152): libradosstriper1-19.0.0-3116.g1f81fb3 4.6 MB/s | 501 kB 00:00 2024-04-24T08:49:54.785 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-24T08:49:54.786 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-24T08:49:55.089 INFO:teuthology.orchestra.run.smithi069.stdout:(18/152): librgw2-19.0.0-3116.g1f81fb39.el9.x86 9.1 MB/s | 5.4 MB 00:00 2024-04-24T08:49:55.204 INFO:teuthology.orchestra.run.smithi069.stdout:(19/152): python3-ceph-argparse-19.0.0-3116.g1f 390 kB/s | 45 kB 00:00 2024-04-24T08:49:55.305 INFO:teuthology.orchestra.run.smithi069.stdout:(20/152): python3-ceph-common-19.0.0-3116.g1f81 1.3 MB/s | 135 kB 00:00 2024-04-24T08:49:55.405 INFO:teuthology.orchestra.run.smithi069.stdout:(21/152): python3-cephfs-19.0.0-3116.g1f81fb39. 1.6 MB/s | 161 kB 00:00 2024-04-24T08:49:55.560 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-04-24T08:49:55.561 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-04-24T08:49:55.776 INFO:teuthology.orchestra.run.smithi069.stdout:(22/152): python3-rados-19.0.0-3116.g1f81fb39.e 866 kB/s | 320 kB 00:00 2024-04-24T08:49:55.880 INFO:teuthology.orchestra.run.smithi069.stdout:(23/152): python3-rbd-19.0.0-3116.g1f81fb39.el9 2.8 MB/s | 298 kB 00:00 2024-04-24T08:49:55.964 INFO:teuthology.orchestra.run.smithi069.stdout:(24/152): python3-rgw-19.0.0-3116.g1f81fb39.el9 1.2 MB/s | 99 kB 00:00 2024-04-24T08:49:56.056 INFO:teuthology.orchestra.run.smithi069.stdout:(25/152): rbd-fuse-19.0.0-3116.g1f81fb39.el9.x8 940 kB/s | 85 kB 00:00 2024-04-24T08:49:56.564 INFO:teuthology.orchestra.run.smithi069.stdout:(26/152): rbd-mirror-19.0.0-3116.g1f81fb39.el9. 6.1 MB/s | 3.1 MB 00:00 2024-04-24T08:49:56.881 INFO:teuthology.orchestra.run.smithi069.stdout:(27/152): ceph-common-19.0.0-3116.g1f81fb39.el9 3.6 MB/s | 21 MB 00:05 2024-04-24T08:49:56.915 INFO:teuthology.orchestra.run.smithi069.stdout:(28/152): rbd-nbd-19.0.0-3116.g1f81fb39.el9.x86 488 kB/s | 170 kB 00:00 2024-04-24T08:49:56.973 INFO:teuthology.orchestra.run.smithi069.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3116.g 377 kB/s | 34 kB 00:00 2024-04-24T08:49:57.007 INFO:teuthology.orchestra.run.smithi069.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3116.g1f81fb3 1.5 MB/s | 142 kB 00:00 2024-04-24T08:49:57.549 INFO:teuthology.orchestra.run.smithi069.stdout:(31/152): ceph-test-19.0.0-3116.g1f81fb39.el9.x 13 MB/s | 49 MB 00:03 2024-04-24T08:49:57.641 INFO:teuthology.orchestra.run.smithi069.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3116.g1f 2.7 MB/s | 249 kB 00:00 2024-04-24T08:49:57.683 INFO:teuthology.orchestra.run.smithi069.stdout:(33/152): ceph-mgr-rook-19.0.0-3116.g1f81fb39.e 1.2 MB/s | 49 kB 00:00 2024-04-24T08:49:57.775 INFO:teuthology.orchestra.run.smithi069.stdout:(34/152): ceph-mgr-dashboard-19.0.0-3116.g1f81f 4.4 MB/s | 3.5 MB 00:00 2024-04-24T08:49:57.861 INFO:teuthology.orchestra.run.smithi069.stdout:(35/152): ceph-volume-19.0.0-3116.g1f81fb39.el9 3.1 MB/s | 273 kB 00:00 2024-04-24T08:49:58.009 INFO:teuthology.orchestra.run.smithi069.stdout:(36/152): ceph-prometheus-alerts-19.0.0-3116.g1 51 kB/s | 16 kB 00:00 2024-04-24T08:49:58.067 INFO:teuthology.orchestra.run.smithi069.stdout:(37/152): cephadm-19.0.0-3116.g1f81fb39.el9.noa 3.6 MB/s | 762 kB 00:00 2024-04-24T08:49:58.092 INFO:teuthology.orchestra.run.smithi069.stdout:(38/152): cephfs-top-19.0.0-3116.g1f81fb39.el9. 221 kB/s | 18 kB 00:00 2024-04-24T08:49:58.368 INFO:teuthology.orchestra.run.smithi069.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 5.4 MB/s | 7.4 MB 00:01 2024-04-24T08:49:58.510 INFO:teuthology.orchestra.run.smithi069.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 182 kB/s | 80 kB 00:00 2024-04-24T08:49:58.935 INFO:teuthology.orchestra.run.smithi069.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 597 kB/s | 253 kB 00:00 2024-04-24T08:49:58.960 INFO:teuthology.orchestra.run.smithi069.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 330 kB/s | 195 kB 00:00 2024-04-24T08:49:59.010 INFO:teuthology.orchestra.run.smithi069.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 876 kB/s | 803 kB 00:00 2024-04-24T08:49:59.077 INFO:teuthology.orchestra.run.smithi069.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 917 kB/s | 106 kB 00:00 2024-04-24T08:49:59.119 INFO:teuthology.orchestra.run.smithi069.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-04-24T08:49:59.178 INFO:teuthology.orchestra.run.smithi069.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-24T08:49:59.228 INFO:teuthology.orchestra.run.smithi069.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-24T08:49:59.295 INFO:teuthology.orchestra.run.smithi069.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 3.4 MB/s | 1.2 MB 00:00 2024-04-24T08:50:00.171 INFO:teuthology.orchestra.run.smithi069.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 122 kB/s | 107 kB 00:00 2024-04-24T08:50:00.754 INFO:teuthology.orchestra.run.smithi069.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 437 kB/s | 689 kB 00:01 2024-04-24T08:50:00.821 INFO:teuthology.orchestra.run.smithi069.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 33 kB/s | 21 kB 00:00 2024-04-24T08:50:01.138 INFO:teuthology.orchestra.run.smithi069.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 494 kB/s | 943 kB 00:01 2024-04-24T08:50:01.488 INFO:teuthology.orchestra.run.smithi069.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 49 kB/s | 33 kB 00:00 2024-04-24T08:50:01.549 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-24T08:50:01.660 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T08:50:01.749 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T08:50:01.880 INFO:teuthology.orchestra.run.smithi069.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 279 kB/s | 313 kB 00:01 2024-04-24T08:50:01.901 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T08:50:01.957 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T08:50:01.994 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T08:50:02.029 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-04-24T08:50:02.111 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T08:50:02.129 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T08:50:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T08:50:02.319 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T08:50:02.373 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T08:50:02.381 INFO:teuthology.orchestra.run.smithi069.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 20 kB/s | 18 kB 00:00 2024-04-24T08:50:02.405 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T08:50:02.535 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T08:50:02.553 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T08:50:02.692 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:50:02.860 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:50:02.893 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:50:02.914 INFO:teuthology.orchestra.run.smithi069.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 11 kB/s | 11 kB 00:01 2024-04-24T08:50:02.995 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T08:50:02.996 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:50:03.033 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T08:50:03.095 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:50:03.269 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:50:03.296 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:50:03.401 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T08:50:03.418 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:50:03.462 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T08:50:03.464 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T08:50:03.489 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:50:03.655 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:50:03.681 INFO:teuthology.orchestra.run.smithi069.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 123 kB/s | 160 kB 00:01 2024-04-24T08:50:03.681 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T08:50:03.767 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T08:50:03.773 INFO:teuthology.orchestra.run.smithi069.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 53 kB/s | 45 kB 00:00 2024-04-24T08:50:03.796 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T08:50:03.799 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T08:50:03.868 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T08:50:03.881 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T08:50:03.892 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:50:04.057 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:50:04.085 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T08:50:04.188 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T08:50:04.219 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T08:50:04.222 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T08:50:04.275 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T08:50:04.319 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T08:50:04.472 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T08:50:04.594 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T08:50:04.599 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T08:50:04.641 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T08:50:04.642 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T08:50:04.691 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T08:50:04.796 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T08:50:04.823 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T08:50:04.854 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T08:50:04.917 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T08:50:04.945 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T08:50:04.967 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T08:50:05.003 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T08:50:05.007 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T08:50:05.054 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T08:50:05.126 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T08:50:05.156 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T08:50:05.197 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T08:50:05.232 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T08:50:05.242 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T08:50:05.297 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T08:50:05.302 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T08:50:05.315 INFO:teuthology.orchestra.run.smithi069.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 406 kB/s | 662 kB 00:01 2024-04-24T08:50:05.346 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T08:50:05.394 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T08:50:05.406 INFO:teuthology.orchestra.run.smithi069.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 357 kB/s | 582 kB 00:01 2024-04-24T08:50:05.409 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T08:50:05.445 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 31/156 2024-04-24T08:50:05.491 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 32/156 2024-04-24T08:50:05.513 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T08:50:05.527 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 33/156 2024-04-24T08:50:05.590 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T08:50:05.629 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T08:50:05.641 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T08:50:05.697 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T08:50:05.707 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T08:50:05.804 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T08:50:05.815 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T08:50:05.840 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 31/156 2024-04-24T08:50:05.857 INFO:teuthology.orchestra.run.smithi069.stdout:(61/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 649 kB/s | 3.0 MB 00:04 2024-04-24T08:50:05.891 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 32/156 2024-04-24T08:50:05.922 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 33/156 2024-04-24T08:50:05.996 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T08:50:06.037 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T08:50:06.076 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T08:50:06.100 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T08:50:06.199 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T08:50:06.374 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T08:50:06.437 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T08:50:06.599 INFO:teuthology.orchestra.run.smithi069.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 20 kB/s | 25 kB 00:01 2024-04-24T08:50:06.682 INFO:teuthology.orchestra.run.smithi069.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 37 kB/s | 47 kB 00:01 2024-04-24T08:50:06.784 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T08:50:06.910 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T08:50:06.951 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T08:50:07.014 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T08:50:07.064 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 43/156 2024-04-24T08:50:07.107 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T08:50:07.136 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T08:50:07.209 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T08:50:07.245 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T08:50:07.290 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T08:50:07.291 INFO:teuthology.orchestra.run.smithi069.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 204 kB/s | 292 kB 00:01 2024-04-24T08:50:07.331 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 43/156 2024-04-24T08:50:07.397 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T08:50:07.412 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T08:50:07.472 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T08:50:07.552 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T08:50:07.647 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T08:50:07.660 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T08:50:07.714 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T08:50:07.746 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T08:50:07.784 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T08:50:07.789 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T08:50:07.870 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T08:50:07.874 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T08:50:07.911 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T08:50:07.915 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T08:50:07.953 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T08:50:07.977 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T08:50:08.019 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T08:50:08.049 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T08:50:08.061 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T08:50:08.123 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T08:50:08.181 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T08:50:08.256 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T08:50:08.274 INFO:teuthology.orchestra.run.smithi069.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 22 kB/s | 35 kB 00:01 2024-04-24T08:50:08.316 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T08:50:08.353 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T08:50:08.376 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T08:50:08.391 INFO:teuthology.orchestra.run.smithi069.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 105 kB/s | 188 kB 00:01 2024-04-24T08:50:08.410 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T08:50:08.435 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T08:50:08.472 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T08:50:08.520 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T08:50:08.549 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T08:50:08.596 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T08:50:08.645 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T08:50:08.647 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T08:50:08.677 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T08:50:08.689 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T08:50:08.710 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T08:50:08.738 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T08:50:08.771 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T08:50:08.803 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T08:50:08.909 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T08:50:09.001 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T08:50:09.137 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T08:50:09.172 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T08:50:09.192 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T08:50:09.227 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T08:50:09.229 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T08:50:09.265 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T08:50:09.295 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T08:50:09.310 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T08:50:09.362 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T08:50:09.363 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T08:50:09.420 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T08:50:09.424 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T08:50:09.463 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T08:50:09.473 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T08:50:09.508 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T08:50:09.526 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T08:50:09.558 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T08:50:09.577 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T08:50:09.633 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T08:50:09.635 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T08:50:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T08:50:09.688 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T08:50:09.829 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T08:50:09.829 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T08:50:09.900 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T08:50:09.900 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T08:50:09.948 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T08:50:09.949 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T08:50:10.018 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T08:50:10.049 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T08:50:10.165 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T08:50:10.202 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T08:50:10.245 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T08:50:10.270 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T08:50:11.216 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T08:50:11.218 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T08:50:11.276 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T08:50:11.278 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T08:50:11.309 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T08:50:11.311 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T08:50:11.429 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T08:50:11.434 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T08:50:11.497 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T08:50:11.505 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T08:50:11.591 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T08:50:11.605 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T08:50:11.653 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T08:50:11.661 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T08:50:11.879 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T08:50:11.888 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T08:50:11.910 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T08:50:11.925 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T08:50:12.018 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T08:50:12.043 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T08:50:12.812 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T08:50:12.871 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T08:50:12.956 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T08:50:13.023 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T08:50:13.158 INFO:teuthology.orchestra.run.smithi069.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 341 kB/s | 1.6 MB 00:04 2024-04-24T08:50:14.517 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T08:50:14.609 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T08:50:14.654 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T08:50:14.673 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T08:50:14.702 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T08:50:14.757 INFO:teuthology.orchestra.run.smithi069.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 650 kB/s | 4.7 MB 00:07 2024-04-24T08:50:14.799 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T08:50:14.865 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T08:50:14.914 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T08:50:15.065 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T08:50:15.113 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T08:50:15.219 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T08:50:15.253 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 100/156 2024-04-24T08:50:15.302 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T08:50:15.310 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T08:50:15.337 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T08:50:15.458 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T08:50:15.493 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 100/156 2024-04-24T08:50:15.548 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T08:50:15.860 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T08:50:15.891 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T08:50:15.982 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T08:50:16.009 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 104/156 2024-04-24T08:50:16.089 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T08:50:16.132 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T08:50:16.223 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T08:50:16.250 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 104/156 2024-04-24T08:50:16.466 INFO:teuthology.orchestra.run.smithi069.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 553 kB/s | 4.4 MB 00:08 2024-04-24T08:50:16.608 INFO:teuthology.orchestra.run.smithi069.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 72 kB/s | 133 kB 00:01 2024-04-24T08:50:17.166 INFO:teuthology.orchestra.run.smithi069.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 227 kB/s | 911 kB 00:04 2024-04-24T08:50:18.308 INFO:teuthology.orchestra.run.smithi069.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 77 kB/s | 87 kB 00:01 2024-04-24T08:50:18.958 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:50:18.998 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:50:19.241 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:50:19.250 INFO:teuthology.orchestra.run.smithi069.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 391 kB/s | 1.0 MB 00:02 2024-04-24T08:50:19.281 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:50:19.642 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:50:19.693 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T08:50:19.861 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T08:50:19.861 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T08:50:19.861 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T08:50:19.861 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:50:19.902 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T08:50:19.937 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:50:19.984 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T08:50:20.147 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T08:50:20.148 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T08:50:20.148 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T08:50:20.148 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:50:20.185 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T08:50:22.475 INFO:teuthology.orchestra.run.smithi069.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 76 kB/s | 244 kB 00:03 2024-04-24T08:50:25.383 INFO:teuthology.orchestra.run.smithi069.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 664 kB/s | 5.8 MB 00:08 2024-04-24T08:50:25.425 INFO:teuthology.orchestra.run.smithi069.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 84 kB/s | 248 kB 00:02 2024-04-24T08:50:26.600 INFO:teuthology.orchestra.run.smithi069.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 736 kB/s | 6.0 MB 00:08 2024-04-24T08:50:27.217 INFO:teuthology.orchestra.run.smithi069.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 11 kB/s | 19 kB 00:01 2024-04-24T08:50:27.259 INFO:teuthology.orchestra.run.smithi069.stdout:(79/152): python3-jsonpatch-1.21-16.el9.noarch. 14 kB/s | 26 kB 00:01 2024-04-24T08:50:28.184 INFO:teuthology.orchestra.run.smithi069.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 109 kB/s | 172 kB 00:01 2024-04-24T08:50:28.426 INFO:teuthology.orchestra.run.smithi069.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 29 kB/s | 35 kB 00:01 2024-04-24T08:50:29.718 INFO:teuthology.orchestra.run.smithi069.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 164 kB/s | 250 kB 00:01 2024-04-24T08:50:29.854 INFO:teuthology.orchestra.run.smithi069.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 156 kB/s | 222 kB 00:01 2024-04-24T08:50:30.543 INFO:teuthology.orchestra.run.smithi069.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 94 kB/s | 77 kB 00:00 2024-04-24T08:50:30.568 INFO:teuthology.orchestra.run.smithi069.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 59 kB/s | 42 kB 00:00 2024-04-24T08:50:31.469 INFO:teuthology.orchestra.run.smithi069.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 177 kB/s | 159 kB 00:00 2024-04-24T08:50:31.586 INFO:teuthology.orchestra.run.smithi069.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 259 kB/s | 269 kB 00:01 2024-04-24T08:50:32.336 INFO:teuthology.orchestra.run.smithi069.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 68 kB/s | 51 kB 00:00 2024-04-24T08:50:32.494 INFO:teuthology.orchestra.run.smithi069.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 272 kB/s | 279 kB 00:01 2024-04-24T08:50:32.574 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T08:50:32.574 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /sys 2024-04-24T08:50:32.574 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /proc 2024-04-24T08:50:32.574 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /mnt 2024-04-24T08:50:32.574 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /var/tmp 2024-04-24T08:50:32.574 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /home 2024-04-24T08:50:32.574 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /root 2024-04-24T08:50:32.574 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /tmp 2024-04-24T08:50:32.575 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:50:32.793 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T08:50:32.843 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T08:50:32.843 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:32.843 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T08:50:32.843 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:50:33.120 INFO:teuthology.orchestra.run.smithi069.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 69 kB/s | 54 kB 00:00 2024-04-24T08:50:33.294 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T08:50:33.294 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /sys 2024-04-24T08:50:33.294 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /proc 2024-04-24T08:50:33.294 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /mnt 2024-04-24T08:50:33.294 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /var/tmp 2024-04-24T08:50:33.294 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /home 2024-04-24T08:50:33.294 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /root 2024-04-24T08:50:33.294 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /tmp 2024-04-24T08:50:33.294 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:50:33.517 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T08:50:33.564 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T08:50:33.564 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:33.564 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T08:50:33.564 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:50:33.838 INFO:teuthology.orchestra.run.smithi069.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 58 kB/s | 42 kB 00:00 2024-04-24T08:50:34.445 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T08:50:34.521 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T08:50:34.521 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:34.521 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T08:50:34.521 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T08:50:34.521 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T08:50:34.522 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:50:34.537 INFO:teuthology.orchestra.run.smithi069.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 63 kB/s | 44 kB 00:00 2024-04-24T08:50:34.787 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T08:50:34.853 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T08:50:34.853 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:34.853 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T08:50:34.853 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T08:50:34.853 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T08:50:34.853 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:50:34.946 INFO:teuthology.orchestra.run.smithi069.stdout:(93/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 679 kB/s | 5.1 MB 00:07 2024-04-24T08:50:35.204 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T08:50:35.283 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T08:50:35.284 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:35.284 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T08:50:35.284 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T08:50:35.284 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T08:50:35.284 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:50:35.337 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T08:50:35.403 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T08:50:35.403 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:35.403 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T08:50:35.403 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T08:50:35.403 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T08:50:35.403 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:50:35.537 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T08:50:35.545 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T08:50:35.546 INFO:teuthology.orchestra.run.smithi069.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 219 kB/s | 221 kB 00:01 2024-04-24T08:50:35.601 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T08:50:35.614 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T08:50:35.614 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:35.614 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T08:50:35.614 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T08:50:35.614 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T08:50:35.614 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:50:35.638 INFO:teuthology.orchestra.run.smithi069.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 52 kB/s | 36 kB 00:00 2024-04-24T08:50:35.674 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T08:50:35.906 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T08:50:35.967 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T08:50:36.052 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T08:50:36.137 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T08:50:36.161 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T08:50:36.209 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T08:50:36.209 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:36.209 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T08:50:36.209 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T08:50:36.209 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T08:50:36.209 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:50:36.272 INFO:teuthology.orchestra.run.smithi069.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 65 kB/s | 47 kB 00:00 2024-04-24T08:50:36.333 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T08:50:36.347 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T08:50:36.396 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T08:50:36.409 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T08:50:36.441 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T08:50:36.483 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T08:50:36.499 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T08:50:36.744 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T08:50:36.801 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T08:50:36.847 INFO:teuthology.orchestra.run.smithi069.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 252 kB/s | 305 kB 00:01 2024-04-24T08:50:36.887 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T08:50:36.997 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T08:50:37.013 INFO:teuthology.orchestra.run.smithi069.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 78 kB/s | 58 kB 00:00 2024-04-24T08:50:37.099 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T08:50:37.162 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T08:50:37.175 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T08:50:37.228 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T08:50:37.245 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 124/156 2024-04-24T08:50:37.266 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T08:50:37.299 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T08:50:37.356 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 125/156 2024-04-24T08:50:37.412 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T08:50:37.748 INFO:teuthology.orchestra.run.smithi069.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 104 kB/s | 94 kB 00:00 2024-04-24T08:50:37.847 INFO:teuthology.orchestra.run.smithi069.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 76 kB/s | 64 kB 00:00 2024-04-24T08:50:38.000 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T08:50:38.036 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T08:50:38.089 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 124/156 2024-04-24T08:50:38.148 INFO:teuthology.orchestra.run.smithi069.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 75 kB/s | 22 kB 00:00 2024-04-24T08:50:38.200 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 125/156 2024-04-24T08:50:38.246 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T08:50:38.515 INFO:teuthology.orchestra.run.smithi069.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 58 kB/s | 45 kB 00:00 2024-04-24T08:50:38.702 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T08:50:38.764 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T08:50:38.824 INFO:teuthology.orchestra.run.smithi069.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-04-24T08:50:39.257 INFO:teuthology.orchestra.run.smithi069.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 740 kB/s | 548 kB 00:00 2024-04-24T08:50:39.374 INFO:teuthology.orchestra.run.smithi069.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 168 kB/s | 19 kB 00:00 2024-04-24T08:50:39.482 INFO:teuthology.orchestra.run.smithi069.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 469 kB/s | 309 kB 00:00 2024-04-24T08:50:39.517 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T08:50:39.565 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T08:50:39.583 INFO:teuthology.orchestra.run.smithi069.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 261 kB/s | 26 kB 00:00 2024-04-24T08:50:39.700 INFO:teuthology.orchestra.run.smithi069.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 419 kB/s | 49 kB 00:00 2024-04-24T08:50:39.833 INFO:teuthology.orchestra.run.smithi069.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-04-24T08:50:39.858 INFO:teuthology.orchestra.run.smithi069.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 428 kB/s | 67 kB 00:00 2024-04-24T08:50:39.860 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T08:50:39.915 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T08:50:39.967 INFO:teuthology.orchestra.run.smithi069.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-04-24T08:50:40.063 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T08:50:40.126 INFO:teuthology.orchestra.run.smithi069.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-04-24T08:50:40.159 INFO:teuthology.orchestra.run.smithi069.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 838 kB 00:00 2024-04-24T08:50:40.255 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 129/156 2024-04-24T08:50:40.285 INFO:teuthology.orchestra.run.smithi069.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 275 kB/s | 43 kB 00:00 2024-04-24T08:50:40.300 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T08:50:40.310 INFO:teuthology.orchestra.run.smithi069.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 215 kB/s | 32 kB 00:00 2024-04-24T08:50:40.367 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T08:50:40.367 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:40.368 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T08:50:40.368 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T08:50:40.368 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T08:50:40.368 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:50:40.402 INFO:teuthology.orchestra.run.smithi069.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 129 kB/s | 15 kB 00:00 2024-04-24T08:50:40.443 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T08:50:40.452 INFO:teuthology.orchestra.run.smithi069.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-04-24T08:50:40.480 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T08:50:40.536 INFO:teuthology.orchestra.run.smithi069.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-04-24T08:50:40.586 INFO:teuthology.orchestra.run.smithi069.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-24T08:50:40.645 INFO:teuthology.orchestra.run.smithi069.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 327 kB/s | 35 kB 00:00 2024-04-24T08:50:40.652 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 132/156 2024-04-24T08:50:40.673 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T08:50:40.711 INFO:teuthology.orchestra.run.smithi069.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-04-24T08:50:40.725 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T08:50:40.750 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 133/156 2024-04-24T08:50:40.845 INFO:teuthology.orchestra.run.smithi069.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-04-24T08:50:40.876 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T08:50:40.879 INFO:teuthology.orchestra.run.smithi069.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 869 kB/s | 144 kB 00:00 2024-04-24T08:50:40.954 INFO:teuthology.orchestra.run.smithi069.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-04-24T08:50:40.988 INFO:teuthology.orchestra.run.smithi069.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-24T08:50:41.065 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 129/156 2024-04-24T08:50:41.071 INFO:teuthology.orchestra.run.smithi069.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 199 kB/s | 23 kB 00:00 2024-04-24T08:50:41.096 INFO:teuthology.orchestra.run.smithi069.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 180 kB/s | 19 kB 00:00 2024-04-24T08:50:41.100 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T08:50:41.172 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T08:50:41.172 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:41.172 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T08:50:41.172 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T08:50:41.172 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T08:50:41.172 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:50:41.188 INFO:teuthology.orchestra.run.smithi069.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 169 kB/s | 20 kB 00:00 2024-04-24T08:50:41.235 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T08:50:41.273 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T08:50:41.289 INFO:teuthology.orchestra.run.smithi069.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-04-24T08:50:41.314 INFO:teuthology.orchestra.run.smithi069.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 371 kB/s | 46 kB 00:00 2024-04-24T08:50:41.398 INFO:teuthology.orchestra.run.smithi069.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 732 kB/s | 79 kB 00:00 2024-04-24T08:50:41.416 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 132/156 2024-04-24T08:50:41.465 INFO:teuthology.orchestra.run.smithi069.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 385 kB/s | 58 kB 00:00 2024-04-24T08:50:41.504 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 133/156 2024-04-24T08:50:41.532 INFO:teuthology.orchestra.run.smithi069.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-24T08:50:41.582 INFO:teuthology.orchestra.run.smithi069.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-04-24T08:50:41.649 INFO:teuthology.orchestra.run.smithi069.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 772 kB/s | 90 kB 00:00 2024-04-24T08:50:41.691 INFO:teuthology.orchestra.run.smithi069.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 261 kB/s | 28 kB 00:00 2024-04-24T08:50:41.808 INFO:teuthology.orchestra.run.smithi069.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 195 kB/s | 31 kB 00:00 2024-04-24T08:50:41.833 INFO:teuthology.orchestra.run.smithi069.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-04-24T08:50:41.864 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T08:50:41.920 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T08:50:41.995 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T08:50:41.995 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:41.996 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T08:50:41.996 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T08:50:41.996 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T08:50:41.996 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:50:42.019 INFO:teuthology.orchestra.run.smithi069.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 193 kB/s | 36 kB 00:00 2024-04-24T08:50:42.059 INFO:teuthology.orchestra.run.smithi069.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 237 kB/s | 59 kB 00:00 2024-04-24T08:50:42.079 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T08:50:42.142 INFO:teuthology.orchestra.run.smithi069.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 419 kB/s | 51 kB 00:00 2024-04-24T08:50:42.150 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T08:50:42.150 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:42.151 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T08:50:42.151 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:50:42.192 INFO:teuthology.orchestra.run.smithi069.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-04-24T08:50:42.238 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T08:50:42.276 INFO:teuthology.orchestra.run.smithi069.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 675 kB/s | 90 kB 00:00 2024-04-24T08:50:42.311 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T08:50:42.311 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:42.311 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T08:50:42.311 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T08:50:42.311 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T08:50:42.311 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:50:42.368 INFO:teuthology.orchestra.run.smithi069.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-04-24T08:50:42.410 INFO:teuthology.orchestra.run.smithi069.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 150 kB/s | 20 kB 00:00 2024-04-24T08:50:42.494 INFO:teuthology.orchestra.run.smithi069.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-04-24T08:50:42.522 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T08:50:42.562 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T08:50:42.577 INFO:teuthology.orchestra.run.smithi069.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-24T08:50:42.635 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T08:50:42.635 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:42.635 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T08:50:42.635 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T08:50:42.635 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T08:50:42.635 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:50:42.680 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T08:50:42.705 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T08:50:42.756 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T08:50:42.756 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:42.756 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T08:50:42.756 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:50:42.773 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T08:50:42.773 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:42.773 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T08:50:42.773 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T08:50:42.774 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T08:50:42.774 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:50:42.847 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T08:50:42.852 INFO:teuthology.orchestra.run.smithi069.stdout:(148/152): librados2-19.0.0-3116.g1f81fb39.el9. 9.5 MB/s | 3.4 MB 00:00 2024-04-24T08:50:42.922 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T08:50:42.922 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:42.922 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T08:50:42.923 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T08:50:42.923 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T08:50:42.923 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:50:43.281 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T08:50:43.350 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T08:50:43.350 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:50:43.350 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T08:50:43.350 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T08:50:43.350 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T08:50:43.350 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:50:43.386 INFO:teuthology.orchestra.run.smithi069.stdout:(149/152): librbd1-19.0.0-3116.g1f81fb39.el9.x8 3.9 MB/s | 3.1 MB 00:00 2024-04-24T08:50:43.745 INFO:teuthology.orchestra.run.smithi069.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 1.5 MB/s | 1.3 MB 00:00 2024-04-24T08:50:44.420 INFO:teuthology.orchestra.run.smithi069.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-04-24T08:50:47.798 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 139/156 2024-04-24T08:50:47.862 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 140/156 2024-04-24T08:50:47.957 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 141/156 2024-04-24T08:50:48.030 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T08:50:48.103 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T08:50:48.430 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 139/156 2024-04-24T08:50:48.446 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 144/156 2024-04-24T08:50:48.488 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 140/156 2024-04-24T08:50:48.537 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T08:50:48.565 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 146/156 2024-04-24T08:50:48.576 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 141/156 2024-04-24T08:50:48.627 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T08:50:48.700 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T08:50:49.038 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 144/156 2024-04-24T08:50:49.109 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T08:50:49.137 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 146/156 2024-04-24T08:50:49.388 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:50:49.430 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:50:49.490 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:50:50.050 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:50:50.094 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:50:50.161 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:50:50.840 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 148/156 2024-04-24T08:50:50.954 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T08:50:51.033 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T08:50:51.244 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T08:50:51.285 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T08:50:51.285 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T08:50:51.420 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T08:50:51.454 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 148/156 2024-04-24T08:50:51.459 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T08:50:51.459 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T08:50:51.559 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T08:50:51.595 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T08:50:51.652 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T08:50:51.729 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:50:51.832 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T08:50:51.873 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T08:50:51.873 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T08:50:51.983 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T08:50:52.027 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:50:52.027 INFO:teuthology.orchestra.run.smithi081.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-24T08:50:52.027 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:50:52.039 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T08:50:52.039 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T08:50:52.182 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T08:50:52.310 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:50:52.655 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:50:52.655 INFO:teuthology.orchestra.run.smithi130.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-24T08:50:52.655 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:50:54.369 INFO:teuthology.orchestra.run.smithi069.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 761 kB/s | 16 MB 00:21 2024-04-24T08:50:54.381 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:50:54.381 INFO:teuthology.orchestra.run.smithi069.stdout:Total 3.5 MB/s | 222 MB 01:03 2024-04-24T08:50:57.235 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-04-24T08:50:57.484 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-04-24T08:50:57.485 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-04-24T08:50:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 1/156 2024-04-24T08:50:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2/156 2024-04-24T08:50:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 3/156 2024-04-24T08:50:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 4/156 2024-04-24T08:50:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 5/156 2024-04-24T08:50:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 6/156 2024-04-24T08:50:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 7/156 2024-04-24T08:50:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:50:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:50:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T08:50:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 11/156 2024-04-24T08:50:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 12/156 2024-04-24T08:50:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:50:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 14/156 2024-04-24T08:50:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 15/156 2024-04-24T08:50:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 16/156 2024-04-24T08:50:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 17/156 2024-04-24T08:50:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 18/156 2024-04-24T08:50:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 19/156 2024-04-24T08:50:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 20/156 2024-04-24T08:50:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 21/156 2024-04-24T08:50:57.923 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 22/156 2024-04-24T08:50:57.923 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 23/156 2024-04-24T08:50:57.923 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 24/156 2024-04-24T08:50:57.923 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 25/156 2024-04-24T08:50:57.923 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 26/156 2024-04-24T08:50:57.923 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 27/156 2024-04-24T08:50:57.923 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 28/156 2024-04-24T08:50:57.923 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 29/156 2024-04-24T08:50:57.923 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 30/156 2024-04-24T08:50:57.935 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 31/156 2024-04-24T08:50:57.935 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 32/156 2024-04-24T08:50:57.935 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 33/156 2024-04-24T08:50:57.935 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 34/156 2024-04-24T08:50:57.935 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 35/156 2024-04-24T08:50:57.935 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 36/156 2024-04-24T08:50:57.935 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 37/156 2024-04-24T08:50:57.935 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 38/156 2024-04-24T08:50:57.935 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 39/156 2024-04-24T08:50:57.935 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T08:50:57.936 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T08:50:57.936 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T08:50:57.936 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T08:50:57.936 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T08:50:57.936 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T08:50:57.936 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T08:50:57.936 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T08:50:57.936 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T08:50:57.936 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T08:50:57.936 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T08:50:57.936 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T08:50:57.936 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T08:50:57.936 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T08:50:57.937 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T08:50:57.937 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T08:50:57.937 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T08:50:57.937 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T08:50:57.937 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T08:50:57.937 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T08:50:57.937 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T08:50:57.937 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T08:50:57.937 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T08:50:57.937 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T08:50:57.937 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T08:50:57.937 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T08:50:57.937 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T08:50:57.937 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T08:50:57.938 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T08:50:57.938 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T08:50:57.938 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T08:50:57.938 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T08:50:57.938 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T08:50:57.938 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T08:50:57.938 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T08:50:57.938 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T08:50:57.938 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T08:50:57.938 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T08:50:57.938 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T08:50:57.938 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T08:50:57.938 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T08:50:57.938 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T08:50:57.939 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T08:50:57.939 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T08:50:57.939 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T08:50:57.939 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T08:50:57.939 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T08:50:57.939 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T08:50:57.939 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T08:50:57.939 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T08:50:57.939 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T08:50:57.939 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T08:50:57.939 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T08:50:57.939 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T08:50:57.939 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T08:50:57.939 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T08:50:57.939 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T08:50:57.940 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T08:50:57.940 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T08:50:57.940 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T08:50:57.940 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T08:50:57.940 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T08:50:57.940 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T08:50:57.940 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T08:50:57.940 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T08:50:57.940 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T08:50:57.940 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T08:50:57.940 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T08:50:57.940 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T08:50:57.940 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T08:50:57.940 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T08:50:57.941 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T08:50:57.941 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T08:50:57.941 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T08:50:57.941 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T08:50:57.941 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T08:50:57.941 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T08:50:57.941 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T08:50:57.941 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T08:50:57.941 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T08:50:57.941 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T08:50:57.941 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T08:50:57.941 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T08:50:57.942 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T08:50:57.942 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T08:50:57.942 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T08:50:57.942 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T08:50:57.942 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T08:50:57.942 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T08:50:57.942 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T08:50:57.942 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T08:50:57.942 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T08:50:57.942 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T08:50:57.942 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T08:50:57.942 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T08:50:57.943 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T08:50:57.943 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T08:50:57.943 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T08:50:57.943 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T08:50:57.943 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T08:50:57.943 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T08:50:57.943 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T08:50:57.943 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T08:50:57.943 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T08:50:57.943 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T08:50:57.943 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T08:50:57.943 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T08:50:57.943 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T08:50:57.943 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T08:50:57.944 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 149/156 2024-04-24T08:50:57.944 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T08:50:57.944 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 151/156 2024-04-24T08:50:57.944 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T08:50:57.944 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T08:50:57.944 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T08:50:57.944 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T08:50:58.558 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 1/156 2024-04-24T08:50:58.558 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2/156 2024-04-24T08:50:58.558 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 3/156 2024-04-24T08:50:58.558 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 4/156 2024-04-24T08:50:58.558 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 5/156 2024-04-24T08:50:58.558 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 6/156 2024-04-24T08:50:58.558 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 7/156 2024-04-24T08:50:58.558 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:50:58.559 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:50:58.559 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T08:50:58.559 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 11/156 2024-04-24T08:50:58.559 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 12/156 2024-04-24T08:50:58.559 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:50:58.559 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 14/156 2024-04-24T08:50:58.559 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 15/156 2024-04-24T08:50:58.559 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 16/156 2024-04-24T08:50:58.559 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 17/156 2024-04-24T08:50:58.559 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 18/156 2024-04-24T08:50:58.559 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 19/156 2024-04-24T08:50:58.559 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 20/156 2024-04-24T08:50:58.559 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 21/156 2024-04-24T08:50:58.559 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 22/156 2024-04-24T08:50:58.560 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 23/156 2024-04-24T08:50:58.560 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 24/156 2024-04-24T08:50:58.560 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 25/156 2024-04-24T08:50:58.560 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 26/156 2024-04-24T08:50:58.560 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 27/156 2024-04-24T08:50:58.560 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 28/156 2024-04-24T08:50:58.560 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 29/156 2024-04-24T08:50:58.560 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 30/156 2024-04-24T08:50:58.560 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 31/156 2024-04-24T08:50:58.560 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 32/156 2024-04-24T08:50:58.560 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 33/156 2024-04-24T08:50:58.561 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 34/156 2024-04-24T08:50:58.561 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 35/156 2024-04-24T08:50:58.561 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 36/156 2024-04-24T08:50:58.561 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 37/156 2024-04-24T08:50:58.561 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 38/156 2024-04-24T08:50:58.561 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 39/156 2024-04-24T08:50:58.561 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T08:50:58.561 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T08:50:58.561 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T08:50:58.561 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T08:50:58.561 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T08:50:58.561 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T08:50:58.561 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T08:50:58.561 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T08:50:58.561 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T08:50:58.561 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T08:50:58.562 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T08:50:58.562 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T08:50:58.562 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T08:50:58.562 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T08:50:58.562 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T08:50:58.562 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T08:50:58.562 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T08:50:58.562 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T08:50:58.562 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T08:50:58.562 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T08:50:58.562 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T08:50:58.563 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T08:50:58.563 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T08:50:58.563 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T08:50:58.563 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T08:50:58.563 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T08:50:58.563 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T08:50:58.563 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T08:50:58.563 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T08:50:58.563 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T08:50:58.563 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T08:50:58.563 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T08:50:58.563 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T08:50:58.563 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T08:50:58.563 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T08:50:58.563 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T08:50:58.564 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T08:50:58.564 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T08:50:58.564 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T08:50:58.564 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T08:50:58.564 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T08:50:58.564 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T08:50:58.564 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T08:50:58.564 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T08:50:58.564 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T08:50:58.564 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T08:50:58.564 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T08:50:58.564 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T08:50:58.564 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T08:50:58.564 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T08:50:58.564 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T08:50:58.565 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T08:50:58.565 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T08:50:58.565 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T08:50:58.565 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T08:50:58.565 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T08:50:58.565 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T08:50:58.565 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T08:50:58.565 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T08:50:58.565 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T08:50:58.565 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T08:50:58.565 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T08:50:58.565 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T08:50:58.565 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T08:50:58.565 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T08:50:58.565 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T08:50:58.566 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T08:50:58.566 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T08:50:58.566 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T08:50:58.566 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T08:50:58.566 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T08:50:58.566 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T08:50:58.566 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T08:50:58.566 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T08:50:58.566 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T08:50:58.566 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T08:50:58.566 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T08:50:58.566 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T08:50:58.566 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T08:50:58.566 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T08:50:58.566 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T08:50:58.567 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T08:50:58.567 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T08:50:58.567 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T08:50:58.567 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T08:50:58.567 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T08:50:58.567 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T08:50:58.567 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T08:50:58.567 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T08:50:58.567 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T08:50:58.567 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T08:50:58.567 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T08:50:58.567 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T08:50:58.567 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T08:50:58.567 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T08:50:58.567 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T08:50:58.568 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T08:50:58.568 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T08:50:58.568 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T08:50:58.568 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T08:50:58.568 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T08:50:58.568 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T08:50:58.568 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T08:50:58.568 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T08:50:58.568 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T08:50:58.568 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T08:50:58.568 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T08:50:58.568 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T08:50:58.568 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T08:50:58.568 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 149/156 2024-04-24T08:50:58.568 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T08:50:58.569 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 151/156 2024-04-24T08:50:58.569 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T08:50:58.569 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T08:50:58.569 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T08:50:58.569 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T08:51:00.082 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-04-24T08:51:00.082 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-04-24T08:51:00.337 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:51:00.337 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:51:00.338 INFO:teuthology.orchestra.run.smithi081.stdout:Upgraded: 2024-04-24T08:51:00.338 INFO:teuthology.orchestra.run.smithi081.stdout: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.338 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.338 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T08:51:00.338 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T08:51:00.338 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-04-24T08:51:00.338 INFO:teuthology.orchestra.run.smithi081.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T08:51:00.338 INFO:teuthology.orchestra.run.smithi081.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T08:51:00.338 INFO:teuthology.orchestra.run.smithi081.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T08:51:00.338 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T08:51:00.338 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.338 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.338 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.338 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.338 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:00.339 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.339 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.339 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.339 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:00.339 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:00.339 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:00.339 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:00.339 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:00.339 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.339 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.339 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:00.339 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.339 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.339 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.339 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:00.340 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:00.340 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.340 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.340 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:00.340 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T08:51:00.340 INFO:teuthology.orchestra.run.smithi081.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T08:51:00.340 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T08:51:00.340 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T08:51:00.340 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T08:51:00.340 INFO:teuthology.orchestra.run.smithi081.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T08:51:00.340 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T08:51:00.340 INFO:teuthology.orchestra.run.smithi081.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T08:51:00.340 INFO:teuthology.orchestra.run.smithi081.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T08:51:00.340 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T08:51:00.340 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T08:51:00.340 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.341 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.341 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.341 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T08:51:00.341 INFO:teuthology.orchestra.run.smithi081.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T08:51:00.341 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T08:51:00.341 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T08:51:00.341 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T08:51:00.341 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.341 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.341 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T08:51:00.341 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.341 INFO:teuthology.orchestra.run.smithi081.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T08:51:00.341 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T08:51:00.341 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T08:51:00.341 INFO:teuthology.orchestra.run.smithi081.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T08:51:00.341 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T08:51:00.342 INFO:teuthology.orchestra.run.smithi081.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T08:51:00.342 INFO:teuthology.orchestra.run.smithi081.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T08:51:00.342 INFO:teuthology.orchestra.run.smithi081.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T08:51:00.342 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T08:51:00.342 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T08:51:00.342 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T08:51:00.342 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T08:51:00.342 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T08:51:00.342 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T08:51:00.342 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T08:51:00.342 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T08:51:00.342 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T08:51:00.342 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T08:51:00.342 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T08:51:00.342 INFO:teuthology.orchestra.run.smithi081.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T08:51:00.343 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T08:51:00.343 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T08:51:00.343 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T08:51:00.343 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.343 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.343 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.343 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T08:51:00.343 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T08:51:00.343 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T08:51:00.343 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T08:51:00.343 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T08:51:00.343 INFO:teuthology.orchestra.run.smithi081.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T08:51:00.343 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T08:51:00.343 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T08:51:00.343 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T08:51:00.343 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T08:51:00.344 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T08:51:00.344 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T08:51:00.344 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T08:51:00.344 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T08:51:00.344 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T08:51:00.344 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T08:51:00.344 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T08:51:00.344 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T08:51:00.344 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T08:51:00.344 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T08:51:00.344 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T08:51:00.344 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T08:51:00.344 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T08:51:00.344 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T08:51:00.344 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T08:51:00.345 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T08:51:00.345 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T08:51:00.345 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T08:51:00.345 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T08:51:00.345 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T08:51:00.345 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T08:51:00.345 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T08:51:00.345 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T08:51:00.345 INFO:teuthology.orchestra.run.smithi081.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T08:51:00.345 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T08:51:00.345 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T08:51:00.345 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T08:51:00.345 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T08:51:00.345 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T08:51:00.345 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T08:51:00.346 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.346 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.346 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T08:51:00.346 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T08:51:00.346 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T08:51:00.346 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.346 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T08:51:00.346 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T08:51:00.346 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T08:51:00.346 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T08:51:00.346 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T08:51:00.346 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T08:51:00.346 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T08:51:00.346 INFO:teuthology.orchestra.run.smithi081.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T08:51:00.347 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T08:51:00.347 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T08:51:00.347 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T08:51:00.347 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T08:51:00.347 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T08:51:00.347 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T08:51:00.347 INFO:teuthology.orchestra.run.smithi081.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T08:51:00.347 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.347 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.347 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:00.347 INFO:teuthology.orchestra.run.smithi081.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T08:51:00.347 INFO:teuthology.orchestra.run.smithi081.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T08:51:00.347 INFO:teuthology.orchestra.run.smithi081.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T08:51:00.347 INFO:teuthology.orchestra.run.smithi081.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T08:51:00.347 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T08:51:00.347 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T08:51:00.348 INFO:teuthology.orchestra.run.smithi081.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T08:51:00.348 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-04-24T08:51:00.348 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-04-24T08:51:01.011 DEBUG:teuthology.parallel:result is None 2024-04-24T08:51:01.044 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:51:01.044 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:51:01.044 INFO:teuthology.orchestra.run.smithi130.stdout:Upgraded: 2024-04-24T08:51:01.044 INFO:teuthology.orchestra.run.smithi130.stdout: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.044 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.044 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T08:51:01.045 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T08:51:01.045 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-24T08:51:01.045 INFO:teuthology.orchestra.run.smithi130.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T08:51:01.045 INFO:teuthology.orchestra.run.smithi130.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T08:51:01.045 INFO:teuthology.orchestra.run.smithi130.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T08:51:01.045 INFO:teuthology.orchestra.run.smithi130.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T08:51:01.045 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.045 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.045 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.045 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.045 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:01.045 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.045 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.045 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.045 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:01.046 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:01.046 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:01.046 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:01.046 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:01.046 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.046 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.046 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:01.046 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.046 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.046 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.046 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:01.046 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:01.046 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.046 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.046 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:51:01.047 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T08:51:01.047 INFO:teuthology.orchestra.run.smithi130.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T08:51:01.047 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T08:51:01.047 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T08:51:01.047 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T08:51:01.047 INFO:teuthology.orchestra.run.smithi130.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T08:51:01.047 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T08:51:01.047 INFO:teuthology.orchestra.run.smithi130.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T08:51:01.047 INFO:teuthology.orchestra.run.smithi130.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T08:51:01.047 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T08:51:01.047 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T08:51:01.047 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.047 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.047 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.048 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T08:51:01.048 INFO:teuthology.orchestra.run.smithi130.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T08:51:01.048 INFO:teuthology.orchestra.run.smithi130.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T08:51:01.048 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T08:51:01.048 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T08:51:01.048 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.048 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.048 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T08:51:01.048 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.048 INFO:teuthology.orchestra.run.smithi130.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T08:51:01.048 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T08:51:01.048 INFO:teuthology.orchestra.run.smithi130.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T08:51:01.048 INFO:teuthology.orchestra.run.smithi130.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T08:51:01.048 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T08:51:01.048 INFO:teuthology.orchestra.run.smithi130.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T08:51:01.049 INFO:teuthology.orchestra.run.smithi130.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T08:51:01.049 INFO:teuthology.orchestra.run.smithi130.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T08:51:01.049 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T08:51:01.049 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T08:51:01.049 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T08:51:01.049 INFO:teuthology.orchestra.run.smithi130.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T08:51:01.049 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T08:51:01.049 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T08:51:01.049 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T08:51:01.049 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T08:51:01.049 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T08:51:01.049 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T08:51:01.049 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T08:51:01.049 INFO:teuthology.orchestra.run.smithi130.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T08:51:01.049 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T08:51:01.050 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T08:51:01.050 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T08:51:01.050 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.050 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.050 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.050 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T08:51:01.050 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T08:51:01.050 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T08:51:01.050 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T08:51:01.050 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T08:51:01.050 INFO:teuthology.orchestra.run.smithi130.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T08:51:01.050 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T08:51:01.050 INFO:teuthology.orchestra.run.smithi130.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T08:51:01.050 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T08:51:01.050 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T08:51:01.051 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T08:51:01.052 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T08:51:01.052 INFO:teuthology.orchestra.run.smithi130.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T08:51:01.052 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T08:51:01.052 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T08:51:01.052 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T08:51:01.052 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T08:51:01.052 INFO:teuthology.orchestra.run.smithi130.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T08:51:01.052 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T08:51:01.052 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T08:51:01.052 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T08:51:01.052 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T08:51:01.052 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T08:51:01.052 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T08:51:01.052 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.052 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.053 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T08:51:01.053 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T08:51:01.053 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T08:51:01.053 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.053 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T08:51:01.053 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T08:51:01.053 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T08:51:01.053 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T08:51:01.053 INFO:teuthology.orchestra.run.smithi130.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T08:51:01.053 INFO:teuthology.orchestra.run.smithi130.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T08:51:01.053 INFO:teuthology.orchestra.run.smithi130.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T08:51:01.053 INFO:teuthology.orchestra.run.smithi130.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T08:51:01.053 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T08:51:01.053 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T08:51:01.053 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T08:51:01.054 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T08:51:01.054 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T08:51:01.054 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T08:51:01.054 INFO:teuthology.orchestra.run.smithi130.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T08:51:01.054 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.054 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.054 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:51:01.054 INFO:teuthology.orchestra.run.smithi130.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T08:51:01.054 INFO:teuthology.orchestra.run.smithi130.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T08:51:01.054 INFO:teuthology.orchestra.run.smithi130.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T08:51:01.054 INFO:teuthology.orchestra.run.smithi130.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T08:51:01.054 INFO:teuthology.orchestra.run.smithi130.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T08:51:01.054 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T08:51:01.055 INFO:teuthology.orchestra.run.smithi130.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T08:51:01.055 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:51:01.055 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-24T08:51:01.605 DEBUG:teuthology.parallel:result is None 2024-04-24T08:51:06.492 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-04-24T08:51:06.600 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T08:51:06.692 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T08:51:06.811 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T08:51:06.871 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T08:51:06.909 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T08:51:07.034 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T08:51:07.441 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T08:51:07.540 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:51:07.719 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:51:07.750 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:51:07.902 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:51:07.949 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T08:51:08.299 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T08:51:08.365 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T08:51:08.389 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:51:08.530 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:51:08.556 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T08:51:08.647 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T08:51:08.674 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T08:51:08.745 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T08:51:09.053 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T08:51:09.116 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T08:51:09.315 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T08:51:09.439 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T08:51:09.483 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T08:51:09.531 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T08:51:09.636 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T08:51:09.686 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T08:51:09.773 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T08:51:09.816 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T08:51:09.865 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T08:51:09.983 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T08:51:10.052 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T08:51:10.091 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T08:51:10.148 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T08:51:10.258 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T08:51:10.302 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 31/156 2024-04-24T08:51:10.360 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 32/156 2024-04-24T08:51:10.393 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 33/156 2024-04-24T08:51:10.513 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T08:51:10.578 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T08:51:10.679 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T08:51:10.877 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T08:51:10.941 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T08:51:11.640 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T08:51:12.045 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T08:51:12.083 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T08:51:12.134 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T08:51:12.183 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 43/156 2024-04-24T08:51:12.293 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T08:51:12.587 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T08:51:12.668 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T08:51:12.752 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T08:51:12.857 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T08:51:12.901 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T08:51:12.940 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T08:51:13.008 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T08:51:13.048 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T08:51:13.108 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T08:51:13.168 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T08:51:13.365 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T08:51:13.421 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T08:51:13.458 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T08:51:13.517 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T08:51:13.656 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T08:51:13.699 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T08:51:13.748 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T08:51:13.806 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T08:51:13.839 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T08:51:14.030 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T08:51:14.212 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T08:51:14.284 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T08:51:14.326 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T08:51:14.372 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T08:51:14.432 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T08:51:14.482 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T08:51:14.523 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T08:51:14.569 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T08:51:14.619 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T08:51:14.695 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T08:51:14.744 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T08:51:14.928 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T08:51:14.994 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T08:51:15.042 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T08:51:15.115 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T08:51:15.270 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T08:51:15.338 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T08:51:16.305 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T08:51:16.361 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T08:51:16.395 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T08:51:16.517 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T08:51:16.588 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T08:51:16.688 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T08:51:16.729 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T08:51:16.959 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T08:51:16.993 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T08:51:17.105 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T08:51:17.948 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T08:51:18.090 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T08:51:19.746 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T08:51:19.859 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T08:51:19.914 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T08:51:19.963 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T08:51:20.352 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T08:51:20.394 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T08:51:20.515 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T08:51:20.559 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 100/156 2024-04-24T08:51:20.607 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T08:51:21.150 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T08:51:21.189 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T08:51:21.263 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T08:51:21.291 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 104/156 2024-04-24T08:51:24.032 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:51:24.078 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:51:24.749 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:51:24.798 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T08:51:25.036 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T08:51:25.036 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T08:51:25.037 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T08:51:25.037 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:51:25.082 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T08:51:38.140 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T08:51:38.140 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /sys 2024-04-24T08:51:38.140 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /proc 2024-04-24T08:51:38.140 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /mnt 2024-04-24T08:51:38.140 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /var/tmp 2024-04-24T08:51:38.140 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /home 2024-04-24T08:51:38.140 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /root 2024-04-24T08:51:38.140 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /tmp 2024-04-24T08:51:38.140 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:51:38.387 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T08:51:38.435 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T08:51:38.435 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:51:38.435 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T08:51:38.435 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:51:40.090 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T08:51:40.176 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T08:51:40.177 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:51:40.177 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T08:51:40.177 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T08:51:40.177 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T08:51:40.177 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:51:40.431 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T08:51:40.499 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T08:51:40.500 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:51:40.500 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T08:51:40.500 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T08:51:40.500 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T08:51:40.500 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:51:40.999 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T08:51:41.066 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T08:51:41.067 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:51:41.067 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T08:51:41.067 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T08:51:41.067 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T08:51:41.067 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:51:41.199 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T08:51:41.259 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T08:51:41.327 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T08:51:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T08:51:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T08:51:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T08:51:41.822 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T08:51:41.994 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T08:51:42.046 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T08:51:42.085 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T08:51:42.118 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T08:51:42.758 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T08:51:42.796 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T08:51:42.849 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 124/156 2024-04-24T08:51:42.988 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 125/156 2024-04-24T08:51:43.023 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T08:51:44.340 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T08:51:44.383 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T08:51:45.480 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T08:51:45.525 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T08:51:45.671 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T08:51:45.848 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 129/156 2024-04-24T08:51:45.893 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T08:51:45.963 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T08:51:45.963 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:51:45.963 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T08:51:45.963 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T08:51:45.963 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T08:51:45.964 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:51:46.019 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T08:51:46.057 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T08:51:46.160 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 132/156 2024-04-24T08:51:46.220 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 133/156 2024-04-24T08:51:47.218 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T08:51:47.255 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T08:51:47.325 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T08:51:47.326 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:51:47.326 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T08:51:47.326 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T08:51:47.326 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T08:51:47.326 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:51:47.373 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T08:51:47.441 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T08:51:47.441 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:51:47.441 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T08:51:47.441 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:51:47.507 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T08:51:47.581 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T08:51:47.582 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:51:47.582 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T08:51:47.582 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T08:51:47.582 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T08:51:47.582 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:51:47.982 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T08:51:48.058 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T08:51:48.058 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:51:48.058 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T08:51:48.058 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T08:51:48.059 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T08:51:48.059 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:51:53.159 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 139/156 2024-04-24T08:51:53.216 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 140/156 2024-04-24T08:51:53.319 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 141/156 2024-04-24T08:51:53.369 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T08:51:53.418 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T08:51:53.768 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 144/156 2024-04-24T08:51:53.826 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T08:51:53.854 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 146/156 2024-04-24T08:51:54.925 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:51:54.968 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:51:55.022 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:51:56.392 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 148/156 2024-04-24T08:51:56.492 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T08:51:56.573 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T08:51:56.764 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T08:51:56.806 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T08:51:56.806 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T08:51:56.967 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T08:51:57.014 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T08:51:57.014 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T08:51:57.212 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T08:51:57.344 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:51:57.630 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:51:57.630 INFO:teuthology.orchestra.run.smithi069.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-24T08:51:57.630 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:52:03.173 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 1/156 2024-04-24T08:52:03.173 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2/156 2024-04-24T08:52:03.173 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 3/156 2024-04-24T08:52:03.173 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 4/156 2024-04-24T08:52:03.173 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 5/156 2024-04-24T08:52:03.173 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 6/156 2024-04-24T08:52:03.173 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 7/156 2024-04-24T08:52:03.173 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:52:03.173 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:52:03.173 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T08:52:03.173 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 11/156 2024-04-24T08:52:03.173 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 12/156 2024-04-24T08:52:03.174 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:52:03.174 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 14/156 2024-04-24T08:52:03.174 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 15/156 2024-04-24T08:52:03.174 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 16/156 2024-04-24T08:52:03.174 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 17/156 2024-04-24T08:52:03.174 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 18/156 2024-04-24T08:52:03.174 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 19/156 2024-04-24T08:52:03.174 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 20/156 2024-04-24T08:52:03.174 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 21/156 2024-04-24T08:52:03.174 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 22/156 2024-04-24T08:52:03.174 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 23/156 2024-04-24T08:52:03.174 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 24/156 2024-04-24T08:52:03.174 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 25/156 2024-04-24T08:52:03.174 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 26/156 2024-04-24T08:52:03.175 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 27/156 2024-04-24T08:52:03.187 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 28/156 2024-04-24T08:52:03.187 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 29/156 2024-04-24T08:52:03.187 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 30/156 2024-04-24T08:52:03.187 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 31/156 2024-04-24T08:52:03.187 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 32/156 2024-04-24T08:52:03.187 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 33/156 2024-04-24T08:52:03.187 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 34/156 2024-04-24T08:52:03.187 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 35/156 2024-04-24T08:52:03.187 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 36/156 2024-04-24T08:52:03.187 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 37/156 2024-04-24T08:52:03.187 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 38/156 2024-04-24T08:52:03.187 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 39/156 2024-04-24T08:52:03.188 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T08:52:03.188 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T08:52:03.188 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T08:52:03.188 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T08:52:03.188 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T08:52:03.188 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T08:52:03.188 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T08:52:03.188 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T08:52:03.188 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T08:52:03.188 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T08:52:03.188 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T08:52:03.188 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T08:52:03.188 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T08:52:03.188 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T08:52:03.188 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T08:52:03.189 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T08:52:03.189 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T08:52:03.189 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T08:52:03.189 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T08:52:03.189 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T08:52:03.189 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T08:52:03.189 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T08:52:03.189 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T08:52:03.189 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T08:52:03.189 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T08:52:03.189 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T08:52:03.189 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T08:52:03.189 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T08:52:03.190 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T08:52:03.190 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T08:52:03.190 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T08:52:03.190 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T08:52:03.190 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T08:52:03.190 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T08:52:03.190 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T08:52:03.190 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T08:52:03.190 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T08:52:03.190 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T08:52:03.190 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T08:52:03.190 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T08:52:03.191 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T08:52:03.191 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T08:52:03.191 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T08:52:03.191 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T08:52:03.191 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T08:52:03.191 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T08:52:03.191 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T08:52:03.191 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T08:52:03.191 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T08:52:03.191 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T08:52:03.191 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T08:52:03.191 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T08:52:03.191 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T08:52:03.191 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T08:52:03.192 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T08:52:03.192 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T08:52:03.192 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T08:52:03.192 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T08:52:03.192 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T08:52:03.192 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T08:52:03.192 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T08:52:03.192 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T08:52:03.192 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T08:52:03.192 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T08:52:03.192 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T08:52:03.192 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T08:52:03.192 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T08:52:03.192 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T08:52:03.192 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T08:52:03.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T08:52:03.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T08:52:03.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T08:52:03.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T08:52:03.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T08:52:03.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T08:52:03.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T08:52:03.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T08:52:03.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T08:52:03.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T08:52:03.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T08:52:03.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T08:52:03.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T08:52:03.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T08:52:03.193 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T08:52:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T08:52:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T08:52:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T08:52:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T08:52:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T08:52:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T08:52:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T08:52:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T08:52:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T08:52:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T08:52:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T08:52:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T08:52:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T08:52:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T08:52:03.194 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T08:52:03.195 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T08:52:03.195 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T08:52:03.195 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T08:52:03.195 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T08:52:03.195 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T08:52:03.195 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T08:52:03.195 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T08:52:03.195 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T08:52:03.195 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T08:52:03.195 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T08:52:03.195 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 149/156 2024-04-24T08:52:03.195 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T08:52:03.195 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 151/156 2024-04-24T08:52:03.195 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T08:52:03.195 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T08:52:03.195 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T08:52:03.196 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T08:52:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:52:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:52:05.672 INFO:teuthology.orchestra.run.smithi069.stdout:Upgraded: 2024-04-24T08:52:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T08:52:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T08:52:05.672 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-04-24T08:52:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T08:52:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T08:52:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T08:52:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T08:52:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:52:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:52:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:52:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:52:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:52:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:52:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:52:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:52:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:52:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:52:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T08:52:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T08:52:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T08:52:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T08:52:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T08:52:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T08:52:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T08:52:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T08:52:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T08:52:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T08:52:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T08:52:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T08:52:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T08:52:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T08:52:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T08:52:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T08:52:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T08:52:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T08:52:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T08:52:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T08:52:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T08:52:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T08:52:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T08:52:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T08:52:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T08:52:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T08:52:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T08:52:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T08:52:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T08:52:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T08:52:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T08:52:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T08:52:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T08:52:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T08:52:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T08:52:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T08:52:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T08:52:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T08:52:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T08:52:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T08:52:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T08:52:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T08:52:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T08:52:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T08:52:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T08:52:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T08:52:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T08:52:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T08:52:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T08:52:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T08:52:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T08:52:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T08:52:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T08:52:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T08:52:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T08:52:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T08:52:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T08:52:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T08:52:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T08:52:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T08:52:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T08:52:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T08:52:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T08:52:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T08:52:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T08:52:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T08:52:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T08:52:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T08:52:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T08:52:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T08:52:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T08:52:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T08:52:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T08:52:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T08:52:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T08:52:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T08:52:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T08:52:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T08:52:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T08:52:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T08:52:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T08:52:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T08:52:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T08:52:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T08:52:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T08:52:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T08:52:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T08:52:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T08:52:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T08:52:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T08:52:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T08:52:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T08:52:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T08:52:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T08:52:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T08:52:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T08:52:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T08:52:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T08:52:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:52:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T08:52:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T08:52:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T08:52:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T08:52:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T08:52:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T08:52:05.683 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T08:52:05.683 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:52:05.683 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-04-24T08:52:06.260 DEBUG:teuthology.parallel:result is None 2024-04-24T08:52:06.260 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:52:06.403 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T08:52:06.456 INFO:teuthology.orchestra.run.smithi069.stdout:19.0.0-3116.g1f81fb39.el9 2024-04-24T08:52:06.456 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3116.g1f81fb39.el9 2024-04-24T08:52:06.457 INFO:teuthology.task.install:The correct ceph version 19.0.0-3116.g1f81fb39 is installed. 2024-04-24T08:52:06.458 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:52:06.602 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T08:52:06.674 INFO:teuthology.orchestra.run.smithi081.stdout:19.0.0-3116.g1f81fb39.el9 2024-04-24T08:52:06.675 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3116.g1f81fb39.el9 2024-04-24T08:52:06.675 INFO:teuthology.task.install:The correct ceph version 19.0.0-3116.g1f81fb39 is installed. 2024-04-24T08:52:06.676 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:52:06.904 DEBUG:teuthology.orchestra.run.smithi130:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T08:52:06.974 INFO:teuthology.orchestra.run.smithi130.stdout:19.0.0-3116.g1f81fb39.el9 2024-04-24T08:52:06.974 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3116.g1f81fb39.el9 2024-04-24T08:52:06.974 INFO:teuthology.task.install:The correct ceph version 19.0.0-3116.g1f81fb39 is installed. 2024-04-24T08:52:06.976 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T08:52:06.976 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T08:52:06.976 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T08:52:07.067 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-04-24T08:52:07.067 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T08:52:07.161 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-24T08:52:07.161 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T08:52:07.256 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T08:52:07.256 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T08:52:07.256 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T08:52:07.326 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T08:52:07.431 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-04-24T08:52:07.431 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T08:52:07.520 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T08:52:07.643 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-24T08:52:07.643 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T08:52:07.733 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T08:52:07.861 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T08:52:07.861 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T08:52:07.861 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T08:52:07.933 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T08:52:08.045 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-04-24T08:52:08.045 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T08:52:08.133 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T08:52:08.263 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-24T08:52:08.263 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T08:52:08.350 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T08:52:08.471 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T08:52:08.472 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T08:52:08.472 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T08:52:08.547 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T08:52:08.662 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-04-24T08:52:08.662 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T08:52:08.751 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T08:52:08.833 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-24T08:52:08.833 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T08:52:08.921 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T08:52:09.004 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-24T08:52:09.104 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-24T08:52:09.104 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:52:09.105 INFO:tasks.cephadm:Cluster fsid is ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:52:09.105 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-24T08:52:09.105 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.69', 'mon.b': '172.21.15.81', 'mon.c': '172.21.15.130'} 2024-04-24T08:52:09.105 INFO:tasks.cephadm:First mon is mon.a on smithi069 2024-04-24T08:52:09.105 INFO:tasks.cephadm:First mgr is x 2024-04-24T08:52:09.105 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-24T08:52:09.105 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname $(hostname -s) 2024-04-24T08:52:09.172 DEBUG:teuthology.orchestra.run.smithi081:> sudo hostname $(hostname -s) 2024-04-24T08:52:09.261 DEBUG:teuthology.orchestra.run.smithi130:> sudo hostname $(hostname -s) 2024-04-24T08:52:09.352 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-24T08:52:09.352 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:52:09.581 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78823/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3116-g1f81fb39', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3116.g1f81fb39'}, 'url': 'https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 01:00:13.634707', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-main-apr17-a654945', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-24T08:52:09.723 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref rishabh-main-apr17-a654945, sha1 1f81fb397ae98da7563d451a78c61574c8f4e6e0 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:52:09.724 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/x86_64/flavors/default/cephadm 2024-04-24T08:52:09.724 DEBUG:teuthology.orchestra.run.smithi069:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T08:52:10.491 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-r--r--. 1 ubuntu ubuntu 781486 Apr 24 08:52 /home/ubuntu/cephtest/cephadm 2024-04-24T08:52:10.492 DEBUG:teuthology.orchestra.run.smithi081:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T08:52:11.483 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 781486 Apr 24 08:52 /home/ubuntu/cephtest/cephadm 2024-04-24T08:52:11.483 DEBUG:teuthology.orchestra.run.smithi130:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T08:52:12.392 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-r--r--. 1 ubuntu ubuntu 781486 Apr 24 08:52 /home/ubuntu/cephtest/cephadm 2024-04-24T08:52:12.392 DEBUG:teuthology.orchestra.run.smithi069:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T08:52:12.442 DEBUG:teuthology.orchestra.run.smithi081:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T08:52:12.503 DEBUG:teuthology.orchestra.run.smithi130:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T08:52:12.579 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 on all hosts... 2024-04-24T08:52:12.580 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 pull 2024-04-24T08:52:12.584 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 pull 2024-04-24T08:52:12.592 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 pull 2024-04-24T08:52:14.405 INFO:teuthology.orchestra.run.smithi130.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0... 2024-04-24T08:52:14.504 INFO:teuthology.orchestra.run.smithi081.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0... 2024-04-24T08:52:15.113 INFO:teuthology.orchestra.run.smithi069.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0... 2024-04-24T08:52:51.039 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-04-24T08:52:51.039 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)", 2024-04-24T08:52:51.039 INFO:teuthology.orchestra.run.smithi130.stdout: "image_id": "b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02", 2024-04-24T08:52:51.039 INFO:teuthology.orchestra.run.smithi130.stdout: "repo_digests": [ 2024-04-24T08:52:51.039 INFO:teuthology.orchestra.run.smithi130.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644" 2024-04-24T08:52:51.039 INFO:teuthology.orchestra.run.smithi130.stdout: ] 2024-04-24T08:52:51.039 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-04-24T08:52:53.492 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T08:52:53.492 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)", 2024-04-24T08:52:53.492 INFO:teuthology.orchestra.run.smithi069.stdout: "image_id": "b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02", 2024-04-24T08:52:53.492 INFO:teuthology.orchestra.run.smithi069.stdout: "repo_digests": [ 2024-04-24T08:52:53.492 INFO:teuthology.orchestra.run.smithi069.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644" 2024-04-24T08:52:53.492 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-24T08:52:53.493 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T08:52:56.757 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-04-24T08:52:56.757 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)", 2024-04-24T08:52:56.757 INFO:teuthology.orchestra.run.smithi081.stdout: "image_id": "b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02", 2024-04-24T08:52:56.757 INFO:teuthology.orchestra.run.smithi081.stdout: "repo_digests": [ 2024-04-24T08:52:56.757 INFO:teuthology.orchestra.run.smithi081.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644" 2024-04-24T08:52:56.757 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-04-24T08:52:56.757 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-04-24T08:52:56.804 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /etc/ceph 2024-04-24T08:52:56.895 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph 2024-04-24T08:52:56.987 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /etc/ceph 2024-04-24T08:52:57.078 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 777 /etc/ceph 2024-04-24T08:52:57.158 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 777 /etc/ceph 2024-04-24T08:52:57.247 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 777 /etc/ceph 2024-04-24T08:52:57.333 INFO:tasks.cephadm:Writing seed config... 2024-04-24T08:52:57.334 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-24T08:52:57.334 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-24T08:52:57.334 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-24T08:52:57.334 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-24T08:52:57.334 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-24T08:52:57.335 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-24T08:52:57.335 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-24T08:52:57.335 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-24T08:52:57.335 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-24T08:52:57.335 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-24T08:52:57.335 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-24T08:52:57.335 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-24T08:52:57.335 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-24T08:52:57.335 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-24T08:52:57.335 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-24T08:52:57.335 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-24T08:52:57.335 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-24T08:52:57.335 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-24T08:52:57.335 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-24T08:52:57.335 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-24T08:52:57.336 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-24T08:52:57.336 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-24T08:52:57.336 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-24T08:52:57.336 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-24T08:52:57.336 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-24T08:52:57.336 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-24T08:52:57.336 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-24T08:52:57.336 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-24T08:52:57.336 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-24T08:52:57.336 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-24T08:52:57.336 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-24T08:52:57.336 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-24T08:52:57.336 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-24T08:52:57.336 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-24T08:52:57.337 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-24T08:52:57.337 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-24T08:52:57.337 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-24T08:52:57.337 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-24T08:52:57.337 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-24T08:52:57.337 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-24T08:52:57.337 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-24T08:52:57.337 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-24T08:52:57.337 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-24T08:52:57.337 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-24T08:52:57.337 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-24T08:52:57.338 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T08:52:57.338 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-24T08:52:57.387 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ef1ec7e8-0217-11ef-bc93-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-24T08:52:57.387 DEBUG:teuthology.orchestra.run.smithi069:mon.a> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mon.a.service 2024-04-24T08:52:57.435 DEBUG:teuthology.orchestra.run.smithi069:mgr.x> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mgr.x.service 2024-04-24T08:52:57.482 INFO:tasks.cephadm:Bootstrapping... 2024-04-24T08:52:57.482 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 -v bootstrap --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.69 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-24T08:52:57.762 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:52:57.762 INFO:teuthology.orchestra.run.smithi069.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0', '-v', 'bootstrap', '--fsid', 'ef1ec7e8-0217-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.69', '--skip-admin-label'] 2024-04-24T08:52:57.762 INFO:teuthology.orchestra.run.smithi069.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-24T08:52:57.762 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying podman|docker is present... 2024-04-24T08:52:57.819 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T08:52:57.819 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying lvm2 is present... 2024-04-24T08:52:57.819 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying time synchronization is in place... 2024-04-24T08:52:57.834 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T08:52:57.834 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T08:52:57.850 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T08:52:57.850 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-04-24T08:52:57.865 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout enabled 2024-04-24T08:52:57.880 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout active 2024-04-24T08:52:57.881 INFO:teuthology.orchestra.run.smithi069.stdout:Unit chronyd.service is enabled and running 2024-04-24T08:52:57.881 INFO:teuthology.orchestra.run.smithi069.stdout:Repeating the final host check... 2024-04-24T08:52:57.933 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T08:52:57.933 INFO:teuthology.orchestra.run.smithi069.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-24T08:52:57.933 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl is present 2024-04-24T08:52:57.934 INFO:teuthology.orchestra.run.smithi069.stdout:lvcreate is present 2024-04-24T08:52:57.949 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T08:52:57.949 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T08:52:57.965 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T08:52:57.965 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-04-24T08:52:57.982 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout enabled 2024-04-24T08:52:57.997 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout active 2024-04-24T08:52:57.997 INFO:teuthology.orchestra.run.smithi069.stdout:Unit chronyd.service is enabled and running 2024-04-24T08:52:57.998 INFO:teuthology.orchestra.run.smithi069.stdout:Host looks OK 2024-04-24T08:52:57.998 INFO:teuthology.orchestra.run.smithi069.stdout:Cluster fsid: ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:52:57.998 INFO:teuthology.orchestra.run.smithi069.stdout:Acquiring lock 140265751474128 on /run/cephadm/ef1ec7e8-0217-11ef-bc93-c7b262605968.lock 2024-04-24T08:52:57.998 INFO:teuthology.orchestra.run.smithi069.stdout:Lock 140265751474128 acquired on /run/cephadm/ef1ec7e8-0217-11ef-bc93-c7b262605968.lock 2024-04-24T08:52:57.999 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying IP 172.21.15.69 port 3300 ... 2024-04-24T08:52:57.999 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying IP 172.21.15.69 port 6789 ... 2024-04-24T08:52:57.999 INFO:teuthology.orchestra.run.smithi069.stdout:Base mon IP(s) is [172.21.15.69:3300, 172.21.15.69:6789], mon addrv is [v2:172.21.15.69:3300,v1:172.21.15.69:6789] 2024-04-24T08:52:58.009 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.69 metric 100 2024-04-24T08:52:58.009 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.69 metric 100 2024-04-24T08:52:58.017 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-24T08:52:58.017 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-24T08:52:58.017 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1766sec hoplimit 64 pref medium 2024-04-24T08:52:58.025 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-24T08:52:58.026 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-24T08:52:58.026 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T08:52:58.026 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-24T08:52:58.026 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d737/64 scope link 2024-04-24T08:52:58.026 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T08:52:58.027 INFO:teuthology.orchestra.run.smithi069.stdout:Mon IP `172.21.15.69` is in CIDR network `172.21.0.0/20` 2024-04-24T08:52:58.027 INFO:teuthology.orchestra.run.smithi069.stdout:Mon IP `172.21.15.69` is in CIDR network `172.21.0.0/20` 2024-04-24T08:52:58.027 INFO:teuthology.orchestra.run.smithi069.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-24T08:52:58.027 INFO:teuthology.orchestra.run.smithi069.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-24T08:52:58.029 INFO:teuthology.orchestra.run.smithi069.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0... 2024-04-24T08:53:04.693 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 2024-04-24T08:53:04.693 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0... 2024-04-24T08:53:04.693 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Getting image source signatures 2024-04-24T08:53:04.694 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying blob sha256:b727accf024e55f70f86e348fc139074bb9b3de1cca3400f8b76b2ed7d35ac83 2024-04-24T08:53:04.694 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-24T08:53:04.694 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying config sha256:b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 2024-04-24T08:53:04.694 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-24T08:53:06.178 INFO:teuthology.orchestra.run.smithi069.stdout:ceph: stdout ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev) 2024-04-24T08:53:06.178 INFO:teuthology.orchestra.run.smithi069.stdout:Ceph version: ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev) 2024-04-24T08:53:06.178 INFO:teuthology.orchestra.run.smithi069.stdout:Extracting ceph user uid/gid from container image... 2024-04-24T08:53:07.410 INFO:teuthology.orchestra.run.smithi069.stdout:stat: stdout 167 167 2024-04-24T08:53:07.410 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial keys... 2024-04-24T08:53:08.515 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQBzyChmRQ/RMxAAQMNJ/WDWGrT3F2g1coxavg== 2024-04-24T08:53:09.939 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQB1yChmArYdAxAADKEDsp5ouixPnOQ6J5e6VQ== 2024-04-24T08:53:11.045 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQB2yChmkM20FhAAwqqmsyFF3uXqN7/uQEwr+g== 2024-04-24T08:53:11.049 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial monmap... 2024-04-24T08:53:12.163 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T08:53:12.164 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-24T08:53:12.164 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:53:12.164 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T08:53:12.164 INFO:teuthology.orchestra.run.smithi069.stdout:monmaptool for a [v2:172.21.15.69:3300,v1:172.21.15.69:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T08:53:12.164 INFO:teuthology.orchestra.run.smithi069.stdout:setting min_mon_release = quincy 2024-04-24T08:53:12.164 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: set fsid to ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:53:12.164 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T08:53:12.164 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:53:12.164 INFO:teuthology.orchestra.run.smithi069.stdout:Creating mon... 2024-04-24T08:53:13.285 INFO:teuthology.orchestra.run.smithi069.stdout:create mon.a on 2024-04-24T08:53:13.651 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-24T08:53:13.889 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-24T08:53:14.198 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968.target. 2024-04-24T08:53:14.198 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968.target. 2024-04-24T08:53:14.619 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mon.a 2024-04-24T08:53:14.619 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-24T08:53:14.902 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968.target.wants/ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@.service. 2024-04-24T08:53:15.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:15 smithi069 podman[27780]: 2024-04-24 08:53:15.324935375 +0000 UTC m=+0.149424224 container create 41acb02e8a7cec60311a0fe3237c470643ed67602cd06ac70e632b0729ed9133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:53:15.763 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T08:53:15.763 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-04-24T08:53:15.781 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T08:53:15.781 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-04-24T08:53:15.781 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-04-24T08:53:15.782 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T08:53:15.782 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mon to start... 2024-04-24T08:53:15.782 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mon... 2024-04-24T08:53:15.935 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:15 smithi069 podman[27780]: 2024-04-24 08:53:15.6358966 +0000 UTC m=+0.460385462 container init 41acb02e8a7cec60311a0fe3237c470643ed67602cd06ac70e632b0729ed9133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-24T08:53:15.935 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:15 smithi069 podman[27780]: 2024-04-24 08:53:15.646378448 +0000 UTC m=+0.470867245 container start 41acb02e8a7cec60311a0fe3237c470643ed67602cd06ac70e632b0729ed9133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-24T08:53:15.935 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:15 smithi069 ceph-mon[27799]: mkfs ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:53:15.935 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:15 smithi069 ceph-mon[27799]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T08:53:15.935 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:15 smithi069 bash[27780]: 41acb02e8a7cec60311a0fe3237c470643ed67602cd06ac70e632b0729ed9133 2024-04-24T08:53:15.935 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:15 smithi069 systemd[1]: Started Ceph mon.a for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T08:53:17.015 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:16 smithi069 ceph-mon[27799]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T08:53:17.015 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:16 smithi069 ceph-mon[27799]: monmap epoch 1 2024-04-24T08:53:17.015 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:16 smithi069 ceph-mon[27799]: fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:53:17.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:16 smithi069 ceph-mon[27799]: last_changed 2024-04-24T08:53:11.516571+0000 2024-04-24T08:53:17.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:16 smithi069 ceph-mon[27799]: created 2024-04-24T08:53:11.516571+0000 2024-04-24T08:53:17.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:16 smithi069 ceph-mon[27799]: min_mon_release 19 (squid) 2024-04-24T08:53:17.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:16 smithi069 ceph-mon[27799]: election_strategy: 1 2024-04-24T08:53:17.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:16 smithi069 ceph-mon[27799]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-04-24T08:53:17.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:16 smithi069 ceph-mon[27799]: fsmap 2024-04-24T08:53:17.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:16 smithi069 ceph-mon[27799]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T08:53:17.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:16 smithi069 ceph-mon[27799]: mgrmap e1: no daemons active 2024-04-24T08:53:17.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:16 smithi069 ceph-mon[27799]: from='client.? 172.21.15.69:0/2261206334' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-24T08:53:17.123 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout cluster: 2024-04-24T08:53:17.130 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout id: ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:53:17.130 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-24T08:53:17.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-24T08:53:17.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout services: 2024-04-24T08:53:17.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.751278s) 2024-04-24T08:53:17.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-24T08:53:17.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-24T08:53:17.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-24T08:53:17.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout data: 2024-04-24T08:53:17.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-24T08:53:17.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-24T08:53:17.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-24T08:53:17.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout pgs: 2024-04-24T08:53:17.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-24T08:53:17.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.383+0000 7f24dce00700 1 Processor -- start 2024-04-24T08:53:17.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.383+0000 7f24dce00700 1 -- start start 2024-04-24T08:53:17.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.384+0000 7f24dce00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24d8108de0 0x7f24d810b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:17.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.384+0000 7f24dce00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f24d807bec0 con 0x7f24d8108de0 2024-04-24T08:53:17.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.384+0000 7f24d5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24d8108de0 0x7f24d810b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:17.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.385+0000 7f24d5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24d8108de0 0x7f24d810b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59860/0 (socket says 172.21.15.69:59860) 2024-04-24T08:53:17.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.385+0000 7f24d5e00700 1 -- 172.21.15.69:0/988873680 learned_addr learned my addr 172.21.15.69:0/988873680 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:17.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.385+0000 7f24d5e00700 1 -- 172.21.15.69:0/988873680 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d810b710 con 0x7f24d8108de0 2024-04-24T08:53:17.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.385+0000 7f24d5e00700 1 --2- 172.21.15.69:0/988873680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24d8108de0 0x7f24d810b1d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f24c80098a0 tx=0x7f24c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcddb6788ca1387 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:17.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.386+0000 7f24d4a00700 1 -- 172.21.15.69:0/988873680 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24c8004160 con 0x7f24d8108de0 2024-04-24T08:53:17.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.386+0000 7f24d4a00700 1 -- 172.21.15.69:0/988873680 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c8004300 con 0x7f24d8108de0 2024-04-24T08:53:17.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.386+0000 7f24d4a00700 1 -- 172.21.15.69:0/988873680 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24c80036a0 con 0x7f24d8108de0 2024-04-24T08:53:17.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.386+0000 7f24dce00700 1 -- 172.21.15.69:0/988873680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24d8108de0 msgr2=0x7f24d810b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:17.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.386+0000 7f24dce00700 1 --2- 172.21.15.69:0/988873680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24d8108de0 0x7f24d810b1d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f24c80098a0 tx=0x7f24c8009bb0 comp rx=0 tx=0).stop 2024-04-24T08:53:17.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.386+0000 7f24dce00700 1 -- 172.21.15.69:0/988873680 shutdown_connections 2024-04-24T08:53:17.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.386+0000 7f24dce00700 1 --2- 172.21.15.69:0/988873680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24d8108de0 0x7f24d810b1d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:17.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.386+0000 7f24dce00700 1 -- 172.21.15.69:0/988873680 >> 172.21.15.69:0/988873680 conn(0x7f24d8102870 msgr2=0x7f24d8104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:17.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.387+0000 7f24dce00700 1 -- 172.21.15.69:0/988873680 shutdown_connections 2024-04-24T08:53:17.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.387+0000 7f24dce00700 1 -- 172.21.15.69:0/988873680 wait complete. 2024-04-24T08:53:17.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.388+0000 7f24dce00700 1 Processor -- start 2024-04-24T08:53:17.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.388+0000 7f24dce00700 1 -- start start 2024-04-24T08:53:17.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.389+0000 7f24dce00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24d819e240 0x7f24d819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:17.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.389+0000 7f24dce00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f24d810bbe0 con 0x7f24d819e240 2024-04-24T08:53:17.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.389+0000 7f24d5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24d819e240 0x7f24d819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:17.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.389+0000 7f24d5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24d819e240 0x7f24d819e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59872/0 (socket says 172.21.15.69:59872) 2024-04-24T08:53:17.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.389+0000 7f24d5e00700 1 -- 172.21.15.69:0/2261206334 learned_addr learned my addr 172.21.15.69:0/2261206334 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:17.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.389+0000 7f24d5e00700 1 -- 172.21.15.69:0/2261206334 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d819eba0 con 0x7f24d819e240 2024-04-24T08:53:17.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.390+0000 7f24d5e00700 1 --2- 172.21.15.69:0/2261206334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24d819e240 0x7f24d819e660 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f24c8000c00 tx=0x7f24c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:17.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.390+0000 7f24cea00700 1 -- 172.21.15.69:0/2261206334 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24c8003ca0 con 0x7f24d819e240 2024-04-24T08:53:17.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.390+0000 7f24cea00700 1 -- 172.21.15.69:0/2261206334 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c8003e40 con 0x7f24d819e240 2024-04-24T08:53:17.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.390+0000 7f24dce00700 1 -- 172.21.15.69:0/2261206334 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d819ee30 con 0x7f24d819e240 2024-04-24T08:53:17.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.390+0000 7f24cea00700 1 -- 172.21.15.69:0/2261206334 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24c80117a0 con 0x7f24d819e240 2024-04-24T08:53:17.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.390+0000 7f24dce00700 1 -- 172.21.15.69:0/2261206334 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d81a1910 con 0x7f24d819e240 2024-04-24T08:53:17.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.390+0000 7f24cea00700 1 -- 172.21.15.69:0/2261206334 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f24c8011cd0 con 0x7f24d819e240 2024-04-24T08:53:17.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.390+0000 7f24cea00700 1 -- 172.21.15.69:0/2261206334 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f24c801a7a0 con 0x7f24d819e240 2024-04-24T08:53:17.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.391+0000 7f24dce00700 1 -- 172.21.15.69:0/2261206334 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d8064150 con 0x7f24d819e240 2024-04-24T08:53:17.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.395+0000 7f24cea00700 1 -- 172.21.15.69:0/2261206334 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f24c8006e20 con 0x7f24d819e240 2024-04-24T08:53:17.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.474+0000 7f24dce00700 1 -- 172.21.15.69:0/2261206334 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f24d802dfe0 con 0x7f24d819e240 2024-04-24T08:53:17.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.475+0000 7f24cea00700 1 -- 172.21.15.69:0/2261206334 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f24c80042c0 con 0x7f24d819e240 2024-04-24T08:53:17.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.475+0000 7f24dce00700 1 -- 172.21.15.69:0/2261206334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24d819e240 msgr2=0x7f24d819e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:17.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.476+0000 7f24dce00700 1 --2- 172.21.15.69:0/2261206334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24d819e240 0x7f24d819e660 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f24c8000c00 tx=0x7f24c8004750 comp rx=0 tx=0).stop 2024-04-24T08:53:17.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.476+0000 7f24dce00700 1 -- 172.21.15.69:0/2261206334 shutdown_connections 2024-04-24T08:53:17.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.476+0000 7f24dce00700 1 --2- 172.21.15.69:0/2261206334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24d819e240 0x7f24d819e660 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:17.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.476+0000 7f24dce00700 1 -- 172.21.15.69:0/2261206334 >> 172.21.15.69:0/2261206334 conn(0x7f24d8102870 msgr2=0x7f24d8104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:17.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.476+0000 7f24dce00700 1 -- 172.21.15.69:0/2261206334 shutdown_connections 2024-04-24T08:53:17.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:16.476+0000 7f24dce00700 1 -- 172.21.15.69:0/2261206334 wait complete. 2024-04-24T08:53:17.135 INFO:teuthology.orchestra.run.smithi069.stdout:mon is available 2024-04-24T08:53:17.135 INFO:teuthology.orchestra.run.smithi069.stdout:Assimilating anything we can from ceph.conf... 2024-04-24T08:53:18.212 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:17 smithi069 ceph-mon[27799]: from='client.? 172.21.15.69:0/485679157' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T08:53:18.213 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:17 smithi069 ceph-mon[27799]: from='client.? 172.21.15.69:0/485679157' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-24T08:53:18.533 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-24T08:53:18.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T08:53:18.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout fsid = ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:53:18.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.69:3300,v1:172.21.15.69:6789] 2024-04-24T08:53:18.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T08:53:18.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T08:53:18.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T08:53:18.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T08:53:18.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-24T08:53:18.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T08:53:18.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T08:53:18.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-24T08:53:18.534 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T08:53:18.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T08:53:18.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T08:53:18.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T08:53:18.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T08:53:18.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T08:53:18.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.765+0000 7f8765200700 1 Processor -- start 2024-04-24T08:53:18.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.766+0000 7f8765200700 1 -- start start 2024-04-24T08:53:18.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.766+0000 7f8765200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f876007dd60 0x7f876007c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:18.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.766+0000 7f8765200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f876007e180 con 0x7f876007dd60 2024-04-24T08:53:18.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.766+0000 7f875de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f876007dd60 0x7f876007c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:18.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.766+0000 7f875de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f876007dd60 0x7f876007c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59886/0 (socket says 172.21.15.69:59886) 2024-04-24T08:53:18.535 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.767+0000 7f875de00700 1 -- 172.21.15.69:0/408430310 learned_addr learned my addr 172.21.15.69:0/408430310 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:18.536 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.767+0000 7f875de00700 1 -- 172.21.15.69:0/408430310 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876007c810 con 0x7f876007dd60 2024-04-24T08:53:18.536 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.767+0000 7f875de00700 1 --2- 172.21.15.69:0/408430310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f876007dd60 0x7f876007c2d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f87500098a0 tx=0x7f8750009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa82b5a4acbe6b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:18.536 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.768+0000 7f875d400700 1 -- 172.21.15.69:0/408430310 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8750004160 con 0x7f876007dd60 2024-04-24T08:53:18.536 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.768+0000 7f875d400700 1 -- 172.21.15.69:0/408430310 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8750004300 con 0x7f876007dd60 2024-04-24T08:53:18.536 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.768+0000 7f875d400700 1 -- 172.21.15.69:0/408430310 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87500036a0 con 0x7f876007dd60 2024-04-24T08:53:18.536 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.768+0000 7f8765200700 1 -- 172.21.15.69:0/408430310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f876007dd60 msgr2=0x7f876007c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:18.536 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.768+0000 7f8765200700 1 --2- 172.21.15.69:0/408430310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f876007dd60 0x7f876007c2d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f87500098a0 tx=0x7f8750009bb0 comp rx=0 tx=0).stop 2024-04-24T08:53:18.536 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.768+0000 7f8765200700 1 -- 172.21.15.69:0/408430310 shutdown_connections 2024-04-24T08:53:18.536 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.768+0000 7f8765200700 1 --2- 172.21.15.69:0/408430310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f876007dd60 0x7f876007c2d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:18.536 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.768+0000 7f8765200700 1 -- 172.21.15.69:0/408430310 >> 172.21.15.69:0/408430310 conn(0x7f8760103a40 msgr2=0x7f8760105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:18.536 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.768+0000 7f8765200700 1 -- 172.21.15.69:0/408430310 shutdown_connections 2024-04-24T08:53:18.536 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.768+0000 7f8765200700 1 -- 172.21.15.69:0/408430310 wait complete. 2024-04-24T08:53:18.536 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.769+0000 7f8765200700 1 Processor -- start 2024-04-24T08:53:18.536 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.770+0000 7f8765200700 1 -- start start 2024-04-24T08:53:18.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.770+0000 7f8765200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f876007dd60 0x7f876019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:18.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.770+0000 7f8765200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f876007cce0 con 0x7f876007dd60 2024-04-24T08:53:18.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.770+0000 7f875de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f876007dd60 0x7f876019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:18.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.770+0000 7f875de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f876007dd60 0x7f876019df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59896/0 (socket says 172.21.15.69:59896) 2024-04-24T08:53:18.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.770+0000 7f875de00700 1 -- 172.21.15.69:0/485679157 learned_addr learned my addr 172.21.15.69:0/485679157 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:18.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.771+0000 7f875de00700 1 -- 172.21.15.69:0/485679157 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876019e490 con 0x7f876007dd60 2024-04-24T08:53:18.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.771+0000 7f875de00700 1 --2- 172.21.15.69:0/485679157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f876007dd60 0x7f876019df50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8750000c00 tx=0x7f8750004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:18.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.771+0000 7f8757400700 1 -- 172.21.15.69:0/485679157 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8750003db0 con 0x7f876007dd60 2024-04-24T08:53:18.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.771+0000 7f8757400700 1 -- 172.21.15.69:0/485679157 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8750011630 con 0x7f876007dd60 2024-04-24T08:53:18.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.771+0000 7f8757400700 1 -- 172.21.15.69:0/485679157 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8750011950 con 0x7f876007dd60 2024-04-24T08:53:18.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.771+0000 7f8765200700 1 -- 172.21.15.69:0/485679157 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876019e720 con 0x7f876007dd60 2024-04-24T08:53:18.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.771+0000 7f8765200700 1 -- 172.21.15.69:0/485679157 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87601a1410 con 0x7f876007dd60 2024-04-24T08:53:18.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.772+0000 7f8757400700 1 -- 172.21.15.69:0/485679157 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8750011af0 con 0x7f876007dd60 2024-04-24T08:53:18.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.772+0000 7f8757400700 1 -- 172.21.15.69:0/485679157 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f875001c450 con 0x7f876007dd60 2024-04-24T08:53:18.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.773+0000 7f8765200700 1 -- 172.21.15.69:0/485679157 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8760064150 con 0x7f876007dd60 2024-04-24T08:53:18.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.775+0000 7f8757400700 1 -- 172.21.15.69:0/485679157 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8750032020 con 0x7f876007dd60 2024-04-24T08:53:18.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.851+0000 7f8765200700 1 -- 172.21.15.69:0/485679157 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f87601968e0 con 0x7f876007dd60 2024-04-24T08:53:18.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.859+0000 7f8757400700 1 -- 172.21.15.69:0/485679157 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f8750004260 con 0x7f876007dd60 2024-04-24T08:53:18.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.860+0000 7f8757400700 1 -- 172.21.15.69:0/485679157 <== mon.0 v2:172.21.15.69:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f875001b450 con 0x7f876007dd60 2024-04-24T08:53:18.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.862+0000 7f8765200700 1 -- 172.21.15.69:0/485679157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f876007dd60 msgr2=0x7f876019df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:18.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.862+0000 7f8765200700 1 --2- 172.21.15.69:0/485679157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f876007dd60 0x7f876019df50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8750000c00 tx=0x7f8750004750 comp rx=0 tx=0).stop 2024-04-24T08:53:18.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.862+0000 7f8765200700 1 -- 172.21.15.69:0/485679157 shutdown_connections 2024-04-24T08:53:18.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.862+0000 7f8765200700 1 --2- 172.21.15.69:0/485679157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f876007dd60 0x7f876019df50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:18.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.862+0000 7f8765200700 1 -- 172.21.15.69:0/485679157 >> 172.21.15.69:0/485679157 conn(0x7f8760103a40 msgr2=0x7f8760105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:18.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.862+0000 7f8765200700 1 -- 172.21.15.69:0/485679157 shutdown_connections 2024-04-24T08:53:18.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:17.862+0000 7f8765200700 1 -- 172.21.15.69:0/485679157 wait complete. 2024-04-24T08:53:18.538 INFO:teuthology.orchestra.run.smithi069.stdout:Generating new minimal ceph.conf... 2024-04-24T08:53:19.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:19 smithi069 ceph-mon[27799]: from='client.? 172.21.15.69:0/4028936368' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:53:19.849 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.090+0000 7f9ffd800700 1 Processor -- start 2024-04-24T08:53:19.849 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.090+0000 7f9ffd800700 1 -- start start 2024-04-24T08:53:19.849 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.091+0000 7f9ffd800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ff8108ea0 0x7f9ff810b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:19.850 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.091+0000 7f9ffd800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff807be00 con 0x7f9ff8108ea0 2024-04-24T08:53:19.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.092+0000 7f9ff6a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ff8108ea0 0x7f9ff810b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:19.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.092+0000 7f9ff6a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ff8108ea0 0x7f9ff810b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59908/0 (socket says 172.21.15.69:59908) 2024-04-24T08:53:19.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.092+0000 7f9ff6a00700 1 -- 172.21.15.69:0/490256770 learned_addr learned my addr 172.21.15.69:0/490256770 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:19.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.092+0000 7f9ff6a00700 1 -- 172.21.15.69:0/490256770 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff810b7d0 con 0x7f9ff8108ea0 2024-04-24T08:53:19.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.092+0000 7f9ff6a00700 1 --2- 172.21.15.69:0/490256770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ff8108ea0 0x7f9ff810b290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80098a0 tx=0x7f9fe8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f54332f84bd40 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:19.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.093+0000 7f9ff5600700 1 -- 172.21.15.69:0/490256770 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe8004160 con 0x7f9ff8108ea0 2024-04-24T08:53:19.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.093+0000 7f9ff5600700 1 -- 172.21.15.69:0/490256770 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9fe8004300 con 0x7f9ff8108ea0 2024-04-24T08:53:19.853 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.093+0000 7f9ff5600700 1 -- 172.21.15.69:0/490256770 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe8011a80 con 0x7f9ff8108ea0 2024-04-24T08:53:19.854 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.093+0000 7f9ffd800700 1 -- 172.21.15.69:0/490256770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ff8108ea0 msgr2=0x7f9ff810b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:19.854 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.093+0000 7f9ffd800700 1 --2- 172.21.15.69:0/490256770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ff8108ea0 0x7f9ff810b290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80098a0 tx=0x7f9fe8009bb0 comp rx=0 tx=0).stop 2024-04-24T08:53:19.854 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.093+0000 7f9ffd800700 1 -- 172.21.15.69:0/490256770 shutdown_connections 2024-04-24T08:53:19.854 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.093+0000 7f9ffd800700 1 --2- 172.21.15.69:0/490256770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ff8108ea0 0x7f9ff810b290 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:19.854 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.093+0000 7f9ffd800700 1 -- 172.21.15.69:0/490256770 >> 172.21.15.69:0/490256770 conn(0x7f9ff8102870 msgr2=0x7f9ff8104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:19.854 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.094+0000 7f9ffd800700 1 -- 172.21.15.69:0/490256770 shutdown_connections 2024-04-24T08:53:19.854 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.094+0000 7f9ffd800700 1 -- 172.21.15.69:0/490256770 wait complete. 2024-04-24T08:53:19.854 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.095+0000 7f9ffd800700 1 Processor -- start 2024-04-24T08:53:19.854 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.095+0000 7f9ffd800700 1 -- start start 2024-04-24T08:53:19.854 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.095+0000 7f9ffd800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ff8108ea0 0x7f9ff8108720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:19.854 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.095+0000 7f9ffd800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff810c6e0 con 0x7f9ff8108ea0 2024-04-24T08:53:19.854 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.095+0000 7f9ff6a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ff8108ea0 0x7f9ff8108720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:19.854 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.096+0000 7f9ff6a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ff8108ea0 0x7f9ff8108720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59912/0 (socket says 172.21.15.69:59912) 2024-04-24T08:53:19.855 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.096+0000 7f9ff6a00700 1 -- 172.21.15.69:0/4028936368 learned_addr learned my addr 172.21.15.69:0/4028936368 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:19.855 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.096+0000 7f9ff6a00700 1 -- 172.21.15.69:0/4028936368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff8108c60 con 0x7f9ff8108ea0 2024-04-24T08:53:19.855 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.096+0000 7f9ff6a00700 1 --2- 172.21.15.69:0/4028936368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ff8108ea0 0x7f9ff8108720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8011a50 tx=0x7f9fe8004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:19.855 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.097+0000 7f9fef400700 1 -- 172.21.15.69:0/4028936368 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe80039f0 con 0x7f9ff8108ea0 2024-04-24T08:53:19.855 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.097+0000 7f9fef400700 1 -- 172.21.15.69:0/4028936368 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9fe8003b90 con 0x7f9ff8108ea0 2024-04-24T08:53:19.857 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.097+0000 7f9ffd800700 1 -- 172.21.15.69:0/4028936368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff8106e50 con 0x7f9ff8108ea0 2024-04-24T08:53:19.857 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.097+0000 7f9fef400700 1 -- 172.21.15.69:0/4028936368 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe801a600 con 0x7f9ff8108ea0 2024-04-24T08:53:19.857 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.097+0000 7f9ffd800700 1 -- 172.21.15.69:0/4028936368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff8107390 con 0x7f9ff8108ea0 2024-04-24T08:53:19.857 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.097+0000 7f9fef400700 1 -- 172.21.15.69:0/4028936368 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9fe801ac60 con 0x7f9ff8108ea0 2024-04-24T08:53:19.857 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.097+0000 7f9fef400700 1 -- 172.21.15.69:0/4028936368 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9fe8019a40 con 0x7f9ff8108ea0 2024-04-24T08:53:19.857 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.098+0000 7f9ffd800700 1 -- 172.21.15.69:0/4028936368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff8064150 con 0x7f9ff8108ea0 2024-04-24T08:53:19.857 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.101+0000 7f9fef400700 1 -- 172.21.15.69:0/4028936368 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9fe8024a40 con 0x7f9ff8108ea0 2024-04-24T08:53:19.857 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.178+0000 7f9ffd800700 1 -- 172.21.15.69:0/4028936368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f9ff8195c40 con 0x7f9ff8108ea0 2024-04-24T08:53:19.857 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.179+0000 7f9fef400700 1 -- 172.21.15.69:0/4028936368 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f9fe8019c40 con 0x7f9ff8108ea0 2024-04-24T08:53:19.857 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.180+0000 7f9ffd800700 1 -- 172.21.15.69:0/4028936368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ff8108ea0 msgr2=0x7f9ff8108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:19.857 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.180+0000 7f9ffd800700 1 --2- 172.21.15.69:0/4028936368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ff8108ea0 0x7f9ff8108720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8011a50 tx=0x7f9fe8004290 comp rx=0 tx=0).stop 2024-04-24T08:53:19.857 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.181+0000 7f9ffd800700 1 -- 172.21.15.69:0/4028936368 shutdown_connections 2024-04-24T08:53:19.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.181+0000 7f9ffd800700 1 --2- 172.21.15.69:0/4028936368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ff8108ea0 0x7f9ff8108720 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:19.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.181+0000 7f9ffd800700 1 -- 172.21.15.69:0/4028936368 >> 172.21.15.69:0/4028936368 conn(0x7f9ff8102870 msgr2=0x7f9ff8103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:19.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.181+0000 7f9ffd800700 1 -- 172.21.15.69:0/4028936368 shutdown_connections 2024-04-24T08:53:19.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:19.181+0000 7f9ffd800700 1 -- 172.21.15.69:0/4028936368 wait complete. 2024-04-24T08:53:19.858 INFO:teuthology.orchestra.run.smithi069.stdout:Restarting the monitor... 2024-04-24T08:53:20.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:19 smithi069 systemd[1]: Stopping Ceph mon.a for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T08:53:20.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:20 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a[27795]: 2024-04-24T08:53:20.661+0000 7fcf20c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:53:20.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:20 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a[27795]: 2024-04-24T08:53:20.661+0000 7fcf20c00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-24T08:53:20.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:20 smithi069 podman[28042]: 2024-04-24 08:53:20.714102291 +0000 UTC m=+0.539378168 container died 41acb02e8a7cec60311a0fe3237c470643ed67602cd06ac70e632b0729ed9133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T08:53:21.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:21 smithi069 podman[28042]: 2024-04-24 08:53:21.095691285 +0000 UTC m=+0.920967152 container cleanup 41acb02e8a7cec60311a0fe3237c470643ed67602cd06ac70e632b0729ed9133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T08:53:21.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:21 smithi069 bash[28042]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a 2024-04-24T08:53:21.630 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:21 smithi069 podman[28056]: 2024-04-24 08:53:21.37485961 +0000 UTC m=+0.651643770 container remove 41acb02e8a7cec60311a0fe3237c470643ed67602cd06ac70e632b0729ed9133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T08:53:22.060 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:21 smithi069 systemd[1]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-24T08:53:22.060 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:21 smithi069 systemd[1]: Stopped Ceph mon.a for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T08:53:22.060 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:21 smithi069 systemd[1]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.750s CPU time. 2024-04-24T08:53:22.060 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:21 smithi069 systemd[1]: Starting Ceph mon.a for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T08:53:22.325 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 podman[28150]: 2024-04-24 08:53:21.967930648 +0000 UTC m=+0.037766834 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:53:22.325 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 podman[28150]: 2024-04-24 08:53:22.080490719 +0000 UTC m=+0.150326926 container create 9d225ee88aa4ed83985c0a2a1dea31c2fcea164d865bdd996e6f8b6033070096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T08:53:22.325 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 podman[28150]: 2024-04-24 08:53:22.325187293 +0000 UTC m=+0.395023506 container init 9d225ee88aa4ed83985c0a2a1dea31c2fcea164d865bdd996e6f8b6033070096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T08:53:22.445 INFO:teuthology.orchestra.run.smithi069.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-24T08:53:22.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 podman[28150]: 2024-04-24 08:53:22.335996061 +0000 UTC m=+0.405832274 container start 9d225ee88aa4ed83985c0a2a1dea31c2fcea164d865bdd996e6f8b6033070096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-24T08:53:22.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T08:53:22.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev), process ceph-mon, pid 2 2024-04-24T08:53:22.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: pidfile_write: ignore empty --pid-file 2024-04-24T08:53:22.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: load: jerasure load: lrc 2024-04-24T08:53:22.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: RocksDB version: 7.9.2 2024-04-24T08:53:22.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Git sha 0 2024-04-24T08:53:22.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Compile date 2024-04-23 21:06:33 2024-04-24T08:53:22.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: DB SUMMARY 2024-04-24T08:53:22.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: DB Session ID: 6Q3QE4ZJFZ5Y84IOD2PF 2024-04-24T08:53:22.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: CURRENT file: CURRENT 2024-04-24T08:53:22.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T08:53:22.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-24T08:53:22.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-24T08:53:22.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104508 ; 2024-04-24T08:53:22.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.error_if_exists: 0 2024-04-24T08:53:22.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.create_if_missing: 0 2024-04-24T08:53:22.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.paranoid_checks: 1 2024-04-24T08:53:22.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T08:53:22.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T08:53:22.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T08:53:22.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.env: 0x564aa817acc0 2024-04-24T08:53:22.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T08:53:22.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.info_log: 0x564ac3bf5c60 2024-04-24T08:53:22.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T08:53:22.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.statistics: (nil) 2024-04-24T08:53:22.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.use_fsync: 0 2024-04-24T08:53:22.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_log_file_size: 0 2024-04-24T08:53:22.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T08:53:22.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T08:53:22.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T08:53:22.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T08:53:22.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.allow_fallocate: 1 2024-04-24T08:53:22.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T08:53:22.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T08:53:22.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.use_direct_reads: 0 2024-04-24T08:53:22.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T08:53:22.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T08:53:22.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.db_log_dir: 2024-04-24T08:53:22.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.wal_dir: 2024-04-24T08:53:22.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T08:53:22.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T08:53:22.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T08:53:22.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T08:53:22.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T08:53:22.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T08:53:22.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T08:53:22.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T08:53:22.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.write_buffer_manager: 0x564ac3bc41e0 2024-04-24T08:53:22.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T08:53:22.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T08:53:22.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T08:53:22.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T08:53:22.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T08:53:22.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T08:53:22.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T08:53:22.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T08:53:22.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.unordered_write: 0 2024-04-24T08:53:22.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T08:53:22.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T08:53:22.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T08:53:22.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T08:53:22.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.row_cache: None 2024-04-24T08:53:22.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.wal_filter: None 2024-04-24T08:53:22.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T08:53:22.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T08:53:22.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.two_write_queues: 0 2024-04-24T08:53:22.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T08:53:22.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.wal_compression: 0 2024-04-24T08:53:22.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.atomic_flush: 0 2024-04-24T08:53:22.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T08:53:22.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T08:53:22.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T08:53:22.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.log_readahead_size: 0 2024-04-24T08:53:22.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T08:53:22.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T08:53:22.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T08:53:22.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T08:53:22.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T08:53:22.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T08:53:22.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T08:53:22.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_background_jobs: 2 2024-04-24T08:53:22.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_background_compactions: -1 2024-04-24T08:53:22.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_subcompactions: 1 2024-04-24T08:53:22.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T08:53:22.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T08:53:22.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T08:53:22.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T08:53:22.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T08:53:22.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T08:53:22.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T08:53:22.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T08:53:22.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_open_files: -1 2024-04-24T08:53:22.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T08:53:22.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T08:53:22.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T08:53:22.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T08:53:22.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_background_flushes: -1 2024-04-24T08:53:22.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Compression algorithms supported: 2024-04-24T08:53:22.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T08:53:22.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: kZSTD supported: 0 2024-04-24T08:53:22.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: kXpressCompression supported: 0 2024-04-24T08:53:22.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T08:53:22.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: kZlibCompression supported: 1 2024-04-24T08:53:22.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: kSnappyCompression supported: 1 2024-04-24T08:53:22.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: kLZ4Compression supported: 1 2024-04-24T08:53:22.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: kBZip2Compression supported: 0 2024-04-24T08:53:22.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T08:53:22.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T08:53:22.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-24T08:53:22.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T08:53:22.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T08:53:22.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.merge_operator: 2024-04-24T08:53:22.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compaction_filter: None 2024-04-24T08:53:22.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T08:53:22.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T08:53:22.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T08:53:22.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T08:53:22.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564ac3bf5e20) 2024-04-24T08:53:22.602 INFO:journalctl@ceph.mon.a.smithi069.stdout: cache_index_and_filter_blocks: 1 2024-04-24T08:53:22.602 INFO:journalctl@ceph.mon.a.smithi069.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T08:53:22.602 INFO:journalctl@ceph.mon.a.smithi069.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T08:53:22.602 INFO:journalctl@ceph.mon.a.smithi069.stdout: pin_top_level_index_and_filter: 1 2024-04-24T08:53:22.602 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_type: 0 2024-04-24T08:53:22.602 INFO:journalctl@ceph.mon.a.smithi069.stdout: data_block_index_type: 0 2024-04-24T08:53:22.603 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_shortening: 1 2024-04-24T08:53:22.603 INFO:journalctl@ceph.mon.a.smithi069.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T08:53:22.603 INFO:journalctl@ceph.mon.a.smithi069.stdout: checksum: 4 2024-04-24T08:53:22.603 INFO:journalctl@ceph.mon.a.smithi069.stdout: no_block_cache: 0 2024-04-24T08:53:22.603 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache: 0x564ac3bf7090 2024-04-24T08:53:22.603 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_name: BinnedLRUCache 2024-04-24T08:53:22.603 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_options: 2024-04-24T08:53:22.603 INFO:journalctl@ceph.mon.a.smithi069.stdout: capacity : 536870912 2024-04-24T08:53:22.603 INFO:journalctl@ceph.mon.a.smithi069.stdout: num_shard_bits : 4 2024-04-24T08:53:22.603 INFO:journalctl@ceph.mon.a.smithi069.stdout: strict_capacity_limit : 0 2024-04-24T08:53:22.603 INFO:journalctl@ceph.mon.a.smithi069.stdout: high_pri_pool_ratio: 0.000 2024-04-24T08:53:22.603 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_compressed: (nil) 2024-04-24T08:53:22.603 INFO:journalctl@ceph.mon.a.smithi069.stdout: persistent_cache: (nil) 2024-04-24T08:53:22.603 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_size: 4096 2024-04-24T08:53:22.603 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_size_deviation: 10 2024-04-24T08:53:22.604 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_restart_interval: 16 2024-04-24T08:53:22.604 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_block_restart_interval: 1 2024-04-24T08:53:22.604 INFO:journalctl@ceph.mon.a.smithi069.stdout: metadata_block_size: 4096 2024-04-24T08:53:22.604 INFO:journalctl@ceph.mon.a.smithi069.stdout: partition_filters: 0 2024-04-24T08:53:22.604 INFO:journalctl@ceph.mon.a.smithi069.stdout: use_delta_encoding: 1 2024-04-24T08:53:22.604 INFO:journalctl@ceph.mon.a.smithi069.stdout: filter_policy: bloomfilter 2024-04-24T08:53:22.604 INFO:journalctl@ceph.mon.a.smithi069.stdout: whole_key_filtering: 1 2024-04-24T08:53:22.604 INFO:journalctl@ceph.mon.a.smithi069.stdout: verify_compression: 0 2024-04-24T08:53:22.604 INFO:journalctl@ceph.mon.a.smithi069.stdout: read_amp_bytes_per_bit: 0 2024-04-24T08:53:22.604 INFO:journalctl@ceph.mon.a.smithi069.stdout: format_version: 5 2024-04-24T08:53:22.604 INFO:journalctl@ceph.mon.a.smithi069.stdout: enable_index_compression: 1 2024-04-24T08:53:22.604 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_align: 0 2024-04-24T08:53:22.604 INFO:journalctl@ceph.mon.a.smithi069.stdout: max_auto_readahead_size: 262144 2024-04-24T08:53:22.604 INFO:journalctl@ceph.mon.a.smithi069.stdout: prepopulate_block_cache: 0 2024-04-24T08:53:22.605 INFO:journalctl@ceph.mon.a.smithi069.stdout: initial_auto_readahead_size: 8192 2024-04-24T08:53:22.605 INFO:journalctl@ceph.mon.a.smithi069.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T08:53:22.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T08:53:22.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T08:53:22.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compression: NoCompression 2024-04-24T08:53:22.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T08:53:22.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T08:53:22.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T08:53:22.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.num_levels: 7 2024-04-24T08:53:22.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T08:53:22.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T08:53:22.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T08:53:22.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T08:53:22.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T08:53:22.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T08:53:22.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T08:53:22.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:53:22.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T08:53:22.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T08:53:22.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:53:22.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:53:22.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T08:53:22.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T08:53:22.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T08:53:22.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T08:53:22.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:53:22.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:53:22.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T08:53:22.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T08:53:22.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:53:22.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T08:53:22.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T08:53:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T08:53:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T08:53:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T08:53:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T08:53:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T08:53:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T08:53:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T08:53:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T08:53:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T08:53:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T08:53:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T08:53:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T08:53:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T08:53:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T08:53:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T08:53:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T08:53:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T08:53:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T08:53:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T08:53:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T08:53:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T08:53:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T08:53:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T08:53:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T08:53:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T08:53:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T08:53:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T08:53:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T08:53:22.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T08:53:22.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T08:53:22.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-24T08:53:22.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.inplace_update_support: 0 2024-04-24T08:53:22.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T08:53:22.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T08:53:22.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T08:53:22.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T08:53:22.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.bloom_locality: 0 2024-04-24T08:53:22.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.max_successive_merges: 0 2024-04-24T08:53:22.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T08:53:22.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T08:53:22.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T08:53:22.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T08:53:22.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.ttl: 2592000 2024-04-24T08:53:22.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T08:53:22.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T08:53:22.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T08:53:22.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.enable_blob_files: false 2024-04-24T08:53:22.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.min_blob_size: 0 2024-04-24T08:53:22.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T08:53:22.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T08:53:22.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T08:53:22.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T08:53:22.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T08:53:22.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T08:53:22.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T08:53:22.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T08:53:22.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-24T08:53:22.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-24T08:53:22.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 80b389ed-346a-4b8f-9e5c-aac0bd36fcec 2024-04-24T08:53:22.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713948802391716, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-24T08:53:22.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-24T08:53:22.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713948802393527, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99759, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97648, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14032, "raw_average_key_size": 49, "raw_value_size": 89971, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713948802, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "80b389ed-346a-4b8f-9e5c-aac0bd36fcec", "db_session_id": "6Q3QE4ZJFZ5Y84IOD2PF", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-24T08:53:22.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713948802393777, "job": 1, "event": "recovery_finished"} 2024-04-24T08:53:22.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-24T08:53:22.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T08:53:22.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564ac3ce4000 2024-04-24T08:53:22.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: DB pointer 0x564ac3cce000 2024-04-24T08:53:22.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T08:53:22.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T08:53:22.612 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** DB Stats ** 2024-04-24T08:53:22.612 INFO:journalctl@ceph.mon.a.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:53:22.612 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T08:53:22.613 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:53:22.613 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:53:22.613 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T08:53:22.613 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:53:22.613 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:53:22.613 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-04-24T08:53:22.613 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** Compaction Stats [default] ** 2024-04-24T08:53:22.613 INFO:journalctl@ceph.mon.a.smithi069.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:53:22.613 INFO:journalctl@ceph.mon.a.smithi069.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T08:53:22.613 INFO:journalctl@ceph.mon.a.smithi069.stdout: L0 2/0 99.28 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:53:22.613 INFO:journalctl@ceph.mon.a.smithi069.stdout: Sum 2/0 99.28 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:53:22.613 INFO:journalctl@ceph.mon.a.smithi069.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:53:22.613 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-04-24T08:53:22.613 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** Compaction Stats [default] ** 2024-04-24T08:53:22.614 INFO:journalctl@ceph.mon.a.smithi069.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:53:22.614 INFO:journalctl@ceph.mon.a.smithi069.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T08:53:22.614 INFO:journalctl@ceph.mon.a.smithi069.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 82.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:53:22.614 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-04-24T08:53:22.614 INFO:journalctl@ceph.mon.a.smithi069.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T08:53:22.614 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-04-24T08:53:22.614 INFO:journalctl@ceph.mon.a.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:53:22.614 INFO:journalctl@ceph.mon.a.smithi069.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T08:53:22.614 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T08:53:22.614 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T08:53:22.614 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T08:53:22.614 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T08:53:22.614 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative compaction: 0.00 GB write, 13.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:53:22.614 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval compaction: 0.00 GB write, 13.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:53:22.614 INFO:journalctl@ceph.mon.a.smithi069.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T08:53:22.615 INFO:journalctl@ceph.mon.a.smithi069.stdout: Block cache BinnedLRUCache@0x564ac3bf7090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-24T08:53:22.615 INFO:journalctl@ceph.mon.a.smithi069.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-24T08:53:22.615 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-04-24T08:53:22.615 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T08:53:22.615 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: starting mon.a rank 0 at public addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] at bind addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:53:22.615 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: mon.a@-1(???) e1 preinit fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:53:22.615 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: mon.a@-1(???).mds e1 new map 2024-04-24T08:53:22.615 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: mon.a@-1(???).mds e1 print_map 2024-04-24T08:53:22.615 INFO:journalctl@ceph.mon.a.smithi069.stdout: e1 2024-04-24T08:53:22.615 INFO:journalctl@ceph.mon.a.smithi069.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:53:22.615 INFO:journalctl@ceph.mon.a.smithi069.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T08:53:22.615 INFO:journalctl@ceph.mon.a.smithi069.stdout: legacy client fscid: -1 2024-04-24T08:53:22.615 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-04-24T08:53:22.616 INFO:journalctl@ceph.mon.a.smithi069.stdout: No filesystems configured 2024-04-24T08:53:22.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T08:53:22.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:53:22.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:53:22.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:53:22.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-24T08:53:22.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 bash[28150]: 9d225ee88aa4ed83985c0a2a1dea31c2fcea164d865bdd996e6f8b6033070096 2024-04-24T08:53:22.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 systemd[1]: Started Ceph mon.a for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T08:53:22.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T08:53:22.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: monmap epoch 1 2024-04-24T08:53:22.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:53:22.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: last_changed 2024-04-24T08:53:11.516571+0000 2024-04-24T08:53:22.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: created 2024-04-24T08:53:11.516571+0000 2024-04-24T08:53:22.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: min_mon_release 19 (squid) 2024-04-24T08:53:22.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: election_strategy: 1 2024-04-24T08:53:22.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-04-24T08:53:22.617 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: fsmap 2024-04-24T08:53:22.617 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T08:53:22.617 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:22 smithi069 ceph-mon[28170]: mgrmap e1: no daemons active 2024-04-24T08:53:24.104 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.315+0000 7f3ee5600700 1 Processor -- start 2024-04-24T08:53:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.316+0000 7f3ee5600700 1 -- start start 2024-04-24T08:53:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.316+0000 7f3ee5600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ee01081f0 0x7f3ee0108610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.316+0000 7f3ee5600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee0108b50 con 0x7f3ee01081f0 2024-04-24T08:53:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.317+0000 7f3edfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ee01081f0 0x7f3ee0108610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.317+0000 7f3edfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ee01081f0 0x7f3ee0108610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56600/0 (socket says 172.21.15.69:56600) 2024-04-24T08:53:24.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.317+0000 7f3edfe00700 1 -- 172.21.15.69:0/1777681155 learned_addr learned my addr 172.21.15.69:0/1777681155 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.317+0000 7f3edfe00700 1 -- 172.21.15.69:0/1777681155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee0108cd0 con 0x7f3ee01081f0 2024-04-24T08:53:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.318+0000 7f3edfe00700 1 --2- 172.21.15.69:0/1777681155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ee01081f0 0x7f3ee0108610 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00098a0 tx=0x7f3ed0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09abee99c706485 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.318+0000 7f3edea00700 1 -- 172.21.15.69:0/1777681155 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ed0004160 con 0x7f3ee01081f0 2024-04-24T08:53:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.318+0000 7f3edea00700 1 -- 172.21.15.69:0/1777681155 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3ed0004300 con 0x7f3ee01081f0 2024-04-24T08:53:24.107 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.318+0000 7f3edea00700 1 -- 172.21.15.69:0/1777681155 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ed0011a80 con 0x7f3ee01081f0 2024-04-24T08:53:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.319+0000 7f3ee5600700 1 -- 172.21.15.69:0/1777681155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ee01081f0 msgr2=0x7f3ee0108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.319+0000 7f3ee5600700 1 --2- 172.21.15.69:0/1777681155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ee01081f0 0x7f3ee0108610 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00098a0 tx=0x7f3ed0009bb0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.319+0000 7f3ee5600700 1 -- 172.21.15.69:0/1777681155 shutdown_connections 2024-04-24T08:53:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.319+0000 7f3ee5600700 1 --2- 172.21.15.69:0/1777681155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ee01081f0 0x7f3ee0108610 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.319+0000 7f3ee5600700 1 -- 172.21.15.69:0/1777681155 >> 172.21.15.69:0/1777681155 conn(0x7f3ee0103870 msgr2=0x7f3ee0105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.319+0000 7f3ee5600700 1 -- 172.21.15.69:0/1777681155 shutdown_connections 2024-04-24T08:53:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.319+0000 7f3ee5600700 1 -- 172.21.15.69:0/1777681155 wait complete. 2024-04-24T08:53:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.320+0000 7f3ee5600700 1 Processor -- start 2024-04-24T08:53:24.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.320+0000 7f3ee5600700 1 -- start start 2024-04-24T08:53:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.320+0000 7f3ee5600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ee01081f0 0x7f3ee019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.321+0000 7f3ee5600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee010a1a0 con 0x7f3ee01081f0 2024-04-24T08:53:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.321+0000 7f3edfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ee01081f0 0x7f3ee019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.321+0000 7f3edfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ee01081f0 0x7f3ee019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56604/0 (socket says 172.21.15.69:56604) 2024-04-24T08:53:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.321+0000 7f3edfe00700 1 -- 172.21.15.69:0/800159195 learned_addr learned my addr 172.21.15.69:0/800159195 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.321+0000 7f3edfe00700 1 -- 172.21.15.69:0/800159195 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee019eb50 con 0x7f3ee01081f0 2024-04-24T08:53:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.322+0000 7f3edfe00700 1 --2- 172.21.15.69:0/800159195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ee01081f0 0x7f3ee019e610 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0011a50 tx=0x7f3ed0011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.322+0000 7f3edcc00700 1 -- 172.21.15.69:0/800159195 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ed0003a80 con 0x7f3ee01081f0 2024-04-24T08:53:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.322+0000 7f3edcc00700 1 -- 172.21.15.69:0/800159195 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3ed0003c20 con 0x7f3ee01081f0 2024-04-24T08:53:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.322+0000 7f3edcc00700 1 -- 172.21.15.69:0/800159195 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ed001a650 con 0x7f3ee01081f0 2024-04-24T08:53:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.322+0000 7f3ee5600700 1 -- 172.21.15.69:0/800159195 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee019ede0 con 0x7f3ee01081f0 2024-04-24T08:53:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.322+0000 7f3ee5600700 1 -- 172.21.15.69:0/800159195 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee01a1a80 con 0x7f3ee01081f0 2024-04-24T08:53:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.322+0000 7f3edcc00700 1 -- 172.21.15.69:0/800159195 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3ed001ace0 con 0x7f3ee01081f0 2024-04-24T08:53:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.323+0000 7f3edcc00700 1 -- 172.21.15.69:0/800159195 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3ed0019700 con 0x7f3ee01081f0 2024-04-24T08:53:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.323+0000 7f3ee5600700 1 -- 172.21.15.69:0/800159195 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee0064150 con 0x7f3ee01081f0 2024-04-24T08:53:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.326+0000 7f3edcc00700 1 -- 172.21.15.69:0/800159195 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3ed0024ad0 con 0x7f3ee01081f0 2024-04-24T08:53:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.402+0000 7f3ee5600700 1 -- 172.21.15.69:0/800159195 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f3ee0076490 con 0x7f3ee01081f0 2024-04-24T08:53:24.113 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.407+0000 7f3edcc00700 1 -- 172.21.15.69:0/800159195 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f3ed0024440 con 0x7f3ee01081f0 2024-04-24T08:53:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.408+0000 7f3ee5600700 1 -- 172.21.15.69:0/800159195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ee01081f0 msgr2=0x7f3ee019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.408+0000 7f3ee5600700 1 --2- 172.21.15.69:0/800159195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ee01081f0 0x7f3ee019e610 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0011a50 tx=0x7f3ed0011b30 comp rx=0 tx=0).stop 2024-04-24T08:53:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.408+0000 7f3ee5600700 1 -- 172.21.15.69:0/800159195 shutdown_connections 2024-04-24T08:53:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.408+0000 7f3ee5600700 1 --2- 172.21.15.69:0/800159195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ee01081f0 0x7f3ee019e610 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.408+0000 7f3ee5600700 1 -- 172.21.15.69:0/800159195 >> 172.21.15.69:0/800159195 conn(0x7f3ee0103870 msgr2=0x7f3ee01045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.409+0000 7f3ee5600700 1 -- 172.21.15.69:0/800159195 shutdown_connections 2024-04-24T08:53:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:23.409+0000 7f3ee5600700 1 -- 172.21.15.69:0/800159195 wait complete. 2024-04-24T08:53:24.114 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-24T08:53:24.138 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-24T08:53:24.139 INFO:teuthology.orchestra.run.smithi069.stdout:Creating mgr... 2024-04-24T08:53:24.140 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-24T08:53:24.141 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-24T08:53:24.595 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mgr.x 2024-04-24T08:53:24.595 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-24T08:53:24.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:24 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/800159195' entity='client.admin' 2024-04-24T08:53:24.864 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968.target.wants/ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@.service. 2024-04-24T08:53:25.257 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:24 smithi069 systemd[1]: Starting Ceph mgr.x for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T08:53:25.521 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:25 smithi069 podman[28397]: 2024-04-24 08:53:25.162680162 +0000 UTC m=+0.033698090 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:53:25.521 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:25 smithi069 podman[28397]: 2024-04-24 08:53:25.267256641 +0000 UTC m=+0.138274626 container create 6e4c09281b32c4f9ae5b210f95e3421b58941d23a09aaf2f14dbea502b1e9a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-24T08:53:25.521 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:25 smithi069 podman[28397]: 2024-04-24 08:53:25.520463106 +0000 UTC m=+0.391481061 container init 6e4c09281b32c4f9ae5b210f95e3421b58941d23a09aaf2f14dbea502b1e9a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-24T08:53:25.623 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T08:53:25.623 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-04-24T08:53:25.638 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T08:53:25.638 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-04-24T08:53:25.638 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-04-24T08:53:25.638 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T08:53:25.656 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T08:53:25.656 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-04-24T08:53:25.671 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T08:53:25.671 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-04-24T08:53:25.671 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-04-24T08:53:25.671 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-24T08:53:25.671 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr to start... 2024-04-24T08:53:25.671 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr... 2024-04-24T08:53:25.819 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:25 smithi069 podman[28397]: 2024-04-24 08:53:25.532228214 +0000 UTC m=+0.403246147 container start 6e4c09281b32c4f9ae5b210f95e3421b58941d23a09aaf2f14dbea502b1e9a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424) 2024-04-24T08:53:25.820 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:25 smithi069 bash[28397]: 6e4c09281b32c4f9ae5b210f95e3421b58941d23a09aaf2f14dbea502b1e9a0b 2024-04-24T08:53:25.820 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:25 smithi069 systemd[1]: Started Ceph mgr.x for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T08:53:25.820 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:25 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:25.767+0000 7f7767fd6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T08:53:26.092 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:26 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:26.026+0000 7f7767fd6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:53:26.381 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:26 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:26.167+0000 7f7767fd6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T08:53:26.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:26 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3849434538' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T08:53:26.699 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:26 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:26.582+0000 7f7767fd6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T08:53:26.700 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:26 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:26.698+0000 7f7767fd6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T08:53:27.080 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-24T08:53:27.080 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:53:27.080 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "ef1ec7e8-0217-11ef-bc93-c7b262605968", 2024-04-24T08:53:27.080 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T08:53:27.080 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T08:53:27.080 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T08:53:27.080 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T08:53:27.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:27.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T08:53:27.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T08:53:27.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-04-24T08:53:27.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:53:27.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T08:53:27.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T08:53:27.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:53:27.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-24T08:53:27.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T08:53:27.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:53:27.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T08:53:27.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T08:53:27.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:27.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T08:53:27.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:53:27.082 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T08:53:27.082 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T08:53:27.082 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T08:53:27.090 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T08:53:27.091 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T08:53:27.091 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T08:53:27.091 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:27.091 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T08:53:27.091 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T08:53:27.091 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T08:53:27.091 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T08:53:27.091 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T08:53:27.091 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T08:53:27.091 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T08:53:27.091 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T08:53:27.091 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T08:53:27.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:27.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T08:53:27.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:53:27.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T08:53:27.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T08:53:27.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:27.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T08:53:27.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T08:53:27.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T08:53:27.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T08:53:27.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T08:53:27.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T08:53:27.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T08:53:27.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:53:27.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:53:27.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:27.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T08:53:27.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:53:27.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T08:53:15.726448+0000", 2024-04-24T08:53:27.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:53:27.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:27.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T08:53:27.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:53:27.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.285+0000 7fb35ee00700 1 Processor -- start 2024-04-24T08:53:27.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.285+0000 7fb35ee00700 1 -- start start 2024-04-24T08:53:27.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.286+0000 7fb35ee00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb35807dd80 0x7fb35807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.286+0000 7fb35ee00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb35807c830 con 0x7fb35807dd80 2024-04-24T08:53:27.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.286+0000 7fb357e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb35807dd80 0x7fb35807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.286+0000 7fb357e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb35807dd80 0x7fb35807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56630/0 (socket says 172.21.15.69:56630) 2024-04-24T08:53:27.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.286+0000 7fb357e00700 1 -- 172.21.15.69:0/3734857007 learned_addr learned my addr 172.21.15.69:0/3734857007 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:27.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.287+0000 7fb357e00700 1 -- 172.21.15.69:0/3734857007 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35807c9b0 con 0x7fb35807dd80 2024-04-24T08:53:27.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.287+0000 7fb357e00700 1 --2- 172.21.15.69:0/3734857007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb35807dd80 0x7fb35807c2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb3480098a0 tx=0x7fb348009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c0da682584190a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:27.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.287+0000 7fb356a00700 1 -- 172.21.15.69:0/3734857007 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb348004160 con 0x7fb35807dd80 2024-04-24T08:53:27.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.287+0000 7fb356a00700 1 -- 172.21.15.69:0/3734857007 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb348004300 con 0x7fb35807dd80 2024-04-24T08:53:27.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.287+0000 7fb356a00700 1 -- 172.21.15.69:0/3734857007 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb348011a80 con 0x7fb35807dd80 2024-04-24T08:53:27.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.288+0000 7fb35ee00700 1 -- 172.21.15.69:0/3734857007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb35807dd80 msgr2=0x7fb35807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.288+0000 7fb35ee00700 1 --2- 172.21.15.69:0/3734857007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb35807dd80 0x7fb35807c2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb3480098a0 tx=0x7fb348009bb0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.288+0000 7fb35ee00700 1 -- 172.21.15.69:0/3734857007 shutdown_connections 2024-04-24T08:53:27.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.288+0000 7fb35ee00700 1 --2- 172.21.15.69:0/3734857007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb35807dd80 0x7fb35807c2f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.288+0000 7fb35ee00700 1 -- 172.21.15.69:0/3734857007 >> 172.21.15.69:0/3734857007 conn(0x7fb358103a40 msgr2=0x7fb358105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:27.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.289+0000 7fb35ee00700 1 -- 172.21.15.69:0/3734857007 shutdown_connections 2024-04-24T08:53:27.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.289+0000 7fb35ee00700 1 -- 172.21.15.69:0/3734857007 wait complete. 2024-04-24T08:53:27.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.290+0000 7fb35ee00700 1 Processor -- start 2024-04-24T08:53:27.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.290+0000 7fb35ee00700 1 -- start start 2024-04-24T08:53:27.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.290+0000 7fb35ee00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb35807dd80 0x7fb35819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.290+0000 7fb35ee00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb35810e810 con 0x7fb35807dd80 2024-04-24T08:53:27.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.290+0000 7fb357e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb35807dd80 0x7fb35819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.290+0000 7fb357e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb35807dd80 0x7fb35819e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56640/0 (socket says 172.21.15.69:56640) 2024-04-24T08:53:27.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.290+0000 7fb357e00700 1 -- 172.21.15.69:0/3849434538 learned_addr learned my addr 172.21.15.69:0/3849434538 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:27.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.291+0000 7fb357e00700 1 -- 172.21.15.69:0/3849434538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35819ed10 con 0x7fb35807dd80 2024-04-24T08:53:27.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.291+0000 7fb357e00700 1 --2- 172.21.15.69:0/3849434538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb35807dd80 0x7fb35819e7d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb348011a50 tx=0x7fb348011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:27.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.291+0000 7fb354c00700 1 -- 172.21.15.69:0/3849434538 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb348003a80 con 0x7fb35807dd80 2024-04-24T08:53:27.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.291+0000 7fb354c00700 1 -- 172.21.15.69:0/3849434538 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb348003c20 con 0x7fb35807dd80 2024-04-24T08:53:27.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.291+0000 7fb354c00700 1 -- 172.21.15.69:0/3849434538 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb34801a650 con 0x7fb35807dd80 2024-04-24T08:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.291+0000 7fb35ee00700 1 -- 172.21.15.69:0/3849434538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35819efa0 con 0x7fb35807dd80 2024-04-24T08:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.291+0000 7fb35ee00700 1 -- 172.21.15.69:0/3849434538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3581a1c90 con 0x7fb35807dd80 2024-04-24T08:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.292+0000 7fb354c00700 1 -- 172.21.15.69:0/3849434538 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb34801ac90 con 0x7fb35807dd80 2024-04-24T08:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.292+0000 7fb354c00700 1 -- 172.21.15.69:0/3849434538 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb3480193c0 con 0x7fb35807dd80 2024-04-24T08:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.293+0000 7fb35ee00700 1 -- 172.21.15.69:0/3849434538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb358064150 con 0x7fb35807dd80 2024-04-24T08:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.295+0000 7fb354c00700 1 -- 172.21.15.69:0/3849434538 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb348024780 con 0x7fb35807dd80 2024-04-24T08:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.375+0000 7fb35ee00700 1 -- 172.21.15.69:0/3849434538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb35802dfe0 con 0x7fb35807dd80 2024-04-24T08:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.375+0000 7fb354c00700 1 -- 172.21.15.69:0/3849434538 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb348024ad0 con 0x7fb35807dd80 2024-04-24T08:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.377+0000 7fb35ee00700 1 -- 172.21.15.69:0/3849434538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb35807dd80 msgr2=0x7fb35819e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.377+0000 7fb35ee00700 1 --2- 172.21.15.69:0/3849434538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb35807dd80 0x7fb35819e7d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb348011a50 tx=0x7fb348011b30 comp rx=0 tx=0).stop 2024-04-24T08:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.377+0000 7fb35ee00700 1 -- 172.21.15.69:0/3849434538 shutdown_connections 2024-04-24T08:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.377+0000 7fb35ee00700 1 --2- 172.21.15.69:0/3849434538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb35807dd80 0x7fb35819e7d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.377+0000 7fb35ee00700 1 -- 172.21.15.69:0/3849434538 >> 172.21.15.69:0/3849434538 conn(0x7fb358103a40 msgr2=0x7fb358104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.377+0000 7fb35ee00700 1 -- 172.21.15.69:0/3849434538 shutdown_connections 2024-04-24T08:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:26.377+0000 7fb35ee00700 1 -- 172.21.15.69:0/3849434538 wait complete. 2024-04-24T08:53:27.096 INFO:teuthology.orchestra.run.smithi069.stdout:mgr not available, waiting (1/15)... 2024-04-24T08:53:28.591 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:28 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:28.268+0000 7f7767fd6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T08:53:28.591 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:28 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:28.490+0000 7f7767fd6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T08:53:29.091 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:28 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:28.709+0000 7f7767fd6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T08:53:29.382 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:29 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:29.091+0000 7f7767fd6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T08:53:29.382 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:29 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:29.218+0000 7f7767fd6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T08:53:29.656 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:29 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:29.474+0000 7f7767fd6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T08:53:30.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:29 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3289045778' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T08:53:30.091 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:29 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:29.734+0000 7f7767fd6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:53:30.478 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-24T08:53:30.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:53:30.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "ef1ec7e8-0217-11ef-bc93-c7b262605968", 2024-04-24T08:53:30.489 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T08:53:30.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T08:53:30.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T08:53:30.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T08:53:30.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:30.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T08:53:30.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T08:53:30.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-04-24T08:53:30.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:53:30.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T08:53:30.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T08:53:30.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:53:30.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-24T08:53:30.490 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T08:53:30.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:53:30.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T08:53:30.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T08:53:30.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:30.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T08:53:30.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:53:30.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T08:53:30.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T08:53:30.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T08:53:30.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T08:53:30.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T08:53:30.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T08:53:30.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:30.491 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T08:53:30.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T08:53:30.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T08:53:30.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T08:53:30.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T08:53:30.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T08:53:30.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T08:53:30.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T08:53:30.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T08:53:30.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:30.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T08:53:30.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:53:30.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T08:53:30.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T08:53:30.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:30.492 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T08:53:30.493 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T08:53:30.493 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T08:53:30.493 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T08:53:30.493 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T08:53:30.493 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T08:53:30.493 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T08:53:30.493 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:53:30.493 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:53:30.493 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:30.493 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T08:53:30.493 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:53:30.493 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T08:53:15.726448+0000", 2024-04-24T08:53:30.493 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:53:30.493 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:30.493 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T08:53:30.494 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:53:30.494 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.656+0000 7f663f000700 1 Processor -- start 2024-04-24T08:53:30.494 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.656+0000 7f663f000700 1 -- start start 2024-04-24T08:53:30.494 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.656+0000 7f663f000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f663807dda0 0x7f663807c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:30.494 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.656+0000 7f663f000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f663807c850 con 0x7f663807dda0 2024-04-24T08:53:30.494 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.657+0000 7f6637e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f663807dda0 0x7f663807c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:30.494 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.657+0000 7f6637e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f663807dda0 0x7f663807c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56644/0 (socket says 172.21.15.69:56644) 2024-04-24T08:53:30.494 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.657+0000 7f6637e00700 1 -- 172.21.15.69:0/1768483643 learned_addr learned my addr 172.21.15.69:0/1768483643 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:30.494 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.657+0000 7f6637e00700 1 -- 172.21.15.69:0/1768483643 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663807c9d0 con 0x7f663807dda0 2024-04-24T08:53:30.494 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.658+0000 7f6637e00700 1 --2- 172.21.15.69:0/1768483643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f663807dda0 0x7f663807c310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6628009e80 tx=0x7f662800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c2ac259460bf83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:30.494 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.658+0000 7f6637400700 1 -- 172.21.15.69:0/1768483643 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6628017070 con 0x7f663807dda0 2024-04-24T08:53:30.494 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.658+0000 7f6637400700 1 -- 172.21.15.69:0/1768483643 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f662800b8c0 con 0x7f663807dda0 2024-04-24T08:53:30.494 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.658+0000 7f6637400700 1 -- 172.21.15.69:0/1768483643 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6628004520 con 0x7f663807dda0 2024-04-24T08:53:30.494 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.658+0000 7f663f000700 1 -- 172.21.15.69:0/1768483643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f663807dda0 msgr2=0x7f663807c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:30.494 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.658+0000 7f663f000700 1 --2- 172.21.15.69:0/1768483643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f663807dda0 0x7f663807c310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6628009e80 tx=0x7f662800b330 comp rx=0 tx=0).stop 2024-04-24T08:53:30.495 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.659+0000 7f663f000700 1 -- 172.21.15.69:0/1768483643 shutdown_connections 2024-04-24T08:53:30.495 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.659+0000 7f663f000700 1 --2- 172.21.15.69:0/1768483643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f663807dda0 0x7f663807c310 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:30.495 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.659+0000 7f663f000700 1 -- 172.21.15.69:0/1768483643 >> 172.21.15.69:0/1768483643 conn(0x7f6638103a40 msgr2=0x7f6638105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:30.495 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.659+0000 7f663f000700 1 -- 172.21.15.69:0/1768483643 shutdown_connections 2024-04-24T08:53:30.495 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.659+0000 7f663f000700 1 -- 172.21.15.69:0/1768483643 wait complete. 2024-04-24T08:53:30.495 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.660+0000 7f663f000700 1 Processor -- start 2024-04-24T08:53:30.495 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.660+0000 7f663f000700 1 -- start start 2024-04-24T08:53:30.495 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.660+0000 7f663f000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f663807dda0 0x7f663819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:30.495 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.660+0000 7f663f000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f663810e7f0 con 0x7f663807dda0 2024-04-24T08:53:30.495 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.660+0000 7f6637e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f663807dda0 0x7f663819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:30.495 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.661+0000 7f6637e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f663807dda0 0x7f663819e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56658/0 (socket says 172.21.15.69:56658) 2024-04-24T08:53:30.495 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.661+0000 7f6637e00700 1 -- 172.21.15.69:0/3289045778 learned_addr learned my addr 172.21.15.69:0/3289045778 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:30.495 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.661+0000 7f6637e00700 1 -- 172.21.15.69:0/3289045778 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663819ecd0 con 0x7f663807dda0 2024-04-24T08:53:30.495 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.661+0000 7f6637e00700 1 --2- 172.21.15.69:0/3289045778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f663807dda0 0x7f663819e790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f662800b7b0 tx=0x7f6628004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:30.495 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.661+0000 7f6635600700 1 -- 172.21.15.69:0/3289045778 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6628020020 con 0x7f663807dda0 2024-04-24T08:53:30.496 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.661+0000 7f6635600700 1 -- 172.21.15.69:0/3289045778 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f662800bcb0 con 0x7f663807dda0 2024-04-24T08:53:30.496 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.661+0000 7f6635600700 1 -- 172.21.15.69:0/3289045778 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6628017040 con 0x7f663807dda0 2024-04-24T08:53:30.496 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.662+0000 7f663f000700 1 -- 172.21.15.69:0/3289045778 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663819ef60 con 0x7f663807dda0 2024-04-24T08:53:30.496 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.662+0000 7f663f000700 1 -- 172.21.15.69:0/3289045778 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66381a1c50 con 0x7f663807dda0 2024-04-24T08:53:30.496 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.662+0000 7f6635600700 1 -- 172.21.15.69:0/3289045778 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6628023020 con 0x7f663807dda0 2024-04-24T08:53:30.496 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.662+0000 7f6635600700 1 -- 172.21.15.69:0/3289045778 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f662800f050 con 0x7f663807dda0 2024-04-24T08:53:30.496 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.663+0000 7f663f000700 1 -- 172.21.15.69:0/3289045778 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6638064150 con 0x7f663807dda0 2024-04-24T08:53:30.496 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.665+0000 7f6635600700 1 -- 172.21.15.69:0/3289045778 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f662802a020 con 0x7f663807dda0 2024-04-24T08:53:30.496 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.745+0000 7f663f000700 1 -- 172.21.15.69:0/3289045778 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f663802dfe0 con 0x7f663807dda0 2024-04-24T08:53:30.496 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.746+0000 7f6635600700 1 -- 172.21.15.69:0/3289045778 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f662800b860 con 0x7f663807dda0 2024-04-24T08:53:30.496 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.747+0000 7f663f000700 1 -- 172.21.15.69:0/3289045778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f663807dda0 msgr2=0x7f663819e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:30.496 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.747+0000 7f663f000700 1 --2- 172.21.15.69:0/3289045778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f663807dda0 0x7f663819e790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f662800b7b0 tx=0x7f6628004670 comp rx=0 tx=0).stop 2024-04-24T08:53:30.496 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.747+0000 7f663f000700 1 -- 172.21.15.69:0/3289045778 shutdown_connections 2024-04-24T08:53:30.496 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.747+0000 7f663f000700 1 --2- 172.21.15.69:0/3289045778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f663807dda0 0x7f663819e790 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:30.497 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.747+0000 7f663f000700 1 -- 172.21.15.69:0/3289045778 >> 172.21.15.69:0/3289045778 conn(0x7f6638103a40 msgr2=0x7f66381047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:30.497 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.748+0000 7f663f000700 1 -- 172.21.15.69:0/3289045778 shutdown_connections 2024-04-24T08:53:30.497 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:29.748+0000 7f663f000700 1 -- 172.21.15.69:0/3289045778 wait complete. 2024-04-24T08:53:30.497 INFO:teuthology.orchestra.run.smithi069.stdout:mgr not available, waiting (2/15)... 2024-04-24T08:53:30.841 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:30 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:30.530+0000 7f7767fd6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T08:53:31.341 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:31 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:31.072+0000 7f7767fd6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T08:53:31.341 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:31 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:31.194+0000 7f7767fd6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T08:53:31.841 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:31 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:31.451+0000 7f7767fd6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T08:53:32.341 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:32 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:32.079+0000 7f7767fd6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T08:53:32.341 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:32 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:32.199+0000 7f7767fd6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T08:53:32.819 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:32 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:32.508+0000 7f7767fd6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T08:53:32.819 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:32 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:32.692+0000 7f7767fd6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T08:53:32.819 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:32 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:32.819+0000 7f7767fd6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T08:53:33.085 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:32 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:32.979+0000 7f7767fd6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T08:53:33.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:33 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/668489820' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T08:53:33.622 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:33 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:33.370+0000 7f7767fd6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T08:53:33.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-24T08:53:33.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:53:33.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "ef1ec7e8-0217-11ef-bc93-c7b262605968", 2024-04-24T08:53:33.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T08:53:33.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T08:53:33.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T08:53:33.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T08:53:33.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:33.858 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T08:53:33.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T08:53:33.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-04-24T08:53:33.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:53:33.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T08:53:33.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T08:53:33.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:53:33.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-24T08:53:33.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T08:53:33.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:53:33.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T08:53:33.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T08:53:33.859 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:33.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T08:53:33.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:53:33.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T08:53:33.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T08:53:33.860 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T08:53:33.870 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T08:53:33.870 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T08:53:33.870 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T08:53:33.871 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:33.871 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T08:53:33.871 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T08:53:33.871 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T08:53:33.871 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T08:53:33.871 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T08:53:33.871 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T08:53:33.871 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T08:53:33.871 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T08:53:33.871 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T08:53:33.871 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:33.871 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T08:53:33.871 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:53:33.872 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T08:53:33.872 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T08:53:33.872 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:33.872 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T08:53:33.872 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T08:53:33.872 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T08:53:33.872 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T08:53:33.872 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T08:53:33.872 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T08:53:33.872 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T08:53:33.872 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:53:33.872 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:53:33.872 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:33.872 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T08:53:33.873 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:53:33.873 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T08:53:15.726448+0000", 2024-04-24T08:53:33.873 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:53:33.873 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:33.873 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T08:53:33.873 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:53:33.873 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.086+0000 7f5c65400700 1 Processor -- start 2024-04-24T08:53:33.873 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.086+0000 7f5c65400700 1 -- start start 2024-04-24T08:53:33.873 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.086+0000 7f5c65400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c60108c30 0x7f5c6010b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:33.873 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.086+0000 7f5c65400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6007bea0 con 0x7f5c60108c30 2024-04-24T08:53:33.873 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.087+0000 7f5c5fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c60108c30 0x7f5c6010b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:33.873 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.087+0000 7f5c5fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c60108c30 0x7f5c6010b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57590/0 (socket says 172.21.15.69:57590) 2024-04-24T08:53:33.873 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.087+0000 7f5c5fe00700 1 -- 172.21.15.69:0/3668001564 learned_addr learned my addr 172.21.15.69:0/3668001564 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:33.874 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.087+0000 7f5c5fe00700 1 -- 172.21.15.69:0/3668001564 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6010b560 con 0x7f5c60108c30 2024-04-24T08:53:33.874 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.088+0000 7f5c5fe00700 1 --2- 172.21.15.69:0/3668001564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c60108c30 0x7f5c6010b020 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5c500098a0 tx=0x7f5c50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a523c57a0a6739d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:33.874 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.088+0000 7f5c5ea00700 1 -- 172.21.15.69:0/3668001564 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c50004160 con 0x7f5c60108c30 2024-04-24T08:53:33.874 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.088+0000 7f5c5ea00700 1 -- 172.21.15.69:0/3668001564 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5c50004300 con 0x7f5c60108c30 2024-04-24T08:53:33.874 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.088+0000 7f5c5ea00700 1 -- 172.21.15.69:0/3668001564 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c50011a80 con 0x7f5c60108c30 2024-04-24T08:53:33.874 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.089+0000 7f5c65400700 1 -- 172.21.15.69:0/3668001564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c60108c30 msgr2=0x7f5c6010b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:33.874 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.089+0000 7f5c65400700 1 --2- 172.21.15.69:0/3668001564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c60108c30 0x7f5c6010b020 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5c500098a0 tx=0x7f5c50009bb0 comp rx=0 tx=0).stop 2024-04-24T08:53:33.874 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.089+0000 7f5c65400700 1 -- 172.21.15.69:0/3668001564 shutdown_connections 2024-04-24T08:53:33.874 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.089+0000 7f5c65400700 1 --2- 172.21.15.69:0/3668001564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c60108c30 0x7f5c6010b020 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:33.874 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.089+0000 7f5c65400700 1 -- 172.21.15.69:0/3668001564 >> 172.21.15.69:0/3668001564 conn(0x7f5c60102640 msgr2=0x7f5c60104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:33.874 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.089+0000 7f5c65400700 1 -- 172.21.15.69:0/3668001564 shutdown_connections 2024-04-24T08:53:33.874 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.089+0000 7f5c65400700 1 -- 172.21.15.69:0/3668001564 wait complete. 2024-04-24T08:53:33.874 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.090+0000 7f5c65400700 1 Processor -- start 2024-04-24T08:53:33.875 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.090+0000 7f5c65400700 1 -- start start 2024-04-24T08:53:33.875 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.090+0000 7f5c65400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c60108c30 0x7f5c6019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:33.875 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.090+0000 7f5c65400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6010c4b0 con 0x7f5c60108c30 2024-04-24T08:53:33.875 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.091+0000 7f5c5fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c60108c30 0x7f5c6019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:33.875 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.091+0000 7f5c5fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c60108c30 0x7f5c6019e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57602/0 (socket says 172.21.15.69:57602) 2024-04-24T08:53:33.875 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.091+0000 7f5c5fe00700 1 -- 172.21.15.69:0/668489820 learned_addr learned my addr 172.21.15.69:0/668489820 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:33.875 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.091+0000 7f5c5fe00700 1 -- 172.21.15.69:0/668489820 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6019ead0 con 0x7f5c60108c30 2024-04-24T08:53:33.875 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.091+0000 7f5c5fe00700 1 --2- 172.21.15.69:0/668489820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c60108c30 0x7f5c6019e590 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5c50011a50 tx=0x7f5c50011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:33.875 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.092+0000 7f5c5cc00700 1 -- 172.21.15.69:0/668489820 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c50003a80 con 0x7f5c60108c30 2024-04-24T08:53:33.875 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.092+0000 7f5c5cc00700 1 -- 172.21.15.69:0/668489820 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5c50003c20 con 0x7f5c60108c30 2024-04-24T08:53:33.875 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.092+0000 7f5c5cc00700 1 -- 172.21.15.69:0/668489820 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c5001a650 con 0x7f5c60108c30 2024-04-24T08:53:33.875 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.092+0000 7f5c65400700 1 -- 172.21.15.69:0/668489820 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c6019edc0 con 0x7f5c60108c30 2024-04-24T08:53:33.875 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.092+0000 7f5c65400700 1 -- 172.21.15.69:0/668489820 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c601a1a50 con 0x7f5c60108c30 2024-04-24T08:53:33.875 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.093+0000 7f5c5cc00700 1 -- 172.21.15.69:0/668489820 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5c5001a900 con 0x7f5c60108c30 2024-04-24T08:53:33.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.093+0000 7f5c5cc00700 1 -- 172.21.15.69:0/668489820 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5c5002dda0 con 0x7f5c60108c30 2024-04-24T08:53:33.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.093+0000 7f5c65400700 1 -- 172.21.15.69:0/668489820 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c60064150 con 0x7f5c60108c30 2024-04-24T08:53:33.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.096+0000 7f5c5cc00700 1 -- 172.21.15.69:0/668489820 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5c500248d0 con 0x7f5c60108c30 2024-04-24T08:53:33.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.184+0000 7f5c65400700 1 -- 172.21.15.69:0/668489820 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5c6002dfe0 con 0x7f5c60108c30 2024-04-24T08:53:33.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.185+0000 7f5c5cc00700 1 -- 172.21.15.69:0/668489820 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f5c50024ad0 con 0x7f5c60108c30 2024-04-24T08:53:33.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.187+0000 7f5c65400700 1 -- 172.21.15.69:0/668489820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c60108c30 msgr2=0x7f5c6019e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:33.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.187+0000 7f5c65400700 1 --2- 172.21.15.69:0/668489820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c60108c30 0x7f5c6019e590 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5c50011a50 tx=0x7f5c50011b30 comp rx=0 tx=0).stop 2024-04-24T08:53:33.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.188+0000 7f5c65400700 1 -- 172.21.15.69:0/668489820 shutdown_connections 2024-04-24T08:53:33.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.188+0000 7f5c65400700 1 --2- 172.21.15.69:0/668489820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c60108c30 0x7f5c6019e590 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:33.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.188+0000 7f5c65400700 1 -- 172.21.15.69:0/668489820 >> 172.21.15.69:0/668489820 conn(0x7f5c60102640 msgr2=0x7f5c60103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:33.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.188+0000 7f5c65400700 1 -- 172.21.15.69:0/668489820 shutdown_connections 2024-04-24T08:53:33.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:33.188+0000 7f5c65400700 1 -- 172.21.15.69:0/668489820 wait complete. 2024-04-24T08:53:33.876 INFO:teuthology.orchestra.run.smithi069.stdout:mgr not available, waiting (3/15)... 2024-04-24T08:53:34.433 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:34 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:34.151+0000 7f7767fd6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T08:53:34.841 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:34 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:34.433+0000 7f7767fd6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T08:53:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:34 smithi069 ceph-mon[28170]: Activating manager daemon x 2024-04-24T08:53:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:34 smithi069 ceph-mon[28170]: mgrmap e2: x(active, starting, since 0.012922s) 2024-04-24T08:53:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:34 smithi069 ceph-mon[28170]: from='mgr.14100 172.21.15.69:0/1396947600' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:53:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:34 smithi069 ceph-mon[28170]: from='mgr.14100 172.21.15.69:0/1396947600' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:53:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:34 smithi069 ceph-mon[28170]: from='mgr.14100 172.21.15.69:0/1396947600' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:53:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:34 smithi069 ceph-mon[28170]: from='mgr.14100 172.21.15.69:0/1396947600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:53:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:34 smithi069 ceph-mon[28170]: from='mgr.14100 172.21.15.69:0/1396947600' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T08:53:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:34 smithi069 ceph-mon[28170]: Manager daemon x is now available 2024-04-24T08:53:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:34 smithi069 ceph-mon[28170]: from='mgr.14100 172.21.15.69:0/1396947600' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:53:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:34 smithi069 ceph-mon[28170]: from='mgr.14100 172.21.15.69:0/1396947600' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T08:53:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:34 smithi069 ceph-mon[28170]: from='mgr.14100 172.21.15.69:0/1396947600' entity='mgr.x' 2024-04-24T08:53:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:34 smithi069 ceph-mon[28170]: from='mgr.14100 172.21.15.69:0/1396947600' entity='mgr.x' 2024-04-24T08:53:35.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:35 smithi069 ceph-mon[28170]: from='mgr.14100 172.21.15.69:0/1396947600' entity='mgr.x' 2024-04-24T08:53:35.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:35 smithi069 ceph-mon[28170]: mgrmap e3: x(active, since 1.02276s) 2024-04-24T08:53:37.537 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-24T08:53:37.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:53:37.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "ef1ec7e8-0217-11ef-bc93-c7b262605968", 2024-04-24T08:53:37.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T08:53:37.538 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T08:53:37.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T08:53:37.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T08:53:37.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:37.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T08:53:37.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T08:53:37.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-04-24T08:53:37.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:53:37.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T08:53:37.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T08:53:37.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:53:37.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-24T08:53:37.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T08:53:37.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:53:37.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T08:53:37.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T08:53:37.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:37.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T08:53:37.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:53:37.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T08:53:37.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T08:53:37.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T08:53:37.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T08:53:37.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T08:53:37.547 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T08:53:37.547 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:37.547 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T08:53:37.547 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T08:53:37.547 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T08:53:37.547 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T08:53:37.547 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T08:53:37.547 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T08:53:37.547 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T08:53:37.548 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T08:53:37.548 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T08:53:37.548 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:37.548 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T08:53:37.548 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:53:37.548 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T08:53:37.548 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T08:53:37.548 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:37.548 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T08:53:37.548 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T08:53:37.548 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T08:53:37.548 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T08:53:37.548 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T08:53:37.548 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T08:53:37.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T08:53:37.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:53:37.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:53:37.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:37.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T08:53:37.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:53:37.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T08:53:15.726448+0000", 2024-04-24T08:53:37.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:53:37.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:53:37.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T08:53:37.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:53:37.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.483+0000 7fa135a00700 1 Processor -- start 2024-04-24T08:53:37.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.483+0000 7fa135a00700 1 -- start start 2024-04-24T08:53:37.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.483+0000 7fa135a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13007de40 0x7fa13007c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:37.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.483+0000 7fa135a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa13007c830 con 0x7fa13007de40 2024-04-24T08:53:37.550 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.484+0000 7fa12ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13007de40 0x7fa13007c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:37.550 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.484+0000 7fa12ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13007de40 0x7fa13007c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57680/0 (socket says 172.21.15.69:57680) 2024-04-24T08:53:37.550 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.484+0000 7fa12ea00700 1 -- 172.21.15.69:0/3821124449 learned_addr learned my addr 172.21.15.69:0/3821124449 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:37.555 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.484+0000 7fa12ea00700 1 -- 172.21.15.69:0/3821124449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13007c9b0 con 0x7fa13007de40 2024-04-24T08:53:37.555 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.485+0000 7fa12ea00700 1 --2- 172.21.15.69:0/3821124449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13007de40 0x7fa13007c2f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa1200098a0 tx=0x7fa120009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d7ad66ae142cd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:37.555 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.485+0000 7fa12e000700 1 -- 172.21.15.69:0/3821124449 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa120004160 con 0x7fa13007de40 2024-04-24T08:53:37.555 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.485+0000 7fa12e000700 1 -- 172.21.15.69:0/3821124449 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa120004300 con 0x7fa13007de40 2024-04-24T08:53:37.555 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.485+0000 7fa12e000700 1 -- 172.21.15.69:0/3821124449 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa120011a80 con 0x7fa13007de40 2024-04-24T08:53:37.555 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.485+0000 7fa135a00700 1 -- 172.21.15.69:0/3821124449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13007de40 msgr2=0x7fa13007c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:37.555 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.485+0000 7fa135a00700 1 --2- 172.21.15.69:0/3821124449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13007de40 0x7fa13007c2f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa1200098a0 tx=0x7fa120009bb0 comp rx=0 tx=0).stop 2024-04-24T08:53:37.555 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.486+0000 7fa135a00700 1 -- 172.21.15.69:0/3821124449 shutdown_connections 2024-04-24T08:53:37.555 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.486+0000 7fa135a00700 1 --2- 172.21.15.69:0/3821124449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13007de40 0x7fa13007c2f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:37.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.486+0000 7fa135a00700 1 -- 172.21.15.69:0/3821124449 >> 172.21.15.69:0/3821124449 conn(0x7fa130103a10 msgr2=0x7fa130105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:37.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.486+0000 7fa135a00700 1 -- 172.21.15.69:0/3821124449 shutdown_connections 2024-04-24T08:53:37.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.486+0000 7fa135a00700 1 -- 172.21.15.69:0/3821124449 wait complete. 2024-04-24T08:53:37.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.487+0000 7fa135a00700 1 Processor -- start 2024-04-24T08:53:37.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.487+0000 7fa135a00700 1 -- start start 2024-04-24T08:53:37.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.487+0000 7fa135a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13007de40 0x7fa13019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:37.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.487+0000 7fa135a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa13010e920 con 0x7fa13007de40 2024-04-24T08:53:37.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.488+0000 7fa12ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13007de40 0x7fa13019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:37.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.488+0000 7fa12ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13007de40 0x7fa13019e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57692/0 (socket says 172.21.15.69:57692) 2024-04-24T08:53:37.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.488+0000 7fa12ea00700 1 -- 172.21.15.69:0/883780847 learned_addr learned my addr 172.21.15.69:0/883780847 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:37.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.488+0000 7fa12ea00700 1 -- 172.21.15.69:0/883780847 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13019ed10 con 0x7fa13007de40 2024-04-24T08:53:37.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.488+0000 7fa12ea00700 1 --2- 172.21.15.69:0/883780847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13007de40 0x7fa13019e7d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa120011a50 tx=0x7fa120011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:37.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.488+0000 7fa127400700 1 -- 172.21.15.69:0/883780847 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa120003a80 con 0x7fa13007de40 2024-04-24T08:53:37.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.488+0000 7fa127400700 1 -- 172.21.15.69:0/883780847 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa120003c20 con 0x7fa13007de40 2024-04-24T08:53:37.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.488+0000 7fa127400700 1 -- 172.21.15.69:0/883780847 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa12001a650 con 0x7fa13007de40 2024-04-24T08:53:37.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.489+0000 7fa135a00700 1 -- 172.21.15.69:0/883780847 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13019efa0 con 0x7fa13007de40 2024-04-24T08:53:37.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.489+0000 7fa135a00700 1 -- 172.21.15.69:0/883780847 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1301a1c40 con 0x7fa13007de40 2024-04-24T08:53:37.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.490+0000 7fa127400700 1 -- 172.21.15.69:0/883780847 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7fa120024ad0 con 0x7fa13007de40 2024-04-24T08:53:37.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.490+0000 7fa135a00700 1 -- 172.21.15.69:0/883780847 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa130064150 con 0x7fa13007de40 2024-04-24T08:53:37.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.490+0000 7fa127400700 1 --2- 172.21.15.69:0/883780847 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fa114039490 0x7fa11403b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:37.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.490+0000 7fa127400700 1 -- 172.21.15.69:0/883780847 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa12004d930 con 0x7fa13007de40 2024-04-24T08:53:37.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.491+0000 7fa127e00700 1 --2- 172.21.15.69:0/883780847 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fa114039490 0x7fa11403b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:37.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.491+0000 7fa127e00700 1 --2- 172.21.15.69:0/883780847 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fa114039490 0x7fa11403b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa118006fd0 tx=0x7fa118006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:37.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.497+0000 7fa127400700 1 -- 172.21.15.69:0/883780847 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa12000f2e0 con 0x7fa13007de40 2024-04-24T08:53:37.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.499+0000 7fa127400700 1 -- 172.21.15.69:0/883780847 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fa12002d3f0 con 0x7fa13007de40 2024-04-24T08:53:37.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.802+0000 7fa135a00700 1 -- 172.21.15.69:0/883780847 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa13002dfe0 con 0x7fa13007de40 2024-04-24T08:53:37.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.803+0000 7fa127400700 1 -- 172.21.15.69:0/883780847 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa120045030 con 0x7fa13007de40 2024-04-24T08:53:37.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.806+0000 7fa135a00700 1 -- 172.21.15.69:0/883780847 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fa114039490 msgr2=0x7fa11403b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:37.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.806+0000 7fa135a00700 1 --2- 172.21.15.69:0/883780847 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fa114039490 0x7fa11403b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa118006fd0 tx=0x7fa118006e40 comp rx=0 tx=0).stop 2024-04-24T08:53:37.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.806+0000 7fa135a00700 1 -- 172.21.15.69:0/883780847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13007de40 msgr2=0x7fa13019e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:37.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.806+0000 7fa135a00700 1 --2- 172.21.15.69:0/883780847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13007de40 0x7fa13019e7d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa120011a50 tx=0x7fa120011b30 comp rx=0 tx=0).stop 2024-04-24T08:53:37.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.806+0000 7fa135a00700 1 -- 172.21.15.69:0/883780847 shutdown_connections 2024-04-24T08:53:37.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.806+0000 7fa135a00700 1 --2- 172.21.15.69:0/883780847 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fa114039490 0x7fa11403b950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:37.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.806+0000 7fa135a00700 1 --2- 172.21.15.69:0/883780847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13007de40 0x7fa13019e7d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:37.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.806+0000 7fa135a00700 1 -- 172.21.15.69:0/883780847 >> 172.21.15.69:0/883780847 conn(0x7fa130103a10 msgr2=0x7fa1301046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:37.559 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.807+0000 7fa135a00700 1 -- 172.21.15.69:0/883780847 shutdown_connections 2024-04-24T08:53:37.559 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:36.807+0000 7fa135a00700 1 -- 172.21.15.69:0/883780847 wait complete. 2024-04-24T08:53:37.559 INFO:teuthology.orchestra.run.smithi069.stdout:mgr is available 2024-04-24T08:53:37.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:37 smithi069 ceph-mon[28170]: mgrmap e4: x(active, since 2s) 2024-04-24T08:53:37.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:37 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/883780847' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T08:53:38.836 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:38 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2137525695' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T08:53:39.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-24T08:53:39.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T08:53:39.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout fsid = ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:53:39.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T08:53:39.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T08:53:39.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T08:53:39.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T08:53:39.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-24T08:53:39.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T08:53:39.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T08:53:39.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-24T08:53:39.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T08:53:39.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T08:53:39.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T08:53:39.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T08:53:39.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T08:53:39.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T08:53:39.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.186+0000 7f0e5dc00700 1 Processor -- start 2024-04-24T08:53:39.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.186+0000 7f0e5dc00700 1 -- start start 2024-04-24T08:53:39.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.187+0000 7f0e5dc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e5807ddf0 0x7f0e5807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:39.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.187+0000 7f0e5dc00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5807c830 con 0x7f0e5807ddf0 2024-04-24T08:53:39.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.187+0000 7f0e56a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e5807ddf0 0x7f0e5807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:39.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.187+0000 7f0e56a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e5807ddf0 0x7f0e5807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57700/0 (socket says 172.21.15.69:57700) 2024-04-24T08:53:39.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.187+0000 7f0e56a00700 1 -- 172.21.15.69:0/3388447868 learned_addr learned my addr 172.21.15.69:0/3388447868 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:39.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.187+0000 7f0e56a00700 1 -- 172.21.15.69:0/3388447868 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5807c9b0 con 0x7f0e5807ddf0 2024-04-24T08:53:39.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.188+0000 7f0e56a00700 1 --2- 172.21.15.69:0/3388447868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e5807ddf0 0x7f0e5807c2f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0e480098a0 tx=0x7f0e48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dce9ca377e2aaa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:39.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.188+0000 7f0e55600700 1 -- 172.21.15.69:0/3388447868 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e48004160 con 0x7f0e5807ddf0 2024-04-24T08:53:39.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.188+0000 7f0e55600700 1 -- 172.21.15.69:0/3388447868 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0e48004300 con 0x7f0e5807ddf0 2024-04-24T08:53:39.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.188+0000 7f0e55600700 1 -- 172.21.15.69:0/3388447868 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e48011a80 con 0x7f0e5807ddf0 2024-04-24T08:53:39.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.189+0000 7f0e5dc00700 1 -- 172.21.15.69:0/3388447868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e5807ddf0 msgr2=0x7f0e5807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:39.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.189+0000 7f0e5dc00700 1 --2- 172.21.15.69:0/3388447868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e5807ddf0 0x7f0e5807c2f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0e480098a0 tx=0x7f0e48009bb0 comp rx=0 tx=0).stop 2024-04-24T08:53:39.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.189+0000 7f0e5dc00700 1 -- 172.21.15.69:0/3388447868 shutdown_connections 2024-04-24T08:53:39.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.189+0000 7f0e5dc00700 1 --2- 172.21.15.69:0/3388447868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e5807ddf0 0x7f0e5807c2f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:39.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.189+0000 7f0e5dc00700 1 -- 172.21.15.69:0/3388447868 >> 172.21.15.69:0/3388447868 conn(0x7f0e581039e0 msgr2=0x7f0e58105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:39.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.189+0000 7f0e5dc00700 1 -- 172.21.15.69:0/3388447868 shutdown_connections 2024-04-24T08:53:39.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.189+0000 7f0e5dc00700 1 -- 172.21.15.69:0/3388447868 wait complete. 2024-04-24T08:53:39.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.190+0000 7f0e5dc00700 1 Processor -- start 2024-04-24T08:53:39.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.190+0000 7f0e5dc00700 1 -- start start 2024-04-24T08:53:39.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.190+0000 7f0e5dc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e5807ddf0 0x7f0e5819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:39.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.190+0000 7f0e5dc00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5810e910 con 0x7f0e5807ddf0 2024-04-24T08:53:39.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.190+0000 7f0e56a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e5807ddf0 0x7f0e5819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:39.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.191+0000 7f0e56a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e5807ddf0 0x7f0e5819e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57706/0 (socket says 172.21.15.69:57706) 2024-04-24T08:53:39.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.191+0000 7f0e56a00700 1 -- 172.21.15.69:0/2137525695 learned_addr learned my addr 172.21.15.69:0/2137525695 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:39.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.191+0000 7f0e56a00700 1 -- 172.21.15.69:0/2137525695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5819ed10 con 0x7f0e5807ddf0 2024-04-24T08:53:39.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.191+0000 7f0e56a00700 1 --2- 172.21.15.69:0/2137525695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e5807ddf0 0x7f0e5819e7d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0e48006af0 tx=0x7f0e48011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:39.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.191+0000 7f0e4f400700 1 -- 172.21.15.69:0/2137525695 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e480039e0 con 0x7f0e5807ddf0 2024-04-24T08:53:39.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.191+0000 7f0e4f400700 1 -- 172.21.15.69:0/2137525695 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0e48003b80 con 0x7f0e5807ddf0 2024-04-24T08:53:39.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.191+0000 7f0e4f400700 1 -- 172.21.15.69:0/2137525695 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e480196a0 con 0x7f0e5807ddf0 2024-04-24T08:53:39.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.191+0000 7f0e5dc00700 1 -- 172.21.15.69:0/2137525695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e5819efa0 con 0x7f0e5807ddf0 2024-04-24T08:53:39.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.192+0000 7f0e5dc00700 1 -- 172.21.15.69:0/2137525695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e581a1c90 con 0x7f0e5807ddf0 2024-04-24T08:53:39.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.193+0000 7f0e4f400700 1 -- 172.21.15.69:0/2137525695 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f0e4801ab50 con 0x7f0e5807ddf0 2024-04-24T08:53:39.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.193+0000 7f0e5dc00700 1 -- 172.21.15.69:0/2137525695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e58064150 con 0x7f0e5807ddf0 2024-04-24T08:53:39.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.193+0000 7f0e4f400700 1 --2- 172.21.15.69:0/2137525695 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7f0e3c0395b0 0x7f0e3c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:39.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.193+0000 7f0e4f400700 1 -- 172.21.15.69:0/2137525695 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0e4804dba0 con 0x7f0e5807ddf0 2024-04-24T08:53:39.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.193+0000 7f0e56000700 1 --2- 172.21.15.69:0/2137525695 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7f0e3c0395b0 0x7f0e3c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:39.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.194+0000 7f0e56000700 1 --2- 172.21.15.69:0/2137525695 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7f0e3c0395b0 0x7f0e3c03ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0e44006fd0 tx=0x7f0e44006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:39.137 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.199+0000 7f0e4f400700 1 -- 172.21.15.69:0/2137525695 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e480172b0 con 0x7f0e5807ddf0 2024-04-24T08:53:39.137 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.423+0000 7f0e5dc00700 1 -- 172.21.15.69:0/2137525695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0e581a2fe0 con 0x7f0e5807ddf0 2024-04-24T08:53:39.137 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.425+0000 7f0e4f400700 1 -- 172.21.15.69:0/2137525695 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f0e48029d50 con 0x7f0e5807ddf0 2024-04-24T08:53:39.137 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.428+0000 7f0e5dc00700 1 -- 172.21.15.69:0/2137525695 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7f0e3c0395b0 msgr2=0x7f0e3c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:39.137 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.428+0000 7f0e5dc00700 1 --2- 172.21.15.69:0/2137525695 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7f0e3c0395b0 0x7f0e3c03ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0e44006fd0 tx=0x7f0e44006e40 comp rx=0 tx=0).stop 2024-04-24T08:53:39.137 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.428+0000 7f0e5dc00700 1 -- 172.21.15.69:0/2137525695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e5807ddf0 msgr2=0x7f0e5819e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:39.137 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.428+0000 7f0e5dc00700 1 --2- 172.21.15.69:0/2137525695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e5807ddf0 0x7f0e5819e7d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0e48006af0 tx=0x7f0e48011c40 comp rx=0 tx=0).stop 2024-04-24T08:53:39.137 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.428+0000 7f0e5dc00700 1 -- 172.21.15.69:0/2137525695 shutdown_connections 2024-04-24T08:53:39.137 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.428+0000 7f0e5dc00700 1 --2- 172.21.15.69:0/2137525695 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7f0e3c0395b0 0x7f0e3c03ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:39.137 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.428+0000 7f0e5dc00700 1 --2- 172.21.15.69:0/2137525695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e5807ddf0 0x7f0e5819e7d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:39.137 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.428+0000 7f0e5dc00700 1 -- 172.21.15.69:0/2137525695 >> 172.21.15.69:0/2137525695 conn(0x7f0e581039e0 msgr2=0x7f0e581047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:39.137 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.428+0000 7f0e5dc00700 1 -- 172.21.15.69:0/2137525695 shutdown_connections 2024-04-24T08:53:39.137 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:38.428+0000 7f0e5dc00700 1 -- 172.21.15.69:0/2137525695 wait complete. 2024-04-24T08:53:39.138 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling cephadm module... 2024-04-24T08:53:40.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:40 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3315667953' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-24T08:53:40.341 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:40 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: ignoring --setuser ceph since I am not root 2024-04-24T08:53:40.341 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:40 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: ignoring --setgroup ceph since I am not root 2024-04-24T08:53:40.744 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:40 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:40.370+0000 7f217d108200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T08:53:40.744 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:40 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:40.609+0000 7f217d108200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:53:40.744 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:40 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:40.743+0000 7f217d108200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T08:53:40.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.742+0000 7fc2bb000700 1 Processor -- start 2024-04-24T08:53:40.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.742+0000 7fc2bb000700 1 -- start start 2024-04-24T08:53:40.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.743+0000 7fc2bb000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b407dde0 0x7fc2b407c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:40.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.743+0000 7fc2bb000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b407c890 con 0x7fc2b407dde0 2024-04-24T08:53:40.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.743+0000 7fc2b9c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b407dde0 0x7fc2b407c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:40.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.743+0000 7fc2b9c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b407dde0 0x7fc2b407c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57716/0 (socket says 172.21.15.69:57716) 2024-04-24T08:53:40.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.743+0000 7fc2b9c00700 1 -- 172.21.15.69:0/1776960439 learned_addr learned my addr 172.21.15.69:0/1776960439 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:40.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.744+0000 7fc2b9c00700 1 -- 172.21.15.69:0/1776960439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b407ca10 con 0x7fc2b407dde0 2024-04-24T08:53:40.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.744+0000 7fc2b9c00700 1 --2- 172.21.15.69:0/1776960439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b407dde0 0x7fc2b407c350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80098a0 tx=0x7fc2a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efdb7b9aa5d6656 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:40.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.744+0000 7fc2b3e00700 1 -- 172.21.15.69:0/1776960439 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2a8004160 con 0x7fc2b407dde0 2024-04-24T08:53:40.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.744+0000 7fc2b3e00700 1 -- 172.21.15.69:0/1776960439 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc2a8004300 con 0x7fc2b407dde0 2024-04-24T08:53:40.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.744+0000 7fc2b3e00700 1 -- 172.21.15.69:0/1776960439 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2a8011a80 con 0x7fc2b407dde0 2024-04-24T08:53:40.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.745+0000 7fc2bb000700 1 -- 172.21.15.69:0/1776960439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b407dde0 msgr2=0x7fc2b407c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:40.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.745+0000 7fc2bb000700 1 --2- 172.21.15.69:0/1776960439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b407dde0 0x7fc2b407c350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80098a0 tx=0x7fc2a8009bb0 comp rx=0 tx=0).stop 2024-04-24T08:53:40.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.745+0000 7fc2bb000700 1 -- 172.21.15.69:0/1776960439 shutdown_connections 2024-04-24T08:53:40.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.745+0000 7fc2bb000700 1 --2- 172.21.15.69:0/1776960439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b407dde0 0x7fc2b407c350 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:40.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.745+0000 7fc2bb000700 1 -- 172.21.15.69:0/1776960439 >> 172.21.15.69:0/1776960439 conn(0x7fc2b4103900 msgr2=0x7fc2b4105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:40.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.745+0000 7fc2bb000700 1 -- 172.21.15.69:0/1776960439 shutdown_connections 2024-04-24T08:53:40.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.745+0000 7fc2bb000700 1 -- 172.21.15.69:0/1776960439 wait complete. 2024-04-24T08:53:40.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.746+0000 7fc2bb000700 1 Processor -- start 2024-04-24T08:53:40.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.747+0000 7fc2bb000700 1 -- start start 2024-04-24T08:53:40.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.747+0000 7fc2bb000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b407dde0 0x7fc2b41a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:40.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.747+0000 7fc2bb000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b410a2a0 con 0x7fc2b407dde0 2024-04-24T08:53:40.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.747+0000 7fc2b9c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b407dde0 0x7fc2b41a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:40.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.747+0000 7fc2b9c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b407dde0 0x7fc2b41a2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57732/0 (socket says 172.21.15.69:57732) 2024-04-24T08:53:40.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.747+0000 7fc2b9c00700 1 -- 172.21.15.69:0/3315667953 learned_addr learned my addr 172.21.15.69:0/3315667953 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:40.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.748+0000 7fc2b9c00700 1 -- 172.21.15.69:0/3315667953 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b41a2fd0 con 0x7fc2b407dde0 2024-04-24T08:53:40.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.748+0000 7fc2b9c00700 1 --2- 172.21.15.69:0/3315667953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b407dde0 0x7fc2b41a2a90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8011a50 tx=0x7fc2a8004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:40.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.748+0000 7fc2b2000700 1 -- 172.21.15.69:0/3315667953 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2a80039f0 con 0x7fc2b407dde0 2024-04-24T08:53:40.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.748+0000 7fc2b2000700 1 -- 172.21.15.69:0/3315667953 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc2a8003b90 con 0x7fc2b407dde0 2024-04-24T08:53:40.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.748+0000 7fc2b2000700 1 -- 172.21.15.69:0/3315667953 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2a801a600 con 0x7fc2b407dde0 2024-04-24T08:53:40.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.748+0000 7fc2bb000700 1 -- 172.21.15.69:0/3315667953 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b41a3260 con 0x7fc2b407dde0 2024-04-24T08:53:40.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.748+0000 7fc2bb000700 1 -- 172.21.15.69:0/3315667953 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b41a5f50 con 0x7fc2b407dde0 2024-04-24T08:53:40.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.749+0000 7fc2b2000700 1 -- 172.21.15.69:0/3315667953 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fc2a8024a40 con 0x7fc2b407dde0 2024-04-24T08:53:40.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.749+0000 7fc2bb000700 1 -- 172.21.15.69:0/3315667953 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b4064150 con 0x7fc2b407dde0 2024-04-24T08:53:40.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.750+0000 7fc2b2000700 1 --2- 172.21.15.69:0/3315667953 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fc29c0395b0 0x7fc29c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:40.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.750+0000 7fc2b2000700 1 -- 172.21.15.69:0/3315667953 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc2a804da80 con 0x7fc2b407dde0 2024-04-24T08:53:40.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.750+0000 7fc2b9200700 1 --2- 172.21.15.69:0/3315667953 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fc29c0395b0 0x7fc29c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:40.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.751+0000 7fc2b9200700 1 --2- 172.21.15.69:0/3315667953 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fc29c0395b0 0x7fc29c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0006fd0 tx=0x7fc2a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:40.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:39.755+0000 7fc2b2000700 1 -- 172.21.15.69:0/3315667953 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2a804d4a0 con 0x7fc2b407dde0 2024-04-24T08:53:40.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:40.041+0000 7fc2bb000700 1 -- 172.21.15.69:0/3315667953 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fc2b402dfe0 con 0x7fc2b407dde0 2024-04-24T08:53:40.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:40.094+0000 7fc2b2000700 1 -- 172.21.15.69:0/3315667953 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fc2a802d770 con 0x7fc2b407dde0 2024-04-24T08:53:40.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:40.097+0000 7fc2b2000700 1 -- 172.21.15.69:0/3315667953 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fc2a80212a0 con 0x7fc2b407dde0 2024-04-24T08:53:40.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:40.099+0000 7fc2bb000700 1 -- 172.21.15.69:0/3315667953 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fc29c0395b0 msgr2=0x7fc29c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:40.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:40.099+0000 7fc2bb000700 1 --2- 172.21.15.69:0/3315667953 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fc29c0395b0 0x7fc29c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0006fd0 tx=0x7fc2a0006e40 comp rx=0 tx=0).stop 2024-04-24T08:53:40.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:40.099+0000 7fc2bb000700 1 -- 172.21.15.69:0/3315667953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b407dde0 msgr2=0x7fc2b41a2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:40.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:40.099+0000 7fc2bb000700 1 --2- 172.21.15.69:0/3315667953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b407dde0 0x7fc2b41a2a90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8011a50 tx=0x7fc2a8004290 comp rx=0 tx=0).stop 2024-04-24T08:53:40.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:40.099+0000 7fc2bb000700 1 -- 172.21.15.69:0/3315667953 shutdown_connections 2024-04-24T08:53:40.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:40.099+0000 7fc2bb000700 1 --2- 172.21.15.69:0/3315667953 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fc29c0395b0 0x7fc29c03ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:40.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:40.099+0000 7fc2bb000700 1 --2- 172.21.15.69:0/3315667953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b407dde0 0x7fc2b41a2a90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:40.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:40.099+0000 7fc2bb000700 1 -- 172.21.15.69:0/3315667953 >> 172.21.15.69:0/3315667953 conn(0x7fc2b4103900 msgr2=0x7fc2b41995b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:40.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:40.099+0000 7fc2bb000700 1 -- 172.21.15.69:0/3315667953 shutdown_connections 2024-04-24T08:53:40.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:40.099+0000 7fc2bb000700 1 -- 172.21.15.69:0/3315667953 wait complete. 2024-04-24T08:53:41.298 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:41 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3315667953' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-24T08:53:41.299 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:41 smithi069 ceph-mon[28170]: mgrmap e5: x(active, since 5s) 2024-04-24T08:53:41.299 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:41 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:41.150+0000 7f217d108200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T08:53:41.299 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:41 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:41.264+0000 7f217d108200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T08:53:42.232 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:42 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3412131330' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T08:53:42.246 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:53:42.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-24T08:53:42.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T08:53:42.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T08:53:42.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T08:53:42.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:53:42.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.298+0000 7f3539e00700 1 Processor -- start 2024-04-24T08:53:42.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.298+0000 7f3539e00700 1 -- start start 2024-04-24T08:53:42.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.298+0000 7f3539e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3534108380 0x7f35341087a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:42.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.299+0000 7f3539e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3534108ce0 con 0x7f3534108380 2024-04-24T08:53:42.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.299+0000 7f3532a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3534108380 0x7f35341087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:42.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.299+0000 7f3532a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3534108380 0x7f35341087a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57764/0 (socket says 172.21.15.69:57764) 2024-04-24T08:53:42.247 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.299+0000 7f3532a00700 1 -- 172.21.15.69:0/2729109563 learned_addr learned my addr 172.21.15.69:0/2729109563 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:42.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.299+0000 7f3532a00700 1 -- 172.21.15.69:0/2729109563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3534108e60 con 0x7f3534108380 2024-04-24T08:53:42.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.300+0000 7f3532a00700 1 --2- 172.21.15.69:0/2729109563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3534108380 0x7f35341087a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f35240098a0 tx=0x7f3524009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a453c21fdc8ef15a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:42.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.300+0000 7f3531600700 1 -- 172.21.15.69:0/2729109563 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3524004160 con 0x7f3534108380 2024-04-24T08:53:42.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.300+0000 7f3531600700 1 -- 172.21.15.69:0/2729109563 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3524004300 con 0x7f3534108380 2024-04-24T08:53:42.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.300+0000 7f3531600700 1 -- 172.21.15.69:0/2729109563 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3524011a80 con 0x7f3534108380 2024-04-24T08:53:42.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.301+0000 7f3539e00700 1 -- 172.21.15.69:0/2729109563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3534108380 msgr2=0x7f35341087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:42.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.301+0000 7f3539e00700 1 --2- 172.21.15.69:0/2729109563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3534108380 0x7f35341087a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f35240098a0 tx=0x7f3524009bb0 comp rx=0 tx=0).stop 2024-04-24T08:53:42.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.301+0000 7f3539e00700 1 -- 172.21.15.69:0/2729109563 shutdown_connections 2024-04-24T08:53:42.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.301+0000 7f3539e00700 1 --2- 172.21.15.69:0/2729109563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3534108380 0x7f35341087a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:42.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.301+0000 7f3539e00700 1 -- 172.21.15.69:0/2729109563 >> 172.21.15.69:0/2729109563 conn(0x7f3534103a40 msgr2=0x7f3534105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:42.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.302+0000 7f3539e00700 1 -- 172.21.15.69:0/2729109563 shutdown_connections 2024-04-24T08:53:42.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.302+0000 7f3539e00700 1 -- 172.21.15.69:0/2729109563 wait complete. 2024-04-24T08:53:42.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.303+0000 7f3539e00700 1 Processor -- start 2024-04-24T08:53:42.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.303+0000 7f3539e00700 1 -- start start 2024-04-24T08:53:42.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.303+0000 7f3539e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3534108380 0x7f353419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:42.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.303+0000 7f3539e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f353410a380 con 0x7f3534108380 2024-04-24T08:53:42.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.303+0000 7f3532a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3534108380 0x7f353419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:42.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.303+0000 7f3532a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3534108380 0x7f353419c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57770/0 (socket says 172.21.15.69:57770) 2024-04-24T08:53:42.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.303+0000 7f3532a00700 1 -- 172.21.15.69:0/3412131330 learned_addr learned my addr 172.21.15.69:0/3412131330 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:42.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.304+0000 7f3532a00700 1 -- 172.21.15.69:0/3412131330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353419cb00 con 0x7f3534108380 2024-04-24T08:53:42.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.304+0000 7f3532a00700 1 --2- 172.21.15.69:0/3412131330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3534108380 0x7f353419c5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3524011a50 tx=0x7f3524011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:42.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.304+0000 7f3538a00700 1 -- 172.21.15.69:0/3412131330 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3524003a80 con 0x7f3534108380 2024-04-24T08:53:42.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.304+0000 7f3538a00700 1 -- 172.21.15.69:0/3412131330 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3524003c20 con 0x7f3534108380 2024-04-24T08:53:42.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.304+0000 7f3539e00700 1 -- 172.21.15.69:0/3412131330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f353419cd90 con 0x7f3534108380 2024-04-24T08:53:42.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.304+0000 7f3538a00700 1 -- 172.21.15.69:0/3412131330 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f352401a650 con 0x7f3534108380 2024-04-24T08:53:42.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.304+0000 7f3539e00700 1 -- 172.21.15.69:0/3412131330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f353419fa80 con 0x7f3534108380 2024-04-24T08:53:42.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.305+0000 7f3538a00700 1 -- 172.21.15.69:0/3412131330 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f352401a960 con 0x7f3534108380 2024-04-24T08:53:42.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.306+0000 7f3539e00700 1 -- 172.21.15.69:0/3412131330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3534064150 con 0x7f3534108380 2024-04-24T08:53:42.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.306+0000 7f3538a00700 1 --2- 172.21.15.69:0/3412131330 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7f3518039600 0x7f351803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:42.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.306+0000 7f3538a00700 1 -- 172.21.15.69:0/3412131330 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f352404dbb0 con 0x7f3534108380 2024-04-24T08:53:42.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.306+0000 7f3532000700 1 -- 172.21.15.69:0/3412131330 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7f3518039600 msgr2=0x7f351803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3911821813 2024-04-24T08:53:42.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.306+0000 7f3532000700 1 --2- 172.21.15.69:0/3412131330 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7f3518039600 0x7f351803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:53:42.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.312+0000 7f3538a00700 1 -- 172.21.15.69:0/3412131330 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3524020240 con 0x7f3534108380 2024-04-24T08:53:42.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.507+0000 7f3532000700 1 -- 172.21.15.69:0/3412131330 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7f3518039600 msgr2=0x7f351803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3911821813 2024-04-24T08:53:42.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.507+0000 7f3532000700 1 --2- 172.21.15.69:0/3412131330 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7f3518039600 0x7f351803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:53:42.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.597+0000 7f3539e00700 1 -- 172.21.15.69:0/3412131330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f353402dfe0 con 0x7f3534108380 2024-04-24T08:53:42.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.598+0000 7f3538a00700 1 -- 172.21.15.69:0/3412131330 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f352400fdd0 con 0x7f3534108380 2024-04-24T08:53:42.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.601+0000 7f3539e00700 1 -- 172.21.15.69:0/3412131330 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7f3518039600 msgr2=0x7f351803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:53:42.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.601+0000 7f3539e00700 1 --2- 172.21.15.69:0/3412131330 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7f3518039600 0x7f351803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:42.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.601+0000 7f3539e00700 1 -- 172.21.15.69:0/3412131330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3534108380 msgr2=0x7f353419c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:42.251 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.601+0000 7f3539e00700 1 --2- 172.21.15.69:0/3412131330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3534108380 0x7f353419c5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3524011a50 tx=0x7f3524011b30 comp rx=0 tx=0).stop 2024-04-24T08:53:42.251 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.601+0000 7f3539e00700 1 -- 172.21.15.69:0/3412131330 shutdown_connections 2024-04-24T08:53:42.251 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.601+0000 7f3539e00700 1 --2- 172.21.15.69:0/3412131330 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7f3518039600 0x7f351803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:42.251 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.601+0000 7f3539e00700 1 --2- 172.21.15.69:0/3412131330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3534108380 0x7f353419c5c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:42.251 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.601+0000 7f3539e00700 1 -- 172.21.15.69:0/3412131330 >> 172.21.15.69:0/3412131330 conn(0x7f3534103a40 msgr2=0x7f3534104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:42.251 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.601+0000 7f3539e00700 1 -- 172.21.15.69:0/3412131330 shutdown_connections 2024-04-24T08:53:42.251 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:41.601+0000 7f3539e00700 1 -- 172.21.15.69:0/3412131330 wait complete. 2024-04-24T08:53:42.251 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for the mgr to restart... 2024-04-24T08:53:42.251 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr epoch 5... 2024-04-24T08:53:43.091 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:42 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:42.843+0000 7f217d108200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T08:53:43.091 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:43 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:43.068+0000 7f217d108200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T08:53:43.591 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:43 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:43.284+0000 7f217d108200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T08:53:44.036 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:43 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:43.664+0000 7f217d108200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T08:53:44.037 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:43 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:43.786+0000 7f217d108200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T08:53:44.291 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:44 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:44.036+0000 7f217d108200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T08:53:44.591 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:44 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:44.290+0000 7f217d108200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:53:45.341 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:45 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:45.060+0000 7f217d108200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T08:53:46.004 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:45 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:45.611+0000 7f217d108200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T08:53:46.005 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:45 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:45.740+0000 7f217d108200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T08:53:46.005 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:46 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:46.004+0000 7f217d108200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T08:53:47.091 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:46 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:46.648+0000 7f217d108200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T08:53:47.091 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:46 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:46.772+0000 7f217d108200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T08:53:47.397 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:47 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:47.093+0000 7f217d108200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T08:53:47.397 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:47 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:47.270+0000 7f217d108200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T08:53:47.841 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:47 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:47.396+0000 7f217d108200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T08:53:47.841 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:47 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:47.548+0000 7f217d108200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T08:53:48.341 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:47 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:47.914+0000 7f217d108200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T08:53:48.944 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:48 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:48.662+0000 7f217d108200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T08:53:49.341 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:48 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:53:48.944+0000 7f217d108200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T08:53:49.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:49 smithi069 ceph-mon[28170]: Active manager daemon x restarted 2024-04-24T08:53:49.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:49 smithi069 ceph-mon[28170]: Activating manager daemon x 2024-04-24T08:53:49.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:49 smithi069 ceph-mon[28170]: osdmap e2: 0 total, 0 up, 0 in 2024-04-24T08:53:49.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:49 smithi069 ceph-mon[28170]: mgrmap e6: x(active, starting, since 0.00966234s) 2024-04-24T08:53:49.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:49 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:53:49.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:49 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T08:53:49.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:49 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:53:49.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:49 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:53:49.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:49 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:53:49.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:49 smithi069 ceph-mon[28170]: Manager daemon x is now available 2024-04-24T08:53:50.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:50 smithi069 ceph-mon[28170]: Found migration_current of "None". Setting to last migration. 2024-04-24T08:53:50.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:50 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' 2024-04-24T08:53:50.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:50 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' 2024-04-24T08:53:50.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:50 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:53:50.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:50 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:53:50.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:50 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:53:50.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:50 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T08:53:50.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:50 smithi069 ceph-mon[28170]: mgrmap e7: x(active, since 1.01603s) 2024-04-24T08:53:50.811 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:53:50.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-24T08:53:50.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T08:53:50.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:53:50.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.823+0000 7feff7000700 1 Processor -- start 2024-04-24T08:53:50.812 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.823+0000 7feff7000700 1 -- start start 2024-04-24T08:53:50.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.823+0000 7feff7000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff0108380 0x7feff01087a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:50.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.823+0000 7feff7000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feff0108ce0 con 0x7feff0108380 2024-04-24T08:53:50.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.824+0000 7fefefe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff0108380 0x7feff01087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:50.813 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.824+0000 7fefefe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff0108380 0x7feff01087a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57772/0 (socket says 172.21.15.69:57772) 2024-04-24T08:53:50.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.824+0000 7fefefe00700 1 -- 172.21.15.69:0/2263228511 learned_addr learned my addr 172.21.15.69:0/2263228511 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:50.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.824+0000 7fefefe00700 1 -- 172.21.15.69:0/2263228511 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff0108e60 con 0x7feff0108380 2024-04-24T08:53:50.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.825+0000 7fefefe00700 1 --2- 172.21.15.69:0/2263228511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff0108380 0x7feff01087a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fefe00098a0 tx=0x7fefe0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd229b536196429 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:50.814 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.825+0000 7fefeea00700 1 -- 172.21.15.69:0/2263228511 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe0004160 con 0x7feff0108380 2024-04-24T08:53:50.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.825+0000 7fefeea00700 1 -- 172.21.15.69:0/2263228511 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fefe0004300 con 0x7feff0108380 2024-04-24T08:53:50.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.825+0000 7fefeea00700 1 -- 172.21.15.69:0/2263228511 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe0011a80 con 0x7feff0108380 2024-04-24T08:53:50.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.826+0000 7feff7000700 1 -- 172.21.15.69:0/2263228511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff0108380 msgr2=0x7feff01087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:50.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.826+0000 7feff7000700 1 --2- 172.21.15.69:0/2263228511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff0108380 0x7feff01087a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fefe00098a0 tx=0x7fefe0009bb0 comp rx=0 tx=0).stop 2024-04-24T08:53:50.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.826+0000 7feff7000700 1 -- 172.21.15.69:0/2263228511 shutdown_connections 2024-04-24T08:53:50.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.826+0000 7feff7000700 1 --2- 172.21.15.69:0/2263228511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff0108380 0x7feff01087a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:50.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.826+0000 7feff7000700 1 -- 172.21.15.69:0/2263228511 >> 172.21.15.69:0/2263228511 conn(0x7feff0103a40 msgr2=0x7feff0105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:50.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.826+0000 7feff7000700 1 -- 172.21.15.69:0/2263228511 shutdown_connections 2024-04-24T08:53:50.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.826+0000 7feff7000700 1 -- 172.21.15.69:0/2263228511 wait complete. 2024-04-24T08:53:50.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.827+0000 7feff7000700 1 Processor -- start 2024-04-24T08:53:50.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.827+0000 7feff7000700 1 -- start start 2024-04-24T08:53:50.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.828+0000 7feff7000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff0108380 0x7feff019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:50.817 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.828+0000 7feff7000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feff010a380 con 0x7feff0108380 2024-04-24T08:53:50.817 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.828+0000 7fefefe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff0108380 0x7feff019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:50.817 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.828+0000 7fefefe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff0108380 0x7feff019e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57784/0 (socket says 172.21.15.69:57784) 2024-04-24T08:53:50.817 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.828+0000 7fefefe00700 1 -- 172.21.15.69:0/3696911082 learned_addr learned my addr 172.21.15.69:0/3696911082 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:50.817 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.828+0000 7fefefe00700 1 -- 172.21.15.69:0/3696911082 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff019ed10 con 0x7feff0108380 2024-04-24T08:53:50.817 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.828+0000 7fefefe00700 1 --2- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff0108380 0x7feff019e7d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fefe0011a50 tx=0x7fefe0011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:50.818 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.829+0000 7fefecc00700 1 -- 172.21.15.69:0/3696911082 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe0003a80 con 0x7feff0108380 2024-04-24T08:53:50.818 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.829+0000 7fefecc00700 1 -- 172.21.15.69:0/3696911082 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fefe0003c20 con 0x7feff0108380 2024-04-24T08:53:50.818 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.829+0000 7feff7000700 1 -- 172.21.15.69:0/3696911082 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff019efa0 con 0x7feff0108380 2024-04-24T08:53:50.818 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.829+0000 7fefecc00700 1 -- 172.21.15.69:0/3696911082 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefe001a650 con 0x7feff0108380 2024-04-24T08:53:50.818 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.829+0000 7feff7000700 1 -- 172.21.15.69:0/3696911082 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff01a1c90 con 0x7feff0108380 2024-04-24T08:53:50.818 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.830+0000 7fefecc00700 1 -- 172.21.15.69:0/3696911082 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fefe0024440 con 0x7feff0108380 2024-04-24T08:53:50.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.831+0000 7fefecc00700 1 --2- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fefd4039600 0x7fefd403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:50.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.831+0000 7fefecc00700 1 -- 172.21.15.69:0/3696911082 --> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fefd403c1d0 con 0x7fefd4039600 2024-04-24T08:53:50.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.831+0000 7fefecc00700 1 -- 172.21.15.69:0/3696911082 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fefe004db40 con 0x7feff0108380 2024-04-24T08:53:50.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.831+0000 7fefef400700 1 -- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fefd4039600 msgr2=0x7fefd403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3911821813 2024-04-24T08:53:50.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:42.831+0000 7fefef400700 1 --2- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fefd4039600 0x7fefd403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:53:50.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:43.032+0000 7fefef400700 1 -- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fefd4039600 msgr2=0x7fefd403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3911821813 2024-04-24T08:53:50.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:43.032+0000 7fefef400700 1 --2- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fefd4039600 0x7fefd403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:53:50.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:43.433+0000 7fefef400700 1 -- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fefd4039600 msgr2=0x7fefd403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3911821813 2024-04-24T08:53:50.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:43.433+0000 7fefef400700 1 --2- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fefd4039600 0x7fefd403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T08:53:50.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:44.234+0000 7fefef400700 1 -- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fefd4039600 msgr2=0x7fefd403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3911821813 2024-04-24T08:53:50.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:44.234+0000 7fefef400700 1 --2- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fefd4039600 0x7fefd403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T08:53:50.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:45.836+0000 7fefef400700 1 -- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fefd4039600 msgr2=0x7fefd403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3911821813 2024-04-24T08:53:50.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:45.836+0000 7fefef400700 1 --2- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fefd4039600 0x7fefd403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T08:53:50.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:48.953+0000 7fefecc00700 1 -- 172.21.15.69:0/3696911082 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7fefe0024ad0 con 0x7feff0108380 2024-04-24T08:53:50.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:48.954+0000 7fefecc00700 1 -- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fefd4039600 msgr2=0x7fefd403bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:53:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:48.954+0000 7fefecc00700 1 --2- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fefd4039600 0x7fefd403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.957+0000 7fefecc00700 1 -- 172.21.15.69:0/3696911082 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fefe0018d00 con 0x7feff0108380 2024-04-24T08:53:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.958+0000 7fefecc00700 1 --2- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7fefd403cef0 0x7fefd403f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.958+0000 7fefecc00700 1 -- 172.21.15.69:0/3696911082 --> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fefe0046c70 con 0x7fefd403cef0 2024-04-24T08:53:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.960+0000 7fefef400700 1 --2- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7fefd403cef0 0x7fefd403f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.963+0000 7fefef400700 1 --2- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7fefd403cef0 0x7fefd403f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fefdc003a10 tx=0x7fefdc0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.964+0000 7fefecc00700 1 -- 172.21.15.69:0/3696911082 <== mgr.14120 v2:172.21.15.69:6800/3470783122 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fefe0046c70 con 0x7fefd403cef0 2024-04-24T08:53:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.971+0000 7feff7000700 1 -- 172.21.15.69:0/3696911082 --> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7feff0064150 con 0x7fefd403cef0 2024-04-24T08:53:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.972+0000 7fefecc00700 1 -- 172.21.15.69:0/3696911082 <== mgr.14120 v2:172.21.15.69:6800/3470783122 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7feff0064150 con 0x7fefd403cef0 2024-04-24T08:53:50.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.972+0000 7feff7000700 1 -- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7fefd403cef0 msgr2=0x7fefd403f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:50.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.972+0000 7feff7000700 1 --2- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7fefd403cef0 0x7fefd403f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fefdc003a10 tx=0x7fefdc0092b0 comp rx=0 tx=0).stop 2024-04-24T08:53:50.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.973+0000 7feff7000700 1 -- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff0108380 msgr2=0x7feff019e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:50.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.973+0000 7feff7000700 1 --2- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff0108380 0x7feff019e7d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fefe0011a50 tx=0x7fefe0011b30 comp rx=0 tx=0).stop 2024-04-24T08:53:50.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.973+0000 7feff7000700 1 -- 172.21.15.69:0/3696911082 shutdown_connections 2024-04-24T08:53:50.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.973+0000 7feff7000700 1 --2- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7fefd403cef0 0x7fefd403f2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:50.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.973+0000 7feff7000700 1 --2- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feff0108380 0x7feff019e7d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:50.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.973+0000 7feff7000700 1 --2- 172.21.15.69:0/3696911082 >> [v2:172.21.15.69:6800/3911821813,v1:172.21.15.69:6801/3911821813] conn(0x7fefd4039600 0x7fefd403bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:50.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.973+0000 7feff7000700 1 -- 172.21.15.69:0/3696911082 >> 172.21.15.69:0/3696911082 conn(0x7feff0103a40 msgr2=0x7feff01046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:50.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.973+0000 7feff7000700 1 -- 172.21.15.69:0/3696911082 shutdown_connections 2024-04-24T08:53:50.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:49.973+0000 7feff7000700 1 -- 172.21.15.69:0/3696911082 wait complete. 2024-04-24T08:53:50.823 INFO:teuthology.orchestra.run.smithi069.stdout:mgr epoch 5 is available 2024-04-24T08:53:50.823 INFO:teuthology.orchestra.run.smithi069.stdout:Setting orchestrator backend to cephadm... 2024-04-24T08:53:51.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:51 smithi069 ceph-mon[28170]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T08:53:51.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:51 smithi069 ceph-mon[28170]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T08:53:51.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:51 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' 2024-04-24T08:53:51.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:51 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' 2024-04-24T08:53:51.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:51 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:53:52.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:52 smithi069 ceph-mon[28170]: [24/Apr/2024:08:53:50] ENGINE Bus STARTING 2024-04-24T08:53:52.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:52 smithi069 ceph-mon[28170]: [24/Apr/2024:08:53:50] ENGINE Serving on https://172.21.15.69:7150 2024-04-24T08:53:52.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:52 smithi069 ceph-mon[28170]: [24/Apr/2024:08:53:50] ENGINE Serving on http://172.21.15.69:8765 2024-04-24T08:53:52.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:52 smithi069 ceph-mon[28170]: [24/Apr/2024:08:53:50] ENGINE Bus STARTED 2024-04-24T08:53:52.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:52 smithi069 ceph-mon[28170]: mgrmap e8: x(active, since 2s) 2024-04-24T08:53:52.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:52 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' 2024-04-24T08:53:52.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:52 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:53:52.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.477+0000 7f760ec00700 1 Processor -- start 2024-04-24T08:53:52.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.477+0000 7f760ec00700 1 -- start start 2024-04-24T08:53:52.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.477+0000 7f760ec00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f760807dda0 0x7f760807c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:52.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.478+0000 7f760ec00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f760807c850 con 0x7f760807dda0 2024-04-24T08:53:52.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.478+0000 7f7607e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f760807dda0 0x7f760807c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:52.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.478+0000 7f7607e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f760807dda0 0x7f760807c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47656/0 (socket says 172.21.15.69:47656) 2024-04-24T08:53:52.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.478+0000 7f7607e00700 1 -- 172.21.15.69:0/1199564673 learned_addr learned my addr 172.21.15.69:0/1199564673 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:52.549 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.478+0000 7f7607e00700 1 -- 172.21.15.69:0/1199564673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760807c9d0 con 0x7f760807dda0 2024-04-24T08:53:52.550 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.479+0000 7f7607e00700 1 --2- 172.21.15.69:0/1199564673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f760807dda0 0x7f760807c310 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f75f80098a0 tx=0x7f75f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6497bee73a8f1333 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:52.550 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.479+0000 7f7607400700 1 -- 172.21.15.69:0/1199564673 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f8004160 con 0x7f760807dda0 2024-04-24T08:53:52.550 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.479+0000 7f7607400700 1 -- 172.21.15.69:0/1199564673 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f75f8004300 con 0x7f760807dda0 2024-04-24T08:53:52.550 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.479+0000 7f7607400700 1 -- 172.21.15.69:0/1199564673 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f8011a80 con 0x7f760807dda0 2024-04-24T08:53:52.550 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.479+0000 7f760ec00700 1 -- 172.21.15.69:0/1199564673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f760807dda0 msgr2=0x7f760807c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:52.550 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.479+0000 7f760ec00700 1 --2- 172.21.15.69:0/1199564673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f760807dda0 0x7f760807c310 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f75f80098a0 tx=0x7f75f8009bb0 comp rx=0 tx=0).stop 2024-04-24T08:53:52.550 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.480+0000 7f760ec00700 1 -- 172.21.15.69:0/1199564673 shutdown_connections 2024-04-24T08:53:52.550 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.480+0000 7f760ec00700 1 --2- 172.21.15.69:0/1199564673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f760807dda0 0x7f760807c310 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:52.550 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.480+0000 7f760ec00700 1 -- 172.21.15.69:0/1199564673 >> 172.21.15.69:0/1199564673 conn(0x7f7608103a40 msgr2=0x7f7608105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:52.550 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.480+0000 7f760ec00700 1 -- 172.21.15.69:0/1199564673 shutdown_connections 2024-04-24T08:53:52.550 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.480+0000 7f760ec00700 1 -- 172.21.15.69:0/1199564673 wait complete. 2024-04-24T08:53:52.550 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.481+0000 7f760ec00700 1 Processor -- start 2024-04-24T08:53:52.550 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.481+0000 7f760ec00700 1 -- start start 2024-04-24T08:53:52.551 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.481+0000 7f760ec00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f760807dda0 0x7f760819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:52.551 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.481+0000 7f760ec00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f760810e7f0 con 0x7f760807dda0 2024-04-24T08:53:52.551 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.481+0000 7f7607e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f760807dda0 0x7f760819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:52.551 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.481+0000 7f7607e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f760807dda0 0x7f760819e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47664/0 (socket says 172.21.15.69:47664) 2024-04-24T08:53:52.551 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.481+0000 7f7607e00700 1 -- 172.21.15.69:0/1508205131 learned_addr learned my addr 172.21.15.69:0/1508205131 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:52.551 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.482+0000 7f7607e00700 1 -- 172.21.15.69:0/1508205131 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760819ed10 con 0x7f760807dda0 2024-04-24T08:53:52.551 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.482+0000 7f7607e00700 1 --2- 172.21.15.69:0/1508205131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f760807dda0 0x7f760819e7d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f75f8011a50 tx=0x7f75f8011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:52.551 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.482+0000 7f7605600700 1 -- 172.21.15.69:0/1508205131 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f8003a80 con 0x7f760807dda0 2024-04-24T08:53:52.551 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.482+0000 7f7605600700 1 -- 172.21.15.69:0/1508205131 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f75f8003c20 con 0x7f760807dda0 2024-04-24T08:53:52.551 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.482+0000 7f7605600700 1 -- 172.21.15.69:0/1508205131 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f801a650 con 0x7f760807dda0 2024-04-24T08:53:52.551 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.482+0000 7f760ec00700 1 -- 172.21.15.69:0/1508205131 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760819efa0 con 0x7f760807dda0 2024-04-24T08:53:52.551 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.482+0000 7f760ec00700 1 -- 172.21.15.69:0/1508205131 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76081a1c90 con 0x7f760807dda0 2024-04-24T08:53:52.551 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.483+0000 7f7605600700 1 -- 172.21.15.69:0/1508205131 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f75f801a900 con 0x7f760807dda0 2024-04-24T08:53:52.551 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.483+0000 7f760ec00700 1 -- 172.21.15.69:0/1508205131 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7608064150 con 0x7f760807dda0 2024-04-24T08:53:52.551 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.484+0000 7f7605600700 1 --2- 172.21.15.69:0/1508205131 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f75ec039600 0x7f75ec03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:52.552 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.484+0000 7f7605600700 1 -- 172.21.15.69:0/1508205131 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f75f804ddf0 con 0x7f760807dda0 2024-04-24T08:53:52.552 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.484+0000 7f75ffe00700 1 --2- 172.21.15.69:0/1508205131 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f75ec039600 0x7f75ec03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:52.552 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.485+0000 7f75ffe00700 1 --2- 172.21.15.69:0/1508205131 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f75ec039600 0x7f75ec03bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f75f0006fd0 tx=0x7f75f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:52.552 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.490+0000 7f7605600700 1 -- 172.21.15.69:0/1508205131 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f75f8019c70 con 0x7f760807dda0 2024-04-24T08:53:52.552 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.733+0000 7f760ec00700 1 -- 172.21.15.69:0/1508205131 --> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f760802e050 con 0x7f75ec039600 2024-04-24T08:53:52.552 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.744+0000 7f7605600700 1 -- 172.21.15.69:0/1508205131 <== mgr.14120 v2:172.21.15.69:6800/3470783122 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f760802e050 con 0x7f75ec039600 2024-04-24T08:53:52.552 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.747+0000 7f760ec00700 1 -- 172.21.15.69:0/1508205131 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f75ec039600 msgr2=0x7f75ec03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:52.552 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.747+0000 7f760ec00700 1 --2- 172.21.15.69:0/1508205131 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f75ec039600 0x7f75ec03bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f75f0006fd0 tx=0x7f75f0006e40 comp rx=0 tx=0).stop 2024-04-24T08:53:52.552 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.747+0000 7f760ec00700 1 -- 172.21.15.69:0/1508205131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f760807dda0 msgr2=0x7f760819e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:52.552 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.747+0000 7f760ec00700 1 --2- 172.21.15.69:0/1508205131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f760807dda0 0x7f760819e7d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f75f8011a50 tx=0x7f75f8011b30 comp rx=0 tx=0).stop 2024-04-24T08:53:52.552 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.747+0000 7f760ec00700 1 -- 172.21.15.69:0/1508205131 shutdown_connections 2024-04-24T08:53:52.552 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.747+0000 7f760ec00700 1 --2- 172.21.15.69:0/1508205131 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f75ec039600 0x7f75ec03bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:52.552 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.747+0000 7f760ec00700 1 --2- 172.21.15.69:0/1508205131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f760807dda0 0x7f760819e7d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:52.552 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.747+0000 7f760ec00700 1 -- 172.21.15.69:0/1508205131 >> 172.21.15.69:0/1508205131 conn(0x7f7608103a40 msgr2=0x7f7608104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:52.553 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.748+0000 7f760ec00700 1 -- 172.21.15.69:0/1508205131 shutdown_connections 2024-04-24T08:53:52.553 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:51.748+0000 7f760ec00700 1 -- 172.21.15.69:0/1508205131 wait complete. 2024-04-24T08:53:53.391 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:53 smithi069 ceph-mon[28170]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:53:54.305 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-24T08:53:54.306 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.151+0000 7f46d0800700 1 Processor -- start 2024-04-24T08:53:54.306 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.152+0000 7f46d0800700 1 -- start start 2024-04-24T08:53:54.306 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.152+0000 7f46d0800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46c810fff0 0x7f46c8110410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:54.306 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.152+0000 7f46d0800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f46c81109e0 con 0x7f46c810fff0 2024-04-24T08:53:54.306 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.152+0000 7f46cde00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46c810fff0 0x7f46c8110410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:54.306 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.152+0000 7f46cde00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46c810fff0 0x7f46c8110410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52902/0 (socket says 172.21.15.69:52902) 2024-04-24T08:53:54.306 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.152+0000 7f46cde00700 1 -- 172.21.15.69:0/1654173565 learned_addr learned my addr 172.21.15.69:0/1654173565 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:54.306 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.153+0000 7f46cde00700 1 -- 172.21.15.69:0/1654173565 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c810ece0 con 0x7f46c810fff0 2024-04-24T08:53:54.306 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.153+0000 7f46cde00700 1 --2- 172.21.15.69:0/1654173565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46c810fff0 0x7f46c8110410 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0098a0 tx=0x7f46bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec322d5d73bbb9f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:54.306 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.153+0000 7f46cca00700 1 -- 172.21.15.69:0/1654173565 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46bc004160 con 0x7f46c810fff0 2024-04-24T08:53:54.306 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.153+0000 7f46cca00700 1 -- 172.21.15.69:0/1654173565 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f46bc004300 con 0x7f46c810fff0 2024-04-24T08:53:54.307 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.153+0000 7f46cca00700 1 -- 172.21.15.69:0/1654173565 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46bc011a80 con 0x7f46c810fff0 2024-04-24T08:53:54.307 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.154+0000 7f46d0800700 1 -- 172.21.15.69:0/1654173565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46c810fff0 msgr2=0x7f46c8110410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:54.307 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.154+0000 7f46d0800700 1 --2- 172.21.15.69:0/1654173565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46c810fff0 0x7f46c8110410 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0098a0 tx=0x7f46bc009bb0 comp rx=0 tx=0).stop 2024-04-24T08:53:54.307 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.154+0000 7f46d0800700 1 -- 172.21.15.69:0/1654173565 shutdown_connections 2024-04-24T08:53:54.307 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.154+0000 7f46d0800700 1 --2- 172.21.15.69:0/1654173565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46c810fff0 0x7f46c8110410 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:54.307 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.154+0000 7f46d0800700 1 -- 172.21.15.69:0/1654173565 >> 172.21.15.69:0/1654173565 conn(0x7f46c807dbd0 msgr2=0x7f46c807fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:54.307 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.154+0000 7f46d0800700 1 -- 172.21.15.69:0/1654173565 shutdown_connections 2024-04-24T08:53:54.307 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.154+0000 7f46d0800700 1 -- 172.21.15.69:0/1654173565 wait complete. 2024-04-24T08:53:54.307 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.155+0000 7f46d0800700 1 Processor -- start 2024-04-24T08:53:54.307 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.155+0000 7f46d0800700 1 -- start start 2024-04-24T08:53:54.307 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.156+0000 7f46d0800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46c810fff0 0x7f46c81a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:54.307 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.156+0000 7f46d0800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f46c8110b60 con 0x7f46c810fff0 2024-04-24T08:53:54.307 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.156+0000 7f46cde00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46c810fff0 0x7f46c81a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:54.307 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.156+0000 7f46cde00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46c810fff0 0x7f46c81a2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52916/0 (socket says 172.21.15.69:52916) 2024-04-24T08:53:54.308 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.156+0000 7f46cde00700 1 -- 172.21.15.69:0/2130901894 learned_addr learned my addr 172.21.15.69:0/2130901894 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:54.308 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.156+0000 7f46cde00700 1 -- 172.21.15.69:0/2130901894 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c81a30f0 con 0x7f46c810fff0 2024-04-24T08:53:54.308 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.157+0000 7f46cde00700 1 --2- 172.21.15.69:0/2130901894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46c810fff0 0x7f46c81a2bb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f46bc006af0 tx=0x7f46bc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:54.308 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.157+0000 7f46c2a00700 1 -- 172.21.15.69:0/2130901894 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46bc0039e0 con 0x7f46c810fff0 2024-04-24T08:53:54.308 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.157+0000 7f46c2a00700 1 -- 172.21.15.69:0/2130901894 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f46bc003b80 con 0x7f46c810fff0 2024-04-24T08:53:54.308 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.157+0000 7f46c2a00700 1 -- 172.21.15.69:0/2130901894 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46bc0196a0 con 0x7f46c810fff0 2024-04-24T08:53:54.308 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.157+0000 7f46d0800700 1 -- 172.21.15.69:0/2130901894 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c81a3380 con 0x7f46c810fff0 2024-04-24T08:53:54.308 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.157+0000 7f46d0800700 1 -- 172.21.15.69:0/2130901894 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c81a6070 con 0x7f46c810fff0 2024-04-24T08:53:54.308 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.158+0000 7f46c2a00700 1 -- 172.21.15.69:0/2130901894 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f46bc01ab50 con 0x7f46c810fff0 2024-04-24T08:53:54.308 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.158+0000 7f46d0800700 1 -- 172.21.15.69:0/2130901894 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c8064150 con 0x7f46c810fff0 2024-04-24T08:53:54.308 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.159+0000 7f46c2a00700 1 --2- 172.21.15.69:0/2130901894 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f46b00395b0 0x7f46b003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:54.308 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.159+0000 7f46c2a00700 1 -- 172.21.15.69:0/2130901894 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f46bc04dc70 con 0x7f46c810fff0 2024-04-24T08:53:54.309 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.159+0000 7f46cd400700 1 --2- 172.21.15.69:0/2130901894 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f46b00395b0 0x7f46b003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:54.309 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.160+0000 7f46cd400700 1 --2- 172.21.15.69:0/2130901894 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f46b00395b0 0x7f46b003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f46b8006fd0 tx=0x7f46b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:54.309 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.165+0000 7f46c2a00700 1 -- 172.21.15.69:0/2130901894 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46bc00b3e0 con 0x7f46c810fff0 2024-04-24T08:53:54.309 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.390+0000 7f46d0800700 1 -- 172.21.15.69:0/2130901894 --> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f46c802e050 con 0x7f46b00395b0 2024-04-24T08:53:54.309 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.393+0000 7f46c2a00700 1 -- 172.21.15.69:0/2130901894 <== mgr.14120 v2:172.21.15.69:6800/3470783122 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f46c802e050 con 0x7f46b00395b0 2024-04-24T08:53:54.309 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.396+0000 7f46d0800700 1 -- 172.21.15.69:0/2130901894 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f46b00395b0 msgr2=0x7f46b003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:54.309 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.396+0000 7f46d0800700 1 --2- 172.21.15.69:0/2130901894 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f46b00395b0 0x7f46b003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f46b8006fd0 tx=0x7f46b8006e40 comp rx=0 tx=0).stop 2024-04-24T08:53:54.309 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.396+0000 7f46d0800700 1 -- 172.21.15.69:0/2130901894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46c810fff0 msgr2=0x7f46c81a2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:54.309 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.396+0000 7f46d0800700 1 --2- 172.21.15.69:0/2130901894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46c810fff0 0x7f46c81a2bb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f46bc006af0 tx=0x7f46bc011c40 comp rx=0 tx=0).stop 2024-04-24T08:53:54.309 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.396+0000 7f46d0800700 1 -- 172.21.15.69:0/2130901894 shutdown_connections 2024-04-24T08:53:54.309 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.396+0000 7f46d0800700 1 --2- 172.21.15.69:0/2130901894 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f46b00395b0 0x7f46b003ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:54.309 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.396+0000 7f46d0800700 1 --2- 172.21.15.69:0/2130901894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46c810fff0 0x7f46c81a2bb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:54.309 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.396+0000 7f46d0800700 1 -- 172.21.15.69:0/2130901894 >> 172.21.15.69:0/2130901894 conn(0x7f46c807dbd0 msgr2=0x7f46c807e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:54.310 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.396+0000 7f46d0800700 1 -- 172.21.15.69:0/2130901894 shutdown_connections 2024-04-24T08:53:54.310 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:53.396+0000 7f46d0800700 1 -- 172.21.15.69:0/2130901894 wait complete. 2024-04-24T08:53:54.310 INFO:teuthology.orchestra.run.smithi069.stdout:Generating ssh key... 2024-04-24T08:53:54.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:54 smithi069 ceph-mon[28170]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:53:55.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-mon[28170]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:53:55.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-mon[28170]: Generating ssh key... 2024-04-24T08:53:56.093 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: Generating public/private rsa key pair. 2024-04-24T08:53:56.093 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: Your identification has been saved in /tmp/tmpq1kmddon/key. 2024-04-24T08:53:56.094 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: Your public key has been saved in /tmp/tmpq1kmddon/key.pub. 2024-04-24T08:53:56.094 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: The key fingerprint is: 2024-04-24T08:53:56.094 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: SHA256:yXyyG0rEHvjeNnTR4S2JaEXlV3ONxlVvqfmE33VcedM ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:53:56.094 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: The key's randomart image is: 2024-04-24T08:53:56.094 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: +---[RSA 3072]----+ 2024-04-24T08:53:56.094 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: | ..... +B| 2024-04-24T08:53:56.094 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: | ... +.O| 2024-04-24T08:53:56.095 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: | o +.=.+E| 2024-04-24T08:53:56.095 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: | o oo.o =.=o+| 2024-04-24T08:53:56.095 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: | . +.S .. = .+| 2024-04-24T08:53:56.095 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: | + ..+. + +| 2024-04-24T08:53:56.095 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: | +.o. o.| 2024-04-24T08:53:56.095 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: | o ooo | 2024-04-24T08:53:56.095 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: | o.o. | 2024-04-24T08:53:56.095 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:53:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: +----[SHA256]-----+ 2024-04-24T08:53:56.497 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.936+0000 7fea72a00700 1 Processor -- start 2024-04-24T08:53:56.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.937+0000 7fea72a00700 1 -- start start 2024-04-24T08:53:56.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.937+0000 7fea72a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c1083a0 0x7fea6c1087c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:56.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.937+0000 7fea72a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c108d90 con 0x7fea6c1083a0 2024-04-24T08:53:56.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.937+0000 7fea6be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c1083a0 0x7fea6c1087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:56.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.938+0000 7fea6be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c1083a0 0x7fea6c1087c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52930/0 (socket says 172.21.15.69:52930) 2024-04-24T08:53:56.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.938+0000 7fea6be00700 1 -- 172.21.15.69:0/2149778778 learned_addr learned my addr 172.21.15.69:0/2149778778 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:56.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.938+0000 7fea6be00700 1 -- 172.21.15.69:0/2149778778 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6c109550 con 0x7fea6c1083a0 2024-04-24T08:53:56.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.938+0000 7fea6be00700 1 --2- 172.21.15.69:0/2149778778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c1083a0 0x7fea6c1087c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0098a0 tx=0x7fea5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ae8f81f5e769bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:56.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.938+0000 7fea6aa00700 1 -- 172.21.15.69:0/2149778778 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea5c004160 con 0x7fea6c1083a0 2024-04-24T08:53:56.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.939+0000 7fea6aa00700 1 -- 172.21.15.69:0/2149778778 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fea5c004300 con 0x7fea6c1083a0 2024-04-24T08:53:56.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.939+0000 7fea6aa00700 1 -- 172.21.15.69:0/2149778778 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea5c011a80 con 0x7fea6c1083a0 2024-04-24T08:53:56.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.939+0000 7fea72a00700 1 -- 172.21.15.69:0/2149778778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c1083a0 msgr2=0x7fea6c1087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:56.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.939+0000 7fea72a00700 1 --2- 172.21.15.69:0/2149778778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c1083a0 0x7fea6c1087c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0098a0 tx=0x7fea5c009bb0 comp rx=0 tx=0).stop 2024-04-24T08:53:56.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.939+0000 7fea72a00700 1 -- 172.21.15.69:0/2149778778 shutdown_connections 2024-04-24T08:53:56.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.939+0000 7fea72a00700 1 --2- 172.21.15.69:0/2149778778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c1083a0 0x7fea6c1087c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:56.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.939+0000 7fea72a00700 1 -- 172.21.15.69:0/2149778778 >> 172.21.15.69:0/2149778778 conn(0x7fea6c1039e0 msgr2=0x7fea6c105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:56.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.939+0000 7fea72a00700 1 -- 172.21.15.69:0/2149778778 shutdown_connections 2024-04-24T08:53:56.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.940+0000 7fea72a00700 1 -- 172.21.15.69:0/2149778778 wait complete. 2024-04-24T08:53:56.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.941+0000 7fea72a00700 1 Processor -- start 2024-04-24T08:53:56.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.942+0000 7fea72a00700 1 -- start start 2024-04-24T08:53:56.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.942+0000 7fea72a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c1083a0 0x7fea6c07db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:56.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.942+0000 7fea72a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c10c900 con 0x7fea6c1083a0 2024-04-24T08:53:56.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.942+0000 7fea6be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c1083a0 0x7fea6c07db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:56.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.943+0000 7fea6be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c1083a0 0x7fea6c07db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52938/0 (socket says 172.21.15.69:52938) 2024-04-24T08:53:56.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.943+0000 7fea6be00700 1 -- 172.21.15.69:0/2737589127 learned_addr learned my addr 172.21.15.69:0/2737589127 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:56.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.943+0000 7fea6be00700 1 -- 172.21.15.69:0/2737589127 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6c07e0c0 con 0x7fea6c1083a0 2024-04-24T08:53:56.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.943+0000 7fea6be00700 1 --2- 172.21.15.69:0/2737589127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c1083a0 0x7fea6c07db80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fea5c000c00 tx=0x7fea5c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:56.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.943+0000 7fea68c00700 1 -- 172.21.15.69:0/2737589127 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea5c0039f0 con 0x7fea6c1083a0 2024-04-24T08:53:56.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.944+0000 7fea68c00700 1 -- 172.21.15.69:0/2737589127 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fea5c003b90 con 0x7fea6c1083a0 2024-04-24T08:53:56.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.944+0000 7fea68c00700 1 -- 172.21.15.69:0/2737589127 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea5c01a740 con 0x7fea6c1083a0 2024-04-24T08:53:56.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.944+0000 7fea72a00700 1 -- 172.21.15.69:0/2737589127 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6c07c2b0 con 0x7fea6c1083a0 2024-04-24T08:53:56.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.944+0000 7fea72a00700 1 -- 172.21.15.69:0/2737589127 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea6c07c7f0 con 0x7fea6c1083a0 2024-04-24T08:53:56.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.945+0000 7fea72a00700 1 -- 172.21.15.69:0/2737589127 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea6c064150 con 0x7fea6c1083a0 2024-04-24T08:53:56.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.945+0000 7fea68c00700 1 -- 172.21.15.69:0/2737589127 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fea5c024af0 con 0x7fea6c1083a0 2024-04-24T08:53:56.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.946+0000 7fea68c00700 1 --2- 172.21.15.69:0/2737589127 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7fea50039600 0x7fea5003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:56.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.946+0000 7fea68c00700 1 -- 172.21.15.69:0/2737589127 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fea5c04dc90 con 0x7fea6c1083a0 2024-04-24T08:53:56.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.946+0000 7fea6b400700 1 --2- 172.21.15.69:0/2737589127 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7fea50039600 0x7fea5003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:56.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.947+0000 7fea6b400700 1 --2- 172.21.15.69:0/2737589127 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7fea50039600 0x7fea5003bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea58006fd0 tx=0x7fea58006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:56.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:54.951+0000 7fea68c00700 1 -- 172.21.15.69:0/2737589127 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea5c00b5a0 con 0x7fea6c1083a0 2024-04-24T08:53:56.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:55.181+0000 7fea72a00700 1 -- 172.21.15.69:0/2737589127 --> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fea6c195db0 con 0x7fea50039600 2024-04-24T08:53:56.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:55.822+0000 7fea68c00700 1 -- 172.21.15.69:0/2737589127 <== mgr.14120 v2:172.21.15.69:6800/3470783122 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fea6c195db0 con 0x7fea50039600 2024-04-24T08:53:56.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:55.824+0000 7fea72a00700 1 -- 172.21.15.69:0/2737589127 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7fea50039600 msgr2=0x7fea5003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:56.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:55.824+0000 7fea72a00700 1 --2- 172.21.15.69:0/2737589127 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7fea50039600 0x7fea5003bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea58006fd0 tx=0x7fea58006e40 comp rx=0 tx=0).stop 2024-04-24T08:53:56.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:55.825+0000 7fea72a00700 1 -- 172.21.15.69:0/2737589127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c1083a0 msgr2=0x7fea6c07db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:56.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:55.825+0000 7fea72a00700 1 --2- 172.21.15.69:0/2737589127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c1083a0 0x7fea6c07db80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fea5c000c00 tx=0x7fea5c011b00 comp rx=0 tx=0).stop 2024-04-24T08:53:56.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:55.825+0000 7fea72a00700 1 -- 172.21.15.69:0/2737589127 shutdown_connections 2024-04-24T08:53:56.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:55.825+0000 7fea72a00700 1 --2- 172.21.15.69:0/2737589127 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7fea50039600 0x7fea5003bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:56.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:55.825+0000 7fea72a00700 1 --2- 172.21.15.69:0/2737589127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c1083a0 0x7fea6c07db80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:56.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:55.825+0000 7fea72a00700 1 -- 172.21.15.69:0/2737589127 >> 172.21.15.69:0/2737589127 conn(0x7fea6c1039e0 msgr2=0x7fea6c104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:56.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:55.825+0000 7fea72a00700 1 -- 172.21.15.69:0/2737589127 shutdown_connections 2024-04-24T08:53:56.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:55.825+0000 7fea72a00700 1 -- 172.21.15.69:0/2737589127 wait complete. 2024-04-24T08:53:57.048 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:56 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' 2024-04-24T08:53:57.048 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:56 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' 2024-04-24T08:53:58.028 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoPmsuDpFJRrlNilvQZllprWXojEuEiZvDWlwE3H384v9WEYLlUN1SpjvcJwIOc4Csxy7oKPSjiCZaTcV4c5rI9XdSBLUgE93khBI9GAxHBszJydJQcWNA1YwpCU0+o/7GjttuFlFIQ+vzydEGHNDE8sqjJXiT5oMLJZG9/vBLgrebReUmDwbTZgjzwKtny3aIKySNl0kItSOPN0yUbbX3IpEN1EAkkn6/TP8DmrKkjMIebwlg6SOrOEaQIInEQ3WEY4beN16u8CwLBgNuVH/mApSCC/2HOfx9l0KWLP738BYZdWNA6V7vdKLfmp82DkIYfBTih5T+ueJmUknY2F2MVsjYxZ70NyfTPC1AsWrdgWgSQ4hACy+oKg0P/92DJsjIelDodKF9XmQLSUxc5+Dfg7QlKwpmjyUZe433mRhsvI+nTWzNU0I4KmtEkCUq2V7ZnIXZHIOuiQYKlMGgZyycZwoZd9pZQOLp9GEAkwc3me5hcbprfUhtSFDHg/ThR4c= ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:53:58.030 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.104+0000 7f1922200700 1 Processor -- start 2024-04-24T08:53:58.030 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.104+0000 7f1922200700 1 -- start start 2024-04-24T08:53:58.030 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.104+0000 7f1922200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f191c10ef40 0x7f191c10f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.104+0000 7f1922200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f191c10f930 con 0x7f191c10ef40 2024-04-24T08:53:58.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.105+0000 7f191b400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f191c10ef40 0x7f191c10f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.105+0000 7f191b400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f191c10ef40 0x7f191c10f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52944/0 (socket says 172.21.15.69:52944) 2024-04-24T08:53:58.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.105+0000 7f191b400700 1 -- 172.21.15.69:0/1602345769 learned_addr learned my addr 172.21.15.69:0/1602345769 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:58.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.105+0000 7f191b400700 1 -- 172.21.15.69:0/1602345769 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191c10fab0 con 0x7f191c10ef40 2024-04-24T08:53:58.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.106+0000 7f191b400700 1 --2- 172.21.15.69:0/1602345769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f191c10ef40 0x7f191c10f360 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f190c0098a0 tx=0x7f190c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee066ee898a0ceff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:58.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.106+0000 7f191a000700 1 -- 172.21.15.69:0/1602345769 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f190c004160 con 0x7f191c10ef40 2024-04-24T08:53:58.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.106+0000 7f191a000700 1 -- 172.21.15.69:0/1602345769 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f190c004300 con 0x7f191c10ef40 2024-04-24T08:53:58.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.106+0000 7f191a000700 1 -- 172.21.15.69:0/1602345769 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f190c011a80 con 0x7f191c10ef40 2024-04-24T08:53:58.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.107+0000 7f1922200700 1 -- 172.21.15.69:0/1602345769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f191c10ef40 msgr2=0x7f191c10f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.032 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.107+0000 7f1922200700 1 --2- 172.21.15.69:0/1602345769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f191c10ef40 0x7f191c10f360 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f190c0098a0 tx=0x7f190c009bb0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.032 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.107+0000 7f1922200700 1 -- 172.21.15.69:0/1602345769 shutdown_connections 2024-04-24T08:53:58.037 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.107+0000 7f1922200700 1 --2- 172.21.15.69:0/1602345769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f191c10ef40 0x7f191c10f360 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.107+0000 7f1922200700 1 -- 172.21.15.69:0/1602345769 >> 172.21.15.69:0/1602345769 conn(0x7f191c07dc20 msgr2=0x7f191c080040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.107+0000 7f1922200700 1 -- 172.21.15.69:0/1602345769 shutdown_connections 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.107+0000 7f1922200700 1 -- 172.21.15.69:0/1602345769 wait complete. 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.108+0000 7f1922200700 1 Processor -- start 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.108+0000 7f1922200700 1 -- start start 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.109+0000 7f1922200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f191c10ef40 0x7f191c1a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.109+0000 7f1922200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f191c110ac0 con 0x7f191c10ef40 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.109+0000 7f191b400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f191c10ef40 0x7f191c1a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.109+0000 7f191b400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f191c10ef40 0x7f191c1a2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52956/0 (socket says 172.21.15.69:52956) 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.109+0000 7f191b400700 1 -- 172.21.15.69:0/3739702637 learned_addr learned my addr 172.21.15.69:0/3739702637 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.109+0000 7f191b400700 1 -- 172.21.15.69:0/3739702637 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191c1a3120 con 0x7f191c10ef40 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.110+0000 7f191b400700 1 --2- 172.21.15.69:0/3739702637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f191c10ef40 0x7f191c1a2be0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f190c000c00 tx=0x7f190c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.110+0000 7f1913e00700 1 -- 172.21.15.69:0/3739702637 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f190c0039f0 con 0x7f191c10ef40 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.110+0000 7f1913e00700 1 -- 172.21.15.69:0/3739702637 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f190c003b90 con 0x7f191c10ef40 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.110+0000 7f1913e00700 1 -- 172.21.15.69:0/3739702637 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f190c01a740 con 0x7f191c10ef40 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.110+0000 7f1922200700 1 -- 172.21.15.69:0/3739702637 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191c1a33b0 con 0x7f191c10ef40 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.110+0000 7f1922200700 1 -- 172.21.15.69:0/3739702637 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191c1a6050 con 0x7f191c10ef40 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.111+0000 7f1922200700 1 -- 172.21.15.69:0/3739702637 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f191c064150 con 0x7f191c10ef40 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.111+0000 7f1913e00700 1 -- 172.21.15.69:0/3739702637 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f190c024af0 con 0x7f191c10ef40 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.112+0000 7f1913e00700 1 --2- 172.21.15.69:0/3739702637 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f19000395b0 0x7f190003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.112+0000 7f1913e00700 1 -- 172.21.15.69:0/3739702637 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f190c04db70 con 0x7f191c10ef40 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.112+0000 7f191aa00700 1 --2- 172.21.15.69:0/3739702637 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f19000395b0 0x7f190003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.113+0000 7f191aa00700 1 --2- 172.21.15.69:0/3739702637 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f19000395b0 0x7f190003ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1908006fd0 tx=0x7f1908006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.117+0000 7f1913e00700 1 -- 172.21.15.69:0/3739702637 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f190c00ba20 con 0x7f191c10ef40 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.342+0000 7f1922200700 1 -- 172.21.15.69:0/3739702637 --> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f191c02e080 con 0x7f19000395b0 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.343+0000 7f1913e00700 1 -- 172.21.15.69:0/3739702637 <== mgr.14120 v2:172.21.15.69:6800/3470783122 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f191c02e080 con 0x7f19000395b0 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.347+0000 7f1922200700 1 -- 172.21.15.69:0/3739702637 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f19000395b0 msgr2=0x7f190003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.347+0000 7f1922200700 1 --2- 172.21.15.69:0/3739702637 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f19000395b0 0x7f190003ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1908006fd0 tx=0x7f1908006e40 comp rx=0 tx=0).stop 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.347+0000 7f1922200700 1 -- 172.21.15.69:0/3739702637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f191c10ef40 msgr2=0x7f191c1a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.347+0000 7f1922200700 1 --2- 172.21.15.69:0/3739702637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f191c10ef40 0x7f191c1a2be0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f190c000c00 tx=0x7f190c011b00 comp rx=0 tx=0).stop 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.347+0000 7f1922200700 1 -- 172.21.15.69:0/3739702637 shutdown_connections 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.347+0000 7f1922200700 1 --2- 172.21.15.69:0/3739702637 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f19000395b0 0x7f190003ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.347+0000 7f1922200700 1 --2- 172.21.15.69:0/3739702637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f191c10ef40 0x7f191c1a2be0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.041 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.347+0000 7f1922200700 1 -- 172.21.15.69:0/3739702637 >> 172.21.15.69:0/3739702637 conn(0x7f191c07dc20 msgr2=0x7f191c07f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:58.041 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.347+0000 7f1922200700 1 -- 172.21.15.69:0/3739702637 shutdown_connections 2024-04-24T08:53:58.041 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:57.347+0000 7f1922200700 1 -- 172.21.15.69:0/3739702637 wait complete. 2024-04-24T08:53:58.041 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-24T08:53:58.041 INFO:teuthology.orchestra.run.smithi069.stdout:Adding key to root@localhost authorized_keys... 2024-04-24T08:53:58.049 INFO:teuthology.orchestra.run.smithi069.stdout:Adding host smithi069... 2024-04-24T08:53:58.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:58 smithi069 ceph-mon[28170]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:53:59.576 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:53:59 smithi069 ceph-mon[28170]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi069", "addr": "172.21.15.69", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:54:00.489 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:00 smithi069 ceph-mon[28170]: Deploying cephadm binary to smithi069 2024-04-24T08:54:05.555 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Added host 'smithi069' with addr '172.21.15.69' 2024-04-24T08:54:05.555 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.658+0000 7f9010a00700 1 Processor -- start 2024-04-24T08:54:05.555 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.658+0000 7f9010a00700 1 -- start start 2024-04-24T08:54:05.555 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.659+0000 7f9010a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f900c10a430 0x7f900c10a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:05.555 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.659+0000 7f9010a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f900c10ae20 con 0x7f900c10a430 2024-04-24T08:54:05.555 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.659+0000 7f900b400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f900c10a430 0x7f900c10a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:05.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.659+0000 7f900b400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f900c10a430 0x7f900c10a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52970/0 (socket says 172.21.15.69:52970) 2024-04-24T08:54:05.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.659+0000 7f900b400700 1 -- 172.21.15.69:0/1401923596 learned_addr learned my addr 172.21.15.69:0/1401923596 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:05.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.660+0000 7f900b400700 1 -- 172.21.15.69:0/1401923596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c10b5e0 con 0x7f900c10a430 2024-04-24T08:54:05.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.660+0000 7f900b400700 1 --2- 172.21.15.69:0/1401923596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f900c10a430 0x7f900c10a850 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc0098a0 tx=0x7f8ffc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c651b72ba45d90fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:05.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.660+0000 7f900a000700 1 -- 172.21.15.69:0/1401923596 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ffc004160 con 0x7f900c10a430 2024-04-24T08:54:05.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.661+0000 7f900a000700 1 -- 172.21.15.69:0/1401923596 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8ffc004300 con 0x7f900c10a430 2024-04-24T08:54:05.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.661+0000 7f900a000700 1 -- 172.21.15.69:0/1401923596 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ffc011a80 con 0x7f900c10a430 2024-04-24T08:54:05.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.661+0000 7f9010a00700 1 -- 172.21.15.69:0/1401923596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f900c10a430 msgr2=0x7f900c10a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:05.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.661+0000 7f9010a00700 1 --2- 172.21.15.69:0/1401923596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f900c10a430 0x7f900c10a850 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc0098a0 tx=0x7f8ffc009bb0 comp rx=0 tx=0).stop 2024-04-24T08:54:05.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.661+0000 7f9010a00700 1 -- 172.21.15.69:0/1401923596 shutdown_connections 2024-04-24T08:54:05.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.661+0000 7f9010a00700 1 --2- 172.21.15.69:0/1401923596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f900c10a430 0x7f900c10a850 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:05.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.661+0000 7f9010a00700 1 -- 172.21.15.69:0/1401923596 >> 172.21.15.69:0/1401923596 conn(0x7f900c07d9b0 msgr2=0x7f900c1084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:05.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.661+0000 7f9010a00700 1 -- 172.21.15.69:0/1401923596 shutdown_connections 2024-04-24T08:54:05.556 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.662+0000 7f9010a00700 1 -- 172.21.15.69:0/1401923596 wait complete. 2024-04-24T08:54:05.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.663+0000 7f9010a00700 1 Processor -- start 2024-04-24T08:54:05.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.663+0000 7f9010a00700 1 -- start start 2024-04-24T08:54:05.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.663+0000 7f9010a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f900c10a430 0x7f900c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:05.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.663+0000 7f9010a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f900c10e990 con 0x7f900c10a430 2024-04-24T08:54:05.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.663+0000 7f900b400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f900c10a430 0x7f900c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:05.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.663+0000 7f900b400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f900c10a430 0x7f900c19e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52974/0 (socket says 172.21.15.69:52974) 2024-04-24T08:54:05.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.663+0000 7f900b400700 1 -- 172.21.15.69:0/2958974610 learned_addr learned my addr 172.21.15.69:0/2958974610 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:05.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.664+0000 7f900b400700 1 -- 172.21.15.69:0/2958974610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c19eb70 con 0x7f900c10a430 2024-04-24T08:54:05.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.664+0000 7f900b400700 1 --2- 172.21.15.69:0/2958974610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f900c10a430 0x7f900c19e630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc006af0 tx=0x7f8ffc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:05.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.664+0000 7f9003e00700 1 -- 172.21.15.69:0/2958974610 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ffc0039e0 con 0x7f900c10a430 2024-04-24T08:54:05.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.664+0000 7f9003e00700 1 -- 172.21.15.69:0/2958974610 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8ffc003b80 con 0x7f900c10a430 2024-04-24T08:54:05.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.664+0000 7f9010a00700 1 -- 172.21.15.69:0/2958974610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c19ee00 con 0x7f900c10a430 2024-04-24T08:54:05.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.664+0000 7f9003e00700 1 -- 172.21.15.69:0/2958974610 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ffc0196a0 con 0x7f900c10a430 2024-04-24T08:54:05.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.665+0000 7f9010a00700 1 -- 172.21.15.69:0/2958974610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c1a1af0 con 0x7f900c10a430 2024-04-24T08:54:05.557 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.666+0000 7f9003e00700 1 -- 172.21.15.69:0/2958974610 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8ffc01a430 con 0x7f900c10a430 2024-04-24T08:54:05.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.666+0000 7f9010a00700 1 -- 172.21.15.69:0/2958974610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900c064150 con 0x7f900c10a430 2024-04-24T08:54:05.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.666+0000 7f9003e00700 1 --2- 172.21.15.69:0/2958974610 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f8ff00395b0 0x7f8ff003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:05.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.666+0000 7f9003e00700 1 -- 172.21.15.69:0/2958974610 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8ffc04dbf0 con 0x7f900c10a430 2024-04-24T08:54:05.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.666+0000 7f900aa00700 1 --2- 172.21.15.69:0/2958974610 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f8ff00395b0 0x7f8ff003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:05.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.667+0000 7f900aa00700 1 --2- 172.21.15.69:0/2958974610 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f8ff00395b0 0x7f8ff003ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8006fd0 tx=0x7f8ff8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:05.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.672+0000 7f9003e00700 1 -- 172.21.15.69:0/2958974610 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ffc0172b0 con 0x7f900c10a430 2024-04-24T08:54:05.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:53:58.904+0000 7f9010a00700 1 -- 172.21.15.69:0/2958974610 --> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi069", "addr": "172.21.15.69", "target": ["mon-mgr", ""]}) -- 0x7f900c02e050 con 0x7f8ff00395b0 2024-04-24T08:54:05.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:04.946+0000 7f9003e00700 1 -- 172.21.15.69:0/2958974610 <== mgr.14120 v2:172.21.15.69:6800/3470783122 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f900c02e050 con 0x7f8ff00395b0 2024-04-24T08:54:05.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:04.951+0000 7f9010a00700 1 -- 172.21.15.69:0/2958974610 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f8ff00395b0 msgr2=0x7f8ff003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:05.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:04.951+0000 7f9010a00700 1 --2- 172.21.15.69:0/2958974610 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f8ff00395b0 0x7f8ff003ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8006fd0 tx=0x7f8ff8006e40 comp rx=0 tx=0).stop 2024-04-24T08:54:05.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:04.951+0000 7f9010a00700 1 -- 172.21.15.69:0/2958974610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f900c10a430 msgr2=0x7f900c19e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:05.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:04.951+0000 7f9010a00700 1 --2- 172.21.15.69:0/2958974610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f900c10a430 0x7f900c19e630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc006af0 tx=0x7f8ffc011c40 comp rx=0 tx=0).stop 2024-04-24T08:54:05.558 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:04.951+0000 7f9010a00700 1 -- 172.21.15.69:0/2958974610 shutdown_connections 2024-04-24T08:54:05.561 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:04.951+0000 7f9010a00700 1 --2- 172.21.15.69:0/2958974610 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f8ff00395b0 0x7f8ff003ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:05.561 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:04.951+0000 7f9010a00700 1 --2- 172.21.15.69:0/2958974610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f900c10a430 0x7f900c19e630 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:05.561 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:04.951+0000 7f9010a00700 1 -- 172.21.15.69:0/2958974610 >> 172.21.15.69:0/2958974610 conn(0x7f900c07d9b0 msgr2=0x7f900c107ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:05.561 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:04.951+0000 7f9010a00700 1 -- 172.21.15.69:0/2958974610 shutdown_connections 2024-04-24T08:54:05.561 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:04.951+0000 7f9010a00700 1 -- 172.21.15.69:0/2958974610 wait complete. 2024-04-24T08:54:05.561 INFO:teuthology.orchestra.run.smithi069.stdout:Deploying unmanaged mon service... 2024-04-24T08:54:06.061 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:05 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' 2024-04-24T08:54:06.061 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:05 smithi069 ceph-mon[28170]: Added host smithi069 2024-04-24T08:54:06.061 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:05 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:54:07.027 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-24T08:54:07.028 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.095+0000 7f5f1f200700 1 Processor -- start 2024-04-24T08:54:07.028 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.095+0000 7f5f1f200700 1 -- start start 2024-04-24T08:54:07.028 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.095+0000 7f5f1f200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f20106ab0 0x7f5f20106ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:07.028 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.096+0000 7f5f1f200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2007be80 con 0x7f5f20106ab0 2024-04-24T08:54:07.028 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.096+0000 7f5f1de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f20106ab0 0x7f5f20106ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:07.029 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.096+0000 7f5f1de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f20106ab0 0x7f5f20106ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58618/0 (socket says 172.21.15.69:58618) 2024-04-24T08:54:07.029 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.096+0000 7f5f1de00700 1 -- 172.21.15.69:0/3318982454 learned_addr learned my addr 172.21.15.69:0/3318982454 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:07.029 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.096+0000 7f5f1de00700 1 -- 172.21.15.69:0/3318982454 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f20107410 con 0x7f5f20106ab0 2024-04-24T08:54:07.029 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.097+0000 7f5f1de00700 1 --2- 172.21.15.69:0/3318982454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f20106ab0 0x7f5f20106ed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5f100098a0 tx=0x7f5f10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8de43820cceca94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:07.029 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.097+0000 7f5f1ca00700 1 -- 172.21.15.69:0/3318982454 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f10004160 con 0x7f5f20106ab0 2024-04-24T08:54:07.029 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.097+0000 7f5f1ca00700 1 -- 172.21.15.69:0/3318982454 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5f10004300 con 0x7f5f20106ab0 2024-04-24T08:54:07.029 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.097+0000 7f5f1ca00700 1 -- 172.21.15.69:0/3318982454 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f10011a80 con 0x7f5f20106ab0 2024-04-24T08:54:07.029 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.098+0000 7f5f1f200700 1 -- 172.21.15.69:0/3318982454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f20106ab0 msgr2=0x7f5f20106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:07.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.098+0000 7f5f1f200700 1 --2- 172.21.15.69:0/3318982454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f20106ab0 0x7f5f20106ed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5f100098a0 tx=0x7f5f10009bb0 comp rx=0 tx=0).stop 2024-04-24T08:54:07.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.098+0000 7f5f1f200700 1 -- 172.21.15.69:0/3318982454 shutdown_connections 2024-04-24T08:54:07.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.098+0000 7f5f1f200700 1 --2- 172.21.15.69:0/3318982454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f20106ab0 0x7f5f20106ed0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:07.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.098+0000 7f5f1f200700 1 -- 172.21.15.69:0/3318982454 >> 172.21.15.69:0/3318982454 conn(0x7f5f201026d0 msgr2=0x7f5f20104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:07.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.098+0000 7f5f1f200700 1 -- 172.21.15.69:0/3318982454 shutdown_connections 2024-04-24T08:54:07.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.098+0000 7f5f1f200700 1 -- 172.21.15.69:0/3318982454 wait complete. 2024-04-24T08:54:07.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.099+0000 7f5f1f200700 1 Processor -- start 2024-04-24T08:54:07.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.099+0000 7f5f1f200700 1 -- start start 2024-04-24T08:54:07.032 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.100+0000 7f5f1f200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f20106ab0 0x7f5f2019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:07.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.100+0000 7f5f1f200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f20108440 con 0x7f5f20106ab0 2024-04-24T08:54:07.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.100+0000 7f5f1de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f20106ab0 0x7f5f2019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:07.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.100+0000 7f5f1de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f20106ab0 0x7f5f2019e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58620/0 (socket says 172.21.15.69:58620) 2024-04-24T08:54:07.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.100+0000 7f5f1de00700 1 -- 172.21.15.69:0/1008220964 learned_addr learned my addr 172.21.15.69:0/1008220964 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:07.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.100+0000 7f5f1de00700 1 -- 172.21.15.69:0/1008220964 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2019eb80 con 0x7f5f20106ab0 2024-04-24T08:54:07.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.100+0000 7f5f1de00700 1 --2- 172.21.15.69:0/1008220964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f20106ab0 0x7f5f2019e640 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5f10011a50 tx=0x7f5f10011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:07.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.101+0000 7f5f16a00700 1 -- 172.21.15.69:0/1008220964 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f10003a80 con 0x7f5f20106ab0 2024-04-24T08:54:07.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.101+0000 7f5f16a00700 1 -- 172.21.15.69:0/1008220964 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5f10003c20 con 0x7f5f20106ab0 2024-04-24T08:54:07.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.101+0000 7f5f1f200700 1 -- 172.21.15.69:0/1008220964 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2019ee10 con 0x7f5f20106ab0 2024-04-24T08:54:07.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.101+0000 7f5f16a00700 1 -- 172.21.15.69:0/1008220964 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f1001a650 con 0x7f5f20106ab0 2024-04-24T08:54:07.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.101+0000 7f5f1f200700 1 -- 172.21.15.69:0/1008220964 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f201a1b00 con 0x7f5f20106ab0 2024-04-24T08:54:07.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.102+0000 7f5f16a00700 1 -- 172.21.15.69:0/1008220964 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5f10024ad0 con 0x7f5f20106ab0 2024-04-24T08:54:07.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.102+0000 7f5f1f200700 1 -- 172.21.15.69:0/1008220964 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f20064150 con 0x7f5f20106ab0 2024-04-24T08:54:07.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.102+0000 7f5f16a00700 1 --2- 172.21.15.69:0/1008220964 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f5f04039600 0x7f5f0403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:07.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.102+0000 7f5f16a00700 1 -- 172.21.15.69:0/1008220964 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5f1004dc50 con 0x7f5f20106ab0 2024-04-24T08:54:07.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.103+0000 7f5f1d400700 1 --2- 172.21.15.69:0/1008220964 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f5f04039600 0x7f5f0403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:07.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.103+0000 7f5f1d400700 1 --2- 172.21.15.69:0/1008220964 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f5f04039600 0x7f5f0403bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c006fd0 tx=0x7f5f0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:07.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.108+0000 7f5f16a00700 1 -- 172.21.15.69:0/1008220964 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f1000f2e0 con 0x7f5f20106ab0 2024-04-24T08:54:07.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.332+0000 7f5f1f200700 1 -- 172.21.15.69:0/1008220964 --> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f5f2002e050 con 0x7f5f04039600 2024-04-24T08:54:07.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.345+0000 7f5f16a00700 1 -- 172.21.15.69:0/1008220964 <== mgr.14120 v2:172.21.15.69:6800/3470783122 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5f2002e050 con 0x7f5f04039600 2024-04-24T08:54:07.037 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.347+0000 7f5f1f200700 1 -- 172.21.15.69:0/1008220964 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f5f04039600 msgr2=0x7f5f0403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:07.037 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.347+0000 7f5f1f200700 1 --2- 172.21.15.69:0/1008220964 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f5f04039600 0x7f5f0403bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c006fd0 tx=0x7f5f0c006e40 comp rx=0 tx=0).stop 2024-04-24T08:54:07.037 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.347+0000 7f5f1f200700 1 -- 172.21.15.69:0/1008220964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f20106ab0 msgr2=0x7f5f2019e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:07.037 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.347+0000 7f5f1f200700 1 --2- 172.21.15.69:0/1008220964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f20106ab0 0x7f5f2019e640 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5f10011a50 tx=0x7f5f10011b30 comp rx=0 tx=0).stop 2024-04-24T08:54:07.037 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.347+0000 7f5f1f200700 1 -- 172.21.15.69:0/1008220964 shutdown_connections 2024-04-24T08:54:07.037 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.347+0000 7f5f1f200700 1 --2- 172.21.15.69:0/1008220964 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f5f04039600 0x7f5f0403bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:07.037 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.347+0000 7f5f1f200700 1 --2- 172.21.15.69:0/1008220964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f20106ab0 0x7f5f2019e640 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:07.037 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.347+0000 7f5f1f200700 1 -- 172.21.15.69:0/1008220964 >> 172.21.15.69:0/1008220964 conn(0x7f5f201026d0 msgr2=0x7f5f20194fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:07.037 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.348+0000 7f5f1f200700 1 -- 172.21.15.69:0/1008220964 shutdown_connections 2024-04-24T08:54:07.037 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:06.348+0000 7f5f1f200700 1 -- 172.21.15.69:0/1008220964 wait complete. 2024-04-24T08:54:07.037 INFO:teuthology.orchestra.run.smithi069.stdout:Deploying unmanaged mgr service... 2024-04-24T08:54:07.547 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:07 smithi069 ceph-mon[28170]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:54:07.547 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:07 smithi069 ceph-mon[28170]: Saving service mon spec with placement count:5 2024-04-24T08:54:07.547 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:07 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' 2024-04-24T08:54:09.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-24T08:54:09.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.880+0000 7ff28ac00700 1 Processor -- start 2024-04-24T08:54:09.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.880+0000 7ff28ac00700 1 -- start start 2024-04-24T08:54:09.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.880+0000 7ff28ac00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28407ddc0 0x7ff28407c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:09.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.880+0000 7ff28ac00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff28407c870 con 0x7ff28407ddc0 2024-04-24T08:54:09.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.881+0000 7ff283e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28407ddc0 0x7ff28407c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:09.266 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.881+0000 7ff283e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28407ddc0 0x7ff28407c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58636/0 (socket says 172.21.15.69:58636) 2024-04-24T08:54:09.266 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.881+0000 7ff283e00700 1 -- 172.21.15.69:0/1200025191 learned_addr learned my addr 172.21.15.69:0/1200025191 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:09.266 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.881+0000 7ff283e00700 1 -- 172.21.15.69:0/1200025191 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28407c9f0 con 0x7ff28407ddc0 2024-04-24T08:54:09.266 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.881+0000 7ff283e00700 1 --2- 172.21.15.69:0/1200025191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28407ddc0 0x7ff28407c330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff2740098a0 tx=0x7ff274009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e79d9efa9c13d7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:09.266 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.882+0000 7ff283400700 1 -- 172.21.15.69:0/1200025191 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff274004160 con 0x7ff28407ddc0 2024-04-24T08:54:09.267 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.882+0000 7ff283400700 1 -- 172.21.15.69:0/1200025191 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff274004300 con 0x7ff28407ddc0 2024-04-24T08:54:09.267 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.882+0000 7ff283400700 1 -- 172.21.15.69:0/1200025191 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff274011a80 con 0x7ff28407ddc0 2024-04-24T08:54:09.267 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.882+0000 7ff28ac00700 1 -- 172.21.15.69:0/1200025191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28407ddc0 msgr2=0x7ff28407c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:09.267 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.882+0000 7ff28ac00700 1 --2- 172.21.15.69:0/1200025191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28407ddc0 0x7ff28407c330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff2740098a0 tx=0x7ff274009bb0 comp rx=0 tx=0).stop 2024-04-24T08:54:09.267 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.883+0000 7ff28ac00700 1 -- 172.21.15.69:0/1200025191 shutdown_connections 2024-04-24T08:54:09.267 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.883+0000 7ff28ac00700 1 --2- 172.21.15.69:0/1200025191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28407ddc0 0x7ff28407c330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:09.267 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.883+0000 7ff28ac00700 1 -- 172.21.15.69:0/1200025191 >> 172.21.15.69:0/1200025191 conn(0x7ff284103a40 msgr2=0x7ff284105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:09.267 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.883+0000 7ff28ac00700 1 -- 172.21.15.69:0/1200025191 shutdown_connections 2024-04-24T08:54:09.267 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.883+0000 7ff28ac00700 1 -- 172.21.15.69:0/1200025191 wait complete. 2024-04-24T08:54:09.267 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.884+0000 7ff28ac00700 1 Processor -- start 2024-04-24T08:54:09.268 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.884+0000 7ff28ac00700 1 -- start start 2024-04-24T08:54:09.268 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.884+0000 7ff28ac00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28407ddc0 0x7ff28419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:09.268 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.884+0000 7ff28ac00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff28410e7d0 con 0x7ff28407ddc0 2024-04-24T08:54:09.268 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.884+0000 7ff283e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28407ddc0 0x7ff28419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:09.268 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.884+0000 7ff283e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28407ddc0 0x7ff28419e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58648/0 (socket says 172.21.15.69:58648) 2024-04-24T08:54:09.268 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.884+0000 7ff283e00700 1 -- 172.21.15.69:0/1189876366 learned_addr learned my addr 172.21.15.69:0/1189876366 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:09.272 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.885+0000 7ff283e00700 1 -- 172.21.15.69:0/1189876366 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28419ed10 con 0x7ff28407ddc0 2024-04-24T08:54:09.272 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.885+0000 7ff283e00700 1 --2- 172.21.15.69:0/1189876366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28407ddc0 0x7ff28419e7d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff274011a50 tx=0x7ff274011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:09.272 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.885+0000 7ff281600700 1 -- 172.21.15.69:0/1189876366 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff274003a80 con 0x7ff28407ddc0 2024-04-24T08:54:09.272 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.885+0000 7ff281600700 1 -- 172.21.15.69:0/1189876366 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff274003c20 con 0x7ff28407ddc0 2024-04-24T08:54:09.272 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.885+0000 7ff28ac00700 1 -- 172.21.15.69:0/1189876366 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28419efa0 con 0x7ff28407ddc0 2024-04-24T08:54:09.272 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.885+0000 7ff281600700 1 -- 172.21.15.69:0/1189876366 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff27401a650 con 0x7ff28407ddc0 2024-04-24T08:54:09.272 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.885+0000 7ff28ac00700 1 -- 172.21.15.69:0/1189876366 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2841a1c90 con 0x7ff28407ddc0 2024-04-24T08:54:09.272 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.886+0000 7ff281600700 1 -- 172.21.15.69:0/1189876366 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff274024440 con 0x7ff28407ddc0 2024-04-24T08:54:09.272 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.887+0000 7ff28ac00700 1 -- 172.21.15.69:0/1189876366 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff284064150 con 0x7ff28407ddc0 2024-04-24T08:54:09.272 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.887+0000 7ff281600700 1 --2- 172.21.15.69:0/1189876366 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7ff268039600 0x7ff26803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:09.272 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.887+0000 7ff281600700 1 -- 172.21.15.69:0/1189876366 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff27404dc00 con 0x7ff28407ddc0 2024-04-24T08:54:09.272 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.887+0000 7ff27be00700 1 --2- 172.21.15.69:0/1189876366 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7ff268039600 0x7ff26803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:09.273 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.888+0000 7ff27be00700 1 --2- 172.21.15.69:0/1189876366 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7ff268039600 0x7ff26803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff26c006fd0 tx=0x7ff26c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:09.273 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:07.893+0000 7ff281600700 1 -- 172.21.15.69:0/1189876366 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff27400e260 con 0x7ff28407ddc0 2024-04-24T08:54:09.273 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:08.140+0000 7ff28ac00700 1 -- 172.21.15.69:0/1189876366 --> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff28402e050 con 0x7ff268039600 2024-04-24T08:54:09.273 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:08.150+0000 7ff281600700 1 -- 172.21.15.69:0/1189876366 <== mgr.14120 v2:172.21.15.69:6800/3470783122 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff28402e050 con 0x7ff268039600 2024-04-24T08:54:09.273 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:08.153+0000 7ff28ac00700 1 -- 172.21.15.69:0/1189876366 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7ff268039600 msgr2=0x7ff26803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:09.273 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:08.153+0000 7ff28ac00700 1 --2- 172.21.15.69:0/1189876366 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7ff268039600 0x7ff26803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff26c006fd0 tx=0x7ff26c006e40 comp rx=0 tx=0).stop 2024-04-24T08:54:09.273 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:08.153+0000 7ff28ac00700 1 -- 172.21.15.69:0/1189876366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28407ddc0 msgr2=0x7ff28419e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:09.273 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:08.153+0000 7ff28ac00700 1 --2- 172.21.15.69:0/1189876366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28407ddc0 0x7ff28419e7d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff274011a50 tx=0x7ff274011b30 comp rx=0 tx=0).stop 2024-04-24T08:54:09.273 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:08.153+0000 7ff28ac00700 1 -- 172.21.15.69:0/1189876366 shutdown_connections 2024-04-24T08:54:09.273 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:08.153+0000 7ff28ac00700 1 --2- 172.21.15.69:0/1189876366 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7ff268039600 0x7ff26803bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:09.273 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:08.153+0000 7ff28ac00700 1 --2- 172.21.15.69:0/1189876366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28407ddc0 0x7ff28419e7d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:09.273 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:08.153+0000 7ff28ac00700 1 -- 172.21.15.69:0/1189876366 >> 172.21.15.69:0/1189876366 conn(0x7ff284103a40 msgr2=0x7ff284104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:09.273 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:08.153+0000 7ff28ac00700 1 -- 172.21.15.69:0/1189876366 shutdown_connections 2024-04-24T08:54:09.274 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:08.154+0000 7ff28ac00700 1 -- 172.21.15.69:0/1189876366 wait complete. 2024-04-24T08:54:09.426 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:09 smithi069 ceph-mon[28170]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:54:09.427 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:09 smithi069 ceph-mon[28170]: Saving service mgr spec with placement count:2 2024-04-24T08:54:09.427 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:09 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' 2024-04-24T08:54:09.427 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:09 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' 2024-04-24T08:54:10.400 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:10 smithi069 ceph-mon[28170]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:54:10.401 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:10 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' 2024-04-24T08:54:10.401 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:10 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/846344715' entity='client.admin' 2024-04-24T08:54:10.705 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.882+0000 7ff01a400700 1 Processor -- start 2024-04-24T08:54:10.705 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.883+0000 7ff01a400700 1 -- start start 2024-04-24T08:54:10.705 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.883+0000 7ff01a400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff01407dda0 0x7ff01407c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:10.705 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.883+0000 7ff01a400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff01407c850 con 0x7ff01407dda0 2024-04-24T08:54:10.706 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.883+0000 7ff013400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff01407dda0 0x7ff01407c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:10.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.883+0000 7ff013400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff01407dda0 0x7ff01407c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58658/0 (socket says 172.21.15.69:58658) 2024-04-24T08:54:10.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.883+0000 7ff013400700 1 -- 172.21.15.69:0/184906440 learned_addr learned my addr 172.21.15.69:0/184906440 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:10.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.884+0000 7ff013400700 1 -- 172.21.15.69:0/184906440 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01407c9d0 con 0x7ff01407dda0 2024-04-24T08:54:10.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.884+0000 7ff013400700 1 --2- 172.21.15.69:0/184906440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff01407dda0 0x7ff01407c310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff0040098a0 tx=0x7ff004009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682dc0a986d2209a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:10.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.884+0000 7ff012000700 1 -- 172.21.15.69:0/184906440 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff004004160 con 0x7ff01407dda0 2024-04-24T08:54:10.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.884+0000 7ff012000700 1 -- 172.21.15.69:0/184906440 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff004004300 con 0x7ff01407dda0 2024-04-24T08:54:10.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.884+0000 7ff012000700 1 -- 172.21.15.69:0/184906440 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff004011a50 con 0x7ff01407dda0 2024-04-24T08:54:10.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.885+0000 7ff01a400700 1 -- 172.21.15.69:0/184906440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff01407dda0 msgr2=0x7ff01407c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:10.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.885+0000 7ff01a400700 1 --2- 172.21.15.69:0/184906440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff01407dda0 0x7ff01407c310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff0040098a0 tx=0x7ff004009bb0 comp rx=0 tx=0).stop 2024-04-24T08:54:10.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.885+0000 7ff01a400700 1 -- 172.21.15.69:0/184906440 shutdown_connections 2024-04-24T08:54:10.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.885+0000 7ff01a400700 1 --2- 172.21.15.69:0/184906440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff01407dda0 0x7ff01407c310 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:10.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.885+0000 7ff01a400700 1 -- 172.21.15.69:0/184906440 >> 172.21.15.69:0/184906440 conn(0x7ff014103a40 msgr2=0x7ff014105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:10.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.885+0000 7ff01a400700 1 -- 172.21.15.69:0/184906440 shutdown_connections 2024-04-24T08:54:10.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.885+0000 7ff01a400700 1 -- 172.21.15.69:0/184906440 wait complete. 2024-04-24T08:54:10.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.886+0000 7ff01a400700 1 Processor -- start 2024-04-24T08:54:10.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.887+0000 7ff01a400700 1 -- start start 2024-04-24T08:54:10.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.887+0000 7ff01a400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff01407dda0 0x7ff01419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:10.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.887+0000 7ff01a400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff01410e830 con 0x7ff01407dda0 2024-04-24T08:54:10.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.887+0000 7ff013400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff01407dda0 0x7ff01419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:10.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.887+0000 7ff013400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff01407dda0 0x7ff01419e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58672/0 (socket says 172.21.15.69:58672) 2024-04-24T08:54:10.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.887+0000 7ff013400700 1 -- 172.21.15.69:0/846344715 learned_addr learned my addr 172.21.15.69:0/846344715 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:10.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.888+0000 7ff013400700 1 -- 172.21.15.69:0/846344715 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01419ecb0 con 0x7ff01407dda0 2024-04-24T08:54:10.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.888+0000 7ff013400700 1 --2- 172.21.15.69:0/846344715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff01407dda0 0x7ff01419e770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff004006b20 tx=0x7ff004011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:10.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.888+0000 7ff00be00700 1 -- 172.21.15.69:0/846344715 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff004003b70 con 0x7ff01407dda0 2024-04-24T08:54:10.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.888+0000 7ff00be00700 1 -- 172.21.15.69:0/846344715 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff00401a8e0 con 0x7ff01407dda0 2024-04-24T08:54:10.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.888+0000 7ff00be00700 1 -- 172.21.15.69:0/846344715 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0040249b0 con 0x7ff01407dda0 2024-04-24T08:54:10.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.888+0000 7ff01a400700 1 -- 172.21.15.69:0/846344715 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01419ef40 con 0x7ff01407dda0 2024-04-24T08:54:10.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.889+0000 7ff01a400700 1 -- 172.21.15.69:0/846344715 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0141a1be0 con 0x7ff01407dda0 2024-04-24T08:54:10.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.890+0000 7ff00be00700 1 -- 172.21.15.69:0/846344715 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff004022020 con 0x7ff01407dda0 2024-04-24T08:54:10.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.890+0000 7ff01a400700 1 -- 172.21.15.69:0/846344715 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff014064150 con 0x7ff01407dda0 2024-04-24T08:54:10.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.890+0000 7ff00be00700 1 --2- 172.21.15.69:0/846344715 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7feff8039600 0x7feff803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:10.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.890+0000 7ff00be00700 1 -- 172.21.15.69:0/846344715 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff00404df60 con 0x7ff01407dda0 2024-04-24T08:54:10.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.890+0000 7ff012a00700 1 --2- 172.21.15.69:0/846344715 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7feff8039600 0x7feff803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:10.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.891+0000 7ff012a00700 1 --2- 172.21.15.69:0/846344715 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7feff8039600 0x7feff803bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff000006fd0 tx=0x7ff000006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:10.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:09.895+0000 7ff00be00700 1 -- 172.21.15.69:0/846344715 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff004020c10 con 0x7ff01407dda0 2024-04-24T08:54:10.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:10.142+0000 7ff01a400700 1 -- 172.21.15.69:0/846344715 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7ff01402dfe0 con 0x7ff01407dda0 2024-04-24T08:54:10.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:10.145+0000 7ff00be00700 1 -- 172.21.15.69:0/846344715 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7ff00404d8b0 con 0x7ff01407dda0 2024-04-24T08:54:10.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:10.148+0000 7ff01a400700 1 -- 172.21.15.69:0/846344715 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7feff8039600 msgr2=0x7feff803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:10.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:10.148+0000 7ff01a400700 1 --2- 172.21.15.69:0/846344715 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7feff8039600 0x7feff803bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff000006fd0 tx=0x7ff000006e40 comp rx=0 tx=0).stop 2024-04-24T08:54:10.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:10.148+0000 7ff01a400700 1 -- 172.21.15.69:0/846344715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff01407dda0 msgr2=0x7ff01419e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:10.713 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:10.148+0000 7ff01a400700 1 --2- 172.21.15.69:0/846344715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff01407dda0 0x7ff01419e770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff004006b20 tx=0x7ff004011ad0 comp rx=0 tx=0).stop 2024-04-24T08:54:10.713 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:10.148+0000 7ff01a400700 1 -- 172.21.15.69:0/846344715 shutdown_connections 2024-04-24T08:54:10.713 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:10.148+0000 7ff01a400700 1 --2- 172.21.15.69:0/846344715 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7feff8039600 0x7feff803bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:10.713 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:10.148+0000 7ff01a400700 1 --2- 172.21.15.69:0/846344715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff01407dda0 0x7ff01419e770 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:10.713 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:10.148+0000 7ff01a400700 1 -- 172.21.15.69:0/846344715 >> 172.21.15.69:0/846344715 conn(0x7ff014103a40 msgr2=0x7ff0141046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:10.713 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:10.149+0000 7ff01a400700 1 -- 172.21.15.69:0/846344715 shutdown_connections 2024-04-24T08:54:10.713 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:10.149+0000 7ff01a400700 1 -- 172.21.15.69:0/846344715 wait complete. 2024-04-24T08:54:12.589 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:12 smithi069 ceph-mon[28170]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:54:12.589 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:12 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1093961793' entity='client.admin' 2024-04-24T08:54:12.601 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.622+0000 7f2804200700 1 Processor -- start 2024-04-24T08:54:12.601 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.622+0000 7f2804200700 1 -- start start 2024-04-24T08:54:12.601 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.623+0000 7f2804200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27fc07dd60 0x7f27fc07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:12.601 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.623+0000 7f2804200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc07e180 con 0x7f27fc07dd60 2024-04-24T08:54:12.601 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.623+0000 7f2801800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27fc07dd60 0x7f27fc07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:12.601 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.623+0000 7f2801800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27fc07dd60 0x7f27fc07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58674/0 (socket says 172.21.15.69:58674) 2024-04-24T08:54:12.602 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.623+0000 7f2801800700 1 -- 172.21.15.69:0/402524416 learned_addr learned my addr 172.21.15.69:0/402524416 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:12.602 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.624+0000 7f2801800700 1 -- 172.21.15.69:0/402524416 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc07c810 con 0x7f27fc07dd60 2024-04-24T08:54:12.602 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.624+0000 7f2801800700 1 --2- 172.21.15.69:0/402524416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27fc07dd60 0x7f27fc07c2d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f27f00098a0 tx=0x7f27f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcca8339f3b40fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:12.602 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.624+0000 7f27fbe00700 1 -- 172.21.15.69:0/402524416 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f0004160 con 0x7f27fc07dd60 2024-04-24T08:54:12.602 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.625+0000 7f27fbe00700 1 -- 172.21.15.69:0/402524416 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27f0004300 con 0x7f27fc07dd60 2024-04-24T08:54:12.602 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.625+0000 7f27fbe00700 1 -- 172.21.15.69:0/402524416 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f0011a50 con 0x7f27fc07dd60 2024-04-24T08:54:12.602 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.625+0000 7f2804200700 1 -- 172.21.15.69:0/402524416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27fc07dd60 msgr2=0x7f27fc07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:12.602 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.625+0000 7f2804200700 1 --2- 172.21.15.69:0/402524416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27fc07dd60 0x7f27fc07c2d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f27f00098a0 tx=0x7f27f0009bb0 comp rx=0 tx=0).stop 2024-04-24T08:54:12.602 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.625+0000 7f2804200700 1 -- 172.21.15.69:0/402524416 shutdown_connections 2024-04-24T08:54:12.602 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.625+0000 7f2804200700 1 --2- 172.21.15.69:0/402524416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27fc07dd60 0x7f27fc07c2d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:12.602 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.625+0000 7f2804200700 1 -- 172.21.15.69:0/402524416 >> 172.21.15.69:0/402524416 conn(0x7f27fc103a40 msgr2=0x7f27fc105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:12.602 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.625+0000 7f2804200700 1 -- 172.21.15.69:0/402524416 shutdown_connections 2024-04-24T08:54:12.603 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.625+0000 7f2804200700 1 -- 172.21.15.69:0/402524416 wait complete. 2024-04-24T08:54:12.603 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.626+0000 7f2804200700 1 Processor -- start 2024-04-24T08:54:12.603 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.627+0000 7f2804200700 1 -- start start 2024-04-24T08:54:12.603 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.627+0000 7f2804200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27fc07dd60 0x7f27fc19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:12.603 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.627+0000 7f2804200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc10e810 con 0x7f27fc07dd60 2024-04-24T08:54:12.603 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.627+0000 7f2801800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27fc07dd60 0x7f27fc19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:12.603 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.627+0000 7f2801800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27fc07dd60 0x7f27fc19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58684/0 (socket says 172.21.15.69:58684) 2024-04-24T08:54:12.603 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.627+0000 7f2801800700 1 -- 172.21.15.69:0/1093961793 learned_addr learned my addr 172.21.15.69:0/1093961793 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:12.603 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.628+0000 7f2801800700 1 -- 172.21.15.69:0/1093961793 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc19ece0 con 0x7f27fc07dd60 2024-04-24T08:54:12.603 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.628+0000 7f2801800700 1 --2- 172.21.15.69:0/1093961793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27fc07dd60 0x7f27fc19e7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f27f0006b20 tx=0x7f27f0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:12.603 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.628+0000 7f27fa000700 1 -- 172.21.15.69:0/1093961793 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f0003b70 con 0x7f27fc07dd60 2024-04-24T08:54:12.603 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.628+0000 7f27fa000700 1 -- 172.21.15.69:0/1093961793 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27f001a8e0 con 0x7f27fc07dd60 2024-04-24T08:54:12.603 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.628+0000 7f2804200700 1 -- 172.21.15.69:0/1093961793 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27fc19ef70 con 0x7f27fc07dd60 2024-04-24T08:54:12.603 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.628+0000 7f27fa000700 1 -- 172.21.15.69:0/1093961793 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f00249b0 con 0x7f27fc07dd60 2024-04-24T08:54:12.604 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.628+0000 7f2804200700 1 -- 172.21.15.69:0/1093961793 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27fc1a1c10 con 0x7f27fc07dd60 2024-04-24T08:54:12.604 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.629+0000 7f27fa000700 1 -- 172.21.15.69:0/1093961793 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f27f0022020 con 0x7f27fc07dd60 2024-04-24T08:54:12.604 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.630+0000 7f2804200700 1 -- 172.21.15.69:0/1093961793 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27fc064150 con 0x7f27fc07dd60 2024-04-24T08:54:12.604 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.630+0000 7f27fa000700 1 --2- 172.21.15.69:0/1093961793 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f27e4039600 0x7f27e403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:12.604 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.630+0000 7f27fa000700 1 -- 172.21.15.69:0/1093961793 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f27f004de00 con 0x7f27fc07dd60 2024-04-24T08:54:12.604 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.630+0000 7f2800e00700 1 --2- 172.21.15.69:0/1093961793 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f27e4039600 0x7f27e403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:12.604 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.631+0000 7f2800e00700 1 --2- 172.21.15.69:0/1093961793 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f27e4039600 0x7f27e403bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f27e8006fd0 tx=0x7f27e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:12.604 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.635+0000 7f27fa000700 1 -- 172.21.15.69:0/1093961793 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27f002c3a0 con 0x7f27fc07dd60 2024-04-24T08:54:12.604 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.859+0000 7f2804200700 1 -- 172.21.15.69:0/1093961793 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f27fc02dfe0 con 0x7f27fc07dd60 2024-04-24T08:54:12.604 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.862+0000 7f27fa000700 1 -- 172.21.15.69:0/1093961793 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f27f000be10 con 0x7f27fc07dd60 2024-04-24T08:54:12.604 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.865+0000 7f2804200700 1 -- 172.21.15.69:0/1093961793 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f27e4039600 msgr2=0x7f27e403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:12.604 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.865+0000 7f2804200700 1 --2- 172.21.15.69:0/1093961793 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f27e4039600 0x7f27e403bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f27e8006fd0 tx=0x7f27e8006e40 comp rx=0 tx=0).stop 2024-04-24T08:54:12.604 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.865+0000 7f2804200700 1 -- 172.21.15.69:0/1093961793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27fc07dd60 msgr2=0x7f27fc19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:12.605 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.865+0000 7f2804200700 1 --2- 172.21.15.69:0/1093961793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27fc07dd60 0x7f27fc19e7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f27f0006b20 tx=0x7f27f0011ad0 comp rx=0 tx=0).stop 2024-04-24T08:54:12.605 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.866+0000 7f2804200700 1 -- 172.21.15.69:0/1093961793 shutdown_connections 2024-04-24T08:54:12.605 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.866+0000 7f2804200700 1 --2- 172.21.15.69:0/1093961793 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f27e4039600 0x7f27e403bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:12.605 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.866+0000 7f2804200700 1 --2- 172.21.15.69:0/1093961793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27fc07dd60 0x7f27fc19e7a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:12.605 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.866+0000 7f2804200700 1 -- 172.21.15.69:0/1093961793 >> 172.21.15.69:0/1093961793 conn(0x7f27fc103a40 msgr2=0x7f27fc104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:12.605 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.866+0000 7f2804200700 1 -- 172.21.15.69:0/1093961793 shutdown_connections 2024-04-24T08:54:12.605 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:11.866+0000 7f2804200700 1 -- 172.21.15.69:0/1093961793 wait complete. 2024-04-24T08:54:12.605 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling the dashboard module... 2024-04-24T08:54:13.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:13 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' 2024-04-24T08:54:13.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:13 smithi069 ceph-mon[28170]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:54:14.579 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:14 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/4062318118' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-24T08:54:14.580 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:14 smithi069 ceph-mon[28170]: from='mgr.14120 172.21.15.69:0/2438601616' entity='mgr.x' 2024-04-24T08:54:14.580 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:14 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: ignoring --setuser ceph since I am not root 2024-04-24T08:54:14.580 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:14 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: ignoring --setgroup ceph since I am not root 2024-04-24T08:54:14.580 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:14 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:14.580+0000 7fed53543200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T08:54:14.841 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:14 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:14.829+0000 7fed53543200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:54:15.003 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.217+0000 7f566f200700 1 Processor -- start 2024-04-24T08:54:15.004 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.217+0000 7f566f200700 1 -- start start 2024-04-24T08:54:15.004 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.217+0000 7f566f200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668108c10 0x7f566810b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:15.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.217+0000 7f566f200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f566807bec0 con 0x7f5668108c10 2024-04-24T08:54:15.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.218+0000 7f566de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668108c10 0x7f566810b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:15.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.218+0000 7f566de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668108c10 0x7f566810b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58682/0 (socket says 172.21.15.69:58682) 2024-04-24T08:54:15.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.218+0000 7f566de00700 1 -- 172.21.15.69:0/2077992717 learned_addr learned my addr 172.21.15.69:0/2077992717 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:15.006 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.218+0000 7f566de00700 1 -- 172.21.15.69:0/2077992717 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566810b540 con 0x7f5668108c10 2024-04-24T08:54:15.006 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.218+0000 7f566de00700 1 --2- 172.21.15.69:0/2077992717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668108c10 0x7f566810b000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f565c0098a0 tx=0x7f565c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b28eab369ee61f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:15.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.219+0000 7f566ca00700 1 -- 172.21.15.69:0/2077992717 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f565c004160 con 0x7f5668108c10 2024-04-24T08:54:15.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.219+0000 7f566ca00700 1 -- 172.21.15.69:0/2077992717 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f565c004300 con 0x7f5668108c10 2024-04-24T08:54:15.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.219+0000 7f566ca00700 1 -- 172.21.15.69:0/2077992717 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f565c011a50 con 0x7f5668108c10 2024-04-24T08:54:15.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.219+0000 7f566f200700 1 -- 172.21.15.69:0/2077992717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668108c10 msgr2=0x7f566810b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:15.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.219+0000 7f566f200700 1 --2- 172.21.15.69:0/2077992717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668108c10 0x7f566810b000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f565c0098a0 tx=0x7f565c009bb0 comp rx=0 tx=0).stop 2024-04-24T08:54:15.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.219+0000 7f566f200700 1 -- 172.21.15.69:0/2077992717 shutdown_connections 2024-04-24T08:54:15.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.219+0000 7f566f200700 1 --2- 172.21.15.69:0/2077992717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668108c10 0x7f566810b000 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:15.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.219+0000 7f566f200700 1 -- 172.21.15.69:0/2077992717 >> 172.21.15.69:0/2077992717 conn(0x7f5668102640 msgr2=0x7f5668104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:15.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.219+0000 7f566f200700 1 -- 172.21.15.69:0/2077992717 shutdown_connections 2024-04-24T08:54:15.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.219+0000 7f566f200700 1 -- 172.21.15.69:0/2077992717 wait complete. 2024-04-24T08:54:15.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.220+0000 7f566f200700 1 Processor -- start 2024-04-24T08:54:15.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.221+0000 7f566f200700 1 -- start start 2024-04-24T08:54:15.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.221+0000 7f566f200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668108c10 0x7f566819e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:15.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.221+0000 7f566f200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f566810c550 con 0x7f5668108c10 2024-04-24T08:54:15.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.221+0000 7f566de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668108c10 0x7f566819e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:15.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.221+0000 7f566de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668108c10 0x7f566819e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58692/0 (socket says 172.21.15.69:58692) 2024-04-24T08:54:15.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.221+0000 7f566de00700 1 -- 172.21.15.69:0/4062318118 learned_addr learned my addr 172.21.15.69:0/4062318118 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:15.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.221+0000 7f566de00700 1 -- 172.21.15.69:0/4062318118 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566819eac0 con 0x7f5668108c10 2024-04-24T08:54:15.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.222+0000 7f566de00700 1 --2- 172.21.15.69:0/4062318118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668108c10 0x7f566819e580 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f565c011a20 tx=0x7f565c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:15.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.222+0000 7f5662a00700 1 -- 172.21.15.69:0/4062318118 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f565c003b70 con 0x7f5668108c10 2024-04-24T08:54:15.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.222+0000 7f5662a00700 1 -- 172.21.15.69:0/4062318118 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f565c01a8e0 con 0x7f5668108c10 2024-04-24T08:54:15.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.222+0000 7f5662a00700 1 -- 172.21.15.69:0/4062318118 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f565c024990 con 0x7f5668108c10 2024-04-24T08:54:15.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.222+0000 7f566f200700 1 -- 172.21.15.69:0/4062318118 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566819ed50 con 0x7f5668108c10 2024-04-24T08:54:15.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.222+0000 7f566f200700 1 -- 172.21.15.69:0/4062318118 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56681a1a40 con 0x7f5668108c10 2024-04-24T08:54:15.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.223+0000 7f566f200700 1 -- 172.21.15.69:0/4062318118 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5668064150 con 0x7f5668108c10 2024-04-24T08:54:15.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.223+0000 7f5662a00700 1 -- 172.21.15.69:0/4062318118 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f565c022020 con 0x7f5668108c10 2024-04-24T08:54:15.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.224+0000 7f5662a00700 1 --2- 172.21.15.69:0/4062318118 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f56500395b0 0x7f565003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:15.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.224+0000 7f5662a00700 1 -- 172.21.15.69:0/4062318118 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f565c04d9b0 con 0x7f5668108c10 2024-04-24T08:54:15.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.224+0000 7f566d400700 1 --2- 172.21.15.69:0/4062318118 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f56500395b0 0x7f565003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:15.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.225+0000 7f566d400700 1 --2- 172.21.15.69:0/4062318118 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f56500395b0 0x7f565003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5658006fd0 tx=0x7f5658006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:15.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.230+0000 7f5662a00700 1 -- 172.21.15.69:0/4062318118 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f565c00b3e0 con 0x7f5668108c10 2024-04-24T08:54:15.014 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:13.542+0000 7f566f200700 1 -- 172.21.15.69:0/4062318118 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f566802dfe0 con 0x7f5668108c10 2024-04-24T08:54:15.014 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:14.258+0000 7f5662a00700 1 -- 172.21.15.69:0/4062318118 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f565c021d80 con 0x7f5668108c10 2024-04-24T08:54:15.014 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:14.262+0000 7f5662a00700 1 -- 172.21.15.69:0/4062318118 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f565c04dd10 con 0x7f5668108c10 2024-04-24T08:54:15.014 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:14.263+0000 7f566f200700 1 -- 172.21.15.69:0/4062318118 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f56500395b0 msgr2=0x7f565003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:15.014 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:14.263+0000 7f566f200700 1 --2- 172.21.15.69:0/4062318118 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f56500395b0 0x7f565003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5658006fd0 tx=0x7f5658006e40 comp rx=0 tx=0).stop 2024-04-24T08:54:15.014 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:14.263+0000 7f566f200700 1 -- 172.21.15.69:0/4062318118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668108c10 msgr2=0x7f566819e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:15.014 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:14.263+0000 7f566f200700 1 --2- 172.21.15.69:0/4062318118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668108c10 0x7f566819e580 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f565c011a20 tx=0x7f565c011b00 comp rx=0 tx=0).stop 2024-04-24T08:54:15.014 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:14.264+0000 7f566f200700 1 -- 172.21.15.69:0/4062318118 shutdown_connections 2024-04-24T08:54:15.014 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:14.264+0000 7f566f200700 1 --2- 172.21.15.69:0/4062318118 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f56500395b0 0x7f565003ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:15.014 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:14.264+0000 7f566f200700 1 --2- 172.21.15.69:0/4062318118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5668108c10 0x7f566819e580 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:15.015 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:14.264+0000 7f566f200700 1 -- 172.21.15.69:0/4062318118 >> 172.21.15.69:0/4062318118 conn(0x7f5668102640 msgr2=0x7f5668103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:15.015 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:14.264+0000 7f566f200700 1 -- 172.21.15.69:0/4062318118 shutdown_connections 2024-04-24T08:54:15.015 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:14.264+0000 7f566f200700 1 -- 172.21.15.69:0/4062318118 wait complete. 2024-04-24T08:54:15.220 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:14 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:14.958+0000 7fed53543200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T08:54:15.481 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:15 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:15.360+0000 7fed53543200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T08:54:15.481 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:15 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/4062318118' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-24T08:54:15.481 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:15 smithi069 ceph-mon[28170]: mgrmap e9: x(active, since 25s) 2024-04-24T08:54:15.841 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:15 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:15.481+0000 7fed53543200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T08:54:16.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:16 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/33499462' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T08:54:16.602 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:54:16.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-24T08:54:16.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T08:54:16.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T08:54:16.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T08:54:16.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:54:16.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.671+0000 7f277e600700 1 Processor -- start 2024-04-24T08:54:16.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.671+0000 7f277e600700 1 -- start start 2024-04-24T08:54:16.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.671+0000 7f277e600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2778108200 0x7f2778108620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:16.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.671+0000 7f277e600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2778108bf0 con 0x7f2778108200 2024-04-24T08:54:16.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.672+0000 7f277d200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2778108200 0x7f2778108620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:16.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.672+0000 7f277d200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2778108200 0x7f2778108620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58716/0 (socket says 172.21.15.69:58716) 2024-04-24T08:54:16.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.672+0000 7f277d200700 1 -- 172.21.15.69:0/1297404025 learned_addr learned my addr 172.21.15.69:0/1297404025 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:16.610 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.672+0000 7f277d200700 1 -- 172.21.15.69:0/1297404025 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27781093b0 con 0x7f2778108200 2024-04-24T08:54:16.610 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.672+0000 7f277d200700 1 --2- 172.21.15.69:0/1297404025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2778108200 0x7f2778108620 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f276c0098a0 tx=0x7f276c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843684c51db5831b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:16.610 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.673+0000 7f2777400700 1 -- 172.21.15.69:0/1297404025 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f276c004160 con 0x7f2778108200 2024-04-24T08:54:16.610 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.673+0000 7f2777400700 1 -- 172.21.15.69:0/1297404025 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f276c004300 con 0x7f2778108200 2024-04-24T08:54:16.610 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.673+0000 7f2777400700 1 -- 172.21.15.69:0/1297404025 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f276c011a50 con 0x7f2778108200 2024-04-24T08:54:16.610 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.673+0000 7f277e600700 1 -- 172.21.15.69:0/1297404025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2778108200 msgr2=0x7f2778108620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:16.610 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.673+0000 7f277e600700 1 --2- 172.21.15.69:0/1297404025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2778108200 0x7f2778108620 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f276c0098a0 tx=0x7f276c009bb0 comp rx=0 tx=0).stop 2024-04-24T08:54:16.610 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.674+0000 7f277e600700 1 -- 172.21.15.69:0/1297404025 shutdown_connections 2024-04-24T08:54:16.610 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.674+0000 7f277e600700 1 --2- 172.21.15.69:0/1297404025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2778108200 0x7f2778108620 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:16.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.674+0000 7f277e600700 1 -- 172.21.15.69:0/1297404025 >> 172.21.15.69:0/1297404025 conn(0x7f2778103860 msgr2=0x7f2778105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:16.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.674+0000 7f277e600700 1 -- 172.21.15.69:0/1297404025 shutdown_connections 2024-04-24T08:54:16.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.674+0000 7f277e600700 1 -- 172.21.15.69:0/1297404025 wait complete. 2024-04-24T08:54:16.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.675+0000 7f277e600700 1 Processor -- start 2024-04-24T08:54:16.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.675+0000 7f277e600700 1 -- start start 2024-04-24T08:54:16.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.675+0000 7f277e600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2778108200 0x7f277819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:16.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.675+0000 7f277e600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f277810c760 con 0x7f2778108200 2024-04-24T08:54:16.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.676+0000 7f277d200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2778108200 0x7f277819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:16.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.676+0000 7f277d200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2778108200 0x7f277819e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58720/0 (socket says 172.21.15.69:58720) 2024-04-24T08:54:16.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.676+0000 7f277d200700 1 -- 172.21.15.69:0/33499462 learned_addr learned my addr 172.21.15.69:0/33499462 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:16.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.676+0000 7f277d200700 1 -- 172.21.15.69:0/33499462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277819eaf0 con 0x7f2778108200 2024-04-24T08:54:16.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.676+0000 7f277d200700 1 --2- 172.21.15.69:0/33499462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2778108200 0x7f277819e5b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f276c006b20 tx=0x7f276c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:16.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.677+0000 7f2775600700 1 -- 172.21.15.69:0/33499462 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f276c0039e0 con 0x7f2778108200 2024-04-24T08:54:16.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.677+0000 7f2775600700 1 -- 172.21.15.69:0/33499462 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f276c01a8e0 con 0x7f2778108200 2024-04-24T08:54:16.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.677+0000 7f2775600700 1 -- 172.21.15.69:0/33499462 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f276c0247e0 con 0x7f2778108200 2024-04-24T08:54:16.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.677+0000 7f277e600700 1 -- 172.21.15.69:0/33499462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277819ed80 con 0x7f2778108200 2024-04-24T08:54:16.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.677+0000 7f277e600700 1 -- 172.21.15.69:0/33499462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27781a1a20 con 0x7f2778108200 2024-04-24T08:54:16.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.678+0000 7f277e600700 1 -- 172.21.15.69:0/33499462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2778064150 con 0x7f2778108200 2024-04-24T08:54:16.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.678+0000 7f2775600700 1 -- 172.21.15.69:0/33499462 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f276c022020 con 0x7f2778108200 2024-04-24T08:54:16.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.679+0000 7f2775600700 1 --2- 172.21.15.69:0/33499462 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f2760039600 0x7f276003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:16.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.679+0000 7f2775600700 1 -- 172.21.15.69:0/33499462 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f276c04dc10 con 0x7f2778108200 2024-04-24T08:54:16.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.679+0000 7f2777e00700 1 -- 172.21.15.69:0/33499462 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f2760039600 msgr2=0x7f276003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3470783122 2024-04-24T08:54:16.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.679+0000 7f2777e00700 1 --2- 172.21.15.69:0/33499462 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f2760039600 0x7f276003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:54:16.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.684+0000 7f2775600700 1 -- 172.21.15.69:0/33499462 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f276c00f2e0 con 0x7f2778108200 2024-04-24T08:54:16.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.880+0000 7f2777e00700 1 -- 172.21.15.69:0/33499462 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f2760039600 msgr2=0x7f276003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3470783122 2024-04-24T08:54:16.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.880+0000 7f2777e00700 1 --2- 172.21.15.69:0/33499462 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f2760039600 0x7f276003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:54:16.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.967+0000 7f277e600700 1 -- 172.21.15.69:0/33499462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f277802dfe0 con 0x7f2778108200 2024-04-24T08:54:16.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.967+0000 7f2775600700 1 -- 172.21.15.69:0/33499462 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f276c04d560 con 0x7f2778108200 2024-04-24T08:54:16.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.970+0000 7f277e600700 1 -- 172.21.15.69:0/33499462 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f2760039600 msgr2=0x7f276003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:54:16.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.970+0000 7f277e600700 1 --2- 172.21.15.69:0/33499462 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f2760039600 0x7f276003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:16.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.970+0000 7f277e600700 1 -- 172.21.15.69:0/33499462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2778108200 msgr2=0x7f277819e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:16.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.970+0000 7f277e600700 1 --2- 172.21.15.69:0/33499462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2778108200 0x7f277819e5b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f276c006b20 tx=0x7f276c011ad0 comp rx=0 tx=0).stop 2024-04-24T08:54:16.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.970+0000 7f277e600700 1 -- 172.21.15.69:0/33499462 shutdown_connections 2024-04-24T08:54:16.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.970+0000 7f277e600700 1 --2- 172.21.15.69:0/33499462 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f2760039600 0x7f276003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:16.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.970+0000 7f277e600700 1 --2- 172.21.15.69:0/33499462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2778108200 0x7f277819e5b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:16.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.970+0000 7f277e600700 1 -- 172.21.15.69:0/33499462 >> 172.21.15.69:0/33499462 conn(0x7f2778103860 msgr2=0x7f2778104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:16.616 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.970+0000 7f277e600700 1 -- 172.21.15.69:0/33499462 shutdown_connections 2024-04-24T08:54:16.616 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:15.970+0000 7f277e600700 1 -- 172.21.15.69:0/33499462 wait complete. 2024-04-24T08:54:16.616 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for the mgr to restart... 2024-04-24T08:54:16.616 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr epoch 9... 2024-04-24T08:54:17.209 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:17 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:17.044+0000 7fed53543200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T08:54:17.492 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:17 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:17.280+0000 7fed53543200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T08:54:17.492 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:17 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:17.492+0000 7fed53543200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T08:54:18.091 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:17 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:17.870+0000 7fed53543200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T08:54:18.091 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:17 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:17.990+0000 7fed53543200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T08:54:18.491 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:18 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:18.240+0000 7fed53543200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T08:54:18.492 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:18 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:18.491+0000 7fed53543200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:54:19.591 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:19 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:19.262+0000 7fed53543200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T08:54:20.091 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:19 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:19.811+0000 7fed53543200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T08:54:20.091 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:19 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:19.938+0000 7fed53543200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T08:54:20.591 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:20 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:20.200+0000 7fed53543200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T08:54:21.288 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:20 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:20.848+0000 7fed53543200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T08:54:21.288 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:20 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:20.972+0000 7fed53543200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T08:54:21.288 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:21 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:21.287+0000 7fed53543200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T08:54:21.591 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:21 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:21.465+0000 7fed53543200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T08:54:22.091 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:21 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:21.591+0000 7fed53543200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T08:54:22.091 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:21 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:21.742+0000 7fed53543200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T08:54:22.462 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:22 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:22.109+0000 7fed53543200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T08:54:23.091 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:22 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:22.897+0000 7fed53543200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T08:54:23.591 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 08:54:23 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T08:54:23.192+0000 7fed53543200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T08:54:23.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:23 smithi069 ceph-mon[28170]: Active manager daemon x restarted 2024-04-24T08:54:23.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:23 smithi069 ceph-mon[28170]: Activating manager daemon x 2024-04-24T08:54:23.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:23 smithi069 ceph-mon[28170]: osdmap e3: 0 total, 0 up, 0 in 2024-04-24T08:54:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:23 smithi069 ceph-mon[28170]: mgrmap e10: x(active, starting, since 0.00771522s) 2024-04-24T08:54:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:23 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:54:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:23 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T08:54:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:23 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:54:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:23 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:54:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:23 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:54:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:23 smithi069 ceph-mon[28170]: Manager daemon x is now available 2024-04-24T08:54:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:23 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:54:24.563 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:54:24.564 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T08:54:24.564 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:24 smithi069 ceph-mon[28170]: mgrmap e11: x(active, since 1.00825s) 2024-04-24T08:54:25.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:54:25.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-24T08:54:25.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T08:54:25.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:54:25.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.210+0000 7f193f400700 1 Processor -- start 2024-04-24T08:54:25.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.210+0000 7f193f400700 1 -- start start 2024-04-24T08:54:25.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.210+0000 7f193f400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1938106c50 0x7f1938107070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:25.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.210+0000 7f193f400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f193807be40 con 0x7f1938106c50 2024-04-24T08:54:25.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.211+0000 7f193ca00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1938106c50 0x7f1938107070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:25.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.211+0000 7f193ca00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1938106c50 0x7f1938107070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58730/0 (socket says 172.21.15.69:58730) 2024-04-24T08:54:25.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.211+0000 7f193ca00700 1 -- 172.21.15.69:0/1174547821 learned_addr learned my addr 172.21.15.69:0/1174547821 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:25.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.211+0000 7f193ca00700 1 -- 172.21.15.69:0/1174547821 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19381075b0 con 0x7f1938106c50 2024-04-24T08:54:25.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.212+0000 7f193ca00700 1 --2- 172.21.15.69:0/1174547821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1938106c50 0x7f1938107070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f192c0098a0 tx=0x7f192c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c76257d0f572229 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:25.020 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.212+0000 7f1937400700 1 -- 172.21.15.69:0/1174547821 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f192c004160 con 0x7f1938106c50 2024-04-24T08:54:25.020 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.212+0000 7f1937400700 1 -- 172.21.15.69:0/1174547821 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f192c004300 con 0x7f1938106c50 2024-04-24T08:54:25.020 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.212+0000 7f1937400700 1 -- 172.21.15.69:0/1174547821 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f192c011a50 con 0x7f1938106c50 2024-04-24T08:54:25.020 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.213+0000 7f193f400700 1 -- 172.21.15.69:0/1174547821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1938106c50 msgr2=0x7f1938107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:25.020 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.213+0000 7f193f400700 1 --2- 172.21.15.69:0/1174547821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1938106c50 0x7f1938107070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f192c0098a0 tx=0x7f192c009bb0 comp rx=0 tx=0).stop 2024-04-24T08:54:25.020 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.213+0000 7f193f400700 1 -- 172.21.15.69:0/1174547821 shutdown_connections 2024-04-24T08:54:25.020 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.213+0000 7f193f400700 1 --2- 172.21.15.69:0/1174547821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1938106c50 0x7f1938107070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:25.020 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.213+0000 7f193f400700 1 -- 172.21.15.69:0/1174547821 >> 172.21.15.69:0/1174547821 conn(0x7f1938102830 msgr2=0x7f1938104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:25.020 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.213+0000 7f193f400700 1 -- 172.21.15.69:0/1174547821 shutdown_connections 2024-04-24T08:54:25.021 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.213+0000 7f193f400700 1 -- 172.21.15.69:0/1174547821 wait complete. 2024-04-24T08:54:25.021 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.215+0000 7f193f400700 1 Processor -- start 2024-04-24T08:54:25.021 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.215+0000 7f193f400700 1 -- start start 2024-04-24T08:54:25.021 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.215+0000 7f193f400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1938106c50 0x7f193819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:25.021 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.215+0000 7f193f400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1938108620 con 0x7f1938106c50 2024-04-24T08:54:25.021 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.215+0000 7f193ca00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1938106c50 0x7f193819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:25.021 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.215+0000 7f193ca00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1938106c50 0x7f193819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58742/0 (socket says 172.21.15.69:58742) 2024-04-24T08:54:25.021 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.215+0000 7f193ca00700 1 -- 172.21.15.69:0/738544785 learned_addr learned my addr 172.21.15.69:0/738544785 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:25.021 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.216+0000 7f193ca00700 1 -- 172.21.15.69:0/738544785 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193819ed30 con 0x7f1938106c50 2024-04-24T08:54:25.021 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.216+0000 7f193ca00700 1 --2- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1938106c50 0x7f193819e7f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f192c011a00 tx=0x7f192c011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:25.021 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.216+0000 7f1935600700 1 -- 172.21.15.69:0/738544785 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f192c003af0 con 0x7f1938106c50 2024-04-24T08:54:25.021 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.216+0000 7f1935600700 1 -- 172.21.15.69:0/738544785 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f192c01a8e0 con 0x7f1938106c50 2024-04-24T08:54:25.021 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.216+0000 7f193f400700 1 -- 172.21.15.69:0/738544785 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193819efc0 con 0x7f1938106c50 2024-04-24T08:54:25.021 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.216+0000 7f1935600700 1 -- 172.21.15.69:0/738544785 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f192c024840 con 0x7f1938106c50 2024-04-24T08:54:25.022 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.217+0000 7f193f400700 1 -- 172.21.15.69:0/738544785 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19381a1cb0 con 0x7f1938106c50 2024-04-24T08:54:25.022 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.218+0000 7f1935600700 1 -- 172.21.15.69:0/738544785 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f192c022020 con 0x7f1938106c50 2024-04-24T08:54:25.022 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.218+0000 7f1935600700 1 --2- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f1920039660 0x7f192003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:25.022 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.218+0000 7f1935600700 1 -- 172.21.15.69:0/738544785 --> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f192003c230 con 0x7f1920039660 2024-04-24T08:54:25.022 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.218+0000 7f1935600700 1 -- 172.21.15.69:0/738544785 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f192c04db00 con 0x7f1938106c50 2024-04-24T08:54:25.022 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.218+0000 7f1937e00700 1 -- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f1920039660 msgr2=0x7f192003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3470783122 2024-04-24T08:54:25.022 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.218+0000 7f1937e00700 1 --2- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f1920039660 0x7f192003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:54:25.022 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.419+0000 7f1937e00700 1 -- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f1920039660 msgr2=0x7f192003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3470783122 2024-04-24T08:54:25.022 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.419+0000 7f1937e00700 1 --2- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f1920039660 0x7f192003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:54:25.022 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.820+0000 7f1937e00700 1 -- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f1920039660 msgr2=0x7f192003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3470783122 2024-04-24T08:54:25.022 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:17.820+0000 7f1937e00700 1 --2- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f1920039660 0x7f192003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T08:54:25.022 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:18.620+0000 7f1937e00700 1 -- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f1920039660 msgr2=0x7f192003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3470783122 2024-04-24T08:54:25.022 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:18.620+0000 7f1937e00700 1 --2- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f1920039660 0x7f192003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T08:54:25.023 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:20.222+0000 7f1937e00700 1 -- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f1920039660 msgr2=0x7f192003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3470783122 2024-04-24T08:54:25.023 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:20.222+0000 7f1937e00700 1 --2- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f1920039660 0x7f192003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T08:54:25.023 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:23.200+0000 7f1935600700 1 -- 172.21.15.69:0/738544785 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f192c01aa90 con 0x7f1938106c50 2024-04-24T08:54:25.023 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:23.200+0000 7f1935600700 1 -- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f1920039660 msgr2=0x7f192003bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:54:25.023 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:23.200+0000 7f1935600700 1 --2- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f1920039660 0x7f192003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:25.023 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.202+0000 7f1935600700 1 -- 172.21.15.69:0/738544785 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f192c04f600 con 0x7f1938106c50 2024-04-24T08:54:25.023 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.202+0000 7f1935600700 1 --2- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f192003cff0 0x7f192003f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:25.023 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.202+0000 7f1935600700 1 -- 172.21.15.69:0/738544785 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f192c02ee50 con 0x7f192003cff0 2024-04-24T08:54:25.023 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.204+0000 7f1937e00700 1 --2- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f192003cff0 0x7f192003f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:25.023 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.207+0000 7f1937e00700 1 --2- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f192003cff0 0x7f192003f3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1924003a10 tx=0x7f1924009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:25.023 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.209+0000 7f1935600700 1 -- 172.21.15.69:0/738544785 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f192c02ee50 con 0x7f192003cff0 2024-04-24T08:54:25.023 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.216+0000 7f193f400700 1 -- 172.21.15.69:0/738544785 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f1938064150 con 0x7f192003cff0 2024-04-24T08:54:25.023 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.216+0000 7f1935600700 1 -- 172.21.15.69:0/738544785 <== mgr.14152 v2:172.21.15.69:6800/1098677190 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f1938064150 con 0x7f192003cff0 2024-04-24T08:54:25.023 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.217+0000 7f193f400700 1 -- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f192003cff0 msgr2=0x7f192003f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:25.024 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.217+0000 7f193f400700 1 --2- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f192003cff0 0x7f192003f3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1924003a10 tx=0x7f1924009270 comp rx=0 tx=0).stop 2024-04-24T08:54:25.024 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.217+0000 7f193f400700 1 -- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1938106c50 msgr2=0x7f193819e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:25.024 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.217+0000 7f193f400700 1 --2- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1938106c50 0x7f193819e7f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f192c011a00 tx=0x7f192c011ae0 comp rx=0 tx=0).stop 2024-04-24T08:54:25.024 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.217+0000 7f193f400700 1 -- 172.21.15.69:0/738544785 shutdown_connections 2024-04-24T08:54:25.024 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.217+0000 7f193f400700 1 --2- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f192003cff0 0x7f192003f3e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:25.024 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.217+0000 7f193f400700 1 --2- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1938106c50 0x7f193819e7f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:25.024 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.217+0000 7f193f400700 1 --2- 172.21.15.69:0/738544785 >> [v2:172.21.15.69:6800/3470783122,v1:172.21.15.69:6801/3470783122] conn(0x7f1920039660 0x7f192003bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:25.024 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.217+0000 7f193f400700 1 -- 172.21.15.69:0/738544785 >> 172.21.15.69:0/738544785 conn(0x7f1938102830 msgr2=0x7f19381951b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:25.024 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.217+0000 7f193f400700 1 -- 172.21.15.69:0/738544785 shutdown_connections 2024-04-24T08:54:25.024 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:24.217+0000 7f193f400700 1 -- 172.21.15.69:0/738544785 wait complete. 2024-04-24T08:54:25.024 INFO:teuthology.orchestra.run.smithi069.stdout:mgr epoch 9 is available 2024-04-24T08:54:25.024 INFO:teuthology.orchestra.run.smithi069.stdout:Generating a dashboard self-signed certificate... 2024-04-24T08:54:25.435 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:25 smithi069 ceph-mon[28170]: [24/Apr/2024:08:54:23] ENGINE Bus STARTING 2024-04-24T08:54:25.436 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:25 smithi069 ceph-mon[28170]: [24/Apr/2024:08:54:23] ENGINE Serving on https://172.21.15.69:7150 2024-04-24T08:54:25.436 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:25 smithi069 ceph-mon[28170]: [24/Apr/2024:08:54:24] ENGINE Serving on http://172.21.15.69:8765 2024-04-24T08:54:25.436 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:25 smithi069 ceph-mon[28170]: [24/Apr/2024:08:54:24] ENGINE Bus STARTED 2024-04-24T08:54:25.436 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:25 smithi069 ceph-mon[28170]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T08:54:25.436 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:25 smithi069 ceph-mon[28170]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T08:54:26.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:26 smithi069 ceph-mon[28170]: mgrmap e12: x(active, since 2s) 2024-04-24T08:54:26.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:26 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:26.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:26 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:26.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:26 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:26.951 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-24T08:54:26.951 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.852+0000 7f56ff200700 1 Processor -- start 2024-04-24T08:54:26.952 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.852+0000 7f56ff200700 1 -- start start 2024-04-24T08:54:26.952 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.852+0000 7f56ff200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570007ddc0 0x7f570007c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:26.952 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.852+0000 7f56ff200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f570007c870 con 0x7f570007ddc0 2024-04-24T08:54:26.952 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.853+0000 7f56fde00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570007ddc0 0x7f570007c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:26.952 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.853+0000 7f56fde00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570007ddc0 0x7f570007c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59340/0 (socket says 172.21.15.69:59340) 2024-04-24T08:54:26.952 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.853+0000 7f56fde00700 1 -- 172.21.15.69:0/747207305 learned_addr learned my addr 172.21.15.69:0/747207305 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:26.952 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.853+0000 7f56fde00700 1 -- 172.21.15.69:0/747207305 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570007c9f0 con 0x7f570007ddc0 2024-04-24T08:54:26.952 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.854+0000 7f56fde00700 1 --2- 172.21.15.69:0/747207305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570007ddc0 0x7f570007c330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f56f00098a0 tx=0x7f56f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4421ebc18d7376eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:26.952 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.854+0000 7f56fca00700 1 -- 172.21.15.69:0/747207305 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56f0004160 con 0x7f570007ddc0 2024-04-24T08:54:26.952 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.854+0000 7f56fca00700 1 -- 172.21.15.69:0/747207305 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56f0004300 con 0x7f570007ddc0 2024-04-24T08:54:26.953 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.854+0000 7f56fca00700 1 -- 172.21.15.69:0/747207305 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56f0011a50 con 0x7f570007ddc0 2024-04-24T08:54:26.953 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.855+0000 7f56ff200700 1 -- 172.21.15.69:0/747207305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570007ddc0 msgr2=0x7f570007c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:26.953 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.855+0000 7f56ff200700 1 --2- 172.21.15.69:0/747207305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570007ddc0 0x7f570007c330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f56f00098a0 tx=0x7f56f0009bb0 comp rx=0 tx=0).stop 2024-04-24T08:54:26.953 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.855+0000 7f56ff200700 1 -- 172.21.15.69:0/747207305 shutdown_connections 2024-04-24T08:54:26.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.855+0000 7f56ff200700 1 --2- 172.21.15.69:0/747207305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570007ddc0 0x7f570007c330 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:26.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.855+0000 7f56ff200700 1 -- 172.21.15.69:0/747207305 >> 172.21.15.69:0/747207305 conn(0x7f5700103900 msgr2=0x7f5700105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:26.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.855+0000 7f56ff200700 1 -- 172.21.15.69:0/747207305 shutdown_connections 2024-04-24T08:54:26.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.855+0000 7f56ff200700 1 -- 172.21.15.69:0/747207305 wait complete. 2024-04-24T08:54:26.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.856+0000 7f56ff200700 1 Processor -- start 2024-04-24T08:54:26.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.856+0000 7f56ff200700 1 -- start start 2024-04-24T08:54:26.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.856+0000 7f56ff200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570007ddc0 0x7f570019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:26.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.856+0000 7f56ff200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f570010e6b0 con 0x7f570007ddc0 2024-04-24T08:54:26.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.857+0000 7f56fde00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570007ddc0 0x7f570019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:26.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.857+0000 7f56fde00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570007ddc0 0x7f570019e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59346/0 (socket says 172.21.15.69:59346) 2024-04-24T08:54:26.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.857+0000 7f56fde00700 1 -- 172.21.15.69:0/4096057761 learned_addr learned my addr 172.21.15.69:0/4096057761 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:26.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.857+0000 7f56fde00700 1 -- 172.21.15.69:0/4096057761 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570019ebb0 con 0x7f570007ddc0 2024-04-24T08:54:26.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.857+0000 7f56fde00700 1 --2- 172.21.15.69:0/4096057761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570007ddc0 0x7f570019e670 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f56f0011a20 tx=0x7f56f0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:26.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.858+0000 7f56f6a00700 1 -- 172.21.15.69:0/4096057761 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56f00039c0 con 0x7f570007ddc0 2024-04-24T08:54:26.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.858+0000 7f56f6a00700 1 -- 172.21.15.69:0/4096057761 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56f0003b60 con 0x7f570007ddc0 2024-04-24T08:54:26.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.858+0000 7f56f6a00700 1 -- 172.21.15.69:0/4096057761 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56f001a6b0 con 0x7f570007ddc0 2024-04-24T08:54:26.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.858+0000 7f56ff200700 1 -- 172.21.15.69:0/4096057761 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570019ee40 con 0x7f570007ddc0 2024-04-24T08:54:26.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.858+0000 7f56ff200700 1 -- 172.21.15.69:0/4096057761 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57001a1ae0 con 0x7f570007ddc0 2024-04-24T08:54:26.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.859+0000 7f56ff200700 1 -- 172.21.15.69:0/4096057761 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5700064150 con 0x7f570007ddc0 2024-04-24T08:54:26.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.859+0000 7f56f6a00700 1 -- 172.21.15.69:0/4096057761 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f56f0024430 con 0x7f570007ddc0 2024-04-24T08:54:26.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.860+0000 7f56f6a00700 1 --2- 172.21.15.69:0/4096057761 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56e40395b0 0x7f56e403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:26.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.860+0000 7f56f6a00700 1 -- 172.21.15.69:0/4096057761 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f56f004de40 con 0x7f570007ddc0 2024-04-24T08:54:26.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.860+0000 7f56fd400700 1 --2- 172.21.15.69:0/4096057761 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56e40395b0 0x7f56e403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:26.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.861+0000 7f56fd400700 1 --2- 172.21.15.69:0/4096057761 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56e40395b0 0x7f56e403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f56ec006fd0 tx=0x7f56ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:26.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:25.865+0000 7f56f6a00700 1 -- 172.21.15.69:0/4096057761 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56f0017ca0 con 0x7f570007ddc0 2024-04-24T08:54:26.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:26.106+0000 7f56ff200700 1 -- 172.21.15.69:0/4096057761 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f5700062f30 con 0x7f56e40395b0 2024-04-24T08:54:26.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:26.239+0000 7f56f6a00700 1 -- 172.21.15.69:0/4096057761 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f5700062f30 con 0x7f56e40395b0 2024-04-24T08:54:26.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:26.242+0000 7f56ff200700 1 -- 172.21.15.69:0/4096057761 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56e40395b0 msgr2=0x7f56e403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:26.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:26.242+0000 7f56ff200700 1 --2- 172.21.15.69:0/4096057761 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56e40395b0 0x7f56e403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f56ec006fd0 tx=0x7f56ec006e40 comp rx=0 tx=0).stop 2024-04-24T08:54:26.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:26.242+0000 7f56ff200700 1 -- 172.21.15.69:0/4096057761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570007ddc0 msgr2=0x7f570019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:26.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:26.242+0000 7f56ff200700 1 --2- 172.21.15.69:0/4096057761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570007ddc0 0x7f570019e670 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f56f0011a20 tx=0x7f56f0011b00 comp rx=0 tx=0).stop 2024-04-24T08:54:26.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:26.242+0000 7f56ff200700 1 -- 172.21.15.69:0/4096057761 shutdown_connections 2024-04-24T08:54:26.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:26.242+0000 7f56ff200700 1 --2- 172.21.15.69:0/4096057761 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56e40395b0 0x7f56e403ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:26.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:26.242+0000 7f56ff200700 1 --2- 172.21.15.69:0/4096057761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f570007ddc0 0x7f570019e670 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:26.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:26.242+0000 7f56ff200700 1 -- 172.21.15.69:0/4096057761 >> 172.21.15.69:0/4096057761 conn(0x7f5700103900 msgr2=0x7f5700104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:26.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:26.242+0000 7f56ff200700 1 -- 172.21.15.69:0/4096057761 shutdown_connections 2024-04-24T08:54:26.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:26.243+0000 7f56ff200700 1 -- 172.21.15.69:0/4096057761 wait complete. 2024-04-24T08:54:26.957 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial admin user... 2024-04-24T08:54:27.494 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:27 smithi069 ceph-mon[28170]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:54:28.741 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$B1Pnz0RLF9tu/eKWxx2Myeri6JszbACdkhGYxoXl96YbGh/Q9CAcC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713948868, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-24T08:54:28.741 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.539+0000 7fbed7200700 1 Processor -- start 2024-04-24T08:54:28.743 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.540+0000 7fbed7200700 1 -- start start 2024-04-24T08:54:28.743 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.540+0000 7fbed7200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed007de30 0x7fbed007c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:28.743 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.540+0000 7fbed7200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbed007c870 con 0x7fbed007de30 2024-04-24T08:54:28.743 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.540+0000 7fbecfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed007de30 0x7fbed007c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:28.743 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.541+0000 7fbecfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed007de30 0x7fbed007c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59348/0 (socket says 172.21.15.69:59348) 2024-04-24T08:54:28.743 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.541+0000 7fbecfe00700 1 -- 172.21.15.69:0/3266308995 learned_addr learned my addr 172.21.15.69:0/3266308995 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:28.743 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.541+0000 7fbecfe00700 1 -- 172.21.15.69:0/3266308995 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed007c9f0 con 0x7fbed007de30 2024-04-24T08:54:28.745 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.541+0000 7fbecfe00700 1 --2- 172.21.15.69:0/3266308995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed007de30 0x7fbed007c330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbec00098a0 tx=0x7fbec0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160df088021d7796 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:28.745 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.542+0000 7fbecea00700 1 -- 172.21.15.69:0/3266308995 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbec0004160 con 0x7fbed007de30 2024-04-24T08:54:28.745 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.542+0000 7fbecea00700 1 -- 172.21.15.69:0/3266308995 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbec0004300 con 0x7fbed007de30 2024-04-24T08:54:28.745 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.542+0000 7fbecea00700 1 -- 172.21.15.69:0/3266308995 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbec0011a50 con 0x7fbed007de30 2024-04-24T08:54:28.745 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.542+0000 7fbed7200700 1 -- 172.21.15.69:0/3266308995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed007de30 msgr2=0x7fbed007c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:28.745 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.542+0000 7fbed7200700 1 --2- 172.21.15.69:0/3266308995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed007de30 0x7fbed007c330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbec00098a0 tx=0x7fbec0009bb0 comp rx=0 tx=0).stop 2024-04-24T08:54:28.745 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.542+0000 7fbed7200700 1 -- 172.21.15.69:0/3266308995 shutdown_connections 2024-04-24T08:54:28.745 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.542+0000 7fbed7200700 1 --2- 172.21.15.69:0/3266308995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed007de30 0x7fbed007c330 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:28.745 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.542+0000 7fbed7200700 1 -- 172.21.15.69:0/3266308995 >> 172.21.15.69:0/3266308995 conn(0x7fbed0103a20 msgr2=0x7fbed0105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:28.746 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.543+0000 7fbed7200700 1 -- 172.21.15.69:0/3266308995 shutdown_connections 2024-04-24T08:54:28.746 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.543+0000 7fbed7200700 1 -- 172.21.15.69:0/3266308995 wait complete. 2024-04-24T08:54:28.746 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.544+0000 7fbed7200700 1 Processor -- start 2024-04-24T08:54:28.747 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.544+0000 7fbed7200700 1 -- start start 2024-04-24T08:54:28.748 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.544+0000 7fbed7200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed007de30 0x7fbed019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:28.748 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.544+0000 7fbed7200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbed010e8e0 con 0x7fbed007de30 2024-04-24T08:54:28.748 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.545+0000 7fbecfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed007de30 0x7fbed019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:28.748 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.545+0000 7fbecfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed007de30 0x7fbed019e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59352/0 (socket says 172.21.15.69:59352) 2024-04-24T08:54:28.748 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.545+0000 7fbecfe00700 1 -- 172.21.15.69:0/2767449745 learned_addr learned my addr 172.21.15.69:0/2767449745 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:28.748 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.545+0000 7fbecfe00700 1 -- 172.21.15.69:0/2767449745 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed019ec70 con 0x7fbed007de30 2024-04-24T08:54:28.748 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.546+0000 7fbecfe00700 1 --2- 172.21.15.69:0/2767449745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed007de30 0x7fbed019e730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbec0006b20 tx=0x7fbec0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:28.748 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.546+0000 7fbeccc00700 1 -- 172.21.15.69:0/2767449745 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbec0003b70 con 0x7fbed007de30 2024-04-24T08:54:28.748 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.546+0000 7fbeccc00700 1 -- 172.21.15.69:0/2767449745 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbec001a8e0 con 0x7fbed007de30 2024-04-24T08:54:28.748 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.546+0000 7fbeccc00700 1 -- 172.21.15.69:0/2767449745 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbec00249b0 con 0x7fbed007de30 2024-04-24T08:54:28.748 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.546+0000 7fbed7200700 1 -- 172.21.15.69:0/2767449745 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed019ef00 con 0x7fbed007de30 2024-04-24T08:54:28.748 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.546+0000 7fbed7200700 1 -- 172.21.15.69:0/2767449745 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed01a1ba0 con 0x7fbed007de30 2024-04-24T08:54:28.748 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.547+0000 7fbeccc00700 1 -- 172.21.15.69:0/2767449745 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fbec0022020 con 0x7fbed007de30 2024-04-24T08:54:28.749 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.548+0000 7fbeccc00700 1 --2- 172.21.15.69:0/2767449745 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbeb4039660 0x7fbeb403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:28.751 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.548+0000 7fbeccc00700 1 -- 172.21.15.69:0/2767449745 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbec004e030 con 0x7fbed007de30 2024-04-24T08:54:28.751 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.548+0000 7fbed7200700 1 -- 172.21.15.69:0/2767449745 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed0064150 con 0x7fbed007de30 2024-04-24T08:54:28.751 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.548+0000 7fbecf400700 1 --2- 172.21.15.69:0/2767449745 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbeb4039660 0x7fbeb403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:28.751 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.549+0000 7fbecf400700 1 --2- 172.21.15.69:0/2767449745 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbeb4039660 0x7fbeb403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbebc006fd0 tx=0x7fbebc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:28.751 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.555+0000 7fbeccc00700 1 -- 172.21.15.69:0/2767449745 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbec002c3a0 con 0x7fbed007de30 2024-04-24T08:54:28.751 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:27.790+0000 7fbed7200700 1 -- 172.21.15.69:0/2767449745 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fbed00765e0 con 0x7fbeb4039660 2024-04-24T08:54:28.751 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:28.019+0000 7fbeccc00700 1 -- 172.21.15.69:0/2767449745 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fbed00765e0 con 0x7fbeb4039660 2024-04-24T08:54:28.751 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:28.022+0000 7fbed7200700 1 -- 172.21.15.69:0/2767449745 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbeb4039660 msgr2=0x7fbeb403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:28.751 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:28.022+0000 7fbed7200700 1 --2- 172.21.15.69:0/2767449745 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbeb4039660 0x7fbeb403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbebc006fd0 tx=0x7fbebc006e40 comp rx=0 tx=0).stop 2024-04-24T08:54:28.751 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:28.022+0000 7fbed7200700 1 -- 172.21.15.69:0/2767449745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed007de30 msgr2=0x7fbed019e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:28.752 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:28.022+0000 7fbed7200700 1 --2- 172.21.15.69:0/2767449745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed007de30 0x7fbed019e730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbec0006b20 tx=0x7fbec0011ad0 comp rx=0 tx=0).stop 2024-04-24T08:54:28.752 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:28.022+0000 7fbed7200700 1 -- 172.21.15.69:0/2767449745 shutdown_connections 2024-04-24T08:54:28.752 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:28.022+0000 7fbed7200700 1 --2- 172.21.15.69:0/2767449745 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbeb4039660 0x7fbeb403bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:28.752 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:28.022+0000 7fbed7200700 1 --2- 172.21.15.69:0/2767449745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbed007de30 0x7fbed019e730 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:28.752 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:28.022+0000 7fbed7200700 1 -- 172.21.15.69:0/2767449745 >> 172.21.15.69:0/2767449745 conn(0x7fbed0103a20 msgr2=0x7fbed0104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:28.752 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:28.023+0000 7fbed7200700 1 -- 172.21.15.69:0/2767449745 shutdown_connections 2024-04-24T08:54:28.752 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:28.023+0000 7fbed7200700 1 -- 172.21.15.69:0/2767449745 wait complete. 2024-04-24T08:54:28.752 INFO:teuthology.orchestra.run.smithi069.stdout:Fetching dashboard port number... 2024-04-24T08:54:29.315 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:29 smithi069 ceph-mon[28170]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:54:29.316 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:29 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:29.316 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:29 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:30.270 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:30 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2841463212' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-24T08:54:31.170 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 8443 2024-04-24T08:54:31.170 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.315+0000 7f5f57600700 1 Processor -- start 2024-04-24T08:54:31.170 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.315+0000 7f5f57600700 1 -- start start 2024-04-24T08:54:31.170 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.316+0000 7f5f57600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f50108e10 0x7f5f5010b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.171 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.316+0000 7f5f57600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5007bec0 con 0x7f5f50108e10 2024-04-24T08:54:31.171 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.316+0000 7f5f54c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f50108e10 0x7f5f5010b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.171 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.316+0000 7f5f54c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f50108e10 0x7f5f5010b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59362/0 (socket says 172.21.15.69:59362) 2024-04-24T08:54:31.171 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.316+0000 7f5f54c00700 1 -- 172.21.15.69:0/3799506240 learned_addr learned my addr 172.21.15.69:0/3799506240 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:31.171 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.317+0000 7f5f54c00700 1 -- 172.21.15.69:0/3799506240 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5010b740 con 0x7f5f50108e10 2024-04-24T08:54:31.171 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.317+0000 7f5f54c00700 1 --2- 172.21.15.69:0/3799506240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f50108e10 0x7f5f5010b200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009ab0 tx=0x7f5f44009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ff714f0ad4e132 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:31.171 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.317+0000 7f5f4f400700 1 -- 172.21.15.69:0/3799506240 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f44004160 con 0x7f5f50108e10 2024-04-24T08:54:31.171 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.317+0000 7f5f4f400700 1 -- 172.21.15.69:0/3799506240 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f4400bc90 con 0x7f5f50108e10 2024-04-24T08:54:31.171 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.317+0000 7f5f4f400700 1 -- 172.21.15.69:0/3799506240 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f44003be0 con 0x7f5f50108e10 2024-04-24T08:54:31.171 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.318+0000 7f5f57600700 1 -- 172.21.15.69:0/3799506240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f50108e10 msgr2=0x7f5f5010b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:31.171 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.318+0000 7f5f57600700 1 --2- 172.21.15.69:0/3799506240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f50108e10 0x7f5f5010b200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009ab0 tx=0x7f5f44009dc0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.171 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.318+0000 7f5f57600700 1 -- 172.21.15.69:0/3799506240 shutdown_connections 2024-04-24T08:54:31.171 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.318+0000 7f5f57600700 1 --2- 172.21.15.69:0/3799506240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f50108e10 0x7f5f5010b200 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.172 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.318+0000 7f5f57600700 1 -- 172.21.15.69:0/3799506240 >> 172.21.15.69:0/3799506240 conn(0x7f5f50102880 msgr2=0x7f5f50104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:31.172 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.318+0000 7f5f57600700 1 -- 172.21.15.69:0/3799506240 shutdown_connections 2024-04-24T08:54:31.172 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.318+0000 7f5f57600700 1 -- 172.21.15.69:0/3799506240 wait complete. 2024-04-24T08:54:31.172 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.320+0000 7f5f57600700 1 Processor -- start 2024-04-24T08:54:31.172 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.320+0000 7f5f57600700 1 -- start start 2024-04-24T08:54:31.172 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.320+0000 7f5f57600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f50108e10 0x7f5f5019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.172 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.320+0000 7f5f57600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5010c750 con 0x7f5f50108e10 2024-04-24T08:54:31.172 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.320+0000 7f5f54c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f50108e10 0x7f5f5019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.172 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.320+0000 7f5f54c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f50108e10 0x7f5f5019c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59364/0 (socket says 172.21.15.69:59364) 2024-04-24T08:54:31.172 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.321+0000 7f5f54c00700 1 -- 172.21.15.69:0/2841463212 learned_addr learned my addr 172.21.15.69:0/2841463212 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:31.172 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.321+0000 7f5f54c00700 1 -- 172.21.15.69:0/2841463212 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5019cbb0 con 0x7f5f50108e10 2024-04-24T08:54:31.172 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.321+0000 7f5f54c00700 1 --2- 172.21.15.69:0/2841463212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f50108e10 0x7f5f5019c670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f440037e0 tx=0x7f5f4400bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:31.172 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.321+0000 7f5f4d600700 1 -- 172.21.15.69:0/2841463212 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f440041d0 con 0x7f5f50108e10 2024-04-24T08:54:31.172 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.321+0000 7f5f4d600700 1 -- 172.21.15.69:0/2841463212 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f44004370 con 0x7f5f50108e10 2024-04-24T08:54:31.172 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.321+0000 7f5f57600700 1 -- 172.21.15.69:0/2841463212 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5019ce40 con 0x7f5f50108e10 2024-04-24T08:54:31.173 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.322+0000 7f5f4d600700 1 -- 172.21.15.69:0/2841463212 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f440115b0 con 0x7f5f50108e10 2024-04-24T08:54:31.173 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.322+0000 7f5f57600700 1 -- 172.21.15.69:0/2841463212 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f5019fb30 con 0x7f5f50108e10 2024-04-24T08:54:31.173 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.323+0000 7f5f4d600700 1 -- 172.21.15.69:0/2841463212 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f5f440117e0 con 0x7f5f50108e10 2024-04-24T08:54:31.173 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.323+0000 7f5f4d600700 1 --2- 172.21.15.69:0/2841463212 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5f38039660 0x7f5f3803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.173 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.323+0000 7f5f4d600700 1 -- 172.21.15.69:0/2841463212 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5f4404e210 con 0x7f5f50108e10 2024-04-24T08:54:31.173 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.323+0000 7f5f57600700 1 -- 172.21.15.69:0/2841463212 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f50064150 con 0x7f5f50108e10 2024-04-24T08:54:31.173 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.324+0000 7f5f4fe00700 1 --2- 172.21.15.69:0/2841463212 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5f38039660 0x7f5f3803bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.173 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.324+0000 7f5f4fe00700 1 --2- 172.21.15.69:0/2841463212 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5f38039660 0x7f5f3803bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c006fd0 tx=0x7f5f3c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:31.173 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.329+0000 7f5f4d600700 1 -- 172.21.15.69:0/2841463212 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f4401f030 con 0x7f5f50108e10 2024-04-24T08:54:31.173 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.560+0000 7f5f57600700 1 -- 172.21.15.69:0/2841463212 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f5f5002dfe0 con 0x7f5f50108e10 2024-04-24T08:54:31.173 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.561+0000 7f5f4d600700 1 -- 172.21.15.69:0/2841463212 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f5f44010380 con 0x7f5f50108e10 2024-04-24T08:54:31.173 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.564+0000 7f5f57600700 1 -- 172.21.15.69:0/2841463212 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5f38039660 msgr2=0x7f5f3803bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:31.173 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.564+0000 7f5f57600700 1 --2- 172.21.15.69:0/2841463212 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5f38039660 0x7f5f3803bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c006fd0 tx=0x7f5f3c006e40 comp rx=0 tx=0).stop 2024-04-24T08:54:31.174 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.564+0000 7f5f57600700 1 -- 172.21.15.69:0/2841463212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f50108e10 msgr2=0x7f5f5019c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:31.174 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.564+0000 7f5f57600700 1 --2- 172.21.15.69:0/2841463212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f50108e10 0x7f5f5019c670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f440037e0 tx=0x7f5f4400bac0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.174 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.564+0000 7f5f57600700 1 -- 172.21.15.69:0/2841463212 shutdown_connections 2024-04-24T08:54:31.174 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.564+0000 7f5f57600700 1 --2- 172.21.15.69:0/2841463212 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5f38039660 0x7f5f3803bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.174 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.564+0000 7f5f57600700 1 --2- 172.21.15.69:0/2841463212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f50108e10 0x7f5f5019c670 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.174 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.564+0000 7f5f57600700 1 -- 172.21.15.69:0/2841463212 >> 172.21.15.69:0/2841463212 conn(0x7f5f50102880 msgr2=0x7f5f50103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:31.174 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.565+0000 7f5f57600700 1 -- 172.21.15.69:0/2841463212 shutdown_connections 2024-04-24T08:54:31.174 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:29.565+0000 7f5f57600700 1 -- 172.21.15.69:0/2841463212 wait complete. 2024-04-24T08:54:31.204 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T08:54:31.205 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-04-24T08:54:31.231 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T08:54:31.231 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-04-24T08:54:31.231 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-04-24T08:54:31.232 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-24T08:54:31.239 INFO:teuthology.orchestra.run.smithi069.stdout:Ceph Dashboard is now available at: 2024-04-24T08:54:31.239 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:54:31.239 INFO:teuthology.orchestra.run.smithi069.stdout: URL: https://smithi069.front.sepia.ceph.com:8443/ 2024-04-24T08:54:31.239 INFO:teuthology.orchestra.run.smithi069.stdout: User: admin 2024-04-24T08:54:31.239 INFO:teuthology.orchestra.run.smithi069.stdout: Password: 17r8b3y2m3 2024-04-24T08:54:31.239 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:54:31.241 INFO:teuthology.orchestra.run.smithi069.stdout:Saving cluster configuration to /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config directory 2024-04-24T08:54:31.242 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling autotune for osd_memory_target 2024-04-24T08:54:31.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:31 smithi069 ceph-mon[28170]: mgrmap e13: x(active, since 6s) 2024-04-24T08:54:32.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.900+0000 7fb4d5000700 1 Processor -- start 2024-04-24T08:54:32.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.900+0000 7fb4d5000700 1 -- start start 2024-04-24T08:54:32.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.901+0000 7fb4d5000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d010a620 0x7fb4d010aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.901+0000 7fb4d5000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d010b010 con 0x7fb4d010a620 2024-04-24T08:54:32.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.901+0000 7fb4cde00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d010a620 0x7fb4d010aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.901+0000 7fb4cde00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d010a620 0x7fb4d010aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59370/0 (socket says 172.21.15.69:59370) 2024-04-24T08:54:32.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.901+0000 7fb4cde00700 1 -- 172.21.15.69:0/4039273351 learned_addr learned my addr 172.21.15.69:0/4039273351 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:32.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.902+0000 7fb4cde00700 1 -- 172.21.15.69:0/4039273351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d010b780 con 0x7fb4d010a620 2024-04-24T08:54:32.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.902+0000 7fb4cde00700 1 --2- 172.21.15.69:0/4039273351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d010a620 0x7fb4d010aa40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00098a0 tx=0x7fb4c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5842ae94229dd58f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.902+0000 7fb4cca00700 1 -- 172.21.15.69:0/4039273351 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4c0004160 con 0x7fb4d010a620 2024-04-24T08:54:32.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.902+0000 7fb4cca00700 1 -- 172.21.15.69:0/4039273351 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4c0004300 con 0x7fb4d010a620 2024-04-24T08:54:32.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.902+0000 7fb4cca00700 1 -- 172.21.15.69:0/4039273351 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4c0011a50 con 0x7fb4d010a620 2024-04-24T08:54:32.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.903+0000 7fb4d5000700 1 -- 172.21.15.69:0/4039273351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d010a620 msgr2=0x7fb4d010aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.903+0000 7fb4d5000700 1 --2- 172.21.15.69:0/4039273351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d010a620 0x7fb4d010aa40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00098a0 tx=0x7fb4c0009bb0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.903+0000 7fb4d5000700 1 -- 172.21.15.69:0/4039273351 shutdown_connections 2024-04-24T08:54:32.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.903+0000 7fb4d5000700 1 --2- 172.21.15.69:0/4039273351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d010a620 0x7fb4d010aa40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.903+0000 7fb4d5000700 1 -- 172.21.15.69:0/4039273351 >> 172.21.15.69:0/4039273351 conn(0x7fb4d007dc00 msgr2=0x7fb4d01084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:32.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.903+0000 7fb4d5000700 1 -- 172.21.15.69:0/4039273351 shutdown_connections 2024-04-24T08:54:32.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.903+0000 7fb4d5000700 1 -- 172.21.15.69:0/4039273351 wait complete. 2024-04-24T08:54:32.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.904+0000 7fb4d5000700 1 Processor -- start 2024-04-24T08:54:32.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.904+0000 7fb4d5000700 1 -- start start 2024-04-24T08:54:32.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.905+0000 7fb4d5000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d010a620 0x7fb4d019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.905+0000 7fb4d5000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d010eb30 con 0x7fb4d010a620 2024-04-24T08:54:32.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.905+0000 7fb4cde00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d010a620 0x7fb4d019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.905+0000 7fb4cde00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d010a620 0x7fb4d019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59372/0 (socket says 172.21.15.69:59372) 2024-04-24T08:54:32.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.905+0000 7fb4cde00700 1 -- 172.21.15.69:0/1364714422 learned_addr learned my addr 172.21.15.69:0/1364714422 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:32.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.905+0000 7fb4cde00700 1 -- 172.21.15.69:0/1364714422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d019ed20 con 0x7fb4d010a620 2024-04-24T08:54:32.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.905+0000 7fb4cde00700 1 --2- 172.21.15.69:0/1364714422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d010a620 0x7fb4d019e7e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0011a20 tx=0x7fb4c0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.906+0000 7fb4c6a00700 1 -- 172.21.15.69:0/1364714422 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4c0003af0 con 0x7fb4d010a620 2024-04-24T08:54:32.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.906+0000 7fb4c6a00700 1 -- 172.21.15.69:0/1364714422 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4c001a8e0 con 0x7fb4d010a620 2024-04-24T08:54:32.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.906+0000 7fb4c6a00700 1 -- 172.21.15.69:0/1364714422 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4c0024860 con 0x7fb4d010a620 2024-04-24T08:54:32.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.906+0000 7fb4d5000700 1 -- 172.21.15.69:0/1364714422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d019efb0 con 0x7fb4d010a620 2024-04-24T08:54:32.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.906+0000 7fb4d5000700 1 -- 172.21.15.69:0/1364714422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d01a1ca0 con 0x7fb4d010a620 2024-04-24T08:54:32.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.907+0000 7fb4d5000700 1 -- 172.21.15.69:0/1364714422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d0064150 con 0x7fb4d010a620 2024-04-24T08:54:32.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.907+0000 7fb4c6a00700 1 -- 172.21.15.69:0/1364714422 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb4c0022020 con 0x7fb4d010a620 2024-04-24T08:54:32.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.908+0000 7fb4c6a00700 1 --2- 172.21.15.69:0/1364714422 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb4b4039700 0x7fb4b403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.908+0000 7fb4c6a00700 1 -- 172.21.15.69:0/1364714422 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb4c004dc00 con 0x7fb4d010a620 2024-04-24T08:54:32.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.908+0000 7fb4cd400700 1 --2- 172.21.15.69:0/1364714422 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb4b4039700 0x7fb4b403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.909+0000 7fb4cd400700 1 --2- 172.21.15.69:0/1364714422 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb4b4039700 0x7fb4b403bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc006fd0 tx=0x7fb4bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:31.913+0000 7fb4c6a00700 1 -- 172.21.15.69:0/1364714422 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4c002c3a0 con 0x7fb4d010a620 2024-04-24T08:54:32.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:32.138+0000 7fb4d5000700 1 -- 172.21.15.69:0/1364714422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fb4d002dfe0 con 0x7fb4d010a620 2024-04-24T08:54:32.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:32.139+0000 7fb4c6a00700 1 -- 172.21.15.69:0/1364714422 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fb4c001a570 con 0x7fb4d010a620 2024-04-24T08:54:32.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:32.141+0000 7fb4d5000700 1 -- 172.21.15.69:0/1364714422 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb4b4039700 msgr2=0x7fb4b403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:32.141+0000 7fb4d5000700 1 --2- 172.21.15.69:0/1364714422 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb4b4039700 0x7fb4b403bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc006fd0 tx=0x7fb4bc006e40 comp rx=0 tx=0).stop 2024-04-24T08:54:32.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:32.141+0000 7fb4d5000700 1 -- 172.21.15.69:0/1364714422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d010a620 msgr2=0x7fb4d019e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:32.141+0000 7fb4d5000700 1 --2- 172.21.15.69:0/1364714422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d010a620 0x7fb4d019e7e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0011a20 tx=0x7fb4c0011b00 comp rx=0 tx=0).stop 2024-04-24T08:54:32.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:32.141+0000 7fb4d5000700 1 -- 172.21.15.69:0/1364714422 shutdown_connections 2024-04-24T08:54:32.845 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:32.141+0000 7fb4d5000700 1 --2- 172.21.15.69:0/1364714422 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb4b4039700 0x7fb4b403bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.845 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:32.141+0000 7fb4d5000700 1 --2- 172.21.15.69:0/1364714422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d010a620 0x7fb4d019e7e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.845 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:32.141+0000 7fb4d5000700 1 -- 172.21.15.69:0/1364714422 >> 172.21.15.69:0/1364714422 conn(0x7fb4d007dc00 msgr2=0x7fb4d0107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:32.845 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:32.142+0000 7fb4d5000700 1 -- 172.21.15.69:0/1364714422 shutdown_connections 2024-04-24T08:54:32.845 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:32.142+0000 7fb4d5000700 1 -- 172.21.15.69:0/1364714422 wait complete. 2024-04-24T08:54:34.627 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.534+0000 7f9770200700 1 Processor -- start 2024-04-24T08:54:34.627 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.534+0000 7f9770200700 1 -- start start 2024-04-24T08:54:34.627 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.535+0000 7f9770200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9768108380 0x7f97681087a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.628 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.535+0000 7f9770200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9768108ce0 con 0x7f9768108380 2024-04-24T08:54:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.535+0000 7f976d800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9768108380 0x7f97681087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.535+0000 7f976d800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9768108380 0x7f97681087a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53768/0 (socket says 172.21.15.69:53768) 2024-04-24T08:54:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.535+0000 7f976d800700 1 -- 172.21.15.69:0/1287495932 learned_addr learned my addr 172.21.15.69:0/1287495932 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.536+0000 7f976d800700 1 -- 172.21.15.69:0/1287495932 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9768108e60 con 0x7f9768108380 2024-04-24T08:54:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.536+0000 7f976d800700 1 --2- 172.21.15.69:0/1287495932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9768108380 0x7f97681087a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f975c0098a0 tx=0x7f975c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdd014a0d8f57a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.536+0000 7f9767e00700 1 -- 172.21.15.69:0/1287495932 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f975c004160 con 0x7f9768108380 2024-04-24T08:54:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.536+0000 7f9767e00700 1 -- 172.21.15.69:0/1287495932 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f975c004300 con 0x7f9768108380 2024-04-24T08:54:34.634 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.536+0000 7f9767e00700 1 -- 172.21.15.69:0/1287495932 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f975c011a50 con 0x7f9768108380 2024-04-24T08:54:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.537+0000 7f9770200700 1 -- 172.21.15.69:0/1287495932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9768108380 msgr2=0x7f97681087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.537+0000 7f9770200700 1 --2- 172.21.15.69:0/1287495932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9768108380 0x7f97681087a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f975c0098a0 tx=0x7f975c009bb0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.537+0000 7f9770200700 1 -- 172.21.15.69:0/1287495932 shutdown_connections 2024-04-24T08:54:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.537+0000 7f9770200700 1 --2- 172.21.15.69:0/1287495932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9768108380 0x7f97681087a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.537+0000 7f9770200700 1 -- 172.21.15.69:0/1287495932 >> 172.21.15.69:0/1287495932 conn(0x7f9768103a40 msgr2=0x7f9768105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.537+0000 7f9770200700 1 -- 172.21.15.69:0/1287495932 shutdown_connections 2024-04-24T08:54:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.537+0000 7f9770200700 1 -- 172.21.15.69:0/1287495932 wait complete. 2024-04-24T08:54:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.538+0000 7f9770200700 1 Processor -- start 2024-04-24T08:54:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.539+0000 7f9770200700 1 -- start start 2024-04-24T08:54:34.637 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.539+0000 7f9770200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9768108380 0x7f976819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.539+0000 7f9770200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f976810a3c0 con 0x7f9768108380 2024-04-24T08:54:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.539+0000 7f976d800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9768108380 0x7f976819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.539+0000 7f976d800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9768108380 0x7f976819c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53784/0 (socket says 172.21.15.69:53784) 2024-04-24T08:54:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.539+0000 7f976d800700 1 -- 172.21.15.69:0/4177242415 learned_addr learned my addr 172.21.15.69:0/4177242415 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.540+0000 7f976d800700 1 -- 172.21.15.69:0/4177242415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976819ca80 con 0x7f9768108380 2024-04-24T08:54:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.540+0000 7f976d800700 1 --2- 172.21.15.69:0/4177242415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9768108380 0x7f976819c540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f975c006b20 tx=0x7f975c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.540+0000 7f9766000700 1 -- 172.21.15.69:0/4177242415 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f975c003b70 con 0x7f9768108380 2024-04-24T08:54:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.540+0000 7f9766000700 1 -- 172.21.15.69:0/4177242415 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f975c01a8e0 con 0x7f9768108380 2024-04-24T08:54:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.540+0000 7f9766000700 1 -- 172.21.15.69:0/4177242415 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f975c0249b0 con 0x7f9768108380 2024-04-24T08:54:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.540+0000 7f9770200700 1 -- 172.21.15.69:0/4177242415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976819cd10 con 0x7f9768108380 2024-04-24T08:54:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.540+0000 7f9770200700 1 -- 172.21.15.69:0/4177242415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f976819fa00 con 0x7f9768108380 2024-04-24T08:54:34.638 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.542+0000 7f9766000700 1 -- 172.21.15.69:0/4177242415 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f975c022020 con 0x7f9768108380 2024-04-24T08:54:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.541+0000 7f975b400700 1 -- 172.21.15.69:0/4177242415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9768064150 con 0x7f9768108380 2024-04-24T08:54:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.542+0000 7f9766000700 1 --2- 172.21.15.69:0/4177242415 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9750039700 0x7f975003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.542+0000 7f9766000700 1 -- 172.21.15.69:0/4177242415 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f975c04e040 con 0x7f9768108380 2024-04-24T08:54:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.542+0000 7f976ce00700 1 --2- 172.21.15.69:0/4177242415 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9750039700 0x7f975003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.543+0000 7f976ce00700 1 --2- 172.21.15.69:0/4177242415 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9750039700 0x7f975003bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9754006fd0 tx=0x7f9754006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.548+0000 7f9766000700 1 -- 172.21.15.69:0/4177242415 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f975c0212c0 con 0x7f9768108380 2024-04-24T08:54:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.890+0000 7f975b400700 1 -- 172.21.15.69:0/4177242415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f9768063700 con 0x7f9768108380 2024-04-24T08:54:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.893+0000 7f9766000700 1 -- 172.21.15.69:0/4177242415 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f975c0094f0 con 0x7f9768108380 2024-04-24T08:54:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.896+0000 7f975b400700 1 -- 172.21.15.69:0/4177242415 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9750039700 msgr2=0x7f975003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.896+0000 7f975b400700 1 --2- 172.21.15.69:0/4177242415 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9750039700 0x7f975003bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9754006fd0 tx=0x7f9754006e40 comp rx=0 tx=0).stop 2024-04-24T08:54:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.896+0000 7f975b400700 1 -- 172.21.15.69:0/4177242415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9768108380 msgr2=0x7f976819c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.896+0000 7f975b400700 1 --2- 172.21.15.69:0/4177242415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9768108380 0x7f976819c540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f975c006b20 tx=0x7f975c011ad0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.896+0000 7f975b400700 1 -- 172.21.15.69:0/4177242415 shutdown_connections 2024-04-24T08:54:34.639 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.896+0000 7f975b400700 1 --2- 172.21.15.69:0/4177242415 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9750039700 0x7f975003bbc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.896+0000 7f975b400700 1 --2- 172.21.15.69:0/4177242415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9768108380 0x7f976819c540 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.896+0000 7f975b400700 1 -- 172.21.15.69:0/4177242415 >> 172.21.15.69:0/4177242415 conn(0x7f9768103a40 msgr2=0x7f9768104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:34.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.897+0000 7f975b400700 1 -- 172.21.15.69:0/4177242415 shutdown_connections 2024-04-24T08:54:34.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-24T08:54:33.897+0000 7f975b400700 1 -- 172.21.15.69:0/4177242415 wait complete. 2024-04-24T08:54:34.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-24T08:54:34.640 INFO:teuthology.orchestra.run.smithi069.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-24T08:54:34.640 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:54:34.640 INFO:teuthology.orchestra.run.smithi069.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-24T08:54:34.640 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:54:34.640 INFO:teuthology.orchestra.run.smithi069.stdout:Or, if you are only running a single cluster on this host: 2024-04-24T08:54:34.640 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:54:34.640 INFO:teuthology.orchestra.run.smithi069.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-24T08:54:34.640 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:54:34.640 INFO:teuthology.orchestra.run.smithi069.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-24T08:54:34.640 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:54:34.640 INFO:teuthology.orchestra.run.smithi069.stdout: ceph telemetry on 2024-04-24T08:54:34.641 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:54:34.641 INFO:teuthology.orchestra.run.smithi069.stdout:For more information see: 2024-04-24T08:54:34.641 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:54:34.641 INFO:teuthology.orchestra.run.smithi069.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-24T08:54:34.641 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:54:34.641 INFO:teuthology.orchestra.run.smithi069.stdout:Bootstrap complete. 2024-04-24T08:54:34.727 INFO:tasks.cephadm:Fetching config... 2024-04-24T08:54:34.727 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T08:54:34.728 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-24T08:54:34.771 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-24T08:54:34.772 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T08:54:34.772 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-24T08:54:34.859 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-24T08:54:34.859 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T08:54:34.859 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-24T08:54:34.968 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-24T08:54:34.968 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T08:54:34.968 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-24T08:54:35.060 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-24T08:54:35.060 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoPmsuDpFJRrlNilvQZllprWXojEuEiZvDWlwE3H384v9WEYLlUN1SpjvcJwIOc4Csxy7oKPSjiCZaTcV4c5rI9XdSBLUgE93khBI9GAxHBszJydJQcWNA1YwpCU0+o/7GjttuFlFIQ+vzydEGHNDE8sqjJXiT5oMLJZG9/vBLgrebReUmDwbTZgjzwKtny3aIKySNl0kItSOPN0yUbbX3IpEN1EAkkn6/TP8DmrKkjMIebwlg6SOrOEaQIInEQ3WEY4beN16u8CwLBgNuVH/mApSCC/2HOfx9l0KWLP738BYZdWNA6V7vdKLfmp82DkIYfBTih5T+ueJmUknY2F2MVsjYxZ70NyfTPC1AsWrdgWgSQ4hACy+oKg0P/92DJsjIelDodKF9XmQLSUxc5+Dfg7QlKwpmjyUZe433mRhsvI+nTWzNU0I4KmtEkCUq2V7ZnIXZHIOuiQYKlMGgZyycZwoZd9pZQOLp9GEAkwc3me5hcbprfUhtSFDHg/ThR4c= ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T08:54:35.160 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:34 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/4177242415' entity='client.admin' 2024-04-24T08:54:35.168 INFO:teuthology.orchestra.run.smithi069.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoPmsuDpFJRrlNilvQZllprWXojEuEiZvDWlwE3H384v9WEYLlUN1SpjvcJwIOc4Csxy7oKPSjiCZaTcV4c5rI9XdSBLUgE93khBI9GAxHBszJydJQcWNA1YwpCU0+o/7GjttuFlFIQ+vzydEGHNDE8sqjJXiT5oMLJZG9/vBLgrebReUmDwbTZgjzwKtny3aIKySNl0kItSOPN0yUbbX3IpEN1EAkkn6/TP8DmrKkjMIebwlg6SOrOEaQIInEQ3WEY4beN16u8CwLBgNuVH/mApSCC/2HOfx9l0KWLP738BYZdWNA6V7vdKLfmp82DkIYfBTih5T+ueJmUknY2F2MVsjYxZ70NyfTPC1AsWrdgWgSQ4hACy+oKg0P/92DJsjIelDodKF9XmQLSUxc5+Dfg7QlKwpmjyUZe433mRhsvI+nTWzNU0I4KmtEkCUq2V7ZnIXZHIOuiQYKlMGgZyycZwoZd9pZQOLp9GEAkwc3me5hcbprfUhtSFDHg/ThR4c= ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:54:35.216 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoPmsuDpFJRrlNilvQZllprWXojEuEiZvDWlwE3H384v9WEYLlUN1SpjvcJwIOc4Csxy7oKPSjiCZaTcV4c5rI9XdSBLUgE93khBI9GAxHBszJydJQcWNA1YwpCU0+o/7GjttuFlFIQ+vzydEGHNDE8sqjJXiT5oMLJZG9/vBLgrebReUmDwbTZgjzwKtny3aIKySNl0kItSOPN0yUbbX3IpEN1EAkkn6/TP8DmrKkjMIebwlg6SOrOEaQIInEQ3WEY4beN16u8CwLBgNuVH/mApSCC/2HOfx9l0KWLP738BYZdWNA6V7vdKLfmp82DkIYfBTih5T+ueJmUknY2F2MVsjYxZ70NyfTPC1AsWrdgWgSQ4hACy+oKg0P/92DJsjIelDodKF9XmQLSUxc5+Dfg7QlKwpmjyUZe433mRhsvI+nTWzNU0I4KmtEkCUq2V7ZnIXZHIOuiQYKlMGgZyycZwoZd9pZQOLp9GEAkwc3me5hcbprfUhtSFDHg/ThR4c= ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T08:54:35.346 INFO:teuthology.orchestra.run.smithi081.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoPmsuDpFJRrlNilvQZllprWXojEuEiZvDWlwE3H384v9WEYLlUN1SpjvcJwIOc4Csxy7oKPSjiCZaTcV4c5rI9XdSBLUgE93khBI9GAxHBszJydJQcWNA1YwpCU0+o/7GjttuFlFIQ+vzydEGHNDE8sqjJXiT5oMLJZG9/vBLgrebReUmDwbTZgjzwKtny3aIKySNl0kItSOPN0yUbbX3IpEN1EAkkn6/TP8DmrKkjMIebwlg6SOrOEaQIInEQ3WEY4beN16u8CwLBgNuVH/mApSCC/2HOfx9l0KWLP738BYZdWNA6V7vdKLfmp82DkIYfBTih5T+ueJmUknY2F2MVsjYxZ70NyfTPC1AsWrdgWgSQ4hACy+oKg0P/92DJsjIelDodKF9XmQLSUxc5+Dfg7QlKwpmjyUZe433mRhsvI+nTWzNU0I4KmtEkCUq2V7ZnIXZHIOuiQYKlMGgZyycZwoZd9pZQOLp9GEAkwc3me5hcbprfUhtSFDHg/ThR4c= ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:54:35.406 DEBUG:teuthology.orchestra.run.smithi130:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoPmsuDpFJRrlNilvQZllprWXojEuEiZvDWlwE3H384v9WEYLlUN1SpjvcJwIOc4Csxy7oKPSjiCZaTcV4c5rI9XdSBLUgE93khBI9GAxHBszJydJQcWNA1YwpCU0+o/7GjttuFlFIQ+vzydEGHNDE8sqjJXiT5oMLJZG9/vBLgrebReUmDwbTZgjzwKtny3aIKySNl0kItSOPN0yUbbX3IpEN1EAkkn6/TP8DmrKkjMIebwlg6SOrOEaQIInEQ3WEY4beN16u8CwLBgNuVH/mApSCC/2HOfx9l0KWLP738BYZdWNA6V7vdKLfmp82DkIYfBTih5T+ueJmUknY2F2MVsjYxZ70NyfTPC1AsWrdgWgSQ4hACy+oKg0P/92DJsjIelDodKF9XmQLSUxc5+Dfg7QlKwpmjyUZe433mRhsvI+nTWzNU0I4KmtEkCUq2V7ZnIXZHIOuiQYKlMGgZyycZwoZd9pZQOLp9GEAkwc3me5hcbprfUhtSFDHg/ThR4c= ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T08:54:35.525 INFO:teuthology.orchestra.run.smithi130.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoPmsuDpFJRrlNilvQZllprWXojEuEiZvDWlwE3H384v9WEYLlUN1SpjvcJwIOc4Csxy7oKPSjiCZaTcV4c5rI9XdSBLUgE93khBI9GAxHBszJydJQcWNA1YwpCU0+o/7GjttuFlFIQ+vzydEGHNDE8sqjJXiT5oMLJZG9/vBLgrebReUmDwbTZgjzwKtny3aIKySNl0kItSOPN0yUbbX3IpEN1EAkkn6/TP8DmrKkjMIebwlg6SOrOEaQIInEQ3WEY4beN16u8CwLBgNuVH/mApSCC/2HOfx9l0KWLP738BYZdWNA6V7vdKLfmp82DkIYfBTih5T+ueJmUknY2F2MVsjYxZ70NyfTPC1AsWrdgWgSQ4hACy+oKg0P/92DJsjIelDodKF9XmQLSUxc5+Dfg7QlKwpmjyUZe433mRhsvI+nTWzNU0I4KmtEkCUq2V7ZnIXZHIOuiQYKlMGgZyycZwoZd9pZQOLp9GEAkwc3me5hcbprfUhtSFDHg/ThR4c= ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:54:35.578 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-24T08:54:35.983 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:54:38.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.802+0000 7fd49a400700 1 -- 172.21.15.69:0/3347711278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd494101450 msgr2=0x7fd494101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:38.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.802+0000 7fd49a400700 1 --2- 172.21.15.69:0/3347711278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd494101450 0x7fd494101870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd484009910 tx=0x7fd484009c20 comp rx=0 tx=0).stop 2024-04-24T08:54:38.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.803+0000 7fd49a400700 1 -- 172.21.15.69:0/3347711278 shutdown_connections 2024-04-24T08:54:38.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.803+0000 7fd49a400700 1 --2- 172.21.15.69:0/3347711278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd494101450 0x7fd494101870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:38.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.803+0000 7fd49a400700 1 -- 172.21.15.69:0/3347711278 >> 172.21.15.69:0/3347711278 conn(0x7fd4940fc9f0 msgr2=0x7fd4940fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:38.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.803+0000 7fd49a400700 1 -- 172.21.15.69:0/3347711278 shutdown_connections 2024-04-24T08:54:38.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.804+0000 7fd49a400700 1 -- 172.21.15.69:0/3347711278 wait complete. 2024-04-24T08:54:38.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.805+0000 7fd49a400700 1 Processor -- start 2024-04-24T08:54:38.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.805+0000 7fd49a400700 1 -- start start 2024-04-24T08:54:38.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.805+0000 7fd49a400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd494101450 0x7fd494107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:38.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.805+0000 7fd49a400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd494108490 con 0x7fd494101450 2024-04-24T08:54:38.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.805+0000 7fd493400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd494101450 0x7fd494107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:38.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.805+0000 7fd493400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd494101450 0x7fd494107440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53796/0 (socket says 172.21.15.69:53796) 2024-04-24T08:54:38.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.805+0000 7fd493400700 1 -- 172.21.15.69:0/3355057696 learned_addr learned my addr 172.21.15.69:0/3355057696 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:38.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.806+0000 7fd493400700 1 -- 172.21.15.69:0/3355057696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd494107980 con 0x7fd494101450 2024-04-24T08:54:38.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.806+0000 7fd493400700 1 --2- 172.21.15.69:0/3355057696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd494101450 0x7fd494107440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd484005800 tx=0x7fd484009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:38.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.806+0000 7fd48be00700 1 -- 172.21.15.69:0/3355057696 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd484027070 con 0x7fd494101450 2024-04-24T08:54:38.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.806+0000 7fd48be00700 1 -- 172.21.15.69:0/3355057696 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd48400be10 con 0x7fd494101450 2024-04-24T08:54:38.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.806+0000 7fd48be00700 1 -- 172.21.15.69:0/3355057696 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd48400f460 con 0x7fd494101450 2024-04-24T08:54:38.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.806+0000 7fd49a400700 1 -- 172.21.15.69:0/3355057696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd494105b70 con 0x7fd494101450 2024-04-24T08:54:38.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.807+0000 7fd49a400700 1 -- 172.21.15.69:0/3355057696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4941060b0 con 0x7fd494101450 2024-04-24T08:54:38.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.808+0000 7fd48be00700 1 -- 172.21.15.69:0/3355057696 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd48402a070 con 0x7fd494101450 2024-04-24T08:54:38.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.808+0000 7fd49a400700 1 -- 172.21.15.69:0/3355057696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd494108c10 con 0x7fd494101450 2024-04-24T08:54:38.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.808+0000 7fd48be00700 1 --2- 172.21.15.69:0/3355057696 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd4780396b0 0x7fd47803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:38.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.808+0000 7fd48be00700 1 -- 172.21.15.69:0/3355057696 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd48401e030 con 0x7fd494101450 2024-04-24T08:54:38.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.808+0000 7fd492a00700 1 --2- 172.21.15.69:0/3355057696 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd4780396b0 0x7fd47803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:38.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.809+0000 7fd492a00700 1 --2- 172.21.15.69:0/3355057696 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd4780396b0 0x7fd47803bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd480006fd0 tx=0x7fd480006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:38.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:38.814+0000 7fd48be00700 1 -- 172.21.15.69:0/3355057696 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd484013070 con 0x7fd494101450 2024-04-24T08:54:39.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:39.036+0000 7fd49a400700 1 -- 172.21.15.69:0/3355057696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fd494190f10 con 0x7fd494101450 2024-04-24T08:54:39.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:39.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:39.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi069", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:54:39.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:39.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:54:39.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:39.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:39.041+0000 7fd48be00700 1 -- 172.21.15.69:0/3355057696 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fd484031020 con 0x7fd494101450 2024-04-24T08:54:39.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:39.045+0000 7fd49a400700 1 -- 172.21.15.69:0/3355057696 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd4780396b0 msgr2=0x7fd47803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:39.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:39.045+0000 7fd49a400700 1 --2- 172.21.15.69:0/3355057696 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd4780396b0 0x7fd47803bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd480006fd0 tx=0x7fd480006e40 comp rx=0 tx=0).stop 2024-04-24T08:54:39.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:39.045+0000 7fd49a400700 1 -- 172.21.15.69:0/3355057696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd494101450 msgr2=0x7fd494107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:39.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:39.045+0000 7fd49a400700 1 --2- 172.21.15.69:0/3355057696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd494101450 0x7fd494107440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd484005800 tx=0x7fd484009f60 comp rx=0 tx=0).stop 2024-04-24T08:54:39.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:39.045+0000 7fd49a400700 1 -- 172.21.15.69:0/3355057696 shutdown_connections 2024-04-24T08:54:39.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:39.045+0000 7fd49a400700 1 --2- 172.21.15.69:0/3355057696 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd4780396b0 0x7fd47803bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:39.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:39.045+0000 7fd49a400700 1 --2- 172.21.15.69:0/3355057696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd494101450 0x7fd494107440 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:39.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:39.045+0000 7fd49a400700 1 -- 172.21.15.69:0/3355057696 >> 172.21.15.69:0/3355057696 conn(0x7fd4940fc9f0 msgr2=0x7fd4940fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:39.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:39.045+0000 7fd49a400700 1 -- 172.21.15.69:0/3355057696 shutdown_connections 2024-04-24T08:54:39.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:39.045+0000 7fd49a400700 1 -- 172.21.15.69:0/3355057696 wait complete. 2024-04-24T08:54:39.849 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-24T08:54:39.850 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-24T08:54:40.275 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:54:40.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:40 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3355057696' entity='client.admin' 2024-04-24T08:54:40.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:54:40.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:42.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.378+0000 7f601f200700 1 -- 172.21.15.69:0/779368178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018076b20 msgr2=0x7f6018074f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:42.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.378+0000 7f601f200700 1 --2- 172.21.15.69:0/779368178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018076b20 0x7f6018074f80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f600c009910 tx=0x7f600c009c20 comp rx=0 tx=0).stop 2024-04-24T08:54:42.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.378+0000 7f601f200700 1 -- 172.21.15.69:0/779368178 shutdown_connections 2024-04-24T08:54:42.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.378+0000 7f601f200700 1 --2- 172.21.15.69:0/779368178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018076b20 0x7f6018074f80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:42.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.378+0000 7f601f200700 1 -- 172.21.15.69:0/779368178 >> 172.21.15.69:0/779368178 conn(0x7f60180fd9a0 msgr2=0x7f60180ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:42.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.378+0000 7f601f200700 1 -- 172.21.15.69:0/779368178 shutdown_connections 2024-04-24T08:54:42.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.378+0000 7f601f200700 1 -- 172.21.15.69:0/779368178 wait complete. 2024-04-24T08:54:42.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.379+0000 7f601f200700 1 Processor -- start 2024-04-24T08:54:42.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.380+0000 7f601f200700 1 -- start start 2024-04-24T08:54:42.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.380+0000 7f601f200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018076b20 0x7f601819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:42.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.380+0000 7f601f200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f601810e980 con 0x7f6018076b20 2024-04-24T08:54:42.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.380+0000 7f601de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018076b20 0x7f601819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:42.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.380+0000 7f601de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018076b20 0x7f601819e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53810/0 (socket says 172.21.15.69:53810) 2024-04-24T08:54:42.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.380+0000 7f601de00700 1 -- 172.21.15.69:0/2103310002 learned_addr learned my addr 172.21.15.69:0/2103310002 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:42.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.381+0000 7f601de00700 1 -- 172.21.15.69:0/2103310002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601819e550 con 0x7f6018076b20 2024-04-24T08:54:42.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.381+0000 7f601de00700 1 --2- 172.21.15.69:0/2103310002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018076b20 0x7f601819e010 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f600c005950 tx=0x7f600c004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:42.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.381+0000 7f6012a00700 1 -- 172.21.15.69:0/2103310002 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f600c026070 con 0x7f6018076b20 2024-04-24T08:54:42.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.381+0000 7f6012a00700 1 -- 172.21.15.69:0/2103310002 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f600c00bc00 con 0x7f6018076b20 2024-04-24T08:54:42.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.381+0000 7f601f200700 1 -- 172.21.15.69:0/2103310002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601819e7e0 con 0x7f6018076b20 2024-04-24T08:54:42.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.381+0000 7f6012a00700 1 -- 172.21.15.69:0/2103310002 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f600c0039a0 con 0x7f6018076b20 2024-04-24T08:54:42.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.381+0000 7f601f200700 1 -- 172.21.15.69:0/2103310002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60181a14d0 con 0x7f6018076b20 2024-04-24T08:54:42.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.382+0000 7f6012a00700 1 -- 172.21.15.69:0/2103310002 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f600c029070 con 0x7f6018076b20 2024-04-24T08:54:42.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.383+0000 7f601f200700 1 -- 172.21.15.69:0/2103310002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f601810f060 con 0x7f6018076b20 2024-04-24T08:54:42.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.383+0000 7f6012a00700 1 --2- 172.21.15.69:0/2103310002 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f60000396b0 0x7f600003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:42.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.383+0000 7f6012a00700 1 -- 172.21.15.69:0/2103310002 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f600c01e070 con 0x7f6018076b20 2024-04-24T08:54:42.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.383+0000 7f601d400700 1 --2- 172.21.15.69:0/2103310002 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f60000396b0 0x7f600003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:42.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.384+0000 7f601d400700 1 --2- 172.21.15.69:0/2103310002 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f60000396b0 0x7f600003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6008006fd0 tx=0x7f6008006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:42.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.389+0000 7f6012a00700 1 -- 172.21.15.69:0/2103310002 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f600c013070 con 0x7f6018076b20 2024-04-24T08:54:42.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.621+0000 7f601f200700 1 -- 172.21.15.69:0/2103310002 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f601802e0f0 con 0x7f60000396b0 2024-04-24T08:54:42.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.629+0000 7f6012a00700 1 -- 172.21.15.69:0/2103310002 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f601802e0f0 con 0x7f60000396b0 2024-04-24T08:54:42.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.632+0000 7f601f200700 1 -- 172.21.15.69:0/2103310002 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f60000396b0 msgr2=0x7f600003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:42.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.632+0000 7f601f200700 1 --2- 172.21.15.69:0/2103310002 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f60000396b0 0x7f600003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6008006fd0 tx=0x7f6008006e40 comp rx=0 tx=0).stop 2024-04-24T08:54:42.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.632+0000 7f601f200700 1 -- 172.21.15.69:0/2103310002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018076b20 msgr2=0x7f601819e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:42.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.632+0000 7f601f200700 1 --2- 172.21.15.69:0/2103310002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018076b20 0x7f601819e010 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f600c005950 tx=0x7f600c004fa0 comp rx=0 tx=0).stop 2024-04-24T08:54:42.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.632+0000 7f601f200700 1 -- 172.21.15.69:0/2103310002 shutdown_connections 2024-04-24T08:54:42.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.632+0000 7f601f200700 1 --2- 172.21.15.69:0/2103310002 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f60000396b0 0x7f600003bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:42.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.632+0000 7f601f200700 1 --2- 172.21.15.69:0/2103310002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018076b20 0x7f601819e010 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:42.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.632+0000 7f601f200700 1 -- 172.21.15.69:0/2103310002 >> 172.21.15.69:0/2103310002 conn(0x7f60180fd9a0 msgr2=0x7f60180fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:42.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.632+0000 7f601f200700 1 -- 172.21.15.69:0/2103310002 shutdown_connections 2024-04-24T08:54:42.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:42.632+0000 7f601f200700 1 -- 172.21.15.69:0/2103310002 wait complete. 2024-04-24T08:54:43.424 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi081 2024-04-24T08:54:43.424 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-04-24T08:54:43.424 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/etc/ceph/ceph.conf 2024-04-24T08:54:43.481 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-04-24T08:54:43.481 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:54:43.578 INFO:tasks.cephadm:Adding host smithi081 to orchestrator... 2024-04-24T08:54:43.578 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch host add smithi081 2024-04-24T08:54:43.986 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:54:44.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:43 smithi069 ceph-mon[28170]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:54:44.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:44.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:54:44.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:54:44.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:54:44.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:43 smithi069 ceph-mon[28170]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-24T08:54:44.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:43 smithi069 ceph-mon[28170]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:54:44.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:43 smithi069 ceph-mon[28170]: Updating smithi069:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:54:44.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:44 smithi069 ceph-mon[28170]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:54:46.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.165+0000 7f44ea200700 1 -- 172.21.15.69:0/459270196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44e4108580 msgr2=0x7f44e4108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:46.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.165+0000 7f44ea200700 1 --2- 172.21.15.69:0/459270196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44e4108580 0x7f44e4108960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44d400b0a0 tx=0x7f44d400b3b0 comp rx=0 tx=0).stop 2024-04-24T08:54:46.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.165+0000 7f44ea200700 1 -- 172.21.15.69:0/459270196 shutdown_connections 2024-04-24T08:54:46.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.165+0000 7f44ea200700 1 --2- 172.21.15.69:0/459270196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44e4108580 0x7f44e4108960 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:46.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.165+0000 7f44ea200700 1 -- 172.21.15.69:0/459270196 >> 172.21.15.69:0/459270196 conn(0x7f44e4076f50 msgr2=0x7f44e4077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:46.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.165+0000 7f44ea200700 1 -- 172.21.15.69:0/459270196 shutdown_connections 2024-04-24T08:54:46.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.165+0000 7f44ea200700 1 -- 172.21.15.69:0/459270196 wait complete. 2024-04-24T08:54:46.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.166+0000 7f44ea200700 1 Processor -- start 2024-04-24T08:54:46.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.167+0000 7f44ea200700 1 -- start start 2024-04-24T08:54:46.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.167+0000 7f44ea200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44e4108580 0x7f44e4197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:46.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.167+0000 7f44ea200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f44e410c970 con 0x7f44e4108580 2024-04-24T08:54:46.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.167+0000 7f44e3400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44e4108580 0x7f44e4197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:46.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.167+0000 7f44e3400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44e4108580 0x7f44e4197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37378/0 (socket says 172.21.15.69:37378) 2024-04-24T08:54:46.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.167+0000 7f44e3400700 1 -- 172.21.15.69:0/1958387871 learned_addr learned my addr 172.21.15.69:0/1958387871 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:46.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.168+0000 7f44e3400700 1 -- 172.21.15.69:0/1958387871 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e41983e0 con 0x7f44e4108580 2024-04-24T08:54:46.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.168+0000 7f44e3400700 1 --2- 172.21.15.69:0/1958387871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44e4108580 0x7f44e4197ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44d400d010 tx=0x7f44d4009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:46.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.168+0000 7f44e0c00700 1 -- 172.21.15.69:0/1958387871 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44d4017040 con 0x7f44e4108580 2024-04-24T08:54:46.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.168+0000 7f44e0c00700 1 -- 172.21.15.69:0/1958387871 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44d4022070 con 0x7f44e4108580 2024-04-24T08:54:46.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.168+0000 7f44ea200700 1 -- 172.21.15.69:0/1958387871 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e4198670 con 0x7f44e4108580 2024-04-24T08:54:46.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.168+0000 7f44e0c00700 1 -- 172.21.15.69:0/1958387871 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44d401d5e0 con 0x7f44e4108580 2024-04-24T08:54:46.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.169+0000 7f44ea200700 1 -- 172.21.15.69:0/1958387871 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e419c770 con 0x7f44e4108580 2024-04-24T08:54:46.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.170+0000 7f44e0c00700 1 -- 172.21.15.69:0/1958387871 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f44d4004040 con 0x7f44e4108580 2024-04-24T08:54:46.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.170+0000 7f44ea200700 1 -- 172.21.15.69:0/1958387871 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e4103c40 con 0x7f44e4108580 2024-04-24T08:54:46.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.170+0000 7f44e0c00700 1 --2- 172.21.15.69:0/1958387871 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f44c80396a0 0x7f44c803bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:46.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.170+0000 7f44e0c00700 1 -- 172.21.15.69:0/1958387871 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f44d400f050 con 0x7f44e4108580 2024-04-24T08:54:46.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.171+0000 7f44daa00700 1 --2- 172.21.15.69:0/1958387871 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f44c80396a0 0x7f44c803bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:46.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.171+0000 7f44daa00700 1 --2- 172.21.15.69:0/1958387871 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f44c80396a0 0x7f44c803bb60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f44cc006fd0 tx=0x7f44cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:46.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.176+0000 7f44e0c00700 1 -- 172.21.15.69:0/1958387871 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44d401cc10 con 0x7f44e4108580 2024-04-24T08:54:46.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:46 smithi069 ceph-mon[28170]: Updating smithi069:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:54:46.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:46 smithi069 ceph-mon[28170]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:54:46.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:46.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:46.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:46.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:46.406+0000 7f44ea200700 1 -- 172.21.15.69:0/1958387871 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi081", "target": ["mon-mgr", ""]}) -- 0x7f44e4064150 con 0x7f44c80396a0 2024-04-24T08:54:47.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:47 smithi069 ceph-mon[28170]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi081", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:54:48.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:48 smithi069 ceph-mon[28170]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:54:49.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:49 smithi069 ceph-mon[28170]: Deploying cephadm binary to smithi081 2024-04-24T08:54:50.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:50 smithi069 ceph-mon[28170]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:54:52.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:52.529+0000 7f44e0c00700 1 -- 172.21.15.69:0/1958387871 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f44e4064150 con 0x7f44c80396a0 2024-04-24T08:54:52.531 INFO:teuthology.orchestra.run.smithi069.stdout:Added host 'smithi081' with addr '172.21.15.81' 2024-04-24T08:54:52.531 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:52 smithi069 ceph-mon[28170]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:54:52.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:52.533+0000 7f44ea200700 1 -- 172.21.15.69:0/1958387871 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f44c80396a0 msgr2=0x7f44c803bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:52.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:52.533+0000 7f44ea200700 1 --2- 172.21.15.69:0/1958387871 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f44c80396a0 0x7f44c803bb60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f44cc006fd0 tx=0x7f44cc006e40 comp rx=0 tx=0).stop 2024-04-24T08:54:52.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:52.533+0000 7f44ea200700 1 -- 172.21.15.69:0/1958387871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44e4108580 msgr2=0x7f44e4197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:52.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:52.533+0000 7f44ea200700 1 --2- 172.21.15.69:0/1958387871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44e4108580 0x7f44e4197ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44d400d010 tx=0x7f44d4009790 comp rx=0 tx=0).stop 2024-04-24T08:54:52.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:52.534+0000 7f44ea200700 1 -- 172.21.15.69:0/1958387871 shutdown_connections 2024-04-24T08:54:52.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:52.534+0000 7f44ea200700 1 --2- 172.21.15.69:0/1958387871 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f44c80396a0 0x7f44c803bb60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:52.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:52.534+0000 7f44ea200700 1 --2- 172.21.15.69:0/1958387871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44e4108580 0x7f44e4197ea0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:52.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:52.534+0000 7f44ea200700 1 -- 172.21.15.69:0/1958387871 >> 172.21.15.69:0/1958387871 conn(0x7f44e4076f50 msgr2=0x7f44e40fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:52.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:52.534+0000 7f44ea200700 1 -- 172.21.15.69:0/1958387871 shutdown_connections 2024-04-24T08:54:52.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:52.534+0000 7f44ea200700 1 -- 172.21.15.69:0/1958387871 wait complete. 2024-04-24T08:54:53.262 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T08:54:53.678 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:54:53.823 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:53.823 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:53 smithi069 ceph-mon[28170]: Added host smithi081 2024-04-24T08:54:53.823 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:54:53.823 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:53 smithi069 ceph-mon[28170]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:54:53.824 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:55.496 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:55 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:55.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.771+0000 7f2d7a400700 1 -- 172.21.15.69:0/453075372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741085d0 msgr2=0x7f2d741089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:55.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.771+0000 7f2d7a400700 1 --2- 172.21.15.69:0/453075372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741085d0 0x7f2d741089b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400b0a0 tx=0x7f2d6400b3b0 comp rx=0 tx=0).stop 2024-04-24T08:54:55.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.771+0000 7f2d7a400700 1 -- 172.21.15.69:0/453075372 shutdown_connections 2024-04-24T08:54:55.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.771+0000 7f2d7a400700 1 --2- 172.21.15.69:0/453075372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741085d0 0x7f2d741089b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:55.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.771+0000 7f2d7a400700 1 -- 172.21.15.69:0/453075372 >> 172.21.15.69:0/453075372 conn(0x7f2d740fe030 msgr2=0x7f2d74100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:55.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.772+0000 7f2d7a400700 1 -- 172.21.15.69:0/453075372 shutdown_connections 2024-04-24T08:54:55.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.772+0000 7f2d7a400700 1 -- 172.21.15.69:0/453075372 wait complete. 2024-04-24T08:54:55.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.773+0000 7f2d7a400700 1 Processor -- start 2024-04-24T08:54:55.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.773+0000 7f2d7a400700 1 -- start start 2024-04-24T08:54:55.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.774+0000 7f2d7a400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741085d0 0x7f2d7419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:55.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.774+0000 7f2d7a400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7410ca40 con 0x7f2d741085d0 2024-04-24T08:54:55.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.774+0000 7f2d73400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741085d0 0x7f2d7419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:55.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.774+0000 7f2d73400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741085d0 0x7f2d7419a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38344/0 (socket says 172.21.15.69:38344) 2024-04-24T08:54:55.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.774+0000 7f2d73400700 1 -- 172.21.15.69:0/1847308452 learned_addr learned my addr 172.21.15.69:0/1847308452 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:55.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.774+0000 7f2d73400700 1 -- 172.21.15.69:0/1847308452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7419a640 con 0x7f2d741085d0 2024-04-24T08:54:55.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.775+0000 7f2d73400700 1 --2- 172.21.15.69:0/1847308452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741085d0 0x7f2d7419a100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2d64015040 tx=0x7f2d64003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:55.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.775+0000 7f2d6be00700 1 -- 172.21.15.69:0/1847308452 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d64017040 con 0x7f2d741085d0 2024-04-24T08:54:55.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.775+0000 7f2d6be00700 1 -- 172.21.15.69:0/1847308452 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d64022070 con 0x7f2d741085d0 2024-04-24T08:54:55.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.775+0000 7f2d7a400700 1 -- 172.21.15.69:0/1847308452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7419a8d0 con 0x7f2d741085d0 2024-04-24T08:54:55.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.775+0000 7f2d6be00700 1 -- 172.21.15.69:0/1847308452 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d6401d5e0 con 0x7f2d741085d0 2024-04-24T08:54:55.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.775+0000 7f2d7a400700 1 -- 172.21.15.69:0/1847308452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7419ad30 con 0x7f2d741085d0 2024-04-24T08:54:55.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.776+0000 7f2d6be00700 1 -- 172.21.15.69:0/1847308452 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2d64004040 con 0x7f2d741085d0 2024-04-24T08:54:55.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.776+0000 7f2d7a400700 1 -- 172.21.15.69:0/1847308452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d74103c40 con 0x7f2d741085d0 2024-04-24T08:54:55.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.777+0000 7f2d6be00700 1 --2- 172.21.15.69:0/1847308452 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2d580396f0 0x7f2d5803bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:55.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.777+0000 7f2d6be00700 1 -- 172.21.15.69:0/1847308452 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2d6400f050 con 0x7f2d741085d0 2024-04-24T08:54:55.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.777+0000 7f2d72a00700 1 --2- 172.21.15.69:0/1847308452 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2d580396f0 0x7f2d5803bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:55.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.778+0000 7f2d72a00700 1 --2- 172.21.15.69:0/1847308452 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2d580396f0 0x7f2d5803bbb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2d60006fd0 tx=0x7f2d60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:55.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:55.783+0000 7f2d6be00700 1 -- 172.21.15.69:0/1847308452 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d6401cc10 con 0x7f2d741085d0 2024-04-24T08:54:56.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:56.016+0000 7f2d7a400700 1 -- 172.21.15.69:0/1847308452 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d7419b4d0 con 0x7f2d580396f0 2024-04-24T08:54:56.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:56.019+0000 7f2d6be00700 1 -- 172.21.15.69:0/1847308452 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f2d7419b4d0 con 0x7f2d580396f0 2024-04-24T08:54:56.018 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:54:56.018 INFO:teuthology.orchestra.run.smithi069.stdout:[{"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}, {"addr": "172.21.15.81", "hostname": "smithi081", "labels": [], "status": ""}] 2024-04-24T08:54:56.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:56.022+0000 7f2d7a400700 1 -- 172.21.15.69:0/1847308452 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2d580396f0 msgr2=0x7f2d5803bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:56.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:56.022+0000 7f2d7a400700 1 --2- 172.21.15.69:0/1847308452 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2d580396f0 0x7f2d5803bbb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2d60006fd0 tx=0x7f2d60006e40 comp rx=0 tx=0).stop 2024-04-24T08:54:56.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:56.022+0000 7f2d7a400700 1 -- 172.21.15.69:0/1847308452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741085d0 msgr2=0x7f2d7419a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:56.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:56.022+0000 7f2d7a400700 1 --2- 172.21.15.69:0/1847308452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741085d0 0x7f2d7419a100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2d64015040 tx=0x7f2d64003ce0 comp rx=0 tx=0).stop 2024-04-24T08:54:56.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:56.022+0000 7f2d7a400700 1 -- 172.21.15.69:0/1847308452 shutdown_connections 2024-04-24T08:54:56.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:56.022+0000 7f2d7a400700 1 --2- 172.21.15.69:0/1847308452 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2d580396f0 0x7f2d5803bbb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:56.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:56.022+0000 7f2d7a400700 1 --2- 172.21.15.69:0/1847308452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741085d0 0x7f2d7419a100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:56.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:56.022+0000 7f2d7a400700 1 -- 172.21.15.69:0/1847308452 >> 172.21.15.69:0/1847308452 conn(0x7f2d740fe030 msgr2=0x7f2d74076710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:56.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:56.022+0000 7f2d7a400700 1 -- 172.21.15.69:0/1847308452 shutdown_connections 2024-04-24T08:54:56.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:56.022+0000 7f2d7a400700 1 -- 172.21.15.69:0/1847308452 wait complete. 2024-04-24T08:54:56.339 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:56 smithi069 ceph-mon[28170]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:54:56.635 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi130 2024-04-24T08:54:56.636 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-24T08:54:56.636 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/etc/ceph/ceph.conf 2024-04-24T08:54:56.694 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-24T08:54:56.694 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:54:56.790 INFO:tasks.cephadm:Adding host smithi130 to orchestrator... 2024-04-24T08:54:56.791 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch host add smithi130 2024-04-24T08:54:57.218 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:54:57.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:57 smithi069 ceph-mon[28170]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:54:57.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:57 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:54:58.458 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:54:58 smithi069 ceph-mon[28170]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:54:59.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.212+0000 7fd052400700 1 -- 172.21.15.69:0/1371781942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd04c1072c0 msgr2=0x7fd04c1076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:59.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.212+0000 7fd052400700 1 --2- 172.21.15.69:0/1371781942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd04c1072c0 0x7fd04c1076a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd040009910 tx=0x7fd040009c20 comp rx=0 tx=0).stop 2024-04-24T08:54:59.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.213+0000 7fd052400700 1 -- 172.21.15.69:0/1371781942 shutdown_connections 2024-04-24T08:54:59.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.213+0000 7fd052400700 1 --2- 172.21.15.69:0/1371781942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd04c1072c0 0x7fd04c1076a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:59.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.213+0000 7fd052400700 1 -- 172.21.15.69:0/1371781942 >> 172.21.15.69:0/1371781942 conn(0x7fd04c0fc4b0 msgr2=0x7fd04c0fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:59.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.213+0000 7fd052400700 1 -- 172.21.15.69:0/1371781942 shutdown_connections 2024-04-24T08:54:59.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.213+0000 7fd052400700 1 -- 172.21.15.69:0/1371781942 wait complete. 2024-04-24T08:54:59.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.214+0000 7fd052400700 1 Processor -- start 2024-04-24T08:54:59.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.214+0000 7fd052400700 1 -- start start 2024-04-24T08:54:59.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.214+0000 7fd052400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd04c1072c0 0x7fd04c19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:59.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.214+0000 7fd052400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c112e70 con 0x7fd04c1072c0 2024-04-24T08:54:59.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.215+0000 7fd051000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd04c1072c0 0x7fd04c19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:59.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.215+0000 7fd051000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd04c1072c0 0x7fd04c19e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38364/0 (socket says 172.21.15.69:38364) 2024-04-24T08:54:59.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.215+0000 7fd051000700 1 -- 172.21.15.69:0/3720744034 learned_addr learned my addr 172.21.15.69:0/3720744034 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:54:59.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.215+0000 7fd051000700 1 -- 172.21.15.69:0/3720744034 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c19e8a0 con 0x7fd04c1072c0 2024-04-24T08:54:59.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.215+0000 7fd051000700 1 --2- 172.21.15.69:0/3720744034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd04c1072c0 0x7fd04c19e360 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd0400056d0 tx=0x7fd0400057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:59.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.216+0000 7fd049600700 1 -- 172.21.15.69:0/3720744034 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd040026070 con 0x7fd04c1072c0 2024-04-24T08:54:59.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.216+0000 7fd049600700 1 -- 172.21.15.69:0/3720744034 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd040004370 con 0x7fd04c1072c0 2024-04-24T08:54:59.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.216+0000 7fd052400700 1 -- 172.21.15.69:0/3720744034 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c19eb30 con 0x7fd04c1072c0 2024-04-24T08:54:59.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.216+0000 7fd049600700 1 -- 172.21.15.69:0/3720744034 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0400039b0 con 0x7fd04c1072c0 2024-04-24T08:54:59.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.216+0000 7fd052400700 1 -- 172.21.15.69:0/3720744034 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c1a2c30 con 0x7fd04c1072c0 2024-04-24T08:54:59.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.217+0000 7fd052400700 1 -- 172.21.15.69:0/3720744034 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04c1019b0 con 0x7fd04c1072c0 2024-04-24T08:54:59.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.217+0000 7fd049600700 1 -- 172.21.15.69:0/3720744034 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd040029070 con 0x7fd04c1072c0 2024-04-24T08:54:59.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.218+0000 7fd049600700 1 --2- 172.21.15.69:0/3720744034 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd0340396b0 0x7fd03403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:59.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.218+0000 7fd049600700 1 -- 172.21.15.69:0/3720744034 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd04001e070 con 0x7fd04c1072c0 2024-04-24T08:54:59.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.218+0000 7fd04be00700 1 --2- 172.21.15.69:0/3720744034 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd0340396b0 0x7fd03403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:59.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.219+0000 7fd04be00700 1 --2- 172.21.15.69:0/3720744034 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd0340396b0 0x7fd03403bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd038006fd0 tx=0x7fd038006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:59.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.223+0000 7fd049600700 1 -- 172.21.15.69:0/3720744034 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd040013070 con 0x7fd04c1072c0 2024-04-24T08:54:59.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:54:59.458+0000 7fd052400700 1 -- 172.21.15.69:0/3720744034 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi130", "target": ["mon-mgr", ""]}) -- 0x7fd04c064150 con 0x7fd0340396b0 2024-04-24T08:55:00.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:00 smithi069 ceph-mon[28170]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:01.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:01 smithi069 ceph-mon[28170]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi130", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:55:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:02 smithi069 ceph-mon[28170]: Deploying cephadm binary to smithi130 2024-04-24T08:55:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:02 smithi069 ceph-mon[28170]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:02.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:02.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:02.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:02.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi081", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:55:02.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:02.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:55:02.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:02 smithi069 ceph-mon[28170]: Updating smithi081:/etc/ceph/ceph.conf 2024-04-24T08:55:04.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:04 smithi069 ceph-mon[28170]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:05.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:05.502+0000 7fd049600700 1 -- 172.21.15.69:0/3720744034 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd04c064150 con 0x7fd0340396b0 2024-04-24T08:55:05.503 INFO:teuthology.orchestra.run.smithi069.stdout:Added host 'smithi130' with addr '172.21.15.130' 2024-04-24T08:55:05.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:05.506+0000 7fd052400700 1 -- 172.21.15.69:0/3720744034 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd0340396b0 msgr2=0x7fd03403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:05.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:05.506+0000 7fd052400700 1 --2- 172.21.15.69:0/3720744034 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd0340396b0 0x7fd03403bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd038006fd0 tx=0x7fd038006e40 comp rx=0 tx=0).stop 2024-04-24T08:55:05.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:05.506+0000 7fd052400700 1 -- 172.21.15.69:0/3720744034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd04c1072c0 msgr2=0x7fd04c19e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:05.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:05.506+0000 7fd052400700 1 --2- 172.21.15.69:0/3720744034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd04c1072c0 0x7fd04c19e360 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd0400056d0 tx=0x7fd0400057b0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:05.507+0000 7fd052400700 1 -- 172.21.15.69:0/3720744034 shutdown_connections 2024-04-24T08:55:05.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:05.507+0000 7fd052400700 1 --2- 172.21.15.69:0/3720744034 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd0340396b0 0x7fd03403bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:05.507+0000 7fd052400700 1 --2- 172.21.15.69:0/3720744034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd04c1072c0 0x7fd04c19e360 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:05.507+0000 7fd052400700 1 -- 172.21.15.69:0/3720744034 >> 172.21.15.69:0/3720744034 conn(0x7fd04c0fc4b0 msgr2=0x7fd04c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:05.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:05.507+0000 7fd052400700 1 -- 172.21.15.69:0/3720744034 shutdown_connections 2024-04-24T08:55:05.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:05.507+0000 7fd052400700 1 -- 172.21.15.69:0/3720744034 wait complete. 2024-04-24T08:55:06.268 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T08:55:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:06 smithi069 ceph-mon[28170]: Updating smithi081:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:55:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:06 smithi069 ceph-mon[28170]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:06 smithi069 ceph-mon[28170]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:55:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:06.688 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:55:07.651 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:07 smithi069 ceph-mon[28170]: Added host smithi130 2024-04-24T08:55:07.651 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:07 smithi069 ceph-mon[28170]: Updating smithi081:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:55:07.651 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:07 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:07.651 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:07 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:07.651 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:07 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:07.651 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:07 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:55:08.573 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:08 smithi069 ceph-mon[28170]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:08.573 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:08.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.738+0000 7f06c1400700 1 -- 172.21.15.69:0/1715865787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06bc10daa0 msgr2=0x7f06bc10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:08.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.738+0000 7f06c1400700 1 --2- 172.21.15.69:0/1715865787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06bc10daa0 0x7f06bc10de80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009910 tx=0x7f06ac009c20 comp rx=0 tx=0).stop 2024-04-24T08:55:08.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.738+0000 7f06c1400700 1 -- 172.21.15.69:0/1715865787 shutdown_connections 2024-04-24T08:55:08.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.738+0000 7f06c1400700 1 --2- 172.21.15.69:0/1715865787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06bc10daa0 0x7f06bc10de80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.739+0000 7f06c1400700 1 -- 172.21.15.69:0/1715865787 >> 172.21.15.69:0/1715865787 conn(0x7f06bc078290 msgr2=0x7f06bc07a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:08.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.739+0000 7f06c1400700 1 -- 172.21.15.69:0/1715865787 shutdown_connections 2024-04-24T08:55:08.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.739+0000 7f06c1400700 1 -- 172.21.15.69:0/1715865787 wait complete. 2024-04-24T08:55:08.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.740+0000 7f06c1400700 1 Processor -- start 2024-04-24T08:55:08.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.740+0000 7f06c1400700 1 -- start start 2024-04-24T08:55:08.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.740+0000 7f06c1400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06bc10daa0 0x7f06bc19e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.740+0000 7f06c1400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc113010 con 0x7f06bc10daa0 2024-04-24T08:55:08.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.740+0000 7f06ba800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06bc10daa0 0x7f06bc19e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.740+0000 7f06ba800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06bc10daa0 0x7f06bc19e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51428/0 (socket says 172.21.15.69:51428) 2024-04-24T08:55:08.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.740+0000 7f06ba800700 1 -- 172.21.15.69:0/2831153566 learned_addr learned my addr 172.21.15.69:0/2831153566 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:55:08.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.741+0000 7f06ba800700 1 -- 172.21.15.69:0/2831153566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06bc19ea80 con 0x7f06bc10daa0 2024-04-24T08:55:08.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.741+0000 7f06ba800700 1 --2- 172.21.15.69:0/2831153566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06bc10daa0 0x7f06bc19e540 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f06ac006010 tx=0x7f06ac005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:08.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.741+0000 7f06b3400700 1 -- 172.21.15.69:0/2831153566 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06ac026070 con 0x7f06bc10daa0 2024-04-24T08:55:08.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.741+0000 7f06b3400700 1 -- 172.21.15.69:0/2831153566 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06ac00bba0 con 0x7f06bc10daa0 2024-04-24T08:55:08.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.741+0000 7f06c1400700 1 -- 172.21.15.69:0/2831153566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06bc19ed70 con 0x7f06bc10daa0 2024-04-24T08:55:08.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.741+0000 7f06b3400700 1 -- 172.21.15.69:0/2831153566 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06ac003910 con 0x7f06bc10daa0 2024-04-24T08:55:08.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.742+0000 7f06c1400700 1 -- 172.21.15.69:0/2831153566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06bc1a2e10 con 0x7f06bc10daa0 2024-04-24T08:55:08.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.743+0000 7f06c1400700 1 -- 172.21.15.69:0/2831153566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06bc10a1f0 con 0x7f06bc10daa0 2024-04-24T08:55:08.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.743+0000 7f06b3400700 1 -- 172.21.15.69:0/2831153566 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f06ac029070 con 0x7f06bc10daa0 2024-04-24T08:55:08.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.743+0000 7f06b3400700 1 --2- 172.21.15.69:0/2831153566 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f06a00396b0 0x7f06a003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.743+0000 7f06b3400700 1 -- 172.21.15.69:0/2831153566 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f06ac01e070 con 0x7f06bc10daa0 2024-04-24T08:55:08.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.744+0000 7f06b9e00700 1 --2- 172.21.15.69:0/2831153566 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f06a00396b0 0x7f06a003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.744+0000 7f06b9e00700 1 --2- 172.21.15.69:0/2831153566 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f06a00396b0 0x7f06a003bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f06a4006fd0 tx=0x7f06a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:08.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.749+0000 7f06b3400700 1 -- 172.21.15.69:0/2831153566 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06ac013070 con 0x7f06bc10daa0 2024-04-24T08:55:08.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.979+0000 7f06c1400700 1 -- 172.21.15.69:0/2831153566 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f06bc19f8a0 con 0x7f06a00396b0 2024-04-24T08:55:08.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.982+0000 7f06b3400700 1 -- 172.21.15.69:0/2831153566 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f06bc19f8a0 con 0x7f06a00396b0 2024-04-24T08:55:08.982 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T08:55:08.982 INFO:teuthology.orchestra.run.smithi069.stdout:[{"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}, {"addr": "172.21.15.81", "hostname": "smithi081", "labels": [], "status": ""}, {"addr": "172.21.15.130", "hostname": "smithi130", "labels": [], "status": ""}] 2024-04-24T08:55:08.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.985+0000 7f06c1400700 1 -- 172.21.15.69:0/2831153566 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f06a00396b0 msgr2=0x7f06a003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:08.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.985+0000 7f06c1400700 1 --2- 172.21.15.69:0/2831153566 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f06a00396b0 0x7f06a003bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f06a4006fd0 tx=0x7f06a4006e40 comp rx=0 tx=0).stop 2024-04-24T08:55:08.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.985+0000 7f06c1400700 1 -- 172.21.15.69:0/2831153566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06bc10daa0 msgr2=0x7f06bc19e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:08.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.985+0000 7f06c1400700 1 --2- 172.21.15.69:0/2831153566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06bc10daa0 0x7f06bc19e540 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f06ac006010 tx=0x7f06ac005780 comp rx=0 tx=0).stop 2024-04-24T08:55:08.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.985+0000 7f06c1400700 1 -- 172.21.15.69:0/2831153566 shutdown_connections 2024-04-24T08:55:08.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.985+0000 7f06c1400700 1 --2- 172.21.15.69:0/2831153566 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f06a00396b0 0x7f06a003bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.985+0000 7f06c1400700 1 --2- 172.21.15.69:0/2831153566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06bc10daa0 0x7f06bc19e540 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.985+0000 7f06c1400700 1 -- 172.21.15.69:0/2831153566 >> 172.21.15.69:0/2831153566 conn(0x7f06bc078290 msgr2=0x7f06bc110640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:08.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.985+0000 7f06c1400700 1 -- 172.21.15.69:0/2831153566 shutdown_connections 2024-04-24T08:55:08.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:08.985+0000 7f06c1400700 1 -- 172.21.15.69:0/2831153566 wait complete. 2024-04-24T08:55:09.675 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-24T08:55:09.676 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-24T08:55:10.107 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:55:10.397 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:10 smithi069 ceph-mon[28170]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:55:10.397 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:10 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:10.397 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:10 smithi069 ceph-mon[28170]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:12.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.103+0000 7f3c29800700 1 -- 172.21.15.69:0/4253345042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c24074e90 msgr2=0x7f3c24075270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:12.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.103+0000 7f3c29800700 1 --2- 172.21.15.69:0/4253345042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c24074e90 0x7f3c24075270 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009910 tx=0x7f3c14009c20 comp rx=0 tx=0).stop 2024-04-24T08:55:12.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.104+0000 7f3c29800700 1 -- 172.21.15.69:0/4253345042 shutdown_connections 2024-04-24T08:55:12.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.104+0000 7f3c29800700 1 --2- 172.21.15.69:0/4253345042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c24074e90 0x7f3c24075270 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:12.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.104+0000 7f3c29800700 1 -- 172.21.15.69:0/4253345042 >> 172.21.15.69:0/4253345042 conn(0x7f3c240fde50 msgr2=0x7f3c24100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:12.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.104+0000 7f3c29800700 1 -- 172.21.15.69:0/4253345042 shutdown_connections 2024-04-24T08:55:12.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.104+0000 7f3c29800700 1 -- 172.21.15.69:0/4253345042 wait complete. 2024-04-24T08:55:12.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.105+0000 7f3c29800700 1 Processor -- start 2024-04-24T08:55:12.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.105+0000 7f3c29800700 1 -- start start 2024-04-24T08:55:12.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.105+0000 7f3c29800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c24074e90 0x7f3c2419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:12.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.105+0000 7f3c29800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3c24112e90 con 0x7f3c24074e90 2024-04-24T08:55:12.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.106+0000 7f3c23e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c24074e90 0x7f3c2419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:12.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.106+0000 7f3c23e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c24074e90 0x7f3c2419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51448/0 (socket says 172.21.15.69:51448) 2024-04-24T08:55:12.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.106+0000 7f3c23e00700 1 -- 172.21.15.69:0/3913175972 learned_addr learned my addr 172.21.15.69:0/3913175972 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:55:12.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.106+0000 7f3c23e00700 1 -- 172.21.15.69:0/3913175972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2419e960 con 0x7f3c24074e90 2024-04-24T08:55:12.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.106+0000 7f3c23e00700 1 --2- 172.21.15.69:0/3913175972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c24074e90 0x7f3c2419e420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3c140056d0 tx=0x7f3c140057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:12.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.107+0000 7f3c20c00700 1 -- 172.21.15.69:0/3913175972 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c14026070 con 0x7f3c24074e90 2024-04-24T08:55:12.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.107+0000 7f3c20c00700 1 -- 172.21.15.69:0/3913175972 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c14004370 con 0x7f3c24074e90 2024-04-24T08:55:12.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.107+0000 7f3c20c00700 1 -- 172.21.15.69:0/3913175972 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c140039b0 con 0x7f3c24074e90 2024-04-24T08:55:12.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.107+0000 7f3c29800700 1 -- 172.21.15.69:0/3913175972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2419ebf0 con 0x7f3c24074e90 2024-04-24T08:55:12.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.107+0000 7f3c29800700 1 -- 172.21.15.69:0/3913175972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2419f050 con 0x7f3c24074e90 2024-04-24T08:55:12.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.108+0000 7f3c29800700 1 -- 172.21.15.69:0/3913175972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c240765d0 con 0x7f3c24074e90 2024-04-24T08:55:12.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.108+0000 7f3c20c00700 1 -- 172.21.15.69:0/3913175972 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3c14029070 con 0x7f3c24074e90 2024-04-24T08:55:12.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.109+0000 7f3c20c00700 1 --2- 172.21.15.69:0/3913175972 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3c080396b0 0x7f3c0803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:12.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.109+0000 7f3c20c00700 1 -- 172.21.15.69:0/3913175972 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3c1401e070 con 0x7f3c24074e90 2024-04-24T08:55:12.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.109+0000 7f3c23400700 1 --2- 172.21.15.69:0/3913175972 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3c080396b0 0x7f3c0803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:12.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.110+0000 7f3c23400700 1 --2- 172.21.15.69:0/3913175972 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3c080396b0 0x7f3c0803bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3c10006fd0 tx=0x7f3c10006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:12.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.114+0000 7f3c20c00700 1 -- 172.21.15.69:0/3913175972 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c14013070 con 0x7f3c24074e90 2024-04-24T08:55:12.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:12.349+0000 7f3c29800700 1 -- 172.21.15.69:0/3913175972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3c2419f780 con 0x7f3c24074e90 2024-04-24T08:55:12.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:12 smithi069 ceph-mon[28170]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:12.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:12 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:13.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:13.233+0000 7f3c20c00700 1 -- 172.21.15.69:0/3913175972 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f3c14013c90 con 0x7f3c24074e90 2024-04-24T08:55:13.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:13.237+0000 7f3c29800700 1 -- 172.21.15.69:0/3913175972 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3c080396b0 msgr2=0x7f3c0803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:13.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:13.237+0000 7f3c29800700 1 --2- 172.21.15.69:0/3913175972 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3c080396b0 0x7f3c0803bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3c10006fd0 tx=0x7f3c10006e40 comp rx=0 tx=0).stop 2024-04-24T08:55:13.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:13.237+0000 7f3c29800700 1 -- 172.21.15.69:0/3913175972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c24074e90 msgr2=0x7f3c2419e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:13.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:13.237+0000 7f3c29800700 1 --2- 172.21.15.69:0/3913175972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c24074e90 0x7f3c2419e420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3c140056d0 tx=0x7f3c140057b0 comp rx=0 tx=0).stop 2024-04-24T08:55:13.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:13.237+0000 7f3c29800700 1 -- 172.21.15.69:0/3913175972 shutdown_connections 2024-04-24T08:55:13.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:13.237+0000 7f3c29800700 1 --2- 172.21.15.69:0/3913175972 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3c080396b0 0x7f3c0803bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:13.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:13.237+0000 7f3c29800700 1 --2- 172.21.15.69:0/3913175972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c24074e90 0x7f3c2419e420 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:13.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:13.237+0000 7f3c29800700 1 -- 172.21.15.69:0/3913175972 >> 172.21.15.69:0/3913175972 conn(0x7f3c240fde50 msgr2=0x7f3c240fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:13.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:13.237+0000 7f3c29800700 1 -- 172.21.15.69:0/3913175972 shutdown_connections 2024-04-24T08:55:13.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:13.238+0000 7f3c29800700 1 -- 172.21.15.69:0/3913175972 wait complete. 2024-04-24T08:55:13.239 INFO:teuthology.orchestra.run.smithi069.stderr:adjusted tunables profile to default 2024-04-24T08:55:13.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:13 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3913175972' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T08:55:13.969 INFO:tasks.cephadm:Adding mon.a on smithi069 2024-04-24T08:55:13.970 INFO:tasks.cephadm:Adding mon.b on smithi081 2024-04-24T08:55:13.970 INFO:tasks.cephadm:Adding mon.c on smithi130 2024-04-24T08:55:13.970 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi069:172.21.15.69=a;smithi081:172.21.15.81=b;smithi130:172.21.15.130=c' 2024-04-24T08:55:14.397 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:55:14.511 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:55:14.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:14 smithi069 ceph-mon[28170]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:14.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:14 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3913175972' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T08:55:14.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:14 smithi069 ceph-mon[28170]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:55:16.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:16 smithi069 ceph-mon[28170]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:17.047 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.047+0000 7f4dc7200700 1 -- 172.21.15.130:0/1215195235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0102550 msgr2=0x7f4dc0102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:17.047 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.047+0000 7f4dc7200700 1 --2- 172.21.15.130:0/1215195235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0102550 0x7f4dc0102970 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4db0009910 tx=0x7f4db0009c20 comp rx=0 tx=0).stop 2024-04-24T08:55:17.047 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.047+0000 7f4dc7200700 1 -- 172.21.15.130:0/1215195235 shutdown_connections 2024-04-24T08:55:17.047 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.047+0000 7f4dc7200700 1 --2- 172.21.15.130:0/1215195235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0102550 0x7f4dc0102970 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:17.047 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.047+0000 7f4dc7200700 1 -- 172.21.15.130:0/1215195235 >> 172.21.15.130:0/1215195235 conn(0x7f4dc00fdaf0 msgr2=0x7f4dc00fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:17.047 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.047+0000 7f4dc7200700 1 -- 172.21.15.130:0/1215195235 shutdown_connections 2024-04-24T08:55:17.047 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.047+0000 7f4dc7200700 1 -- 172.21.15.130:0/1215195235 wait complete. 2024-04-24T08:55:17.048 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.048+0000 7f4dc7200700 1 Processor -- start 2024-04-24T08:55:17.048 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.048+0000 7f4dc7200700 1 -- start start 2024-04-24T08:55:17.048 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.048+0000 7f4dc7200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0102550 0x7f4dc0199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:17.048 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.048+0000 7f4dc7200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc0108490 con 0x7f4dc0102550 2024-04-24T08:55:17.049 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.049+0000 7f4dbfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0102550 0x7f4dc0199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:17.049 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.049+0000 7f4dbfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0102550 0x7f4dc0199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.130:49878/0 (socket says 172.21.15.130:49878) 2024-04-24T08:55:17.049 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.049+0000 7f4dbfe00700 1 -- 172.21.15.130:0/2880651071 learned_addr learned my addr 172.21.15.130:0/2880651071 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-24T08:55:17.049 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.050+0000 7f4dbfe00700 1 -- 172.21.15.130:0/2880651071 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc019a2f0 con 0x7f4dc0102550 2024-04-24T08:55:17.049 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.050+0000 7f4dbfe00700 1 --2- 172.21.15.130:0/2880651071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0102550 0x7f4dc0199db0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4db0005950 tx=0x7f4db0004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:17.050 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.050+0000 7f4dbcc00700 1 -- 172.21.15.130:0/2880651071 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4db0026070 con 0x7f4dc0102550 2024-04-24T08:55:17.050 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.051+0000 7f4dbcc00700 1 -- 172.21.15.130:0/2880651071 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4db000bc00 con 0x7f4dc0102550 2024-04-24T08:55:17.050 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.051+0000 7f4dbcc00700 1 -- 172.21.15.130:0/2880651071 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4db00039a0 con 0x7f4dc0102550 2024-04-24T08:55:17.050 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.051+0000 7f4dc7200700 1 -- 172.21.15.130:0/2880651071 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc019a580 con 0x7f4dc0102550 2024-04-24T08:55:17.051 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.051+0000 7f4dc7200700 1 -- 172.21.15.130:0/2880651071 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc019d270 con 0x7f4dc0102550 2024-04-24T08:55:17.051 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.052+0000 7f4dc7200700 1 -- 172.21.15.130:0/2880651071 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc0108c10 con 0x7f4dc0102550 2024-04-24T08:55:17.052 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.052+0000 7f4dbcc00700 1 -- 172.21.15.130:0/2880651071 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4db0029070 con 0x7f4dc0102550 2024-04-24T08:55:17.052 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.053+0000 7f4dbcc00700 1 --2- 172.21.15.130:0/2880651071 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4da40396b0 0x7f4da403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:17.053 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.053+0000 7f4dbcc00700 1 -- 172.21.15.130:0/2880651071 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4db001e070 con 0x7f4dc0102550 2024-04-24T08:55:17.053 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.054+0000 7f4dbf400700 1 --2- 172.21.15.130:0/2880651071 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4da40396b0 0x7f4da403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:17.054 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.055+0000 7f4dbf400700 1 --2- 172.21.15.130:0/2880651071 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4da40396b0 0x7f4da403bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4dac006fd0 tx=0x7f4dac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:17.068 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.069+0000 7f4dbcc00700 1 -- 172.21.15.130:0/2880651071 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4db0013070 con 0x7f4dc0102550 2024-04-24T08:55:17.310 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.310+0000 7f4dc7200700 1 -- 172.21.15.130:0/2880651071 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi081:172.21.15.81=b;smithi130:172.21.15.130=c", "target": ["mon-mgr", ""]}) -- 0x7f4dc002e270 con 0x7f4da40396b0 2024-04-24T08:55:17.326 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.326+0000 7f4dbcc00700 1 -- 172.21.15.130:0/2880651071 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4dc002e270 con 0x7f4da40396b0 2024-04-24T08:55:17.326 INFO:teuthology.orchestra.run.smithi130.stdout:Scheduled mon update... 2024-04-24T08:55:17.329 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.329+0000 7f4dc7200700 1 -- 172.21.15.130:0/2880651071 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4da40396b0 msgr2=0x7f4da403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:17.329 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.329+0000 7f4dc7200700 1 --2- 172.21.15.130:0/2880651071 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4da40396b0 0x7f4da403bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4dac006fd0 tx=0x7f4dac006e40 comp rx=0 tx=0).stop 2024-04-24T08:55:17.329 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.329+0000 7f4dc7200700 1 -- 172.21.15.130:0/2880651071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0102550 msgr2=0x7f4dc0199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:17.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.329+0000 7f4dc7200700 1 --2- 172.21.15.130:0/2880651071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0102550 0x7f4dc0199db0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4db0005950 tx=0x7f4db0004fa0 comp rx=0 tx=0).stop 2024-04-24T08:55:17.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.329+0000 7f4dc7200700 1 -- 172.21.15.130:0/2880651071 shutdown_connections 2024-04-24T08:55:17.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.329+0000 7f4dc7200700 1 --2- 172.21.15.130:0/2880651071 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4da40396b0 0x7f4da403bb70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:17.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.329+0000 7f4dc7200700 1 --2- 172.21.15.130:0/2880651071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dc0102550 0x7f4dc0199db0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:17.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.329+0000 7f4dc7200700 1 -- 172.21.15.130:0/2880651071 >> 172.21.15.130:0/2880651071 conn(0x7f4dc00fdaf0 msgr2=0x7f4dc00fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:17.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.329+0000 7f4dc7200700 1 -- 172.21.15.130:0/2880651071 shutdown_connections 2024-04-24T08:55:17.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:17.329+0000 7f4dc7200700 1 -- 172.21.15.130:0/2880651071 wait complete. 2024-04-24T08:55:17.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi130", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:55:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:55:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:17 smithi069 ceph-mon[28170]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-24T08:55:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:17 smithi069 ceph-mon[28170]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:17 smithi069 ceph-mon[28170]: Updating smithi130:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:55:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:17 smithi069 ceph-mon[28170]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi081:172.21.15.81=b;smithi130:172.21.15.130=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:55:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:17 smithi069 ceph-mon[28170]: Saving service mon spec with placement smithi069:172.21.15.69=a;smithi081:172.21.15.81=b;smithi130:172.21.15.130=c;count:3 2024-04-24T08:55:17.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:18.084 DEBUG:teuthology.orchestra.run.smithi081:mon.b> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mon.b.service 2024-04-24T08:55:18.092 DEBUG:teuthology.orchestra.run.smithi130:mon.c> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mon.c.service 2024-04-24T08:55:18.099 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T08:55:18.099 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:55:18.517 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:55:18.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:18 smithi069 ceph-mon[28170]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:55:20.466 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.466+0000 7f2ef2e00700 1 -- 172.21.15.130:0/1731986662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2eec1083f0 msgr2=0x7f2eec1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:20.466 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.466+0000 7f2ef2e00700 1 --2- 172.21.15.130:0/1731986662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2eec1083f0 0x7f2eec1087d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009910 tx=0x7f2ee0009c20 comp rx=0 tx=0).stop 2024-04-24T08:55:20.467 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.466+0000 7f2ef2e00700 1 -- 172.21.15.130:0/1731986662 shutdown_connections 2024-04-24T08:55:20.467 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.466+0000 7f2ef2e00700 1 --2- 172.21.15.130:0/1731986662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2eec1083f0 0x7f2eec1087d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.467 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.466+0000 7f2ef2e00700 1 -- 172.21.15.130:0/1731986662 >> 172.21.15.130:0/1731986662 conn(0x7f2eec076da0 msgr2=0x7f2eec0771b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:20.467 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.466+0000 7f2ef2e00700 1 -- 172.21.15.130:0/1731986662 shutdown_connections 2024-04-24T08:55:20.467 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.466+0000 7f2ef2e00700 1 -- 172.21.15.130:0/1731986662 wait complete. 2024-04-24T08:55:20.467 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.467+0000 7f2ef2e00700 1 Processor -- start 2024-04-24T08:55:20.468 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.467+0000 7f2ef2e00700 1 -- start start 2024-04-24T08:55:20.468 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.468+0000 7f2ef2e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2eec1083f0 0x7f2eec199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.468 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.468+0000 7f2ef2e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec10c820 con 0x7f2eec1083f0 2024-04-24T08:55:20.468 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.468+0000 7f2ef1a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2eec1083f0 0x7f2eec199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.468 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.469+0000 7f2ef1a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2eec1083f0 0x7f2eec199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.130:55846/0 (socket says 172.21.15.130:55846) 2024-04-24T08:55:20.468 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.469+0000 7f2ef1a00700 1 -- 172.21.15.130:0/1049631722 learned_addr learned my addr 172.21.15.130:0/1049631722 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-24T08:55:20.469 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.469+0000 7f2ef1a00700 1 -- 172.21.15.130:0/1049631722 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec19a470 con 0x7f2eec1083f0 2024-04-24T08:55:20.469 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.470+0000 7f2ef1a00700 1 --2- 172.21.15.130:0/1049631722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2eec1083f0 0x7f2eec199f30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0006010 tx=0x7f2ee0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:20.469 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.470+0000 7f2eea000700 1 -- 172.21.15.130:0/1049631722 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee0026070 con 0x7f2eec1083f0 2024-04-24T08:55:20.469 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.470+0000 7f2ef2e00700 1 -- 172.21.15.130:0/1049631722 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eec19a760 con 0x7f2eec1083f0 2024-04-24T08:55:20.470 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.470+0000 7f2eea000700 1 -- 172.21.15.130:0/1049631722 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ee000bba0 con 0x7f2eec1083f0 2024-04-24T08:55:20.470 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.470+0000 7f2eea000700 1 -- 172.21.15.130:0/1049631722 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee00038a0 con 0x7f2eec1083f0 2024-04-24T08:55:20.470 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.470+0000 7f2ef2e00700 1 -- 172.21.15.130:0/1049631722 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eec19e800 con 0x7f2eec1083f0 2024-04-24T08:55:20.471 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.472+0000 7f2ef2e00700 1 -- 172.21.15.130:0/1049631722 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eec103aa0 con 0x7f2eec1083f0 2024-04-24T08:55:20.471 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.472+0000 7f2eea000700 1 -- 172.21.15.130:0/1049631722 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2ee0029070 con 0x7f2eec1083f0 2024-04-24T08:55:20.472 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.472+0000 7f2eea000700 1 --2- 172.21.15.130:0/1049631722 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2ed40396b0 0x7f2ed403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.472 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.472+0000 7f2eea000700 1 -- 172.21.15.130:0/1049631722 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2ee001e070 con 0x7f2eec1083f0 2024-04-24T08:55:20.472 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.473+0000 7f2ef1000700 1 --2- 172.21.15.130:0/1049631722 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2ed40396b0 0x7f2ed403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.474 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.474+0000 7f2ef1000700 1 --2- 172.21.15.130:0/1049631722 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2ed40396b0 0x7f2ed403bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8006fd0 tx=0x7f2ed8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:20.488 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.488+0000 7f2eea000700 1 -- 172.21.15.130:0/1049631722 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ee0013070 con 0x7f2eec1083f0 2024-04-24T08:55:20.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:20 smithi069 ceph-mon[28170]: Updating smithi130:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:55:20.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:20 smithi069 ceph-mon[28170]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:20.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:20 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:20.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:20 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:20.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:20 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:20.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:20 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:55:20.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:20 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:20.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:20 smithi069 ceph-mon[28170]: Deploying daemon mon.c on smithi130 2024-04-24T08:55:20.804 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.804+0000 7f2ef2e00700 1 -- 172.21.15.130:0/1049631722 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2eec19b290 con 0x7f2eec1083f0 2024-04-24T08:55:20.805 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.806+0000 7f2eea000700 1 -- 172.21.15.130:0/1049631722 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f2ee0030090 con 0x7f2eec1083f0 2024-04-24T08:55:20.806 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:55:20.806 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","modified":"2024-04-24T08:53:11.516571Z","created":"2024-04-24T08:53:11.516571Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:55:20.809 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.809+0000 7f2ef2e00700 1 -- 172.21.15.130:0/1049631722 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2ed40396b0 msgr2=0x7f2ed403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:20.809 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.809+0000 7f2ef2e00700 1 --2- 172.21.15.130:0/1049631722 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2ed40396b0 0x7f2ed403bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8006fd0 tx=0x7f2ed8006e40 comp rx=0 tx=0).stop 2024-04-24T08:55:20.809 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.809+0000 7f2ef2e00700 1 -- 172.21.15.130:0/1049631722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2eec1083f0 msgr2=0x7f2eec199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:20.809 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.809+0000 7f2ef2e00700 1 --2- 172.21.15.130:0/1049631722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2eec1083f0 0x7f2eec199f30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0006010 tx=0x7f2ee0005780 comp rx=0 tx=0).stop 2024-04-24T08:55:20.809 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.809+0000 7f2ef2e00700 1 -- 172.21.15.130:0/1049631722 shutdown_connections 2024-04-24T08:55:20.809 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.809+0000 7f2ef2e00700 1 --2- 172.21.15.130:0/1049631722 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2ed40396b0 0x7f2ed403bb70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.809 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.809+0000 7f2ef2e00700 1 --2- 172.21.15.130:0/1049631722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2eec1083f0 0x7f2eec199f30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.809 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.809+0000 7f2ef2e00700 1 -- 172.21.15.130:0/1049631722 >> 172.21.15.130:0/1049631722 conn(0x7f2eec076da0 msgr2=0x7f2eec10aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:20.809 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.809+0000 7f2ef2e00700 1 -- 172.21.15.130:0/1049631722 shutdown_connections 2024-04-24T08:55:20.809 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:20.809+0000 7f2ef2e00700 1 -- 172.21.15.130:0/1049631722 wait complete. 2024-04-24T08:55:20.810 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-04-24T08:55:21.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:21 smithi069 ceph-mon[28170]: from='client.? 172.21.15.130:0/1049631722' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:55:22.527 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:22 smithi069 ceph-mon[28170]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:23.288 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T08:55:23.288 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:55:23.729 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:55:24.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:24 smithi069 ceph-mon[28170]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:25.538 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:25 smithi130 systemd[1]: Starting Ceph mon.c for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T08:55:25.966 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.965+0000 7fb04dc00700 1 -- 172.21.15.130:0/833795868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb048074e90 msgr2=0x7fb048075270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:25.966 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.965+0000 7fb04dc00700 1 --2- 172.21.15.130:0/833795868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb048074e90 0x7fb048075270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb03800b0a0 tx=0x7fb03800b3b0 comp rx=0 tx=0).stop 2024-04-24T08:55:25.966 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.965+0000 7fb04dc00700 1 -- 172.21.15.130:0/833795868 shutdown_connections 2024-04-24T08:55:25.966 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.965+0000 7fb04dc00700 1 --2- 172.21.15.130:0/833795868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb048074e90 0x7fb048075270 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:25.966 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.965+0000 7fb04dc00700 1 -- 172.21.15.130:0/833795868 >> 172.21.15.130:0/833795868 conn(0x7fb0480fe0c0 msgr2=0x7fb0481004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:25.967 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.965+0000 7fb04dc00700 1 -- 172.21.15.130:0/833795868 shutdown_connections 2024-04-24T08:55:25.967 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.965+0000 7fb04dc00700 1 -- 172.21.15.130:0/833795868 wait complete. 2024-04-24T08:55:25.967 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.966+0000 7fb04dc00700 1 Processor -- start 2024-04-24T08:55:25.967 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.966+0000 7fb04dc00700 1 -- start start 2024-04-24T08:55:25.967 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.967+0000 7fb04dc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb048074e90 0x7fb04819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:25.968 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.967+0000 7fb04dc00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb048113120 con 0x7fb048074e90 2024-04-24T08:55:25.968 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.967+0000 7fb046a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb048074e90 0x7fb04819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:25.968 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.968+0000 7fb046a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb048074e90 0x7fb04819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.130:55866/0 (socket says 172.21.15.130:55866) 2024-04-24T08:55:25.968 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.968+0000 7fb046a00700 1 -- 172.21.15.130:0/1421973930 learned_addr learned my addr 172.21.15.130:0/1421973930 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-24T08:55:25.969 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.968+0000 7fb046a00700 1 -- 172.21.15.130:0/1421973930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04819ebc0 con 0x7fb048074e90 2024-04-24T08:55:25.969 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.968+0000 7fb046a00700 1 --2- 172.21.15.130:0/1421973930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb048074e90 0x7fb04819e680 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb038000f80 tx=0x7fb038009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:25.970 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.969+0000 7fb03f400700 1 -- 172.21.15.130:0/1421973930 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb038017040 con 0x7fb048074e90 2024-04-24T08:55:25.970 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.969+0000 7fb04dc00700 1 -- 172.21.15.130:0/1421973930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb04819ee50 con 0x7fb048074e90 2024-04-24T08:55:25.970 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.969+0000 7fb03f400700 1 -- 172.21.15.130:0/1421973930 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb038022070 con 0x7fb048074e90 2024-04-24T08:55:25.970 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.969+0000 7fb03f400700 1 -- 172.21.15.130:0/1421973930 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb038004910 con 0x7fb048074e90 2024-04-24T08:55:25.970 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.969+0000 7fb04dc00700 1 -- 172.21.15.130:0/1421973930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb04819f2b0 con 0x7fb048074e90 2024-04-24T08:55:25.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.970+0000 7fb04dc00700 1 -- 172.21.15.130:0/1421973930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0480765d0 con 0x7fb048074e90 2024-04-24T08:55:25.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.971+0000 7fb03f400700 1 -- 172.21.15.130:0/1421973930 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb038004040 con 0x7fb048074e90 2024-04-24T08:55:25.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.971+0000 7fb03f400700 1 --2- 172.21.15.130:0/1421973930 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb02c0396f0 0x7fb02c03bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:25.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.971+0000 7fb03f400700 1 -- 172.21.15.130:0/1421973930 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb03800f050 con 0x7fb048074e90 2024-04-24T08:55:25.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.972+0000 7fb046000700 1 --2- 172.21.15.130:0/1421973930 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb02c0396f0 0x7fb02c03bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:25.973 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.973+0000 7fb046000700 1 --2- 172.21.15.130:0/1421973930 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb02c0396f0 0x7fb02c03bbb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb034006fd0 tx=0x7fb034006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:25.988 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:25.988+0000 7fb03f400700 1 -- 172.21.15.130:0/1421973930 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb0380192e0 con 0x7fb048074e90 2024-04-24T08:55:26.309 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:26.307+0000 7fb04dc00700 1 -- 172.21.15.130:0/1421973930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb04819f9e0 con 0x7fb048074e90 2024-04-24T08:55:26.309 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:26.308+0000 7fb03f400700 1 -- 172.21.15.130:0/1421973930 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fb038007990 con 0x7fb048074e90 2024-04-24T08:55:26.309 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:55:26.309 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","modified":"2024-04-24T08:53:11.516571Z","created":"2024-04-24T08:53:11.516571Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:55:26.312 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:26.311+0000 7fb04dc00700 1 -- 172.21.15.130:0/1421973930 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb02c0396f0 msgr2=0x7fb02c03bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:26.312 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:26.311+0000 7fb04dc00700 1 --2- 172.21.15.130:0/1421973930 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb02c0396f0 0x7fb02c03bbb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb034006fd0 tx=0x7fb034006e40 comp rx=0 tx=0).stop 2024-04-24T08:55:26.312 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:26.311+0000 7fb04dc00700 1 -- 172.21.15.130:0/1421973930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb048074e90 msgr2=0x7fb04819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:26.312 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:26.311+0000 7fb04dc00700 1 --2- 172.21.15.130:0/1421973930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb048074e90 0x7fb04819e680 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb038000f80 tx=0x7fb038009790 comp rx=0 tx=0).stop 2024-04-24T08:55:26.312 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:26.311+0000 7fb04dc00700 1 -- 172.21.15.130:0/1421973930 shutdown_connections 2024-04-24T08:55:26.312 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:26.311+0000 7fb04dc00700 1 --2- 172.21.15.130:0/1421973930 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb02c0396f0 0x7fb02c03bbb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:26.312 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:26.311+0000 7fb04dc00700 1 --2- 172.21.15.130:0/1421973930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb048074e90 0x7fb04819e680 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:26.312 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:26.311+0000 7fb04dc00700 1 -- 172.21.15.130:0/1421973930 >> 172.21.15.130:0/1421973930 conn(0x7fb0480fe0c0 msgr2=0x7fb0480fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:26.312 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:26.311+0000 7fb04dc00700 1 -- 172.21.15.130:0/1421973930 shutdown_connections 2024-04-24T08:55:26.313 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:26.311+0000 7fb04dc00700 1 -- 172.21.15.130:0/1421973930 wait complete. 2024-04-24T08:55:26.313 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-04-24T08:55:26.410 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 podman[31500]: 2024-04-24 08:55:26.005906759 +0000 UTC m=+0.264214206 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:55:26.413 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 podman[31500]: 2024-04-24 08:55:26.123991386 +0000 UTC m=+0.382298888 container create dc34ddfa715916bad147fd274e09d1172a4e655672bc44baae5397ee8d51e69e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-24T08:55:26.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:26 smithi069 ceph-mon[28170]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 podman[31500]: 2024-04-24 08:55:26.41588882 +0000 UTC m=+0.674196299 container init dc34ddfa715916bad147fd274e09d1172a4e655672bc44baae5397ee8d51e69e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-24T08:55:26.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 podman[31500]: 2024-04-24 08:55:26.424138484 +0000 UTC m=+0.682445951 container start dc34ddfa715916bad147fd274e09d1172a4e655672bc44baae5397ee8d51e69e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-24T08:55:26.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T08:55:26.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev), process ceph-mon, pid 2 2024-04-24T08:55:26.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pidfile_write: ignore empty --pid-file 2024-04-24T08:55:26.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: load: jerasure load: lrc 2024-04-24T08:55:26.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: RocksDB version: 7.9.2 2024-04-24T08:55:26.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Git sha 0 2024-04-24T08:55:26.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Compile date 2024-04-23 21:06:33 2024-04-24T08:55:26.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: DB SUMMARY 2024-04-24T08:55:26.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: DB Session ID: 668VUP8A70MDO5MCPZ0Q 2024-04-24T08:55:26.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: CURRENT file: CURRENT 2024-04-24T08:55:26.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T08:55:26.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-24T08:55:26.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-24T08:55:26.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-24T08:55:26.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.error_if_exists: 0 2024-04-24T08:55:26.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.create_if_missing: 0 2024-04-24T08:55:26.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.paranoid_checks: 1 2024-04-24T08:55:26.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T08:55:26.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T08:55:26.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T08:55:26.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.env: 0x56085fe03cc0 2024-04-24T08:55:26.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T08:55:26.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.info_log: 0x560860b3fc60 2024-04-24T08:55:26.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T08:55:26.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.statistics: (nil) 2024-04-24T08:55:26.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.use_fsync: 0 2024-04-24T08:55:26.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_log_file_size: 0 2024-04-24T08:55:26.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T08:55:26.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T08:55:26.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T08:55:26.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T08:55:26.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.allow_fallocate: 1 2024-04-24T08:55:26.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T08:55:26.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T08:55:26.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.use_direct_reads: 0 2024-04-24T08:55:26.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T08:55:26.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T08:55:26.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.db_log_dir: 2024-04-24T08:55:26.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.wal_dir: 2024-04-24T08:55:26.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T08:55:26.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T08:55:26.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T08:55:26.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T08:55:26.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T08:55:26.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T08:55:26.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T08:55:26.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T08:55:26.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.write_buffer_manager: 0x560860b0e1e0 2024-04-24T08:55:26.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T08:55:26.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T08:55:26.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T08:55:26.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T08:55:26.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T08:55:26.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T08:55:26.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T08:55:26.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T08:55:26.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.unordered_write: 0 2024-04-24T08:55:26.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T08:55:26.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T08:55:26.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T08:55:26.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T08:55:26.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.row_cache: None 2024-04-24T08:55:26.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.wal_filter: None 2024-04-24T08:55:26.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T08:55:26.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T08:55:26.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.two_write_queues: 0 2024-04-24T08:55:26.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T08:55:26.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.wal_compression: 0 2024-04-24T08:55:26.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.atomic_flush: 0 2024-04-24T08:55:26.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T08:55:26.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T08:55:26.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T08:55:26.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.log_readahead_size: 0 2024-04-24T08:55:26.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T08:55:26.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T08:55:26.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T08:55:26.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T08:55:26.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T08:55:26.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T08:55:26.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T08:55:26.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_background_jobs: 2 2024-04-24T08:55:26.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_background_compactions: -1 2024-04-24T08:55:26.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_subcompactions: 1 2024-04-24T08:55:26.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T08:55:26.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T08:55:26.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T08:55:26.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T08:55:26.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T08:55:26.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T08:55:26.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T08:55:26.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T08:55:26.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_open_files: -1 2024-04-24T08:55:26.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T08:55:26.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T08:55:26.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T08:55:26.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T08:55:26.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_background_flushes: -1 2024-04-24T08:55:26.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Compression algorithms supported: 2024-04-24T08:55:26.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T08:55:26.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: kZSTD supported: 0 2024-04-24T08:55:26.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: kXpressCompression supported: 0 2024-04-24T08:55:26.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T08:55:26.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: kZlibCompression supported: 1 2024-04-24T08:55:26.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: kSnappyCompression supported: 1 2024-04-24T08:55:26.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: kLZ4Compression supported: 1 2024-04-24T08:55:26.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: kBZip2Compression supported: 0 2024-04-24T08:55:26.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T08:55:26.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T08:55:26.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-24T08:55:26.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T08:55:26.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T08:55:26.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.merge_operator: 2024-04-24T08:55:26.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compaction_filter: None 2024-04-24T08:55:26.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T08:55:26.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T08:55:26.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T08:55:26.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T08:55:26.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560860b3fe20) 2024-04-24T08:55:26.715 INFO:journalctl@ceph.mon.c.smithi130.stdout: cache_index_and_filter_blocks: 1 2024-04-24T08:55:26.715 INFO:journalctl@ceph.mon.c.smithi130.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T08:55:26.716 INFO:journalctl@ceph.mon.c.smithi130.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T08:55:26.716 INFO:journalctl@ceph.mon.c.smithi130.stdout: pin_top_level_index_and_filter: 1 2024-04-24T08:55:26.716 INFO:journalctl@ceph.mon.c.smithi130.stdout: index_type: 0 2024-04-24T08:55:26.716 INFO:journalctl@ceph.mon.c.smithi130.stdout: data_block_index_type: 0 2024-04-24T08:55:26.716 INFO:journalctl@ceph.mon.c.smithi130.stdout: index_shortening: 1 2024-04-24T08:55:26.716 INFO:journalctl@ceph.mon.c.smithi130.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T08:55:26.716 INFO:journalctl@ceph.mon.c.smithi130.stdout: checksum: 4 2024-04-24T08:55:26.716 INFO:journalctl@ceph.mon.c.smithi130.stdout: no_block_cache: 0 2024-04-24T08:55:26.716 INFO:journalctl@ceph.mon.c.smithi130.stdout: block_cache: 0x560860b41090 2024-04-24T08:55:26.716 INFO:journalctl@ceph.mon.c.smithi130.stdout: block_cache_name: BinnedLRUCache 2024-04-24T08:55:26.716 INFO:journalctl@ceph.mon.c.smithi130.stdout: block_cache_options: 2024-04-24T08:55:26.716 INFO:journalctl@ceph.mon.c.smithi130.stdout: capacity : 536870912 2024-04-24T08:55:26.716 INFO:journalctl@ceph.mon.c.smithi130.stdout: num_shard_bits : 4 2024-04-24T08:55:26.716 INFO:journalctl@ceph.mon.c.smithi130.stdout: strict_capacity_limit : 0 2024-04-24T08:55:26.716 INFO:journalctl@ceph.mon.c.smithi130.stdout: high_pri_pool_ratio: 0.000 2024-04-24T08:55:26.716 INFO:journalctl@ceph.mon.c.smithi130.stdout: block_cache_compressed: (nil) 2024-04-24T08:55:26.717 INFO:journalctl@ceph.mon.c.smithi130.stdout: persistent_cache: (nil) 2024-04-24T08:55:26.717 INFO:journalctl@ceph.mon.c.smithi130.stdout: block_size: 4096 2024-04-24T08:55:26.717 INFO:journalctl@ceph.mon.c.smithi130.stdout: block_size_deviation: 10 2024-04-24T08:55:26.717 INFO:journalctl@ceph.mon.c.smithi130.stdout: block_restart_interval: 16 2024-04-24T08:55:26.717 INFO:journalctl@ceph.mon.c.smithi130.stdout: index_block_restart_interval: 1 2024-04-24T08:55:26.717 INFO:journalctl@ceph.mon.c.smithi130.stdout: metadata_block_size: 4096 2024-04-24T08:55:26.717 INFO:journalctl@ceph.mon.c.smithi130.stdout: partition_filters: 0 2024-04-24T08:55:26.717 INFO:journalctl@ceph.mon.c.smithi130.stdout: use_delta_encoding: 1 2024-04-24T08:55:26.717 INFO:journalctl@ceph.mon.c.smithi130.stdout: filter_policy: bloomfilter 2024-04-24T08:55:26.717 INFO:journalctl@ceph.mon.c.smithi130.stdout: whole_key_filtering: 1 2024-04-24T08:55:26.717 INFO:journalctl@ceph.mon.c.smithi130.stdout: verify_compression: 0 2024-04-24T08:55:26.717 INFO:journalctl@ceph.mon.c.smithi130.stdout: read_amp_bytes_per_bit: 0 2024-04-24T08:55:26.717 INFO:journalctl@ceph.mon.c.smithi130.stdout: format_version: 5 2024-04-24T08:55:26.717 INFO:journalctl@ceph.mon.c.smithi130.stdout: enable_index_compression: 1 2024-04-24T08:55:26.717 INFO:journalctl@ceph.mon.c.smithi130.stdout: block_align: 0 2024-04-24T08:55:26.717 INFO:journalctl@ceph.mon.c.smithi130.stdout: max_auto_readahead_size: 262144 2024-04-24T08:55:26.718 INFO:journalctl@ceph.mon.c.smithi130.stdout: prepopulate_block_cache: 0 2024-04-24T08:55:26.718 INFO:journalctl@ceph.mon.c.smithi130.stdout: initial_auto_readahead_size: 8192 2024-04-24T08:55:26.718 INFO:journalctl@ceph.mon.c.smithi130.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T08:55:26.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T08:55:26.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T08:55:26.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compression: NoCompression 2024-04-24T08:55:26.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T08:55:26.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T08:55:26.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T08:55:26.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.num_levels: 7 2024-04-24T08:55:26.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T08:55:26.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T08:55:26.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T08:55:26.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T08:55:26.719 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T08:55:26.719 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T08:55:26.719 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T08:55:26.719 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:55:26.719 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T08:55:26.719 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T08:55:26.719 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:55:26.719 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:55:26.719 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T08:55:26.719 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T08:55:26.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T08:55:26.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T08:55:26.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:55:26.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:55:26.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T08:55:26.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T08:55:26.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:55:26.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T08:55:26.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T08:55:26.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T08:55:26.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T08:55:26.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T08:55:26.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T08:55:26.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T08:55:26.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T08:55:26.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T08:55:26.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T08:55:26.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T08:55:26.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T08:55:26.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T08:55:26.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T08:55:26.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T08:55:26.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T08:55:26.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T08:55:26.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T08:55:26.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T08:55:26.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T08:55:26.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T08:55:26.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T08:55:26.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T08:55:26.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T08:55:26.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T08:55:26.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T08:55:26.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T08:55:26.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T08:55:26.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T08:55:26.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T08:55:26.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T08:55:26.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T08:55:26.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-24T08:55:26.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.inplace_update_support: 0 2024-04-24T08:55:26.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T08:55:26.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T08:55:26.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T08:55:26.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T08:55:26.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.bloom_locality: 0 2024-04-24T08:55:26.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.max_successive_merges: 0 2024-04-24T08:55:26.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T08:55:26.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T08:55:26.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T08:55:26.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T08:55:26.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.ttl: 2592000 2024-04-24T08:55:26.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T08:55:26.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T08:55:26.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T08:55:26.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.enable_blob_files: false 2024-04-24T08:55:26.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.min_blob_size: 0 2024-04-24T08:55:26.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T08:55:26.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T08:55:26.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T08:55:26.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T08:55:26.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T08:55:26.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T08:55:26.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T08:55:26.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T08:55:26.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T08:55:26.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-24T08:55:26.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c7b146bd-f161-4b05-a46a-0d1dfc876961 2024-04-24T08:55:26.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713948926480590, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-24T08:55:26.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-24T08:55:26.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713948926482046, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713948926, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c7b146bd-f161-4b05-a46a-0d1dfc876961", "db_session_id": "668VUP8A70MDO5MCPZ0Q", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-24T08:55:26.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713948926482287, "job": 1, "event": "recovery_finished"} 2024-04-24T08:55:26.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-24T08:55:26.725 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T08:55:26.725 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560860c2e000 2024-04-24T08:55:26.725 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: DB pointer 0x560860c18000 2024-04-24T08:55:26.725 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-24T08:55:26.725 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T08:55:26.725 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T08:55:26.725 INFO:journalctl@ceph.mon.c.smithi130.stdout: ** DB Stats ** 2024-04-24T08:55:26.725 INFO:journalctl@ceph.mon.c.smithi130.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:55:26.725 INFO:journalctl@ceph.mon.c.smithi130.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T08:55:26.725 INFO:journalctl@ceph.mon.c.smithi130.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:55:26.725 INFO:journalctl@ceph.mon.c.smithi130.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:55:26.725 INFO:journalctl@ceph.mon.c.smithi130.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T08:55:26.725 INFO:journalctl@ceph.mon.c.smithi130.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:55:26.725 INFO:journalctl@ceph.mon.c.smithi130.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:55:26.725 INFO:journalctl@ceph.mon.c.smithi130.stdout: 2024-04-24T08:55:26.726 INFO:journalctl@ceph.mon.c.smithi130.stdout: ** Compaction Stats [default] ** 2024-04-24T08:55:26.726 INFO:journalctl@ceph.mon.c.smithi130.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:55:26.726 INFO:journalctl@ceph.mon.c.smithi130.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T08:55:26.726 INFO:journalctl@ceph.mon.c.smithi130.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:55:26.726 INFO:journalctl@ceph.mon.c.smithi130.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:55:26.726 INFO:journalctl@ceph.mon.c.smithi130.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:55:26.726 INFO:journalctl@ceph.mon.c.smithi130.stdout: 2024-04-24T08:55:26.726 INFO:journalctl@ceph.mon.c.smithi130.stdout: ** Compaction Stats [default] ** 2024-04-24T08:55:26.726 INFO:journalctl@ceph.mon.c.smithi130.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:55:26.726 INFO:journalctl@ceph.mon.c.smithi130.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T08:55:26.726 INFO:journalctl@ceph.mon.c.smithi130.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:55:26.726 INFO:journalctl@ceph.mon.c.smithi130.stdout: 2024-04-24T08:55:26.726 INFO:journalctl@ceph.mon.c.smithi130.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T08:55:26.726 INFO:journalctl@ceph.mon.c.smithi130.stdout: 2024-04-24T08:55:26.727 INFO:journalctl@ceph.mon.c.smithi130.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:55:26.727 INFO:journalctl@ceph.mon.c.smithi130.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T08:55:26.727 INFO:journalctl@ceph.mon.c.smithi130.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T08:55:26.727 INFO:journalctl@ceph.mon.c.smithi130.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T08:55:26.727 INFO:journalctl@ceph.mon.c.smithi130.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T08:55:26.727 INFO:journalctl@ceph.mon.c.smithi130.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T08:55:26.727 INFO:journalctl@ceph.mon.c.smithi130.stdout: Cumulative compaction: 0.00 GB write, 0.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:55:26.727 INFO:journalctl@ceph.mon.c.smithi130.stdout: Interval compaction: 0.00 GB write, 0.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:55:26.727 INFO:journalctl@ceph.mon.c.smithi130.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T08:55:26.727 INFO:journalctl@ceph.mon.c.smithi130.stdout: Block cache BinnedLRUCache@0x560860b41090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-24T08:55:26.727 INFO:journalctl@ceph.mon.c.smithi130.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-24T08:55:26.727 INFO:journalctl@ceph.mon.c.smithi130.stdout: 2024-04-24T08:55:26.727 INFO:journalctl@ceph.mon.c.smithi130.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T08:55:26.727 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: using public_addr v2:172.21.15.130:0/0 -> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] 2024-04-24T08:55:26.727 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: starting mon.c rank -1 at public addrs [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] at bind addrs [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:55:26.728 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: mon.c@-1(???) e0 preinit fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:55:26.728 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: mon.c@-1(synchronizing).mds e1 new map 2024-04-24T08:55:26.728 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-24T08:55:26.728 INFO:journalctl@ceph.mon.c.smithi130.stdout: e1 2024-04-24T08:55:26.728 INFO:journalctl@ceph.mon.c.smithi130.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:55:26.728 INFO:journalctl@ceph.mon.c.smithi130.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T08:55:26.728 INFO:journalctl@ceph.mon.c.smithi130.stdout: legacy client fscid: -1 2024-04-24T08:55:26.728 INFO:journalctl@ceph.mon.c.smithi130.stdout: 2024-04-24T08:55:26.728 INFO:journalctl@ceph.mon.c.smithi130.stdout: No filesystems configured 2024-04-24T08:55:26.728 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-24T08:55:26.728 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T08:55:26.728 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T08:55:26.728 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T08:55:26.728 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T08:55:26.728 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T08:55:26.729 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T08:55:26.729 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:55:26.729 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:55:26.729 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:55:26.729 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi081", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:55:26.729 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.729 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: Deploying cephadm binary to smithi081 2024-04-24T08:55:26.729 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.729 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.729 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.729 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: Added host smithi081 2024-04-24T08:55:26.729 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:55:26.729 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.729 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.730 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.730 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.730 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:55:26.730 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.730 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.730 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.730 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi130", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:55:26.730 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: Deploying cephadm binary to smithi130 2024-04-24T08:55:26.730 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.730 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.730 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.730 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.730 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.730 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi081", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:55:26.730 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:26.731 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:55:26.731 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: Updating smithi081:/etc/ceph/ceph.conf 2024-04-24T08:55:26.731 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.731 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: Updating smithi081:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:55:26.731 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.731 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:55:26.731 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.731 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: Added host smithi130 2024-04-24T08:55:26.731 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: Updating smithi081:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:55:26.731 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.731 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.731 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.731 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:55:26.732 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.732 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.732 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:55:26.732 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.732 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.732 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.732 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.732 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3913175972' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T08:55:26.732 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.732 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3913175972' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T08:55:26.732 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:55:26.733 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.733 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.733 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.733 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.733 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.733 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi130", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:55:26.733 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:26.733 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:55:26.733 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-24T08:55:26.733 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.733 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: Updating smithi130:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:55:26.733 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi081:172.21.15.81=b;smithi130:172.21.15.130=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:55:26.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: Saving service mon spec with placement smithi069:172.21.15.69=a;smithi081:172.21.15.81=b;smithi130:172.21.15.130=c;count:3 2024-04-24T08:55:26.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:55:26.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: Updating smithi130:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:55:26.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:26.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:55:26.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:26.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: Deploying daemon mon.c on smithi130 2024-04-24T08:55:26.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: from='client.? 172.21.15.130:0/1049631722' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:55:26.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.735 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:26.735 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 ceph-mon[31553]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-24T08:55:27.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 bash[31500]: dc34ddfa715916bad147fd274e09d1172a4e655672bc44baae5397ee8d51e69e 2024-04-24T08:55:27.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:26 smithi130 systemd[1]: Started Ceph mon.c for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T08:55:28.283 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T08:55:28.283 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:55:28.722 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:55:31.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:55:31.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: mon.a calling monitor election 2024-04-24T08:55:31.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:31.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:31.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:31.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:31.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: mon.c calling monitor election 2024-04-24T08:55:31.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:31.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:31.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:31.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:31.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:31.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T08:55:31.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: monmap epoch 2 2024-04-24T08:55:31.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:55:31.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: last_changed 2024-04-24T08:55:26.528456+0000 2024-04-24T08:55:31.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: created 2024-04-24T08:53:11.516571+0000 2024-04-24T08:55:31.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: min_mon_release 19 (squid) 2024-04-24T08:55:31.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: election_strategy: 1 2024-04-24T08:55:31.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-04-24T08:55:31.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-04-24T08:55:31.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: fsmap 2024-04-24T08:55:31.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:55:31.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: mgrmap e13: x(active, since 68s) 2024-04-24T08:55:31.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: overall HEALTH_OK 2024-04-24T08:55:31.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:31.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:31.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:31.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:55:31.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:32.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:55:32.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: mon.a calling monitor election 2024-04-24T08:55:32.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:32.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:32.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:32.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:32.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: mon.c calling monitor election 2024-04-24T08:55:32.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:32.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:32.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:32.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:32.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:32.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T08:55:32.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: monmap epoch 2 2024-04-24T08:55:32.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:55:32.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: last_changed 2024-04-24T08:55:26.528456+0000 2024-04-24T08:55:32.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: created 2024-04-24T08:53:11.516571+0000 2024-04-24T08:55:32.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: min_mon_release 19 (squid) 2024-04-24T08:55:32.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: election_strategy: 1 2024-04-24T08:55:32.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-04-24T08:55:32.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-04-24T08:55:32.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: fsmap 2024-04-24T08:55:32.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:55:32.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: mgrmap e13: x(active, since 68s) 2024-04-24T08:55:32.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: overall HEALTH_OK 2024-04-24T08:55:32.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:32.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:32.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:32.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:55:32.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:32.153 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.152+0000 7f7a43600700 1 -- 172.21.15.130:0/1705699784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a3c074f20 msgr2=0x7f7a3c075300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:32.153 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.152+0000 7f7a43600700 1 --2- 172.21.15.130:0/1705699784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a3c074f20 0x7f7a3c075300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7a30005fa0 tx=0x7f7a3000fe80 comp rx=0 tx=0).stop 2024-04-24T08:55:32.153 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.153+0000 7f7a43600700 1 -- 172.21.15.130:0/1705699784 shutdown_connections 2024-04-24T08:55:32.154 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.153+0000 7f7a43600700 1 --2- 172.21.15.130:0/1705699784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a3c074f20 0x7f7a3c075300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:32.154 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.153+0000 7f7a43600700 1 -- 172.21.15.130:0/1705699784 >> 172.21.15.130:0/1705699784 conn(0x7f7a3c0fe0c0 msgr2=0x7f7a3c1004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:32.154 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.153+0000 7f7a43600700 1 -- 172.21.15.130:0/1705699784 shutdown_connections 2024-04-24T08:55:32.154 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.153+0000 7f7a43600700 1 -- 172.21.15.130:0/1705699784 wait complete. 2024-04-24T08:55:32.155 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.155+0000 7f7a43600700 1 Processor -- start 2024-04-24T08:55:32.155 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.155+0000 7f7a43600700 1 -- start start 2024-04-24T08:55:32.156 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.156+0000 7f7a43600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a3c074f20 0x7f7a3c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:32.156 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.156+0000 7f7a43600700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3c19a920 0x7f7a3c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:32.156 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.156+0000 7f7a43600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c10ed10 con 0x7f7a3c074f20 2024-04-24T08:55:32.156 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.156+0000 7f7a43600700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c10eb90 con 0x7f7a3c19a920 2024-04-24T08:55:32.157 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.157+0000 7f7a40c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a3c074f20 0x7f7a3c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:32.157 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.157+0000 7f7a3be00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3c19a920 0x7f7a3c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:32.157 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.157+0000 7f7a40c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a3c074f20 0x7f7a3c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.130:60966/0 (socket says 172.21.15.130:60966) 2024-04-24T08:55:32.157 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.157+0000 7f7a40c00700 1 -- 172.21.15.130:0/1526172146 learned_addr learned my addr 172.21.15.130:0/1526172146 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-24T08:55:32.158 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.157+0000 7f7a3be00700 1 -- 172.21.15.130:0/1526172146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3c19a920 msgr2=0x7f7a3c19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-24T08:55:32.158 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.157+0000 7f7a3be00700 1 -- 172.21.15.130:0/1526172146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3c19a920 msgr2=0x7f7a3c19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T08:55:32.158 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.157+0000 7f7a3be00700 1 --2- 172.21.15.130:0/1526172146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3c19a920 0x7f7a3c19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T08:55:32.158 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.157+0000 7f7a3be00700 1 --2- 172.21.15.130:0/1526172146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3c19a920 0x7f7a3c19ec60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:55:32.158 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.158+0000 7f7a40c00700 1 -- 172.21.15.130:0/1526172146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3c19a920 msgr2=0x7f7a3c19ec60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:55:32.158 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.158+0000 7f7a40c00700 1 --2- 172.21.15.130:0/1526172146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3c19a920 0x7f7a3c19ec60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:32.158 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.158+0000 7f7a40c00700 1 -- 172.21.15.130:0/1526172146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3c19af60 con 0x7f7a3c074f20 2024-04-24T08:55:32.158 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.158+0000 7f7a40c00700 1 --2- 172.21.15.130:0/1526172146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a3c074f20 0x7f7a3c19a3e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000f850 tx=0x7f7a30003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:32.159 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.159+0000 7f7a39600700 1 -- 172.21.15.130:0/1526172146 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7a30005200 con 0x7f7a3c074f20 2024-04-24T08:55:32.159 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.159+0000 7f7a39600700 1 -- 172.21.15.130:0/1526172146 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a30012450 con 0x7f7a3c074f20 2024-04-24T08:55:32.159 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.159+0000 7f7a39600700 1 -- 172.21.15.130:0/1526172146 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7a30004240 con 0x7f7a3c074f20 2024-04-24T08:55:32.159 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.159+0000 7f7a43600700 1 -- 172.21.15.130:0/1526172146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3c19f1a0 con 0x7f7a3c074f20 2024-04-24T08:55:32.159 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.159+0000 7f7a43600700 1 -- 172.21.15.130:0/1526172146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a3c19f640 con 0x7f7a3c074f20 2024-04-24T08:55:32.160 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.160+0000 7f7a43600700 1 -- 172.21.15.130:0/1526172146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3c0765d0 con 0x7f7a3c074f20 2024-04-24T08:55:32.161 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.161+0000 7f7a39600700 1 -- 172.21.15.130:0/1526172146 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7a3002c060 con 0x7f7a3c074f20 2024-04-24T08:55:32.161 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.162+0000 7f7a39600700 1 --2- 172.21.15.130:0/1526172146 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7a24039410 0x7f7a2403b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:32.161 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.162+0000 7f7a39600700 1 -- 172.21.15.130:0/1526172146 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7a30022070 con 0x7f7a3c074f20 2024-04-24T08:55:32.162 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.162+0000 7f7a3be00700 1 --2- 172.21.15.130:0/1526172146 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7a24039410 0x7f7a2403b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:32.163 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.164+0000 7f7a3be00700 1 --2- 172.21.15.130:0/1526172146 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7a24039410 0x7f7a2403b8d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009850 tx=0x7f7a28008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:32.177 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.178+0000 7f7a39600700 1 -- 172.21.15.130:0/1526172146 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a30024a00 con 0x7f7a3c074f20 2024-04-24T08:55:32.497 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.498+0000 7f7a43600700 1 -- 172.21.15.130:0/1526172146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7a3c19fa20 con 0x7f7a3c074f20 2024-04-24T08:55:32.499 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.499+0000 7f7a39600700 1 -- 172.21.15.130:0/1526172146 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f7a3002bd80 con 0x7f7a3c074f20 2024-04-24T08:55:32.499 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:55:32.499 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":2,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","modified":"2024-04-24T08:55:26.528456Z","created":"2024-04-24T08:53:11.516571Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:3300","nonce":0},{"type":"v1","addr":"172.21.15.130:6789","nonce":0}]},"addr":"172.21.15.130:6789/0","public_addr":"172.21.15.130:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T08:55:32.502 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.502+0000 7f7a43600700 1 -- 172.21.15.130:0/1526172146 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7a24039410 msgr2=0x7f7a2403b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:32.502 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.502+0000 7f7a43600700 1 --2- 172.21.15.130:0/1526172146 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7a24039410 0x7f7a2403b8d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009850 tx=0x7f7a28008040 comp rx=0 tx=0).stop 2024-04-24T08:55:32.502 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.502+0000 7f7a43600700 1 -- 172.21.15.130:0/1526172146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a3c074f20 msgr2=0x7f7a3c19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:32.502 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.502+0000 7f7a43600700 1 --2- 172.21.15.130:0/1526172146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a3c074f20 0x7f7a3c19a3e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000f850 tx=0x7f7a30003e10 comp rx=0 tx=0).stop 2024-04-24T08:55:32.503 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.502+0000 7f7a43600700 1 -- 172.21.15.130:0/1526172146 shutdown_connections 2024-04-24T08:55:32.503 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.502+0000 7f7a43600700 1 --2- 172.21.15.130:0/1526172146 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7a24039410 0x7f7a2403b8d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:32.503 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.502+0000 7f7a43600700 1 --2- 172.21.15.130:0/1526172146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a3c074f20 0x7f7a3c19a3e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:32.503 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.502+0000 7f7a43600700 1 --2- 172.21.15.130:0/1526172146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a3c19a920 0x7f7a3c19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:32.503 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.502+0000 7f7a43600700 1 -- 172.21.15.130:0/1526172146 >> 172.21.15.130:0/1526172146 conn(0x7f7a3c0fe0c0 msgr2=0x7f7a3c108880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:32.503 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.502+0000 7f7a43600700 1 -- 172.21.15.130:0/1526172146 shutdown_connections 2024-04-24T08:55:32.503 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:32.502+0000 7f7a43600700 1 -- 172.21.15.130:0/1526172146 wait complete. 2024-04-24T08:55:32.504 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 2 2024-04-24T08:55:32.857 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:32 smithi130 ceph-mon[31553]: Deploying daemon mon.b on smithi081 2024-04-24T08:55:32.857 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:32 smithi130 ceph-mon[31553]: from='client.? 172.21.15.130:0/1526172146' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:55:32.857 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:32 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:33.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:32 smithi069 ceph-mon[28170]: Deploying daemon mon.b on smithi081 2024-04-24T08:55:33.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:32 smithi069 ceph-mon[28170]: from='client.? 172.21.15.130:0/1526172146' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:55:33.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:32 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:33.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:33 smithi069 ceph-mon[28170]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:33.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:33 smithi130 ceph-mon[31553]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:34.248 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T08:55:34.248 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:55:34.675 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:55:36.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:36 smithi130 ceph-mon[31553]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:36.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:36 smithi069 ceph-mon[28170]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:36.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.627+0000 7f4732600700 1 -- 172.21.15.130:0/2669081390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c10a630 msgr2=0x7f472c10aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:36.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.627+0000 7f4732600700 1 --2- 172.21.15.130:0/2669081390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c10a630 0x7f472c10aa10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4720009910 tx=0x7f4720009c20 comp rx=0 tx=0).stop 2024-04-24T08:55:36.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.628+0000 7f4732600700 1 -- 172.21.15.130:0/2669081390 shutdown_connections 2024-04-24T08:55:36.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.628+0000 7f4732600700 1 --2- 172.21.15.130:0/2669081390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c10a630 0x7f472c10aa10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:36.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.628+0000 7f4732600700 1 -- 172.21.15.130:0/2669081390 >> 172.21.15.130:0/2669081390 conn(0x7f472c100140 msgr2=0x7f472c102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:36.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.628+0000 7f4732600700 1 -- 172.21.15.130:0/2669081390 shutdown_connections 2024-04-24T08:55:36.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.628+0000 7f4732600700 1 -- 172.21.15.130:0/2669081390 wait complete. 2024-04-24T08:55:36.629 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.629+0000 7f4732600700 1 Processor -- start 2024-04-24T08:55:36.629 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.629+0000 7f4732600700 1 -- start start 2024-04-24T08:55:36.630 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.630+0000 7f4732600700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f472c10a630 0x7f472c19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:36.630 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.630+0000 7f4732600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c19a730 0x7f472c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:36.630 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.630+0000 7f4732600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10eb20 con 0x7f472c19a730 2024-04-24T08:55:36.630 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.630+0000 7f4732600700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10e9a0 con 0x7f472c10a630 2024-04-24T08:55:36.630 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.630+0000 7f4731200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f472c10a630 0x7f472c19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:36.630 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.630+0000 7f4731200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f472c10a630 0x7f472c19a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:49476/0 (socket says 172.21.15.130:49476) 2024-04-24T08:55:36.630 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.630+0000 7f4731200700 1 -- 172.21.15.130:0/2686201943 learned_addr learned my addr 172.21.15.130:0/2686201943 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-24T08:55:36.631 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.630+0000 7f472be00700 1 --2- 172.21.15.130:0/2686201943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c19a730 0x7f472c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:36.631 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.631+0000 7f4731200700 1 -- 172.21.15.130:0/2686201943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c19a730 msgr2=0x7f472c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:36.631 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.631+0000 7f4731200700 1 --2- 172.21.15.130:0/2686201943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c19a730 0x7f472c19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:36.631 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.631+0000 7f4731200700 1 -- 172.21.15.130:0/2686201943 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c19ad70 con 0x7f472c10a630 2024-04-24T08:55:36.631 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.631+0000 7f4731200700 1 --2- 172.21.15.130:0/2686201943 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f472c10a630 0x7f472c19a1f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4720004f80 tx=0x7f4720004fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:36.631 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.632+0000 7f4729600700 1 -- 172.21.15.130:0/2686201943 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4720005dc0 con 0x7f472c10a630 2024-04-24T08:55:36.631 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.632+0000 7f4729600700 1 -- 172.21.15.130:0/2686201943 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f472000b6f0 con 0x7f472c10a630 2024-04-24T08:55:36.632 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.632+0000 7f4732600700 1 -- 172.21.15.130:0/2686201943 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472c19f000 con 0x7f472c10a630 2024-04-24T08:55:36.632 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.632+0000 7f4729600700 1 -- 172.21.15.130:0/2686201943 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4720003880 con 0x7f472c10a630 2024-04-24T08:55:36.632 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.632+0000 7f4732600700 1 -- 172.21.15.130:0/2686201943 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472c19f560 con 0x7f472c10a630 2024-04-24T08:55:36.632 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.633+0000 7f4732600700 1 -- 172.21.15.130:0/2686201943 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f472c105d70 con 0x7f472c10a630 2024-04-24T08:55:36.633 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.633+0000 7f4729600700 1 -- 172.21.15.130:0/2686201943 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4720029070 con 0x7f472c10a630 2024-04-24T08:55:36.633 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.634+0000 7f4729600700 1 --2- 172.21.15.130:0/2686201943 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4714039440 0x7f471403b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:36.633 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.634+0000 7f4729600700 1 -- 172.21.15.130:0/2686201943 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f472001e070 con 0x7f472c10a630 2024-04-24T08:55:36.634 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.634+0000 7f472be00700 1 --2- 172.21.15.130:0/2686201943 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4714039440 0x7f471403b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:36.635 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.635+0000 7f472be00700 1 --2- 172.21.15.130:0/2686201943 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4714039440 0x7f471403b900 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4718009ad0 tx=0x7f4718009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:36.640 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.640+0000 7f4729600700 1 -- 172.21.15.130:0/2686201943 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4720013070 con 0x7f472c10a630 2024-04-24T08:55:36.945 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.946+0000 7f4732600700 1 -- 172.21.15.130:0/2686201943 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f472c19f810 con 0x7f472c10a630 2024-04-24T08:55:36.946 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.947+0000 7f4729600700 1 -- 172.21.15.130:0/2686201943 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f4720030090 con 0x7f472c10a630 2024-04-24T08:55:36.947 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:55:36.947 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":2,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","modified":"2024-04-24T08:55:26.528456Z","created":"2024-04-24T08:53:11.516571Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:3300","nonce":0},{"type":"v1","addr":"172.21.15.130:6789","nonce":0}]},"addr":"172.21.15.130:6789/0","public_addr":"172.21.15.130:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T08:55:36.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.950+0000 7f4732600700 1 -- 172.21.15.130:0/2686201943 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4714039440 msgr2=0x7f471403b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:36.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.950+0000 7f4732600700 1 --2- 172.21.15.130:0/2686201943 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4714039440 0x7f471403b900 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4718009ad0 tx=0x7f4718009210 comp rx=0 tx=0).stop 2024-04-24T08:55:36.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.950+0000 7f4732600700 1 -- 172.21.15.130:0/2686201943 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f472c10a630 msgr2=0x7f472c19a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:36.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.950+0000 7f4732600700 1 --2- 172.21.15.130:0/2686201943 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f472c10a630 0x7f472c19a1f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4720004f80 tx=0x7f4720004fb0 comp rx=0 tx=0).stop 2024-04-24T08:55:36.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.950+0000 7f4732600700 1 -- 172.21.15.130:0/2686201943 shutdown_connections 2024-04-24T08:55:36.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.950+0000 7f4732600700 1 --2- 172.21.15.130:0/2686201943 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4714039440 0x7f471403b900 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:36.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.950+0000 7f4732600700 1 --2- 172.21.15.130:0/2686201943 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f472c10a630 0x7f472c19a1f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:36.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.950+0000 7f4732600700 1 --2- 172.21.15.130:0/2686201943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f472c19a730 0x7f472c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:36.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.950+0000 7f4732600700 1 -- 172.21.15.130:0/2686201943 >> 172.21.15.130:0/2686201943 conn(0x7f472c100140 msgr2=0x7f472c106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:36.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.950+0000 7f4732600700 1 -- 172.21.15.130:0/2686201943 shutdown_connections 2024-04-24T08:55:36.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:36.950+0000 7f4732600700 1 -- 172.21.15.130:0/2686201943 wait complete. 2024-04-24T08:55:36.951 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 2 2024-04-24T08:55:37.406 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 podman[30943]: 2024-04-24 08:55:37.149051946 +0000 UTC m=+0.173786642 container create 500bbac1214c29cf70c55ca09dac6ecba873f4511b29c675563a13624cb57d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-04-24T08:55:37.407 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 podman[30943]: 2024-04-24 08:55:37.404587323 +0000 UTC m=+0.429322025 container init 500bbac1214c29cf70c55ca09dac6ecba873f4511b29c675563a13624cb57d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-24T08:55:37.583 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:37 smithi130 ceph-mon[31553]: from='client.? 172.21.15.130:0/2686201943' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:55:37.583 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:37 smithi130 ceph-mon[31553]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 podman[30943]: 2024-04-24 08:55:37.41573652 +0000 UTC m=+0.440471189 container start 500bbac1214c29cf70c55ca09dac6ecba873f4511b29c675563a13624cb57d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-24T08:55:37.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T08:55:37.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev), process ceph-mon, pid 2 2024-04-24T08:55:37.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pidfile_write: ignore empty --pid-file 2024-04-24T08:55:37.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: load: jerasure load: lrc 2024-04-24T08:55:37.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: RocksDB version: 7.9.2 2024-04-24T08:55:37.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Git sha 0 2024-04-24T08:55:37.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Compile date 2024-04-23 21:06:33 2024-04-24T08:55:37.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: DB SUMMARY 2024-04-24T08:55:37.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: DB Session ID: TR9H0IBY7MT33DLOGYEG 2024-04-24T08:55:37.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: CURRENT file: CURRENT 2024-04-24T08:55:37.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T08:55:37.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-24T08:55:37.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-24T08:55:37.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-24T08:55:37.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.error_if_exists: 0 2024-04-24T08:55:37.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.create_if_missing: 0 2024-04-24T08:55:37.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.paranoid_checks: 1 2024-04-24T08:55:37.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T08:55:37.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T08:55:37.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T08:55:37.821 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.env: 0x55801dcbccc0 2024-04-24T08:55:37.821 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T08:55:37.821 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.info_log: 0x558036087c60 2024-04-24T08:55:37.821 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T08:55:37.821 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.statistics: (nil) 2024-04-24T08:55:37.821 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.use_fsync: 0 2024-04-24T08:55:37.821 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_log_file_size: 0 2024-04-24T08:55:37.821 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T08:55:37.821 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T08:55:37.821 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T08:55:37.821 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T08:55:37.821 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.allow_fallocate: 1 2024-04-24T08:55:37.822 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T08:55:37.822 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T08:55:37.822 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.use_direct_reads: 0 2024-04-24T08:55:37.822 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T08:55:37.822 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T08:55:37.822 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.db_log_dir: 2024-04-24T08:55:37.822 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.wal_dir: 2024-04-24T08:55:37.822 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T08:55:37.822 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T08:55:37.822 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T08:55:37.822 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T08:55:37.822 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T08:55:37.822 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T08:55:37.823 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T08:55:37.823 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T08:55:37.823 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.write_buffer_manager: 0x5580360561e0 2024-04-24T08:55:37.823 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T08:55:37.823 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T08:55:37.823 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T08:55:37.823 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T08:55:37.823 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T08:55:37.823 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T08:55:37.823 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T08:55:37.823 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T08:55:37.823 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.unordered_write: 0 2024-04-24T08:55:37.823 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T08:55:37.823 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T08:55:37.824 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T08:55:37.824 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T08:55:37.824 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.row_cache: None 2024-04-24T08:55:37.824 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.wal_filter: None 2024-04-24T08:55:37.824 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T08:55:37.824 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T08:55:37.824 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.two_write_queues: 0 2024-04-24T08:55:37.824 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T08:55:37.824 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.wal_compression: 0 2024-04-24T08:55:37.824 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.atomic_flush: 0 2024-04-24T08:55:37.824 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T08:55:37.824 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T08:55:37.824 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T08:55:37.825 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.log_readahead_size: 0 2024-04-24T08:55:37.825 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T08:55:37.825 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T08:55:37.825 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T08:55:37.825 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T08:55:37.825 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T08:55:37.825 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T08:55:37.825 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T08:55:37.825 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_background_jobs: 2 2024-04-24T08:55:37.825 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_background_compactions: -1 2024-04-24T08:55:37.825 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_subcompactions: 1 2024-04-24T08:55:37.826 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T08:55:37.826 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T08:55:37.826 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T08:55:37.826 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T08:55:37.826 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T08:55:37.826 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T08:55:37.826 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T08:55:37.826 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T08:55:37.826 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_open_files: -1 2024-04-24T08:55:37.826 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T08:55:37.826 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T08:55:37.826 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T08:55:37.826 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T08:55:37.827 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_background_flushes: -1 2024-04-24T08:55:37.827 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Compression algorithms supported: 2024-04-24T08:55:37.827 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T08:55:37.827 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: kZSTD supported: 0 2024-04-24T08:55:37.827 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: kXpressCompression supported: 0 2024-04-24T08:55:37.827 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T08:55:37.827 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: kZlibCompression supported: 1 2024-04-24T08:55:37.827 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: kSnappyCompression supported: 1 2024-04-24T08:55:37.827 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: kLZ4Compression supported: 1 2024-04-24T08:55:37.827 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: kBZip2Compression supported: 0 2024-04-24T08:55:37.827 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T08:55:37.827 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T08:55:37.827 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-24T08:55:37.827 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T08:55:37.828 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T08:55:37.828 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.merge_operator: 2024-04-24T08:55:37.828 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compaction_filter: None 2024-04-24T08:55:37.828 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T08:55:37.828 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T08:55:37.828 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T08:55:37.828 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T08:55:37.829 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558036087e20) 2024-04-24T08:55:37.829 INFO:journalctl@ceph.mon.b.smithi081.stdout: cache_index_and_filter_blocks: 1 2024-04-24T08:55:37.829 INFO:journalctl@ceph.mon.b.smithi081.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T08:55:37.829 INFO:journalctl@ceph.mon.b.smithi081.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T08:55:37.829 INFO:journalctl@ceph.mon.b.smithi081.stdout: pin_top_level_index_and_filter: 1 2024-04-24T08:55:37.829 INFO:journalctl@ceph.mon.b.smithi081.stdout: index_type: 0 2024-04-24T08:55:37.829 INFO:journalctl@ceph.mon.b.smithi081.stdout: data_block_index_type: 0 2024-04-24T08:55:37.829 INFO:journalctl@ceph.mon.b.smithi081.stdout: index_shortening: 1 2024-04-24T08:55:37.829 INFO:journalctl@ceph.mon.b.smithi081.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T08:55:37.829 INFO:journalctl@ceph.mon.b.smithi081.stdout: checksum: 4 2024-04-24T08:55:37.829 INFO:journalctl@ceph.mon.b.smithi081.stdout: no_block_cache: 0 2024-04-24T08:55:37.829 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_cache: 0x558036089090 2024-04-24T08:55:37.829 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_cache_name: BinnedLRUCache 2024-04-24T08:55:37.829 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_cache_options: 2024-04-24T08:55:37.830 INFO:journalctl@ceph.mon.b.smithi081.stdout: capacity : 536870912 2024-04-24T08:55:37.830 INFO:journalctl@ceph.mon.b.smithi081.stdout: num_shard_bits : 4 2024-04-24T08:55:37.830 INFO:journalctl@ceph.mon.b.smithi081.stdout: strict_capacity_limit : 0 2024-04-24T08:55:37.830 INFO:journalctl@ceph.mon.b.smithi081.stdout: high_pri_pool_ratio: 0.000 2024-04-24T08:55:37.830 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_cache_compressed: (nil) 2024-04-24T08:55:37.830 INFO:journalctl@ceph.mon.b.smithi081.stdout: persistent_cache: (nil) 2024-04-24T08:55:37.830 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_size: 4096 2024-04-24T08:55:37.830 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_size_deviation: 10 2024-04-24T08:55:37.830 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_restart_interval: 16 2024-04-24T08:55:37.830 INFO:journalctl@ceph.mon.b.smithi081.stdout: index_block_restart_interval: 1 2024-04-24T08:55:37.830 INFO:journalctl@ceph.mon.b.smithi081.stdout: metadata_block_size: 4096 2024-04-24T08:55:37.830 INFO:journalctl@ceph.mon.b.smithi081.stdout: partition_filters: 0 2024-04-24T08:55:37.830 INFO:journalctl@ceph.mon.b.smithi081.stdout: use_delta_encoding: 1 2024-04-24T08:55:37.831 INFO:journalctl@ceph.mon.b.smithi081.stdout: filter_policy: bloomfilter 2024-04-24T08:55:37.831 INFO:journalctl@ceph.mon.b.smithi081.stdout: whole_key_filtering: 1 2024-04-24T08:55:37.831 INFO:journalctl@ceph.mon.b.smithi081.stdout: verify_compression: 0 2024-04-24T08:55:37.831 INFO:journalctl@ceph.mon.b.smithi081.stdout: read_amp_bytes_per_bit: 0 2024-04-24T08:55:37.831 INFO:journalctl@ceph.mon.b.smithi081.stdout: format_version: 5 2024-04-24T08:55:37.831 INFO:journalctl@ceph.mon.b.smithi081.stdout: enable_index_compression: 1 2024-04-24T08:55:37.831 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_align: 0 2024-04-24T08:55:37.831 INFO:journalctl@ceph.mon.b.smithi081.stdout: max_auto_readahead_size: 262144 2024-04-24T08:55:37.831 INFO:journalctl@ceph.mon.b.smithi081.stdout: prepopulate_block_cache: 0 2024-04-24T08:55:37.831 INFO:journalctl@ceph.mon.b.smithi081.stdout: initial_auto_readahead_size: 8192 2024-04-24T08:55:37.831 INFO:journalctl@ceph.mon.b.smithi081.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T08:55:37.831 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T08:55:37.831 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T08:55:37.831 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compression: NoCompression 2024-04-24T08:55:37.832 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T08:55:37.832 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T08:55:37.832 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T08:55:37.832 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.num_levels: 7 2024-04-24T08:55:37.832 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T08:55:37.832 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T08:55:37.832 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T08:55:37.832 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T08:55:37.832 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T08:55:37.832 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T08:55:37.832 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T08:55:37.832 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:55:37.832 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T08:55:37.833 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T08:55:37.833 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:55:37.833 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:55:37.833 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T08:55:37.833 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T08:55:37.833 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T08:55:37.833 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T08:55:37.833 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:55:37.833 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:55:37.833 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T08:55:37.833 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T08:55:37.833 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:55:37.833 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T08:55:37.833 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T08:55:37.834 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T08:55:37.834 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T08:55:37.834 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T08:55:37.834 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T08:55:37.834 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T08:55:37.834 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T08:55:37.834 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T08:55:37.834 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T08:55:37.834 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T08:55:37.834 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T08:55:37.834 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T08:55:37.834 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T08:55:37.834 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T08:55:37.834 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T08:55:37.835 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T08:55:37.835 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T08:55:37.835 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T08:55:37.835 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T08:55:37.835 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T08:55:37.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T08:55:37.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T08:55:37.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T08:55:37.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T08:55:37.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T08:55:37.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T08:55:37.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T08:55:37.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T08:55:37.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T08:55:37.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T08:55:37.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T08:55:37.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-24T08:55:37.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.inplace_update_support: 0 2024-04-24T08:55:37.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T08:55:37.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T08:55:37.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T08:55:37.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T08:55:37.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.bloom_locality: 0 2024-04-24T08:55:37.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.max_successive_merges: 0 2024-04-24T08:55:37.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T08:55:37.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T08:55:37.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T08:55:37.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T08:55:37.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.ttl: 2592000 2024-04-24T08:55:37.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T08:55:37.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T08:55:37.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T08:55:37.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.enable_blob_files: false 2024-04-24T08:55:37.838 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.min_blob_size: 0 2024-04-24T08:55:37.838 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T08:55:37.838 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T08:55:37.838 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T08:55:37.838 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T08:55:37.838 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T08:55:37.838 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T08:55:37.838 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T08:55:37.838 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T08:55:37.838 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T08:55:37.838 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-24T08:55:37.838 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f6dab784-2f61-4fb9-bfdd-c39855c62ccd 2024-04-24T08:55:37.838 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713948937473194, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-24T08:55:37.838 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-24T08:55:37.839 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713948937474579, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713948937, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f6dab784-2f61-4fb9-bfdd-c39855c62ccd", "db_session_id": "TR9H0IBY7MT33DLOGYEG", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-24T08:55:37.839 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713948937474842, "job": 1, "event": "recovery_finished"} 2024-04-24T08:55:37.839 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-24T08:55:37.839 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T08:55:37.839 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558036176000 2024-04-24T08:55:37.839 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: DB pointer 0x558036160000 2024-04-24T08:55:37.839 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T08:55:37.839 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T08:55:37.839 INFO:journalctl@ceph.mon.b.smithi081.stdout: ** DB Stats ** 2024-04-24T08:55:37.839 INFO:journalctl@ceph.mon.b.smithi081.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:55:37.839 INFO:journalctl@ceph.mon.b.smithi081.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T08:55:37.839 INFO:journalctl@ceph.mon.b.smithi081.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:55:37.839 INFO:journalctl@ceph.mon.b.smithi081.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:55:37.839 INFO:journalctl@ceph.mon.b.smithi081.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T08:55:37.839 INFO:journalctl@ceph.mon.b.smithi081.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:55:37.840 INFO:journalctl@ceph.mon.b.smithi081.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:55:37.840 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2024-04-24T08:55:37.840 INFO:journalctl@ceph.mon.b.smithi081.stdout: ** Compaction Stats [default] ** 2024-04-24T08:55:37.840 INFO:journalctl@ceph.mon.b.smithi081.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:55:37.840 INFO:journalctl@ceph.mon.b.smithi081.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T08:55:37.840 INFO:journalctl@ceph.mon.b.smithi081.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:55:37.840 INFO:journalctl@ceph.mon.b.smithi081.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:55:37.840 INFO:journalctl@ceph.mon.b.smithi081.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:55:37.840 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2024-04-24T08:55:37.840 INFO:journalctl@ceph.mon.b.smithi081.stdout: ** Compaction Stats [default] ** 2024-04-24T08:55:37.840 INFO:journalctl@ceph.mon.b.smithi081.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:55:37.840 INFO:journalctl@ceph.mon.b.smithi081.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T08:55:37.840 INFO:journalctl@ceph.mon.b.smithi081.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:55:37.841 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2024-04-24T08:55:37.841 INFO:journalctl@ceph.mon.b.smithi081.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T08:55:37.841 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2024-04-24T08:55:37.841 INFO:journalctl@ceph.mon.b.smithi081.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:55:37.841 INFO:journalctl@ceph.mon.b.smithi081.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T08:55:37.841 INFO:journalctl@ceph.mon.b.smithi081.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T08:55:37.841 INFO:journalctl@ceph.mon.b.smithi081.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T08:55:37.841 INFO:journalctl@ceph.mon.b.smithi081.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T08:55:37.841 INFO:journalctl@ceph.mon.b.smithi081.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T08:55:37.841 INFO:journalctl@ceph.mon.b.smithi081.stdout: Cumulative compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:55:37.841 INFO:journalctl@ceph.mon.b.smithi081.stdout: Interval compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:55:37.841 INFO:journalctl@ceph.mon.b.smithi081.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T08:55:37.841 INFO:journalctl@ceph.mon.b.smithi081.stdout: Block cache BinnedLRUCache@0x558036089090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-24T08:55:37.841 INFO:journalctl@ceph.mon.b.smithi081.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-24T08:55:37.841 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2024-04-24T08:55:37.842 INFO:journalctl@ceph.mon.b.smithi081.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T08:55:37.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-24T08:55:37.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: using public_addr v2:172.21.15.81:0/0 -> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] 2024-04-24T08:55:37.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: starting mon.b rank -1 at public addrs [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] at bind addrs [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:55:37.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.b@-1(???) e0 preinit fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:55:37.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 bash[30943]: 500bbac1214c29cf70c55ca09dac6ecba873f4511b29c675563a13624cb57d64 2024-04-24T08:55:37.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 systemd[1]: Started Ceph mon.b for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T08:55:37.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.b@-1(synchronizing).mds e1 new map 2024-04-24T08:55:37.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-24T08:55:37.842 INFO:journalctl@ceph.mon.b.smithi081.stdout: e1 2024-04-24T08:55:37.842 INFO:journalctl@ceph.mon.b.smithi081.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:55:37.842 INFO:journalctl@ceph.mon.b.smithi081.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T08:55:37.842 INFO:journalctl@ceph.mon.b.smithi081.stdout: legacy client fscid: -1 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout: No filesystems configured 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi081", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: Deploying cephadm binary to smithi081 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.844 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.844 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: Added host smithi081 2024-04-24T08:55:37.844 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:55:37.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:37 smithi069 ceph-mon[28170]: from='client.? 172.21.15.130:0/2686201943' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:55:37.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:37 smithi069 ceph-mon[28170]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.845 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.845 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.845 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.845 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.845 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:55:37.845 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.845 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.845 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.845 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi130", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:55:37.845 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: Deploying cephadm binary to smithi130 2024-04-24T08:55:37.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi081", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:55:37.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:37.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:55:37.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: Updating smithi081:/etc/ceph/ceph.conf 2024-04-24T08:55:37.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: Updating smithi081:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:55:37.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.846 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:55:37.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: Added host smithi130 2024-04-24T08:55:37.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: Updating smithi081:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:55:37.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:55:37.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:55:37.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.847 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3913175972' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T08:55:37.848 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.848 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3913175972' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T08:55:37.848 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:55:37.848 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.848 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.848 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.848 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.848 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.848 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi130", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:55:37.848 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:37.848 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:55:37.848 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-24T08:55:37.848 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.849 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: Updating smithi130:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:55:37.849 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi081:172.21.15.81=b;smithi130:172.21.15.130=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:55:37.849 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: Saving service mon spec with placement smithi069:172.21.15.69=a;smithi081:172.21.15.81=b;smithi130:172.21.15.130=c;count:3 2024-04-24T08:55:37.849 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.849 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:55:37.849 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: Updating smithi130:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:55:37.849 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.849 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.849 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.849 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.849 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:55:37.849 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:37.849 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: Deploying daemon mon.c on smithi130 2024-04-24T08:55:37.849 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='client.? 172.21.15.130:0/1049631722' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:55:37.849 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.849 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.850 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.850 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:55:37.850 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.a calling monitor election 2024-04-24T08:55:37.850 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:37.850 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.850 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:37.850 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:37.850 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.c calling monitor election 2024-04-24T08:55:37.850 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.850 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:37.850 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:37.850 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.850 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:37.851 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T08:55:37.851 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: monmap epoch 2 2024-04-24T08:55:37.851 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:55:37.851 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: last_changed 2024-04-24T08:55:26.528456+0000 2024-04-24T08:55:37.851 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: created 2024-04-24T08:53:11.516571+0000 2024-04-24T08:55:37.851 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: min_mon_release 19 (squid) 2024-04-24T08:55:37.851 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: election_strategy: 1 2024-04-24T08:55:37.851 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-04-24T08:55:37.851 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-04-24T08:55:37.851 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: fsmap 2024-04-24T08:55:37.851 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:55:37.851 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mgrmap e13: x(active, since 68s) 2024-04-24T08:55:37.851 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: overall HEALTH_OK 2024-04-24T08:55:37.851 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.851 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.852 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:37.852 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:55:37.852 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:37.852 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: Deploying daemon mon.b on smithi081 2024-04-24T08:55:37.852 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='client.? 172.21.15.130:0/1526172146' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:55:37.852 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:37.852 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.852 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.852 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: from='client.? 172.21.15.130:0/2686201943' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:55:37.852 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:37.852 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:37 smithi081 ceph-mon[30961]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-04-24T08:55:38.652 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T08:55:38.653 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:55:39.090 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:55:42.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:55:42.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: mon.a calling monitor election 2024-04-24T08:55:42.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: mon.c calling monitor election 2024-04-24T08:55:42.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:42.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:42.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:42.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:42.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:42.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: mon.b calling monitor election 2024-04-24T08:55:42.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:42.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:42.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:42.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:42.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T08:55:42.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: monmap epoch 3 2024-04-24T08:55:42.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:55:42.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: last_changed 2024-04-24T08:55:37.527038+0000 2024-04-24T08:55:42.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: created 2024-04-24T08:53:11.516571+0000 2024-04-24T08:55:42.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: min_mon_release 19 (squid) 2024-04-24T08:55:42.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: election_strategy: 1 2024-04-24T08:55:42.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-04-24T08:55:42.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-04-24T08:55:42.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: 2: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-04-24T08:55:42.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: fsmap 2024-04-24T08:55:42.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:55:42.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: mgrmap e13: x(active, since 79s) 2024-04-24T08:55:42.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: overall HEALTH_OK 2024-04-24T08:55:42.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:42.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:43.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:55:43.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: mon.a calling monitor election 2024-04-24T08:55:43.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: mon.c calling monitor election 2024-04-24T08:55:43.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:43.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:43.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:43.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:43.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:43.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: mon.b calling monitor election 2024-04-24T08:55:43.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:43.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:43.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:43.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:43.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T08:55:43.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: monmap epoch 3 2024-04-24T08:55:43.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:55:43.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: last_changed 2024-04-24T08:55:37.527038+0000 2024-04-24T08:55:43.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: created 2024-04-24T08:53:11.516571+0000 2024-04-24T08:55:43.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: min_mon_release 19 (squid) 2024-04-24T08:55:43.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: election_strategy: 1 2024-04-24T08:55:43.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-04-24T08:55:43.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-04-24T08:55:43.071 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: 2: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-04-24T08:55:43.071 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: fsmap 2024-04-24T08:55:43.071 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:55:43.071 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: mgrmap e13: x(active, since 79s) 2024-04-24T08:55:43.071 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: overall HEALTH_OK 2024-04-24T08:55:43.071 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:43.071 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:43.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:55:43.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: mon.a calling monitor election 2024-04-24T08:55:43.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: mon.c calling monitor election 2024-04-24T08:55:43.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:43.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:55:43.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:43.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:43.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:43.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: mon.b calling monitor election 2024-04-24T08:55:43.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:43.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:43.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:43.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:43.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T08:55:43.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: monmap epoch 3 2024-04-24T08:55:43.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:55:43.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: last_changed 2024-04-24T08:55:37.527038+0000 2024-04-24T08:55:43.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: created 2024-04-24T08:53:11.516571+0000 2024-04-24T08:55:43.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: min_mon_release 19 (squid) 2024-04-24T08:55:43.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: election_strategy: 1 2024-04-24T08:55:43.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-04-24T08:55:43.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-04-24T08:55:43.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: 2: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2024-04-24T08:55:43.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: fsmap 2024-04-24T08:55:43.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:55:43.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: mgrmap e13: x(active, since 79s) 2024-04-24T08:55:43.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: overall HEALTH_OK 2024-04-24T08:55:43.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:43.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:43.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:43 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:43.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:43 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:43.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:43 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:55:43.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:43 smithi130 ceph-mon[31553]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:43.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:43 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:43.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:43 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:44.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:43 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:44.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:43 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:44.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:43 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:55:44.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:43 smithi081 ceph-mon[30961]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:44.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:43 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:44.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:43 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:44.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:44.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:44.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:55:44.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:43 smithi069 ceph-mon[28170]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:44.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:44.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:55:44.136 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.136+0000 7f6924200700 1 -- 172.21.15.130:0/1869766866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c108660 msgr2=0x7f6904005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:44.136 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.136+0000 7f6924200700 1 --2- 172.21.15.130:0/1869766866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c108660 0x7f6904005610 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f691000fc90 tx=0x7f691000ffa0 comp rx=0 tx=0).stop 2024-04-24T08:55:44.136 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.136+0000 7f6924200700 1 -- 172.21.15.130:0/1869766866 shutdown_connections 2024-04-24T08:55:44.137 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.136+0000 7f6924200700 1 --2- 172.21.15.130:0/1869766866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c108660 0x7f6904005610 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:44.137 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.136+0000 7f6924200700 1 -- 172.21.15.130:0/1869766866 >> 172.21.15.130:0/1869766866 conn(0x7f691c0fe0c0 msgr2=0x7f691c1004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:44.137 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.136+0000 7f6924200700 1 -- 172.21.15.130:0/1869766866 shutdown_connections 2024-04-24T08:55:44.137 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.136+0000 7f6924200700 1 -- 172.21.15.130:0/1869766866 wait complete. 2024-04-24T08:55:44.137 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.138+0000 7f6924200700 1 Processor -- start 2024-04-24T08:55:44.138 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.138+0000 7f6924200700 1 -- start start 2024-04-24T08:55:44.138 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.139+0000 7f6924200700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f691c108660 0x7f691c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:44.139 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.139+0000 7f6924200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f691c19a850 0x7f691c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:44.139 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.139+0000 7f6924200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c19f120 0x7f691c19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:44.139 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.139+0000 7f6924200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f691c10a540 con 0x7f691c19f120 2024-04-24T08:55:44.139 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.139+0000 7f6924200700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f691c10a3c0 con 0x7f691c108660 2024-04-24T08:55:44.139 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.139+0000 7f6924200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f691c10a6c0 con 0x7f691c19a850 2024-04-24T08:55:44.141 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.139+0000 7f6922200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c19f120 0x7f691c19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:44.141 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.139+0000 7f6922200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c19f120 0x7f691c19f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.130:59514/0 (socket says 172.21.15.130:59514) 2024-04-24T08:55:44.141 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.139+0000 7f6922200700 1 -- 172.21.15.130:0/3511249610 learned_addr learned my addr 172.21.15.130:0/3511249610 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-24T08:55:44.141 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.140+0000 7f6920e00700 1 --2- 172.21.15.130:0/3511249610 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f691c19a850 0x7f691c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:44.141 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.140+0000 7f6921800700 1 --2- 172.21.15.130:0/3511249610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f691c108660 0x7f691c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:44.141 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.140+0000 7f6922200700 1 -- 172.21.15.130:0/3511249610 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f691c19a850 msgr2=0x7f691c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:44.141 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.140+0000 7f6922200700 1 --2- 172.21.15.130:0/3511249610 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f691c19a850 0x7f691c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:44.141 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.140+0000 7f6922200700 1 -- 172.21.15.130:0/3511249610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f691c108660 msgr2=0x7f691c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:44.141 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.140+0000 7f6922200700 1 --2- 172.21.15.130:0/3511249610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f691c108660 0x7f691c19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:44.142 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.140+0000 7f6922200700 1 -- 172.21.15.130:0/3511249610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c1a6e90 con 0x7f691c19f120 2024-04-24T08:55:44.142 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.140+0000 7f6922200700 1 --2- 172.21.15.130:0/3511249610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c19f120 0x7f691c19f5d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f691400ba50 tx=0x7f691400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:44.142 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.141+0000 7f691a000700 1 -- 172.21.15.130:0/3511249610 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914019070 con 0x7f691c19f120 2024-04-24T08:55:44.142 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.141+0000 7f6924200700 1 -- 172.21.15.130:0/3511249610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691c1a70c0 con 0x7f691c19f120 2024-04-24T08:55:44.142 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.141+0000 7f691a000700 1 -- 172.21.15.130:0/3511249610 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6914014440 con 0x7f691c19f120 2024-04-24T08:55:44.142 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.141+0000 7f691a000700 1 -- 172.21.15.130:0/3511249610 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691400ce60 con 0x7f691c19f120 2024-04-24T08:55:44.142 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.141+0000 7f6924200700 1 -- 172.21.15.130:0/3511249610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691c1a76d0 con 0x7f691c19f120 2024-04-24T08:55:44.143 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.142+0000 7f6924200700 1 -- 172.21.15.130:0/3511249610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691c064150 con 0x7f691c19f120 2024-04-24T08:55:44.143 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.143+0000 7f691a000700 1 -- 172.21.15.130:0/3511249610 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f691401c070 con 0x7f691c19f120 2024-04-24T08:55:44.143 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.143+0000 7f691a000700 1 --2- 172.21.15.130:0/3511249610 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f68fc039ae0 0x7f68fc03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:44.143 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.143+0000 7f691a000700 1 -- 172.21.15.130:0/3511249610 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f691404bda0 con 0x7f691c19f120 2024-04-24T08:55:44.143 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.144+0000 7f6921800700 1 --2- 172.21.15.130:0/3511249610 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f68fc039ae0 0x7f68fc03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:44.144 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.145+0000 7f6921800700 1 --2- 172.21.15.130:0/3511249610 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f68fc039ae0 0x7f68fc03bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6910010010 tx=0x7f691001b4f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:44.161 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.161+0000 7f691a000700 1 -- 172.21.15.130:0/3511249610 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6914014980 con 0x7f691c19f120 2024-04-24T08:55:44.477 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.477+0000 7f6924200700 1 -- 172.21.15.130:0/3511249610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f691c02dfe0 con 0x7f691c19f120 2024-04-24T08:55:44.478 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.478+0000 7f691a000700 1 -- 172.21.15.130:0/3511249610 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f691401c520 con 0x7f691c19f120 2024-04-24T08:55:44.478 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-24T08:55:44.478 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":3,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","modified":"2024-04-24T08:55:37.527038Z","created":"2024-04-24T08:53:11.516571Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:3300","nonce":0},{"type":"v1","addr":"172.21.15.130:6789","nonce":0}]},"addr":"172.21.15.130:6789/0","public_addr":"172.21.15.130:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-24T08:55:44.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.481+0000 7f6924200700 1 -- 172.21.15.130:0/3511249610 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f68fc039ae0 msgr2=0x7f68fc03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:44.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.481+0000 7f6924200700 1 --2- 172.21.15.130:0/3511249610 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f68fc039ae0 0x7f68fc03bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6910010010 tx=0x7f691001b4f0 comp rx=0 tx=0).stop 2024-04-24T08:55:44.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.481+0000 7f6924200700 1 -- 172.21.15.130:0/3511249610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c19f120 msgr2=0x7f691c19f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:44.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.481+0000 7f6924200700 1 --2- 172.21.15.130:0/3511249610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c19f120 0x7f691c19f5d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f691400ba50 tx=0x7f691400bd60 comp rx=0 tx=0).stop 2024-04-24T08:55:44.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.481+0000 7f6924200700 1 -- 172.21.15.130:0/3511249610 shutdown_connections 2024-04-24T08:55:44.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.481+0000 7f6924200700 1 --2- 172.21.15.130:0/3511249610 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f68fc039ae0 0x7f68fc03bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:44.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.481+0000 7f6924200700 1 --2- 172.21.15.130:0/3511249610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f691c19f120 0x7f691c19f5d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:44.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.481+0000 7f6924200700 1 --2- 172.21.15.130:0/3511249610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f691c108660 0x7f691c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:44.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.481+0000 7f6924200700 1 --2- 172.21.15.130:0/3511249610 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f691c19a850 0x7f691c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:44.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.481+0000 7f6924200700 1 -- 172.21.15.130:0/3511249610 >> 172.21.15.130:0/3511249610 conn(0x7f691c0fe0c0 msgr2=0x7f691c075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:44.482 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.481+0000 7f6924200700 1 -- 172.21.15.130:0/3511249610 shutdown_connections 2024-04-24T08:55:44.482 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:44.481+0000 7f6924200700 1 -- 172.21.15.130:0/3511249610 wait complete. 2024-04-24T08:55:44.482 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 3 2024-04-24T08:55:44.797 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:44 smithi130 ceph-mon[31553]: from='client.? 172.21.15.130:0/3511249610' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:55:45.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:44 smithi081 ceph-mon[30961]: from='client.? 172.21.15.130:0/3511249610' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:55:45.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:44 smithi069 ceph-mon[28170]: from='client.? 172.21.15.130:0/3511249610' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:55:45.124 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-24T08:55:45.125 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-24T08:55:45.560 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:55:46.046 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:45 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:46.046 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:45 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:46.046 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:45 smithi069 ceph-mon[28170]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:46.046 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:45 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:46.046 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:45 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:46.046 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:45 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:46.046 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:45 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:55:46.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:45 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:46.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:45 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:46.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:45 smithi081 ceph-mon[30961]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:46.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:45 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:46.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:45 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:46.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:45 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:46.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:45 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:55:46.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:45 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:46.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:45 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:46.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:45 smithi130 ceph-mon[31553]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:46.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:45 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:46.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:45 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:46.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:45 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:46.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:45 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:55:47.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:46 smithi069 ceph-mon[28170]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-24T08:55:47.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:46 smithi069 ceph-mon[28170]: Updating smithi081:/etc/ceph/ceph.conf 2024-04-24T08:55:47.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:46 smithi069 ceph-mon[28170]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-24T08:55:47.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:46 smithi069 ceph-mon[28170]: Updating smithi069:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:55:47.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:46 smithi069 ceph-mon[28170]: Updating smithi130:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:55:47.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:46 smithi069 ceph-mon[28170]: Updating smithi081:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:55:47.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:47.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:47.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:46 smithi130 ceph-mon[31553]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-24T08:55:47.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:46 smithi130 ceph-mon[31553]: Updating smithi081:/etc/ceph/ceph.conf 2024-04-24T08:55:47.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:46 smithi130 ceph-mon[31553]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-24T08:55:47.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:46 smithi130 ceph-mon[31553]: Updating smithi069:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:55:47.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:46 smithi130 ceph-mon[31553]: Updating smithi130:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:55:47.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:46 smithi130 ceph-mon[31553]: Updating smithi081:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:55:47.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:47.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:47.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:46 smithi081 ceph-mon[30961]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-24T08:55:47.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:46 smithi081 ceph-mon[30961]: Updating smithi081:/etc/ceph/ceph.conf 2024-04-24T08:55:47.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:46 smithi081 ceph-mon[30961]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-24T08:55:47.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:46 smithi081 ceph-mon[30961]: Updating smithi069:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:55:47.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:46 smithi081 ceph-mon[30961]: Updating smithi130:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:55:47.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:46 smithi081 ceph-mon[30961]: Updating smithi081:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:55:47.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:47.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:47.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.737+0000 7fa97c000700 1 -- 172.21.15.69:0/175222011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9741085d0 msgr2=0x7fa9741089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:47.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.737+0000 7fa97c000700 1 --2- 172.21.15.69:0/175222011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9741085d0 0x7fa9741089b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa968009910 tx=0x7fa968009c20 comp rx=0 tx=0).stop 2024-04-24T08:55:47.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.737+0000 7fa97c000700 1 -- 172.21.15.69:0/175222011 shutdown_connections 2024-04-24T08:55:47.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.737+0000 7fa97c000700 1 --2- 172.21.15.69:0/175222011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9741085d0 0x7fa9741089b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.737+0000 7fa97c000700 1 -- 172.21.15.69:0/175222011 >> 172.21.15.69:0/175222011 conn(0x7fa9740fe030 msgr2=0x7fa974100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:47.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.737+0000 7fa97c000700 1 -- 172.21.15.69:0/175222011 shutdown_connections 2024-04-24T08:55:47.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.737+0000 7fa97c000700 1 -- 172.21.15.69:0/175222011 wait complete. 2024-04-24T08:55:47.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.738+0000 7fa97c000700 1 Processor -- start 2024-04-24T08:55:47.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.739+0000 7fa97c000700 1 -- start start 2024-04-24T08:55:47.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.739+0000 7fa97c000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9741085d0 0x7fa974074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.739+0000 7fa97c000700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa974075330 0x7fa9740757b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.739+0000 7fa97c000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9741a4340 0x7fa9741a6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.739+0000 7fa97c000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa97410ca40 con 0x7fa9741085d0 2024-04-24T08:55:47.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.739+0000 7fa97c000700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa97410c8c0 con 0x7fa974075330 2024-04-24T08:55:47.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.739+0000 7fa97c000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa97410cbc0 con 0x7fa9741a4340 2024-04-24T08:55:47.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.739+0000 7fa979600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9741085d0 0x7fa974074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.739+0000 7fa979600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9741085d0 0x7fa974074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56112/0 (socket says 172.21.15.69:56112) 2024-04-24T08:55:47.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.739+0000 7fa979600700 1 -- 172.21.15.69:0/1436108758 learned_addr learned my addr 172.21.15.69:0/1436108758 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:55:47.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.740+0000 7fa978c00700 1 --2- 172.21.15.69:0/1436108758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa974075330 0x7fa9740757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.740+0000 7fa979600700 1 -- 172.21.15.69:0/1436108758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9741a4340 msgr2=0x7fa9741a6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:47.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.740+0000 7fa97a000700 1 --2- 172.21.15.69:0/1436108758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9741a4340 0x7fa9741a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.740+0000 7fa979600700 1 --2- 172.21.15.69:0/1436108758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9741a4340 0x7fa9741a6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.740+0000 7fa979600700 1 -- 172.21.15.69:0/1436108758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa974075330 msgr2=0x7fa9740757b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:47.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.740+0000 7fa979600700 1 --2- 172.21.15.69:0/1436108758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa974075330 0x7fa9740757b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.740+0000 7fa979600700 1 -- 172.21.15.69:0/1436108758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9741a6e60 con 0x7fa9741085d0 2024-04-24T08:55:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.740+0000 7fa978c00700 1 --2- 172.21.15.69:0/1436108758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa974075330 0x7fa9740757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.740+0000 7fa97a000700 1 --2- 172.21.15.69:0/1436108758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9741a4340 0x7fa9741a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.740+0000 7fa979600700 1 --2- 172.21.15.69:0/1436108758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9741085d0 0x7fa974074df0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa968005b40 tx=0x7fa968005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.741+0000 7fa972000700 1 -- 172.21.15.69:0/1436108758 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968026070 con 0x7fa9741085d0 2024-04-24T08:55:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.741+0000 7fa972000700 1 -- 172.21.15.69:0/1436108758 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa96800b750 con 0x7fa9741085d0 2024-04-24T08:55:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.741+0000 7fa97c000700 1 -- 172.21.15.69:0/1436108758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9741a70f0 con 0x7fa9741085d0 2024-04-24T08:55:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.741+0000 7fa972000700 1 -- 172.21.15.69:0/1436108758 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9680038e0 con 0x7fa9741085d0 2024-04-24T08:55:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.741+0000 7fa97c000700 1 -- 172.21.15.69:0/1436108758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9741a75d0 con 0x7fa9741085d0 2024-04-24T08:55:47.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.742+0000 7fa97c000700 1 -- 172.21.15.69:0/1436108758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa974064150 con 0x7fa9741085d0 2024-04-24T08:55:47.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.743+0000 7fa972000700 1 -- 172.21.15.69:0/1436108758 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa968029070 con 0x7fa9741085d0 2024-04-24T08:55:47.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.743+0000 7fa972000700 1 --2- 172.21.15.69:0/1436108758 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa95c039ae0 0x7fa95c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.743+0000 7fa972000700 1 -- 172.21.15.69:0/1436108758 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa96804d6f0 con 0x7fa9741085d0 2024-04-24T08:55:47.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.743+0000 7fa978c00700 1 --2- 172.21.15.69:0/1436108758 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa95c039ae0 0x7fa95c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.744+0000 7fa978c00700 1 --2- 172.21.15.69:0/1436108758 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa95c039ae0 0x7fa95c03bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa9600079b0 tx=0x7fa960008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:47.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.749+0000 7fa972000700 1 -- 172.21.15.69:0/1436108758 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa968013070 con 0x7fa9741085d0 2024-04-24T08:55:47.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.977+0000 7fa97c000700 1 -- 172.21.15.69:0/1436108758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa9741a78f0 con 0x7fa9741085d0 2024-04-24T08:55:47.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.978+0000 7fa972000700 1 -- 172.21.15.69:0/1436108758 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fa96800bc30 con 0x7fa9741085d0 2024-04-24T08:55:47.978 INFO:teuthology.orchestra.run.smithi069.stdout:# minimal ceph.conf for ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:55:47.978 INFO:teuthology.orchestra.run.smithi069.stdout:[global] 2024-04-24T08:55:47.979 INFO:teuthology.orchestra.run.smithi069.stdout: fsid = ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T08:55:47.979 INFO:teuthology.orchestra.run.smithi069.stdout: mon_host = [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] 2024-04-24T08:55:47.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.981+0000 7fa97c000700 1 -- 172.21.15.69:0/1436108758 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa95c039ae0 msgr2=0x7fa95c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:47.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.981+0000 7fa97c000700 1 --2- 172.21.15.69:0/1436108758 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa95c039ae0 0x7fa95c03bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa9600079b0 tx=0x7fa960008040 comp rx=0 tx=0).stop 2024-04-24T08:55:47.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.981+0000 7fa97c000700 1 -- 172.21.15.69:0/1436108758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9741085d0 msgr2=0x7fa974074df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:47.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.981+0000 7fa97c000700 1 --2- 172.21.15.69:0/1436108758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9741085d0 0x7fa974074df0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa968005b40 tx=0x7fa968005030 comp rx=0 tx=0).stop 2024-04-24T08:55:47.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.981+0000 7fa97c000700 1 -- 172.21.15.69:0/1436108758 shutdown_connections 2024-04-24T08:55:47.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.981+0000 7fa97c000700 1 --2- 172.21.15.69:0/1436108758 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa95c039ae0 0x7fa95c03bfa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.981+0000 7fa97c000700 1 --2- 172.21.15.69:0/1436108758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9741a4340 0x7fa9741a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.981+0000 7fa97c000700 1 --2- 172.21.15.69:0/1436108758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9741085d0 0x7fa974074df0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.981+0000 7fa97c000700 1 --2- 172.21.15.69:0/1436108758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa974075330 0x7fa9740757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.981+0000 7fa97c000700 1 -- 172.21.15.69:0/1436108758 >> 172.21.15.69:0/1436108758 conn(0x7fa9740fe030 msgr2=0x7fa9740fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:47.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.982+0000 7fa97c000700 1 -- 172.21.15.69:0/1436108758 shutdown_connections 2024-04-24T08:55:47.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:55:47.982+0000 7fa97c000700 1 -- 172.21.15.69:0/1436108758 wait complete. 2024-04-24T08:55:48.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:47 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:47 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:47 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:47 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.205 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:47 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.205 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:47 smithi130 ceph-mon[31553]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:48.205 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:47 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.205 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:47 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.205 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:47 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.205 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:47 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.205 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:47 smithi130 ceph-mon[31553]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T08:55:48.205 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:47 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:55:48.205 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:47 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:55:48.205 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:47 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:48.205 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:47 smithi130 ceph-mon[31553]: Reconfiguring daemon mon.a on smithi069 2024-04-24T08:55:48.275 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:47 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.275 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:47 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.275 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:47 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.275 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:47 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.275 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:47 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.275 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:47 smithi069 ceph-mon[28170]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:48.275 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:47 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.275 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:47 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.275 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:47 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.275 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:47 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.275 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:47 smithi069 ceph-mon[28170]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T08:55:48.276 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:47 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:55:48.276 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:47 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:55:48.276 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:47 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:48.276 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:47 smithi069 ceph-mon[28170]: Reconfiguring daemon mon.a on smithi069 2024-04-24T08:55:48.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:47 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:47 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:47 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:47 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:47 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:47 smithi081 ceph-mon[30961]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:48.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:47 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:47 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:47 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:47 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:48.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:47 smithi081 ceph-mon[30961]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T08:55:48.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:47 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:55:48.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:47 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:55:48.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:47 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:48.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:47 smithi081 ceph-mon[30961]: Reconfiguring daemon mon.a on smithi069 2024-04-24T08:55:48.688 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-24T08:55:48.688 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T08:55:48.688 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T08:55:48.753 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T08:55:48.753 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:55:48.865 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-04-24T08:55:48.865 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T08:55:48.922 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:48 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1436108758' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:48.955 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-04-24T08:55:48.956 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:55:49.083 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-24T08:55:49.083 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T08:55:49.159 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:48 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1436108758' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:49.172 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-24T08:55:49.172 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:55:49.301 INFO:tasks.cephadm:Adding mgr.x on smithi069 2024-04-24T08:55:49.301 INFO:tasks.cephadm:Adding mgr.y on smithi081 2024-04-24T08:55:49.301 INFO:tasks.cephadm:Adding mgr.z on smithi130 2024-04-24T08:55:49.301 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi069=x;smithi081=y;smithi130=z' 2024-04-24T08:55:49.316 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:48 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1436108758' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:49.769 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:55:50.125 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:49 smithi130 ceph-mon[31553]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:50.316 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:49 smithi081 ceph-mon[30961]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:50.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:49 smithi069 ceph-mon[28170]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:51.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:51 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:51.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:51 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:51.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:51 smithi130 ceph-mon[31553]: Reconfiguring mon.b (monmap changed)... 2024-04-24T08:55:51.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:51 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:55:51.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:51 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:55:51.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:51 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:51.205 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:51 smithi130 ceph-mon[31553]: Reconfiguring daemon mon.b on smithi081 2024-04-24T08:55:51.209 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:51 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:51.209 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:51 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:51.209 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:51 smithi081 ceph-mon[30961]: Reconfiguring mon.b (monmap changed)... 2024-04-24T08:55:51.210 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:51 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:55:51.210 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:51 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:55:51.210 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:51 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:51.210 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:51 smithi081 ceph-mon[30961]: Reconfiguring daemon mon.b on smithi081 2024-04-24T08:55:51.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:51 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:51.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:51 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:51.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:51 smithi069 ceph-mon[28170]: Reconfiguring mon.b (monmap changed)... 2024-04-24T08:55:51.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:51 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:55:51.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:51 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:55:51.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:51 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:51.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:51 smithi069 ceph-mon[28170]: Reconfiguring daemon mon.b on smithi081 2024-04-24T08:55:52.016 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.016+0000 7f94f5200700 1 -- 172.21.15.130:0/992911836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94f0102650 msgr2=0x7f94f0102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:52.016 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.016+0000 7f94f5200700 1 --2- 172.21.15.130:0/992911836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94f0102650 0x7f94f0102a70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009910 tx=0x7f94e0009c20 comp rx=0 tx=0).stop 2024-04-24T08:55:52.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.016+0000 7f94f5200700 1 -- 172.21.15.130:0/992911836 shutdown_connections 2024-04-24T08:55:52.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.016+0000 7f94f5200700 1 --2- 172.21.15.130:0/992911836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94f0102650 0x7f94f0102a70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:52.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.016+0000 7f94f5200700 1 -- 172.21.15.130:0/992911836 >> 172.21.15.130:0/992911836 conn(0x7f94f00fdbf0 msgr2=0x7f94f0100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:52.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.016+0000 7f94f5200700 1 -- 172.21.15.130:0/992911836 shutdown_connections 2024-04-24T08:55:52.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.016+0000 7f94f5200700 1 -- 172.21.15.130:0/992911836 wait complete. 2024-04-24T08:55:52.018 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.018+0000 7f94f5200700 1 Processor -- start 2024-04-24T08:55:52.018 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.018+0000 7f94f5200700 1 -- start start 2024-04-24T08:55:52.019 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.019+0000 7f94f5200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94f0102650 0x7f94f0197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:52.019 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.019+0000 7f94f5200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94f01982e0 0x7f94f019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:52.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.019+0000 7f94f5200700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94f0198760 0x7f94f019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:52.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.019+0000 7f94f5200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f94f0108590 con 0x7f94f0102650 2024-04-24T08:55:52.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.019+0000 7f94f5200700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f94f0108410 con 0x7f94f0198760 2024-04-24T08:55:52.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.019+0000 7f94f5200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f94f0108710 con 0x7f94f01982e0 2024-04-24T08:55:52.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.019+0000 7f94e7e00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94f01982e0 0x7f94f019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:52.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.019+0000 7f94e7e00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94f01982e0 0x7f94f019d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:34584/0 (socket says 172.21.15.130:34584) 2024-04-24T08:55:52.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.019+0000 7f94e7e00700 1 -- 172.21.15.130:0/469032035 learned_addr learned my addr 172.21.15.130:0/469032035 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-24T08:55:52.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.019+0000 7f94ede00700 1 --2- 172.21.15.130:0/469032035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94f0102650 0x7f94f0197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:52.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.020+0000 7f94ee800700 1 --2- 172.21.15.130:0/469032035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94f0198760 0x7f94f019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:52.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.020+0000 7f94e7e00700 1 -- 172.21.15.130:0/469032035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94f0198760 msgr2=0x7f94f019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:52.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.020+0000 7f94e7e00700 1 --2- 172.21.15.130:0/469032035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94f0198760 0x7f94f019f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:52.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.020+0000 7f94e7e00700 1 -- 172.21.15.130:0/469032035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94f0102650 msgr2=0x7f94f0197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:52.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.020+0000 7f94e7e00700 1 --2- 172.21.15.130:0/469032035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94f0102650 0x7f94f0197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:52.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.020+0000 7f94e7e00700 1 -- 172.21.15.130:0/469032035 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f01a01b0 con 0x7f94f01982e0 2024-04-24T08:55:52.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.020+0000 7f94ee800700 1 --2- 172.21.15.130:0/469032035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94f0198760 0x7f94f019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:52.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.020+0000 7f94e7e00700 1 --2- 172.21.15.130:0/469032035 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94f01982e0 0x7f94f019d350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f94d800b780 tx=0x7f94d800bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:52.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.020+0000 7f94e6a00700 1 -- 172.21.15.130:0/469032035 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8004230 con 0x7f94f01982e0 2024-04-24T08:55:52.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.020+0000 7f94f5200700 1 -- 172.21.15.130:0/469032035 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f0074d80 con 0x7f94f01982e0 2024-04-24T08:55:52.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.020+0000 7f94e6a00700 1 -- 172.21.15.130:0/469032035 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94d80043d0 con 0x7f94f01982e0 2024-04-24T08:55:52.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.020+0000 7f94e6a00700 1 -- 172.21.15.130:0/469032035 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8011b80 con 0x7f94f01982e0 2024-04-24T08:55:52.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.021+0000 7f94f5200700 1 -- 172.21.15.130:0/469032035 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f0075390 con 0x7f94f01982e0 2024-04-24T08:55:52.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.021+0000 7f94f5200700 1 -- 172.21.15.130:0/469032035 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f0064150 con 0x7f94f01982e0 2024-04-24T08:55:52.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.022+0000 7f94e6a00700 1 -- 172.21.15.130:0/469032035 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f94d80036a0 con 0x7f94f01982e0 2024-04-24T08:55:52.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.022+0000 7f94e6a00700 1 --2- 172.21.15.130:0/469032035 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f94d4039ae0 0x7f94d403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:52.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.022+0000 7f94e6a00700 1 -- 172.21.15.130:0/469032035 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f94d804c510 con 0x7f94f01982e0 2024-04-24T08:55:52.023 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.023+0000 7f94ede00700 1 --2- 172.21.15.130:0/469032035 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f94d4039ae0 0x7f94d403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:52.023 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.024+0000 7f94ede00700 1 --2- 172.21.15.130:0/469032035 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f94d4039ae0 0x7f94d403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f94e00050f0 tx=0x7f94e001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:52.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.029+0000 7f94e6a00700 1 -- 172.21.15.130:0/469032035 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f94d8021e00 con 0x7f94f01982e0 2024-04-24T08:55:52.125 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:52 smithi081 ceph-mon[30961]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:52.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:52 smithi130 ceph-mon[31553]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:52.268 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.268+0000 7f94f5200700 1 -- 172.21.15.130:0/469032035 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi081=y;smithi130=z", "target": ["mon-mgr", ""]}) -- 0x7f94f0062fc0 con 0x7f94d4039ae0 2024-04-24T08:55:52.284 INFO:teuthology.orchestra.run.smithi130.stdout:Scheduled mgr update... 2024-04-24T08:55:52.284 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.284+0000 7f94e6a00700 1 -- 172.21.15.130:0/469032035 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f94f0062fc0 con 0x7f94d4039ae0 2024-04-24T08:55:52.287 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.287+0000 7f94f5200700 1 -- 172.21.15.130:0/469032035 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f94d4039ae0 msgr2=0x7f94d403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:52.287 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.287+0000 7f94f5200700 1 --2- 172.21.15.130:0/469032035 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f94d4039ae0 0x7f94d403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f94e00050f0 tx=0x7f94e001a040 comp rx=0 tx=0).stop 2024-04-24T08:55:52.287 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.287+0000 7f94f5200700 1 -- 172.21.15.130:0/469032035 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94f01982e0 msgr2=0x7f94f019d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:52.287 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.287+0000 7f94f5200700 1 --2- 172.21.15.130:0/469032035 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94f01982e0 0x7f94f019d350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f94d800b780 tx=0x7f94d800bb40 comp rx=0 tx=0).stop 2024-04-24T08:55:52.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.287+0000 7f94f5200700 1 -- 172.21.15.130:0/469032035 shutdown_connections 2024-04-24T08:55:52.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.287+0000 7f94f5200700 1 --2- 172.21.15.130:0/469032035 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f94d4039ae0 0x7f94d403bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:52.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.287+0000 7f94f5200700 1 --2- 172.21.15.130:0/469032035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94f0198760 0x7f94f019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:52.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.287+0000 7f94f5200700 1 --2- 172.21.15.130:0/469032035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94f0102650 0x7f94f0197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:52.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.287+0000 7f94f5200700 1 --2- 172.21.15.130:0/469032035 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94f01982e0 0x7f94f019d350 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:52.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.287+0000 7f94f5200700 1 -- 172.21.15.130:0/469032035 >> 172.21.15.130:0/469032035 conn(0x7f94f00fdbf0 msgr2=0x7f94f00fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:52.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.287+0000 7f94f5200700 1 -- 172.21.15.130:0/469032035 shutdown_connections 2024-04-24T08:55:52.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:55:52.288+0000 7f94f5200700 1 -- 172.21.15.130:0/469032035 wait complete. 2024-04-24T08:55:52.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:52 smithi069 ceph-mon[28170]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:53.096 DEBUG:teuthology.orchestra.run.smithi081:mgr.y> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mgr.y.service 2024-04-24T08:55:53.104 DEBUG:teuthology.orchestra.run.smithi130:mgr.z> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mgr.z.service 2024-04-24T08:55:53.110 INFO:tasks.cephadm:Deploying OSDs... 2024-04-24T08:55:53.110 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T08:55:53.110 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T08:55:53.165 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T08:55:53.165 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_1 2024-04-24T08:55:53.215 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T08:55:53.215 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:55:53.215 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-24T08:55:53.215 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:55:53.215 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:55:53.215 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-24 08:54:36.333580093 +0000 2024-04-24T08:55:53.215 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-24 08:48:28.796031273 +0000 2024-04-24T08:55:53.215 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-24 08:48:28.796031273 +0000 2024-04-24T08:55:53.215 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-04-24 08:48:28.796031273 +0000 2024-04-24T08:55:53.216 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T08:55:53.299 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-04-24T08:55:53.299 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-04-24T08:55:53.299 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.00093288 s, 549 kB/s 2024-04-24T08:55:53.302 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T08:55:53.308 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:53 smithi130 ceph-mon[31553]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi081=y;smithi130=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:55:53.308 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:53 smithi130 ceph-mon[31553]: Saving service mgr spec with placement smithi069=x;smithi081=y;smithi130=z;count:3 2024-04-24T08:55:53.308 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:53.308 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:53.308 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:53.308 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:53 smithi130 ceph-mon[31553]: Reconfiguring mon.c (monmap changed)... 2024-04-24T08:55:53.308 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:55:53.308 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:55:53.308 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:53.309 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:53 smithi130 ceph-mon[31553]: Reconfiguring daemon mon.c on smithi130 2024-04-24T08:55:53.355 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_2 2024-04-24T08:55:53.405 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T08:55:53.405 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:55:53.405 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-24T08:55:53.405 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:55:53.405 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:55:53.405 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-24 08:54:36.401578496 +0000 2024-04-24T08:55:53.405 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-24 08:48:29.413016591 +0000 2024-04-24T08:55:53.405 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-24 08:48:29.413016591 +0000 2024-04-24T08:55:53.405 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-04-24 08:48:29.413016591 +0000 2024-04-24T08:55:53.405 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T08:55:53.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:53 smithi069 ceph-mon[28170]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi081=y;smithi130=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:55:53.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:53 smithi069 ceph-mon[28170]: Saving service mgr spec with placement smithi069=x;smithi081=y;smithi130=z;count:3 2024-04-24T08:55:53.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:53.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:53.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:53.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:53 smithi069 ceph-mon[28170]: Reconfiguring mon.c (monmap changed)... 2024-04-24T08:55:53.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:55:53.477 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:55:53.477 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:53.477 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:53 smithi069 ceph-mon[28170]: Reconfiguring daemon mon.c on smithi130 2024-04-24T08:55:53.484 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-04-24T08:55:53.484 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-04-24T08:55:53.485 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000444298 s, 1.2 MB/s 2024-04-24T08:55:53.489 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T08:55:53.541 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_3 2024-04-24T08:55:53.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:53 smithi081 ceph-mon[30961]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi081=y;smithi130=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:55:53.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:53 smithi081 ceph-mon[30961]: Saving service mgr spec with placement smithi069=x;smithi081=y;smithi130=z;count:3 2024-04-24T08:55:53.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:53.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:53.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:53.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:53 smithi081 ceph-mon[30961]: Reconfiguring mon.c (monmap changed)... 2024-04-24T08:55:53.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:55:53.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:55:53.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:53.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:53 smithi081 ceph-mon[30961]: Reconfiguring daemon mon.c on smithi130 2024-04-24T08:55:53.587 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T08:55:53.587 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:55:53.587 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-24T08:55:53.587 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:55:53.587 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:55:53.587 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-24 08:54:36.453577274 +0000 2024-04-24T08:55:53.587 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-24 08:48:29.997002700 +0000 2024-04-24T08:55:53.587 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-24 08:48:29.997002700 +0000 2024-04-24T08:55:53.587 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-04-24 08:48:29.997002700 +0000 2024-04-24T08:55:53.588 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T08:55:53.659 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-04-24T08:55:53.659 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-04-24T08:55:53.659 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000359021 s, 1.4 MB/s 2024-04-24T08:55:53.663 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T08:55:53.714 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_4 2024-04-24T08:55:53.763 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T08:55:53.763 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:55:53.764 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-24T08:55:53.764 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:55:53.764 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:55:53.764 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-24 08:54:36.508575982 +0000 2024-04-24T08:55:53.764 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-24 08:48:30.581988784 +0000 2024-04-24T08:55:53.764 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-24 08:48:30.581988784 +0000 2024-04-24T08:55:53.764 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-04-24 08:48:30.581988784 +0000 2024-04-24T08:55:53.764 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T08:55:53.842 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-04-24T08:55:53.843 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-04-24T08:55:53.843 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000368819 s, 1.4 MB/s 2024-04-24T08:55:53.847 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T08:55:53.900 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-04-24T08:55:53.901 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T08:55:53.960 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T08:55:53.960 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_1 2024-04-24T08:55:54.013 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T08:55:54.013 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:55:54.013 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-24T08:55:54.013 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:55:54.013 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:55:54.013 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-04-24 08:55:36.585892205 +0000 2024-04-24T08:55:54.013 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-04-24 08:48:29.718568937 +0000 2024-04-24T08:55:54.013 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-04-24 08:48:29.718568937 +0000 2024-04-24T08:55:54.013 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-04-24 08:48:29.718568937 +0000 2024-04-24T08:55:54.014 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T08:55:54.090 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-04-24T08:55:54.090 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-04-24T08:55:54.090 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.00035751 s, 1.4 MB/s 2024-04-24T08:55:54.093 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T08:55:54.148 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_2 2024-04-24T08:55:54.195 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T08:55:54.196 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:55:54.196 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 6h/6d Inode: 655 Links: 1 2024-04-24T08:55:54.196 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:55:54.196 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:55:54.196 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-04-24 08:55:36.586892187 +0000 2024-04-24T08:55:54.196 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-04-24 08:48:30.292557990 +0000 2024-04-24T08:55:54.196 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-04-24 08:48:30.292557990 +0000 2024-04-24T08:55:54.196 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-04-24 08:48:30.292557990 +0000 2024-04-24T08:55:54.196 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T08:55:54.267 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-04-24T08:55:54.268 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-04-24T08:55:54.268 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000392885 s, 1.3 MB/s 2024-04-24T08:55:54.270 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T08:55:54.324 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_3 2024-04-24T08:55:54.373 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T08:55:54.373 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:55:54.373 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-24T08:55:54.373 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:55:54.373 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:55:54.373 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-04-24 08:55:36.586892187 +0000 2024-04-24T08:55:54.373 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-04-24 08:48:30.900546394 +0000 2024-04-24T08:55:54.373 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-04-24 08:48:30.900546394 +0000 2024-04-24T08:55:54.373 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-04-24 08:48:30.900546394 +0000 2024-04-24T08:55:54.374 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T08:55:54.441 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:54 smithi081 ceph-mon[30961]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:54.448 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-04-24T08:55:54.449 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-04-24T08:55:54.449 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.00035731 s, 1.4 MB/s 2024-04-24T08:55:54.452 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T08:55:54.505 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_4 2024-04-24T08:55:54.554 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T08:55:54.554 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:55:54.554 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-24T08:55:54.554 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:55:54.554 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:55:54.554 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-04-24 08:55:36.587892168 +0000 2024-04-24T08:55:54.555 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-04-24 08:48:31.493535084 +0000 2024-04-24T08:55:54.555 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-04-24 08:48:31.493535084 +0000 2024-04-24T08:55:54.555 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-04-24 08:48:31.493535084 +0000 2024-04-24T08:55:54.555 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T08:55:54.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:54 smithi069 ceph-mon[28170]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:54.633 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-04-24T08:55:54.633 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-04-24T08:55:54.633 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000374637 s, 1.4 MB/s 2024-04-24T08:55:54.638 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T08:55:54.694 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-24T08:55:54.694 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T08:55:54.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:54 smithi130 ceph-mon[31553]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:54.740 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T08:55:54.740 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_1 2024-04-24T08:55:54.832 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T08:55:54.832 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:55:54.832 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-24T08:55:54.832 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:55:54.832 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:55:54.832 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-04-24 08:55:25.154989410 +0000 2024-04-24T08:55:54.832 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-04-24 08:48:29.921833379 +0000 2024-04-24T08:55:54.832 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-04-24 08:48:29.921833379 +0000 2024-04-24T08:55:54.832 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-04-24 08:48:29.921833379 +0000 2024-04-24T08:55:54.832 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T08:55:54.893 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-04-24T08:55:54.894 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-04-24T08:55:54.894 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000321879 s, 1.6 MB/s 2024-04-24T08:55:54.897 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T08:55:54.945 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_2 2024-04-24T08:55:54.986 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T08:55:54.986 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:55:54.986 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-04-24T08:55:54.986 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:55:54.986 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:55:54.986 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-04-24 08:55:25.155989392 +0000 2024-04-24T08:55:54.986 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-04-24 08:48:30.507822545 +0000 2024-04-24T08:55:54.986 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-04-24 08:48:30.507822545 +0000 2024-04-24T08:55:54.986 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-04-24 08:48:30.507822545 +0000 2024-04-24T08:55:54.986 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T08:55:55.099 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-04-24T08:55:55.099 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-04-24T08:55:55.099 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000346757 s, 1.5 MB/s 2024-04-24T08:55:55.103 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T08:55:55.153 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_3 2024-04-24T08:55:55.195 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T08:55:55.195 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:55:55.195 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-24T08:55:55.195 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:55:55.195 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:55:55.195 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-04-24 08:55:25.156989374 +0000 2024-04-24T08:55:55.195 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-04-24 08:48:31.071812119 +0000 2024-04-24T08:55:55.195 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-04-24 08:48:31.071812119 +0000 2024-04-24T08:55:55.195 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-04-24 08:48:31.071812119 +0000 2024-04-24T08:55:55.196 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T08:55:55.307 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-04-24T08:55:55.307 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-04-24T08:55:55.307 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000307856 s, 1.7 MB/s 2024-04-24T08:55:55.310 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T08:55:55.354 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_4 2024-04-24T08:55:55.432 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T08:55:55.433 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:55:55.433 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-24T08:55:55.433 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:55:55.433 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:55:55.433 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-04-24 08:55:25.156989374 +0000 2024-04-24T08:55:55.433 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-04-24 08:48:31.657801285 +0000 2024-04-24T08:55:55.433 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-04-24 08:48:31.657801285 +0000 2024-04-24T08:55:55.433 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-04-24 08:48:31.657801285 +0000 2024-04-24T08:55:55.433 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T08:55:55.542 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-04-24T08:55:55.542 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-04-24T08:55:55.542 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000365488 s, 1.4 MB/s 2024-04-24T08:55:55.545 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T08:55:55.597 INFO:tasks.cephadm:Deploying osd.0 on smithi069 with /dev/vg_nvme/lv_4... 2024-04-24T08:55:55.597 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T08:55:55.612 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:55 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:55.612 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:55 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:55.612 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:55 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:55:55.613 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:55 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:55.613 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:55 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:55:55.613 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:55 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:55.613 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:55 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:55:55.613 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:55 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T08:55:55.613 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:55 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:55:55.613 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:55 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:55.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:55 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:55.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:55 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:55.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:55 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:55:55.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:55 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:55.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:55 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:55:55.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:55 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:55.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:55 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:55:55.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:55 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T08:55:55.676 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:55 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:55:55.677 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:55 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:55.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:55 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:55.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:55 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:55.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:55 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:55:55.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:55 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:55.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:55 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:55:55.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:55 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:55.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:55 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:55:55.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:55 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T08:55:55.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:55 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:55:55.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:55 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:56.033 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:55:56.506 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:56 smithi130 ceph-mon[31553]: Deploying daemon mgr.z on smithi130 2024-04-24T08:55:56.506 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:56 smithi130 ceph-mon[31553]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:56.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:56 smithi069 ceph-mon[28170]: Deploying daemon mgr.z on smithi130 2024-04-24T08:55:56.552 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:56 smithi069 ceph-mon[28170]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:56.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:56 smithi081 ceph-mon[30961]: Deploying daemon mgr.z on smithi130 2024-04-24T08:55:56.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:56 smithi081 ceph-mon[30961]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:57.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:57 smithi069 ceph-mon[28170]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:57.661 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:57 smithi130 ceph-mon[31553]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:57.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:57 smithi081 ceph-mon[30961]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:55:58.305 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:55:58 smithi130 systemd[1]: Started Ceph mgr.z for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T08:55:58.563 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:55:58 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:55:58.304+0000 7f843d981200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T08:55:58.564 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:55:58 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:55:58.563+0000 7f843d981200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:55:58.841 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T08:55:58.842 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T08:55:58.842 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-04-24T08:55:58.842 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-04-24T08:55:58.842 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0358107 s, 293 MB/s 2024-04-24T08:55:58.842 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-04-24T08:55:58.953 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:55:58 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:55:58.701+0000 7f843d981200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T08:55:59.453 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:55:59 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:55:59.098+0000 7f843d981200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T08:55:59.453 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:55:59 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:55:59.208+0000 7f843d981200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T08:55:59.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:59.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:59.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:59.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:55:59.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T08:55:59.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:55:59.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:59.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:55:59 smithi130 ceph-mon[31553]: Deploying daemon mgr.y on smithi081 2024-04-24T08:55:59.550 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:59.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:59.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:59.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:55:59.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T08:55:59.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:55:59.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:59.551 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:55:59 smithi069 ceph-mon[28170]: Deploying daemon mgr.y on smithi081 2024-04-24T08:55:59.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:59.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:59.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:55:59.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:55:59.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T08:55:59.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:55:59.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:59.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:55:59 smithi081 ceph-mon[30961]: Deploying daemon mgr.y on smithi081 2024-04-24T08:55:59.608 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_4 2024-04-24T08:56:00.038 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:56:00.415 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:00 smithi081 ceph-mon[30961]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:00.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:00 smithi069 ceph-mon[28170]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:00.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:00 smithi130 ceph-mon[31553]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:01.151 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:00 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:00.789+0000 7f843d981200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T08:56:01.151 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:01.022+0000 7f843d981200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T08:56:01.453 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:01.246+0000 7f843d981200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T08:56:01.494 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:01 smithi081 systemd[1]: Starting Ceph mgr.y for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T08:56:01.755 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:01 smithi081 podman[32322]: 2024-04-24 08:56:01.40410431 +0000 UTC m=+0.043224298 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:56:01.756 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:01 smithi081 podman[32322]: 2024-04-24 08:56:01.509012018 +0000 UTC m=+0.148132036 container create 86e9cccf0aaa04a29533f7030aee547b1f4202d3e2d05c571da4936c7ab54895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-24T08:56:01.756 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:01 smithi081 podman[32322]: 2024-04-24 08:56:01.754667221 +0000 UTC m=+0.393787242 container init 86e9cccf0aaa04a29533f7030aee547b1f4202d3e2d05c571da4936c7ab54895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:56:01.953 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:01.636+0000 7f843d981200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T08:56:01.953 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:01.759+0000 7f843d981200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T08:56:02.067 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:01 smithi081 podman[32322]: 2024-04-24 08:56:01.764921668 +0000 UTC m=+0.404041649 container start 86e9cccf0aaa04a29533f7030aee547b1f4202d3e2d05c571da4936c7ab54895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-24T08:56:02.067 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:01 smithi081 ceph-mgr[32341]: -- 172.21.15.81:0/1137497443 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d057a4a5a0 con 0x55d057a1ec00 2024-04-24T08:56:02.067 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:01 smithi081 bash[32322]: 86e9cccf0aaa04a29533f7030aee547b1f4202d3e2d05c571da4936c7ab54895 2024-04-24T08:56:02.067 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:01 smithi081 systemd[1]: Started Ceph mgr.y for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T08:56:02.067 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:02 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:02.000+0000 7f005df3b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T08:56:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.106+0000 7fcadba00700 1 -- 172.21.15.69:0/663160736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcad4106900 msgr2=0x7fcad410d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.106+0000 7fcadba00700 1 --2- 172.21.15.69:0/663160736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcad4106900 0x7fcad410d1b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00b0d0 tx=0x7fcacc00b3e0 comp rx=0 tx=0).stop 2024-04-24T08:56:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.106+0000 7fcadba00700 1 -- 172.21.15.69:0/663160736 shutdown_connections 2024-04-24T08:56:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.106+0000 7fcadba00700 1 --2- 172.21.15.69:0/663160736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcad4106900 0x7fcad410d1b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.106+0000 7fcadba00700 1 --2- 172.21.15.69:0/663160736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcad4104d60 0x7fcad4105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.106+0000 7fcadba00700 1 --2- 172.21.15.69:0/663160736 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcad4105f60 0x7fcad41063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:02.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.106+0000 7fcadba00700 1 -- 172.21.15.69:0/663160736 >> 172.21.15.69:0/663160736 conn(0x7fcad4100300 msgr2=0x7fcad4102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.106+0000 7fcadba00700 1 -- 172.21.15.69:0/663160736 shutdown_connections 2024-04-24T08:56:02.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.106+0000 7fcadba00700 1 -- 172.21.15.69:0/663160736 wait complete. 2024-04-24T08:56:02.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.107+0000 7fcadba00700 1 Processor -- start 2024-04-24T08:56:02.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.108+0000 7fcadba00700 1 -- start start 2024-04-24T08:56:02.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.108+0000 7fcadba00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcad4104d60 0x7fcad419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:02.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.108+0000 7fcadba00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcad4105f60 0x7fcad419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:02.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.108+0000 7fcadba00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcad4106900 0x7fcad41a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:02.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.108+0000 7fcadba00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4110850 con 0x7fcad4104d60 2024-04-24T08:56:02.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.108+0000 7fcadba00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41106d0 con 0x7fcad4106900 2024-04-24T08:56:02.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.108+0000 7fcadba00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41109d0 con 0x7fcad4105f60 2024-04-24T08:56:02.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.108+0000 7fcad9000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcad4104d60 0x7fcad419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:02.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.108+0000 7fcad9000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcad4104d60 0x7fcad419ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47002/0 (socket says 172.21.15.69:47002) 2024-04-24T08:56:02.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.108+0000 7fcad9000700 1 -- 172.21.15.69:0/757990422 learned_addr learned my addr 172.21.15.69:0/757990422 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:56:02.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.109+0000 7fcad3e00700 1 --2- 172.21.15.69:0/757990422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcad4105f60 0x7fcad419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:02.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.109+0000 7fcad9a00700 1 --2- 172.21.15.69:0/757990422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcad4106900 0x7fcad41a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:02.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.109+0000 7fcad9000700 1 -- 172.21.15.69:0/757990422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcad4105f60 msgr2=0x7fcad419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:02.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.109+0000 7fcad9000700 1 --2- 172.21.15.69:0/757990422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcad4105f60 0x7fcad419b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:02.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.109+0000 7fcad9000700 1 -- 172.21.15.69:0/757990422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcad4106900 msgr2=0x7fcad41a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:02.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.109+0000 7fcad9000700 1 --2- 172.21.15.69:0/757990422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcad4106900 0x7fcad41a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:02.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.109+0000 7fcad9000700 1 -- 172.21.15.69:0/757990422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad41a2aa0 con 0x7fcad4104d60 2024-04-24T08:56:02.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.109+0000 7fcad9a00700 1 --2- 172.21.15.69:0/757990422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcad4106900 0x7fcad41a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:56:02.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.109+0000 7fcad9000700 1 --2- 172.21.15.69:0/757990422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcad4104d60 0x7fcad419ae40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcac800d9b0 tx=0x7fcac800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:02.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.110+0000 7fcad1600700 1 -- 172.21.15.69:0/757990422 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8017030 con 0x7fcad4104d60 2024-04-24T08:56:02.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.110+0000 7fcad1600700 1 -- 172.21.15.69:0/757990422 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcac80044d0 con 0x7fcad4104d60 2024-04-24T08:56:02.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.110+0000 7fcadba00700 1 -- 172.21.15.69:0/757990422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad41a2d90 con 0x7fcad4104d60 2024-04-24T08:56:02.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.110+0000 7fcad1600700 1 -- 172.21.15.69:0/757990422 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac80039e0 con 0x7fcad4104d60 2024-04-24T08:56:02.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.110+0000 7fcadba00700 1 -- 172.21.15.69:0/757990422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad41a3240 con 0x7fcad4104d60 2024-04-24T08:56:02.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.111+0000 7fcadba00700 1 -- 172.21.15.69:0/757990422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad402dfe0 con 0x7fcad4104d60 2024-04-24T08:56:02.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.112+0000 7fcad1600700 1 -- 172.21.15.69:0/757990422 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fcac801a070 con 0x7fcad4104d60 2024-04-24T08:56:02.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.112+0000 7fcad1600700 1 --2- 172.21.15.69:0/757990422 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcabc039890 0x7fcabc03bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:02.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.112+0000 7fcad1600700 1 -- 172.21.15.69:0/757990422 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcac804c310 con 0x7fcad4104d60 2024-04-24T08:56:02.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.112+0000 7fcad3e00700 1 --2- 172.21.15.69:0/757990422 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcabc039890 0x7fcabc03bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:02.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.113+0000 7fcad3e00700 1 --2- 172.21.15.69:0/757990422 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcabc039890 0x7fcabc03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009c60 tx=0x7fcac0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:02.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.118+0000 7fcad1600700 1 -- 172.21.15.69:0/757990422 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcac800f3c0 con 0x7fcad4104d60 2024-04-24T08:56:02.275 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:02 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:02.020+0000 7f843d981200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T08:56:02.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:02.372+0000 7fcadba00700 1 -- 172.21.15.69:0/757990422 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fcad4062f30 con 0x7fcabc039890 2024-04-24T08:56:02.393 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:02 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:02.260+0000 7f005df3b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:56:02.393 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:02 smithi081 ceph-mon[30961]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:02.393 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:02 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:02.393 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:02 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:02.393 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:02 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:02.393 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:02 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:02.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:02 smithi069 ceph-mon[28170]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:02.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:02.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:02.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:02.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:02.703 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:02 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:02.275+0000 7f843d981200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:56:02.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:02 smithi130 ceph-mon[31553]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:02.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:02 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:02.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:02 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:02.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:02 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:02.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:02 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:02.809 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:02 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:02.392+0000 7f005df3b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T08:56:03.066 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:02 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:02.809+0000 7f005df3b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T08:56:03.067 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:02 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:02.925+0000 7f005df3b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T08:56:03.431 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:03 smithi069 ceph-mon[28170]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:56:03.431 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:03 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:56:03.431 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:03 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:56:03.431 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:03 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:03.431 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:03 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:03.431 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:03 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:03.453 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:03 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:03.093+0000 7f843d981200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T08:56:03.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:03 smithi130 ceph-mon[31553]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:56:03.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:03 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:56:03.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:03 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:56:03.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:03 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:03.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:03 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:03.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:03 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:03.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:03 smithi081 ceph-mon[30961]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:56:03.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:03 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:56:03.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:03 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:56:03.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:03 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:03.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:03 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:03.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:03 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:03.953 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:03 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:03.673+0000 7f843d981200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T08:56:03.953 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:03 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:03.803+0000 7f843d981200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T08:56:04.415 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:04 smithi069 ceph-mon[28170]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:04.415 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:04 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:04.416 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:04 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:04.416 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:04 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:04.416 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:04 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:04.454 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:04 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:04.079+0000 7f843d981200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T08:56:04.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:04 smithi130 ceph-mon[31553]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:04.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:04 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:04.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:04 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:04.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:04 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:04.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:04 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:04.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:04 smithi081 ceph-mon[30961]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:04.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:04 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:04.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:04 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:04.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:04 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:04.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:04 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:05.046 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:04 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:04.593+0000 7f005df3b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T08:56:05.046 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:04 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:04.826+0000 7f005df3b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T08:56:05.046 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:05 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:05.045+0000 7f005df3b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T08:56:05.167 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:04 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:04.729+0000 7f843d981200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T08:56:05.167 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:04 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:04.853+0000 7f843d981200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T08:56:05.167 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:05 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:05.166+0000 7f843d981200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T08:56:05.453 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:05 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:05.342+0000 7f843d981200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T08:56:05.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:05 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:05.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:05 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:56:05.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:05 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:05.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:05 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:05.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:05 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:56:05.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:05 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:05.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:05 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:05.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:05 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:56:05.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:05 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:05.817 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:05 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:05.435+0000 7f005df3b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T08:56:05.817 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:05 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:05.563+0000 7f005df3b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T08:56:05.862 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:05 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:05.468+0000 7f843d981200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T08:56:05.862 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:05 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:05.619+0000 7f843d981200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T08:56:06.097 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:05 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:05.837+0000 7f005df3b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T08:56:06.097 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:06 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:06.097+0000 7f005df3b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:56:06.203 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:05 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:05.993+0000 7f843d981200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T08:56:06.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:06 smithi081 ceph-mon[30961]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:06.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:56:06.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:56:06.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:06.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:06 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2146798031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c221fcb-ab5f-4f37-b5b4-40366081fbb9"}]: dispatch 2024-04-24T08:56:06.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:06 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2146798031' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c221fcb-ab5f-4f37-b5b4-40366081fbb9"}]': finished 2024-04-24T08:56:06.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:06 smithi081 ceph-mon[30961]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T08:56:06.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:56:06.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:06 smithi069 ceph-mon[28170]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:06.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:56:06.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:56:06.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:06.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:06 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2146798031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c221fcb-ab5f-4f37-b5b4-40366081fbb9"}]: dispatch 2024-04-24T08:56:06.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:06 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2146798031' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c221fcb-ab5f-4f37-b5b4-40366081fbb9"}]': finished 2024-04-24T08:56:06.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:06 smithi069 ceph-mon[28170]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T08:56:06.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:56:06.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:06 smithi130 ceph-mon[31553]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:06.727 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:56:06.727 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:56:06.727 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:06.728 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:06 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2146798031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c221fcb-ab5f-4f37-b5b4-40366081fbb9"}]: dispatch 2024-04-24T08:56:06.728 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:06 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2146798031' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c221fcb-ab5f-4f37-b5b4-40366081fbb9"}]': finished 2024-04-24T08:56:06.728 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:06 smithi130 ceph-mon[31553]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T08:56:06.728 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:56:07.028 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:06 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:06.745+0000 7f843d981200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T08:56:07.317 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:06 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:06.878+0000 7f005df3b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T08:56:07.346 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 08:56:07 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T08:56:07.028+0000 7f843d981200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T08:56:07.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:07.355+0000 7fcad1600700 1 -- 172.21.15.69:0/757990422 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7fcac8003b80 con 0x7fcad4104d60 2024-04-24T08:56:07.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:07 smithi130 ceph-mon[31553]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T08:56:07.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:07 smithi130 ceph-mon[31553]: Reconfiguring daemon mgr.x on smithi069 2024-04-24T08:56:07.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:07 smithi130 ceph-mon[31553]: Standby manager daemon z started 2024-04-24T08:56:07.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:07 smithi130 ceph-mon[31553]: from='mgr.? 172.21.15.130:0/2241268869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T08:56:07.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:07 smithi130 ceph-mon[31553]: from='mgr.? 172.21.15.130:0/2241268869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:56:07.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:07 smithi130 ceph-mon[31553]: from='mgr.? 172.21.15.130:0/2241268869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T08:56:07.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:07 smithi130 ceph-mon[31553]: from='mgr.? 172.21.15.130:0/2241268869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:56:07.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:07 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1835333165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:56:07.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:07 smithi081 ceph-mon[30961]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T08:56:07.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:07 smithi081 ceph-mon[30961]: Reconfiguring daemon mgr.x on smithi069 2024-04-24T08:56:07.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:07 smithi081 ceph-mon[30961]: Standby manager daemon z started 2024-04-24T08:56:07.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:07 smithi081 ceph-mon[30961]: from='mgr.? 172.21.15.130:0/2241268869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T08:56:07.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:07 smithi081 ceph-mon[30961]: from='mgr.? 172.21.15.130:0/2241268869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:56:07.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:07 smithi081 ceph-mon[30961]: from='mgr.? 172.21.15.130:0/2241268869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T08:56:07.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:07 smithi081 ceph-mon[30961]: from='mgr.? 172.21.15.130:0/2241268869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:56:07.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:07 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1835333165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:56:07.817 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:07 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:07.429+0000 7f005df3b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T08:56:07.818 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:07 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:07.556+0000 7f005df3b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T08:56:07.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:07 smithi069 ceph-mon[28170]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T08:56:07.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:07 smithi069 ceph-mon[28170]: Reconfiguring daemon mgr.x on smithi069 2024-04-24T08:56:07.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:07 smithi069 ceph-mon[28170]: Standby manager daemon z started 2024-04-24T08:56:07.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:07 smithi069 ceph-mon[28170]: from='mgr.? 172.21.15.130:0/2241268869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T08:56:07.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:07 smithi069 ceph-mon[28170]: from='mgr.? 172.21.15.130:0/2241268869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:56:07.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:07 smithi069 ceph-mon[28170]: from='mgr.? 172.21.15.130:0/2241268869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T08:56:07.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:07 smithi069 ceph-mon[28170]: from='mgr.? 172.21.15.130:0/2241268869' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:56:07.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:07 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1835333165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:56:08.316 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:07 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:07.818+0000 7f005df3b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T08:56:08.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:08 smithi130 ceph-mon[31553]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:08.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:08 smithi130 ceph-mon[31553]: mgrmap e14: x(active, since 104s), standbys: z 2024-04-24T08:56:08.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:08 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T08:56:08.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:08 smithi081 ceph-mon[30961]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:08.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:08 smithi081 ceph-mon[30961]: mgrmap e14: x(active, since 104s), standbys: z 2024-04-24T08:56:08.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:08 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T08:56:08.818 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:08.465+0000 7f005df3b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T08:56:08.818 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:08.586+0000 7f005df3b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T08:56:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:08 smithi069 ceph-mon[28170]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:08 smithi069 ceph-mon[28170]: mgrmap e14: x(active, since 104s), standbys: z 2024-04-24T08:56:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T08:56:09.206 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:08.905+0000 7f005df3b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T08:56:09.206 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:09 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:09.081+0000 7f005df3b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T08:56:09.206 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:09 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:09.205+0000 7f005df3b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T08:56:09.567 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:09 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:09.354+0000 7f005df3b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T08:56:10.066 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:09 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:09.718+0000 7f005df3b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T08:56:10.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:10 smithi130 ceph-mon[31553]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:10.758 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:10 smithi081 ceph-mon[30961]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:10.758 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:10 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:10.474+0000 7f005df3b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T08:56:10.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:10 smithi069 ceph-mon[28170]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:11.067 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 08:56:10 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T08:56:10.757+0000 7f005df3b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T08:56:11.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:11.372+0000 7fcad1600700 1 -- 172.21.15.69:0/757990422 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fcac8021200 con 0x7fcad4104d60 2024-04-24T08:56:11.432 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:11 smithi069 ceph-mon[28170]: Standby manager daemon y started 2024-04-24T08:56:11.432 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:11 smithi069 ceph-mon[28170]: from='mgr.? 172.21.15.81:0/720624501' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T08:56:11.432 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:11 smithi069 ceph-mon[28170]: from='mgr.? 172.21.15.81:0/720624501' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:56:11.432 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:11 smithi069 ceph-mon[28170]: from='mgr.? 172.21.15.81:0/720624501' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T08:56:11.432 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:11 smithi069 ceph-mon[28170]: from='mgr.? 172.21.15.81:0/720624501' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:56:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:11 smithi130 ceph-mon[31553]: Standby manager daemon y started 2024-04-24T08:56:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:11 smithi130 ceph-mon[31553]: from='mgr.? 172.21.15.81:0/720624501' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T08:56:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:11 smithi130 ceph-mon[31553]: from='mgr.? 172.21.15.81:0/720624501' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:56:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:11 smithi130 ceph-mon[31553]: from='mgr.? 172.21.15.81:0/720624501' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T08:56:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:11 smithi130 ceph-mon[31553]: from='mgr.? 172.21.15.81:0/720624501' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:56:11.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:11 smithi081 ceph-mon[30961]: Standby manager daemon y started 2024-04-24T08:56:11.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:11 smithi081 ceph-mon[30961]: from='mgr.? 172.21.15.81:0/720624501' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T08:56:11.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:11 smithi081 ceph-mon[30961]: from='mgr.? 172.21.15.81:0/720624501' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:56:11.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:11 smithi081 ceph-mon[30961]: from='mgr.? 172.21.15.81:0/720624501' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T08:56:11.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:11 smithi081 ceph-mon[30961]: from='mgr.? 172.21.15.81:0/720624501' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:56:12.404 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:12 smithi069 ceph-mon[28170]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:12.404 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:12 smithi069 ceph-mon[28170]: mgrmap e15: x(active, since 108s), standbys: z, y 2024-04-24T08:56:12.404 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:12 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T08:56:12.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:12 smithi130 ceph-mon[31553]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:12.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:12 smithi130 ceph-mon[31553]: mgrmap e15: x(active, since 108s), standbys: z, y 2024-04-24T08:56:12.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:12 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T08:56:12.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:12 smithi081 ceph-mon[30961]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:12.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:12 smithi081 ceph-mon[30961]: mgrmap e15: x(active, since 108s), standbys: z, y 2024-04-24T08:56:12.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:12 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T08:56:13.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:13 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:13.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:13 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:13.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:13 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:13.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:13 smithi069 ceph-mon[28170]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:13.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:13 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:13.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:13 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:13.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:13 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:13.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:13 smithi081 ceph-mon[30961]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:13.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:13 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:13.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:13 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:13.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:13 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:13.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:13 smithi130 ceph-mon[31553]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:14.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:14 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:14.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:14 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:56:14.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:14 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:14.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:14 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:14.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:14 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:56:14.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:14 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:14.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:14 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:14.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:14 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:56:14.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:14 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:15.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:15 smithi081 ceph-mon[30961]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:15.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:15 smithi069 ceph-mon[28170]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:15.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:15 smithi130 ceph-mon[31553]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:16.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:16 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T08:56:16.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:16 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:16.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:16 smithi069 ceph-mon[28170]: Deploying daemon osd.0 on smithi069 2024-04-24T08:56:16.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:16 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T08:56:16.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:16 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:16.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:16 smithi130 ceph-mon[31553]: Deploying daemon osd.0 on smithi069 2024-04-24T08:56:17.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:16 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T08:56:17.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:16 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:17.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:16 smithi081 ceph-mon[30961]: Deploying daemon osd.0 on smithi069 2024-04-24T08:56:17.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:17 smithi069 ceph-mon[28170]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:17.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:17 smithi130 ceph-mon[31553]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:18.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:17 smithi081 ceph-mon[30961]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:20.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:20 smithi081 ceph-mon[30961]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:20.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:20 smithi069 ceph-mon[28170]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:20.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:20 smithi130 ceph-mon[31553]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:22.493 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:22 smithi069 ceph-mon[28170]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:22.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:22 smithi081 ceph-mon[30961]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:22.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:22 smithi130 ceph-mon[31553]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:24.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:24 smithi081 ceph-mon[30961]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:24.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:24.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:24.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:24.590 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:24 smithi069 ceph-mon[28170]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:24.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:24.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:24.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:24.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:24 smithi130 ceph-mon[31553]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:24.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:24.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:24.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:26.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:26 smithi081 ceph-mon[30961]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:26.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:26 smithi069 ceph-mon[28170]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:26.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:26 smithi130 ceph-mon[31553]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:28.556 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:28 smithi069 ceph-mon[28170]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:28.557 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:28 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:28.557 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:28 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:28.557 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:28 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:28.557 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:28 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:56:28.557 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:28 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:28.557 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:28 smithi069 ceph-mon[28170]: from='osd.0 [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T08:56:28.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:28.675+0000 7fcad1600700 1 -- 172.21.15.69:0/757990422 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcad4062f30 con 0x7fcabc039890 2024-04-24T08:56:28.676 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 0 on host 'smithi069' 2024-04-24T08:56:28.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:28.678+0000 7fcadba00700 1 -- 172.21.15.69:0/757990422 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcabc039890 msgr2=0x7fcabc03bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:28.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:28.678+0000 7fcadba00700 1 --2- 172.21.15.69:0/757990422 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcabc039890 0x7fcabc03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009c60 tx=0x7fcac0009260 comp rx=0 tx=0).stop 2024-04-24T08:56:28.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:28.678+0000 7fcadba00700 1 -- 172.21.15.69:0/757990422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcad4104d60 msgr2=0x7fcad419ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:28.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:28.678+0000 7fcadba00700 1 --2- 172.21.15.69:0/757990422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcad4104d60 0x7fcad419ae40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcac800d9b0 tx=0x7fcac800dcc0 comp rx=0 tx=0).stop 2024-04-24T08:56:28.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:28.678+0000 7fcadba00700 1 -- 172.21.15.69:0/757990422 shutdown_connections 2024-04-24T08:56:28.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:28.679+0000 7fcadba00700 1 --2- 172.21.15.69:0/757990422 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcabc039890 0x7fcabc03bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:28.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:28.679+0000 7fcadba00700 1 --2- 172.21.15.69:0/757990422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcad4106900 0x7fcad41a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:28.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:28.679+0000 7fcadba00700 1 --2- 172.21.15.69:0/757990422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcad4104d60 0x7fcad419ae40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:28.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:28.679+0000 7fcadba00700 1 --2- 172.21.15.69:0/757990422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcad4105f60 0x7fcad419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:28.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:28.679+0000 7fcadba00700 1 -- 172.21.15.69:0/757990422 >> 172.21.15.69:0/757990422 conn(0x7fcad4100300 msgr2=0x7fcad41092e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:28.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:28.679+0000 7fcadba00700 1 -- 172.21.15.69:0/757990422 shutdown_connections 2024-04-24T08:56:28.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:56:28.679+0000 7fcadba00700 1 -- 172.21.15.69:0/757990422 wait complete. 2024-04-24T08:56:28.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:28 smithi130 ceph-mon[31553]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:28.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:28 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:28.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:28 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:28.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:28 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:28.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:28 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:56:28.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:28 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:28.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:28 smithi130 ceph-mon[31553]: from='osd.0 [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T08:56:28.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:28 smithi081 ceph-mon[30961]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:28.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:28 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:28.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:28 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:28.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:28 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:28.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:28 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:56:28.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:28 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:28.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:28 smithi081 ceph-mon[30961]: from='osd.0 [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T08:56:29.367 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:29 smithi069 ceph-mon[28170]: from='osd.0 [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T08:56:29.367 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:29 smithi069 ceph-mon[28170]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T08:56:29.367 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:29 smithi069 ceph-mon[28170]: from='osd.0 [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-24T08:56:29.368 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:29 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:56:29.368 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:29 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:29.368 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:29 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:29.368 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:29 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:29.491 DEBUG:teuthology.orchestra.run.smithi069:osd.0> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.0.service 2024-04-24T08:56:29.496 INFO:tasks.cephadm:Deploying osd.1 on smithi081 with /dev/vg_nvme/lv_4... 2024-04-24T08:56:29.497 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T08:56:29.607 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:29 smithi130 ceph-mon[31553]: from='osd.0 [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T08:56:29.607 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:29 smithi130 ceph-mon[31553]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T08:56:29.607 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:29 smithi130 ceph-mon[31553]: from='osd.0 [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-24T08:56:29.607 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:29 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:56:29.607 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:29 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:29.607 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:29 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:29.607 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:29 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:29.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:29 smithi081 ceph-mon[30961]: from='osd.0 [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T08:56:29.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:29 smithi081 ceph-mon[30961]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T08:56:29.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:29 smithi081 ceph-mon[30961]: from='osd.0 [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-24T08:56:29.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:29 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:56:29.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:29 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:29.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:29 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:29.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:29 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:29.925 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T08:56:30.639 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:30 smithi069 ceph-mon[28170]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:30.639 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:30 smithi069 ceph-mon[28170]: from='osd.0 [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-24T08:56:30.639 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:30 smithi069 ceph-mon[28170]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T08:56:30.639 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:30 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:56:30.639 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:30 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:56:30.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:30 smithi130 ceph-mon[31553]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:30.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:30 smithi130 ceph-mon[31553]: from='osd.0 [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-24T08:56:30.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:30 smithi130 ceph-mon[31553]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T08:56:30.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:30 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:56:30.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:30 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:56:30.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:30 smithi081 ceph-mon[30961]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:30.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:30 smithi081 ceph-mon[30961]: from='osd.0 [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-24T08:56:30.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:30 smithi081 ceph-mon[30961]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T08:56:30.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:30 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:56:30.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:30 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:56:31.452 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:31 smithi081 ceph-mon[30961]: purged_snaps scrub starts 2024-04-24T08:56:31.452 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:31 smithi081 ceph-mon[30961]: purged_snaps scrub ok 2024-04-24T08:56:31.452 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:31 smithi081 ceph-mon[30961]: osd.0 [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595] boot 2024-04-24T08:56:31.452 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:31 smithi081 ceph-mon[30961]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T08:56:31.452 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:31 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:56:31.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:31 smithi130 ceph-mon[31553]: purged_snaps scrub starts 2024-04-24T08:56:31.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:31 smithi130 ceph-mon[31553]: purged_snaps scrub ok 2024-04-24T08:56:31.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:31 smithi130 ceph-mon[31553]: osd.0 [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595] boot 2024-04-24T08:56:31.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:31 smithi130 ceph-mon[31553]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T08:56:31.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:56:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:31 smithi069 ceph-mon[28170]: purged_snaps scrub starts 2024-04-24T08:56:31.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:31 smithi069 ceph-mon[28170]: purged_snaps scrub ok 2024-04-24T08:56:31.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:31 smithi069 ceph-mon[28170]: osd.0 [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595] boot 2024-04-24T08:56:31.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:31 smithi069 ceph-mon[28170]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T08:56:31.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:56:32.666 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:32 smithi081 ceph-mon[30961]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:32.667 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:32 smithi081 ceph-mon[30961]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T08:56:32.667 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T08:56:32.667 INFO:teuthology.orchestra.run.smithi081.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T08:56:32.667 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10+0 records in 2024-04-24T08:56:32.667 INFO:teuthology.orchestra.run.smithi081.stderr:10+0 records out 2024-04-24T08:56:32.667 INFO:teuthology.orchestra.run.smithi081.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0365632 s, 287 MB/s 2024-04-24T08:56:32.667 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping successful for: 2024-04-24T08:56:32.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:32 smithi130 ceph-mon[31553]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:32.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:32 smithi130 ceph-mon[31553]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T08:56:32.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:32 smithi069 ceph-mon[28170]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:56:32.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:32 smithi069 ceph-mon[28170]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T08:56:33.391 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi081:vg_nvme/lv_4 2024-04-24T08:56:33.819 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T08:56:34.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:34 smithi081 ceph-mon[30961]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:34.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:34 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:34.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:34 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:34.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:34 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:56:34.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:34 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:34.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:34 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:34.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:34 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:56:34.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:34 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:34.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:34 smithi130 ceph-mon[31553]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:34.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:34 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:34.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:34 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:34.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:34 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:56:34.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:34 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:34.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:34 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:34.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:34 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:56:34.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:34 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:34.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:34 smithi069 ceph-mon[28170]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:34.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:34 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:34 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:34 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:56:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:34 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:34 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:34 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:56:34.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:34 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:35.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:35 smithi081 ceph-mon[30961]: Detected new or changed devices on smithi069 2024-04-24T08:56:35.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:35 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi069 to 17089M 2024-04-24T08:56:35.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:35 smithi130 ceph-mon[31553]: Detected new or changed devices on smithi069 2024-04-24T08:56:35.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:35 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi069 to 17089M 2024-04-24T08:56:35.781 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.780+0000 7fc97c400700 1 -- 172.21.15.81:0/1057986829 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc974102b60 msgr2=0x7fc974102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:35.781 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.780+0000 7fc97c400700 1 --2- 172.21.15.81:0/1057986829 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc974102b60 0x7fc974102f80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc9680099c0 tx=0x7fc968009cd0 comp rx=0 tx=0).stop 2024-04-24T08:56:35.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.780+0000 7fc97c400700 1 -- 172.21.15.81:0/1057986829 shutdown_connections 2024-04-24T08:56:35.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.780+0000 7fc97c400700 1 --2- 172.21.15.81:0/1057986829 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc974104700 0x7fc97410afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:35.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.780+0000 7fc97c400700 1 --2- 172.21.15.81:0/1057986829 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc974102b60 0x7fc974102f80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:35.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.780+0000 7fc97c400700 1 --2- 172.21.15.81:0/1057986829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc974103d60 0x7fc9741041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:35.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.780+0000 7fc97c400700 1 -- 172.21.15.81:0/1057986829 >> 172.21.15.81:0/1057986829 conn(0x7fc9740fe0c0 msgr2=0x7fc974100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:35.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.780+0000 7fc97c400700 1 -- 172.21.15.81:0/1057986829 shutdown_connections 2024-04-24T08:56:35.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.780+0000 7fc97c400700 1 -- 172.21.15.81:0/1057986829 wait complete. 2024-04-24T08:56:35.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.781+0000 7fc97c400700 1 Processor -- start 2024-04-24T08:56:35.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.781+0000 7fc97c400700 1 -- start start 2024-04-24T08:56:35.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.782+0000 7fc97c400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc974102b60 0x7fc97419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:35.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.782+0000 7fc97c400700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc974103d60 0x7fc97419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:35.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.782+0000 7fc97c400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc974104700 0x7fc9741a2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:35.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.782+0000 7fc97c400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc97410e620 con 0x7fc974104700 2024-04-24T08:56:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.782+0000 7fc97c400700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc97410e4a0 con 0x7fc974103d60 2024-04-24T08:56:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.782+0000 7fc97c400700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc97410e7a0 con 0x7fc974102b60 2024-04-24T08:56:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.782+0000 7fc979000700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc974103d60 0x7fc97419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.782+0000 7fc979000700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc974103d60 0x7fc97419b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53058/0 (socket says 172.21.15.81:53058) 2024-04-24T08:56:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.782+0000 7fc979000700 1 -- 172.21.15.81:0/2156500125 learned_addr learned my addr 172.21.15.81:0/2156500125 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-04-24T08:56:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.782+0000 7fc979a00700 1 --2- 172.21.15.81:0/2156500125 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc974102b60 0x7fc97419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.782+0000 7fc97a400700 1 --2- 172.21.15.81:0/2156500125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc974104700 0x7fc9741a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.783+0000 7fc979000700 1 -- 172.21.15.81:0/2156500125 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc974102b60 msgr2=0x7fc97419aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.783+0000 7fc979000700 1 --2- 172.21.15.81:0/2156500125 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc974102b60 0x7fc97419aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.783+0000 7fc979000700 1 -- 172.21.15.81:0/2156500125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc974104700 msgr2=0x7fc9741a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.783+0000 7fc979000700 1 --2- 172.21.15.81:0/2156500125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc974104700 0x7fc9741a2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.783+0000 7fc979000700 1 -- 172.21.15.81:0/2156500125 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9741a2ae0 con 0x7fc974103d60 2024-04-24T08:56:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.783+0000 7fc97a400700 1 --2- 172.21.15.81:0/2156500125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc974104700 0x7fc9741a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.783+0000 7fc979000700 1 --2- 172.21.15.81:0/2156500125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc974103d60 0x7fc97419b410 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc96000c920 tx=0x7fc96000cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:35.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.783+0000 7fc972000700 1 -- 172.21.15.81:0/2156500125 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc960007b00 con 0x7fc974103d60 2024-04-24T08:56:35.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.783+0000 7fc972000700 1 -- 172.21.15.81:0/2156500125 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc960004ce0 con 0x7fc974103d60 2024-04-24T08:56:35.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.783+0000 7fc97c400700 1 -- 172.21.15.81:0/2156500125 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9741a2dd0 con 0x7fc974103d60 2024-04-24T08:56:35.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.783+0000 7fc972000700 1 -- 172.21.15.81:0/2156500125 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc960005600 con 0x7fc974103d60 2024-04-24T08:56:35.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.783+0000 7fc97c400700 1 -- 172.21.15.81:0/2156500125 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9741a3280 con 0x7fc974103d60 2024-04-24T08:56:35.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.785+0000 7fc97c400700 1 -- 172.21.15.81:0/2156500125 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc974069810 con 0x7fc974103d60 2024-04-24T08:56:35.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.787+0000 7fc972000700 1 -- 172.21.15.81:0/2156500125 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fc960004750 con 0x7fc974103d60 2024-04-24T08:56:35.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.789+0000 7fc972000700 1 --2- 172.21.15.81:0/2156500125 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc95c0a3bb0 0x7fc95c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:35.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.790+0000 7fc972000700 1 -- 172.21.15.81:0/2156500125 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fc9600ce990 con 0x7fc974103d60 2024-04-24T08:56:35.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.790+0000 7fc979a00700 1 --2- 172.21.15.81:0/2156500125 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc95c0a3bb0 0x7fc95c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:35.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.791+0000 7fc979a00700 1 --2- 172.21.15.81:0/2156500125 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc95c0a3bb0 0x7fc95c0a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc968000c00 tx=0x7fc968005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:35.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:35.793+0000 7fc972000700 1 -- 172.21.15.81:0/2156500125 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc974069810 con 0x7fc974103d60 2024-04-24T08:56:35.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:35 smithi069 ceph-mon[28170]: Detected new or changed devices on smithi069 2024-04-24T08:56:35.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:35 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi069 to 17089M 2024-04-24T08:56:36.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:56:36.028+0000 7fc97c400700 1 -- 172.21.15.81:0/2156500125 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc97410ac20 con 0x7fc95c0a3bb0 2024-04-24T08:56:36.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:36 smithi130 ceph-mon[31553]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:36.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:36 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:56:36.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:36 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:56:36.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:36 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:36.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:36 smithi081 ceph-mon[30961]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:36.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:36 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:56:36.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:36 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:56:36.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:36 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:36.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:36 smithi069 ceph-mon[28170]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:36.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:36 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:56:36.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:36 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:56:36.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:36 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:37.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:37 smithi081 ceph-mon[30961]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:56:37.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:37 smithi130 ceph-mon[31553]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:56:37.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:37 smithi069 ceph-mon[28170]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:56:38.534 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:38 smithi081 ceph-mon[30961]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:38.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:38 smithi130 ceph-mon[31553]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:38.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:38 smithi069 ceph-mon[28170]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:40.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:40 smithi130 ceph-mon[31553]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:40.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:40 smithi130 ceph-mon[31553]: from='client.? 172.21.15.81:0/3634047086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dae4a019-875a-48ff-96d8-c19efa5987ff"}]: dispatch 2024-04-24T08:56:40.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:40 smithi130 ceph-mon[31553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dae4a019-875a-48ff-96d8-c19efa5987ff"}]: dispatch 2024-04-24T08:56:40.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:40 smithi130 ceph-mon[31553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dae4a019-875a-48ff-96d8-c19efa5987ff"}]': finished 2024-04-24T08:56:40.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:40 smithi130 ceph-mon[31553]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T08:56:40.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:56:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:40 smithi081 ceph-mon[30961]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:40 smithi081 ceph-mon[30961]: from='client.? 172.21.15.81:0/3634047086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dae4a019-875a-48ff-96d8-c19efa5987ff"}]: dispatch 2024-04-24T08:56:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:40 smithi081 ceph-mon[30961]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dae4a019-875a-48ff-96d8-c19efa5987ff"}]: dispatch 2024-04-24T08:56:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:40 smithi081 ceph-mon[30961]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dae4a019-875a-48ff-96d8-c19efa5987ff"}]': finished 2024-04-24T08:56:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:40 smithi081 ceph-mon[30961]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T08:56:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:56:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:40 smithi069 ceph-mon[28170]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:40 smithi069 ceph-mon[28170]: from='client.? 172.21.15.81:0/3634047086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dae4a019-875a-48ff-96d8-c19efa5987ff"}]: dispatch 2024-04-24T08:56:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:40 smithi069 ceph-mon[28170]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dae4a019-875a-48ff-96d8-c19efa5987ff"}]: dispatch 2024-04-24T08:56:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:40 smithi069 ceph-mon[28170]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dae4a019-875a-48ff-96d8-c19efa5987ff"}]': finished 2024-04-24T08:56:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:40 smithi069 ceph-mon[28170]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T08:56:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:56:41.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:41 smithi130 ceph-mon[31553]: from='client.? 172.21.15.81:0/1702653536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:56:41.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:41 smithi081 ceph-mon[30961]: from='client.? 172.21.15.81:0/1702653536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:56:41.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:41 smithi069 ceph-mon[28170]: from='client.? 172.21.15.81:0/1702653536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:56:42.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:42 smithi130 ceph-mon[31553]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:42 smithi081 ceph-mon[30961]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:42.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:42 smithi069 ceph-mon[28170]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:43.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:43 smithi081 ceph-mon[30961]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:43.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:43 smithi069 ceph-mon[28170]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:43.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:43 smithi130 ceph-mon[31553]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:46.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:46 smithi081 ceph-mon[30961]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:46.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:46 smithi069 ceph-mon[28170]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:46.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:46 smithi130 ceph-mon[31553]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:48.533 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:48 smithi081 ceph-mon[30961]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:48.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:48 smithi069 ceph-mon[28170]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:48.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:48 smithi130 ceph-mon[31553]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:49.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T08:56:49.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:49.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T08:56:49.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:49.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T08:56:49.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:50.553 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:50 smithi081 ceph-mon[30961]: Deploying daemon osd.1 on smithi081 2024-04-24T08:56:50.553 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:50 smithi081 ceph-mon[30961]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:50.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:50 smithi069 ceph-mon[28170]: Deploying daemon osd.1 on smithi081 2024-04-24T08:56:50.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:50 smithi069 ceph-mon[28170]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:50.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:50 smithi130 ceph-mon[31553]: Deploying daemon osd.1 on smithi081 2024-04-24T08:56:50.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:50 smithi130 ceph-mon[31553]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:52.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:52 smithi069 ceph-mon[28170]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:52.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:52 smithi130 ceph-mon[31553]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:52.704 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:52 smithi081 ceph-mon[30961]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:54.418 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:54 smithi081 ceph-mon[30961]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:54.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:54 smithi069 ceph-mon[28170]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:54.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:54 smithi130 ceph-mon[31553]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:56.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:56 smithi069 ceph-mon[28170]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:56.649 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:56 smithi081 ceph-mon[30961]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:56.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:56 smithi130 ceph-mon[31553]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:58.043 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:57 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:58.043 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:57 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:58.044 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:57 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:58.044 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:56:57 smithi081 ceph-mon[30961]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:58.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:57 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:58.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:57 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:58.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:57 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:58.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:56:57 smithi130 ceph-mon[31553]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:56:58.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:57 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:58.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:57 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:56:58.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:57 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:58.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:56:57 smithi069 ceph-mon[28170]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:57:00.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:00 smithi081 ceph-mon[30961]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:57:00.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:00 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:00.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:00 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:00.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:00 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:00.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:00 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:57:00.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:00 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:00.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:00 smithi069 ceph-mon[28170]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:57:00.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:00 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:00.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:00 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:00.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:00 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:00.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:00 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:57:00.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:00 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:00.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:00 smithi130 ceph-mon[31553]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:57:00.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:00 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:00.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:00 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:00.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:00 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:00.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:00 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:57:00.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:00 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:01.379 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:01 smithi130 ceph-mon[31553]: from='osd.1 [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T08:57:01.379 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:01 smithi130 ceph-mon[31553]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T08:57:01.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:01 smithi081 ceph-mon[30961]: from='osd.1 [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T08:57:01.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:01 smithi081 ceph-mon[30961]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T08:57:01.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:01 smithi069 ceph-mon[28170]: from='osd.1 [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T08:57:01.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:01 smithi069 ceph-mon[28170]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T08:57:01.735 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:57:01.734+0000 7fc972000700 1 -- 172.21.15.81:0/2156500125 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc97410ac20 con 0x7fc95c0a3bb0 2024-04-24T08:57:01.735 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 1 on host 'smithi081' 2024-04-24T08:57:01.739 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:57:01.738+0000 7fc97c400700 1 -- 172.21.15.81:0/2156500125 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc95c0a3bb0 msgr2=0x7fc95c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:01.739 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:57:01.738+0000 7fc97c400700 1 --2- 172.21.15.81:0/2156500125 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc95c0a3bb0 0x7fc95c0a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc968000c00 tx=0x7fc968005fb0 comp rx=0 tx=0).stop 2024-04-24T08:57:01.739 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:57:01.738+0000 7fc97c400700 1 -- 172.21.15.81:0/2156500125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc974103d60 msgr2=0x7fc97419b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:01.740 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:57:01.738+0000 7fc97c400700 1 --2- 172.21.15.81:0/2156500125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc974103d60 0x7fc97419b410 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc96000c920 tx=0x7fc96000cce0 comp rx=0 tx=0).stop 2024-04-24T08:57:01.740 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:57:01.738+0000 7fc97c400700 1 -- 172.21.15.81:0/2156500125 shutdown_connections 2024-04-24T08:57:01.740 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:57:01.738+0000 7fc97c400700 1 --2- 172.21.15.81:0/2156500125 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc95c0a3bb0 0x7fc95c0a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:01.740 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:57:01.738+0000 7fc97c400700 1 --2- 172.21.15.81:0/2156500125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc974104700 0x7fc9741a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:01.740 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:57:01.738+0000 7fc97c400700 1 --2- 172.21.15.81:0/2156500125 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc974102b60 0x7fc97419aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:01.740 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:57:01.738+0000 7fc97c400700 1 --2- 172.21.15.81:0/2156500125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc974103d60 0x7fc97419b410 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:01.741 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:57:01.738+0000 7fc97c400700 1 -- 172.21.15.81:0/2156500125 >> 172.21.15.81:0/2156500125 conn(0x7fc9740fe0c0 msgr2=0x7fc974109500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:01.741 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:57:01.739+0000 7fc97c400700 1 -- 172.21.15.81:0/2156500125 shutdown_connections 2024-04-24T08:57:01.741 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:57:01.739+0000 7fc97c400700 1 -- 172.21.15.81:0/2156500125 wait complete. 2024-04-24T08:57:02.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:02 smithi069 ceph-mon[28170]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:57:02.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:02 smithi069 ceph-mon[28170]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T08:57:02.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:02 smithi069 ceph-mon[28170]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T08:57:02.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:57:02.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:02 smithi069 ceph-mon[28170]: from='osd.1 [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T08:57:02.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:02 smithi069 ceph-mon[28170]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T08:57:02.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:02.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:02.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:57:02.594 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:02 smithi081 ceph-mon[30961]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:57:02.594 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:02 smithi081 ceph-mon[30961]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T08:57:02.594 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:02 smithi081 ceph-mon[30961]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T08:57:02.594 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:02 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:57:02.594 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:02 smithi081 ceph-mon[30961]: from='osd.1 [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T08:57:02.594 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:02 smithi081 ceph-mon[30961]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T08:57:02.594 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:02 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:02.594 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:02 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:02.594 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:02 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:57:02.654 DEBUG:teuthology.orchestra.run.smithi081:osd.1> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.1.service 2024-04-24T08:57:02.659 INFO:tasks.cephadm:Deploying osd.2 on smithi130 with /dev/vg_nvme/lv_4... 2024-04-24T08:57:02.659 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T08:57:02.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:02 smithi130 ceph-mon[31553]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:57:02.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:02 smithi130 ceph-mon[31553]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T08:57:02.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:02 smithi130 ceph-mon[31553]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T08:57:02.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:02 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:57:02.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:02 smithi130 ceph-mon[31553]: from='osd.1 [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T08:57:02.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:02 smithi130 ceph-mon[31553]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T08:57:02.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:02 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:02.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:02 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:02.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:02 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:57:03.084 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:57:03.483 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:03 smithi130 ceph-mon[31553]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-04-24T08:57:03.483 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:03 smithi130 ceph-mon[31553]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T08:57:03.483 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:03 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:57:03.483 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:03 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:57:03.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:03 smithi081 ceph-mon[30961]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-04-24T08:57:03.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:03 smithi081 ceph-mon[30961]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T08:57:03.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:03 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:57:03.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:03 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:57:03.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:03 smithi069 ceph-mon[28170]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-04-24T08:57:03.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:03 smithi069 ceph-mon[28170]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T08:57:03.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:03 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:57:03.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:03 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:57:04.527 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:04 smithi130 ceph-mon[31553]: purged_snaps scrub starts 2024-04-24T08:57:04.527 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:04 smithi130 ceph-mon[31553]: purged_snaps scrub ok 2024-04-24T08:57:04.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:04 smithi130 ceph-mon[31553]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:57:04.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:04 smithi130 ceph-mon[31553]: osd.1 [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330] boot 2024-04-24T08:57:04.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:04 smithi130 ceph-mon[31553]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T08:57:04.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:04 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:57:04.544 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:04 smithi081 ceph-mon[30961]: purged_snaps scrub starts 2024-04-24T08:57:04.544 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:04 smithi081 ceph-mon[30961]: purged_snaps scrub ok 2024-04-24T08:57:04.544 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:04 smithi081 ceph-mon[30961]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:57:04.544 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:04 smithi081 ceph-mon[30961]: osd.1 [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330] boot 2024-04-24T08:57:04.545 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:04 smithi081 ceph-mon[30961]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T08:57:04.545 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:04 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:57:04.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:04 smithi069 ceph-mon[28170]: purged_snaps scrub starts 2024-04-24T08:57:04.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:04 smithi069 ceph-mon[28170]: purged_snaps scrub ok 2024-04-24T08:57:04.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:04 smithi069 ceph-mon[28170]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:57:04.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:04 smithi069 ceph-mon[28170]: osd.1 [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330] boot 2024-04-24T08:57:04.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:04 smithi069 ceph-mon[28170]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T08:57:04.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:04 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:57:05.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:05 smithi069 ceph-mon[28170]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T08:57:05.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:05 smithi130 ceph-mon[31553]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T08:57:05.768 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T08:57:05.768 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T08:57:05.768 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-04-24T08:57:05.768 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-04-24T08:57:05.768 INFO:teuthology.orchestra.run.smithi130.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0362593 s, 289 MB/s 2024-04-24T08:57:05.768 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-04-24T08:57:05.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:05 smithi081 ceph-mon[30961]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T08:57:06.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:06 smithi130 ceph-mon[31553]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:06.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:06 smithi081 ceph-mon[30961]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:06.599 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_4 2024-04-24T08:57:06.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:06 smithi069 ceph-mon[28170]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:07.072 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:57:08.271 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:08 smithi130 ceph-mon[31553]: Detected new or changed devices on smithi081 2024-04-24T08:57:08.271 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:08 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:08.271 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:08 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:08.272 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:08 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:57:08.272 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:08 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi081 to 17089M 2024-04-24T08:57:08.272 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:08 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:08.272 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:08 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:08.272 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:08 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:57:08.272 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:08 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:08.272 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:08 smithi130 ceph-mon[31553]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:08.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:08 smithi081 ceph-mon[30961]: Detected new or changed devices on smithi081 2024-04-24T08:57:08.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:08 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:08.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:08 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:08.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:08 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:57:08.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:08 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi081 to 17089M 2024-04-24T08:57:08.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:08 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:08.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:08 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:08.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:08 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:57:08.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:08 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:08.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:08 smithi081 ceph-mon[30961]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:08.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:08 smithi069 ceph-mon[28170]: Detected new or changed devices on smithi081 2024-04-24T08:57:08.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:08.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:08.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:57:08.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:08 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi081 to 17089M 2024-04-24T08:57:08.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:08.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:08.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:57:08.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:08.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:08 smithi069 ceph-mon[28170]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:09.122 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.121+0000 7efe60200700 1 -- 172.21.15.130:0/2445919454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe58105f90 msgr2=0x7efe581063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:09.122 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.121+0000 7efe60200700 1 --2- 172.21.15.130:0/2445919454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe58105f90 0x7efe581063f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efe48009990 tx=0x7efe48009ca0 comp rx=0 tx=0).stop 2024-04-24T08:57:09.122 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.121+0000 7efe60200700 1 -- 172.21.15.130:0/2445919454 shutdown_connections 2024-04-24T08:57:09.123 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.121+0000 7efe60200700 1 --2- 172.21.15.130:0/2445919454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe58106930 0x7efe5810d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:09.123 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.121+0000 7efe60200700 1 --2- 172.21.15.130:0/2445919454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe58104d90 0x7efe581051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:09.123 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.121+0000 7efe60200700 1 --2- 172.21.15.130:0/2445919454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe58105f90 0x7efe581063f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:09.123 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.121+0000 7efe60200700 1 -- 172.21.15.130:0/2445919454 >> 172.21.15.130:0/2445919454 conn(0x7efe58100350 msgr2=0x7efe58102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:09.123 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.122+0000 7efe60200700 1 -- 172.21.15.130:0/2445919454 shutdown_connections 2024-04-24T08:57:09.123 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.122+0000 7efe60200700 1 -- 172.21.15.130:0/2445919454 wait complete. 2024-04-24T08:57:09.123 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.123+0000 7efe60200700 1 Processor -- start 2024-04-24T08:57:09.123 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.123+0000 7efe60200700 1 -- start start 2024-04-24T08:57:09.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.123+0000 7efe60200700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe58104d90 0x7efe5819af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:09.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.123+0000 7efe60200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe58105f90 0x7efe5819b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:09.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.123+0000 7efe60200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe58106930 0x7efe581a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:09.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.123+0000 7efe60200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efe581108f0 con 0x7efe58106930 2024-04-24T08:57:09.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.123+0000 7efe60200700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efe58110770 con 0x7efe58104d90 2024-04-24T08:57:09.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.123+0000 7efe60200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efe58110a70 con 0x7efe58105f90 2024-04-24T08:57:09.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.123+0000 7efe5ce00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe58105f90 0x7efe5819b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:09.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.123+0000 7efe5ce00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe58105f90 0x7efe5819b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:41698/0 (socket says 172.21.15.130:41698) 2024-04-24T08:57:09.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.123+0000 7efe5ce00700 1 -- 172.21.15.130:0/76248324 learned_addr learned my addr 172.21.15.130:0/76248324 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-24T08:57:09.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.124+0000 7efe5d800700 1 --2- 172.21.15.130:0/76248324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe58104d90 0x7efe5819af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:09.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.124+0000 7efe5ce00700 1 -- 172.21.15.130:0/76248324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe58104d90 msgr2=0x7efe5819af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:09.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.124+0000 7efe5ce00700 1 --2- 172.21.15.130:0/76248324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe58104d90 0x7efe5819af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:09.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.124+0000 7efe5ce00700 1 -- 172.21.15.130:0/76248324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe58106930 msgr2=0x7efe581a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:09.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.124+0000 7efe5e200700 1 --2- 172.21.15.130:0/76248324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe58106930 0x7efe581a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:09.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.124+0000 7efe5ce00700 1 --2- 172.21.15.130:0/76248324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe58106930 0x7efe581a24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:09.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.124+0000 7efe5ce00700 1 -- 172.21.15.130:0/76248324 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe581a2b90 con 0x7efe58105f90 2024-04-24T08:57:09.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.124+0000 7efe5e200700 1 --2- 172.21.15.130:0/76248324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe58106930 0x7efe581a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:09.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.124+0000 7efe5ce00700 1 --2- 172.21.15.130:0/76248324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe58105f90 0x7efe5819b470 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efe48005910 tx=0x7efe48009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:09.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.125+0000 7efe56000700 1 -- 172.21.15.130:0/76248324 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48027070 con 0x7efe58105f90 2024-04-24T08:57:09.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.125+0000 7efe60200700 1 -- 172.21.15.130:0/76248324 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe581a2e20 con 0x7efe58105f90 2024-04-24T08:57:09.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.125+0000 7efe56000700 1 -- 172.21.15.130:0/76248324 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe48022470 con 0x7efe58105f90 2024-04-24T08:57:09.127 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.125+0000 7efe56000700 1 -- 172.21.15.130:0/76248324 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48004230 con 0x7efe58105f90 2024-04-24T08:57:09.127 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.125+0000 7efe60200700 1 -- 172.21.15.130:0/76248324 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe581a32a0 con 0x7efe58105f90 2024-04-24T08:57:09.127 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.126+0000 7efe60200700 1 -- 172.21.15.130:0/76248324 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe580697e0 con 0x7efe58105f90 2024-04-24T08:57:09.127 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.128+0000 7efe56000700 1 -- 172.21.15.130:0/76248324 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7efe4802a070 con 0x7efe58105f90 2024-04-24T08:57:09.130 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.130+0000 7efe56000700 1 --2- 172.21.15.130:0/76248324 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efe400a3bb0 0x7efe400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:09.130 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.130+0000 7efe56000700 1 -- 172.21.15.130:0/76248324 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7efe480cfcf0 con 0x7efe58105f90 2024-04-24T08:57:09.130 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.131+0000 7efe5d800700 1 --2- 172.21.15.130:0/76248324 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efe400a3bb0 0x7efe400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:09.131 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.132+0000 7efe5d800700 1 --2- 172.21.15.130:0/76248324 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efe400a3bb0 0x7efe400a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efe4c006fd0 tx=0x7efe4c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:09.135 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.136+0000 7efe56000700 1 -- 172.21.15.130:0/76248324 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe4805eff0 con 0x7efe58105f90 2024-04-24T08:57:09.372 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:09.372+0000 7efe60200700 1 -- 172.21.15.130:0/76248324 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7efe58062f30 con 0x7efe400a3bb0 2024-04-24T08:57:10.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:10 smithi081 ceph-mon[30961]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:10.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:10 smithi081 ceph-mon[30961]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:57:10.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:10 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:57:10.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:10 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:57:10.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:10 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:10.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:10 smithi069 ceph-mon[28170]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:10.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:10 smithi069 ceph-mon[28170]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:57:10.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:10 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:57:10.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:10 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:57:10.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:10 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:10.677 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:10 smithi130 ceph-mon[31553]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:10.677 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:10 smithi130 ceph-mon[31553]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:57:10.677 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:10 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:57:10.677 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:10 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:57:10.677 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:10 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:12.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:12 smithi081 ceph-mon[30961]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:12.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:12 smithi069 ceph-mon[28170]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:12.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:12 smithi130 ceph-mon[31553]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:13.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:13 smithi081 ceph-mon[30961]: from='client.? 172.21.15.130:0/2130504272' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62f0fd9a-64fe-4680-9107-ae8a97490dec"}]: dispatch 2024-04-24T08:57:13.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:13 smithi081 ceph-mon[30961]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62f0fd9a-64fe-4680-9107-ae8a97490dec"}]: dispatch 2024-04-24T08:57:13.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:13 smithi081 ceph-mon[30961]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62f0fd9a-64fe-4680-9107-ae8a97490dec"}]': finished 2024-04-24T08:57:13.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:13 smithi081 ceph-mon[30961]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T08:57:13.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:13 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:13.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:13 smithi069 ceph-mon[28170]: from='client.? 172.21.15.130:0/2130504272' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62f0fd9a-64fe-4680-9107-ae8a97490dec"}]: dispatch 2024-04-24T08:57:13.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:13 smithi069 ceph-mon[28170]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62f0fd9a-64fe-4680-9107-ae8a97490dec"}]: dispatch 2024-04-24T08:57:13.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:13 smithi069 ceph-mon[28170]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62f0fd9a-64fe-4680-9107-ae8a97490dec"}]': finished 2024-04-24T08:57:13.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:13 smithi069 ceph-mon[28170]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T08:57:13.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:13 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:13.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:13 smithi130 ceph-mon[31553]: from='client.? 172.21.15.130:0/2130504272' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62f0fd9a-64fe-4680-9107-ae8a97490dec"}]: dispatch 2024-04-24T08:57:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:13 smithi130 ceph-mon[31553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62f0fd9a-64fe-4680-9107-ae8a97490dec"}]: dispatch 2024-04-24T08:57:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:13 smithi130 ceph-mon[31553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62f0fd9a-64fe-4680-9107-ae8a97490dec"}]': finished 2024-04-24T08:57:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:13 smithi130 ceph-mon[31553]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T08:57:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:13 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:14.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:14 smithi081 ceph-mon[30961]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:14.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:14 smithi081 ceph-mon[30961]: from='client.? 172.21.15.130:0/1026871544' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:57:14.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:14 smithi069 ceph-mon[28170]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:14.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:14 smithi069 ceph-mon[28170]: from='client.? 172.21.15.130:0/1026871544' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:57:14.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:14 smithi130 ceph-mon[31553]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:14.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:14 smithi130 ceph-mon[31553]: from='client.? 172.21.15.130:0/1026871544' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:57:16.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:16 smithi069 ceph-mon[28170]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:16.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:16 smithi130 ceph-mon[31553]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:16.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:16 smithi081 ceph-mon[30961]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:18.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:18 smithi130 ceph-mon[31553]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:18.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:18 smithi069 ceph-mon[28170]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:18.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:18 smithi081 ceph-mon[30961]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:20.569 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:20 smithi130 ceph-mon[31553]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:20.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:20 smithi069 ceph-mon[28170]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:20.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:20 smithi081 ceph-mon[30961]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:22.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:22 smithi130 ceph-mon[31553]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:22.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T08:57:22.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:22.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:22 smithi069 ceph-mon[28170]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:22.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T08:57:22.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:22.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:22 smithi081 ceph-mon[30961]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:22.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T08:57:22.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:23.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:23 smithi130 ceph-mon[31553]: Deploying daemon osd.2 on smithi130 2024-04-24T08:57:23.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:23 smithi081 ceph-mon[30961]: Deploying daemon osd.2 on smithi130 2024-04-24T08:57:23.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:23 smithi069 ceph-mon[28170]: Deploying daemon osd.2 on smithi130 2024-04-24T08:57:24.672 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:24 smithi130 ceph-mon[31553]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:24.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:24 smithi081 ceph-mon[30961]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:24.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:24 smithi069 ceph-mon[28170]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:26.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:26 smithi130 ceph-mon[31553]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:26.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:26 smithi081 ceph-mon[30961]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:26.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:26 smithi069 ceph-mon[28170]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:28.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:28 smithi130 ceph-mon[31553]: pgmap v97: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:28.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:28 smithi081 ceph-mon[30961]: pgmap v97: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:28.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:28 smithi069 ceph-mon[28170]: pgmap v97: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:30.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:30 smithi130 ceph-mon[31553]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:30.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:30 smithi081 ceph-mon[30961]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:30.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:30 smithi069 ceph-mon[28170]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:31.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:31.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:31.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:57:31.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:31 smithi130 ceph-mon[31553]: pgmap v99: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:31.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:31 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:31.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:31 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:31.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:31 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:57:31.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:31 smithi081 ceph-mon[30961]: pgmap v99: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:31.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:31.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:31.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:57:31.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:31 smithi069 ceph-mon[28170]: pgmap v99: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:34.563 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:34 smithi130 ceph-mon[31553]: pgmap v100: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:34.563 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:34 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:34.563 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:34 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:34.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:34 smithi081 ceph-mon[30961]: pgmap v100: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:34.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:34 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:34.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:34 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:34.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:34 smithi069 ceph-mon[28170]: pgmap v100: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:34.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:34 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:34.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:34 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:35.345 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 2 on host 'smithi130' 2024-04-24T08:57:35.345 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:35.344+0000 7efe56000700 1 -- 172.21.15.130:0/76248324 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efe58062f30 con 0x7efe400a3bb0 2024-04-24T08:57:35.348 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:35.348+0000 7efe60200700 1 -- 172.21.15.130:0/76248324 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efe400a3bb0 msgr2=0x7efe400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:35.348 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:35.348+0000 7efe60200700 1 --2- 172.21.15.130:0/76248324 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efe400a3bb0 0x7efe400a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efe4c006fd0 tx=0x7efe4c008040 comp rx=0 tx=0).stop 2024-04-24T08:57:35.348 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:35.348+0000 7efe60200700 1 -- 172.21.15.130:0/76248324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe58105f90 msgr2=0x7efe5819b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:35.348 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:35.348+0000 7efe60200700 1 --2- 172.21.15.130:0/76248324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe58105f90 0x7efe5819b470 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efe48005910 tx=0x7efe48009f90 comp rx=0 tx=0).stop 2024-04-24T08:57:35.349 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:35.348+0000 7efe60200700 1 -- 172.21.15.130:0/76248324 shutdown_connections 2024-04-24T08:57:35.349 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:35.348+0000 7efe60200700 1 --2- 172.21.15.130:0/76248324 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efe400a3bb0 0x7efe400a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:35.349 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:35.348+0000 7efe60200700 1 --2- 172.21.15.130:0/76248324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe58106930 0x7efe581a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:35.349 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:35.348+0000 7efe60200700 1 --2- 172.21.15.130:0/76248324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe58104d90 0x7efe5819af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:35.349 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:35.348+0000 7efe60200700 1 --2- 172.21.15.130:0/76248324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe58105f90 0x7efe5819b470 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:35.349 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:35.348+0000 7efe60200700 1 -- 172.21.15.130:0/76248324 >> 172.21.15.130:0/76248324 conn(0x7efe58100350 msgr2=0x7efe58109290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:35.349 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:35.348+0000 7efe60200700 1 -- 172.21.15.130:0/76248324 shutdown_connections 2024-04-24T08:57:35.349 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:57:35.348+0000 7efe60200700 1 -- 172.21.15.130:0/76248324 wait complete. 2024-04-24T08:57:35.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:35 smithi081 ceph-mon[30961]: from='osd.2 [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T08:57:35.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:35 smithi081 ceph-mon[30961]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T08:57:35.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:35 smithi069 ceph-mon[28170]: from='osd.2 [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T08:57:35.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:35 smithi069 ceph-mon[28170]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T08:57:35.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:35 smithi130 ceph-mon[31553]: from='osd.2 [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T08:57:35.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:35 smithi130 ceph-mon[31553]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T08:57:36.172 DEBUG:teuthology.orchestra.run.smithi130:osd.2> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.2.service 2024-04-24T08:57:36.176 INFO:tasks.cephadm:Deploying osd.3 on smithi069 with /dev/vg_nvme/lv_3... 2024-04-24T08:57:36.176 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T08:57:36.426 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:36 smithi130 ceph-mon[31553]: pgmap v101: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:36.427 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:36 smithi130 ceph-mon[31553]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T08:57:36.427 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:36 smithi130 ceph-mon[31553]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T08:57:36.427 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:36 smithi130 ceph-mon[31553]: from='osd.2 [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T08:57:36.427 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:36 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:36.427 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:36 smithi130 ceph-mon[31553]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T08:57:36.427 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:36 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:36.427 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:36 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:36.427 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 08:57:36 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-2[36838]: 2024-04-24T08:57:36.340+0000 7fa626200700 -1 osd.2 0 waiting for initial osdmap 2024-04-24T08:57:36.427 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 08:57:36 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-2[36838]: 2024-04-24T08:57:36.346+0000 7fa621000700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:57:36.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:36 smithi081 ceph-mon[30961]: pgmap v101: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:36.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:36 smithi081 ceph-mon[30961]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T08:57:36.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:36 smithi081 ceph-mon[30961]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T08:57:36.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:36 smithi081 ceph-mon[30961]: from='osd.2 [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T08:57:36.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:36 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:36.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:36 smithi081 ceph-mon[30961]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T08:57:36.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:36 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:36.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:36 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:36.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:36 smithi069 ceph-mon[28170]: pgmap v101: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:36.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:36 smithi069 ceph-mon[28170]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T08:57:36.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:36 smithi069 ceph-mon[28170]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T08:57:36.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:36 smithi069 ceph-mon[28170]: from='osd.2 [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T08:57:36.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:36 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:36.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:36 smithi069 ceph-mon[28170]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T08:57:36.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:36 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:36.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:36 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:36.618 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:57:37.348 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:37 smithi069 ceph-mon[28170]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-24T08:57:37.349 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:37 smithi069 ceph-mon[28170]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T08:57:37.349 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:37 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:37.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:37 smithi081 ceph-mon[30961]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-24T08:57:37.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:37 smithi081 ceph-mon[30961]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T08:57:37.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:37.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:37.702 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:37 smithi130 ceph-mon[31553]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-24T08:57:37.702 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:37 smithi130 ceph-mon[31553]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T08:57:37.702 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:37 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:37.702 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:37 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:37.779 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:37 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:38 smithi069 ceph-mon[28170]: purged_snaps scrub starts 2024-04-24T08:57:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:38 smithi069 ceph-mon[28170]: purged_snaps scrub ok 2024-04-24T08:57:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:38 smithi069 ceph-mon[28170]: pgmap v104: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:38 smithi069 ceph-mon[28170]: osd.2 [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638] boot 2024-04-24T08:57:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:38 smithi069 ceph-mon[28170]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T08:57:38.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:38.533 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T08:57:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:38 smithi130 ceph-mon[31553]: purged_snaps scrub starts 2024-04-24T08:57:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:38 smithi130 ceph-mon[31553]: purged_snaps scrub ok 2024-04-24T08:57:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:38 smithi130 ceph-mon[31553]: pgmap v104: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:38 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:38 smithi130 ceph-mon[31553]: osd.2 [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638] boot 2024-04-24T08:57:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:38 smithi130 ceph-mon[31553]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T08:57:38.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:38 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:38.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:38 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T08:57:38.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:38 smithi081 ceph-mon[30961]: purged_snaps scrub starts 2024-04-24T08:57:38.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:38 smithi081 ceph-mon[30961]: purged_snaps scrub ok 2024-04-24T08:57:38.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:38 smithi081 ceph-mon[30961]: pgmap v104: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:57:38.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:38 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:38.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:38 smithi081 ceph-mon[30961]: osd.2 [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638] boot 2024-04-24T08:57:38.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:38 smithi081 ceph-mon[30961]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T08:57:38.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:38 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:57:38.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:38 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T08:57:39.428 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T08:57:39.428 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T08:57:39.428 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-04-24T08:57:39.428 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-04-24T08:57:39.428 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0368245 s, 285 MB/s 2024-04-24T08:57:39.428 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-04-24T08:57:39.660 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:39 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T08:57:39.660 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:39 smithi069 ceph-mon[28170]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T08:57:39.660 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:39 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T08:57:39.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:39 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T08:57:39.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:39 smithi130 ceph-mon[31553]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T08:57:39.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:39 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T08:57:39.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:39 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T08:57:39.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:39 smithi081 ceph-mon[30961]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T08:57:39.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:39 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T08:57:39.917 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 08:57:39 smithi069 sudo[44432]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:57:39.917 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 08:57:39 smithi069 sudo[44432]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:57:39.917 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 08:57:39 smithi069 sudo[44432]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:57:39.917 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 08:57:39 smithi069 sudo[44432]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:57:39.917 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 08:57:39 smithi069 sudo[44435]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:57:39.917 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 08:57:39 smithi069 sudo[44435]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:57:39.917 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 08:57:39 smithi069 sudo[44435]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:57:39.917 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 08:57:39 smithi069 sudo[44435]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:57:40.254 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 08:57:39 smithi130 sudo[40533]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:57:40.255 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 08:57:39 smithi130 sudo[40533]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:57:40.255 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 08:57:39 smithi130 sudo[40533]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:57:40.255 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 08:57:40 smithi130 sudo[40533]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:57:40.255 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 08:57:40 smithi130 sudo[40592]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:57:40.255 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 08:57:40 smithi130 sudo[40592]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:57:40.255 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 08:57:40 smithi130 sudo[40592]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:57:40.255 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 08:57:40 smithi130 sudo[40592]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:57:40.255 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 sudo[40605]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:57:40.255 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 sudo[40605]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:57:40.255 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 sudo[40605]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:57:40.256 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 sudo[40605]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:57:40.298 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_3 2024-04-24T08:57:40.317 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 08:57:39 smithi081 sudo[39407]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:57:40.317 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 08:57:39 smithi081 sudo[39407]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:57:40.318 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 08:57:39 smithi081 sudo[39407]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:57:40.318 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 08:57:39 smithi081 sudo[39407]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:57:40.318 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 08:57:39 smithi081 sudo[39410]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:57:40.318 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 08:57:39 smithi081 sudo[39410]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:57:40.318 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 08:57:39 smithi081 sudo[39410]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:57:40.318 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 08:57:39 smithi081 sudo[39410]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:57:40.363 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 sudo[44448]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:57:40.363 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 sudo[44448]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:57:40.363 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 sudo[44448]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:57:40.363 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 sudo[44448]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:57:40.363 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 sudo[44451]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:57:40.363 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 sudo[44451]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:57:40.363 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 sudo[44451]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:57:40.363 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 sudo[44451]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:57:40.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 sudo[40611]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:57:40.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 sudo[40611]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:57:40.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 sudo[40611]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:57:40.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 sudo[40611]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:57:40.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: pgmap v107: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:40.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T08:57:40.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T08:57:40.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T08:57:40.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:40.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:40.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:40.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:57:40.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:57:40.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:57:40.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:57:40.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:57:40.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:57:40.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:57:40.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:57:40.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:57:40.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:57:40.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:57:40.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:57:40.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:57:40.728 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:57:40.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 sudo[39413]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:57:40.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 sudo[39413]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:57:40.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 sudo[39413]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:57:40.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 sudo[39413]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:57:40.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 sudo[39416]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:57:40.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 sudo[39416]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:57:40.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 sudo[39416]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:57:40.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 sudo[39416]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:57:40.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: pgmap v107: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:40.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T08:57:40.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T08:57:40.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T08:57:40.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:40.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:40.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:40.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:57:40.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:57:40.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:57:40.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:57:40.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:57:40.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:57:40.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:57:40.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:57:40.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:57:40.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:57:40.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:57:40.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:57:40.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:57:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: pgmap v107: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T08:57:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T08:57:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T08:57:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:40.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:57:40.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:57:40.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:57:40.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:57:40.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:57:40.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:57:40.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:57:40.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:57:40.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:57:40.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:57:40.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:57:40.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:57:40.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:57:41.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:41 smithi069 ceph-mon[28170]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:57:41.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:41 smithi069 ceph-mon[28170]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:57:41.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:41 smithi069 ceph-mon[28170]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T08:57:41.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:41.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:41.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:57:41.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:41.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:41.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:57:41.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:41.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:57:41.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:41.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:57:41.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:41.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:41 smithi130 ceph-mon[31553]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:57:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:41 smithi130 ceph-mon[31553]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:57:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:41 smithi130 ceph-mon[31553]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T08:57:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:57:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:57:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:57:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:57:41.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:41.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:41 smithi081 ceph-mon[30961]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:57:41.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:41 smithi081 ceph-mon[30961]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:57:41.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:41 smithi081 ceph-mon[30961]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T08:57:41.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:41.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:41.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:57:41.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:41.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:41.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:57:41.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:41.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:57:41.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:41.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:57:41.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:57:42.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:42 smithi130 ceph-mon[31553]: Detected new or changed devices on smithi130 2024-04-24T08:57:42.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:42 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi130 to 17089M 2024-04-24T08:57:42.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:42 smithi130 ceph-mon[31553]: pgmap v110: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:42.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:42 smithi130 ceph-mon[31553]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T08:57:42.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:42 smithi130 ceph-mon[31553]: Cluster is now healthy 2024-04-24T08:57:42.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:42 smithi130 ceph-mon[31553]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-24T08:57:42.732 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:42 smithi069 ceph-mon[28170]: Detected new or changed devices on smithi130 2024-04-24T08:57:42.732 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:42 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi130 to 17089M 2024-04-24T08:57:42.732 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:42 smithi069 ceph-mon[28170]: pgmap v110: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:42.732 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:42 smithi069 ceph-mon[28170]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T08:57:42.732 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:42 smithi069 ceph-mon[28170]: Cluster is now healthy 2024-04-24T08:57:42.732 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:42 smithi069 ceph-mon[28170]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-24T08:57:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:42 smithi081 ceph-mon[30961]: Detected new or changed devices on smithi130 2024-04-24T08:57:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:42 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi130 to 17089M 2024-04-24T08:57:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:42 smithi081 ceph-mon[30961]: pgmap v110: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:42 smithi081 ceph-mon[30961]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T08:57:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:42 smithi081 ceph-mon[30961]: Cluster is now healthy 2024-04-24T08:57:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:42 smithi081 ceph-mon[30961]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-24T08:57:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.904+0000 7f9b5be00700 1 -- 172.21.15.69:0/1023315125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b5c102a70 msgr2=0x7f9b5c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.904+0000 7f9b5be00700 1 --2- 172.21.15.69:0/1023315125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b5c102a70 0x7f9b5c102e90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0099c0 tx=0x7f9b4c009cd0 comp rx=0 tx=0).stop 2024-04-24T08:57:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.904+0000 7f9b5be00700 1 -- 172.21.15.69:0/1023315125 shutdown_connections 2024-04-24T08:57:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.904+0000 7f9b5be00700 1 --2- 172.21.15.69:0/1023315125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5c104610 0x7f9b5c10aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.904+0000 7f9b5be00700 1 --2- 172.21.15.69:0/1023315125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b5c102a70 0x7f9b5c102e90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.904+0000 7f9b5be00700 1 --2- 172.21.15.69:0/1023315125 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b5c103c70 0x7f9b5c1040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.904+0000 7f9b5be00700 1 -- 172.21.15.69:0/1023315125 >> 172.21.15.69:0/1023315125 conn(0x7f9b5c0fe030 msgr2=0x7f9b5c100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.904+0000 7f9b5be00700 1 -- 172.21.15.69:0/1023315125 shutdown_connections 2024-04-24T08:57:42.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.904+0000 7f9b5be00700 1 -- 172.21.15.69:0/1023315125 wait complete. 2024-04-24T08:57:42.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.906+0000 7f9b5be00700 1 Processor -- start 2024-04-24T08:57:42.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.906+0000 7f9b5be00700 1 -- start start 2024-04-24T08:57:42.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.906+0000 7f9b5be00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5c102a70 0x7f9b5c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:42.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.906+0000 7f9b5be00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b5c103c70 0x7f9b5c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:42.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.906+0000 7f9b5be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b5c104610 0x7f9b5c1a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:42.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.906+0000 7f9b5be00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c10e550 con 0x7f9b5c104610 2024-04-24T08:57:42.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.906+0000 7f9b5be00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c10e3d0 con 0x7f9b5c102a70 2024-04-24T08:57:42.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.906+0000 7f9b5be00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c10e6d0 con 0x7f9b5c103c70 2024-04-24T08:57:42.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.906+0000 7f9b5b400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b5c104610 0x7f9b5c1a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:42.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.906+0000 7f9b5b400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b5c104610 0x7f9b5c1a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54754/0 (socket says 172.21.15.69:54754) 2024-04-24T08:57:42.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.906+0000 7f9b5b400700 1 -- 172.21.15.69:0/331144549 learned_addr learned my addr 172.21.15.69:0/331144549 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:57:42.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.907+0000 7f9b5aa00700 1 --2- 172.21.15.69:0/331144549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5c102a70 0x7f9b5c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:42.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.907+0000 7f9b53e00700 1 --2- 172.21.15.69:0/331144549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b5c103c70 0x7f9b5c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:42.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.907+0000 7f9b5b400700 1 -- 172.21.15.69:0/331144549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b5c103c70 msgr2=0x7f9b5c19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:42.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.907+0000 7f9b5b400700 1 --2- 172.21.15.69:0/331144549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b5c103c70 0x7f9b5c19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:42.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.907+0000 7f9b5b400700 1 -- 172.21.15.69:0/331144549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5c102a70 msgr2=0x7f9b5c19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:42.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.907+0000 7f9b5b400700 1 --2- 172.21.15.69:0/331144549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5c102a70 0x7f9b5c19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:42.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.907+0000 7f9b5b400700 1 -- 172.21.15.69:0/331144549 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c1a2a60 con 0x7f9b5c104610 2024-04-24T08:57:42.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.907+0000 7f9b5b400700 1 --2- 172.21.15.69:0/331144549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b5c104610 0x7f9b5c1a23c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9b54007f10 tx=0x7f9b5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:42.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.908+0000 7f9b53400700 1 -- 172.21.15.69:0/331144549 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400ddd0 con 0x7f9b5c104610 2024-04-24T08:57:42.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.908+0000 7f9b53400700 1 -- 172.21.15.69:0/331144549 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b54004020 con 0x7f9b5c104610 2024-04-24T08:57:42.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.908+0000 7f9b5be00700 1 -- 172.21.15.69:0/331144549 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5c1a2d50 con 0x7f9b5c104610 2024-04-24T08:57:42.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.908+0000 7f9b53400700 1 -- 172.21.15.69:0/331144549 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54015660 con 0x7f9b5c104610 2024-04-24T08:57:42.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.908+0000 7f9b5be00700 1 -- 172.21.15.69:0/331144549 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5c1a3230 con 0x7f9b5c104610 2024-04-24T08:57:42.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.909+0000 7f9b5be00700 1 -- 172.21.15.69:0/331144549 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b5c0697e0 con 0x7f9b5c104610 2024-04-24T08:57:42.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.911+0000 7f9b53400700 1 -- 172.21.15.69:0/331144549 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9b54003a00 con 0x7f9b5c104610 2024-04-24T08:57:42.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.913+0000 7f9b53400700 1 --2- 172.21.15.69:0/331144549 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9b400a3cc0 0x7f9b400a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:42.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.913+0000 7f9b53400700 1 -- 172.21.15.69:0/331144549 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f9b540ceac0 con 0x7f9b5c104610 2024-04-24T08:57:42.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.913+0000 7f9b5aa00700 1 --2- 172.21.15.69:0/331144549 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9b400a3cc0 0x7f9b400a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:42.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.914+0000 7f9b5aa00700 1 --2- 172.21.15.69:0/331144549 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9b400a3cc0 0x7f9b400a6180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c000c00 tx=0x7f9b4c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:42.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:42.918+0000 7f9b53400700 1 -- 172.21.15.69:0/331144549 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b54010070 con 0x7f9b5c104610 2024-04-24T08:57:43.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:57:43.153+0000 7f9b5be00700 1 -- 172.21.15.69:0/331144549 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9b5c076440 con 0x7f9b400a3cc0 2024-04-24T08:57:43.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:43 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:57:43.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:43 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:57:43.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:43 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:43.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:43 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:57:43.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:43 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:57:43.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:43 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:43.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:57:43.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:57:43.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:44.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:44 smithi130 ceph-mon[31553]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:57:44.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:44 smithi130 ceph-mon[31553]: pgmap v111: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:44.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:44 smithi081 ceph-mon[30961]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:57:44.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:44 smithi081 ceph-mon[30961]: pgmap v111: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:44.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:44 smithi069 ceph-mon[28170]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:57:44.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:44 smithi069 ceph-mon[28170]: pgmap v111: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:46.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:46 smithi130 ceph-mon[31553]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:46.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:46 smithi081 ceph-mon[30961]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:46.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:46 smithi069 ceph-mon[28170]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:47.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:47 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1672194653' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09c5d15c-129e-45cf-9891-5b7a7f4a450f"}]: dispatch 2024-04-24T08:57:47.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:47 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1672194653' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09c5d15c-129e-45cf-9891-5b7a7f4a450f"}]': finished 2024-04-24T08:57:47.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:47 smithi081 ceph-mon[30961]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T08:57:47.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:47 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:57:47.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:47 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1672194653' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09c5d15c-129e-45cf-9891-5b7a7f4a450f"}]: dispatch 2024-04-24T08:57:47.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:47 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1672194653' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09c5d15c-129e-45cf-9891-5b7a7f4a450f"}]': finished 2024-04-24T08:57:47.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:47 smithi130 ceph-mon[31553]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T08:57:47.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:47 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:57:47.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:47 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1672194653' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09c5d15c-129e-45cf-9891-5b7a7f4a450f"}]: dispatch 2024-04-24T08:57:47.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:47 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1672194653' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09c5d15c-129e-45cf-9891-5b7a7f4a450f"}]': finished 2024-04-24T08:57:47.769 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:47 smithi069 ceph-mon[28170]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T08:57:47.769 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:47 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:57:48.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:48 smithi130 ceph-mon[31553]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:48.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:48 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2834591641' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:57:48.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:48 smithi081 ceph-mon[30961]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:48.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:48 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2834591641' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:57:48.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:48 smithi069 ceph-mon[28170]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:48.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:48 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2834591641' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:57:49.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:49 smithi081 ceph-mon[30961]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:49.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:49 smithi069 ceph-mon[28170]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:49.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:49 smithi130 ceph-mon[31553]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:52.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:52 smithi081 ceph-mon[30961]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:52.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:52 smithi069 ceph-mon[28170]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:52.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:52 smithi130 ceph-mon[31553]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:54.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:54 smithi081 ceph-mon[30961]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:54.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:54 smithi069 ceph-mon[28170]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:54.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:54 smithi130 ceph-mon[31553]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:56.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:56 smithi069 ceph-mon[28170]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:56.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:56 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T08:57:56.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:56 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:56.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:56 smithi130 ceph-mon[31553]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:56.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:56 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T08:57:56.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:56 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:56.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:56 smithi081 ceph-mon[30961]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:56.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:56 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T08:57:56.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:56 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:57.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:57 smithi069 ceph-mon[28170]: Deploying daemon osd.3 on smithi069 2024-04-24T08:57:57.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:57 smithi130 ceph-mon[31553]: Deploying daemon osd.3 on smithi069 2024-04-24T08:57:57.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:57 smithi081 ceph-mon[30961]: Deploying daemon osd.3 on smithi069 2024-04-24T08:57:58.478 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:57:58 smithi069 ceph-mon[28170]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:58.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:57:58 smithi130 ceph-mon[31553]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:57:58.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:57:58 smithi081 ceph-mon[30961]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:00.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:00 smithi069 ceph-mon[28170]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:00.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:00 smithi130 ceph-mon[31553]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:00.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:00 smithi081 ceph-mon[30961]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:02.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:02 smithi130 ceph-mon[31553]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:02.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:02 smithi081 ceph-mon[30961]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:02 smithi069 ceph-mon[28170]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:04.619 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:04 smithi081 ceph-mon[30961]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:04.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:04 smithi130 ceph-mon[31553]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:04.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:04 smithi069 ceph-mon[28170]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:05.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:05 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:05.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:05 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:58:05.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:05 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:05.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:05 smithi069 ceph-mon[28170]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:05.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:05 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:05.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:05 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:58:05.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:05 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:05.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:05 smithi130 ceph-mon[31553]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:06.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:05 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:06.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:05 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:58:06.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:05 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:06.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:05 smithi081 ceph-mon[30961]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:08.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:08 smithi081 ceph-mon[30961]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:08.576 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:08 smithi069 ceph-mon[28170]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:08.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:08 smithi130 ceph-mon[31553]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:09.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:09 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:09.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:09 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:09.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:09 smithi081 ceph-mon[30961]: from='osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T08:58:09.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:09 smithi081 ceph-mon[30961]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:09.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:09 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:09.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:09 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:09.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:09 smithi069 ceph-mon[28170]: from='osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T08:58:09.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:09 smithi069 ceph-mon[28170]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:09.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:58:09.856+0000 7f9b53400700 1 -- 172.21.15.69:0/331144549 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9b5c076440 con 0x7f9b400a3cc0 2024-04-24T08:58:09.857 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 3 on host 'smithi069' 2024-04-24T08:58:09.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:58:09.860+0000 7f9b5be00700 1 -- 172.21.15.69:0/331144549 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9b400a3cc0 msgr2=0x7f9b400a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:09.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:58:09.860+0000 7f9b5be00700 1 --2- 172.21.15.69:0/331144549 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9b400a3cc0 0x7f9b400a6180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c000c00 tx=0x7f9b4c005fb0 comp rx=0 tx=0).stop 2024-04-24T08:58:09.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:58:09.861+0000 7f9b5be00700 1 -- 172.21.15.69:0/331144549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b5c104610 msgr2=0x7f9b5c1a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:09.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:58:09.861+0000 7f9b5be00700 1 --2- 172.21.15.69:0/331144549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b5c104610 0x7f9b5c1a23c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9b54007f10 tx=0x7f9b5400d3b0 comp rx=0 tx=0).stop 2024-04-24T08:58:09.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:58:09.861+0000 7f9b5be00700 1 -- 172.21.15.69:0/331144549 shutdown_connections 2024-04-24T08:58:09.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:58:09.861+0000 7f9b5be00700 1 --2- 172.21.15.69:0/331144549 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9b400a3cc0 0x7f9b400a6180 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:09.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:58:09.861+0000 7f9b5be00700 1 --2- 172.21.15.69:0/331144549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b5c104610 0x7f9b5c1a23c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:09.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:58:09.861+0000 7f9b5be00700 1 --2- 172.21.15.69:0/331144549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5c102a70 0x7f9b5c19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:09.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:58:09.861+0000 7f9b5be00700 1 --2- 172.21.15.69:0/331144549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b5c103c70 0x7f9b5c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:09.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:58:09.861+0000 7f9b5be00700 1 -- 172.21.15.69:0/331144549 >> 172.21.15.69:0/331144549 conn(0x7f9b5c0fe030 msgr2=0x7f9b5c074d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:09.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:58:09.861+0000 7f9b5be00700 1 -- 172.21.15.69:0/331144549 shutdown_connections 2024-04-24T08:58:09.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:58:09.861+0000 7f9b5be00700 1 -- 172.21.15.69:0/331144549 wait complete. 2024-04-24T08:58:09.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:09 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:09.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:09 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:09.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:09 smithi130 ceph-mon[31553]: from='osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T08:58:09.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:09 smithi130 ceph-mon[31553]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:10.709 DEBUG:teuthology.orchestra.run.smithi069:osd.3> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.3.service 2024-04-24T08:58:10.714 INFO:tasks.cephadm:Deploying osd.4 on smithi081 with /dev/vg_nvme/lv_3... 2024-04-24T08:58:10.714 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T08:58:10.798 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:10 smithi081 ceph-mon[30961]: from='osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T08:58:10.798 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:10 smithi081 ceph-mon[30961]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T08:58:10.798 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:10 smithi081 ceph-mon[30961]: from='osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-24T08:58:10.798 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:10 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:58:10.798 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:10 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:10.798 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:10 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:10.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:10 smithi069 ceph-mon[28170]: from='osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T08:58:10.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:10 smithi069 ceph-mon[28170]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T08:58:10.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:10 smithi069 ceph-mon[28170]: from='osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-24T08:58:10.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:10 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:58:10.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:10 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:10.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:10 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:10.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:10 smithi130 ceph-mon[31553]: from='osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T08:58:10.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:10 smithi130 ceph-mon[31553]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T08:58:10.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:10 smithi130 ceph-mon[31553]: from='osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-24T08:58:10.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:10 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:58:10.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:10 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:10.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:10 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:11.155 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T08:58:11.555 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 08:58:11 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3[46907]: 2024-04-24T08:58:11.381+0000 7f1a4a400700 -1 osd.3 0 waiting for initial osdmap 2024-04-24T08:58:11.555 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 08:58:11 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3[46907]: 2024-04-24T08:58:11.401+0000 7f1a40200700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:58:11.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:11 smithi081 ceph-mon[30961]: from='osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-24T08:58:11.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:11 smithi081 ceph-mon[30961]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T08:58:11.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:11 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:58:11.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:11 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:58:11.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:11 smithi081 ceph-mon[30961]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:11.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:11 smithi081 ceph-mon[30961]: from='osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529]' entity='osd.3' 2024-04-24T08:58:11.818 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:11 smithi069 ceph-mon[28170]: from='osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-24T08:58:11.818 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:11 smithi069 ceph-mon[28170]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T08:58:11.818 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:11 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:58:11.819 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:11 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:58:11.819 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:11 smithi069 ceph-mon[28170]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:11.819 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:11 smithi069 ceph-mon[28170]: from='osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529]' entity='osd.3' 2024-04-24T08:58:11.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:11 smithi130 ceph-mon[31553]: from='osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-24T08:58:11.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:11 smithi130 ceph-mon[31553]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T08:58:11.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:11 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:58:11.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:11 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:58:11.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:11 smithi130 ceph-mon[31553]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:58:11.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:11 smithi130 ceph-mon[31553]: from='osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529]' entity='osd.3' 2024-04-24T08:58:12.745 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:12 smithi081 ceph-mon[30961]: purged_snaps scrub starts 2024-04-24T08:58:12.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:12 smithi081 ceph-mon[30961]: purged_snaps scrub ok 2024-04-24T08:58:12.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:12 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:58:12.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:12 smithi081 ceph-mon[30961]: osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529] boot 2024-04-24T08:58:12.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:12 smithi081 ceph-mon[30961]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T08:58:12.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:12 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:58:12.746 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:12 smithi081 ceph-mon[30961]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T08:58:12.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:12 smithi069 ceph-mon[28170]: purged_snaps scrub starts 2024-04-24T08:58:12.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:12 smithi069 ceph-mon[28170]: purged_snaps scrub ok 2024-04-24T08:58:12.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:12 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:58:12.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:12 smithi069 ceph-mon[28170]: osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529] boot 2024-04-24T08:58:12.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:12 smithi069 ceph-mon[28170]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T08:58:12.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:12 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:58:12.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:12 smithi069 ceph-mon[28170]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T08:58:12.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:12 smithi130 ceph-mon[31553]: purged_snaps scrub starts 2024-04-24T08:58:12.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:12 smithi130 ceph-mon[31553]: purged_snaps scrub ok 2024-04-24T08:58:12.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:12 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:58:12.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:12 smithi130 ceph-mon[31553]: osd.3 [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529] boot 2024-04-24T08:58:12.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:12 smithi130 ceph-mon[31553]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T08:58:12.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:12 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:58:12.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:12 smithi130 ceph-mon[31553]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T08:58:13.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:13 smithi069 ceph-mon[28170]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:13.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:13 smithi069 ceph-mon[28170]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T08:58:13.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:13 smithi130 ceph-mon[31553]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:13.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:13 smithi130 ceph-mon[31553]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T08:58:13.956 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:13 smithi081 ceph-mon[30961]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:13.956 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:13 smithi081 ceph-mon[30961]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T08:58:13.956 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T08:58:13.957 INFO:teuthology.orchestra.run.smithi081.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T08:58:13.957 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10+0 records in 2024-04-24T08:58:13.957 INFO:teuthology.orchestra.run.smithi081.stderr:10+0 records out 2024-04-24T08:58:13.957 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0325898 s, 322 MB/s 2024-04-24T08:58:13.957 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping successful for: 2024-04-24T08:58:14.696 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi081:vg_nvme/lv_3 2024-04-24T08:58:15.130 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T08:58:15.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:15 smithi081 ceph-mon[30961]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T08:58:15.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:15 smithi081 ceph-mon[30961]: pgmap v134: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T08:58:15.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:15 smithi069 ceph-mon[28170]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T08:58:15.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:15 smithi069 ceph-mon[28170]: pgmap v134: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T08:58:15.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:15 smithi130 ceph-mon[31553]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T08:58:15.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:15 smithi130 ceph-mon[31553]: pgmap v134: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T08:58:16.760 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:16 smithi081 ceph-mon[30961]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T08:58:16.761 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:16 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:16.761 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:16 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:16.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:16 smithi069 ceph-mon[28170]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T08:58:16.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:16 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:16.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:16 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:16.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:16 smithi130 ceph-mon[31553]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T08:58:16.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:16 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:16.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:16 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.298+0000 7f68c4e00700 1 -- 172.21.15.81:0/3126788046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c010aaa0 msgr2=0x7f68c010cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.298+0000 7f68c4e00700 1 --2- 172.21.15.81:0/3126788046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c010aaa0 0x7f68c010cf10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f68b800b0a0 tx=0x7f68b800b3b0 comp rx=0 tx=0).stop 2024-04-24T08:58:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.298+0000 7f68c4e00700 1 -- 172.21.15.81:0/3126788046 shutdown_connections 2024-04-24T08:58:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.298+0000 7f68c4e00700 1 --2- 172.21.15.81:0/3126788046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c010aaa0 0x7f68c010cf10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.298+0000 7f68c4e00700 1 --2- 172.21.15.81:0/3126788046 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f68c0076aa0 0x7f68c0074f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.298+0000 7f68c4e00700 1 --2- 172.21.15.81:0/3126788046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68c00754d0 0x7f68c0075950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.298+0000 7f68c4e00700 1 -- 172.21.15.81:0/3126788046 >> 172.21.15.81:0/3126788046 conn(0x7f68c00fe140 msgr2=0x7f68c0100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.298+0000 7f68c4e00700 1 -- 172.21.15.81:0/3126788046 shutdown_connections 2024-04-24T08:58:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.299+0000 7f68c4e00700 1 -- 172.21.15.81:0/3126788046 wait complete. 2024-04-24T08:58:17.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.300+0000 7f68c4e00700 1 Processor -- start 2024-04-24T08:58:17.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.300+0000 7f68c4e00700 1 -- start start 2024-04-24T08:58:17.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.301+0000 7f68c4e00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f68c00754d0 0x7f68c019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:17.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.301+0000 7f68c4e00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c0076aa0 0x7f68c019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:17.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.301+0000 7f68c4e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68c010aaa0 0x7f68c01a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:17.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.301+0000 7f68c4e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01107f0 con 0x7f68c010aaa0 2024-04-24T08:58:17.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.301+0000 7f68c4e00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f68c0110670 con 0x7f68c0076aa0 2024-04-24T08:58:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.301+0000 7f68c4e00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f68c0110970 con 0x7f68c00754d0 2024-04-24T08:58:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.301+0000 7f68bd400700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c0076aa0 0x7f68c019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.301+0000 7f68bd400700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c0076aa0 0x7f68c019b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55142/0 (socket says 172.21.15.81:55142) 2024-04-24T08:58:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.301+0000 7f68bd400700 1 -- 172.21.15.81:0/3482681252 learned_addr learned my addr 172.21.15.81:0/3482681252 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-04-24T08:58:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.301+0000 7f68bde00700 1 --2- 172.21.15.81:0/3482681252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f68c00754d0 0x7f68c019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.301+0000 7f68be800700 1 --2- 172.21.15.81:0/3482681252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68c010aaa0 0x7f68c01a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.301+0000 7f68bd400700 1 -- 172.21.15.81:0/3482681252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f68c00754d0 msgr2=0x7f68c019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.301+0000 7f68bd400700 1 --2- 172.21.15.81:0/3482681252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f68c00754d0 0x7f68c019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.301+0000 7f68bd400700 1 -- 172.21.15.81:0/3482681252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68c010aaa0 msgr2=0x7f68c01a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.301+0000 7f68bd400700 1 --2- 172.21.15.81:0/3482681252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68c010aaa0 0x7f68c01a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.301+0000 7f68bd400700 1 -- 172.21.15.81:0/3482681252 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c01a2a30 con 0x7f68c0076aa0 2024-04-24T08:58:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.302+0000 7f68bd400700 1 --2- 172.21.15.81:0/3482681252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c0076aa0 0x7f68c019b310 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00d950 tx=0x7f68ac00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.302+0000 7f68b6a00700 1 -- 172.21.15.81:0/3482681252 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac016070 con 0x7f68c0076aa0 2024-04-24T08:58:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.302+0000 7f68c4e00700 1 -- 172.21.15.81:0/3482681252 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c01a2d20 con 0x7f68c0076aa0 2024-04-24T08:58:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.302+0000 7f68b6a00700 1 -- 172.21.15.81:0/3482681252 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68ac009df0 con 0x7f68c0076aa0 2024-04-24T08:58:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.302+0000 7f68b6a00700 1 -- 172.21.15.81:0/3482681252 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00f620 con 0x7f68c0076aa0 2024-04-24T08:58:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.302+0000 7f68c4e00700 1 -- 172.21.15.81:0/3482681252 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c01a3200 con 0x7f68c0076aa0 2024-04-24T08:58:17.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.304+0000 7f68c4e00700 1 -- 172.21.15.81:0/3482681252 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c00697e0 con 0x7f68c0076aa0 2024-04-24T08:58:17.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.305+0000 7f68b6a00700 1 -- 172.21.15.81:0/3482681252 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f68ac019070 con 0x7f68c0076aa0 2024-04-24T08:58:17.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.307+0000 7f68b6a00700 1 --2- 172.21.15.81:0/3482681252 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f68a40a3c40 0x7f68a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:17.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.307+0000 7f68b6a00700 1 -- 172.21.15.81:0/3482681252 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f68ac0cee00 con 0x7f68c0076aa0 2024-04-24T08:58:17.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.307+0000 7f68bde00700 1 --2- 172.21.15.81:0/3482681252 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f68a40a3c40 0x7f68a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:17.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.308+0000 7f68bde00700 1 --2- 172.21.15.81:0/3482681252 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f68a40a3c40 0x7f68a40a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f68b0007be0 tx=0x7f68b00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:17.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.312+0000 7f68b6a00700 1 -- 172.21.15.81:0/3482681252 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f68ac09c960 con 0x7f68c0076aa0 2024-04-24T08:58:17.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:17.544+0000 7f68c4e00700 1 -- 172.21.15.81:0/3482681252 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f68c0062f30 con 0x7f68a40a3c40 2024-04-24T08:58:17.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:17 smithi081 ceph-mon[30961]: Detected new or changed devices on smithi069 2024-04-24T08:58:17.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:58:17.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:58:17.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:17 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi069 to 8544M 2024-04-24T08:58:17.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:17.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:17.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:58:17.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:17.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:58:17.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:17.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:58:17.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:17.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:17 smithi081 ceph-mon[30961]: pgmap v136: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-04-24T08:58:17.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:17 smithi069 ceph-mon[28170]: Detected new or changed devices on smithi069 2024-04-24T08:58:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:58:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:58:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:17 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi069 to 8544M 2024-04-24T08:58:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:58:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:58:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:58:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:17.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:17 smithi069 ceph-mon[28170]: pgmap v136: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-04-24T08:58:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:17 smithi130 ceph-mon[31553]: Detected new or changed devices on smithi069 2024-04-24T08:58:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:58:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:58:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:17 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi069 to 8544M 2024-04-24T08:58:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:58:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:58:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:58:17.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:17.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:17 smithi130 ceph-mon[31553]: pgmap v136: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-04-24T08:58:18.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:18 smithi081 ceph-mon[30961]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:58:18.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:18 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:58:18.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:18 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:58:18.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:18 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:18.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:18 smithi069 ceph-mon[28170]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:58:18.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:18 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:58:18.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:18 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:58:18.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:18 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:18.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:18 smithi130 ceph-mon[31553]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:58:18.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:18 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:58:18.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:18 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:58:18.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:18 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:19.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:19 smithi081 ceph-mon[30961]: pgmap v137: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T08:58:19.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:19 smithi069 ceph-mon[28170]: pgmap v137: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T08:58:19.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:19 smithi130 ceph-mon[31553]: pgmap v137: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T08:58:22.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:22 smithi069 ceph-mon[28170]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-24T08:58:22.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:22 smithi069 ceph-mon[28170]: from='client.? 172.21.15.81:0/2162231150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae0c645d-b56e-4348-88c5-bf4f6738e0a1"}]: dispatch 2024-04-24T08:58:22.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:22 smithi069 ceph-mon[28170]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae0c645d-b56e-4348-88c5-bf4f6738e0a1"}]: dispatch 2024-04-24T08:58:22.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:22 smithi069 ceph-mon[28170]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae0c645d-b56e-4348-88c5-bf4f6738e0a1"}]': finished 2024-04-24T08:58:22.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:22 smithi069 ceph-mon[28170]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T08:58:22.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:58:22.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:22 smithi069 ceph-mon[28170]: from='client.? 172.21.15.81:0/3334205093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:58:22.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:22 smithi130 ceph-mon[31553]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-24T08:58:22.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:22 smithi130 ceph-mon[31553]: from='client.? 172.21.15.81:0/2162231150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae0c645d-b56e-4348-88c5-bf4f6738e0a1"}]: dispatch 2024-04-24T08:58:22.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:22 smithi130 ceph-mon[31553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae0c645d-b56e-4348-88c5-bf4f6738e0a1"}]: dispatch 2024-04-24T08:58:22.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:22 smithi130 ceph-mon[31553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae0c645d-b56e-4348-88c5-bf4f6738e0a1"}]': finished 2024-04-24T08:58:22.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:22 smithi130 ceph-mon[31553]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T08:58:22.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:58:22.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:22 smithi130 ceph-mon[31553]: from='client.? 172.21.15.81:0/3334205093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:58:22.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:22 smithi081 ceph-mon[30961]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-24T08:58:22.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:22 smithi081 ceph-mon[30961]: from='client.? 172.21.15.81:0/2162231150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae0c645d-b56e-4348-88c5-bf4f6738e0a1"}]: dispatch 2024-04-24T08:58:22.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:22 smithi081 ceph-mon[30961]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae0c645d-b56e-4348-88c5-bf4f6738e0a1"}]: dispatch 2024-04-24T08:58:22.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:22 smithi081 ceph-mon[30961]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae0c645d-b56e-4348-88c5-bf4f6738e0a1"}]': finished 2024-04-24T08:58:22.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:22 smithi081 ceph-mon[30961]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T08:58:22.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:58:22.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:22 smithi081 ceph-mon[30961]: from='client.? 172.21.15.81:0/3334205093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:58:24.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:24 smithi069 ceph-mon[28170]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:24.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:24 smithi130 ceph-mon[31553]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:24.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:24 smithi081 ceph-mon[30961]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:26.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:26 smithi069 ceph-mon[28170]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:26.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:26 smithi130 ceph-mon[31553]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:26.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:26 smithi081 ceph-mon[30961]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:28.433 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:28 smithi081 ceph-mon[30961]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:28.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:28 smithi069 ceph-mon[28170]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:28.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:28 smithi130 ceph-mon[31553]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:30.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:30 smithi130 ceph-mon[31553]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:30.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:30 smithi081 ceph-mon[30961]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:30.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:30 smithi069 ceph-mon[28170]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:31.610 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:31 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T08:58:31.611 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:31 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:31.611 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:31 smithi081 ceph-mon[30961]: Deploying daemon osd.4 on smithi081 2024-04-24T08:58:31.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T08:58:31.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:31.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:31 smithi130 ceph-mon[31553]: Deploying daemon osd.4 on smithi081 2024-04-24T08:58:31.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T08:58:31.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:31.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:31 smithi069 ceph-mon[28170]: Deploying daemon osd.4 on smithi081 2024-04-24T08:58:32.537 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:32 smithi081 ceph-mon[30961]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:32.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:32 smithi130 ceph-mon[31553]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:32.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:32 smithi069 ceph-mon[28170]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:34.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:34 smithi081 ceph-mon[30961]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:34.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:34 smithi069 ceph-mon[28170]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:34.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:34 smithi130 ceph-mon[31553]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:36.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:36 smithi130 ceph-mon[31553]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:36.787 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:36 smithi081 ceph-mon[30961]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:36.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:36 smithi069 ceph-mon[28170]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:38.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:38 smithi081 ceph-mon[30961]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:38.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:38 smithi130 ceph-mon[31553]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:38.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:38 smithi069 ceph-mon[28170]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:39.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:39 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:39.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:39 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:58:39.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:39 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:39.736 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:39 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:39.736 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:39 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:58:39.736 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:39 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:39.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:39 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:39.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:39 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:58:39.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:39 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:40 smithi081 ceph-mon[30961]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:40 smithi069 ceph-mon[28170]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:40.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:40 smithi130 ceph-mon[31553]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:42.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:42 smithi081 ceph-mon[30961]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:42.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:42.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:42.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:42 smithi081 ceph-mon[30961]: from='osd.4 [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:58:42.719 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:42 smithi081 ceph-mon[30961]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:58:42.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:42 smithi069 ceph-mon[28170]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:42.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:42.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:42.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:42 smithi069 ceph-mon[28170]: from='osd.4 [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:58:42.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:42 smithi069 ceph-mon[28170]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:58:42.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:42 smithi130 ceph-mon[31553]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:42.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:42.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:42.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:42 smithi130 ceph-mon[31553]: from='osd.4 [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:58:42.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:42 smithi130 ceph-mon[31553]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:58:43.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:43.163+0000 7f68b6a00700 1 -- 172.21.15.81:0/3482681252 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f68c0062f30 con 0x7f68a40a3c40 2024-04-24T08:58:43.164 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 4 on host 'smithi081' 2024-04-24T08:58:43.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:43.166+0000 7f68c4e00700 1 -- 172.21.15.81:0/3482681252 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f68a40a3c40 msgr2=0x7f68a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:43.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:43.166+0000 7f68c4e00700 1 --2- 172.21.15.81:0/3482681252 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f68a40a3c40 0x7f68a40a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f68b0007be0 tx=0x7f68b00073d0 comp rx=0 tx=0).stop 2024-04-24T08:58:43.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:43.167+0000 7f68c4e00700 1 -- 172.21.15.81:0/3482681252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c0076aa0 msgr2=0x7f68c019b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:43.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:43.167+0000 7f68c4e00700 1 --2- 172.21.15.81:0/3482681252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c0076aa0 0x7f68c019b310 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00d950 tx=0x7f68ac00dc60 comp rx=0 tx=0).stop 2024-04-24T08:58:43.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:43.167+0000 7f68c4e00700 1 -- 172.21.15.81:0/3482681252 shutdown_connections 2024-04-24T08:58:43.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:43.167+0000 7f68c4e00700 1 --2- 172.21.15.81:0/3482681252 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f68a40a3c40 0x7f68a40a6100 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:43.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:43.167+0000 7f68c4e00700 1 --2- 172.21.15.81:0/3482681252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68c010aaa0 0x7f68c01a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:43.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:43.167+0000 7f68c4e00700 1 --2- 172.21.15.81:0/3482681252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f68c00754d0 0x7f68c019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:43.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:43.167+0000 7f68c4e00700 1 --2- 172.21.15.81:0/3482681252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c0076aa0 0x7f68c019b310 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:43.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:43.167+0000 7f68c4e00700 1 -- 172.21.15.81:0/3482681252 >> 172.21.15.81:0/3482681252 conn(0x7f68c00fe140 msgr2=0x7f68c01069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:43.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:43.167+0000 7f68c4e00700 1 -- 172.21.15.81:0/3482681252 shutdown_connections 2024-04-24T08:58:43.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T08:58:43.167+0000 7f68c4e00700 1 -- 172.21.15.81:0/3482681252 wait complete. 2024-04-24T08:58:43.961 DEBUG:teuthology.orchestra.run.smithi081:osd.4> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.4.service 2024-04-24T08:58:43.965 INFO:tasks.cephadm:Deploying osd.5 on smithi130 with /dev/vg_nvme/lv_3... 2024-04-24T08:58:43.966 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T08:58:44.160 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:43 smithi081 ceph-mon[30961]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T08:58:44.160 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:43 smithi081 ceph-mon[30961]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T08:58:44.160 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:43 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:58:44.160 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:43 smithi081 ceph-mon[30961]: from='osd.4 [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T08:58:44.160 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:43 smithi081 ceph-mon[30961]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T08:58:44.160 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:43 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:44.160 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:43 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:44.160 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:43 smithi081 ceph-mon[30961]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:44.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:43 smithi069 ceph-mon[28170]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T08:58:44.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:43 smithi069 ceph-mon[28170]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T08:58:44.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:58:44.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:43 smithi069 ceph-mon[28170]: from='osd.4 [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T08:58:44.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:43 smithi069 ceph-mon[28170]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T08:58:44.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:44.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:44.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:43 smithi069 ceph-mon[28170]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:44.412 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:58:44.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:43 smithi130 ceph-mon[31553]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T08:58:44.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:43 smithi130 ceph-mon[31553]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T08:58:44.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:43 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:58:44.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:43 smithi130 ceph-mon[31553]: from='osd.4 [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T08:58:44.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:43 smithi130 ceph-mon[31553]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T08:58:44.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:43 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:44.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:43 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:44.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:43 smithi130 ceph-mon[31553]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:45.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:44 smithi130 ceph-mon[31553]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-04-24T08:58:45.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:44 smithi130 ceph-mon[31553]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T08:58:45.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:44 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:58:45.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:44 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:58:45.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:44 smithi081 ceph-mon[30961]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-04-24T08:58:45.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:44 smithi081 ceph-mon[30961]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T08:58:45.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:44 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:58:45.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:44 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:58:45.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:44 smithi069 ceph-mon[28170]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-04-24T08:58:45.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:44 smithi069 ceph-mon[28170]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T08:58:45.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:44 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:58:45.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:44 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:58:46.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:45 smithi081 ceph-mon[30961]: purged_snaps scrub starts 2024-04-24T08:58:46.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:45 smithi081 ceph-mon[30961]: purged_snaps scrub ok 2024-04-24T08:58:46.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:45 smithi081 ceph-mon[30961]: osd.4 [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990] boot 2024-04-24T08:58:46.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:45 smithi081 ceph-mon[30961]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T08:58:46.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:45 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:58:46.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:45 smithi081 ceph-mon[30961]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:46.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:45 smithi069 ceph-mon[28170]: purged_snaps scrub starts 2024-04-24T08:58:46.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:45 smithi069 ceph-mon[28170]: purged_snaps scrub ok 2024-04-24T08:58:46.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:45 smithi069 ceph-mon[28170]: osd.4 [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990] boot 2024-04-24T08:58:46.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:45 smithi069 ceph-mon[28170]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T08:58:46.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:45 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:58:46.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:45 smithi069 ceph-mon[28170]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:46.395 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:45 smithi130 ceph-mon[31553]: purged_snaps scrub starts 2024-04-24T08:58:46.395 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:45 smithi130 ceph-mon[31553]: purged_snaps scrub ok 2024-04-24T08:58:46.395 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:45 smithi130 ceph-mon[31553]: osd.4 [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990] boot 2024-04-24T08:58:46.395 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:45 smithi130 ceph-mon[31553]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T08:58:46.395 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:45 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:58:46.395 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:45 smithi130 ceph-mon[31553]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:58:47.099 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:47 smithi130 ceph-mon[31553]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T08:58:47.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:47 smithi081 ceph-mon[30961]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T08:58:47.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:47 smithi069 ceph-mon[28170]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T08:58:47.419 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T08:58:47.420 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T08:58:47.420 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-04-24T08:58:47.420 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-04-24T08:58:47.420 INFO:teuthology.orchestra.run.smithi130.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0359887 s, 291 MB/s 2024-04-24T08:58:47.420 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-04-24T08:58:48.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:48 smithi130 ceph-mon[31553]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:48.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:48 smithi081 ceph-mon[30961]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:48.330 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_3 2024-04-24T08:58:48.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:48 smithi069 ceph-mon[28170]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:48.752 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:58:50.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:49 smithi069 ceph-mon[28170]: Detected new or changed devices on smithi081 2024-04-24T08:58:50.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:50.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:50.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:58:50.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:58:50.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:49 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi081 to 8544M 2024-04-24T08:58:50.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:50.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:50.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:58:50.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:50.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:49 smithi069 ceph-mon[28170]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:50.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:58:50.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:50.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:58:50.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:50.140 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:49 smithi130 ceph-mon[31553]: Detected new or changed devices on smithi081 2024-04-24T08:58:50.140 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:50.140 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:50.140 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:58:50.140 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:58:50.141 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:49 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi081 to 8544M 2024-04-24T08:58:50.141 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:50.141 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:50.141 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:58:50.141 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:50.141 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:49 smithi130 ceph-mon[31553]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:50.141 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:58:50.141 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:50.141 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:58:50.141 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:50.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:49 smithi081 ceph-mon[30961]: Detected new or changed devices on smithi081 2024-04-24T08:58:50.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:50.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:50.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:58:50.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:58:50.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:49 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi081 to 8544M 2024-04-24T08:58:50.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:50.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:50.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:58:50.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:50.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:49 smithi081 ceph-mon[30961]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:50.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:58:50.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:50.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:58:50.319 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:58:50.968 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.968+0000 7f420ce00700 1 -- 172.21.15.130:0/3795782392 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4208101440 msgr2=0x7f4208101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:50.968 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.968+0000 7f420ce00700 1 --2- 172.21.15.130:0/3795782392 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4208101440 0x7f4208101860 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009990 tx=0x7f41f8009ca0 comp rx=0 tx=0).stop 2024-04-24T08:58:50.968 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.968+0000 7f420ce00700 1 -- 172.21.15.130:0/3795782392 shutdown_connections 2024-04-24T08:58:50.968 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.968+0000 7f420ce00700 1 --2- 172.21.15.130:0/3795782392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4208102760 0x7f4208102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:50.968 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.968+0000 7f420ce00700 1 --2- 172.21.15.130:0/3795782392 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4208101440 0x7f4208101860 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:50.968 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.968+0000 7f420ce00700 1 --2- 172.21.15.130:0/3795782392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4208101da0 0x7f4208102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:50.968 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.968+0000 7f420ce00700 1 -- 172.21.15.130:0/3795782392 >> 172.21.15.130:0/3795782392 conn(0x7f42080fcfe0 msgr2=0x7f42080ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:50.968 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.968+0000 7f420ce00700 1 -- 172.21.15.130:0/3795782392 shutdown_connections 2024-04-24T08:58:50.968 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.968+0000 7f420ce00700 1 -- 172.21.15.130:0/3795782392 wait complete. 2024-04-24T08:58:50.969 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.969+0000 7f420ce00700 1 Processor -- start 2024-04-24T08:58:50.969 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.970+0000 7f420ce00700 1 -- start start 2024-04-24T08:58:50.970 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.970+0000 7f420ce00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4208101440 0x7f420819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:50.970 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.970+0000 7f420ce00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4208101da0 0x7f420819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:50.970 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.970+0000 7f420ce00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4208102760 0x7f42081a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:50.970 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.970+0000 7f420ce00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4208110790 con 0x7f4208101da0 2024-04-24T08:58:50.970 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.970+0000 7f420ce00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4208110610 con 0x7f4208101440 2024-04-24T08:58:50.970 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.970+0000 7f420ce00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4208110910 con 0x7f4208102760 2024-04-24T08:58:50.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.970+0000 7f4206800700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4208102760 0x7f42081a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:50.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.971+0000 7f4206800700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4208102760 0x7f42081a23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:57954/0 (socket says 172.21.15.130:57954) 2024-04-24T08:58:50.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.971+0000 7f4206800700 1 -- 172.21.15.130:0/2306880435 learned_addr learned my addr 172.21.15.130:0/2306880435 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-24T08:58:50.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.971+0000 7f4205e00700 1 --2- 172.21.15.130:0/2306880435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4208101440 0x7f420819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:50.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.971+0000 7f41fd400700 1 --2- 172.21.15.130:0/2306880435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4208101da0 0x7f420819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:50.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.971+0000 7f4206800700 1 -- 172.21.15.130:0/2306880435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4208101440 msgr2=0x7f420819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:50.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.971+0000 7f4206800700 1 --2- 172.21.15.130:0/2306880435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4208101440 0x7f420819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:50.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.971+0000 7f4206800700 1 -- 172.21.15.130:0/2306880435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4208101da0 msgr2=0x7f420819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:50.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.971+0000 7f4206800700 1 --2- 172.21.15.130:0/2306880435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4208101da0 0x7f420819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:50.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.971+0000 7f4206800700 1 -- 172.21.15.130:0/2306880435 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42081a2a80 con 0x7f4208102760 2024-04-24T08:58:50.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.971+0000 7f4206800700 1 --2- 172.21.15.130:0/2306880435 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4208102760 0x7f42081a23e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f420000c9a0 tx=0x7f420000ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:50.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.972+0000 7f41ff400700 1 -- 172.21.15.130:0/2306880435 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200007ac0 con 0x7f4208102760 2024-04-24T08:58:50.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.972+0000 7f41ff400700 1 -- 172.21.15.130:0/2306880435 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4200004ce0 con 0x7f4208102760 2024-04-24T08:58:50.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.972+0000 7f420ce00700 1 -- 172.21.15.130:0/2306880435 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42081a2d70 con 0x7f4208102760 2024-04-24T08:58:50.973 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.972+0000 7f41ff400700 1 -- 172.21.15.130:0/2306880435 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200005470 con 0x7f4208102760 2024-04-24T08:58:50.973 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.972+0000 7f420ce00700 1 -- 172.21.15.130:0/2306880435 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42081a3250 con 0x7f4208102760 2024-04-24T08:58:50.973 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.973+0000 7f420ce00700 1 -- 172.21.15.130:0/2306880435 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42080697e0 con 0x7f4208102760 2024-04-24T08:58:50.976 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.977+0000 7f41ff400700 1 -- 172.21.15.130:0/2306880435 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4200004750 con 0x7f4208102760 2024-04-24T08:58:50.977 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.978+0000 7f41ff400700 1 --2- 172.21.15.130:0/2306880435 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f41ec0a3cc0 0x7f41ec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:50.978 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.978+0000 7f41ff400700 1 -- 172.21.15.130:0/2306880435 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f42000ceec0 con 0x7f4208102760 2024-04-24T08:58:50.978 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.979+0000 7f4205e00700 1 --2- 172.21.15.130:0/2306880435 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f41ec0a3cc0 0x7f41ec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:50.979 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.980+0000 7f4205e00700 1 --2- 172.21.15.130:0/2306880435 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f41ec0a3cc0 0x7f41ec0a6180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f41f8003780 tx=0x7f41f8003850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:50.981 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:50.981+0000 7f41ff400700 1 -- 172.21.15.130:0/2306880435 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4200014030 con 0x7f4208102760 2024-04-24T08:58:51.216 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:58:51.216+0000 7f420ce00700 1 -- 172.21.15.130:0/2306880435 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4208062f30 con 0x7f41ec0a3cc0 2024-04-24T08:58:51.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:51 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:58:51.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:51 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:58:51.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:51 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:51.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:51 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:58:51.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:51 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:58:51.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:51 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:51.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:51 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:58:51.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:51 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:58:51.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:51 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:58:52.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:52 smithi130 ceph-mon[31553]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:58:52.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:52 smithi130 ceph-mon[31553]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:52.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:52 smithi069 ceph-mon[28170]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:58:52.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:52 smithi069 ceph-mon[28170]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:52.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:52 smithi081 ceph-mon[30961]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:58:52.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:52 smithi081 ceph-mon[30961]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:54.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:54 smithi081 ceph-mon[30961]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:54.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:54 smithi069 ceph-mon[28170]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:54.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:54 smithi130 ceph-mon[31553]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:55.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:55 smithi069 ceph-mon[28170]: from='client.? 172.21.15.130:0/2115115513' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd082142-bbe7-40f2-85c3-94cf0cf96466"}]: dispatch 2024-04-24T08:58:55.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:55 smithi069 ceph-mon[28170]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd082142-bbe7-40f2-85c3-94cf0cf96466"}]: dispatch 2024-04-24T08:58:55.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:55 smithi069 ceph-mon[28170]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd082142-bbe7-40f2-85c3-94cf0cf96466"}]': finished 2024-04-24T08:58:55.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:55 smithi069 ceph-mon[28170]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T08:58:55.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:55 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:58:55.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:55 smithi130 ceph-mon[31553]: from='client.? 172.21.15.130:0/2115115513' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd082142-bbe7-40f2-85c3-94cf0cf96466"}]: dispatch 2024-04-24T08:58:55.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:55 smithi130 ceph-mon[31553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd082142-bbe7-40f2-85c3-94cf0cf96466"}]: dispatch 2024-04-24T08:58:55.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:55 smithi130 ceph-mon[31553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd082142-bbe7-40f2-85c3-94cf0cf96466"}]': finished 2024-04-24T08:58:55.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:55 smithi130 ceph-mon[31553]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T08:58:55.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:55 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:58:55.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:55 smithi081 ceph-mon[30961]: from='client.? 172.21.15.130:0/2115115513' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd082142-bbe7-40f2-85c3-94cf0cf96466"}]: dispatch 2024-04-24T08:58:55.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:55 smithi081 ceph-mon[30961]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd082142-bbe7-40f2-85c3-94cf0cf96466"}]: dispatch 2024-04-24T08:58:55.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:55 smithi081 ceph-mon[30961]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd082142-bbe7-40f2-85c3-94cf0cf96466"}]': finished 2024-04-24T08:58:55.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:55 smithi081 ceph-mon[30961]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T08:58:55.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:55 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:58:56.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:56 smithi130 ceph-mon[31553]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:56.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:56 smithi130 ceph-mon[31553]: from='client.? 172.21.15.130:0/283650944' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:58:56.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:56 smithi081 ceph-mon[30961]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:56.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:56 smithi081 ceph-mon[30961]: from='client.? 172.21.15.130:0/283650944' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:58:56.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:56 smithi069 ceph-mon[28170]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:56.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:56 smithi069 ceph-mon[28170]: from='client.? 172.21.15.130:0/283650944' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:58:58.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:58:58 smithi130 ceph-mon[31553]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:58.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:58:58 smithi081 ceph-mon[30961]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:58:58.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:58:58 smithi069 ceph-mon[28170]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:00.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:00 smithi130 ceph-mon[31553]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:00.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:00 smithi081 ceph-mon[30961]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:00.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:00 smithi069 ceph-mon[28170]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:02.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:02 smithi130 ceph-mon[31553]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:02.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:02 smithi081 ceph-mon[30961]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:02 smithi069 ceph-mon[28170]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:04.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:04 smithi069 ceph-mon[28170]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:04.619 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:04 smithi081 ceph-mon[30961]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:04.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:04 smithi130 ceph-mon[31553]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:05.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:05 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T08:59:05.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:05 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:05.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:05 smithi081 ceph-mon[30961]: Deploying daemon osd.5 on smithi130 2024-04-24T08:59:05.646 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:05 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T08:59:05.646 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:05 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:05.647 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:05 smithi130 ceph-mon[31553]: Deploying daemon osd.5 on smithi130 2024-04-24T08:59:05.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:05 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T08:59:05.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:05 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:05.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:05 smithi069 ceph-mon[28170]: Deploying daemon osd.5 on smithi130 2024-04-24T08:59:06.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:06 smithi081 ceph-mon[30961]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:06.677 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:06 smithi130 ceph-mon[31553]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:06.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:06 smithi069 ceph-mon[28170]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:08.590 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:08 smithi130 ceph-mon[31553]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:08.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:08 smithi081 ceph-mon[30961]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:08 smithi069 ceph-mon[28170]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:10.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:10 smithi130 ceph-mon[31553]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:10.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:10 smithi081 ceph-mon[30961]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:10.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:10 smithi069 ceph-mon[28170]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:12.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:12 smithi130 ceph-mon[31553]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:12.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:12 smithi081 ceph-mon[30961]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:12.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:12 smithi069 ceph-mon[28170]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:14.258 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:14 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:14.258 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:14 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:14.258 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:14 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:59:14.258 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:14 smithi130 ceph-mon[31553]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:14.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:14 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:14.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:14 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:14.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:14 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:59:14.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:14 smithi069 ceph-mon[28170]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:14.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:14 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:14.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:14 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:14.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:14 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:59:14.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:14 smithi081 ceph-mon[30961]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:16.377 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:16 smithi130 ceph-mon[31553]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:16.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:16 smithi069 ceph-mon[28170]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:16.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:16 smithi081 ceph-mon[30961]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:17.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:17.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:17.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:17.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:59:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:17 smithi130 ceph-mon[31553]: from='osd.5 [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:59:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:17 smithi130 ceph-mon[31553]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:59:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:17 smithi130 ceph-mon[31553]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:18.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:18.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:18.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:18.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:59:18.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:18.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:17 smithi069 ceph-mon[28170]: from='osd.5 [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:59:18.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:17 smithi069 ceph-mon[28170]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:59:18.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:17 smithi069 ceph-mon[28170]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:18.132 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:59:18.130+0000 7f41ff400700 1 -- 172.21.15.130:0/2306880435 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4208062f30 con 0x7f41ec0a3cc0 2024-04-24T08:59:18.132 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 5 on host 'smithi130' 2024-04-24T08:59:18.133 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:59:18.134+0000 7f420ce00700 1 -- 172.21.15.130:0/2306880435 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f41ec0a3cc0 msgr2=0x7f41ec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:18.133 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:59:18.134+0000 7f420ce00700 1 --2- 172.21.15.130:0/2306880435 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f41ec0a3cc0 0x7f41ec0a6180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f41f8003780 tx=0x7f41f8003850 comp rx=0 tx=0).stop 2024-04-24T08:59:18.133 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:59:18.134+0000 7f420ce00700 1 -- 172.21.15.130:0/2306880435 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4208102760 msgr2=0x7f42081a23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:18.134 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:59:18.134+0000 7f420ce00700 1 --2- 172.21.15.130:0/2306880435 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4208102760 0x7f42081a23e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f420000c9a0 tx=0x7f420000ccb0 comp rx=0 tx=0).stop 2024-04-24T08:59:18.134 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:59:18.134+0000 7f420ce00700 1 -- 172.21.15.130:0/2306880435 shutdown_connections 2024-04-24T08:59:18.134 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:59:18.134+0000 7f420ce00700 1 --2- 172.21.15.130:0/2306880435 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f41ec0a3cc0 0x7f41ec0a6180 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:18.134 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:59:18.134+0000 7f420ce00700 1 --2- 172.21.15.130:0/2306880435 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4208102760 0x7f42081a23e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:18.134 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:59:18.134+0000 7f420ce00700 1 --2- 172.21.15.130:0/2306880435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4208101440 0x7f420819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:18.134 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:59:18.134+0000 7f420ce00700 1 --2- 172.21.15.130:0/2306880435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4208101da0 0x7f420819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:18.134 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:59:18.134+0000 7f420ce00700 1 -- 172.21.15.130:0/2306880435 >> 172.21.15.130:0/2306880435 conn(0x7f42080fcfe0 msgr2=0x7f42080feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:18.135 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:59:18.134+0000 7f420ce00700 1 -- 172.21.15.130:0/2306880435 shutdown_connections 2024-04-24T08:59:18.135 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T08:59:18.135+0000 7f420ce00700 1 -- 172.21.15.130:0/2306880435 wait complete. 2024-04-24T08:59:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:59:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:17 smithi081 ceph-mon[30961]: from='osd.5 [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:59:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:17 smithi081 ceph-mon[30961]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:59:18.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:17 smithi081 ceph-mon[30961]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:18.985 DEBUG:teuthology.orchestra.run.smithi130:osd.5> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.5.service 2024-04-24T08:59:18.990 INFO:tasks.cephadm:Deploying osd.6 on smithi069 with /dev/vg_nvme/lv_2... 2024-04-24T08:59:18.991 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T08:59:19.178 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:18 smithi130 ceph-mon[31553]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T08:59:19.179 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:18 smithi130 ceph-mon[31553]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T08:59:19.179 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:18 smithi130 ceph-mon[31553]: from='osd.5 [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T08:59:19.179 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:18 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:59:19.179 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:18 smithi130 ceph-mon[31553]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T08:59:19.179 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:18 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:19.179 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:18 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:19.179 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:18 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:59:19.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:18 smithi081 ceph-mon[30961]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T08:59:19.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:18 smithi081 ceph-mon[30961]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T08:59:19.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:18 smithi081 ceph-mon[30961]: from='osd.5 [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T08:59:19.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:18 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:59:19.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:18 smithi081 ceph-mon[30961]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T08:59:19.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:18 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:19.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:18 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:19.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:18 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:59:19.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:18 smithi069 ceph-mon[28170]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T08:59:19.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:18 smithi069 ceph-mon[28170]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T08:59:19.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:18 smithi069 ceph-mon[28170]: from='osd.5 [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T08:59:19.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:18 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:59:19.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:18 smithi069 ceph-mon[28170]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T08:59:19.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:18 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:19.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:18 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:19.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:18 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:59:19.434 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:59:19.890 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 08:59:19 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T08:59:19.672+0000 7fd707800700 -1 osd.5 0 waiting for initial osdmap 2024-04-24T08:59:19.891 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 08:59:19 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T08:59:19.702+0000 7fd6fcc00700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:59:20.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:19 smithi130 ceph-mon[31553]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-24T08:59:20.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:19 smithi130 ceph-mon[31553]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T08:59:20.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:19 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:59:20.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:19 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:59:20.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:19 smithi130 ceph-mon[31553]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:20.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:19 smithi130 ceph-mon[31553]: from='osd.5 ' entity='osd.5' 2024-04-24T08:59:20.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:19 smithi081 ceph-mon[30961]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-24T08:59:20.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:19 smithi081 ceph-mon[30961]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T08:59:20.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:19 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:59:20.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:19 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:59:20.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:19 smithi081 ceph-mon[30961]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:20.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:19 smithi081 ceph-mon[30961]: from='osd.5 ' entity='osd.5' 2024-04-24T08:59:20.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:19 smithi069 ceph-mon[28170]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-24T08:59:20.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:19 smithi069 ceph-mon[28170]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T08:59:20.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:19 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:59:20.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:19 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:59:20.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:19 smithi069 ceph-mon[28170]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:59:20.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:19 smithi069 ceph-mon[28170]: from='osd.5 ' entity='osd.5' 2024-04-24T08:59:21.083 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:20 smithi130 ceph-mon[31553]: purged_snaps scrub starts 2024-04-24T08:59:21.083 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:20 smithi130 ceph-mon[31553]: purged_snaps scrub ok 2024-04-24T08:59:21.083 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:20 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:59:21.083 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:20 smithi130 ceph-mon[31553]: osd.5 [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184] boot 2024-04-24T08:59:21.083 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:20 smithi130 ceph-mon[31553]: osdmap e38: 6 total, 6 up, 6 in 2024-04-24T08:59:21.083 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:20 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:59:21.268 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:20 smithi069 ceph-mon[28170]: purged_snaps scrub starts 2024-04-24T08:59:21.269 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:20 smithi069 ceph-mon[28170]: purged_snaps scrub ok 2024-04-24T08:59:21.269 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:20 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:59:21.269 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:20 smithi069 ceph-mon[28170]: osd.5 [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184] boot 2024-04-24T08:59:21.269 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:20 smithi069 ceph-mon[28170]: osdmap e38: 6 total, 6 up, 6 in 2024-04-24T08:59:21.269 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:20 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:59:21.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:20 smithi081 ceph-mon[30961]: purged_snaps scrub starts 2024-04-24T08:59:21.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:20 smithi081 ceph-mon[30961]: purged_snaps scrub ok 2024-04-24T08:59:21.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:20 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:59:21.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:20 smithi081 ceph-mon[30961]: osd.5 [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184] boot 2024-04-24T08:59:21.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:20 smithi081 ceph-mon[30961]: osdmap e38: 6 total, 6 up, 6 in 2024-04-24T08:59:21.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:20 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:59:22.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:21 smithi069 ceph-mon[28170]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T08:59:22.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:21 smithi069 ceph-mon[28170]: pgmap v178: 1 pgs: 1 peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:22.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:21 smithi130 ceph-mon[31553]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T08:59:22.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:21 smithi130 ceph-mon[31553]: pgmap v178: 1 pgs: 1 peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:22.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:21 smithi081 ceph-mon[30961]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T08:59:22.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:21 smithi081 ceph-mon[30961]: pgmap v178: 1 pgs: 1 peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:22.568 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T08:59:22.568 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T08:59:22.568 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-04-24T08:59:22.569 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-04-24T08:59:22.569 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0332237 s, 316 MB/s 2024-04-24T08:59:22.569 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-04-24T08:59:23.174 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:22 smithi069 ceph-mon[28170]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T08:59:23.174 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:22 smithi069 ceph-mon[28170]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T08:59:23.184 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:22 smithi130 ceph-mon[31553]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T08:59:23.184 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:22 smithi130 ceph-mon[31553]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T08:59:23.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:22 smithi081 ceph-mon[30961]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T08:59:23.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:22 smithi081 ceph-mon[30961]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T08:59:23.532 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_2 2024-04-24T08:59:23.943 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:59:24.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:23 smithi130 ceph-mon[31553]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:24.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:23 smithi130 ceph-mon[31553]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T08:59:24.276 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:23 smithi069 ceph-mon[28170]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:24.276 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:23 smithi069 ceph-mon[28170]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T08:59:24.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:23 smithi081 ceph-mon[30961]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:24.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:23 smithi081 ceph-mon[30961]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T08:59:25.339 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:25 smithi069 ceph-mon[28170]: Detected new or changed devices on smithi130 2024-04-24T08:59:25.339 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:25 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:25.339 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:25 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:25.339 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:25 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:59:25.339 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:25 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:59:25.339 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:25 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi130 to 8544M 2024-04-24T08:59:25.339 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:25 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:25.340 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:25 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:25.340 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:25 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:59:25.340 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:25 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:25.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:25 smithi130 ceph-mon[31553]: Detected new or changed devices on smithi130 2024-04-24T08:59:25.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:25 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:25.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:25 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:25.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:25 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:59:25.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:25 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:59:25.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:25 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi130 to 8544M 2024-04-24T08:59:25.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:25 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:25.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:25 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:25.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:25 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:59:25.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:25 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:25 smithi081 ceph-mon[30961]: Detected new or changed devices on smithi130 2024-04-24T08:59:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:25 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:25 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:25 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:59:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:25 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:59:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:25 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi130 to 8544M 2024-04-24T08:59:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:25 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:25 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:25.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:25 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:59:25.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:25 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:26.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.206+0000 7f8ff7a00700 1 -- 172.21.15.69:0/2486511613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ff0076ac0 msgr2=0x7f8ff0074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:26.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.206+0000 7f8ff7a00700 1 --2- 172.21.15.69:0/2486511613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ff0076ac0 0x7f8ff0074f20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400b0a0 tx=0x7f8fe400b3b0 comp rx=0 tx=0).stop 2024-04-24T08:59:26.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.206+0000 7f8ff7a00700 1 -- 172.21.15.69:0/2486511613 shutdown_connections 2024-04-24T08:59:26.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.206+0000 7f8ff7a00700 1 --2- 172.21.15.69:0/2486511613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ff010aa40 0x7f8ff010ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:26.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.206+0000 7f8ff7a00700 1 --2- 172.21.15.69:0/2486511613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ff0076ac0 0x7f8ff0074f20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:26.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.206+0000 7f8ff7a00700 1 --2- 172.21.15.69:0/2486511613 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ff00754f0 0x7f8ff0075970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:26.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.206+0000 7f8ff7a00700 1 -- 172.21.15.69:0/2486511613 >> 172.21.15.69:0/2486511613 conn(0x7f8ff00fe0c0 msgr2=0x7f8ff0100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:26.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.206+0000 7f8ff7a00700 1 -- 172.21.15.69:0/2486511613 shutdown_connections 2024-04-24T08:59:26.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.206+0000 7f8ff7a00700 1 -- 172.21.15.69:0/2486511613 wait complete. 2024-04-24T08:59:26.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.207+0000 7f8ff7a00700 1 Processor -- start 2024-04-24T08:59:26.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.208+0000 7f8ff7a00700 1 -- start start 2024-04-24T08:59:26.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.208+0000 7f8ff7a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ff00754f0 0x7f8ff019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:26.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.208+0000 7f8ff7a00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ff0076ac0 0x7f8ff019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:26.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.208+0000 7f8ff7a00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ff010aa40 0x7f8ff01a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:26.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.208+0000 7f8ff7a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0110800 con 0x7f8ff00754f0 2024-04-24T08:59:26.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.208+0000 7f8ff7a00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0110680 con 0x7f8ff010aa40 2024-04-24T08:59:26.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.208+0000 7f8ff7a00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0110980 con 0x7f8ff0076ac0 2024-04-24T08:59:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.208+0000 7f8ff5000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ff00754f0 0x7f8ff019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.208+0000 7f8ff5000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ff00754f0 0x7f8ff019ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46294/0 (socket says 172.21.15.69:46294) 2024-04-24T08:59:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.208+0000 7f8ff5000700 1 -- 172.21.15.69:0/2085949926 learned_addr learned my addr 172.21.15.69:0/2085949926 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T08:59:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.208+0000 7f8fefe00700 1 --2- 172.21.15.69:0/2085949926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ff0076ac0 0x7f8ff019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.208+0000 7f8ff5a00700 1 --2- 172.21.15.69:0/2085949926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ff010aa40 0x7f8ff01a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.209+0000 7f8ff5000700 1 -- 172.21.15.69:0/2085949926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ff0076ac0 msgr2=0x7f8ff019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.209+0000 7f8ff5000700 1 --2- 172.21.15.69:0/2085949926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ff0076ac0 0x7f8ff019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.209+0000 7f8ff5000700 1 -- 172.21.15.69:0/2085949926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ff010aa40 msgr2=0x7f8ff01a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.209+0000 7f8ff5000700 1 --2- 172.21.15.69:0/2085949926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ff010aa40 0x7f8ff01a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.209+0000 7f8ff5000700 1 -- 172.21.15.69:0/2085949926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff01a2a80 con 0x7f8ff00754f0 2024-04-24T08:59:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.209+0000 7f8ff5000700 1 --2- 172.21.15.69:0/2085949926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ff00754f0 0x7f8ff019ae20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4006070 tx=0x7f8fe4007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.209+0000 7f8fed600700 1 -- 172.21.15.69:0/2085949926 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4021020 con 0x7f8ff00754f0 2024-04-24T08:59:26.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.209+0000 7f8fed600700 1 -- 172.21.15.69:0/2085949926 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fe4007d10 con 0x7f8ff00754f0 2024-04-24T08:59:26.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.209+0000 7f8ff7a00700 1 -- 172.21.15.69:0/2085949926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff01a2d10 con 0x7f8ff00754f0 2024-04-24T08:59:26.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.209+0000 7f8fed600700 1 -- 172.21.15.69:0/2085949926 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe401d970 con 0x7f8ff00754f0 2024-04-24T08:59:26.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.209+0000 7f8ff7a00700 1 -- 172.21.15.69:0/2085949926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff01a31e0 con 0x7f8ff00754f0 2024-04-24T08:59:26.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.211+0000 7f8ff7a00700 1 -- 172.21.15.69:0/2085949926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff00697e0 con 0x7f8ff00754f0 2024-04-24T08:59:26.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.213+0000 7f8fed600700 1 -- 172.21.15.69:0/2085949926 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8fe4024020 con 0x7f8ff00754f0 2024-04-24T08:59:26.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.214+0000 7f8fed600700 1 --2- 172.21.15.69:0/2085949926 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8fd80a3cc0 0x7f8fd80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:26.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.214+0000 7f8fed600700 1 -- 172.21.15.69:0/2085949926 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f8fe400f020 con 0x7f8ff00754f0 2024-04-24T08:59:26.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.214+0000 7f8fefe00700 1 --2- 172.21.15.69:0/2085949926 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8fd80a3cc0 0x7f8fd80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:26.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.215+0000 7f8fefe00700 1 --2- 172.21.15.69:0/2085949926 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8fd80a3cc0 0x7f8fd80a6180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc005ea0 tx=0x7f8fdc005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:26.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.217+0000 7f8fed600700 1 -- 172.21.15.69:0/2085949926 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8fe409ec40 con 0x7f8ff00754f0 2024-04-24T08:59:26.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:26.452+0000 7f8ff7a00700 1 -- 172.21.15.69:0/2085949926 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8ff0062f30 con 0x7f8fd80a3cc0 2024-04-24T08:59:26.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:26 smithi069 ceph-mon[28170]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-04-24T08:59:26.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:26 smithi130 ceph-mon[31553]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-04-24T08:59:26.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:26 smithi081 ceph-mon[30961]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-04-24T08:59:27.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:27 smithi130 ceph-mon[31553]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:59:27.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:27 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:59:27.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:27 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:59:27.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:27 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:27.548 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:27 smithi069 ceph-mon[28170]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:59:27.548 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:27 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:59:27.548 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:27 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:59:27.548 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:27 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:27.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:27 smithi081 ceph-mon[30961]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:59:27.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:27 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:59:27.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:27 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:59:27.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:27 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:28.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:28 smithi130 ceph-mon[31553]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T08:59:28.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:28 smithi081 ceph-mon[30961]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T08:59:28.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:28 smithi069 ceph-mon[28170]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T08:59:30.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:30 smithi069 ceph-mon[28170]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-24T08:59:30.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:30 smithi130 ceph-mon[31553]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-24T08:59:30.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:30 smithi081 ceph-mon[30961]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-24T08:59:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:31 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2521969700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58e89c83-2d93-42a3-9fa4-e6b08810c776"}]: dispatch 2024-04-24T08:59:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:31 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2521969700' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58e89c83-2d93-42a3-9fa4-e6b08810c776"}]': finished 2024-04-24T08:59:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:31 smithi069 ceph-mon[28170]: osdmap e43: 7 total, 6 up, 7 in 2024-04-24T08:59:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:59:31.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:31 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2521969700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58e89c83-2d93-42a3-9fa4-e6b08810c776"}]: dispatch 2024-04-24T08:59:31.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:31 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2521969700' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58e89c83-2d93-42a3-9fa4-e6b08810c776"}]': finished 2024-04-24T08:59:31.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:31 smithi130 ceph-mon[31553]: osdmap e43: 7 total, 6 up, 7 in 2024-04-24T08:59:31.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:59:31.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:31 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2521969700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58e89c83-2d93-42a3-9fa4-e6b08810c776"}]: dispatch 2024-04-24T08:59:31.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:31 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2521969700' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58e89c83-2d93-42a3-9fa4-e6b08810c776"}]': finished 2024-04-24T08:59:31.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:31 smithi081 ceph-mon[30961]: osdmap e43: 7 total, 6 up, 7 in 2024-04-24T08:59:31.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:31 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:59:32.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:32 smithi069 ceph-mon[28170]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T08:59:32.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:32 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/236366737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:59:32.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:32 smithi130 ceph-mon[31553]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T08:59:32.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:32 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/236366737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:59:32.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:32 smithi081 ceph-mon[30961]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T08:59:32.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:32 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/236366737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:59:34.619 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:34 smithi081 ceph-mon[30961]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-24T08:59:34.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:34 smithi130 ceph-mon[31553]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-24T08:59:34.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:34 smithi069 ceph-mon[28170]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-24T08:59:36.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:36 smithi130 ceph-mon[31553]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:36.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:36 smithi081 ceph-mon[30961]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:36.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:36 smithi069 ceph-mon[28170]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:38.644 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:38 smithi069 ceph-mon[28170]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:38.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:38 smithi130 ceph-mon[31553]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:38.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:38 smithi081 ceph-mon[30961]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:40.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:40 smithi069 ceph-mon[28170]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:40.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T08:59:40.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:40.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:40 smithi130 ceph-mon[31553]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:40.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T08:59:40.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:40 smithi081 ceph-mon[30961]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T08:59:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:41.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:41 smithi069 ceph-mon[28170]: Deploying daemon osd.6 on smithi069 2024-04-24T08:59:41.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:41 smithi130 ceph-mon[31553]: Deploying daemon osd.6 on smithi069 2024-04-24T08:59:41.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:41 smithi081 ceph-mon[30961]: Deploying daemon osd.6 on smithi069 2024-04-24T08:59:42.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:42 smithi130 ceph-mon[31553]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:42 smithi081 ceph-mon[30961]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:42.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:42 smithi069 ceph-mon[28170]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:44.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:44 smithi130 ceph-mon[31553]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:44.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:44 smithi081 ceph-mon[30961]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:44.830 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:44 smithi069 ceph-mon[28170]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:46.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:46 smithi069 ceph-mon[28170]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:46.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:46 smithi130 ceph-mon[31553]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:46.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:46 smithi081 ceph-mon[30961]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:48.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:48 smithi069 ceph-mon[28170]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:48.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:48 smithi130 ceph-mon[31553]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:48.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:48 smithi081 ceph-mon[30961]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:50.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:50.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:50.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:59:50.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:49 smithi069 ceph-mon[28170]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:50.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:50.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:50.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:59:50.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:49 smithi130 ceph-mon[31553]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:50.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:50.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:50.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:59:50.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:49 smithi081 ceph-mon[30961]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:52.523 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:52 smithi069 ceph-mon[28170]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:52.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:52 smithi130 ceph-mon[31553]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:52.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:52 smithi081 ceph-mon[30961]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:53.793 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 6 on host 'smithi069' 2024-04-24T08:59:53.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:53.793+0000 7f8fed600700 1 -- 172.21.15.69:0/2085949926 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8ff0062f30 con 0x7f8fd80a3cc0 2024-04-24T08:59:53.794 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:53.794 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:53.794 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:53.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:59:53.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:53.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:53 smithi069 ceph-mon[28170]: from='osd.6 [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T08:59:53.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:53 smithi069 ceph-mon[28170]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:53.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:53.796+0000 7f8ff7a00700 1 -- 172.21.15.69:0/2085949926 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8fd80a3cc0 msgr2=0x7f8fd80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:53.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:53.796+0000 7f8ff7a00700 1 --2- 172.21.15.69:0/2085949926 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8fd80a3cc0 0x7f8fd80a6180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc005ea0 tx=0x7f8fdc005e30 comp rx=0 tx=0).stop 2024-04-24T08:59:53.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:53.796+0000 7f8ff7a00700 1 -- 172.21.15.69:0/2085949926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ff00754f0 msgr2=0x7f8ff019ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:53.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:53.796+0000 7f8ff7a00700 1 --2- 172.21.15.69:0/2085949926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ff00754f0 0x7f8ff019ae20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4006070 tx=0x7f8fe4007770 comp rx=0 tx=0).stop 2024-04-24T08:59:53.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:53.796+0000 7f8ff7a00700 1 -- 172.21.15.69:0/2085949926 shutdown_connections 2024-04-24T08:59:53.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:53.796+0000 7f8ff7a00700 1 --2- 172.21.15.69:0/2085949926 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8fd80a3cc0 0x7f8fd80a6180 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:53.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:53.796+0000 7f8ff7a00700 1 --2- 172.21.15.69:0/2085949926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ff010aa40 0x7f8ff01a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:53.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:53.796+0000 7f8ff7a00700 1 --2- 172.21.15.69:0/2085949926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ff00754f0 0x7f8ff019ae20 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4006070 tx=0x7f8fe4007770 comp rx=0 tx=0).stop 2024-04-24T08:59:53.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:53.796+0000 7f8ff7a00700 1 --2- 172.21.15.69:0/2085949926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ff0076ac0 0x7f8ff019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:53.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:53.796+0000 7f8ff7a00700 1 -- 172.21.15.69:0/2085949926 >> 172.21.15.69:0/2085949926 conn(0x7f8ff00fe0c0 msgr2=0x7f8ff0106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:53.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:53.797+0000 7f8ff7a00700 1 -- 172.21.15.69:0/2085949926 shutdown_connections 2024-04-24T08:59:53.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T08:59:53.797+0000 7f8ff7a00700 1 -- 172.21.15.69:0/2085949926 wait complete. 2024-04-24T08:59:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:59:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:53 smithi081 ceph-mon[30961]: from='osd.6 [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T08:59:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:53 smithi081 ceph-mon[30961]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:53.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:53.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:53.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:53.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:59:53.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:53.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:53 smithi130 ceph-mon[31553]: from='osd.6 [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T08:59:53.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:53 smithi130 ceph-mon[31553]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:54.575 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:54 smithi069 ceph-mon[28170]: from='osd.6 [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T08:59:54.575 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:54 smithi069 ceph-mon[28170]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T08:59:54.575 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:54 smithi069 ceph-mon[28170]: from='osd.6 [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-24T08:59:54.575 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:54 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:59:54.575 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:54 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:54.575 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:54 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:59:54.575 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:54 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:54.771 DEBUG:teuthology.orchestra.run.smithi069:osd.6> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.6.service 2024-04-24T08:59:54.778 INFO:tasks.cephadm:Deploying osd.7 on smithi081 with /dev/vg_nvme/lv_2... 2024-04-24T08:59:54.778 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T08:59:54.858 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:54 smithi081 ceph-mon[30961]: from='osd.6 [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T08:59:54.858 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:54 smithi081 ceph-mon[30961]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T08:59:54.858 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:54 smithi081 ceph-mon[30961]: from='osd.6 [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-24T08:59:54.859 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:54 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:59:54.859 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:54 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:54.859 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:54 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:59:54.859 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:54 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:54 smithi130 ceph-mon[31553]: from='osd.6 [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T08:59:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:54 smithi130 ceph-mon[31553]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T08:59:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:54 smithi130 ceph-mon[31553]: from='osd.6 [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-24T08:59:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:54 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:59:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:54 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:54 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:59:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:54 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T08:59:55.222 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T08:59:55.714 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:55 smithi081 ceph-mon[30961]: from='osd.6 [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-24T08:59:55.714 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:55 smithi081 ceph-mon[30961]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T08:59:55.714 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:55 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:59:55.714 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:55 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:59:55.714 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:55 smithi081 ceph-mon[30961]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:55.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:55 smithi069 ceph-mon[28170]: from='osd.6 [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-24T08:59:55.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:55 smithi069 ceph-mon[28170]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T08:59:55.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:55 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:59:55.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:55 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:59:55.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:55 smithi069 ceph-mon[28170]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:55.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:55 smithi130 ceph-mon[31553]: from='osd.6 [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-24T08:59:55.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:55 smithi130 ceph-mon[31553]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T08:59:55.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:55 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:59:55.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:55 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:59:55.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:55 smithi130 ceph-mon[31553]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:59:56.788 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:56 smithi081 ceph-mon[30961]: purged_snaps scrub starts 2024-04-24T08:59:56.788 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:56 smithi081 ceph-mon[30961]: purged_snaps scrub ok 2024-04-24T08:59:56.788 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:56 smithi081 ceph-mon[30961]: osd.6 [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387] boot 2024-04-24T08:59:56.789 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:56 smithi081 ceph-mon[30961]: osdmap e46: 7 total, 7 up, 7 in 2024-04-24T08:59:56.789 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:56 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:59:56.869 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:56 smithi069 ceph-mon[28170]: purged_snaps scrub starts 2024-04-24T08:59:56.869 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:56 smithi069 ceph-mon[28170]: purged_snaps scrub ok 2024-04-24T08:59:56.869 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:56 smithi069 ceph-mon[28170]: osd.6 [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387] boot 2024-04-24T08:59:56.869 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:56 smithi069 ceph-mon[28170]: osdmap e46: 7 total, 7 up, 7 in 2024-04-24T08:59:56.869 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:56 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:59:56.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:56 smithi130 ceph-mon[31553]: purged_snaps scrub starts 2024-04-24T08:59:56.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:56 smithi130 ceph-mon[31553]: purged_snaps scrub ok 2024-04-24T08:59:56.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:56 smithi130 ceph-mon[31553]: osd.6 [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387] boot 2024-04-24T08:59:56.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:56 smithi130 ceph-mon[31553]: osdmap e46: 7 total, 7 up, 7 in 2024-04-24T08:59:56.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:56 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:59:57.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:57 smithi130 ceph-mon[31553]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T08:59:57.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 08:59:57 smithi130 ceph-mon[31553]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:59:58.000 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T08:59:58.000 INFO:teuthology.orchestra.run.smithi081.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T08:59:58.000 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10+0 records in 2024-04-24T08:59:58.000 INFO:teuthology.orchestra.run.smithi081.stderr:10+0 records out 2024-04-24T08:59:58.000 INFO:teuthology.orchestra.run.smithi081.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.036241 s, 289 MB/s 2024-04-24T08:59:58.000 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping successful for: 2024-04-24T08:59:58.001 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:57 smithi081 ceph-mon[30961]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T08:59:58.001 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 08:59:57 smithi081 ceph-mon[30961]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:59:58.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:57 smithi069 ceph-mon[28170]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T08:59:58.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 08:59:57 smithi069 ceph-mon[28170]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:59:58.925 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi081:vg_nvme/lv_2 2024-04-24T08:59:59.360 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:00:00.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:00 smithi081 ceph-mon[30961]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:00.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:00 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:00.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:00 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:00.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:00 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:00.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:00 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:00.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:00 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:00.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:00 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:00.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:00 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:00.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:00 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:00:00.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:00 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:00.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:00 smithi081 ceph-mon[30961]: overall HEALTH_OK 2024-04-24T09:00:00.658 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:00 smithi130 ceph-mon[31553]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:00.658 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:00 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:00.658 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:00 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:00.658 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:00 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:00.658 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:00 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:00.658 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:00 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:00.658 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:00 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:00.658 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:00 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:00.658 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:00 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:00:00.658 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:00 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:00.658 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:00 smithi130 ceph-mon[31553]: overall HEALTH_OK 2024-04-24T09:00:00.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:00 smithi069 ceph-mon[28170]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:00.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:00 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:00.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:00 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:00.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:00 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:00.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:00 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:00.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:00 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:00.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:00 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:00.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:00 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:00.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:00 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:00:00.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:00 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:00.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:00 smithi069 ceph-mon[28170]: overall HEALTH_OK 2024-04-24T09:00:01.511 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:01 smithi081 ceph-mon[30961]: Detected new or changed devices on smithi069 2024-04-24T09:00:01.511 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:01 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi069 to 5696M 2024-04-24T09:00:01.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.682+0000 7f13cda00700 1 -- 172.21.15.81:0/3135836387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13c810a280 msgr2=0x7f13c810a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:01.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.682+0000 7f13cda00700 1 --2- 172.21.15.81:0/3135836387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13c810a280 0x7f13c810a6e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f13b40099c0 tx=0x7f13b4009cd0 comp rx=0 tx=0).stop 2024-04-24T09:00:01.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.682+0000 7f13cda00700 1 -- 172.21.15.81:0/3135836387 shutdown_connections 2024-04-24T09:00:01.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.682+0000 7f13cda00700 1 --2- 172.21.15.81:0/3135836387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13c810ac20 0x7f13c81114d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:01.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.682+0000 7f13cda00700 1 --2- 172.21.15.81:0/3135836387 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13c8109080 0x7f13c81094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:01.682 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.682+0000 7f13cda00700 1 --2- 172.21.15.81:0/3135836387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13c810a280 0x7f13c810a6e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:01.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.682+0000 7f13cda00700 1 -- 172.21.15.81:0/3135836387 >> 172.21.15.81:0/3135836387 conn(0x7f13c8078050 msgr2=0x7f13c807a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:01.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.682+0000 7f13cda00700 1 -- 172.21.15.81:0/3135836387 shutdown_connections 2024-04-24T09:00:01.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.682+0000 7f13cda00700 1 -- 172.21.15.81:0/3135836387 wait complete. 2024-04-24T09:00:01.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.683+0000 7f13cda00700 1 Processor -- start 2024-04-24T09:00:01.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.684+0000 7f13cda00700 1 -- start start 2024-04-24T09:00:01.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.684+0000 7f13cda00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13c8109080 0x7f13c819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:01.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.684+0000 7f13cda00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13c810a280 0x7f13c819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:01.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.684+0000 7f13cda00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13c810ac20 0x7f13c81a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:01.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.684+0000 7f13cda00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8114b60 con 0x7f13c8109080 2024-04-24T09:00:01.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.684+0000 7f13cda00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f13c81149e0 con 0x7f13c810ac20 2024-04-24T09:00:01.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.684+0000 7f13cda00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8114ce0 con 0x7f13c810a280 2024-04-24T09:00:01.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.684+0000 7f13cd000700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13c810ac20 0x7f13c81a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:01.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.684+0000 7f13cd000700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13c810ac20 0x7f13c81a66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45080/0 (socket says 172.21.15.81:45080) 2024-04-24T09:00:01.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.684+0000 7f13cd000700 1 -- 172.21.15.81:0/608227375 learned_addr learned my addr 172.21.15.81:0/608227375 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-04-24T09:00:01.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.685+0000 7f13c7400700 1 --2- 172.21.15.81:0/608227375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13c810a280 0x7f13c819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:01.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.685+0000 7f13c7e00700 1 --2- 172.21.15.81:0/608227375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13c8109080 0x7f13c819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:01.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.685+0000 7f13cd000700 1 -- 172.21.15.81:0/608227375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13c810a280 msgr2=0x7f13c819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:01.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.685+0000 7f13cd000700 1 --2- 172.21.15.81:0/608227375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13c810a280 0x7f13c819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:01.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.685+0000 7f13cd000700 1 -- 172.21.15.81:0/608227375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13c8109080 msgr2=0x7f13c819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:01.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.685+0000 7f13cd000700 1 --2- 172.21.15.81:0/608227375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13c8109080 0x7f13c819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:01.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.685+0000 7f13cd000700 1 -- 172.21.15.81:0/608227375 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c81a6d70 con 0x7f13c810ac20 2024-04-24T09:00:01.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.685+0000 7f13cd000700 1 --2- 172.21.15.81:0/608227375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13c810ac20 0x7f13c81a66d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f13c0007d70 tx=0x7f13c000a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:01.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.685+0000 7f13c4c00700 1 -- 172.21.15.81:0/608227375 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0016020 con 0x7f13c810ac20 2024-04-24T09:00:01.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.685+0000 7f13c4c00700 1 -- 172.21.15.81:0/608227375 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13c000ad40 con 0x7f13c810ac20 2024-04-24T09:00:01.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.685+0000 7f13cda00700 1 -- 172.21.15.81:0/608227375 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c81a7060 con 0x7f13c810ac20 2024-04-24T09:00:01.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.685+0000 7f13c4c00700 1 -- 172.21.15.81:0/608227375 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c00115e0 con 0x7f13c810ac20 2024-04-24T09:00:01.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.686+0000 7f13cda00700 1 -- 172.21.15.81:0/608227375 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c81a7550 con 0x7f13c810ac20 2024-04-24T09:00:01.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.687+0000 7f13cda00700 1 -- 172.21.15.81:0/608227375 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c80697e0 con 0x7f13c810ac20 2024-04-24T09:00:01.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.689+0000 7f13c4c00700 1 -- 172.21.15.81:0/608227375 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f13c0019020 con 0x7f13c810ac20 2024-04-24T09:00:01.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.691+0000 7f13c4c00700 1 --2- 172.21.15.81:0/608227375 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13ac0a3c40 0x7f13ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:01.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.691+0000 7f13c4c00700 1 -- 172.21.15.81:0/608227375 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f13c00cfbd0 con 0x7f13c810ac20 2024-04-24T09:00:01.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.692+0000 7f13c7e00700 1 --2- 172.21.15.81:0/608227375 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13ac0a3c40 0x7f13ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:01.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.693+0000 7f13c7e00700 1 --2- 172.21.15.81:0/608227375 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13ac0a3c40 0x7f13ac0a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f13b8005950 tx=0x7f13b80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:01.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.695+0000 7f13c4c00700 1 -- 172.21.15.81:0/608227375 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f13c0020020 con 0x7f13c810ac20 2024-04-24T09:00:01.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:01 smithi130 ceph-mon[31553]: Detected new or changed devices on smithi069 2024-04-24T09:00:01.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:01 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi069 to 5696M 2024-04-24T09:00:01.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:01 smithi069 ceph-mon[28170]: Detected new or changed devices on smithi069 2024-04-24T09:00:01.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:01 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi069 to 5696M 2024-04-24T09:00:01.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:01.931+0000 7f13cda00700 1 -- 172.21.15.81:0/608227375 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f13c8062f30 con 0x7f13ac0a3c40 2024-04-24T09:00:02.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:02 smithi130 ceph-mon[31553]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:02.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:02 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:00:02.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:02 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:00:02.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:02 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:02.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:02 smithi081 ceph-mon[30961]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:02.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:02 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:00:02.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:02 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:00:02.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:02 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:02 smithi069 ceph-mon[28170]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:00:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:00:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:03.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:03 smithi081 ceph-mon[30961]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:00:03.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:03 smithi130 ceph-mon[31553]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:00:03.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:03 smithi069 ceph-mon[28170]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:00:04.542 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:04 smithi081 ceph-mon[30961]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:04.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:04 smithi069 ceph-mon[28170]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:04.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:04 smithi130 ceph-mon[31553]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:06.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:06 smithi130 ceph-mon[31553]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:06.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:06 smithi130 ceph-mon[31553]: from='client.? 172.21.15.81:0/685373659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1daca7a-630f-4a56-bd3d-713fc3f88ede"}]: dispatch 2024-04-24T09:00:06.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:06 smithi130 ceph-mon[31553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1daca7a-630f-4a56-bd3d-713fc3f88ede"}]: dispatch 2024-04-24T09:00:06.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:06 smithi130 ceph-mon[31553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1daca7a-630f-4a56-bd3d-713fc3f88ede"}]': finished 2024-04-24T09:00:06.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:06 smithi130 ceph-mon[31553]: osdmap e48: 8 total, 7 up, 8 in 2024-04-24T09:00:06.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:00:06.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:06 smithi081 ceph-mon[30961]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:06.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:06 smithi081 ceph-mon[30961]: from='client.? 172.21.15.81:0/685373659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1daca7a-630f-4a56-bd3d-713fc3f88ede"}]: dispatch 2024-04-24T09:00:06.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:06 smithi081 ceph-mon[30961]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1daca7a-630f-4a56-bd3d-713fc3f88ede"}]: dispatch 2024-04-24T09:00:06.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:06 smithi081 ceph-mon[30961]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1daca7a-630f-4a56-bd3d-713fc3f88ede"}]': finished 2024-04-24T09:00:06.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:06 smithi081 ceph-mon[30961]: osdmap e48: 8 total, 7 up, 8 in 2024-04-24T09:00:06.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:00:06.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:06 smithi069 ceph-mon[28170]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:06.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:06 smithi069 ceph-mon[28170]: from='client.? 172.21.15.81:0/685373659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1daca7a-630f-4a56-bd3d-713fc3f88ede"}]: dispatch 2024-04-24T09:00:06.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:06 smithi069 ceph-mon[28170]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1daca7a-630f-4a56-bd3d-713fc3f88ede"}]: dispatch 2024-04-24T09:00:06.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:06 smithi069 ceph-mon[28170]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1daca7a-630f-4a56-bd3d-713fc3f88ede"}]': finished 2024-04-24T09:00:06.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:06 smithi069 ceph-mon[28170]: osdmap e48: 8 total, 7 up, 8 in 2024-04-24T09:00:06.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:00:07.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:07 smithi130 ceph-mon[31553]: from='client.? 172.21.15.81:0/710169238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:00:07.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:07 smithi081 ceph-mon[30961]: from='client.? 172.21.15.81:0/710169238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:00:07.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:07 smithi069 ceph-mon[28170]: from='client.? 172.21.15.81:0/710169238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:00:08.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:08 smithi130 ceph-mon[31553]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:08.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:08 smithi081 ceph-mon[30961]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:08 smithi069 ceph-mon[28170]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:10.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:10 smithi130 ceph-mon[31553]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:10.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:10 smithi081 ceph-mon[30961]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:10.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:10 smithi069 ceph-mon[28170]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:12.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:12 smithi130 ceph-mon[31553]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:12.781 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:12 smithi081 ceph-mon[30961]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:12.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:12 smithi069 ceph-mon[28170]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:14.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:14 smithi081 ceph-mon[30961]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:14.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:14 smithi130 ceph-mon[31553]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:14.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:14 smithi069 ceph-mon[28170]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:15.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:15 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T09:00:15.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:15 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:15.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:15 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T09:00:15.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:15 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:15.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:15 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T09:00:15.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:15 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:16.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:16 smithi130 ceph-mon[31553]: Deploying daemon osd.7 on smithi081 2024-04-24T09:00:16.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:16 smithi130 ceph-mon[31553]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:16.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:16 smithi081 ceph-mon[30961]: Deploying daemon osd.7 on smithi081 2024-04-24T09:00:16.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:16 smithi081 ceph-mon[30961]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:16.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:16 smithi069 ceph-mon[28170]: Deploying daemon osd.7 on smithi081 2024-04-24T09:00:16.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:16 smithi069 ceph-mon[28170]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:18.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:18 smithi130 ceph-mon[31553]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:18.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:18 smithi081 ceph-mon[30961]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:18.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:18 smithi069 ceph-mon[28170]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:20.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:20 smithi081 ceph-mon[30961]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:20.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:20 smithi130 ceph-mon[31553]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:20.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:20 smithi069 ceph-mon[28170]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:22.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:22 smithi081 ceph-mon[30961]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:22.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:22 smithi069 ceph-mon[28170]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:22.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:22 smithi130 ceph-mon[31553]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:24.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:24 smithi069 ceph-mon[28170]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:24.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:24.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:24.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:00:24.742 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:24 smithi081 ceph-mon[30961]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:24.742 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:24.742 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:24.742 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:00:24.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:24 smithi130 ceph-mon[31553]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:24.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:24.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:24.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:00:26.532 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:26 smithi081 ceph-mon[30961]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:26.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:26 smithi069 ceph-mon[28170]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:26.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:26 smithi130 ceph-mon[31553]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:28.257 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:28 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:28.257 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:28 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:28.257 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:28 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:28.257 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:28 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:00:28.257 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:28 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:28.257 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:28 smithi081 ceph-mon[30961]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:28.257 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:28 smithi081 ceph-mon[30961]: from='osd.7 [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T09:00:28.258 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:28 smithi081 ceph-mon[30961]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T09:00:28.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:28.387+0000 7f13c4c00700 1 -- 172.21.15.81:0/608227375 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f13c8062f30 con 0x7f13ac0a3c40 2024-04-24T09:00:28.388 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 7 on host 'smithi081' 2024-04-24T09:00:28.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:28.390+0000 7f13cda00700 1 -- 172.21.15.81:0/608227375 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13ac0a3c40 msgr2=0x7f13ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:28.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:28.390+0000 7f13cda00700 1 --2- 172.21.15.81:0/608227375 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13ac0a3c40 0x7f13ac0a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f13b8005950 tx=0x7f13b80058e0 comp rx=0 tx=0).stop 2024-04-24T09:00:28.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:28.390+0000 7f13cda00700 1 -- 172.21.15.81:0/608227375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13c810ac20 msgr2=0x7f13c81a66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:28.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:28.390+0000 7f13cda00700 1 --2- 172.21.15.81:0/608227375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13c810ac20 0x7f13c81a66d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f13c0007d70 tx=0x7f13c000a340 comp rx=0 tx=0).stop 2024-04-24T09:00:28.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:28.390+0000 7f13cda00700 1 -- 172.21.15.81:0/608227375 shutdown_connections 2024-04-24T09:00:28.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:28.390+0000 7f13cda00700 1 --2- 172.21.15.81:0/608227375 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13ac0a3c40 0x7f13ac0a6100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:28.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:28.390+0000 7f13cda00700 1 --2- 172.21.15.81:0/608227375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13c810ac20 0x7f13c81a66d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:28.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:28.390+0000 7f13cda00700 1 --2- 172.21.15.81:0/608227375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13c8109080 0x7f13c819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:28.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:28.390+0000 7f13cda00700 1 --2- 172.21.15.81:0/608227375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13c810a280 0x7f13c819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:28.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:28.390+0000 7f13cda00700 1 -- 172.21.15.81:0/608227375 >> 172.21.15.81:0/608227375 conn(0x7f13c8078050 msgr2=0x7f13c810d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:28.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:28.391+0000 7f13cda00700 1 -- 172.21.15.81:0/608227375 shutdown_connections 2024-04-24T09:00:28.392 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:00:28.391+0000 7f13cda00700 1 -- 172.21.15.81:0/608227375 wait complete. 2024-04-24T09:00:28.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:28 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:28.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:28 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:28.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:28 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:28.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:28 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:00:28.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:28 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:28.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:28 smithi130 ceph-mon[31553]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:28.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:28 smithi130 ceph-mon[31553]: from='osd.7 [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T09:00:28.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:28 smithi130 ceph-mon[31553]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T09:00:28.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:28 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:28.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:28 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:28.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:28 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:28.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:28 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:00:28.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:28 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:28.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:28 smithi069 ceph-mon[28170]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:28.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:28 smithi069 ceph-mon[28170]: from='osd.7 [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T09:00:28.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:28 smithi069 ceph-mon[28170]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T09:00:29.113 DEBUG:teuthology.orchestra.run.smithi081:osd.7> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.7.service 2024-04-24T09:00:29.117 INFO:tasks.cephadm:Deploying osd.8 on smithi130 with /dev/vg_nvme/lv_2... 2024-04-24T09:00:29.117 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T09:00:29.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:29 smithi081 ceph-mon[30961]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T09:00:29.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:29 smithi081 ceph-mon[30961]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T09:00:29.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:29 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:00:29.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:29 smithi081 ceph-mon[30961]: from='osd.7 [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T09:00:29.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:29 smithi081 ceph-mon[30961]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T09:00:29.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:29 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:29.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:29 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:29.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:29 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:00:29.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:29 smithi130 ceph-mon[31553]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T09:00:29.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:29 smithi130 ceph-mon[31553]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T09:00:29.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:29 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:00:29.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:29 smithi130 ceph-mon[31553]: from='osd.7 [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T09:00:29.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:29 smithi130 ceph-mon[31553]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T09:00:29.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:29 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:29.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:29 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:29.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:29 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:00:29.551 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:00:29.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:29 smithi069 ceph-mon[28170]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T09:00:29.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:29 smithi069 ceph-mon[28170]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T09:00:29.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:29 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:00:29.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:29 smithi069 ceph-mon[28170]: from='osd.7 [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T09:00:29.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:29 smithi069 ceph-mon[28170]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T09:00:29.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:29 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:29.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:29 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:29.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:29 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:00:30.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:30 smithi081 ceph-mon[30961]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-04-24T09:00:30.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:30 smithi081 ceph-mon[30961]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T09:00:30.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:30 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:00:30.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:30 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:00:30.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:30 smithi081 ceph-mon[30961]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:30.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:30 smithi081 ceph-mon[30961]: from='osd.7 ' entity='osd.7' 2024-04-24T09:00:30.318 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:00:30 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:00:30.006+0000 7f544f000700 -1 osd.7 0 waiting for initial osdmap 2024-04-24T09:00:30.318 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:00:30 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:00:30.036+0000 7f5444400700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T09:00:30.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:30 smithi130 ceph-mon[31553]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-04-24T09:00:30.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:30 smithi130 ceph-mon[31553]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T09:00:30.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:30 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:00:30.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:30 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:00:30.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:30 smithi130 ceph-mon[31553]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:30.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:30 smithi130 ceph-mon[31553]: from='osd.7 ' entity='osd.7' 2024-04-24T09:00:30.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:30 smithi069 ceph-mon[28170]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-04-24T09:00:30.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:30 smithi069 ceph-mon[28170]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T09:00:30.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:30 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:00:30.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:30 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:00:30.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:30 smithi069 ceph-mon[28170]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:00:30.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:30 smithi069 ceph-mon[28170]: from='osd.7 ' entity='osd.7' 2024-04-24T09:00:31.439 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:31 smithi081 ceph-mon[30961]: purged_snaps scrub starts 2024-04-24T09:00:31.439 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:31 smithi081 ceph-mon[30961]: purged_snaps scrub ok 2024-04-24T09:00:31.440 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:31 smithi081 ceph-mon[30961]: osd.7 [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153] boot 2024-04-24T09:00:31.440 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:31 smithi081 ceph-mon[30961]: osdmap e51: 8 total, 8 up, 8 in 2024-04-24T09:00:31.440 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:31 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:00:31.440 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:31 smithi081 ceph-mon[30961]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T09:00:31.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:31 smithi130 ceph-mon[31553]: purged_snaps scrub starts 2024-04-24T09:00:31.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:31 smithi130 ceph-mon[31553]: purged_snaps scrub ok 2024-04-24T09:00:31.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:31 smithi130 ceph-mon[31553]: osd.7 [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153] boot 2024-04-24T09:00:31.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:31 smithi130 ceph-mon[31553]: osdmap e51: 8 total, 8 up, 8 in 2024-04-24T09:00:31.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:31 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:00:31.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:31 smithi130 ceph-mon[31553]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T09:00:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:31 smithi069 ceph-mon[28170]: purged_snaps scrub starts 2024-04-24T09:00:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:31 smithi069 ceph-mon[28170]: purged_snaps scrub ok 2024-04-24T09:00:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:31 smithi069 ceph-mon[28170]: osd.7 [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153] boot 2024-04-24T09:00:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:31 smithi069 ceph-mon[28170]: osdmap e51: 8 total, 8 up, 8 in 2024-04-24T09:00:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:31 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:00:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:31 smithi069 ceph-mon[28170]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T09:00:32.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:32 smithi130 ceph-mon[31553]: pgmap v226: 1 pgs: 1 remapped; 577 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:32.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:32 smithi130 ceph-mon[31553]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T09:00:32.519 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T09:00:32.519 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T09:00:32.519 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-04-24T09:00:32.520 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-04-24T09:00:32.520 INFO:teuthology.orchestra.run.smithi130.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0368316 s, 285 MB/s 2024-04-24T09:00:32.520 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-04-24T09:00:32.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:32 smithi081 ceph-mon[30961]: pgmap v226: 1 pgs: 1 remapped; 577 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:32.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:32 smithi081 ceph-mon[30961]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T09:00:32.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:32 smithi069 ceph-mon[28170]: pgmap v226: 1 pgs: 1 remapped; 577 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:32.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:32 smithi069 ceph-mon[28170]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T09:00:33.436 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:33 smithi130 ceph-mon[31553]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-24T09:00:33.437 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:33 smithi130 ceph-mon[31553]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T09:00:33.492 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_2 2024-04-24T09:00:33.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:33 smithi081 ceph-mon[30961]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-24T09:00:33.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:33 smithi081 ceph-mon[30961]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T09:00:33.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:33 smithi069 ceph-mon[28170]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-24T09:00:33.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:33 smithi069 ceph-mon[28170]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T09:00:33.935 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:00:34.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:34 smithi081 ceph-mon[30961]: pgmap v229: 1 pgs: 1 remapped; 577 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:34.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:34 smithi081 ceph-mon[30961]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T09:00:34.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:34 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:34.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:34 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:34.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:34 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:34.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:34 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:34.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:34 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:34.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:34 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:34.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:34 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:34.341 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:34 smithi130 ceph-mon[31553]: pgmap v229: 1 pgs: 1 remapped; 577 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:34.341 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:34 smithi130 ceph-mon[31553]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T09:00:34.342 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:34 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:34.342 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:34 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:34.342 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:34 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:34.342 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:34 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:34.342 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:34 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:34.342 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:34 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:34.342 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:34 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:34.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:34 smithi069 ceph-mon[28170]: pgmap v229: 1 pgs: 1 remapped; 577 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:34.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:34 smithi069 ceph-mon[28170]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T09:00:34.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:34 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:34.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:34 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:34.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:34 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:34.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:34 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:34.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:34 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:00:34.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:34 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:34.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:34 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:35.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:35 smithi130 ceph-mon[31553]: Detected new or changed devices on smithi081 2024-04-24T09:00:35.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:35 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi081 to 5696M 2024-04-24T09:00:35.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:35 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:00:35.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:35 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:35.457 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:35 smithi081 ceph-mon[30961]: Detected new or changed devices on smithi081 2024-04-24T09:00:35.457 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:35 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi081 to 5696M 2024-04-24T09:00:35.457 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:35 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:00:35.457 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:35 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:35.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:35 smithi069 ceph-mon[28170]: Detected new or changed devices on smithi081 2024-04-24T09:00:35.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:35 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi081 to 5696M 2024-04-24T09:00:35.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:35 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:00:35.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:35 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:36.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.332+0000 7feb8fa00700 1 -- 172.21.15.130:0/172998618 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb88109190 msgr2=0x7feb881095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:36.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.332+0000 7feb8fa00700 1 --2- 172.21.15.130:0/172998618 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb88109190 0x7feb881095b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009990 tx=0x7feb7c009ca0 comp rx=0 tx=0).stop 2024-04-24T09:00:36.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.332+0000 7feb8fa00700 1 -- 172.21.15.130:0/172998618 shutdown_connections 2024-04-24T09:00:36.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.332+0000 7feb8fa00700 1 --2- 172.21.15.130:0/172998618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb8810ad30 0x7feb881115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:36.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.332+0000 7feb8fa00700 1 --2- 172.21.15.130:0/172998618 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb88109190 0x7feb881095b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:36.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.332+0000 7feb8fa00700 1 --2- 172.21.15.130:0/172998618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb8810a390 0x7feb8810a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:36.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.332+0000 7feb8fa00700 1 -- 172.21.15.130:0/172998618 >> 172.21.15.130:0/172998618 conn(0x7feb88078160 msgr2=0x7feb8807a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:36.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.332+0000 7feb8fa00700 1 -- 172.21.15.130:0/172998618 shutdown_connections 2024-04-24T09:00:36.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.332+0000 7feb8fa00700 1 -- 172.21.15.130:0/172998618 wait complete. 2024-04-24T09:00:36.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.333+0000 7feb8fa00700 1 Processor -- start 2024-04-24T09:00:36.335 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.333+0000 7feb8fa00700 1 -- start start 2024-04-24T09:00:36.335 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.334+0000 7feb8fa00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb88109190 0x7feb8819f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:36.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.334+0000 7feb8fa00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb8810a390 0x7feb8819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:36.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.334+0000 7feb8fa00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb8810ad30 0x7feb881a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:36.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.334+0000 7feb8fa00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feb88114c70 con 0x7feb8810ad30 2024-04-24T09:00:36.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.334+0000 7feb8fa00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7feb88114af0 con 0x7feb8810a390 2024-04-24T09:00:36.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.334+0000 7feb8fa00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7feb88114df0 con 0x7feb88109190 2024-04-24T09:00:36.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.334+0000 7feb8d000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb88109190 0x7feb8819f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:36.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.334+0000 7feb8d000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb88109190 0x7feb8819f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:39468/0 (socket says 172.21.15.130:39468) 2024-04-24T09:00:36.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.334+0000 7feb8d000700 1 -- 172.21.15.130:0/1439493264 learned_addr learned my addr 172.21.15.130:0/1439493264 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-24T09:00:36.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.334+0000 7feb87e00700 1 --2- 172.21.15.130:0/1439493264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb8810a390 0x7feb8819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:36.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.334+0000 7feb8da00700 1 --2- 172.21.15.130:0/1439493264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb8810ad30 0x7feb881a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:36.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.335+0000 7feb8d000700 1 -- 172.21.15.130:0/1439493264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb8810a390 msgr2=0x7feb8819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:36.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.335+0000 7feb8d000700 1 --2- 172.21.15.130:0/1439493264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb8810a390 0x7feb8819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:36.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.335+0000 7feb8d000700 1 -- 172.21.15.130:0/1439493264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb8810ad30 msgr2=0x7feb881a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:36.337 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.335+0000 7feb8d000700 1 --2- 172.21.15.130:0/1439493264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb8810ad30 0x7feb881a68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:36.337 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.335+0000 7feb8d000700 1 -- 172.21.15.130:0/1439493264 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb881a6f40 con 0x7feb88109190 2024-04-24T09:00:36.337 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.335+0000 7feb8da00700 1 --2- 172.21.15.130:0/1439493264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb8810ad30 0x7feb881a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:36.337 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.335+0000 7feb8d000700 1 --2- 172.21.15.130:0/1439493264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb88109190 0x7feb8819f2e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0037f0 tx=0x7feb7c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:36.337 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.335+0000 7feb85600700 1 -- 172.21.15.130:0/1439493264 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c003c90 con 0x7feb88109190 2024-04-24T09:00:36.337 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.335+0000 7feb85600700 1 -- 172.21.15.130:0/1439493264 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb7c004830 con 0x7feb88109190 2024-04-24T09:00:36.337 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.335+0000 7feb8fa00700 1 -- 172.21.15.130:0/1439493264 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb881a71d0 con 0x7feb88109190 2024-04-24T09:00:36.337 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.335+0000 7feb85600700 1 -- 172.21.15.130:0/1439493264 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c0054e0 con 0x7feb88109190 2024-04-24T09:00:36.337 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.336+0000 7feb8fa00700 1 -- 172.21.15.130:0/1439493264 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb881a75c0 con 0x7feb88109190 2024-04-24T09:00:36.337 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.337+0000 7feb8fa00700 1 -- 172.21.15.130:0/1439493264 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb880697e0 con 0x7feb88109190 2024-04-24T09:00:36.338 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.339+0000 7feb85600700 1 -- 172.21.15.130:0/1439493264 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feb7c0049e0 con 0x7feb88109190 2024-04-24T09:00:36.340 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.340+0000 7feb85600700 1 --2- 172.21.15.130:0/1439493264 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7feb700a3c40 0x7feb700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:36.340 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.340+0000 7feb85600700 1 -- 172.21.15.130:0/1439493264 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7feb7c0d0e30 con 0x7feb88109190 2024-04-24T09:00:36.340 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.341+0000 7feb87e00700 1 --2- 172.21.15.130:0/1439493264 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7feb700a3c40 0x7feb700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:36.341 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.342+0000 7feb87e00700 1 --2- 172.21.15.130:0/1439493264 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7feb700a3c40 0x7feb700a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feb74005950 tx=0x7feb740058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:36.342 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.343+0000 7feb85600700 1 -- 172.21.15.130:0/1439493264 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb7c013070 con 0x7feb88109190 2024-04-24T09:00:36.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:36 smithi130 ceph-mon[31553]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:36.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:36 smithi081 ceph-mon[30961]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:36.579 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:00:36.579+0000 7feb8fa00700 1 -- 172.21.15.130:0/1439493264 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7feb88062f30 con 0x7feb700a3c40 2024-04-24T09:00:36.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:36 smithi069 ceph-mon[28170]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:37.562 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:37 smithi130 ceph-mon[31553]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-24T09:00:37.562 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:37 smithi130 ceph-mon[31553]: Cluster is now healthy 2024-04-24T09:00:37.562 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:37 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:00:37.562 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:37 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:00:37.562 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:37 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:37.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:37 smithi081 ceph-mon[30961]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-24T09:00:37.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:37 smithi081 ceph-mon[30961]: Cluster is now healthy 2024-04-24T09:00:37.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:00:37.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:00:37.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:37 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:37.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:37 smithi069 ceph-mon[28170]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-24T09:00:37.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:37 smithi069 ceph-mon[28170]: Cluster is now healthy 2024-04-24T09:00:37.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:37 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:00:37.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:37 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:00:37.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:37 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:38.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:38 smithi081 ceph-mon[30961]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:00:38.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:38 smithi081 ceph-mon[30961]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:38.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:38 smithi069 ceph-mon[28170]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:00:38.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:38 smithi069 ceph-mon[28170]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:38.602 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:38 smithi130 ceph-mon[31553]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:00:38.602 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:38 smithi130 ceph-mon[31553]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:40.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:40 smithi130 ceph-mon[31553]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:40 smithi081 ceph-mon[30961]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:40 smithi069 ceph-mon[28170]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:41.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:41 smithi130 ceph-mon[31553]: from='client.? 172.21.15.130:0/2621654367' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "151820a5-68cc-4d46-af26-eff7f37e3186"}]: dispatch 2024-04-24T09:00:41.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:41 smithi130 ceph-mon[31553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "151820a5-68cc-4d46-af26-eff7f37e3186"}]: dispatch 2024-04-24T09:00:41.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:41 smithi130 ceph-mon[31553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "151820a5-68cc-4d46-af26-eff7f37e3186"}]': finished 2024-04-24T09:00:41.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:41 smithi130 ceph-mon[31553]: osdmap e56: 9 total, 8 up, 9 in 2024-04-24T09:00:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:00:41.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:41 smithi081 ceph-mon[30961]: from='client.? 172.21.15.130:0/2621654367' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "151820a5-68cc-4d46-af26-eff7f37e3186"}]: dispatch 2024-04-24T09:00:41.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:41 smithi081 ceph-mon[30961]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "151820a5-68cc-4d46-af26-eff7f37e3186"}]: dispatch 2024-04-24T09:00:41.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:41 smithi081 ceph-mon[30961]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "151820a5-68cc-4d46-af26-eff7f37e3186"}]': finished 2024-04-24T09:00:41.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:41 smithi081 ceph-mon[30961]: osdmap e56: 9 total, 8 up, 9 in 2024-04-24T09:00:41.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:00:41.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:41 smithi069 ceph-mon[28170]: from='client.? 172.21.15.130:0/2621654367' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "151820a5-68cc-4d46-af26-eff7f37e3186"}]: dispatch 2024-04-24T09:00:41.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:41 smithi069 ceph-mon[28170]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "151820a5-68cc-4d46-af26-eff7f37e3186"}]: dispatch 2024-04-24T09:00:41.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:41 smithi069 ceph-mon[28170]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "151820a5-68cc-4d46-af26-eff7f37e3186"}]': finished 2024-04-24T09:00:41.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:41 smithi069 ceph-mon[28170]: osdmap e56: 9 total, 8 up, 9 in 2024-04-24T09:00:41.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:00:42.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:42 smithi130 ceph-mon[31553]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:42.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:42 smithi130 ceph-mon[31553]: from='client.? 172.21.15.130:0/2033606839' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:00:42.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:42 smithi081 ceph-mon[30961]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:42.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:42 smithi081 ceph-mon[30961]: from='client.? 172.21.15.130:0/2033606839' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:00:42.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:42 smithi069 ceph-mon[28170]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:42.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:42 smithi069 ceph-mon[28170]: from='client.? 172.21.15.130:0/2033606839' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:00:44.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:44 smithi130 ceph-mon[31553]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:44.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:44 smithi081 ceph-mon[30961]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:44.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:44 smithi069 ceph-mon[28170]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:46.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:46 smithi130 ceph-mon[31553]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:46.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:46 smithi081 ceph-mon[30961]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:46.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:46 smithi069 ceph-mon[28170]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:48.523 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:48 smithi130 ceph-mon[31553]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:48.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:48 smithi081 ceph-mon[30961]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:48.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:48 smithi069 ceph-mon[28170]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:50.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:50 smithi130 ceph-mon[31553]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:50.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:50 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T09:00:50.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:50 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:50.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:50 smithi081 ceph-mon[30961]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:50.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:50 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T09:00:50.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:50 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:50.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:50 smithi069 ceph-mon[28170]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:50.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:50 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T09:00:50.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:50 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:51.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:51 smithi130 ceph-mon[31553]: Deploying daemon osd.8 on smithi130 2024-04-24T09:00:51.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:51 smithi081 ceph-mon[30961]: Deploying daemon osd.8 on smithi130 2024-04-24T09:00:51.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:51 smithi069 ceph-mon[28170]: Deploying daemon osd.8 on smithi130 2024-04-24T09:00:52.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:52 smithi130 ceph-mon[31553]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:52.743 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:52 smithi069 ceph-mon[28170]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:52.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:52 smithi081 ceph-mon[30961]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:54.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:54 smithi130 ceph-mon[31553]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:54.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:54 smithi081 ceph-mon[30961]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:54.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:54 smithi069 ceph-mon[28170]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:56.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:56 smithi130 ceph-mon[31553]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:56.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:56 smithi081 ceph-mon[30961]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:56.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:56 smithi069 ceph-mon[28170]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:58.649 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:58 smithi130 ceph-mon[31553]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:58.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:58 smithi081 ceph-mon[30961]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:58.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:58 smithi069 ceph-mon[28170]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:00:59.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:00:59.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:59.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:00:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:59.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:00:59.894 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:59.894 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:00:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:59.895 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:00:59.895 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:00:59.895 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:00:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:00.563 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:00 smithi130 ceph-mon[31553]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:01:00.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:00 smithi081 ceph-mon[30961]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:01:00.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:00 smithi069 ceph-mon[28170]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:01:02.664 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:02 smithi130 ceph-mon[31553]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:01:02.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:02 smithi081 ceph-mon[30961]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:01:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:02 smithi069 ceph-mon[28170]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:01:04.112 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:03 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:04.112 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:03 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:04.112 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:03 smithi130 ceph-mon[31553]: from='osd.8 [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T09:01:04.112 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:03 smithi130 ceph-mon[31553]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T09:01:04.112 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:03 smithi130 ceph-mon[31553]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:01:04.255 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:01:04.256+0000 7feb85600700 1 -- 172.21.15.130:0/1439493264 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7feb88062f30 con 0x7feb700a3c40 2024-04-24T09:01:04.256 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 8 on host 'smithi130' 2024-04-24T09:01:04.259 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:01:04.260+0000 7feb8fa00700 1 -- 172.21.15.130:0/1439493264 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7feb700a3c40 msgr2=0x7feb700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:04.260 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:01:04.260+0000 7feb8fa00700 1 --2- 172.21.15.130:0/1439493264 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7feb700a3c40 0x7feb700a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feb74005950 tx=0x7feb740058e0 comp rx=0 tx=0).stop 2024-04-24T09:01:04.260 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:01:04.260+0000 7feb8fa00700 1 -- 172.21.15.130:0/1439493264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb88109190 msgr2=0x7feb8819f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:04.260 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:01:04.260+0000 7feb8fa00700 1 --2- 172.21.15.130:0/1439493264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb88109190 0x7feb8819f2e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0037f0 tx=0x7feb7c0047b0 comp rx=0 tx=0).stop 2024-04-24T09:01:04.260 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:01:04.260+0000 7feb8fa00700 1 -- 172.21.15.130:0/1439493264 shutdown_connections 2024-04-24T09:01:04.260 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:01:04.260+0000 7feb8fa00700 1 --2- 172.21.15.130:0/1439493264 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7feb700a3c40 0x7feb700a6100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:04.260 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:01:04.260+0000 7feb8fa00700 1 --2- 172.21.15.130:0/1439493264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb8810ad30 0x7feb881a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:04.260 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:01:04.260+0000 7feb8fa00700 1 --2- 172.21.15.130:0/1439493264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb88109190 0x7feb8819f2e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:04.260 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:01:04.260+0000 7feb8fa00700 1 --2- 172.21.15.130:0/1439493264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb8810a390 0x7feb8819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:04.260 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:01:04.260+0000 7feb8fa00700 1 -- 172.21.15.130:0/1439493264 >> 172.21.15.130:0/1439493264 conn(0x7feb88078160 msgr2=0x7feb8810d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:04.260 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:01:04.260+0000 7feb8fa00700 1 -- 172.21.15.130:0/1439493264 shutdown_connections 2024-04-24T09:01:04.260 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:01:04.260+0000 7feb8fa00700 1 -- 172.21.15.130:0/1439493264 wait complete. 2024-04-24T09:01:04.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:03 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:04.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:03 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:04.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:03 smithi081 ceph-mon[30961]: from='osd.8 [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T09:01:04.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:03 smithi081 ceph-mon[30961]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T09:01:04.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:03 smithi081 ceph-mon[30961]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:01:04.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:03 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:04.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:03 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:04.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:03 smithi069 ceph-mon[28170]: from='osd.8 [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T09:01:04.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:03 smithi069 ceph-mon[28170]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T09:01:04.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:03 smithi069 ceph-mon[28170]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:01:04.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:04 smithi130 ceph-mon[31553]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T09:01:04.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:04 smithi130 ceph-mon[31553]: from='osd.8 [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T09:01:04.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:04 smithi130 ceph-mon[31553]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T09:01:04.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:04 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:01:04.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:04 smithi130 ceph-mon[31553]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T09:01:04.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:04 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:04.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:04 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:05.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:04 smithi069 ceph-mon[28170]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T09:01:05.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:04 smithi069 ceph-mon[28170]: from='osd.8 [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T09:01:05.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:04 smithi069 ceph-mon[28170]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T09:01:05.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:04 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:01:05.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:04 smithi069 ceph-mon[28170]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T09:01:05.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:04 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:05.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:04 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:05.163 DEBUG:teuthology.orchestra.run.smithi130:osd.8> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.8.service 2024-04-24T09:01:05.169 INFO:tasks.cephadm:Deploying osd.9 on smithi069 with /dev/vg_nvme/lv_1... 2024-04-24T09:01:05.170 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T09:01:05.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:04 smithi081 ceph-mon[30961]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T09:01:05.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:04 smithi081 ceph-mon[30961]: from='osd.8 [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T09:01:05.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:04 smithi081 ceph-mon[30961]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T09:01:05.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:04 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:01:05.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:04 smithi081 ceph-mon[30961]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T09:01:05.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:04 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:05.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:04 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:05.626 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:01:06.027 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:05 smithi069 ceph-mon[28170]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-24T09:01:06.027 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:05 smithi069 ceph-mon[28170]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T09:01:06.027 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:05 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:01:06.027 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:05 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:01:06.027 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:05 smithi069 ceph-mon[28170]: pgmap v249: 1 pgs: 1 peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:01:06.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:05 smithi130 ceph-mon[31553]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-24T09:01:06.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:05 smithi130 ceph-mon[31553]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T09:01:06.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:05 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:01:06.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:05 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:01:06.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:05 smithi130 ceph-mon[31553]: pgmap v249: 1 pgs: 1 peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:01:06.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:05 smithi081 ceph-mon[30961]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-24T09:01:06.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:05 smithi081 ceph-mon[30961]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T09:01:06.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:05 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:01:06.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:05 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:01:06.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:05 smithi081 ceph-mon[30961]: pgmap v249: 1 pgs: 1 peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:01:07.073 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:06 smithi130 ceph-mon[31553]: purged_snaps scrub starts 2024-04-24T09:01:07.073 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:06 smithi130 ceph-mon[31553]: purged_snaps scrub ok 2024-04-24T09:01:07.074 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:01:07.074 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:06 smithi130 ceph-mon[31553]: osd.8 [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143] boot 2024-04-24T09:01:07.074 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:06 smithi130 ceph-mon[31553]: osdmap e59: 9 total, 9 up, 9 in 2024-04-24T09:01:07.074 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:01:07.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:06 smithi081 ceph-mon[30961]: purged_snaps scrub starts 2024-04-24T09:01:07.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:06 smithi081 ceph-mon[30961]: purged_snaps scrub ok 2024-04-24T09:01:07.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:01:07.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:06 smithi081 ceph-mon[30961]: osd.8 [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143] boot 2024-04-24T09:01:07.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:06 smithi081 ceph-mon[30961]: osdmap e59: 9 total, 9 up, 9 in 2024-04-24T09:01:07.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:01:07.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:06 smithi069 ceph-mon[28170]: purged_snaps scrub starts 2024-04-24T09:01:07.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:06 smithi069 ceph-mon[28170]: purged_snaps scrub ok 2024-04-24T09:01:07.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:01:07.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:06 smithi069 ceph-mon[28170]: osd.8 [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143] boot 2024-04-24T09:01:07.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:06 smithi069 ceph-mon[28170]: osdmap e59: 9 total, 9 up, 9 in 2024-04-24T09:01:07.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:01:08.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:07 smithi069 ceph-mon[28170]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T09:01:08.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:07 smithi069 ceph-mon[28170]: pgmap v252: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:08.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:07 smithi081 ceph-mon[30961]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T09:01:08.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:07 smithi081 ceph-mon[30961]: pgmap v252: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:08.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:07 smithi130 ceph-mon[31553]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T09:01:08.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:07 smithi130 ceph-mon[31553]: pgmap v252: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:08.750 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T09:01:08.750 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T09:01:08.750 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-04-24T09:01:08.750 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-04-24T09:01:08.750 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.036316 s, 289 MB/s 2024-04-24T09:01:08.750 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-04-24T09:01:09.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:08 smithi069 ceph-mon[28170]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T09:01:09.316 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:08 smithi081 ceph-mon[30961]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T09:01:09.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:08 smithi130 ceph-mon[31553]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T09:01:09.717 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_1 2024-04-24T09:01:09.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:09 smithi069 ceph-mon[28170]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T09:01:09.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:09 smithi069 ceph-mon[28170]: pgmap v255: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:10.135 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:01:10.316 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:09 smithi081 ceph-mon[30961]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T09:01:10.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:09 smithi081 ceph-mon[30961]: pgmap v255: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:10.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:09 smithi130 ceph-mon[31553]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T09:01:10.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:09 smithi130 ceph-mon[31553]: pgmap v255: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:11.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:10 smithi081 ceph-mon[30961]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T09:01:11.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:10 smithi081 ceph-mon[30961]: Detected new or changed devices on smithi130 2024-04-24T09:01:11.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:10 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:11.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:10 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:11.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:10 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:11.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:10 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:11.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:10 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:11.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:10 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi130 to 5696M 2024-04-24T09:01:11.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:10 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:11.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:10 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:11.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:10 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:01:11.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:10 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:11.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:10 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:01:11.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:11 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:11.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:11 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:01:11.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:11 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:11.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:10 smithi069 ceph-mon[28170]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T09:01:11.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:10 smithi069 ceph-mon[28170]: Detected new or changed devices on smithi130 2024-04-24T09:01:11.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:10 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:11.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:10 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:11.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:10 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:11.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:10 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:11.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:10 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:11.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:10 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi130 to 5696M 2024-04-24T09:01:11.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:10 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:11.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:11 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:11.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:11 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:01:11.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:11 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:11.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:11 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:01:11.328 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:11 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:11.328 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:11 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:01:11.328 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:11 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:11.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:10 smithi130 ceph-mon[31553]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T09:01:11.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:10 smithi130 ceph-mon[31553]: Detected new or changed devices on smithi130 2024-04-24T09:01:11.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:10 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:11.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:10 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:11.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:10 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:11.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:10 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:11.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:11 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:11.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:11 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi130 to 5696M 2024-04-24T09:01:11.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:11 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:11.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:11 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:11.455 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:11 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:01:11.455 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:11 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:11.455 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:11 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:01:11.455 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:11 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:11.455 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:11 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:01:11.455 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:11 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:12.221 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:12 smithi069 ceph-mon[28170]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-04-24T09:01:12.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:12 smithi081 ceph-mon[30961]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-04-24T09:01:12.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:12 smithi130 ceph-mon[31553]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-04-24T09:01:12.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.509+0000 7f7255e00700 1 -- 172.21.15.69:0/438803875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7250101eb0 msgr2=0x7f7250102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:12.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.509+0000 7f7255e00700 1 --2- 172.21.15.69:0/438803875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7250101eb0 0x7f7250102310 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f723c009990 tx=0x7f723c009ca0 comp rx=0 tx=0).stop 2024-04-24T09:01:12.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.509+0000 7f7255e00700 1 -- 172.21.15.69:0/438803875 shutdown_connections 2024-04-24T09:01:12.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.509+0000 7f7255e00700 1 --2- 172.21.15.69:0/438803875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7250102850 0x7f7250109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:12.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.509+0000 7f7255e00700 1 --2- 172.21.15.69:0/438803875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7250100cb0 0x7f72501010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:12.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.509+0000 7f7255e00700 1 --2- 172.21.15.69:0/438803875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7250101eb0 0x7f7250102310 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:12.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.509+0000 7f7255e00700 1 -- 172.21.15.69:0/438803875 >> 172.21.15.69:0/438803875 conn(0x7f72500fc270 msgr2=0x7f72500fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:12.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.509+0000 7f7255e00700 1 -- 172.21.15.69:0/438803875 shutdown_connections 2024-04-24T09:01:12.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.509+0000 7f7255e00700 1 -- 172.21.15.69:0/438803875 wait complete. 2024-04-24T09:01:12.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.510+0000 7f7255e00700 1 Processor -- start 2024-04-24T09:01:12.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.510+0000 7f7255e00700 1 -- start start 2024-04-24T09:01:12.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.511+0000 7f7255e00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7250100cb0 0x7f725019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:12.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.511+0000 7f7255e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7250101eb0 0x7f725019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:12.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.511+0000 7f7255e00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7250102850 0x7f72501a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:12.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.511+0000 7f7255e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f725010d450 con 0x7f7250101eb0 2024-04-24T09:01:12.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.511+0000 7f7255e00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f725010d2d0 con 0x7f7250102850 2024-04-24T09:01:12.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.511+0000 7f7255e00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f725010d5d0 con 0x7f7250100cb0 2024-04-24T09:01:12.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.511+0000 7f724fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7250101eb0 0x7f725019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:12.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.511+0000 7f724fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7250101eb0 0x7f725019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40546/0 (socket says 172.21.15.69:40546) 2024-04-24T09:01:12.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.511+0000 7f724fe00700 1 -- 172.21.15.69:0/2650380760 learned_addr learned my addr 172.21.15.69:0/2650380760 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:01:12.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.511+0000 7f7254a00700 1 --2- 172.21.15.69:0/2650380760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7250100cb0 0x7f725019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:12.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.512+0000 7f7255400700 1 --2- 172.21.15.69:0/2650380760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7250102850 0x7f72501a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:12.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.512+0000 7f724fe00700 1 -- 172.21.15.69:0/2650380760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7250100cb0 msgr2=0x7f725019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:12.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.512+0000 7f724fe00700 1 --2- 172.21.15.69:0/2650380760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7250100cb0 0x7f725019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:12.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.512+0000 7f724fe00700 1 -- 172.21.15.69:0/2650380760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7250102850 msgr2=0x7f72501a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:12.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.512+0000 7f724fe00700 1 --2- 172.21.15.69:0/2650380760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7250102850 0x7f72501a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:12.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.512+0000 7f724fe00700 1 -- 172.21.15.69:0/2650380760 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72501a2a60 con 0x7f7250101eb0 2024-04-24T09:01:12.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.512+0000 7f7255400700 1 --2- 172.21.15.69:0/2650380760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7250102850 0x7f72501a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:12.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.512+0000 7f724fe00700 1 --2- 172.21.15.69:0/2650380760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7250101eb0 0x7f725019b390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f723c000c00 tx=0x7f723c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:12.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.512+0000 7f724d600700 1 -- 172.21.15.69:0/2650380760 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c027070 con 0x7f7250101eb0 2024-04-24T09:01:12.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.512+0000 7f724d600700 1 -- 172.21.15.69:0/2650380760 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f723c022470 con 0x7f7250101eb0 2024-04-24T09:01:12.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.512+0000 7f7255e00700 1 -- 172.21.15.69:0/2650380760 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72501a2cf0 con 0x7f7250101eb0 2024-04-24T09:01:12.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.512+0000 7f724d600700 1 -- 172.21.15.69:0/2650380760 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c00f6c0 con 0x7f7250101eb0 2024-04-24T09:01:12.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.512+0000 7f7255e00700 1 -- 172.21.15.69:0/2650380760 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72501a31a0 con 0x7f7250101eb0 2024-04-24T09:01:12.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.514+0000 7f7255e00700 1 -- 172.21.15.69:0/2650380760 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72500697e0 con 0x7f7250101eb0 2024-04-24T09:01:12.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.516+0000 7f724d600700 1 -- 172.21.15.69:0/2650380760 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f723c02a070 con 0x7f7250101eb0 2024-04-24T09:01:12.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.517+0000 7f724d600700 1 --2- 172.21.15.69:0/2650380760 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f72380a3cc0 0x7f72380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:12.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.517+0000 7f724d600700 1 -- 172.21.15.69:0/2650380760 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f723c0d08d0 con 0x7f7250101eb0 2024-04-24T09:01:12.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.517+0000 7f7254a00700 1 --2- 172.21.15.69:0/2650380760 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f72380a3cc0 0x7f72380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:12.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.518+0000 7f7254a00700 1 --2- 172.21.15.69:0/2650380760 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f72380a3cc0 0x7f72380a6180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7244005950 tx=0x7f72440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:12.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.521+0000 7f724d600700 1 -- 172.21.15.69:0/2650380760 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f723c05eff0 con 0x7f7250101eb0 2024-04-24T09:01:12.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:12.759+0000 7f7255e00700 1 -- 172.21.15.69:0/2650380760 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f72500fe490 con 0x7f72380a3cc0 2024-04-24T09:01:13.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:13 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:01:13.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:13 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:01:13.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:13 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:13.340 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:13 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:01:13.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:13 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:01:13.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:13 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:13.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:13 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:01:13.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:13 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:01:13.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:13 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:14.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:14 smithi081 ceph-mon[30961]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:01:14.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:14 smithi081 ceph-mon[30961]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T09:01:14.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:14 smithi069 ceph-mon[28170]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:01:14.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:14 smithi069 ceph-mon[28170]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T09:01:14.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:14 smithi130 ceph-mon[31553]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:01:14.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:14 smithi130 ceph-mon[31553]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T09:01:16.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:16 smithi130 ceph-mon[31553]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-24T09:01:16.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:16 smithi081 ceph-mon[30961]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-24T09:01:16.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:16 smithi069 ceph-mon[28170]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-24T09:01:17.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:17 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1528003202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9155e0e5-52f4-48d3-b024-cf346de31211"}]: dispatch 2024-04-24T09:01:17.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:17 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1528003202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9155e0e5-52f4-48d3-b024-cf346de31211"}]': finished 2024-04-24T09:01:17.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:17 smithi130 ceph-mon[31553]: osdmap e64: 10 total, 9 up, 10 in 2024-04-24T09:01:17.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:17 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1528003202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9155e0e5-52f4-48d3-b024-cf346de31211"}]: dispatch 2024-04-24T09:01:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:17 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1528003202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9155e0e5-52f4-48d3-b024-cf346de31211"}]': finished 2024-04-24T09:01:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:17 smithi081 ceph-mon[30961]: osdmap e64: 10 total, 9 up, 10 in 2024-04-24T09:01:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:17.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:17 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1528003202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9155e0e5-52f4-48d3-b024-cf346de31211"}]: dispatch 2024-04-24T09:01:17.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:17 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1528003202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9155e0e5-52f4-48d3-b024-cf346de31211"}]': finished 2024-04-24T09:01:17.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:17 smithi069 ceph-mon[28170]: osdmap e64: 10 total, 9 up, 10 in 2024-04-24T09:01:17.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:18.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:18 smithi130 ceph-mon[31553]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T09:01:18.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:18 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1560363127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:01:18.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:18 smithi081 ceph-mon[30961]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T09:01:18.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:18 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1560363127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:01:18.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:18 smithi069 ceph-mon[28170]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T09:01:18.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:18 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1560363127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:01:20.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:20 smithi130 ceph-mon[31553]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-24T09:01:20.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:20 smithi081 ceph-mon[30961]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-24T09:01:20.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:20 smithi069 ceph-mon[28170]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-24T09:01:22.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:22 smithi130 ceph-mon[31553]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:22.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:22 smithi081 ceph-mon[30961]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:22.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:22 smithi069 ceph-mon[28170]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:24.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:24 smithi130 ceph-mon[31553]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:24.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:24 smithi081 ceph-mon[30961]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:24.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:24 smithi069 ceph-mon[28170]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:26.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:26 smithi130 ceph-mon[31553]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:26.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:26 smithi081 ceph-mon[30961]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:26.840 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:26 smithi069 ceph-mon[28170]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:27.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:27 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T09:01:27.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:27 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:27.813 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:27 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T09:01:27.813 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:27 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:27.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:27 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T09:01:27.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:27 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:28.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:28 smithi130 ceph-mon[31553]: Deploying daemon osd.9 on smithi069 2024-04-24T09:01:28.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:28 smithi130 ceph-mon[31553]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:28.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:28 smithi081 ceph-mon[30961]: Deploying daemon osd.9 on smithi069 2024-04-24T09:01:28.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:28 smithi081 ceph-mon[30961]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:28.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:28 smithi069 ceph-mon[28170]: Deploying daemon osd.9 on smithi069 2024-04-24T09:01:28.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:28 smithi069 ceph-mon[28170]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:30.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:30 smithi130 ceph-mon[31553]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:30.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:30 smithi081 ceph-mon[30961]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:30.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:30 smithi069 ceph-mon[28170]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:32.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:32 smithi130 ceph-mon[31553]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:32.546 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:32 smithi069 ceph-mon[28170]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:32.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:32 smithi081 ceph-mon[30961]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:34.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:34 smithi130 ceph-mon[31553]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:34.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:34 smithi081 ceph-mon[30961]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:34.828 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:34 smithi069 ceph-mon[28170]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:36.698 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:36 smithi069 ceph-mon[28170]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:36.698 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:36 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:36.698 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:36 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:01:36.698 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:36 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:36.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:36 smithi130 ceph-mon[31553]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:36.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:36 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:36.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:36 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:01:36.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:36 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:36.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:36 smithi081 ceph-mon[30961]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:36.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:36 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:36.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:36 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:01:36.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:36 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:38.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:38 smithi130 ceph-mon[31553]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:38.773 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:38 smithi069 ceph-mon[28170]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:38.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:38 smithi081 ceph-mon[30961]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:40.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:40.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:40 smithi069 ceph-mon[28170]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:40.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:40 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:40.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:40 smithi069 ceph-mon[28170]: from='osd.9 [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T09:01:40.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:40.660+0000 7f724d600700 1 -- 172.21.15.69:0/2650380760 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f72500fe490 con 0x7f72380a3cc0 2024-04-24T09:01:40.661 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 9 on host 'smithi069' 2024-04-24T09:01:40.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:40.664+0000 7f7255e00700 1 -- 172.21.15.69:0/2650380760 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f72380a3cc0 msgr2=0x7f72380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:40.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:40.664+0000 7f7255e00700 1 --2- 172.21.15.69:0/2650380760 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f72380a3cc0 0x7f72380a6180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7244005950 tx=0x7f72440058e0 comp rx=0 tx=0).stop 2024-04-24T09:01:40.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:40.664+0000 7f7255e00700 1 -- 172.21.15.69:0/2650380760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7250101eb0 msgr2=0x7f725019b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:40.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:40.664+0000 7f7255e00700 1 --2- 172.21.15.69:0/2650380760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7250101eb0 0x7f725019b390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f723c000c00 tx=0x7f723c005ef0 comp rx=0 tx=0).stop 2024-04-24T09:01:40.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:40.664+0000 7f7255e00700 1 -- 172.21.15.69:0/2650380760 shutdown_connections 2024-04-24T09:01:40.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:40.664+0000 7f7255e00700 1 --2- 172.21.15.69:0/2650380760 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f72380a3cc0 0x7f72380a6180 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:40.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:40.664+0000 7f7255e00700 1 --2- 172.21.15.69:0/2650380760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7250102850 0x7f72501a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:40.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:40.664+0000 7f7255e00700 1 --2- 172.21.15.69:0/2650380760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7250100cb0 0x7f725019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:40.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:40.664+0000 7f7255e00700 1 --2- 172.21.15.69:0/2650380760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7250101eb0 0x7f725019b390 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:40.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:40.664+0000 7f7255e00700 1 -- 172.21.15.69:0/2650380760 >> 172.21.15.69:0/2650380760 conn(0x7f72500fc270 msgr2=0x7f72500fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:40.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:40.664+0000 7f7255e00700 1 -- 172.21.15.69:0/2650380760 shutdown_connections 2024-04-24T09:01:40.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:01:40.664+0000 7f7255e00700 1 -- 172.21.15.69:0/2650380760 wait complete. 2024-04-24T09:01:40.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:40.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:40 smithi130 ceph-mon[31553]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:40.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:40 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:40.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:40 smithi130 ceph-mon[31553]: from='osd.9 [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T09:01:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:40 smithi081 ceph-mon[30961]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:40 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:40 smithi081 ceph-mon[30961]: from='osd.9 [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T09:01:41.519 DEBUG:teuthology.orchestra.run.smithi069:osd.9> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.9.service 2024-04-24T09:01:41.524 INFO:tasks.cephadm:Deploying osd.10 on smithi081 with /dev/vg_nvme/lv_1... 2024-04-24T09:01:41.525 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T09:01:41.611 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:41 smithi081 ceph-mon[30961]: from='osd.9 [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T09:01:41.611 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:41 smithi081 ceph-mon[30961]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T09:01:41.611 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:41 smithi081 ceph-mon[30961]: from='osd.9 [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-24T09:01:41.611 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:41.611 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:41.611 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:01:41.611 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:41.611 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:41.611 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:41.611 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:41 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:01:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:41 smithi130 ceph-mon[31553]: from='osd.9 [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T09:01:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:41 smithi130 ceph-mon[31553]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T09:01:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:41 smithi130 ceph-mon[31553]: from='osd.9 [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-24T09:01:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:01:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:41.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:41.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:41.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:41 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:01:41.822 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:41 smithi069 ceph-mon[28170]: from='osd.9 [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T09:01:41.822 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:41 smithi069 ceph-mon[28170]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T09:01:41.822 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:41 smithi069 ceph-mon[28170]: from='osd.9 [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-24T09:01:41.822 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:41.822 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:41.822 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:01:41.823 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:41.823 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:41.823 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:41.823 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:41 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:01:41.970 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:01:42.406 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:42 smithi081 ceph-mon[30961]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:42.406 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:42 smithi081 ceph-mon[30961]: from='osd.9 [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-24T09:01:42.406 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:42 smithi081 ceph-mon[30961]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T09:01:42.406 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:42.406 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:42.637 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:42 smithi069 ceph-mon[28170]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:42.638 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:42 smithi069 ceph-mon[28170]: from='osd.9 [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-24T09:01:42.638 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:42 smithi069 ceph-mon[28170]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T09:01:42.638 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:42.638 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:42.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:42 smithi130 ceph-mon[31553]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:42.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:42 smithi130 ceph-mon[31553]: from='osd.9 [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-24T09:01:42.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:42 smithi130 ceph-mon[31553]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T09:01:42.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:42.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:43.581 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:43 smithi069 ceph-mon[28170]: purged_snaps scrub starts 2024-04-24T09:01:43.581 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:43 smithi069 ceph-mon[28170]: purged_snaps scrub ok 2024-04-24T09:01:43.581 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:43.581 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:43 smithi069 ceph-mon[28170]: osd.9 [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159] boot 2024-04-24T09:01:43.581 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:43 smithi069 ceph-mon[28170]: osdmap e67: 10 total, 10 up, 10 in 2024-04-24T09:01:43.581 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:43 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:43.689 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:43 smithi081 ceph-mon[30961]: purged_snaps scrub starts 2024-04-24T09:01:43.689 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:43 smithi081 ceph-mon[30961]: purged_snaps scrub ok 2024-04-24T09:01:43.689 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:43 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:43.689 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:43 smithi081 ceph-mon[30961]: osd.9 [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159] boot 2024-04-24T09:01:43.689 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:43 smithi081 ceph-mon[30961]: osdmap e67: 10 total, 10 up, 10 in 2024-04-24T09:01:43.689 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:43 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:43.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:43 smithi130 ceph-mon[31553]: purged_snaps scrub starts 2024-04-24T09:01:43.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:43 smithi130 ceph-mon[31553]: purged_snaps scrub ok 2024-04-24T09:01:43.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:43 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:43.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:43 smithi130 ceph-mon[31553]: osd.9 [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159] boot 2024-04-24T09:01:43.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:43 smithi130 ceph-mon[31553]: osdmap e67: 10 total, 10 up, 10 in 2024-04-24T09:01:43.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:43 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:01:44.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:44 smithi130 ceph-mon[31553]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:44.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:44 smithi130 ceph-mon[31553]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T09:01:44.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:44 smithi081 ceph-mon[30961]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:44.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:44 smithi081 ceph-mon[30961]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T09:01:44.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:44 smithi069 ceph-mon[28170]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:01:44.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:44 smithi069 ceph-mon[28170]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T09:01:44.912 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T09:01:44.912 INFO:teuthology.orchestra.run.smithi081.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T09:01:44.912 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10+0 records in 2024-04-24T09:01:44.912 INFO:teuthology.orchestra.run.smithi081.stderr:10+0 records out 2024-04-24T09:01:44.912 INFO:teuthology.orchestra.run.smithi081.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0359088 s, 292 MB/s 2024-04-24T09:01:44.912 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping successful for: 2024-04-24T09:01:45.872 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi081:vg_nvme/lv_1 2024-04-24T09:01:46.283 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:01:46.694 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:46 smithi081 ceph-mon[30961]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:46.694 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:46.695 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:46.695 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:46.695 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:46.695 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:46.695 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:46.695 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:46.695 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:46.695 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:01:46.695 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:46.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:46 smithi130 ceph-mon[31553]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:46.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:01:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:46.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:46 smithi069 ceph-mon[28170]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:46.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:46.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:46.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:46.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:46.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:46.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:01:46.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:46.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:46.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:01:46.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:01:47.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:47 smithi130 ceph-mon[31553]: Detected new or changed devices on smithi069 2024-04-24T09:01:47.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:47 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi069 to 4272M 2024-04-24T09:01:47.760 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:47 smithi081 ceph-mon[30961]: Detected new or changed devices on smithi069 2024-04-24T09:01:47.760 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:47 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi069 to 4272M 2024-04-24T09:01:47.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:47 smithi069 ceph-mon[28170]: Detected new or changed devices on smithi069 2024-04-24T09:01:47.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:47 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi069 to 4272M 2024-04-24T09:01:48.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.604+0000 7fd611600700 1 -- 172.21.15.81:0/2563260397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd60c102b50 msgr2=0x7fd60c102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:48.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.604+0000 7fd611600700 1 --2- 172.21.15.81:0/2563260397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd60c102b50 0x7fd60c102f70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0099c0 tx=0x7fd5fc009cd0 comp rx=0 tx=0).stop 2024-04-24T09:01:48.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.605+0000 7fd611600700 1 -- 172.21.15.81:0/2563260397 shutdown_connections 2024-04-24T09:01:48.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.605+0000 7fd611600700 1 --2- 172.21.15.81:0/2563260397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd60c1046f0 0x7fd60c10afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:48.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.605+0000 7fd611600700 1 --2- 172.21.15.81:0/2563260397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd60c102b50 0x7fd60c102f70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:48.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.605+0000 7fd611600700 1 --2- 172.21.15.81:0/2563260397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd60c103d50 0x7fd60c1041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:48.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.605+0000 7fd611600700 1 -- 172.21.15.81:0/2563260397 >> 172.21.15.81:0/2563260397 conn(0x7fd60c0fe0f0 msgr2=0x7fd60c100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:48.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.605+0000 7fd611600700 1 -- 172.21.15.81:0/2563260397 shutdown_connections 2024-04-24T09:01:48.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.605+0000 7fd611600700 1 -- 172.21.15.81:0/2563260397 wait complete. 2024-04-24T09:01:48.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.606+0000 7fd611600700 1 Processor -- start 2024-04-24T09:01:48.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.606+0000 7fd611600700 1 -- start start 2024-04-24T09:01:48.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.607+0000 7fd611600700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd60c102b50 0x7fd60c198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:48.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.607+0000 7fd611600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd60c103d50 0x7fd60c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:48.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.607+0000 7fd611600700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd60c1046f0 0x7fd60c1a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:48.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.607+0000 7fd611600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c10e5e0 con 0x7fd60c103d50 2024-04-24T09:01:48.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.607+0000 7fd611600700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c10e460 con 0x7fd60c102b50 2024-04-24T09:01:48.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.607+0000 7fd611600700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c10e760 con 0x7fd60c1046f0 2024-04-24T09:01:48.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.607+0000 7fd60aa00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd60c102b50 0x7fd60c198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:48.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.607+0000 7fd60aa00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd60c102b50 0x7fd60c198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40662/0 (socket says 172.21.15.81:40662) 2024-04-24T09:01:48.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.607+0000 7fd60aa00700 1 -- 172.21.15.81:0/3191593578 learned_addr learned my addr 172.21.15.81:0/3191593578 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-04-24T09:01:48.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.607+0000 7fd60b400700 1 --2- 172.21.15.81:0/3191593578 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd60c1046f0 0x7fd60c1a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.607+0000 7fd60aa00700 1 -- 172.21.15.81:0/3191593578 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd60c1046f0 msgr2=0x7fd60c1a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.608+0000 7fd60aa00700 1 --2- 172.21.15.81:0/3191593578 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd60c1046f0 0x7fd60c1a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.608+0000 7fd60aa00700 1 -- 172.21.15.81:0/3191593578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd60c103d50 msgr2=0x7fd60c199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.608+0000 7fd60aa00700 1 --2- 172.21.15.81:0/3191593578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd60c103d50 0x7fd60c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.608+0000 7fd60aa00700 1 -- 172.21.15.81:0/3191593578 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c1a0940 con 0x7fd60c102b50 2024-04-24T09:01:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.608+0000 7fd60aa00700 1 --2- 172.21.15.81:0/3191593578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd60c102b50 0x7fd60c198cf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0036f0 tx=0x7fd5fc00f7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.608+0000 7fd603400700 1 -- 172.21.15.81:0/3191593578 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc009e60 con 0x7fd60c102b50 2024-04-24T09:01:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.608+0000 7fd603400700 1 -- 172.21.15.81:0/3191593578 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5fc005940 con 0x7fd60c102b50 2024-04-24T09:01:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.608+0000 7fd611600700 1 -- 172.21.15.81:0/3191593578 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60c1a0bd0 con 0x7fd60c102b50 2024-04-24T09:01:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.608+0000 7fd603400700 1 -- 172.21.15.81:0/3191593578 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc005620 con 0x7fd60c102b50 2024-04-24T09:01:48.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.608+0000 7fd611600700 1 -- 172.21.15.81:0/3191593578 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60c1a10b0 con 0x7fd60c102b50 2024-04-24T09:01:48.610 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.610+0000 7fd611600700 1 -- 172.21.15.81:0/3191593578 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd60c0697e0 con 0x7fd60c102b50 2024-04-24T09:01:48.611 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.612+0000 7fd603400700 1 -- 172.21.15.81:0/3191593578 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd5fc0038c0 con 0x7fd60c102b50 2024-04-24T09:01:48.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.614+0000 7fd603400700 1 --2- 172.21.15.81:0/3191593578 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd5f00a3c40 0x7fd5f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:48.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.614+0000 7fd603400700 1 -- 172.21.15.81:0/3191593578 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7fd5fc0d0c40 con 0x7fd60c102b50 2024-04-24T09:01:48.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.615+0000 7fd60a000700 1 --2- 172.21.15.81:0/3191593578 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd5f00a3c40 0x7fd5f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:48.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.616+0000 7fd60a000700 1 --2- 172.21.15.81:0/3191593578 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd5f00a3c40 0x7fd5f00a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8005950 tx=0x7fd5f80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:48.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.617+0000 7fd603400700 1 -- 172.21.15.81:0/3191593578 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd5fc013070 con 0x7fd60c102b50 2024-04-24T09:01:48.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:48 smithi130 ceph-mon[31553]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:48.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:48 smithi081 ceph-mon[30961]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:48.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:48 smithi069 ceph-mon[28170]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:48.853 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:01:48.853+0000 7fd611600700 1 -- 172.21.15.81:0/3191593578 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd60c062f30 con 0x7fd5f00a3c40 2024-04-24T09:01:49.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:01:49.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:01:49.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:49.808 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:01:49.808 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:01:49.808 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:49.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:01:49.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:01:49.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:50.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:50 smithi130 ceph-mon[31553]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:01:50.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:50 smithi130 ceph-mon[31553]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:50.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:50 smithi081 ceph-mon[30961]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:01:50.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:50 smithi081 ceph-mon[30961]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:50.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:50 smithi069 ceph-mon[28170]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:01:50.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:50 smithi069 ceph-mon[28170]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:52.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:52 smithi130 ceph-mon[31553]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:52.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:52 smithi081 ceph-mon[30961]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:52.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:52 smithi069 ceph-mon[28170]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:53.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:53 smithi130 ceph-mon[31553]: from='client.? 172.21.15.81:0/2607095151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2f8028d-7966-4ceb-9e12-a728d89665ee"}]: dispatch 2024-04-24T09:01:53.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:53 smithi130 ceph-mon[31553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2f8028d-7966-4ceb-9e12-a728d89665ee"}]: dispatch 2024-04-24T09:01:53.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:53 smithi130 ceph-mon[31553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2f8028d-7966-4ceb-9e12-a728d89665ee"}]': finished 2024-04-24T09:01:53.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:53 smithi130 ceph-mon[31553]: osdmap e69: 11 total, 10 up, 11 in 2024-04-24T09:01:53.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:01:53.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:53 smithi081 ceph-mon[30961]: from='client.? 172.21.15.81:0/2607095151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2f8028d-7966-4ceb-9e12-a728d89665ee"}]: dispatch 2024-04-24T09:01:53.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:53 smithi081 ceph-mon[30961]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2f8028d-7966-4ceb-9e12-a728d89665ee"}]: dispatch 2024-04-24T09:01:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:53 smithi081 ceph-mon[30961]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2f8028d-7966-4ceb-9e12-a728d89665ee"}]': finished 2024-04-24T09:01:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:53 smithi081 ceph-mon[30961]: osdmap e69: 11 total, 10 up, 11 in 2024-04-24T09:01:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:01:53.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:53 smithi069 ceph-mon[28170]: from='client.? 172.21.15.81:0/2607095151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2f8028d-7966-4ceb-9e12-a728d89665ee"}]: dispatch 2024-04-24T09:01:53.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:53 smithi069 ceph-mon[28170]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2f8028d-7966-4ceb-9e12-a728d89665ee"}]: dispatch 2024-04-24T09:01:53.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:53 smithi069 ceph-mon[28170]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2f8028d-7966-4ceb-9e12-a728d89665ee"}]': finished 2024-04-24T09:01:53.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:53 smithi069 ceph-mon[28170]: osdmap e69: 11 total, 10 up, 11 in 2024-04-24T09:01:53.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:01:54.541 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:54 smithi069 ceph-mon[28170]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:54.542 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:54 smithi069 ceph-mon[28170]: from='client.? 172.21.15.81:0/2330845015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:01:54.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:54 smithi130 ceph-mon[31553]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:54.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:54 smithi130 ceph-mon[31553]: from='client.? 172.21.15.81:0/2330845015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:01:54.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:54 smithi081 ceph-mon[30961]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:54.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:54 smithi081 ceph-mon[30961]: from='client.? 172.21.15.81:0/2330845015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:01:56.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:56 smithi130 ceph-mon[31553]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:56.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:56 smithi081 ceph-mon[30961]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:56.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:56 smithi069 ceph-mon[28170]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:58.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:01:58 smithi130 ceph-mon[31553]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:58.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:01:58 smithi081 ceph-mon[30961]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:01:58.840 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:01:58 smithi069 ceph-mon[28170]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:00.717 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:00 smithi081 ceph-mon[30961]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:00.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:00 smithi069 ceph-mon[28170]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:00.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:00 smithi130 ceph-mon[31553]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:02.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:02 smithi081 ceph-mon[30961]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:02.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:02 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T09:02:02.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:02 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:02 smithi069 ceph-mon[28170]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T09:02:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:02.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:02 smithi130 ceph-mon[31553]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:02.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:02 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T09:02:02.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:02 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:03.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:03 smithi081 ceph-mon[30961]: Deploying daemon osd.10 on smithi081 2024-04-24T09:02:03.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:03 smithi069 ceph-mon[28170]: Deploying daemon osd.10 on smithi081 2024-04-24T09:02:03.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:03 smithi130 ceph-mon[31553]: Deploying daemon osd.10 on smithi081 2024-04-24T09:02:04.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:04 smithi081 ceph-mon[30961]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:04.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:04 smithi069 ceph-mon[28170]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:04.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:04 smithi130 ceph-mon[31553]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:06.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:06 smithi081 ceph-mon[30961]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:06.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:06 smithi069 ceph-mon[28170]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:06.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:06 smithi130 ceph-mon[31553]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:08.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:08 smithi081 ceph-mon[30961]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:08 smithi069 ceph-mon[28170]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:08.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:08 smithi130 ceph-mon[31553]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:10.806 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:10 smithi081 ceph-mon[30961]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:10.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:10 smithi069 ceph-mon[28170]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:10.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:10 smithi130 ceph-mon[31553]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:12.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:12 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:12.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:12 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:12.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:12 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:02:12.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:12 smithi081 ceph-mon[30961]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:12.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:12 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:12.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:12 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:12.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:12 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:02:12.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:12 smithi069 ceph-mon[28170]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:12.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:12 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:12.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:12 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:12.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:12 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:02:12.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:12 smithi130 ceph-mon[31553]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:14.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:14 smithi130 ceph-mon[31553]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:14.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:14 smithi081 ceph-mon[30961]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:14.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:14 smithi069 ceph-mon[28170]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:15.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:02:15.763+0000 7fd603400700 1 -- 172.21.15.81:0/3191593578 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd60c062f30 con 0x7fd5f00a3c40 2024-04-24T09:02:15.764 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 10 on host 'smithi081' 2024-04-24T09:02:15.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:02:15.767+0000 7fd611600700 1 -- 172.21.15.81:0/3191593578 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd5f00a3c40 msgr2=0x7fd5f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:15.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:02:15.767+0000 7fd611600700 1 --2- 172.21.15.81:0/3191593578 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd5f00a3c40 0x7fd5f00a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8005950 tx=0x7fd5f80058e0 comp rx=0 tx=0).stop 2024-04-24T09:02:15.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:02:15.767+0000 7fd611600700 1 -- 172.21.15.81:0/3191593578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd60c102b50 msgr2=0x7fd60c198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:15.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:02:15.767+0000 7fd611600700 1 --2- 172.21.15.81:0/3191593578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd60c102b50 0x7fd60c198cf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0036f0 tx=0x7fd5fc00f7c0 comp rx=0 tx=0).stop 2024-04-24T09:02:15.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:02:15.767+0000 7fd611600700 1 -- 172.21.15.81:0/3191593578 shutdown_connections 2024-04-24T09:02:15.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:02:15.767+0000 7fd611600700 1 --2- 172.21.15.81:0/3191593578 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd5f00a3c40 0x7fd5f00a6100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:15.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:02:15.767+0000 7fd611600700 1 --2- 172.21.15.81:0/3191593578 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd60c1046f0 0x7fd60c1a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:15.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:02:15.767+0000 7fd611600700 1 --2- 172.21.15.81:0/3191593578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd60c102b50 0x7fd60c198cf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:15.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:02:15.767+0000 7fd611600700 1 --2- 172.21.15.81:0/3191593578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd60c103d50 0x7fd60c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:15.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:02:15.767+0000 7fd611600700 1 -- 172.21.15.81:0/3191593578 >> 172.21.15.81:0/3191593578 conn(0x7fd60c0fe0f0 msgr2=0x7fd60c107170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:15.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:02:15.768+0000 7fd611600700 1 -- 172.21.15.81:0/3191593578 shutdown_connections 2024-04-24T09:02:15.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-04-24T09:02:15.768+0000 7fd611600700 1 -- 172.21.15.81:0/3191593578 wait complete. 2024-04-24T09:02:16.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:15 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:16.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:15 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:16.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:15 smithi081 ceph-mon[30961]: from='osd.10 [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T09:02:16.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:15 smithi081 ceph-mon[30961]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T09:02:16.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:15 smithi081 ceph-mon[30961]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:16.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:15 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:16.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:15 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:16.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:15 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:16.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:15 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:16.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:15 smithi069 ceph-mon[28170]: from='osd.10 [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T09:02:16.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:15 smithi069 ceph-mon[28170]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T09:02:16.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:15 smithi069 ceph-mon[28170]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:16.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:15 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:16.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:15 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:16.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:15 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:16.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:15 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:16.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:15 smithi130 ceph-mon[31553]: from='osd.10 [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T09:02:16.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:15 smithi130 ceph-mon[31553]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T09:02:16.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:15 smithi130 ceph-mon[31553]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:16.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:15 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:16.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:15 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:16.590 DEBUG:teuthology.orchestra.run.smithi081:osd.10> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.10.service 2024-04-24T09:02:16.596 INFO:tasks.cephadm:Deploying osd.11 on smithi130 with /dev/vg_nvme/lv_1... 2024-04-24T09:02:16.596 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T09:02:16.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:16 smithi130 ceph-mon[31553]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T09:02:16.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:16 smithi130 ceph-mon[31553]: from='osd.10 [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T09:02:16.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:16 smithi130 ceph-mon[31553]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T09:02:16.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:16 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:16.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:16 smithi130 ceph-mon[31553]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T09:02:17.034 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:02:17.066 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:16 smithi081 ceph-mon[30961]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T09:02:17.066 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:16 smithi081 ceph-mon[30961]: from='osd.10 [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T09:02:17.066 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:16 smithi081 ceph-mon[30961]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T09:02:17.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:16 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:17.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:16 smithi081 ceph-mon[30961]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T09:02:17.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:16 smithi069 ceph-mon[28170]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T09:02:17.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:16 smithi069 ceph-mon[28170]: from='osd.10 [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T09:02:17.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:16 smithi069 ceph-mon[28170]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T09:02:17.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:16 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:17.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:16 smithi069 ceph-mon[28170]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-04-24T09:02:17.816 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:02:17 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:02:17.674+0000 7f012a200700 -1 osd.10 0 waiting for initial osdmap 2024-04-24T09:02:17.816 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:02:17 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:02:17.701+0000 7f0120000700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T09:02:18.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:17 smithi081 ceph-mon[30961]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-04-24T09:02:18.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:17 smithi081 ceph-mon[30961]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T09:02:18.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:18.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:18.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:17 smithi081 ceph-mon[30961]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:18.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:17 smithi081 ceph-mon[30961]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T09:02:18.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:18.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:17 smithi081 ceph-mon[30961]: from='osd.10 ' entity='osd.10' 2024-04-24T09:02:18.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:17 smithi069 ceph-mon[28170]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-04-24T09:02:18.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:17 smithi069 ceph-mon[28170]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T09:02:18.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:18.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:18.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:17 smithi069 ceph-mon[28170]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:18.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:17 smithi069 ceph-mon[28170]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T09:02:18.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:18.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:17 smithi069 ceph-mon[28170]: from='osd.10 ' entity='osd.10' 2024-04-24T09:02:18.126 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:17 smithi130 ceph-mon[31553]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-04-24T09:02:18.126 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:17 smithi130 ceph-mon[31553]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T09:02:18.126 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:18.126 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:18.126 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:17 smithi130 ceph-mon[31553]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:02:18.126 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:17 smithi130 ceph-mon[31553]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T09:02:18.126 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:18.126 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:17 smithi130 ceph-mon[31553]: from='osd.10 ' entity='osd.10' 2024-04-24T09:02:18.903 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:18 smithi130 ceph-mon[31553]: purged_snaps scrub starts 2024-04-24T09:02:18.904 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:18 smithi130 ceph-mon[31553]: purged_snaps scrub ok 2024-04-24T09:02:18.904 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:18 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:18.904 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:18 smithi130 ceph-mon[31553]: osd.10 [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549] boot 2024-04-24T09:02:18.904 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:18 smithi130 ceph-mon[31553]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T09:02:18.904 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:18 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:18.939 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:18 smithi081 ceph-mon[30961]: purged_snaps scrub starts 2024-04-24T09:02:18.940 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:18 smithi081 ceph-mon[30961]: purged_snaps scrub ok 2024-04-24T09:02:18.940 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:18 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:18.940 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:18 smithi081 ceph-mon[30961]: osd.10 [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549] boot 2024-04-24T09:02:18.940 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:18 smithi081 ceph-mon[30961]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T09:02:18.940 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:18 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:19.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:18 smithi069 ceph-mon[28170]: purged_snaps scrub starts 2024-04-24T09:02:19.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:18 smithi069 ceph-mon[28170]: purged_snaps scrub ok 2024-04-24T09:02:19.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:18 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:19.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:18 smithi069 ceph-mon[28170]: osd.10 [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549] boot 2024-04-24T09:02:19.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:18 smithi069 ceph-mon[28170]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T09:02:19.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:18 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:02:20.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:19 smithi069 ceph-mon[28170]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:20.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:19 smithi069 ceph-mon[28170]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T09:02:20.135 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:19 smithi130 ceph-mon[31553]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:20.135 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:19 smithi130 ceph-mon[31553]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T09:02:20.136 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T09:02:20.136 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T09:02:20.136 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-04-24T09:02:20.136 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-04-24T09:02:20.136 INFO:teuthology.orchestra.run.smithi130.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0347956 s, 301 MB/s 2024-04-24T09:02:20.136 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-04-24T09:02:20.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:19 smithi081 ceph-mon[30961]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:20.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:19 smithi081 ceph-mon[30961]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T09:02:21.131 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_1 2024-04-24T09:02:21.572 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:02:21.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:21 smithi081 ceph-mon[30961]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T09:02:21.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:21 smithi081 ceph-mon[30961]: pgmap v304: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:21.927 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:21 smithi130 ceph-mon[31553]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T09:02:21.927 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:21 smithi130 ceph-mon[31553]: pgmap v304: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:22.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:21 smithi069 ceph-mon[28170]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T09:02:22.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:21 smithi069 ceph-mon[28170]: pgmap v304: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:22.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: Detected new or changed devices on smithi081 2024-04-24T09:02:22.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T09:02:22.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:22.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:22.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:22.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:22.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:22.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:22.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi081 to 4272M 2024-04-24T09:02:22.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:22.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:22.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:02:22.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:22.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:02:22.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:22.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:02:22.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:22 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:22.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: Detected new or changed devices on smithi081 2024-04-24T09:02:22.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T09:02:22.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:22.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:22.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:22.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:22.994 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:22.994 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:22.994 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi081 to 4272M 2024-04-24T09:02:22.994 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:22.994 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:22.994 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:02:22.994 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:22.994 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:02:22.994 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:22.994 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:02:22.994 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:22 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:23.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: Detected new or changed devices on smithi081 2024-04-24T09:02:23.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T09:02:23.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:23.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:23.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:23.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:23.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:23.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:23.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi081 to 4272M 2024-04-24T09:02:23.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:23.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:23.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:02:23.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:23.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:02:23.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:23.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:02:23.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:22 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:23.693 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:23 smithi130 ceph-mon[31553]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T09:02:23.693 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:23 smithi130 ceph-mon[31553]: pgmap v307: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:24.002 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.002+0000 7fef4de00700 1 -- 172.21.15.130:0/2103232658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef48108840 msgr2=0x7fef4810acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.002 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.002+0000 7fef4de00700 1 --2- 172.21.15.130:0/2103232658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef48108840 0x7fef4810acb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fef4000b0a0 tx=0x7fef4000b3b0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.002 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.002+0000 7fef4de00700 1 -- 172.21.15.130:0/2103232658 shutdown_connections 2024-04-24T09:02:24.002 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.002+0000 7fef4de00700 1 --2- 172.21.15.130:0/2103232658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef48108840 0x7fef4810acb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.002 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.002+0000 7fef4de00700 1 --2- 172.21.15.130:0/2103232658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef481035e0 0x7fef481059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.002 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.002+0000 7fef4de00700 1 --2- 172.21.15.130:0/2103232658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef48105f10 0x7fef48108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.002 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.002+0000 7fef4de00700 1 -- 172.21.15.130:0/2103232658 >> 172.21.15.130:0/2103232658 conn(0x7fef480fd010 msgr2=0x7fef480ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:24.003 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.002+0000 7fef4de00700 1 -- 172.21.15.130:0/2103232658 shutdown_connections 2024-04-24T09:02:24.003 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.002+0000 7fef4de00700 1 -- 172.21.15.130:0/2103232658 wait complete. 2024-04-24T09:02:24.004 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.003+0000 7fef4de00700 1 Processor -- start 2024-04-24T09:02:24.004 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.003+0000 7fef4de00700 1 -- start start 2024-04-24T09:02:24.004 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.004+0000 7fef4de00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef481035e0 0x7fef4819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.004 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.004+0000 7fef4de00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef48105f10 0x7fef4819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.004 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.004+0000 7fef4de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef48108840 0x7fef481a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.005 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.004+0000 7fef4de00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fef4810e600 con 0x7fef48108840 2024-04-24T09:02:24.005 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.004+0000 7fef4de00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fef4810e480 con 0x7fef481035e0 2024-04-24T09:02:24.005 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.004+0000 7fef4de00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fef4810e780 con 0x7fef48105f10 2024-04-24T09:02:24.005 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.004+0000 7fef46000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef48105f10 0x7fef4819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.005 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.004+0000 7fef46000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef48105f10 0x7fef4819b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:38648/0 (socket says 172.21.15.130:38648) 2024-04-24T09:02:24.005 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.004+0000 7fef46000700 1 -- 172.21.15.130:0/4005328224 learned_addr learned my addr 172.21.15.130:0/4005328224 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-24T09:02:24.005 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.004+0000 7fef46a00700 1 --2- 172.21.15.130:0/4005328224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef481035e0 0x7fef4819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.005 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.004+0000 7fef47400700 1 --2- 172.21.15.130:0/4005328224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef48108840 0x7fef481a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.005 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.005+0000 7fef46000700 1 -- 172.21.15.130:0/4005328224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef481035e0 msgr2=0x7fef4819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.005 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.005+0000 7fef46000700 1 --2- 172.21.15.130:0/4005328224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef481035e0 0x7fef4819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.005 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.005+0000 7fef46000700 1 -- 172.21.15.130:0/4005328224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef48108840 msgr2=0x7fef481a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.005 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.005+0000 7fef46000700 1 --2- 172.21.15.130:0/4005328224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef48108840 0x7fef481a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.005 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.005+0000 7fef46000700 1 -- 172.21.15.130:0/4005328224 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef481a29c0 con 0x7fef48105f10 2024-04-24T09:02:24.006 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.005+0000 7fef47400700 1 --2- 172.21.15.130:0/4005328224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef48108840 0x7fef481a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:24.006 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.005+0000 7fef46000700 1 --2- 172.21.15.130:0/4005328224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef48105f10 0x7fef4819b2a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fef3400c9e0 tx=0x7fef3400cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:24.006 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.005+0000 7fef4ca00700 1 -- 172.21.15.130:0/4005328224 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34007ac0 con 0x7fef48105f10 2024-04-24T09:02:24.006 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.005+0000 7fef4ca00700 1 -- 172.21.15.130:0/4005328224 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef34007c60 con 0x7fef48105f10 2024-04-24T09:02:24.007 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.005+0000 7fef4de00700 1 -- 172.21.15.130:0/4005328224 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef481a2cb0 con 0x7fef48105f10 2024-04-24T09:02:24.007 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.005+0000 7fef4ca00700 1 -- 172.21.15.130:0/4005328224 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34018660 con 0x7fef48105f10 2024-04-24T09:02:24.007 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.005+0000 7fef4de00700 1 -- 172.21.15.130:0/4005328224 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef481a31a0 con 0x7fef48105f10 2024-04-24T09:02:24.007 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.006+0000 7fef4de00700 1 -- 172.21.15.130:0/4005328224 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef480697e0 con 0x7fef48105f10 2024-04-24T09:02:24.008 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.009+0000 7fef4ca00700 1 -- 172.21.15.130:0/4005328224 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fef3400f450 con 0x7fef48105f10 2024-04-24T09:02:24.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.010+0000 7fef4ca00700 1 --2- 172.21.15.130:0/4005328224 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fef2c0a3c40 0x7fef2c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.010+0000 7fef4ca00700 1 -- 172.21.15.130:0/4005328224 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6694+0+0 (secure 0 0 0) 0x7fef340cfe10 con 0x7fef48105f10 2024-04-24T09:02:24.010 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.010+0000 7fef46a00700 1 --2- 172.21.15.130:0/4005328224 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fef2c0a3c40 0x7fef2c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.011 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.011+0000 7fef46a00700 1 --2- 172.21.15.130:0/4005328224 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fef2c0a3c40 0x7fef2c0a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fef38005ea0 tx=0x7fef38005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:24.015 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.015+0000 7fef4ca00700 1 -- 172.21.15.130:0/4005328224 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef340d3030 con 0x7fef48105f10 2024-04-24T09:02:24.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:23 smithi081 ceph-mon[30961]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T09:02:24.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:23 smithi081 ceph-mon[30961]: pgmap v307: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:24.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:23 smithi069 ceph-mon[28170]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T09:02:24.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:23 smithi069 ceph-mon[28170]: pgmap v307: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:24.248 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:24.249+0000 7fef4de00700 1 -- 172.21.15.130:0/4005328224 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fef48062f30 con 0x7fef2c0a3c40 2024-04-24T09:02:24.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:24 smithi130 ceph-mon[31553]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:02:24.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:02:24.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:02:24.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:25.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:24 smithi081 ceph-mon[30961]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:02:25.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:02:25.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:02:25.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:25.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:24 smithi069 ceph-mon[28170]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:02:25.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:02:25.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:02:25.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:25.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:25 smithi130 ceph-mon[31553]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-24T09:02:26.066 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:25 smithi081 ceph-mon[30961]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-24T09:02:26.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:25 smithi069 ceph-mon[28170]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-24T09:02:28.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:28 smithi130 ceph-mon[31553]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-24T09:02:28.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:28 smithi081 ceph-mon[30961]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-24T09:02:28.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:28 smithi069 ceph-mon[28170]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-24T09:02:29.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:29 smithi130 ceph-mon[31553]: from='client.? 172.21.15.130:0/2758206856' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebe21375-9751-4541-9dbf-0d5e0c4bbd5b"}]: dispatch 2024-04-24T09:02:29.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:29 smithi130 ceph-mon[31553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebe21375-9751-4541-9dbf-0d5e0c4bbd5b"}]: dispatch 2024-04-24T09:02:29.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:29 smithi130 ceph-mon[31553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebe21375-9751-4541-9dbf-0d5e0c4bbd5b"}]': finished 2024-04-24T09:02:29.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:29 smithi130 ceph-mon[31553]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T09:02:29.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:29 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:29.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:29 smithi081 ceph-mon[30961]: from='client.? 172.21.15.130:0/2758206856' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebe21375-9751-4541-9dbf-0d5e0c4bbd5b"}]: dispatch 2024-04-24T09:02:29.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:29 smithi081 ceph-mon[30961]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebe21375-9751-4541-9dbf-0d5e0c4bbd5b"}]: dispatch 2024-04-24T09:02:29.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:29 smithi081 ceph-mon[30961]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebe21375-9751-4541-9dbf-0d5e0c4bbd5b"}]': finished 2024-04-24T09:02:29.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:29 smithi081 ceph-mon[30961]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T09:02:29.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:29 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:29.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:29 smithi069 ceph-mon[28170]: from='client.? 172.21.15.130:0/2758206856' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebe21375-9751-4541-9dbf-0d5e0c4bbd5b"}]: dispatch 2024-04-24T09:02:29.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:29 smithi069 ceph-mon[28170]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebe21375-9751-4541-9dbf-0d5e0c4bbd5b"}]: dispatch 2024-04-24T09:02:29.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:29 smithi069 ceph-mon[28170]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebe21375-9751-4541-9dbf-0d5e0c4bbd5b"}]': finished 2024-04-24T09:02:29.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:29 smithi069 ceph-mon[28170]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T09:02:29.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:29 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:30.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:30 smithi130 ceph-mon[31553]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-24T09:02:30.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:30 smithi130 ceph-mon[31553]: from='client.? 172.21.15.130:0/1594850436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:02:30.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:30 smithi081 ceph-mon[30961]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-24T09:02:30.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:30 smithi081 ceph-mon[30961]: from='client.? 172.21.15.130:0/1594850436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:02:30.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:30 smithi069 ceph-mon[28170]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-24T09:02:30.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:30 smithi069 ceph-mon[28170]: from='client.? 172.21.15.130:0/1594850436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:02:32.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:32 smithi130 ceph-mon[31553]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-24T09:02:32.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:32 smithi081 ceph-mon[30961]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-24T09:02:32.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:32 smithi069 ceph-mon[28170]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-24T09:02:34.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:34 smithi130 ceph-mon[31553]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T09:02:34.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:34 smithi081 ceph-mon[30961]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T09:02:34.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:34 smithi069 ceph-mon[28170]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T09:02:36.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:36 smithi130 ceph-mon[31553]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:36.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:36 smithi081 ceph-mon[30961]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:36.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:36 smithi069 ceph-mon[28170]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:38.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:38 smithi130 ceph-mon[31553]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:38.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:38 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T09:02:38.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:38 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:38.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:38 smithi081 ceph-mon[30961]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:38.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:38 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T09:02:38.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:38 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:38.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:38 smithi069 ceph-mon[28170]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:38.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T09:02:38.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:39.625 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:39 smithi130 ceph-mon[31553]: Deploying daemon osd.11 on smithi130 2024-04-24T09:02:39.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:39 smithi081 ceph-mon[30961]: Deploying daemon osd.11 on smithi130 2024-04-24T09:02:39.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:39 smithi069 ceph-mon[28170]: Deploying daemon osd.11 on smithi130 2024-04-24T09:02:40.686 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:40 smithi130 ceph-mon[31553]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:40 smithi081 ceph-mon[30961]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:40 smithi069 ceph-mon[28170]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:42.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:42 smithi130 ceph-mon[31553]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:42.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:42 smithi081 ceph-mon[30961]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:42.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:42 smithi069 ceph-mon[28170]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:44.544 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:44 smithi130 ceph-mon[31553]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:44.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:44 smithi081 ceph-mon[30961]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:44.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:44 smithi069 ceph-mon[28170]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:46.678 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:46 smithi130 ceph-mon[31553]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:46.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:46 smithi081 ceph-mon[30961]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:46.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:46 smithi069 ceph-mon[28170]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:48.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:48 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:48.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:48 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:48.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:48 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:02:48.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:48 smithi081 ceph-mon[30961]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:48.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:48 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:48.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:48 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:48.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:48 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:02:48.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:48 smithi069 ceph-mon[28170]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:48.596 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:48 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:48.596 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:48 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:48.596 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:48 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:02:48.596 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:48 smithi130 ceph-mon[31553]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:50.537 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:50 smithi130 ceph-mon[31553]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:50.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:50 smithi081 ceph-mon[30961]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:50.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:50 smithi069 ceph-mon[28170]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:51.677 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:51 smithi130 ceph-mon[31553]: from='osd.11 [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T09:02:51.677 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:51 smithi130 ceph-mon[31553]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T09:02:51.677 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:51 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:51.677 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:51 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:51.677 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:51 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:51.677 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:51 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:02:51.678 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:51 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:51.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:51 smithi081 ceph-mon[30961]: from='osd.11 [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T09:02:51.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:51 smithi081 ceph-mon[30961]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T09:02:51.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:51 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:51.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:51 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:51.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:51 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:51.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:51 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:02:51.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:51 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:51.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:51 smithi069 ceph-mon[28170]: from='osd.11 [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T09:02:51.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:51 smithi069 ceph-mon[28170]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T09:02:51.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:51 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:51.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:51 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:51.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:51 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:51.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:51 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:02:51.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:51 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:52.513 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:52.512+0000 7fef4ca00700 1 -- 172.21.15.130:0/4005328224 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fef48062f30 con 0x7fef2c0a3c40 2024-04-24T09:02:52.514 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 11 on host 'smithi130' 2024-04-24T09:02:52.515 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:52.516+0000 7fef4de00700 1 -- 172.21.15.130:0/4005328224 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fef2c0a3c40 msgr2=0x7fef2c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:52.516 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:52.516+0000 7fef4de00700 1 --2- 172.21.15.130:0/4005328224 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fef2c0a3c40 0x7fef2c0a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fef38005ea0 tx=0x7fef38005e30 comp rx=0 tx=0).stop 2024-04-24T09:02:52.516 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:52.516+0000 7fef4de00700 1 -- 172.21.15.130:0/4005328224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef48105f10 msgr2=0x7fef4819b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:52.516 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:52.516+0000 7fef4de00700 1 --2- 172.21.15.130:0/4005328224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef48105f10 0x7fef4819b2a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fef3400c9e0 tx=0x7fef3400cda0 comp rx=0 tx=0).stop 2024-04-24T09:02:52.516 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:52.516+0000 7fef4de00700 1 -- 172.21.15.130:0/4005328224 shutdown_connections 2024-04-24T09:02:52.516 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:52.516+0000 7fef4de00700 1 --2- 172.21.15.130:0/4005328224 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fef2c0a3c40 0x7fef2c0a6100 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:52.516 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:52.516+0000 7fef4de00700 1 --2- 172.21.15.130:0/4005328224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef48108840 0x7fef481a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:52.516 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:52.516+0000 7fef4de00700 1 --2- 172.21.15.130:0/4005328224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef481035e0 0x7fef4819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:52.516 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:52.516+0000 7fef4de00700 1 --2- 172.21.15.130:0/4005328224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef48105f10 0x7fef4819b2a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:52.516 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:52.516+0000 7fef4de00700 1 -- 172.21.15.130:0/4005328224 >> 172.21.15.130:0/4005328224 conn(0x7fef480fd010 msgr2=0x7fef48102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:52.516 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:52.516+0000 7fef4de00700 1 -- 172.21.15.130:0/4005328224 shutdown_connections 2024-04-24T09:02:52.516 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:02:52.516+0000 7fef4de00700 1 -- 172.21.15.130:0/4005328224 wait complete. 2024-04-24T09:02:52.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:52 smithi130 ceph-mon[31553]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:52.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:52 smithi130 ceph-mon[31553]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T09:02:52.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:52 smithi130 ceph-mon[31553]: from='osd.11 [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T09:02:52.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:52 smithi130 ceph-mon[31553]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T09:02:52.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:52 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:52.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:52 smithi130 ceph-mon[31553]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T09:02:52.743 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:52 smithi069 ceph-mon[28170]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:52.743 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:52 smithi069 ceph-mon[28170]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T09:02:52.744 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:52 smithi069 ceph-mon[28170]: from='osd.11 [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T09:02:52.744 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:52 smithi069 ceph-mon[28170]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T09:02:52.744 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:52 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:52.744 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:52 smithi069 ceph-mon[28170]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T09:02:52.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:52 smithi081 ceph-mon[30961]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:52.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:52 smithi081 ceph-mon[30961]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T09:02:52.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:52 smithi081 ceph-mon[30961]: from='osd.11 [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T09:02:52.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:52 smithi081 ceph-mon[30961]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T09:02:52.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:52 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:52.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:52 smithi081 ceph-mon[30961]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-24T09:02:53.280 DEBUG:teuthology.orchestra.run.smithi130:osd.11> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.11.service 2024-04-24T09:02:53.287 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-24T09:02:53.287 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T09:02:53.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:53 smithi130 ceph-mon[31553]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-24T09:02:53.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:53 smithi130 ceph-mon[31553]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T09:02:53.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:53.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:53.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:53.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:02:53.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:53 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:53.705 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:02:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:53 smithi081 ceph-mon[30961]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-24T09:02:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:53 smithi081 ceph-mon[30961]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T09:02:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:02:53.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:53 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:53.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:53 smithi069 ceph-mon[28170]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-24T09:02:53.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:53 smithi069 ceph-mon[28170]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T09:02:53.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:53.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:53.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:53.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:02:53.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:53 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:54.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:54 smithi069 ceph-mon[28170]: purged_snaps scrub starts 2024-04-24T09:02:54.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:54 smithi069 ceph-mon[28170]: purged_snaps scrub ok 2024-04-24T09:02:54.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:54 smithi069 ceph-mon[28170]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:54.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:54 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:54.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:54 smithi069 ceph-mon[28170]: osd.11 [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980] boot 2024-04-24T09:02:54.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:54 smithi069 ceph-mon[28170]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T09:02:54.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:54 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:54.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:54 smithi130 ceph-mon[31553]: purged_snaps scrub starts 2024-04-24T09:02:54.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:54 smithi130 ceph-mon[31553]: purged_snaps scrub ok 2024-04-24T09:02:54.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:54 smithi130 ceph-mon[31553]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:54.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:54 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:54.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:54 smithi130 ceph-mon[31553]: osd.11 [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980] boot 2024-04-24T09:02:54.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:54 smithi130 ceph-mon[31553]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T09:02:54.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:54 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:54.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:54 smithi081 ceph-mon[30961]: purged_snaps scrub starts 2024-04-24T09:02:54.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:54 smithi081 ceph-mon[30961]: purged_snaps scrub ok 2024-04-24T09:02:54.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:54 smithi081 ceph-mon[30961]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:02:54.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:54 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:54.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:54 smithi081 ceph-mon[30961]: osd.11 [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980] boot 2024-04-24T09:02:54.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:54 smithi081 ceph-mon[30961]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T09:02:54.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:54 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:02:55.769 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:55 smithi069 ceph-mon[28170]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T09:02:55.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:55 smithi081 ceph-mon[30961]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T09:02:55.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:55 smithi130 ceph-mon[31553]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T09:02:56.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.052+0000 7fdd5b200700 1 -- 172.21.15.69:0/1627193438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd541013d0 msgr2=0x7fdd541017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:56.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.052+0000 7fdd5b200700 1 --2- 172.21.15.69:0/1627193438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd541013d0 0x7fdd541017f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009990 tx=0x7fdd44009ca0 comp rx=0 tx=0).stop 2024-04-24T09:02:56.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.052+0000 7fdd5b200700 1 -- 172.21.15.69:0/1627193438 shutdown_connections 2024-04-24T09:02:56.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.052+0000 7fdd5b200700 1 --2- 172.21.15.69:0/1627193438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd541026f0 0x7fdd54102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:56.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.052+0000 7fdd5b200700 1 --2- 172.21.15.69:0/1627193438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd541013d0 0x7fdd541017f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:56.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.052+0000 7fdd5b200700 1 --2- 172.21.15.69:0/1627193438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd54101d30 0x7fdd541021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:56.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.052+0000 7fdd5b200700 1 -- 172.21.15.69:0/1627193438 >> 172.21.15.69:0/1627193438 conn(0x7fdd540fd010 msgr2=0x7fdd540ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:56.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.053+0000 7fdd5b200700 1 -- 172.21.15.69:0/1627193438 shutdown_connections 2024-04-24T09:02:56.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.053+0000 7fdd5b200700 1 -- 172.21.15.69:0/1627193438 wait complete. 2024-04-24T09:02:56.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.054+0000 7fdd5b200700 1 Processor -- start 2024-04-24T09:02:56.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.054+0000 7fdd5b200700 1 -- start start 2024-04-24T09:02:56.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.054+0000 7fdd5b200700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd541013d0 0x7fdd5419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:56.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.054+0000 7fdd5b200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd54101d30 0x7fdd5419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:56.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.054+0000 7fdd5b200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd541026f0 0x7fdd541a2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:56.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.054+0000 7fdd5b200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541107a0 con 0x7fdd54101d30 2024-04-24T09:02:56.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.054+0000 7fdd5b200700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdd54110620 con 0x7fdd541013d0 2024-04-24T09:02:56.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.054+0000 7fdd5b200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdd54110920 con 0x7fdd541026f0 2024-04-24T09:02:56.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.055+0000 7fdd53400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd54101d30 0x7fdd5419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:56.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.055+0000 7fdd53400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd54101d30 0x7fdd5419b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59836/0 (socket says 172.21.15.69:59836) 2024-04-24T09:02:56.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.055+0000 7fdd53400700 1 -- 172.21.15.69:0/3612014410 learned_addr learned my addr 172.21.15.69:0/3612014410 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:02:56.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.055+0000 7fdd53e00700 1 --2- 172.21.15.69:0/3612014410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd541013d0 0x7fdd5419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:56.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.055+0000 7fdd59200700 1 --2- 172.21.15.69:0/3612014410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd541026f0 0x7fdd541a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:56.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.055+0000 7fdd53400700 1 -- 172.21.15.69:0/3612014410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd541026f0 msgr2=0x7fdd541a2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:56.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.055+0000 7fdd53400700 1 --2- 172.21.15.69:0/3612014410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd541026f0 0x7fdd541a2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:56.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.055+0000 7fdd53400700 1 -- 172.21.15.69:0/3612014410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd541013d0 msgr2=0x7fdd5419ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:56.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.055+0000 7fdd53400700 1 --2- 172.21.15.69:0/3612014410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd541013d0 0x7fdd5419ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:56.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.055+0000 7fdd53400700 1 -- 172.21.15.69:0/3612014410 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd541a2ad0 con 0x7fdd54101d30 2024-04-24T09:02:56.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.056+0000 7fdd53400700 1 --2- 172.21.15.69:0/3612014410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd54101d30 0x7fdd5419b3b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000d9b0 tx=0x7fdd4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:56.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.056+0000 7fdd50c00700 1 -- 172.21.15.69:0/3612014410 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd40016070 con 0x7fdd54101d30 2024-04-24T09:02:56.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.056+0000 7fdd50c00700 1 -- 172.21.15.69:0/3612014410 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd40004830 con 0x7fdd54101d30 2024-04-24T09:02:56.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.056+0000 7fdd5b200700 1 -- 172.21.15.69:0/3612014410 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd541a2dc0 con 0x7fdd54101d30 2024-04-24T09:02:56.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.056+0000 7fdd50c00700 1 -- 172.21.15.69:0/3612014410 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd400054f0 con 0x7fdd54101d30 2024-04-24T09:02:56.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.056+0000 7fdd5b200700 1 -- 172.21.15.69:0/3612014410 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd541a3270 con 0x7fdd54101d30 2024-04-24T09:02:56.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.057+0000 7fdd5b200700 1 -- 172.21.15.69:0/3612014410 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd5402dfe0 con 0x7fdd54101d30 2024-04-24T09:02:56.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.059+0000 7fdd50c00700 1 -- 172.21.15.69:0/3612014410 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdd400049e0 con 0x7fdd54101d30 2024-04-24T09:02:56.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.061+0000 7fdd50c00700 1 --2- 172.21.15.69:0/3612014410 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd380a3c40 0x7fdd380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:56.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.061+0000 7fdd50c00700 1 -- 172.21.15.69:0/3612014410 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdd400cfde0 con 0x7fdd54101d30 2024-04-24T09:02:56.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.061+0000 7fdd53e00700 1 --2- 172.21.15.69:0/3612014410 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd380a3c40 0x7fdd380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:56.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.062+0000 7fdd53e00700 1 --2- 172.21.15.69:0/3612014410 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd380a3c40 0x7fdd380a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400b5c0 tx=0x7fdd44005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:56.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.066+0000 7fdd50c00700 1 -- 172.21.15.69:0/3612014410 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd4001f020 con 0x7fdd54101d30 2024-04-24T09:02:56.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.291+0000 7fdd5b200700 1 -- 172.21.15.69:0/3612014410 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fdd540637a0 con 0x7fdd54101d30 2024-04-24T09:02:56.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.292+0000 7fdd50c00700 1 -- 172.21.15.69:0/3612014410 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7fdd4009cbc0 con 0x7fdd54101d30 2024-04-24T09:02:56.292 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:02:56.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.295+0000 7fdd5b200700 1 -- 172.21.15.69:0/3612014410 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd380a3c40 msgr2=0x7fdd380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:56.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.295+0000 7fdd5b200700 1 --2- 172.21.15.69:0/3612014410 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd380a3c40 0x7fdd380a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400b5c0 tx=0x7fdd44005fd0 comp rx=0 tx=0).stop 2024-04-24T09:02:56.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.295+0000 7fdd5b200700 1 -- 172.21.15.69:0/3612014410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd54101d30 msgr2=0x7fdd5419b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:56.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.295+0000 7fdd5b200700 1 --2- 172.21.15.69:0/3612014410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd54101d30 0x7fdd5419b3b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000d9b0 tx=0x7fdd4000dcc0 comp rx=0 tx=0).stop 2024-04-24T09:02:56.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.295+0000 7fdd5b200700 1 -- 172.21.15.69:0/3612014410 shutdown_connections 2024-04-24T09:02:56.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.295+0000 7fdd5b200700 1 --2- 172.21.15.69:0/3612014410 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd380a3c40 0x7fdd380a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:56.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.295+0000 7fdd5b200700 1 --2- 172.21.15.69:0/3612014410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd541026f0 0x7fdd541a2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:56.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.295+0000 7fdd5b200700 1 --2- 172.21.15.69:0/3612014410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd541013d0 0x7fdd5419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:56.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.295+0000 7fdd5b200700 1 --2- 172.21.15.69:0/3612014410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd54101d30 0x7fdd5419b3b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:56.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.295+0000 7fdd5b200700 1 -- 172.21.15.69:0/3612014410 >> 172.21.15.69:0/3612014410 conn(0x7fdd540fd010 msgr2=0x7fdd540feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:56.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.295+0000 7fdd5b200700 1 -- 172.21.15.69:0/3612014410 shutdown_connections 2024-04-24T09:02:56.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:02:56.295+0000 7fdd5b200700 1 -- 172.21.15.69:0/3612014410 wait complete. 2024-04-24T09:02:56.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:56 smithi069 ceph-mon[28170]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:02:56.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:56 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3612014410' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T09:02:56.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:56 smithi081 ceph-mon[30961]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:02:56.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:56 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3612014410' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T09:02:56.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:56 smithi130 ceph-mon[31553]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:02:56.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:56 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3612014410' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T09:02:57.249 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1713949373,"num_in_osds":12,"osd_in_since":1713949348,"num_remapped_pgs":0} 2024-04-24T09:02:57.250 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T09:02:57.694 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:02:57.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:57 smithi130 ceph-mon[31553]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:02:57.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:57 smithi081 ceph-mon[30961]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:02:57.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:57 smithi069 ceph-mon[28170]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:02:59.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:59 smithi081 ceph-mon[30961]: Detected new or changed devices on smithi130 2024-04-24T09:02:59.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:59.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:59.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:59.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:59.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:59.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:59.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:59 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi130 to 4272M 2024-04-24T09:02:59.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:59.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:59.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:02:59.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:02:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:59.580 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:59 smithi069 ceph-mon[28170]: Detected new or changed devices on smithi130 2024-04-24T09:02:59.580 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:59.580 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:59.580 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:59.580 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:59.580 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:59.580 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:59.580 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:59 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi130 to 4272M 2024-04-24T09:02:59.580 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:59.580 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:59.580 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:02:59.581 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:02:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:59.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:59 smithi130 ceph-mon[31553]: Detected new or changed devices on smithi130 2024-04-24T09:02:59.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:59.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:59.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:59.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:59.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:59.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:59.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:59 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi130 to 4272M 2024-04-24T09:02:59.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:02:59.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:59.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:02:59.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:02:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:00.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.152+0000 7f6f49e00700 1 -- 172.21.15.69:0/2110125156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f44105940 msgr2=0x7f6f44109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.152+0000 7f6f49e00700 1 --2- 172.21.15.69:0/2110125156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f44105940 0x7f6f44109990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0099c0 tx=0x7f6f2c009cd0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.153+0000 7f6f49e00700 1 -- 172.21.15.69:0/2110125156 shutdown_connections 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.153+0000 7f6f49e00700 1 --2- 172.21.15.69:0/2110125156 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f4410a070 0x7f6f44111c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.153+0000 7f6f49e00700 1 --2- 172.21.15.69:0/2110125156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f44104f90 0x7f6f44105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.153+0000 7f6f49e00700 1 --2- 172.21.15.69:0/2110125156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f44105940 0x7f6f44109990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.153+0000 7f6f49e00700 1 -- 172.21.15.69:0/2110125156 >> 172.21.15.69:0/2110125156 conn(0x7f6f44100820 msgr2=0x7f6f44102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.153+0000 7f6f49e00700 1 -- 172.21.15.69:0/2110125156 shutdown_connections 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.153+0000 7f6f49e00700 1 -- 172.21.15.69:0/2110125156 wait complete. 2024-04-24T09:03:00.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.154+0000 7f6f49e00700 1 Processor -- start 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.154+0000 7f6f49e00700 1 -- start start 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.154+0000 7f6f49e00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f44104f90 0x7f6f4419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.154+0000 7f6f49e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f44105940 0x7f6f4419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.154+0000 7f6f49e00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f4410a070 0x7f6f4419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.154+0000 7f6f49e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6f44114ee0 con 0x7f6f44105940 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.154+0000 7f6f49e00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6f44114d60 con 0x7f6f4410a070 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.154+0000 7f6f49e00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6f44115060 con 0x7f6f44104f90 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.155+0000 7f6f3be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f44105940 0x7f6f4419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.155+0000 7f6f3be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f44105940 0x7f6f4419b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59852/0 (socket says 172.21.15.69:59852) 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.155+0000 7f6f3be00700 1 -- 172.21.15.69:0/3946200529 learned_addr learned my addr 172.21.15.69:0/3946200529 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.155+0000 7f6f42a00700 1 --2- 172.21.15.69:0/3946200529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f44104f90 0x7f6f4419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.155+0000 7f6f43400700 1 --2- 172.21.15.69:0/3946200529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f4410a070 0x7f6f4419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.155+0000 7f6f3be00700 1 -- 172.21.15.69:0/3946200529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f44104f90 msgr2=0x7f6f4419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.155+0000 7f6f3be00700 1 --2- 172.21.15.69:0/3946200529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f44104f90 0x7f6f4419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.155+0000 7f6f3be00700 1 -- 172.21.15.69:0/3946200529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f4410a070 msgr2=0x7f6f4419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.155+0000 7f6f3be00700 1 --2- 172.21.15.69:0/3946200529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f4410a070 0x7f6f4419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.155+0000 7f6f3be00700 1 -- 172.21.15.69:0/3946200529 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f441a02e0 con 0x7f6f44105940 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.155+0000 7f6f3be00700 1 --2- 172.21.15.69:0/3946200529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f44105940 0x7f6f4419b810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009fd0 tx=0x7f6f2c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.156+0000 7f6f48a00700 1 -- 172.21.15.69:0/3946200529 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c004830 con 0x7f6f44105940 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.156+0000 7f6f48a00700 1 -- 172.21.15.69:0/3946200529 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f2c003ba0 con 0x7f6f44105940 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.156+0000 7f6f48a00700 1 -- 172.21.15.69:0/3946200529 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c005380 con 0x7f6f44105940 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.156+0000 7f6f49e00700 1 -- 172.21.15.69:0/3946200529 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f441a0570 con 0x7f6f44105940 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.156+0000 7f6f49e00700 1 -- 172.21.15.69:0/3946200529 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f441a09a0 con 0x7f6f44105940 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.157+0000 7f6f49e00700 1 -- 172.21.15.69:0/3946200529 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f440697e0 con 0x7f6f44105940 2024-04-24T09:03:00.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.159+0000 7f6f48a00700 1 -- 172.21.15.69:0/3946200529 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6f2c005560 con 0x7f6f44105940 2024-04-24T09:03:00.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.161+0000 7f6f48a00700 1 --2- 172.21.15.69:0/3946200529 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6f280a3cc0 0x7f6f280a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.161+0000 7f6f48a00700 1 -- 172.21.15.69:0/3946200529 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6f2c0d0ff0 con 0x7f6f44105940 2024-04-24T09:03:00.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.161+0000 7f6f42a00700 1 --2- 172.21.15.69:0/3946200529 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6f280a3cc0 0x7f6f280a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.162+0000 7f6f42a00700 1 --2- 172.21.15.69:0/3946200529 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6f280a3cc0 0x7f6f280a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6f34005ea0 tx=0x7f6f34005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:00.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.165+0000 7f6f48a00700 1 -- 172.21.15.69:0/3946200529 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f2c013070 con 0x7f6f44105940 2024-04-24T09:03:00.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.392+0000 7f6f49e00700 1 -- 172.21.15.69:0/3946200529 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f441a7fc0 con 0x7f6f44105940 2024-04-24T09:03:00.393 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:00 smithi069 ceph-mon[28170]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:00.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.397+0000 7f6f48a00700 1 -- 172.21.15.69:0/3946200529 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18422 (secure 0 0 0) 0x7f6f2c09de60 con 0x7f6f44105940 2024-04-24T09:03:00.397 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:00.397 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":82,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:02:54.504455+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:03:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.400+0000 7f6f49e00700 1 -- 172.21.15.69:0/3946200529 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6f280a3cc0 msgr2=0x7f6f280a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.400+0000 7f6f49e00700 1 --2- 172.21.15.69:0/3946200529 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6f280a3cc0 0x7f6f280a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6f34005ea0 tx=0x7f6f34005e30 comp rx=0 tx=0).stop 2024-04-24T09:03:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.400+0000 7f6f49e00700 1 -- 172.21.15.69:0/3946200529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f44105940 msgr2=0x7f6f4419b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.400+0000 7f6f49e00700 1 --2- 172.21.15.69:0/3946200529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f44105940 0x7f6f4419b810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009fd0 tx=0x7f6f2c005f10 comp rx=0 tx=0).stop 2024-04-24T09:03:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.400+0000 7f6f49e00700 1 -- 172.21.15.69:0/3946200529 shutdown_connections 2024-04-24T09:03:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.400+0000 7f6f49e00700 1 --2- 172.21.15.69:0/3946200529 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6f280a3cc0 0x7f6f280a6180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.400+0000 7f6f49e00700 1 --2- 172.21.15.69:0/3946200529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f4410a070 0x7f6f4419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.400+0000 7f6f49e00700 1 --2- 172.21.15.69:0/3946200529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f44104f90 0x7f6f4419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.400+0000 7f6f49e00700 1 --2- 172.21.15.69:0/3946200529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6f44105940 0x7f6f4419b810 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.400+0000 7f6f49e00700 1 -- 172.21.15.69:0/3946200529 >> 172.21.15.69:0/3946200529 conn(0x7f6f44100820 msgr2=0x7f6f44101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.401+0000 7f6f49e00700 1 -- 172.21.15.69:0/3946200529 shutdown_connections 2024-04-24T09:03:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:00.401+0000 7f6f49e00700 1 -- 172.21.15.69:0/3946200529 wait complete. 2024-04-24T09:03:00.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:00 smithi081 ceph-mon[30961]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:00.605 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:00 smithi130 ceph-mon[31553]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:01.319 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:57:37.377599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T09:03:01.320 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-24T09:03:01.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:01 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3946200529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:03:01.569 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:01 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3946200529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:03:01.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:01 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3946200529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:03:01.757 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:03:02.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:02 smithi081 ceph-mon[30961]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:02.590 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:02 smithi069 ceph-mon[28170]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:02.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:02 smithi130 ceph-mon[31553]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:03.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.988+0000 7f223a200700 1 -- 172.21.15.69:0/2356583301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f223406bb80 msgr2=0x7f223410f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:03.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.988+0000 7f223a200700 1 --2- 172.21.15.69:0/2356583301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f223406bb80 0x7f223410f9b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f222c00b0a0 tx=0x7f222c00b3b0 comp rx=0 tx=0).stop 2024-04-24T09:03:03.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.988+0000 7f223a200700 1 -- 172.21.15.69:0/2356583301 shutdown_connections 2024-04-24T09:03:03.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.988+0000 7f223a200700 1 --2- 172.21.15.69:0/2356583301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f223406bb80 0x7f223410f9b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:03.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.988+0000 7f223a200700 1 --2- 172.21.15.69:0/2356583301 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2234108b90 0x7f2234108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:03.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.988+0000 7f223a200700 1 --2- 172.21.15.69:0/2356583301 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f223406b1c0 0x7f223406b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:03.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.988+0000 7f223a200700 1 -- 172.21.15.69:0/2356583301 >> 172.21.15.69:0/2356583301 conn(0x7f2234078a30 msgr2=0x7f2234078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:03.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.988+0000 7f223a200700 1 -- 172.21.15.69:0/2356583301 shutdown_connections 2024-04-24T09:03:03.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.988+0000 7f223a200700 1 -- 172.21.15.69:0/2356583301 wait complete. 2024-04-24T09:03:03.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.989+0000 7f223a200700 1 Processor -- start 2024-04-24T09:03:03.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.989+0000 7f223a200700 1 -- start start 2024-04-24T09:03:03.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.989+0000 7f223a200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f223406b1c0 0x7f2234101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:03.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.989+0000 7f223a200700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f223406bb80 0x7f2234102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:03.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.990+0000 7f223a200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2234108b90 0x7f2234102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:03.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.990+0000 7f223a200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2234112cc0 con 0x7f2234108b90 2024-04-24T09:03:03.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.990+0000 7f223a200700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2234112b40 con 0x7f223406bb80 2024-04-24T09:03:03.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.990+0000 7f223a200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2234112e40 con 0x7f223406b1c0 2024-04-24T09:03:03.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.990+0000 7f2233e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2234108b90 0x7f2234102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:03.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.990+0000 7f2233e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2234108b90 0x7f2234102570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36896/0 (socket says 172.21.15.69:36896) 2024-04-24T09:03:03.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.990+0000 7f2233e00700 1 -- 172.21.15.69:0/2356777701 learned_addr learned my addr 172.21.15.69:0/2356777701 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:03.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.990+0000 7f2233400700 1 --2- 172.21.15.69:0/2356777701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f223406b1c0 0x7f2234101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:03.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.990+0000 7f2232a00700 1 --2- 172.21.15.69:0/2356777701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f223406bb80 0x7f2234102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:03.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.990+0000 7f2233e00700 1 -- 172.21.15.69:0/2356777701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f223406b1c0 msgr2=0x7f2234101af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:03.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.990+0000 7f2233e00700 1 --2- 172.21.15.69:0/2356777701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f223406b1c0 0x7f2234101af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:03.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.990+0000 7f2233e00700 1 -- 172.21.15.69:0/2356777701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f223406bb80 msgr2=0x7f2234102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:03.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.990+0000 7f2233e00700 1 --2- 172.21.15.69:0/2356777701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f223406bb80 0x7f2234102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:03.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.990+0000 7f2233e00700 1 -- 172.21.15.69:0/2356777701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22341a7620 con 0x7f2234108b90 2024-04-24T09:03:03.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.991+0000 7f2233e00700 1 --2- 172.21.15.69:0/2356777701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2234108b90 0x7f2234102570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f222c0062a0 tx=0x7f222c004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:03.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.991+0000 7f222be00700 1 -- 172.21.15.69:0/2356777701 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c021070 con 0x7f2234108b90 2024-04-24T09:03:03.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.991+0000 7f222be00700 1 -- 172.21.15.69:0/2356777701 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f222c007d00 con 0x7f2234108b90 2024-04-24T09:03:03.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.991+0000 7f223a200700 1 -- 172.21.15.69:0/2356777701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22341a78b0 con 0x7f2234108b90 2024-04-24T09:03:03.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.991+0000 7f222be00700 1 -- 172.21.15.69:0/2356777701 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c01d450 con 0x7f2234108b90 2024-04-24T09:03:03.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.991+0000 7f223a200700 1 -- 172.21.15.69:0/2356777701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22341a7d60 con 0x7f2234108b90 2024-04-24T09:03:03.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.992+0000 7f223a200700 1 -- 172.21.15.69:0/2356777701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22341041e0 con 0x7f2234108b90 2024-04-24T09:03:03.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.994+0000 7f222be00700 1 -- 172.21.15.69:0/2356777701 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f222c003a00 con 0x7f2234108b90 2024-04-24T09:03:03.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.995+0000 7f222be00700 1 --2- 172.21.15.69:0/2356777701 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f22180a3cc0 0x7f22180a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:03.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.996+0000 7f222be00700 1 -- 172.21.15.69:0/2356777701 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f222c00f020 con 0x7f2234108b90 2024-04-24T09:03:03.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.996+0000 7f2233400700 1 --2- 172.21.15.69:0/2356777701 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f22180a3cc0 0x7f22180a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:03.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:03.997+0000 7f2233400700 1 --2- 172.21.15.69:0/2356777701 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f22180a3cc0 0x7f22180a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2224005fd0 tx=0x7f222400af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:04.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:04.000+0000 7f222be00700 1 -- 172.21.15.69:0/2356777701 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f222c02a020 con 0x7f2234108b90 2024-04-24T09:03:04.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:04.227+0000 7f223a200700 1 -- 172.21.15.69:0/2356777701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f223402e1a0 con 0x7f2234108b90 2024-04-24T09:03:04.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:04.228+0000 7f222be00700 1 -- 172.21.15.69:0/2356777701 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f222c09e5b0 con 0x7f2234108b90 2024-04-24T09:03:04.228 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-24T09:03:04.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:04.231+0000 7f223a200700 1 -- 172.21.15.69:0/2356777701 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f22180a3cc0 msgr2=0x7f22180a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:04.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:04.231+0000 7f223a200700 1 --2- 172.21.15.69:0/2356777701 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f22180a3cc0 0x7f22180a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2224005fd0 tx=0x7f222400af20 comp rx=0 tx=0).stop 2024-04-24T09:03:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:04.231+0000 7f223a200700 1 -- 172.21.15.69:0/2356777701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2234108b90 msgr2=0x7f2234102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:04.231+0000 7f223a200700 1 --2- 172.21.15.69:0/2356777701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2234108b90 0x7f2234102570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f222c0062a0 tx=0x7f222c004ed0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:04.231+0000 7f223a200700 1 -- 172.21.15.69:0/2356777701 shutdown_connections 2024-04-24T09:03:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:04.231+0000 7f223a200700 1 --2- 172.21.15.69:0/2356777701 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f22180a3cc0 0x7f22180a6180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:04.231+0000 7f223a200700 1 --2- 172.21.15.69:0/2356777701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2234108b90 0x7f2234102570 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:04.231+0000 7f223a200700 1 --2- 172.21.15.69:0/2356777701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f223406b1c0 0x7f2234101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:04.231+0000 7f223a200700 1 --2- 172.21.15.69:0/2356777701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f223406bb80 0x7f2234102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:04.231+0000 7f223a200700 1 -- 172.21.15.69:0/2356777701 >> 172.21.15.69:0/2356777701 conn(0x7f2234078a30 msgr2=0x7f2234100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:04.231+0000 7f223a200700 1 -- 172.21.15.69:0/2356777701 shutdown_connections 2024-04-24T09:03:04.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:04.231+0000 7f223a200700 1 -- 172.21.15.69:0/2356777701 wait complete. 2024-04-24T09:03:04.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:04 smithi069 ceph-mon[28170]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:04.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:04 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2356777701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:03:04.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:04 smithi130 ceph-mon[31553]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:04.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:04 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2356777701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:03:04.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:04 smithi081 ceph-mon[30961]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:04.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:04 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2356777701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:03:05.174 INFO:tasks.cephadm:Adding mds.a on smithi069 2024-04-24T09:03:05.174 INFO:tasks.cephadm:Adding mds.d on smithi069 2024-04-24T09:03:05.174 INFO:tasks.cephadm:Adding mds.g on smithi069 2024-04-24T09:03:05.174 INFO:tasks.cephadm:Adding mds.j on smithi069 2024-04-24T09:03:05.174 INFO:tasks.cephadm:Adding mds.b on smithi081 2024-04-24T09:03:05.174 INFO:tasks.cephadm:Adding mds.e on smithi081 2024-04-24T09:03:05.174 INFO:tasks.cephadm:Adding mds.h on smithi081 2024-04-24T09:03:05.174 INFO:tasks.cephadm:Adding mds.k on smithi081 2024-04-24T09:03:05.174 INFO:tasks.cephadm:Adding mds.c on smithi130 2024-04-24T09:03:05.174 INFO:tasks.cephadm:Adding mds.f on smithi130 2024-04-24T09:03:05.175 INFO:tasks.cephadm:Adding mds.i on smithi130 2024-04-24T09:03:05.175 INFO:tasks.cephadm:Adding mds.l on smithi130 2024-04-24T09:03:05.175 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l' 2024-04-24T09:03:05.618 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:03:06.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:06 smithi130 ceph-mon[31553]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:06.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:06 smithi081 ceph-mon[30961]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:06.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:06 smithi069 ceph-mon[28170]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:07.947 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.947+0000 7f5a24400700 1 -- 172.21.15.130:0/2570910043 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a1c105f10 msgr2=0x7f5a1c108300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:07.947 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.947+0000 7f5a24400700 1 --2- 172.21.15.130:0/2570910043 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a1c105f10 0x7f5a1c108300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009990 tx=0x7f5a08009ca0 comp rx=0 tx=0).stop 2024-04-24T09:03:07.947 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.947+0000 7f5a24400700 1 -- 172.21.15.130:0/2570910043 shutdown_connections 2024-04-24T09:03:07.947 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.947+0000 7f5a24400700 1 --2- 172.21.15.130:0/2570910043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a1c108840 0x7f5a1c10acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:07.947 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.947+0000 7f5a24400700 1 --2- 172.21.15.130:0/2570910043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a1c1035e0 0x7f5a1c1059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:07.948 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.947+0000 7f5a24400700 1 --2- 172.21.15.130:0/2570910043 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a1c105f10 0x7f5a1c108300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:07.948 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.947+0000 7f5a24400700 1 -- 172.21.15.130:0/2570910043 >> 172.21.15.130:0/2570910043 conn(0x7f5a1c0fd010 msgr2=0x7f5a1c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:07.948 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.947+0000 7f5a24400700 1 -- 172.21.15.130:0/2570910043 shutdown_connections 2024-04-24T09:03:07.948 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.947+0000 7f5a24400700 1 -- 172.21.15.130:0/2570910043 wait complete. 2024-04-24T09:03:07.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.948+0000 7f5a24400700 1 Processor -- start 2024-04-24T09:03:07.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.948+0000 7f5a24400700 1 -- start start 2024-04-24T09:03:07.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.949+0000 7f5a24400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a1c1035e0 0x7f5a1c19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:07.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.949+0000 7f5a24400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a1c105f10 0x7f5a1c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:07.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.949+0000 7f5a24400700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a1c108840 0x7f5a1c1a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:07.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.949+0000 7f5a24400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c10e630 con 0x7f5a1c1035e0 2024-04-24T09:03:07.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.949+0000 7f5a24400700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c10e4b0 con 0x7f5a1c108840 2024-04-24T09:03:07.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.949+0000 7f5a24400700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c10e7b0 con 0x7f5a1c105f10 2024-04-24T09:03:07.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.949+0000 7f5a21000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a1c105f10 0x7f5a1c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:07.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.949+0000 7f5a21000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a1c105f10 0x7f5a1c19b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:50588/0 (socket says 172.21.15.130:50588) 2024-04-24T09:03:07.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.949+0000 7f5a21000700 1 -- 172.21.15.130:0/542889499 learned_addr learned my addr 172.21.15.130:0/542889499 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-24T09:03:07.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.949+0000 7f5a21a00700 1 --2- 172.21.15.130:0/542889499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a1c1035e0 0x7f5a1c19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:07.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.949+0000 7f5a22400700 1 --2- 172.21.15.130:0/542889499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a1c108840 0x7f5a1c1a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:07.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.950+0000 7f5a21000700 1 -- 172.21.15.130:0/542889499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a1c108840 msgr2=0x7f5a1c1a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:07.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.950+0000 7f5a21000700 1 --2- 172.21.15.130:0/542889499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a1c108840 0x7f5a1c1a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:07.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.950+0000 7f5a21000700 1 -- 172.21.15.130:0/542889499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a1c1035e0 msgr2=0x7f5a1c19af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:07.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.950+0000 7f5a21000700 1 --2- 172.21.15.130:0/542889499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a1c1035e0 0x7f5a1c19af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:07.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.950+0000 7f5a21000700 1 -- 172.21.15.130:0/542889499 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c1a2b80 con 0x7f5a1c105f10 2024-04-24T09:03:07.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.950+0000 7f5a21000700 1 --2- 172.21.15.130:0/542889499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a1c105f10 0x7f5a1c19b460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5a08006010 tx=0x7f5a08003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:07.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.950+0000 7f5a1a000700 1 -- 172.21.15.130:0/542889499 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08027070 con 0x7f5a1c105f10 2024-04-24T09:03:07.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.950+0000 7f5a1a000700 1 -- 172.21.15.130:0/542889499 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a0800bd90 con 0x7f5a1c105f10 2024-04-24T09:03:07.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.950+0000 7f5a1a000700 1 -- 172.21.15.130:0/542889499 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0800f460 con 0x7f5a1c105f10 2024-04-24T09:03:07.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.950+0000 7f5a24400700 1 -- 172.21.15.130:0/542889499 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1c1a2e10 con 0x7f5a1c105f10 2024-04-24T09:03:07.952 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.950+0000 7f5a24400700 1 -- 172.21.15.130:0/542889499 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a1c1a31b0 con 0x7f5a1c105f10 2024-04-24T09:03:07.952 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.951+0000 7f5a24400700 1 -- 172.21.15.130:0/542889499 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1c0697e0 con 0x7f5a1c105f10 2024-04-24T09:03:07.953 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.954+0000 7f5a1a000700 1 -- 172.21.15.130:0/542889499 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5a0802a070 con 0x7f5a1c105f10 2024-04-24T09:03:07.954 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.955+0000 7f5a1a000700 1 --2- 172.21.15.130:0/542889499 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5a040a3cc0 0x7f5a040a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:07.954 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.955+0000 7f5a1a000700 1 -- 172.21.15.130:0/542889499 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5a080d1260 con 0x7f5a1c105f10 2024-04-24T09:03:07.955 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.956+0000 7f5a21a00700 1 --2- 172.21.15.130:0/542889499 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5a040a3cc0 0x7f5a040a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:07.956 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.957+0000 7f5a21a00700 1 --2- 172.21.15.130:0/542889499 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5a040a3cc0 0x7f5a040a6180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5a10005950 tx=0x7f5a1000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:07.957 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:07.958+0000 7f5a1a000700 1 -- 172.21.15.130:0/542889499 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a08013070 con 0x7f5a1c105f10 2024-04-24T09:03:08.192 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:08.192+0000 7f5a24400700 1 -- 172.21.15.130:0/542889499 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l", "target": ["mon-mgr", ""]}) -- 0x7f5a1c0631f0 con 0x7f5a040a3cc0 2024-04-24T09:03:08.210 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:08.210+0000 7f5a1a000700 1 -- 172.21.15.130:0/542889499 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f5a1c0631f0 con 0x7f5a040a3cc0 2024-04-24T09:03:08.210 INFO:teuthology.orchestra.run.smithi130.stdout:Scheduled mds.all update... 2024-04-24T09:03:08.213 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:08.213+0000 7f5a24400700 1 -- 172.21.15.130:0/542889499 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5a040a3cc0 msgr2=0x7f5a040a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:08.213 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:08.213+0000 7f5a24400700 1 --2- 172.21.15.130:0/542889499 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5a040a3cc0 0x7f5a040a6180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5a10005950 tx=0x7f5a1000b410 comp rx=0 tx=0).stop 2024-04-24T09:03:08.213 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:08.213+0000 7f5a24400700 1 -- 172.21.15.130:0/542889499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a1c105f10 msgr2=0x7f5a1c19b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:08.213 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:08.213+0000 7f5a24400700 1 --2- 172.21.15.130:0/542889499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a1c105f10 0x7f5a1c19b460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5a08006010 tx=0x7f5a08003b80 comp rx=0 tx=0).stop 2024-04-24T09:03:08.213 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:08.213+0000 7f5a24400700 1 -- 172.21.15.130:0/542889499 shutdown_connections 2024-04-24T09:03:08.213 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:08.213+0000 7f5a24400700 1 --2- 172.21.15.130:0/542889499 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5a040a3cc0 0x7f5a040a6180 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:08.213 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:08.213+0000 7f5a24400700 1 --2- 172.21.15.130:0/542889499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a1c108840 0x7f5a1c1a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:08.213 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:08.213+0000 7f5a24400700 1 --2- 172.21.15.130:0/542889499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a1c1035e0 0x7f5a1c19af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:08.213 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:08.213+0000 7f5a24400700 1 --2- 172.21.15.130:0/542889499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a1c105f10 0x7f5a1c19b460 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:08.213 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:08.213+0000 7f5a24400700 1 -- 172.21.15.130:0/542889499 >> 172.21.15.130:0/542889499 conn(0x7f5a1c0fd010 msgr2=0x7f5a1c102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:08.214 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:08.214+0000 7f5a24400700 1 -- 172.21.15.130:0/542889499 shutdown_connections 2024-04-24T09:03:08.214 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-24T09:03:08.214+0000 7f5a24400700 1 -- 172.21.15.130:0/542889499 wait complete. 2024-04-24T09:03:08.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:08 smithi130 ceph-mon[31553]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:08.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:08 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:08.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:08 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:03:08.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:08 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:08.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:08 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:03:08.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:08 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:08.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:08 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:08.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:08 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:08.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:08 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:08.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:08 smithi081 ceph-mon[30961]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:08.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:08 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:08.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:08 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:03:08.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:08 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:08.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:08 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:03:08.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:08 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:08.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:08 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:08.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:08 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:08.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:08 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:08 smithi069 ceph-mon[28170]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:03:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:03:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:08.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:08.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:08 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:09.083 DEBUG:teuthology.orchestra.run.smithi069:mds.a> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.a.service 2024-04-24T09:03:09.091 DEBUG:teuthology.orchestra.run.smithi069:mds.d> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.d.service 2024-04-24T09:03:09.137 DEBUG:teuthology.orchestra.run.smithi069:mds.g> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.g.service 2024-04-24T09:03:09.183 DEBUG:teuthology.orchestra.run.smithi069:mds.j> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.j.service 2024-04-24T09:03:09.227 DEBUG:teuthology.orchestra.run.smithi081:mds.b> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.b.service 2024-04-24T09:03:09.235 DEBUG:teuthology.orchestra.run.smithi081:mds.e> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.e.service 2024-04-24T09:03:09.282 DEBUG:teuthology.orchestra.run.smithi081:mds.h> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.h.service 2024-04-24T09:03:09.327 DEBUG:teuthology.orchestra.run.smithi081:mds.k> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.k.service 2024-04-24T09:03:09.372 DEBUG:teuthology.orchestra.run.smithi130:mds.c> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.c.service 2024-04-24T09:03:09.380 DEBUG:teuthology.orchestra.run.smithi130:mds.f> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.f.service 2024-04-24T09:03:09.426 DEBUG:teuthology.orchestra.run.smithi130:mds.i> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.i.service 2024-04-24T09:03:09.472 DEBUG:teuthology.orchestra.run.smithi130:mds.l> sudo journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.l.service 2024-04-24T09:03:09.517 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-24T09:03:09.518 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:03:09.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:09 smithi081 ceph-mon[30961]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:03:09.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:09 smithi081 ceph-mon[30961]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l;count:12 2024-04-24T09:03:09.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:09 smithi081 ceph-mon[30961]: Deploying daemon mds.l on smithi130 2024-04-24T09:03:09.666 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:09 smithi130 ceph-mon[31553]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:03:09.667 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:09 smithi130 ceph-mon[31553]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l;count:12 2024-04-24T09:03:09.667 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:09 smithi130 ceph-mon[31553]: Deploying daemon mds.l on smithi130 2024-04-24T09:03:09.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.746+0000 7efc4ca00640 1 -- 172.21.15.69:0/2793258161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc48109d30 msgr2=0x7efc4810c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:09.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.746+0000 7efc4ca00640 1 --2- 172.21.15.69:0/2793258161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc48109d30 0x7efc4810c1c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efc40009ca0 tx=0x7efc4002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:09.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.746+0000 7efc4ca00640 1 -- 172.21.15.69:0/2793258161 shutdown_connections 2024-04-24T09:03:09.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.746+0000 7efc4ca00640 1 --2- 172.21.15.69:0/2793258161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc48109d30 0x7efc4810c1c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:09.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.746+0000 7efc4ca00640 1 --2- 172.21.15.69:0/2793258161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc48074a60 0x7efc48074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:09.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.746+0000 7efc4ca00640 1 --2- 172.21.15.69:0/2793258161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc48076040 0x7efc48074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:09.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.746+0000 7efc4ca00640 1 -- 172.21.15.69:0/2793258161 >> 172.21.15.69:0/2793258161 conn(0x7efc480fd3e0 msgr2=0x7efc480ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:09.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.746+0000 7efc4ca00640 1 -- 172.21.15.69:0/2793258161 shutdown_connections 2024-04-24T09:03:09.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.746+0000 7efc4ca00640 1 -- 172.21.15.69:0/2793258161 wait complete. 2024-04-24T09:03:09.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.748+0000 7efc4ca00640 1 Processor -- start 2024-04-24T09:03:09.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.748+0000 7efc4ca00640 1 -- start start 2024-04-24T09:03:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.748+0000 7efc4ca00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc48074a60 0x7efc4819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.748+0000 7efc4ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc48076040 0x7efc4819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.748+0000 7efc4ca00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc48109d30 0x7efc481a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.748+0000 7efc4ca00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efc4810fbb0 con 0x7efc48076040 2024-04-24T09:03:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.748+0000 7efc4ca00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efc4810fa30 con 0x7efc48109d30 2024-04-24T09:03:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.748+0000 7efc4ca00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efc4810fd30 con 0x7efc48074a60 2024-04-24T09:03:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.749+0000 7efc3fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc48076040 0x7efc4819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.749+0000 7efc3fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc48076040 0x7efc4819a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36918/0 (socket says 172.21.15.69:36918) 2024-04-24T09:03:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.749+0000 7efc3fe00640 1 -- 172.21.15.69:0/4197313881 learned_addr learned my addr 172.21.15.69:0/4197313881 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.749+0000 7efc47400640 1 --2- 172.21.15.69:0/4197313881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc48074a60 0x7efc4819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.749+0000 7efc47e00640 1 --2- 172.21.15.69:0/4197313881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc48109d30 0x7efc481a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.749+0000 7efc3fe00640 1 -- 172.21.15.69:0/4197313881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc48074a60 msgr2=0x7efc4819a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.749+0000 7efc3fe00640 1 --2- 172.21.15.69:0/4197313881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc48074a60 0x7efc4819a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.749+0000 7efc3fe00640 1 -- 172.21.15.69:0/4197313881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc48109d30 msgr2=0x7efc481a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:09.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.749+0000 7efc3fe00640 1 --2- 172.21.15.69:0/4197313881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc48109d30 0x7efc481a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:09.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.749+0000 7efc3fe00640 1 -- 172.21.15.69:0/4197313881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc481a1e20 con 0x7efc48076040 2024-04-24T09:03:09.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.750+0000 7efc3fe00640 1 --2- 172.21.15.69:0/4197313881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc48076040 0x7efc4819a740 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efc3000d950 tx=0x7efc3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:09.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.750+0000 7efc44c00640 1 -- 172.21.15.69:0/4197313881 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc30017070 con 0x7efc48076040 2024-04-24T09:03:09.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.750+0000 7efc4ca00640 1 -- 172.21.15.69:0/4197313881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc481a2110 con 0x7efc48076040 2024-04-24T09:03:09.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.750+0000 7efc44c00640 1 -- 172.21.15.69:0/4197313881 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc30009c30 con 0x7efc48076040 2024-04-24T09:03:09.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.750+0000 7efc44c00640 1 -- 172.21.15.69:0/4197313881 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3000f620 con 0x7efc48076040 2024-04-24T09:03:09.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.750+0000 7efc4ca00640 1 -- 172.21.15.69:0/4197313881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc481a2660 con 0x7efc48076040 2024-04-24T09:03:09.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.751+0000 7efc4ca00640 1 -- 172.21.15.69:0/4197313881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc48108a60 con 0x7efc48076040 2024-04-24T09:03:09.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.753+0000 7efc44c00640 1 -- 172.21.15.69:0/4197313881 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efc300109a0 con 0x7efc48076040 2024-04-24T09:03:09.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.755+0000 7efc44c00640 1 --2- 172.21.15.69:0/4197313881 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efc2c0a3bb0 0x7efc2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:09.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.755+0000 7efc44c00640 1 -- 172.21.15.69:0/4197313881 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efc300cfc20 con 0x7efc48076040 2024-04-24T09:03:09.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.755+0000 7efc47400640 1 --2- 172.21.15.69:0/4197313881 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efc2c0a3bb0 0x7efc2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:09.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.756+0000 7efc47400640 1 --2- 172.21.15.69:0/4197313881 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efc2c0a3bb0 0x7efc2c0a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efc3800bbe0 tx=0x7efc3800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:09.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.760+0000 7efc44c00640 1 -- 172.21.15.69:0/4197313881 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc30009de0 con 0x7efc48076040 2024-04-24T09:03:09.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:09 smithi069 ceph-mon[28170]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:03:09.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:09 smithi069 ceph-mon[28170]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l;count:12 2024-04-24T09:03:09.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:09 smithi069 ceph-mon[28170]: Deploying daemon mds.l on smithi130 2024-04-24T09:03:09.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.967+0000 7efc4ca00640 1 -- 172.21.15.69:0/4197313881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc4802d870 con 0x7efc48076040 2024-04-24T09:03:09.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.970+0000 7efc44c00640 1 -- 172.21.15.69:0/4197313881 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18422 (secure 0 0 0) 0x7efc3009ca00 con 0x7efc48076040 2024-04-24T09:03:09.970 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:09.970 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":82,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:02:54.504455+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:03:09.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.973+0000 7efc4ca00640 1 -- 172.21.15.69:0/4197313881 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efc2c0a3bb0 msgr2=0x7efc2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:09.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.973+0000 7efc4ca00640 1 --2- 172.21.15.69:0/4197313881 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efc2c0a3bb0 0x7efc2c0a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efc3800bbe0 tx=0x7efc3800b390 comp rx=0 tx=0).stop 2024-04-24T09:03:09.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.973+0000 7efc4ca00640 1 -- 172.21.15.69:0/4197313881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc48076040 msgr2=0x7efc4819a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:09.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.973+0000 7efc4ca00640 1 --2- 172.21.15.69:0/4197313881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc48076040 0x7efc4819a740 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efc3000d950 tx=0x7efc3000de10 comp rx=0 tx=0).stop 2024-04-24T09:03:09.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.973+0000 7efc4ca00640 1 -- 172.21.15.69:0/4197313881 shutdown_connections 2024-04-24T09:03:09.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.973+0000 7efc4ca00640 1 --2- 172.21.15.69:0/4197313881 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efc2c0a3bb0 0x7efc2c0a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:09.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.973+0000 7efc4ca00640 1 --2- 172.21.15.69:0/4197313881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc48109d30 0x7efc481a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:09.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.973+0000 7efc4ca00640 1 --2- 172.21.15.69:0/4197313881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc48076040 0x7efc4819a740 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:09.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.973+0000 7efc4ca00640 1 --2- 172.21.15.69:0/4197313881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc48074a60 0x7efc4819a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:09.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.973+0000 7efc4ca00640 1 -- 172.21.15.69:0/4197313881 >> 172.21.15.69:0/4197313881 conn(0x7efc480fd3e0 msgr2=0x7efc48105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:09.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.974+0000 7efc4ca00640 1 -- 172.21.15.69:0/4197313881 shutdown_connections 2024-04-24T09:03:09.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:09.974+0000 7efc4ca00640 1 -- 172.21.15.69:0/4197313881 wait complete. 2024-04-24T09:03:09.994 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:57:37.377599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T09:03:09.994 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:03:10.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.194+0000 7f0aa9600640 1 -- 172.21.15.69:0/1690942402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aa4109730 msgr2=0x7f0aa4109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:10.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.194+0000 7f0aa9600640 1 --2- 172.21.15.69:0/1690942402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aa4109730 0x7f0aa4109bb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009900 tx=0x7f0a8c02a410 comp rx=0 tx=0).stop 2024-04-24T09:03:10.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.194+0000 7f0aa9600640 1 -- 172.21.15.69:0/1690942402 shutdown_connections 2024-04-24T09:03:10.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.194+0000 7f0aa9600640 1 --2- 172.21.15.69:0/1690942402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0aa410a0f0 0x7f0aa4110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.194+0000 7f0aa9600640 1 --2- 172.21.15.69:0/1690942402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aa4109730 0x7f0aa4109bb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.194+0000 7f0aa9600640 1 --2- 172.21.15.69:0/1690942402 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0aa4108540 0x7f0aa4108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.194+0000 7f0aa9600640 1 -- 172.21.15.69:0/1690942402 >> 172.21.15.69:0/1690942402 conn(0x7f0aa40774d0 msgr2=0x7f0aa4079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:10.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.194+0000 7f0aa9600640 1 -- 172.21.15.69:0/1690942402 shutdown_connections 2024-04-24T09:03:10.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.194+0000 7f0aa9600640 1 -- 172.21.15.69:0/1690942402 wait complete. 2024-04-24T09:03:10.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.195+0000 7f0aa9600640 1 Processor -- start 2024-04-24T09:03:10.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.195+0000 7f0aa9600640 1 -- start start 2024-04-24T09:03:10.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.195+0000 7f0aa9600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aa4108540 0x7f0aa419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:10.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.195+0000 7f0aa9600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0aa4109730 0x7f0aa419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:10.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.195+0000 7f0aa9600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0aa410a0f0 0x7f0aa41a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:10.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.195+0000 7f0aa9600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa4114080 con 0x7f0aa4108540 2024-04-24T09:03:10.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.195+0000 7f0aa9600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa4113f00 con 0x7f0aa410a0f0 2024-04-24T09:03:10.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.195+0000 7f0aa9600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa4114200 con 0x7f0aa4109730 2024-04-24T09:03:10.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.196+0000 7f0aa3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aa4108540 0x7f0aa419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:10.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.196+0000 7f0aa3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aa4108540 0x7f0aa419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36924/0 (socket says 172.21.15.69:36924) 2024-04-24T09:03:10.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.196+0000 7f0aa3e00640 1 -- 172.21.15.69:0/2549527254 learned_addr learned my addr 172.21.15.69:0/2549527254 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:10.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.196+0000 7f0a9b400640 1 --2- 172.21.15.69:0/2549527254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0aa4109730 0x7f0aa419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:10.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.196+0000 7f0aa8c00640 1 --2- 172.21.15.69:0/2549527254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0aa410a0f0 0x7f0aa41a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:10.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.196+0000 7f0aa3e00640 1 -- 172.21.15.69:0/2549527254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0aa4109730 msgr2=0x7f0aa419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:10.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.196+0000 7f0aa3e00640 1 --2- 172.21.15.69:0/2549527254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0aa4109730 0x7f0aa419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.196+0000 7f0aa3e00640 1 -- 172.21.15.69:0/2549527254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0aa410a0f0 msgr2=0x7f0aa41a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:10.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.196+0000 7f0aa3e00640 1 --2- 172.21.15.69:0/2549527254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0aa410a0f0 0x7f0aa41a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.196+0000 7f0aa3e00640 1 -- 172.21.15.69:0/2549527254 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa41a6290 con 0x7f0aa4108540 2024-04-24T09:03:10.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.196+0000 7f0aa8c00640 1 --2- 172.21.15.69:0/2549527254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0aa410a0f0 0x7f0aa41a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:10.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.196+0000 7f0aa3e00640 1 --2- 172.21.15.69:0/2549527254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aa4108540 0x7f0aa419e710 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400c970 tx=0x7f0a9400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:10.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.197+0000 7f0aa1600640 1 -- 172.21.15.69:0/2549527254 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94007c10 con 0x7f0aa4108540 2024-04-24T09:03:10.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.197+0000 7f0aa1600640 1 -- 172.21.15.69:0/2549527254 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a94007db0 con 0x7f0aa4108540 2024-04-24T09:03:10.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.197+0000 7f0aa9600640 1 -- 172.21.15.69:0/2549527254 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa41a6580 con 0x7f0aa4108540 2024-04-24T09:03:10.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.197+0000 7f0aa1600640 1 -- 172.21.15.69:0/2549527254 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9400f870 con 0x7f0aa4108540 2024-04-24T09:03:10.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.197+0000 7f0aa9600640 1 -- 172.21.15.69:0/2549527254 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa41a6ad0 con 0x7f0aa4108540 2024-04-24T09:03:10.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.198+0000 7f0aa9600640 1 -- 172.21.15.69:0/2549527254 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa410f860 con 0x7f0aa4108540 2024-04-24T09:03:10.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.200+0000 7f0aa1600640 1 -- 172.21.15.69:0/2549527254 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0a9400fa10 con 0x7f0aa4108540 2024-04-24T09:03:10.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.201+0000 7f0aa1600640 1 --2- 172.21.15.69:0/2549527254 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0a880a3bb0 0x7f0a880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:10.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.202+0000 7f0aa1600640 1 -- 172.21.15.69:0/2549527254 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0a940d0680 con 0x7f0aa4108540 2024-04-24T09:03:10.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.202+0000 7f0a9b400640 1 --2- 172.21.15.69:0/2549527254 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0a880a3bb0 0x7f0a880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:10.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.202+0000 7f0a9b400640 1 --2- 172.21.15.69:0/2549527254 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0a880a3bb0 0x7f0a880a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0aa419fb80 tx=0x7f0a8c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:10.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.204+0000 7f0aa1600640 1 -- 172.21.15.69:0/2549527254 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0a9409d580 con 0x7f0aa4108540 2024-04-24T09:03:10.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.404+0000 7f0aa9600640 1 -- 172.21.15.69:0/2549527254 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0aa4062c00 con 0x7f0aa4108540 2024-04-24T09:03:10.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.405+0000 7f0aa1600640 1 -- 172.21.15.69:0/2549527254 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f0a9409ce80 con 0x7f0aa4108540 2024-04-24T09:03:10.405 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-24T09:03:10.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.408+0000 7f0aa9600640 1 -- 172.21.15.69:0/2549527254 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0a880a3bb0 msgr2=0x7f0a880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:10.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.408+0000 7f0aa9600640 1 --2- 172.21.15.69:0/2549527254 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0a880a3bb0 0x7f0a880a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0aa419fb80 tx=0x7f0a8c0314a0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.408+0000 7f0aa9600640 1 -- 172.21.15.69:0/2549527254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aa4108540 msgr2=0x7f0aa419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:10.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.408+0000 7f0aa9600640 1 --2- 172.21.15.69:0/2549527254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aa4108540 0x7f0aa419e710 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400c970 tx=0x7f0a9400ce30 comp rx=0 tx=0).stop 2024-04-24T09:03:10.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.408+0000 7f0aa9600640 1 -- 172.21.15.69:0/2549527254 shutdown_connections 2024-04-24T09:03:10.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.408+0000 7f0aa9600640 1 --2- 172.21.15.69:0/2549527254 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0a880a3bb0 0x7f0a880a6070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.408+0000 7f0aa9600640 1 --2- 172.21.15.69:0/2549527254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0aa410a0f0 0x7f0aa41a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.408+0000 7f0aa9600640 1 --2- 172.21.15.69:0/2549527254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0aa4109730 0x7f0aa419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.408+0000 7f0aa9600640 1 --2- 172.21.15.69:0/2549527254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aa4108540 0x7f0aa419e710 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.408+0000 7f0aa9600640 1 -- 172.21.15.69:0/2549527254 >> 172.21.15.69:0/2549527254 conn(0x7f0aa40774d0 msgr2=0x7f0aa410ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:10.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.408+0000 7f0aa9600640 1 -- 172.21.15.69:0/2549527254 shutdown_connections 2024-04-24T09:03:10.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.408+0000 7f0aa9600640 1 -- 172.21.15.69:0/2549527254 wait complete. 2024-04-24T09:03:10.428 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:10.538 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:10 smithi130 ceph-mon[31553]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:10.538 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:10 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/4197313881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:03:10.538 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:10 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2549527254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:03:10.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.630+0000 7fc0ca600640 1 -- 172.21.15.69:0/3241925552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0c4105a10 msgr2=0x7fc0c4105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:10.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.630+0000 7fc0ca600640 1 --2- 172.21.15.69:0/3241925552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0c4105a10 0x7fc0c4105e90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009ca0 tx=0x7fc0bc02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.631+0000 7fc0ca600640 1 -- 172.21.15.69:0/3241925552 shutdown_connections 2024-04-24T09:03:10.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.631+0000 7fc0ca600640 1 --2- 172.21.15.69:0/3241925552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0c4105a10 0x7fc0c4105e90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.631+0000 7fc0ca600640 1 --2- 172.21.15.69:0/3241925552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0c4103060 0x7fc0c41054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.631+0000 7fc0ca600640 1 --2- 172.21.15.69:0/3241925552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0c41006f0 0x7fc0c4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.631+0000 7fc0ca600640 1 -- 172.21.15.69:0/3241925552 >> 172.21.15.69:0/3241925552 conn(0x7fc0c40fc2d0 msgr2=0x7fc0c40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:10.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.631+0000 7fc0ca600640 1 -- 172.21.15.69:0/3241925552 shutdown_connections 2024-04-24T09:03:10.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.631+0000 7fc0ca600640 1 -- 172.21.15.69:0/3241925552 wait complete. 2024-04-24T09:03:10.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.632+0000 7fc0ca600640 1 Processor -- start 2024-04-24T09:03:10.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.632+0000 7fc0ca600640 1 -- start start 2024-04-24T09:03:10.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.632+0000 7fc0ca600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0c41006f0 0x7fc0c419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:10.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.632+0000 7fc0ca600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0c4103060 0x7fc0c419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:10.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.632+0000 7fc0ca600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0c4105a10 0x7fc0c41a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:10.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.632+0000 7fc0ca600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c4114040 con 0x7fc0c4103060 2024-04-24T09:03:10.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.632+0000 7fc0ca600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c4113ec0 con 0x7fc0c4105a10 2024-04-24T09:03:10.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.632+0000 7fc0ca600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c41141c0 con 0x7fc0c41006f0 2024-04-24T09:03:10.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.632+0000 7fc0c3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0c4103060 0x7fc0c419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:10.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.632+0000 7fc0c3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0c4103060 0x7fc0c419eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36942/0 (socket says 172.21.15.69:36942) 2024-04-24T09:03:10.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.632+0000 7fc0c3e00640 1 -- 172.21.15.69:0/2951763822 learned_addr learned my addr 172.21.15.69:0/2951763822 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:10.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.633+0000 7fc0c9200640 1 --2- 172.21.15.69:0/2951763822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0c41006f0 0x7fc0c419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:10.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.633+0000 7fc0c9c00640 1 --2- 172.21.15.69:0/2951763822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0c4105a10 0x7fc0c41a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:10.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.633+0000 7fc0c3e00640 1 -- 172.21.15.69:0/2951763822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0c41006f0 msgr2=0x7fc0c419e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:10.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.633+0000 7fc0c3e00640 1 --2- 172.21.15.69:0/2951763822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0c41006f0 0x7fc0c419e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.633+0000 7fc0c3e00640 1 -- 172.21.15.69:0/2951763822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0c4105a10 msgr2=0x7fc0c41a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:10.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.633+0000 7fc0c3e00640 1 --2- 172.21.15.69:0/2951763822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0c4105a10 0x7fc0c41a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.633+0000 7fc0c3e00640 1 -- 172.21.15.69:0/2951763822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c41a6210 con 0x7fc0c4103060 2024-04-24T09:03:10.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.633+0000 7fc0c3e00640 1 --2- 172.21.15.69:0/2951763822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0c4103060 0x7fc0c419eb30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000c920 tx=0x7fc0b000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:10.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.633+0000 7fc0c1600640 1 -- 172.21.15.69:0/2951763822 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0007c70 con 0x7fc0c4103060 2024-04-24T09:03:10.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.633+0000 7fc0c1600640 1 -- 172.21.15.69:0/2951763822 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0b0007e10 con 0x7fc0c4103060 2024-04-24T09:03:10.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.633+0000 7fc0ca600640 1 -- 172.21.15.69:0/2951763822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c41a6500 con 0x7fc0c4103060 2024-04-24T09:03:10.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.633+0000 7fc0c1600640 1 -- 172.21.15.69:0/2951763822 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0018820 con 0x7fc0c4103060 2024-04-24T09:03:10.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.634+0000 7fc0ca600640 1 -- 172.21.15.69:0/2951763822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c41a6aa0 con 0x7fc0c4103060 2024-04-24T09:03:10.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.635+0000 7fc0ca600640 1 -- 172.21.15.69:0/2951763822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c40fddc0 con 0x7fc0c4103060 2024-04-24T09:03:10.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.637+0000 7fc0c1600640 1 -- 172.21.15.69:0/2951763822 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc0b00189c0 con 0x7fc0c4103060 2024-04-24T09:03:10.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.638+0000 7fc0c1600640 1 --2- 172.21.15.69:0/2951763822 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc0ac0a3c30 0x7fc0ac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:10.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.638+0000 7fc0c1600640 1 -- 172.21.15.69:0/2951763822 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc0b00d01a0 con 0x7fc0c4103060 2024-04-24T09:03:10.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.638+0000 7fc0c9200640 1 --2- 172.21.15.69:0/2951763822 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc0ac0a3c30 0x7fc0ac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:10.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.639+0000 7fc0c9200640 1 --2- 172.21.15.69:0/2951763822 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc0ac0a3c30 0x7fc0ac0a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009890 tx=0x7fc0b8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:10.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.642+0000 7fc0c1600640 1 -- 172.21.15.69:0/2951763822 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0b0014030 con 0x7fc0c4103060 2024-04-24T09:03:10.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:10 smithi081 ceph-mon[30961]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:10.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:10 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/4197313881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:03:10.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:10 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2549527254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:03:10.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:10 smithi069 ceph-mon[28170]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:10.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:10 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/4197313881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:03:10.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:10 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2549527254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:03:10.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.885+0000 7fc0ca600640 1 -- 172.21.15.69:0/2951763822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0c402d620 con 0x7fc0c4103060 2024-04-24T09:03:10.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.886+0000 7fc0c1600640 1 -- 172.21.15.69:0/2951763822 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fc0b009cf80 con 0x7fc0c4103060 2024-04-24T09:03:10.886 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:10.886 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-24T09:03:10.886 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 1 2024-04-24T09:03:10.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.889+0000 7fc0ca600640 1 -- 172.21.15.69:0/2951763822 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc0ac0a3c30 msgr2=0x7fc0ac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:10.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.889+0000 7fc0ca600640 1 --2- 172.21.15.69:0/2951763822 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc0ac0a3c30 0x7fc0ac0a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009890 tx=0x7fc0b8008040 comp rx=0 tx=0).stop 2024-04-24T09:03:10.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.889+0000 7fc0ca600640 1 -- 172.21.15.69:0/2951763822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0c4103060 msgr2=0x7fc0c419eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:10.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.889+0000 7fc0ca600640 1 --2- 172.21.15.69:0/2951763822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0c4103060 0x7fc0c419eb30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000c920 tx=0x7fc0b000cde0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.889+0000 7fc0ca600640 1 -- 172.21.15.69:0/2951763822 shutdown_connections 2024-04-24T09:03:10.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.889+0000 7fc0ca600640 1 --2- 172.21.15.69:0/2951763822 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc0ac0a3c30 0x7fc0ac0a60f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.889+0000 7fc0ca600640 1 --2- 172.21.15.69:0/2951763822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0c4105a10 0x7fc0c41a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.889+0000 7fc0ca600640 1 --2- 172.21.15.69:0/2951763822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0c4103060 0x7fc0c419eb30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.889+0000 7fc0ca600640 1 --2- 172.21.15.69:0/2951763822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0c41006f0 0x7fc0c419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:10.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.889+0000 7fc0ca600640 1 -- 172.21.15.69:0/2951763822 >> 172.21.15.69:0/2951763822 conn(0x7fc0c40fc2d0 msgr2=0x7fc0c4103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:10.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.889+0000 7fc0ca600640 1 -- 172.21.15.69:0/2951763822 shutdown_connections 2024-04-24T09:03:10.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:10.889+0000 7fc0ca600640 1 -- 172.21.15.69:0/2951763822 wait complete. 2024-04-24T09:03:11.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:11 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2951763822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:11.703 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 24 09:03:11 smithi130 systemd[1]: Starting Ceph mds.l for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:03:11.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:11 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2951763822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:11.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:11 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2951763822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:12.171 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 24 09:03:11 smithi130 podman[61886]: 2024-04-24 09:03:11.756995418 +0000 UTC m=+0.033102740 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:03:12.171 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 24 09:03:11 smithi130 podman[61886]: 2024-04-24 09:03:11.924516197 +0000 UTC m=+0.200623578 container create 0ea7a1311dc2e20018728a8402ece7102758cb0ca2a9d747982b6176816926ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T09:03:12.171 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 24 09:03:12 smithi130 podman[61886]: 2024-04-24 09:03:12.171028546 +0000 UTC m=+0.447135889 container init 0ea7a1311dc2e20018728a8402ece7102758cb0ca2a9d747982b6176816926ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-24T09:03:12.454 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 24 09:03:12 smithi130 podman[61886]: 2024-04-24 09:03:12.181763049 +0000 UTC m=+0.457870437 container start 0ea7a1311dc2e20018728a8402ece7102758cb0ca2a9d747982b6176816926ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-24T09:03:12.454 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 24 09:03:12 smithi130 ceph-mds[61906]: -- 172.21.15.130:0/1502273508 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5637c2909a40 con 0x5637c28dc800 2024-04-24T09:03:12.454 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 24 09:03:12 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-l[61902]: starting mds.l at 2024-04-24T09:03:12.454 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 24 09:03:12 smithi130 bash[61886]: 0ea7a1311dc2e20018728a8402ece7102758cb0ca2a9d747982b6176816926ab 2024-04-24T09:03:12.454 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 24 09:03:12 smithi130 systemd[1]: Started Ceph mds.l for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T09:03:12.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:12 smithi081 ceph-mon[30961]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:12.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:12 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.130:6832/1510374637,v1:172.21.15.130:6833/1510374637] up:boot 2024-04-24T09:03:12.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:12 smithi081 ceph-mon[30961]: fsmap 1 up:standby 2024-04-24T09:03:12.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:12 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:03:12.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:12 smithi069 ceph-mon[28170]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:12.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:12 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.130:6832/1510374637,v1:172.21.15.130:6833/1510374637] up:boot 2024-04-24T09:03:12.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:12 smithi069 ceph-mon[28170]: fsmap 1 up:standby 2024-04-24T09:03:12.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:12 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:03:12.909 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:12.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:12 smithi130 ceph-mon[31553]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:12.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:12 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.130:6832/1510374637,v1:172.21.15.130:6833/1510374637] up:boot 2024-04-24T09:03:12.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:12 smithi130 ceph-mon[31553]: fsmap 1 up:standby 2024-04-24T09:03:12.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:12 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:03:13.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.121+0000 7f1de3e00640 1 -- 172.21.15.69:0/2634668404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1de4102900 msgr2=0x7f1de4104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:13.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.121+0000 7f1de3e00640 1 --2- 172.21.15.69:0/2634668404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1de4102900 0x7f1de4104cf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009ca0 tx=0x7f1dd402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:13.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.121+0000 7f1de3e00640 1 -- 172.21.15.69:0/2634668404 shutdown_connections 2024-04-24T09:03:13.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.121+0000 7f1de3e00640 1 --2- 172.21.15.69:0/2634668404 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1de4107b60 0x7f1de4109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:13.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.121+0000 7f1de3e00640 1 --2- 172.21.15.69:0/2634668404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1de4105230 0x7f1de4107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:13.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.121+0000 7f1de3e00640 1 --2- 172.21.15.69:0/2634668404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1de4102900 0x7f1de4104cf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:13.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.121+0000 7f1de3e00640 1 -- 172.21.15.69:0/2634668404 >> 172.21.15.69:0/2634668404 conn(0x7f1de40fc2d0 msgr2=0x7f1de40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:13.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.121+0000 7f1de3e00640 1 -- 172.21.15.69:0/2634668404 shutdown_connections 2024-04-24T09:03:13.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.121+0000 7f1de3e00640 1 -- 172.21.15.69:0/2634668404 wait complete. 2024-04-24T09:03:13.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.122+0000 7f1de3e00640 1 Processor -- start 2024-04-24T09:03:13.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.122+0000 7f1de3e00640 1 -- start start 2024-04-24T09:03:13.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.122+0000 7f1de3e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1de4102900 0x7f1de4198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:13.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.122+0000 7f1de3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1de4105230 0x7f1de4198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:13.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.122+0000 7f1de3e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1de4107b60 0x7f1de419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:13.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.122+0000 7f1de3e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1de410da70 con 0x7f1de4105230 2024-04-24T09:03:13.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.122+0000 7f1de3e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1de410d8f0 con 0x7f1de4102900 2024-04-24T09:03:13.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.122+0000 7f1de3e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1de410dbf0 con 0x7f1de4107b60 2024-04-24T09:03:13.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.123+0000 7f1dda000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1de4105230 0x7f1de4198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:13.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.123+0000 7f1dda000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1de4105230 0x7f1de4198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52824/0 (socket says 172.21.15.69:52824) 2024-04-24T09:03:13.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.123+0000 7f1dda000640 1 -- 172.21.15.69:0/2655688644 learned_addr learned my addr 172.21.15.69:0/2655688644 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:13.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.123+0000 7f1de2a00640 1 --2- 172.21.15.69:0/2655688644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1de4102900 0x7f1de4198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:13.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.123+0000 7f1de3400640 1 --2- 172.21.15.69:0/2655688644 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1de4107b60 0x7f1de419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:13.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.123+0000 7f1dda000640 1 -- 172.21.15.69:0/2655688644 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1de4107b60 msgr2=0x7f1de419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:13.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.123+0000 7f1dda000640 1 --2- 172.21.15.69:0/2655688644 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1de4107b60 0x7f1de419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:13.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.123+0000 7f1dda000640 1 -- 172.21.15.69:0/2655688644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1de4102900 msgr2=0x7f1de4198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:13.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.123+0000 7f1dda000640 1 --2- 172.21.15.69:0/2655688644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1de4102900 0x7f1de4198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:13.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.123+0000 7f1dda000640 1 -- 172.21.15.69:0/2655688644 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de419fd60 con 0x7f1de4105230 2024-04-24T09:03:13.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.123+0000 7f1dda000640 1 --2- 172.21.15.69:0/2655688644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1de4105230 0x7f1de4198680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00d770 tx=0x7f1dcc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:13.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.124+0000 7f1ddbe00640 1 -- 172.21.15.69:0/2655688644 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc017070 con 0x7f1de4105230 2024-04-24T09:03:13.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.124+0000 7f1ddbe00640 1 -- 172.21.15.69:0/2655688644 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dcc009c30 con 0x7f1de4105230 2024-04-24T09:03:13.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.124+0000 7f1ddbe00640 1 -- 172.21.15.69:0/2655688644 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00f620 con 0x7f1de4105230 2024-04-24T09:03:13.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.124+0000 7f1de3e00640 1 -- 172.21.15.69:0/2655688644 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de419fff0 con 0x7f1de4105230 2024-04-24T09:03:13.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.124+0000 7f1de3e00640 1 -- 172.21.15.69:0/2655688644 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de41a0560 con 0x7f1de4105230 2024-04-24T09:03:13.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.125+0000 7f1de3e00640 1 -- 172.21.15.69:0/2655688644 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de40fddc0 con 0x7f1de4105230 2024-04-24T09:03:13.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.128+0000 7f1ddbe00640 1 -- 172.21.15.69:0/2655688644 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1dcc01a070 con 0x7f1de4105230 2024-04-24T09:03:13.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.129+0000 7f1ddbe00640 1 --2- 172.21.15.69:0/2655688644 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f1dc80a3c30 0x7f1dc80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:13.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.129+0000 7f1ddbe00640 1 -- 172.21.15.69:0/2655688644 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1dcc0cfaf0 con 0x7f1de4105230 2024-04-24T09:03:13.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.129+0000 7f1de2a00640 1 --2- 172.21.15.69:0/2655688644 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f1dc80a3c30 0x7f1dc80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:13.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.130+0000 7f1de2a00640 1 --2- 172.21.15.69:0/2655688644 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f1dc80a3c30 0x7f1dc80a60f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4031040 tx=0x7f1dd4000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:13.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.134+0000 7f1ddbe00640 1 -- 172.21.15.69:0/2655688644 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1dcc009de0 con 0x7f1de4105230 2024-04-24T09:03:13.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.378+0000 7f1de3e00640 1 -- 172.21.15.69:0/2655688644 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1de4062ca0 con 0x7f1de4105230 2024-04-24T09:03:13.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.379+0000 7f1ddbe00640 1 -- 172.21.15.69:0/2655688644 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f1dcc09c8d0 con 0x7f1de4105230 2024-04-24T09:03:13.378 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:13.379 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T09:03:13.379 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 2 2024-04-24T09:03:13.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.381+0000 7f1de3e00640 1 -- 172.21.15.69:0/2655688644 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f1dc80a3c30 msgr2=0x7f1dc80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:13.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.381+0000 7f1de3e00640 1 --2- 172.21.15.69:0/2655688644 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f1dc80a3c30 0x7f1dc80a60f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4031040 tx=0x7f1dd4000f40 comp rx=0 tx=0).stop 2024-04-24T09:03:13.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.381+0000 7f1de3e00640 1 -- 172.21.15.69:0/2655688644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1de4105230 msgr2=0x7f1de4198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:13.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.381+0000 7f1de3e00640 1 --2- 172.21.15.69:0/2655688644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1de4105230 0x7f1de4198680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00d770 tx=0x7f1dcc00dc30 comp rx=0 tx=0).stop 2024-04-24T09:03:13.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.381+0000 7f1de3e00640 1 -- 172.21.15.69:0/2655688644 shutdown_connections 2024-04-24T09:03:13.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.381+0000 7f1de3e00640 1 --2- 172.21.15.69:0/2655688644 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1de4107b60 0x7f1de419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:13.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.381+0000 7f1de3e00640 1 --2- 172.21.15.69:0/2655688644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1de4105230 0x7f1de4198680 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:13.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.381+0000 7f1de3e00640 1 --2- 172.21.15.69:0/2655688644 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f1dc80a3c30 0x7f1dc80a60f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:13.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.381+0000 7f1de3e00640 1 --2- 172.21.15.69:0/2655688644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1de4102900 0x7f1de4198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:13.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.381+0000 7f1de3e00640 1 -- 172.21.15.69:0/2655688644 >> 172.21.15.69:0/2655688644 conn(0x7f1de40fc2d0 msgr2=0x7f1de4103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:13.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.381+0000 7f1de3e00640 1 -- 172.21.15.69:0/2655688644 shutdown_connections 2024-04-24T09:03:13.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:13.381+0000 7f1de3e00640 1 -- 172.21.15.69:0/2655688644 wait complete. 2024-04-24T09:03:13.815 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:13 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:13.815 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:13 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:13.815 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:13 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:13.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:13 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:13.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:13 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:13.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:13 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:13.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:13 smithi081 ceph-mon[30961]: Deploying daemon mds.k on smithi081 2024-04-24T09:03:13.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:13 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2655688644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:13.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:13 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:13.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:13 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:13.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:13 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:13.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:13 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:13.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:13 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:13.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:13 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:13.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:13 smithi069 ceph-mon[28170]: Deploying daemon mds.k on smithi081 2024-04-24T09:03:13.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:13 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2655688644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:13.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:13 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:13.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:13 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:13.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:13 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:13.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:13 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:13.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:13 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:13.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:13 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:13.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:13 smithi130 ceph-mon[31553]: Deploying daemon mds.k on smithi081 2024-04-24T09:03:13.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:13 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2655688644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:14.793 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:14 smithi081 ceph-mon[30961]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:14.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:14 smithi069 ceph-mon[28170]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:14.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:14 smithi130 ceph-mon[31553]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:15.401 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.611+0000 7fd016800640 1 -- 172.21.15.69:0/2946659182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0100769c0 msgr2=0x7fd010076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.611+0000 7fd016800640 1 --2- 172.21.15.69:0/2946659182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0100769c0 0x7fd010076de0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd000009ca0 tx=0x7fd00002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.611+0000 7fd016800640 1 -- 172.21.15.69:0/2946659182 shutdown_connections 2024-04-24T09:03:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.611+0000 7fd016800640 1 --2- 172.21.15.69:0/2946659182 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0100785a0 0x7fd010078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.611+0000 7fd016800640 1 --2- 172.21.15.69:0/2946659182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd010077c00 0x7fd010078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.611+0000 7fd016800640 1 --2- 172.21.15.69:0/2946659182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0100769c0 0x7fd010076de0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.611+0000 7fd016800640 1 -- 172.21.15.69:0/2946659182 >> 172.21.15.69:0/2946659182 conn(0x7fd0100ff7f0 msgr2=0x7fd010101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.611+0000 7fd016800640 1 -- 172.21.15.69:0/2946659182 shutdown_connections 2024-04-24T09:03:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.611+0000 7fd016800640 1 -- 172.21.15.69:0/2946659182 wait complete. 2024-04-24T09:03:15.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.612+0000 7fd016800640 1 Processor -- start 2024-04-24T09:03:15.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.612+0000 7fd016800640 1 -- start start 2024-04-24T09:03:15.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.613+0000 7fd016800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd010077c00 0x7fd01019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:15.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.613+0000 7fd016800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0100785a0 0x7fd01019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:15.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.613+0000 7fd016800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0101a40e0 0x7fd0101a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:15.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.613+0000 7fd016800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd010114200 con 0x7fd0101a40e0 2024-04-24T09:03:15.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.613+0000 7fd016800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd010114080 con 0x7fd010077c00 2024-04-24T09:03:15.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.613+0000 7fd016800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd010114380 con 0x7fd0100785a0 2024-04-24T09:03:15.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.613+0000 7fd00fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0101a40e0 0x7fd0101a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:15.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.613+0000 7fd00fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0101a40e0 0x7fd0101a64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52842/0 (socket says 172.21.15.69:52842) 2024-04-24T09:03:15.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.613+0000 7fd00fe00640 1 -- 172.21.15.69:0/601259390 learned_addr learned my addr 172.21.15.69:0/601259390 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:15.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.613+0000 7fd00f400640 1 --2- 172.21.15.69:0/601259390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd010077c00 0x7fd01019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:15.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.613+0000 7fd00fe00640 1 -- 172.21.15.69:0/601259390 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0100785a0 msgr2=0x7fd01019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:15.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.613+0000 7fd00ea00640 1 --2- 172.21.15.69:0/601259390 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0100785a0 0x7fd01019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:15.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.613+0000 7fd00fe00640 1 --2- 172.21.15.69:0/601259390 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0100785a0 0x7fd01019f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:15.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.613+0000 7fd00fe00640 1 -- 172.21.15.69:0/601259390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd010077c00 msgr2=0x7fd01019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:15.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.613+0000 7fd00fe00640 1 --2- 172.21.15.69:0/601259390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd010077c00 0x7fd01019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:15.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.613+0000 7fd00fe00640 1 -- 172.21.15.69:0/601259390 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0101a6b40 con 0x7fd0101a40e0 2024-04-24T09:03:15.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.614+0000 7fd00fe00640 1 --2- 172.21.15.69:0/601259390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0101a40e0 0x7fd0101a64d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd00800e9b0 tx=0x7fd00800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:15.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.614+0000 7fd007e00640 1 -- 172.21.15.69:0/601259390 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00800cd80 con 0x7fd0101a40e0 2024-04-24T09:03:15.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.614+0000 7fd007e00640 1 -- 172.21.15.69:0/601259390 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0080035f0 con 0x7fd0101a40e0 2024-04-24T09:03:15.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.614+0000 7fd007e00640 1 -- 172.21.15.69:0/601259390 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008010690 con 0x7fd0101a40e0 2024-04-24T09:03:15.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.614+0000 7fd016800640 1 -- 172.21.15.69:0/601259390 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0101a6e30 con 0x7fd0101a40e0 2024-04-24T09:03:15.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.614+0000 7fd016800640 1 -- 172.21.15.69:0/601259390 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0101a72a0 con 0x7fd0101a40e0 2024-04-24T09:03:15.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.615+0000 7fd016800640 1 -- 172.21.15.69:0/601259390 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd010110eb0 con 0x7fd0101a40e0 2024-04-24T09:03:15.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.617+0000 7fd007e00640 1 -- 172.21.15.69:0/601259390 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd0080037a0 con 0x7fd0101a40e0 2024-04-24T09:03:15.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.619+0000 7fd007e00640 1 --2- 172.21.15.69:0/601259390 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcff40a3bb0 0x7fcff40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:15.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.619+0000 7fd007e00640 1 -- 172.21.15.69:0/601259390 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd0080d0420 con 0x7fd0101a40e0 2024-04-24T09:03:15.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.619+0000 7fd00f400640 1 --2- 172.21.15.69:0/601259390 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcff40a3bb0 0x7fcff40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:15.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.620+0000 7fd00f400640 1 --2- 172.21.15.69:0/601259390 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcff40a3bb0 0x7fcff40a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd00002aeb0 tx=0x7fd000000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:15.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.623+0000 7fd007e00640 1 -- 172.21.15.69:0/601259390 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd00809d200 con 0x7fd0101a40e0 2024-04-24T09:03:15.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.868+0000 7fd016800640 1 -- 172.21.15.69:0/601259390 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd01002d620 con 0x7fd0101a40e0 2024-04-24T09:03:15.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.869+0000 7fd007e00640 1 -- 172.21.15.69:0/601259390 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fd00809cb00 con 0x7fd0101a40e0 2024-04-24T09:03:15.868 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:15.868 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T09:03:15.869 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 2 2024-04-24T09:03:15.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.871+0000 7fd016800640 1 -- 172.21.15.69:0/601259390 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcff40a3bb0 msgr2=0x7fcff40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:15.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.871+0000 7fd016800640 1 --2- 172.21.15.69:0/601259390 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcff40a3bb0 0x7fcff40a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd00002aeb0 tx=0x7fd000000f40 comp rx=0 tx=0).stop 2024-04-24T09:03:15.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.871+0000 7fd016800640 1 -- 172.21.15.69:0/601259390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0101a40e0 msgr2=0x7fd0101a64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:15.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.871+0000 7fd016800640 1 --2- 172.21.15.69:0/601259390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0101a40e0 0x7fd0101a64d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd00800e9b0 tx=0x7fd00800ee70 comp rx=0 tx=0).stop 2024-04-24T09:03:15.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.872+0000 7fd016800640 1 -- 172.21.15.69:0/601259390 shutdown_connections 2024-04-24T09:03:15.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.872+0000 7fd016800640 1 --2- 172.21.15.69:0/601259390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0101a40e0 0x7fd0101a64d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:15.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.872+0000 7fd016800640 1 --2- 172.21.15.69:0/601259390 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0100785a0 0x7fd01019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:15.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.872+0000 7fd016800640 1 --2- 172.21.15.69:0/601259390 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcff40a3bb0 0x7fcff40a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:15.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.872+0000 7fd016800640 1 --2- 172.21.15.69:0/601259390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd010077c00 0x7fd01019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:15.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.872+0000 7fd016800640 1 -- 172.21.15.69:0/601259390 >> 172.21.15.69:0/601259390 conn(0x7fd0100ff7f0 msgr2=0x7fd010101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:15.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.872+0000 7fd016800640 1 -- 172.21.15.69:0/601259390 shutdown_connections 2024-04-24T09:03:15.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:15.872+0000 7fd016800640 1 -- 172.21.15.69:0/601259390 wait complete. 2024-04-24T09:03:16.185 INFO:journalctl@ceph.mds.k.smithi081.stdout:Apr 24 09:03:16 smithi081 podman[60496]: 2024-04-24 09:03:16.020206869 +0000 UTC m=+0.163659892 container create ec97d78eb5b7728d88713380c054eb5ba85f3bea6dd6e3e742f0cbccb9cceb92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-k, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:03:16.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:16 smithi081 ceph-mon[30961]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:16.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:16 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/601259390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:16.567 INFO:journalctl@ceph.mds.k.smithi081.stdout:Apr 24 09:03:16 smithi081 podman[60496]: 2024-04-24 09:03:16.282526046 +0000 UTC m=+0.425979069 container init ec97d78eb5b7728d88713380c054eb5ba85f3bea6dd6e3e742f0cbccb9cceb92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-24T09:03:16.567 INFO:journalctl@ceph.mds.k.smithi081.stdout:Apr 24 09:03:16 smithi081 podman[60496]: 2024-04-24 09:03:16.292401476 +0000 UTC m=+0.435854459 container start ec97d78eb5b7728d88713380c054eb5ba85f3bea6dd6e3e742f0cbccb9cceb92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-24T09:03:16.567 INFO:journalctl@ceph.mds.k.smithi081.stdout:Apr 24 09:03:16 smithi081 ceph-mds[60522]: -- 172.21.15.81:0/2352949685 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5592874bfa40 con 0x559287493000 2024-04-24T09:03:16.567 INFO:journalctl@ceph.mds.k.smithi081.stdout:Apr 24 09:03:16 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-k[60516]: starting mds.k at 2024-04-24T09:03:16.568 INFO:journalctl@ceph.mds.k.smithi081.stdout:Apr 24 09:03:16 smithi081 bash[60496]: ec97d78eb5b7728d88713380c054eb5ba85f3bea6dd6e3e742f0cbccb9cceb92 2024-04-24T09:03:16.568 INFO:journalctl@ceph.mds.k.smithi081.stdout:Apr 24 09:03:16 smithi081 systemd[1]: Started Ceph mds.k for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T09:03:16.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:16 smithi069 ceph-mon[28170]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:16.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:16 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/601259390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:16.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:16 smithi130 ceph-mon[31553]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:16.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:16 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/601259390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:17.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:17 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.81:6832/3240654002,v1:172.21.15.81:6833/3240654002] up:boot 2024-04-24T09:03:17.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:17 smithi069 ceph-mon[28170]: fsmap 2 up:standby 2024-04-24T09:03:17.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T09:03:17.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:17.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:17.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:17.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:17.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:17.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:17 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:17 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.81:6832/3240654002,v1:172.21.15.81:6833/3240654002] up:boot 2024-04-24T09:03:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:17 smithi081 ceph-mon[30961]: fsmap 2 up:standby 2024-04-24T09:03:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T09:03:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:17.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:17.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:17 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:17.893 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:17 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.81:6832/3240654002,v1:172.21.15.81:6833/3240654002] up:boot 2024-04-24T09:03:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:17 smithi130 ceph-mon[31553]: fsmap 2 up:standby 2024-04-24T09:03:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T09:03:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:17 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:18.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.096+0000 7f009d600640 1 -- 172.21.15.69:0/1017750108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00980785a0 msgr2=0x7f0098078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:18.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.096+0000 7f009d600640 1 --2- 172.21.15.69:0/1017750108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00980785a0 0x7f0098078e40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0090009ca0 tx=0x7f009002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.096+0000 7f009d600640 1 -- 172.21.15.69:0/1017750108 shutdown_connections 2024-04-24T09:03:18.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.096+0000 7f009d600640 1 --2- 172.21.15.69:0/1017750108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00980785a0 0x7f0098078e40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.096+0000 7f009d600640 1 --2- 172.21.15.69:0/1017750108 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0098077c00 0x7f0098078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.096+0000 7f009d600640 1 --2- 172.21.15.69:0/1017750108 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00980769c0 0x7f0098076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.096+0000 7f009d600640 1 -- 172.21.15.69:0/1017750108 >> 172.21.15.69:0/1017750108 conn(0x7f00980ff7f0 msgr2=0x7f0098101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:18.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.096+0000 7f009d600640 1 -- 172.21.15.69:0/1017750108 shutdown_connections 2024-04-24T09:03:18.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.096+0000 7f009d600640 1 -- 172.21.15.69:0/1017750108 wait complete. 2024-04-24T09:03:18.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.097+0000 7f009d600640 1 Processor -- start 2024-04-24T09:03:18.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.098+0000 7f009d600640 1 -- start start 2024-04-24T09:03:18.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.098+0000 7f009d600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00980769c0 0x7f009819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:18.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.098+0000 7f009d600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0098077c00 0x7f009819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:18.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.098+0000 7f009d600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00980785a0 0x7f00981a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:18.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.098+0000 7f009d600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f00981141e0 con 0x7f00980769c0 2024-04-24T09:03:18.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.098+0000 7f009d600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0098114060 con 0x7f0098077c00 2024-04-24T09:03:18.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.098+0000 7f009d600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0098114360 con 0x7f00980785a0 2024-04-24T09:03:18.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.098+0000 7f0096a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00980769c0 0x7f009819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:18.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.098+0000 7f0096a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00980769c0 0x7f009819e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52872/0 (socket says 172.21.15.69:52872) 2024-04-24T09:03:18.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.098+0000 7f0096a00640 1 -- 172.21.15.69:0/958852909 learned_addr learned my addr 172.21.15.69:0/958852909 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:18.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.099+0000 7f0097400640 1 --2- 172.21.15.69:0/958852909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00980785a0 0x7f00981a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:18.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.099+0000 7f008fe00640 1 --2- 172.21.15.69:0/958852909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0098077c00 0x7f009819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:18.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.099+0000 7f0096a00640 1 -- 172.21.15.69:0/958852909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00980785a0 msgr2=0x7f00981a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:18.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.099+0000 7f0096a00640 1 --2- 172.21.15.69:0/958852909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00980785a0 0x7f00981a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.099+0000 7f0096a00640 1 -- 172.21.15.69:0/958852909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0098077c00 msgr2=0x7f009819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:18.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.099+0000 7f0096a00640 1 --2- 172.21.15.69:0/958852909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0098077c00 0x7f009819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.099+0000 7f0096a00640 1 -- 172.21.15.69:0/958852909 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00981a63a0 con 0x7f00980769c0 2024-04-24T09:03:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.099+0000 7f008fe00640 1 --2- 172.21.15.69:0/958852909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0098077c00 0x7f009819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.099+0000 7f0096a00640 1 --2- 172.21.15.69:0/958852909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00980769c0 0x7f009819e780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f008800d770 tx=0x7f008800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.099+0000 7f008f400640 1 -- 172.21.15.69:0/958852909 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0088017070 con 0x7f00980769c0 2024-04-24T09:03:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.099+0000 7f009d600640 1 -- 172.21.15.69:0/958852909 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00981a6690 con 0x7f00980769c0 2024-04-24T09:03:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.099+0000 7f008f400640 1 -- 172.21.15.69:0/958852909 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00880054d0 con 0x7f00980769c0 2024-04-24T09:03:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.100+0000 7f009d600640 1 -- 172.21.15.69:0/958852909 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00981a6b90 con 0x7f00980769c0 2024-04-24T09:03:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.100+0000 7f008f400640 1 -- 172.21.15.69:0/958852909 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0088020060 con 0x7f00980769c0 2024-04-24T09:03:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.101+0000 7f009d600640 1 -- 172.21.15.69:0/958852909 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0098110ce0 con 0x7f00980769c0 2024-04-24T09:03:18.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.103+0000 7f008f400640 1 -- 172.21.15.69:0/958852909 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0088004c90 con 0x7f00980769c0 2024-04-24T09:03:18.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.104+0000 7f008f400640 1 --2- 172.21.15.69:0/958852909 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f007c0a3bb0 0x7f007c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:18.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.105+0000 7f008f400640 1 -- 172.21.15.69:0/958852909 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f00880cfe90 con 0x7f00980769c0 2024-04-24T09:03:18.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.105+0000 7f008fe00640 1 --2- 172.21.15.69:0/958852909 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f007c0a3bb0 0x7f007c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:18.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.105+0000 7f008fe00640 1 --2- 172.21.15.69:0/958852909 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f007c0a3bb0 0x7f007c0a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f009819fc90 tx=0x7f0080005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:18.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.109+0000 7f008f400640 1 -- 172.21.15.69:0/958852909 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f008809cc70 con 0x7f00980769c0 2024-04-24T09:03:18.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.361+0000 7f009d600640 1 -- 172.21.15.69:0/958852909 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f009802d620 con 0x7f00980769c0 2024-04-24T09:03:18.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.362+0000 7f008f400640 1 -- 172.21.15.69:0/958852909 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f00880d18c0 con 0x7f00980769c0 2024-04-24T09:03:18.362 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:18.362 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T09:03:18.363 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 3 2024-04-24T09:03:18.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.365+0000 7f009d600640 1 -- 172.21.15.69:0/958852909 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f007c0a3bb0 msgr2=0x7f007c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:18.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.365+0000 7f009d600640 1 --2- 172.21.15.69:0/958852909 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f007c0a3bb0 0x7f007c0a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f009819fc90 tx=0x7f0080005d90 comp rx=0 tx=0).stop 2024-04-24T09:03:18.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.365+0000 7f009d600640 1 -- 172.21.15.69:0/958852909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00980769c0 msgr2=0x7f009819e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:18.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.365+0000 7f009d600640 1 --2- 172.21.15.69:0/958852909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00980769c0 0x7f009819e780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f008800d770 tx=0x7f008800dc30 comp rx=0 tx=0).stop 2024-04-24T09:03:18.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.365+0000 7f009d600640 1 -- 172.21.15.69:0/958852909 shutdown_connections 2024-04-24T09:03:18.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.365+0000 7f009d600640 1 --2- 172.21.15.69:0/958852909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00980785a0 0x7f00981a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.365+0000 7f009d600640 1 --2- 172.21.15.69:0/958852909 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f007c0a3bb0 0x7f007c0a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.365+0000 7f009d600640 1 --2- 172.21.15.69:0/958852909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0098077c00 0x7f009819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.365+0000 7f009d600640 1 --2- 172.21.15.69:0/958852909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00980769c0 0x7f009819e780 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.365+0000 7f009d600640 1 -- 172.21.15.69:0/958852909 >> 172.21.15.69:0/958852909 conn(0x7f00980ff7f0 msgr2=0x7f0098100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:18.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.366+0000 7f009d600640 1 -- 172.21.15.69:0/958852909 shutdown_connections 2024-04-24T09:03:18.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:18.366+0000 7f009d600640 1 -- 172.21.15.69:0/958852909 wait complete. 2024-04-24T09:03:18.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:18 smithi081 ceph-mon[30961]: Deploying daemon mds.j on smithi069 2024-04-24T09:03:18.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:18 smithi081 ceph-mon[30961]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:18.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:18 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/958852909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:18.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:18 smithi069 ceph-mon[28170]: Deploying daemon mds.j on smithi069 2024-04-24T09:03:18.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:18 smithi069 ceph-mon[28170]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:18.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:18 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/958852909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:18.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:18 smithi130 ceph-mon[31553]: Deploying daemon mds.j on smithi069 2024-04-24T09:03:18.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:18 smithi130 ceph-mon[31553]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:18.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:18 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/958852909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:19.724 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 24 09:03:19 smithi069 systemd[1]: Starting Ceph mds.j for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:03:20.342 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 24 09:03:20 smithi069 podman[65879]: 2024-04-24 09:03:19.987559177 +0000 UTC m=+0.037584414 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:03:20.342 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 24 09:03:20 smithi069 podman[65879]: 2024-04-24 09:03:20.117405433 +0000 UTC m=+0.167430707 container create 85b5983e6d728bd153c1e096cc747f11a0ff7135a070fea97d53e879ca500756 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T09:03:20.384 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:20.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.582+0000 7f21c2c00640 1 -- 172.21.15.69:0/1757981140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc107d30 msgr2=0x7f21bc10a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:20.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.582+0000 7f21c2c00640 1 --2- 172.21.15.69:0/1757981140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc107d30 0x7f21bc10a180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009ca0 tx=0x7f21b402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:20.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.582+0000 7f21c2c00640 1 -- 172.21.15.69:0/1757981140 shutdown_connections 2024-04-24T09:03:20.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.582+0000 7f21c2c00640 1 --2- 172.21.15.69:0/1757981140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc107d30 0x7f21bc10a180 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:20.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.582+0000 7f21c2c00640 1 --2- 172.21.15.69:0/1757981140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21bc105400 0x7f21bc1077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:20.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.582+0000 7f21c2c00640 1 --2- 172.21.15.69:0/1757981140 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f21bc102ad0 0x7f21bc104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:20.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.582+0000 7f21c2c00640 1 -- 172.21.15.69:0/1757981140 >> 172.21.15.69:0/1757981140 conn(0x7f21bc0fc4e0 msgr2=0x7f21bc0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:20.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.582+0000 7f21c2c00640 1 -- 172.21.15.69:0/1757981140 shutdown_connections 2024-04-24T09:03:20.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.582+0000 7f21c2c00640 1 -- 172.21.15.69:0/1757981140 wait complete. 2024-04-24T09:03:20.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.583+0000 7f21c2c00640 1 Processor -- start 2024-04-24T09:03:20.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.583+0000 7f21c2c00640 1 -- start start 2024-04-24T09:03:20.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.583+0000 7f21c2c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc102ad0 0x7f21bc102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:20.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.583+0000 7f21c2c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21bc105400 0x7f21bc100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:20.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.583+0000 7f21c2c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f21bc107d30 0x7f21bc100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:20.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.584+0000 7f21c2c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc10db90 con 0x7f21bc102ad0 2024-04-24T09:03:20.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.584+0000 7f21c2c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc10da10 con 0x7f21bc105400 2024-04-24T09:03:20.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.584+0000 7f21c2c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc10dd10 con 0x7f21bc107d30 2024-04-24T09:03:20.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.584+0000 7f21bbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc102ad0 0x7f21bc102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:20.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.584+0000 7f21bbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc102ad0 0x7f21bc102440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52900/0 (socket says 172.21.15.69:52900) 2024-04-24T09:03:20.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.584+0000 7f21bbe00640 1 -- 172.21.15.69:0/2461958137 learned_addr learned my addr 172.21.15.69:0/2461958137 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:20.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.584+0000 7f21c0c00640 1 --2- 172.21.15.69:0/2461958137 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f21bc107d30 0x7f21bc100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:20.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.584+0000 7f21bb400640 1 --2- 172.21.15.69:0/2461958137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21bc105400 0x7f21bc100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:20.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.584+0000 7f21bbe00640 1 -- 172.21.15.69:0/2461958137 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f21bc107d30 msgr2=0x7f21bc100fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:20.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.584+0000 7f21bbe00640 1 --2- 172.21.15.69:0/2461958137 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f21bc107d30 0x7f21bc100fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:20.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.584+0000 7f21bbe00640 1 -- 172.21.15.69:0/2461958137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21bc105400 msgr2=0x7f21bc100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:20.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.584+0000 7f21bbe00640 1 --2- 172.21.15.69:0/2461958137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21bc105400 0x7f21bc100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:20.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.584+0000 7f21bbe00640 1 -- 172.21.15.69:0/2461958137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21bc101850 con 0x7f21bc102ad0 2024-04-24T09:03:20.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.584+0000 7f21bb400640 1 --2- 172.21.15.69:0/2461958137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21bc105400 0x7f21bc100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:20.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.585+0000 7f21bbe00640 1 --2- 172.21.15.69:0/2461958137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc102ad0 0x7f21bc102440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00c970 tx=0x7f21ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:20.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.585+0000 7f21b8c00640 1 -- 172.21.15.69:0/2461958137 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac007bb0 con 0x7f21bc102ad0 2024-04-24T09:03:20.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.585+0000 7f21c2c00640 1 -- 172.21.15.69:0/2461958137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21bc1a2320 con 0x7f21bc102ad0 2024-04-24T09:03:20.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.585+0000 7f21b8c00640 1 -- 172.21.15.69:0/2461958137 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21ac007d50 con 0x7f21bc102ad0 2024-04-24T09:03:20.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.585+0000 7f21b8c00640 1 -- 172.21.15.69:0/2461958137 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac00f450 con 0x7f21bc102ad0 2024-04-24T09:03:20.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.585+0000 7f21c2c00640 1 -- 172.21.15.69:0/2461958137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21bc1a2820 con 0x7f21bc102ad0 2024-04-24T09:03:20.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.586+0000 7f21c2c00640 1 -- 172.21.15.69:0/2461958137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21bc0fdfd0 con 0x7f21bc102ad0 2024-04-24T09:03:20.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.588+0000 7f21b8c00640 1 -- 172.21.15.69:0/2461958137 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f21ac004f20 con 0x7f21bc102ad0 2024-04-24T09:03:20.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.590+0000 7f21b8c00640 1 --2- 172.21.15.69:0/2461958137 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f21a00a3bb0 0x7f21a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:20.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.590+0000 7f21b8c00640 1 -- 172.21.15.69:0/2461958137 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f21ac0cfe90 con 0x7f21bc102ad0 2024-04-24T09:03:20.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.590+0000 7f21bb400640 1 --2- 172.21.15.69:0/2461958137 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f21a00a3bb0 0x7f21a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.591+0000 7f21bb400640 1 --2- 172.21.15.69:0/2461958137 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f21a00a3bb0 0x7f21a00a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f21bc1021b0 tx=0x7f21a4005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:20.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.595+0000 7f21b8c00640 1 -- 172.21.15.69:0/2461958137 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f21ac014030 con 0x7f21bc102ad0 2024-04-24T09:03:20.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:20 smithi081 ceph-mon[30961]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:20.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:20 smithi069 ceph-mon[28170]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:20.841 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 24 09:03:20 smithi069 podman[65879]: 2024-04-24 09:03:20.396380466 +0000 UTC m=+0.446405747 container init 85b5983e6d728bd153c1e096cc747f11a0ff7135a070fea97d53e879ca500756 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD) 2024-04-24T09:03:20.841 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 24 09:03:20 smithi069 podman[65879]: 2024-04-24 09:03:20.406589851 +0000 UTC m=+0.456615103 container start 85b5983e6d728bd153c1e096cc747f11a0ff7135a070fea97d53e879ca500756 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-j, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:03:20.841 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 24 09:03:20 smithi069 ceph-mds[65902]: -- 172.21.15.69:0/255426006 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f1cb8a3a40 con 0x55f1cb876c00 2024-04-24T09:03:20.841 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 24 09:03:20 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-j[65894]: starting mds.j at 2024-04-24T09:03:20.841 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 24 09:03:20 smithi069 bash[65879]: 85b5983e6d728bd153c1e096cc747f11a0ff7135a070fea97d53e879ca500756 2024-04-24T09:03:20.841 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 24 09:03:20 smithi069 systemd[1]: Started Ceph mds.j for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T09:03:20.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.867+0000 7f21c2c00640 1 -- 172.21.15.69:0/2461958137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21bc02d620 con 0x7f21bc102ad0 2024-04-24T09:03:20.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.868+0000 7f21b8c00640 1 -- 172.21.15.69:0/2461958137 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f21ac0cfc00 con 0x7f21bc102ad0 2024-04-24T09:03:20.868 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:20.868 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T09:03:20.868 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 4 2024-04-24T09:03:20.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.870+0000 7f21c2c00640 1 -- 172.21.15.69:0/2461958137 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f21a00a3bb0 msgr2=0x7f21a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:20.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.870+0000 7f21c2c00640 1 --2- 172.21.15.69:0/2461958137 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f21a00a3bb0 0x7f21a00a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f21bc1021b0 tx=0x7f21a4005d90 comp rx=0 tx=0).stop 2024-04-24T09:03:20.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.870+0000 7f21c2c00640 1 -- 172.21.15.69:0/2461958137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc102ad0 msgr2=0x7f21bc102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:20.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.870+0000 7f21c2c00640 1 --2- 172.21.15.69:0/2461958137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc102ad0 0x7f21bc102440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00c970 tx=0x7f21ac00ce30 comp rx=0 tx=0).stop 2024-04-24T09:03:20.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.870+0000 7f21c2c00640 1 -- 172.21.15.69:0/2461958137 shutdown_connections 2024-04-24T09:03:20.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.870+0000 7f21c2c00640 1 --2- 172.21.15.69:0/2461958137 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f21bc107d30 0x7f21bc100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:20.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.870+0000 7f21c2c00640 1 --2- 172.21.15.69:0/2461958137 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f21a00a3bb0 0x7f21a00a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:20.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.870+0000 7f21c2c00640 1 --2- 172.21.15.69:0/2461958137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21bc105400 0x7f21bc100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:20.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.870+0000 7f21c2c00640 1 --2- 172.21.15.69:0/2461958137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21bc102ad0 0x7f21bc102440 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:20.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.870+0000 7f21c2c00640 1 -- 172.21.15.69:0/2461958137 >> 172.21.15.69:0/2461958137 conn(0x7f21bc0fc4e0 msgr2=0x7f21bc103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:20.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.871+0000 7f21c2c00640 1 -- 172.21.15.69:0/2461958137 shutdown_connections 2024-04-24T09:03:20.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:20.871+0000 7f21c2c00640 1 -- 172.21.15.69:0/2461958137 wait complete. 2024-04-24T09:03:20.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:20 smithi130 ceph-mon[31553]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:21.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:21 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.69:6834/106106849,v1:172.21.15.69:6835/106106849] up:boot 2024-04-24T09:03:21.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:21 smithi081 ceph-mon[30961]: fsmap 3 up:standby 2024-04-24T09:03:21.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:21 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T09:03:21.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:21 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:21.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:21 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:21.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:21 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:21.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:21 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:21.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:21 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:21.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:21 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:21.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:21 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2461958137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:21.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:21 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.69:6834/106106849,v1:172.21.15.69:6835/106106849] up:boot 2024-04-24T09:03:21.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:21 smithi069 ceph-mon[28170]: fsmap 3 up:standby 2024-04-24T09:03:21.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:21 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T09:03:21.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:21 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:21.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:21 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:21.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:21 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:21.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:21 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:21.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:21 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:21.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:21 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:21.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:21 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2461958137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:21.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:21 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.69:6834/106106849,v1:172.21.15.69:6835/106106849] up:boot 2024-04-24T09:03:21.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:21 smithi130 ceph-mon[31553]: fsmap 3 up:standby 2024-04-24T09:03:21.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:21 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T09:03:21.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:21 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:21.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:21 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:21.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:21 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:21.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:21 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:21.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:21 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:21.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:21 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:21.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:21 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2461958137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:22.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:22 smithi081 ceph-mon[30961]: Deploying daemon mds.d on smithi069 2024-04-24T09:03:22.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:22 smithi081 ceph-mon[30961]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:22.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:22 smithi069 ceph-mon[28170]: Deploying daemon mds.d on smithi069 2024-04-24T09:03:22.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:22 smithi069 ceph-mon[28170]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:22.889 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:22.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:22 smithi130 ceph-mon[31553]: Deploying daemon mds.d on smithi069 2024-04-24T09:03:22.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:22 smithi130 ceph-mon[31553]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:23.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.101+0000 7f2eee200640 1 -- 172.21.15.69:0/2969448259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ee8105470 msgr2=0x7f2ee81058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:23.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.101+0000 7f2eee200640 1 --2- 172.21.15.69:0/2969448259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ee8105470 0x7f2ee81058f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009900 tx=0x7f2ed402a410 comp rx=0 tx=0).stop 2024-04-24T09:03:23.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.101+0000 7f2eee200640 1 -- 172.21.15.69:0/2969448259 shutdown_connections 2024-04-24T09:03:23.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.101+0000 7f2eee200640 1 --2- 172.21.15.69:0/2969448259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ee8105e30 0x7f2ee810c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:23.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.101+0000 7f2eee200640 1 --2- 172.21.15.69:0/2969448259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ee8105470 0x7f2ee81058f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:23.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.101+0000 7f2eee200640 1 --2- 172.21.15.69:0/2969448259 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ee8104280 0x7f2ee81046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:23.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.101+0000 7f2eee200640 1 -- 172.21.15.69:0/2969448259 >> 172.21.15.69:0/2969448259 conn(0x7f2ee80ff7f0 msgr2=0x7f2ee8101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:23.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.101+0000 7f2eee200640 1 -- 172.21.15.69:0/2969448259 shutdown_connections 2024-04-24T09:03:23.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.101+0000 7f2eee200640 1 -- 172.21.15.69:0/2969448259 wait complete. 2024-04-24T09:03:23.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.102+0000 7f2eee200640 1 Processor -- start 2024-04-24T09:03:23.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.102+0000 7f2eee200640 1 -- start start 2024-04-24T09:03:23.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.102+0000 7f2eee200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ee8104280 0x7f2ee819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:23.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.102+0000 7f2eee200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ee8105470 0x7f2ee819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:23.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.102+0000 7f2eee200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ee8105e30 0x7f2ee81a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:23.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.102+0000 7f2eee200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee810fdc0 con 0x7f2ee8105e30 2024-04-24T09:03:23.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.102+0000 7f2eee200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee810fc40 con 0x7f2ee8104280 2024-04-24T09:03:23.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.102+0000 7f2eee200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee810ff40 con 0x7f2ee8105470 2024-04-24T09:03:23.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.103+0000 7f2ee7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ee8105e30 0x7f2ee81a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:23.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.103+0000 7f2ee7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ee8105e30 0x7f2ee81a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36316/0 (socket says 172.21.15.69:36316) 2024-04-24T09:03:23.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.103+0000 7f2ee7e00640 1 -- 172.21.15.69:0/2983288877 learned_addr learned my addr 172.21.15.69:0/2983288877 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:23.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.103+0000 7f2ee6a00640 1 --2- 172.21.15.69:0/2983288877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ee8105470 0x7f2ee819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:23.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.103+0000 7f2ee7400640 1 --2- 172.21.15.69:0/2983288877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ee8104280 0x7f2ee819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:23.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.103+0000 7f2ee7e00640 1 -- 172.21.15.69:0/2983288877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ee8105470 msgr2=0x7f2ee819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:23.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.103+0000 7f2ee7e00640 1 --2- 172.21.15.69:0/2983288877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ee8105470 0x7f2ee819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:23.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.103+0000 7f2ee7e00640 1 -- 172.21.15.69:0/2983288877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ee8104280 msgr2=0x7f2ee819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:23.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.103+0000 7f2ee7e00640 1 --2- 172.21.15.69:0/2983288877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ee8104280 0x7f2ee819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:23.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.104+0000 7f2ee7e00640 1 -- 172.21.15.69:0/2983288877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee81a20c0 con 0x7f2ee8105e30 2024-04-24T09:03:23.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.104+0000 7f2ee7e00640 1 --2- 172.21.15.69:0/2983288877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ee8105e30 0x7f2ee81a1a50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000eaa0 tx=0x7f2ee000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:23.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.104+0000 7f2edfe00640 1 -- 172.21.15.69:0/2983288877 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee000cdb0 con 0x7f2ee8105e30 2024-04-24T09:03:23.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.104+0000 7f2eee200640 1 -- 172.21.15.69:0/2983288877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee81a23b0 con 0x7f2ee8105e30 2024-04-24T09:03:23.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.104+0000 7f2edfe00640 1 -- 172.21.15.69:0/2983288877 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ee0005020 con 0x7f2ee8105e30 2024-04-24T09:03:23.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.104+0000 7f2edfe00640 1 -- 172.21.15.69:0/2983288877 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee001f060 con 0x7f2ee8105e30 2024-04-24T09:03:23.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.104+0000 7f2eee200640 1 -- 172.21.15.69:0/2983288877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee81a2880 con 0x7f2ee8105e30 2024-04-24T09:03:23.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.106+0000 7f2eee200640 1 -- 172.21.15.69:0/2983288877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee810b640 con 0x7f2ee8105e30 2024-04-24T09:03:23.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.108+0000 7f2edfe00640 1 -- 172.21.15.69:0/2983288877 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2ee00051d0 con 0x7f2ee8105e30 2024-04-24T09:03:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.110+0000 7f2edfe00640 1 --2- 172.21.15.69:0/2983288877 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2ecc0a3bb0 0x7f2ecc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.110+0000 7f2edfe00640 1 -- 172.21.15.69:0/2983288877 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2ee00d00f0 con 0x7f2ee8105e30 2024-04-24T09:03:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.110+0000 7f2ee7400640 1 --2- 172.21.15.69:0/2983288877 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2ecc0a3bb0 0x7f2ecc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:23.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.111+0000 7f2ee7400640 1 --2- 172.21.15.69:0/2983288877 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2ecc0a3bb0 0x7f2ecc0a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8005fd0 tx=0x7f2ed800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:23.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.114+0000 7f2edfe00640 1 -- 172.21.15.69:0/2983288877 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ee0016030 con 0x7f2ee8105e30 2024-04-24T09:03:23.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.455+0000 7f2eee200640 1 -- 172.21.15.69:0/2983288877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ee802d620 con 0x7f2ee8105e30 2024-04-24T09:03:23.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.456+0000 7f2edfe00640 1 -- 172.21.15.69:0/2983288877 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f2ee009c880 con 0x7f2ee8105e30 2024-04-24T09:03:23.456 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:23.456 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T09:03:23.456 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 4 2024-04-24T09:03:23.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.458+0000 7f2eee200640 1 -- 172.21.15.69:0/2983288877 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2ecc0a3bb0 msgr2=0x7f2ecc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:23.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.458+0000 7f2eee200640 1 --2- 172.21.15.69:0/2983288877 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2ecc0a3bb0 0x7f2ecc0a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8005fd0 tx=0x7f2ed800b390 comp rx=0 tx=0).stop 2024-04-24T09:03:23.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.458+0000 7f2eee200640 1 -- 172.21.15.69:0/2983288877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ee8105e30 msgr2=0x7f2ee81a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:23.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.458+0000 7f2eee200640 1 --2- 172.21.15.69:0/2983288877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ee8105e30 0x7f2ee81a1a50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000eaa0 tx=0x7f2ee000ef60 comp rx=0 tx=0).stop 2024-04-24T09:03:23.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.458+0000 7f2eee200640 1 -- 172.21.15.69:0/2983288877 shutdown_connections 2024-04-24T09:03:23.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.458+0000 7f2eee200640 1 --2- 172.21.15.69:0/2983288877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ee8105e30 0x7f2ee81a1a50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:23.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.458+0000 7f2eee200640 1 --2- 172.21.15.69:0/2983288877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ee8105470 0x7f2ee819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:23.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.458+0000 7f2eee200640 1 --2- 172.21.15.69:0/2983288877 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2ecc0a3bb0 0x7f2ecc0a6070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:23.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.458+0000 7f2eee200640 1 --2- 172.21.15.69:0/2983288877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ee8104280 0x7f2ee819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:23.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.458+0000 7f2eee200640 1 -- 172.21.15.69:0/2983288877 >> 172.21.15.69:0/2983288877 conn(0x7f2ee80ff7f0 msgr2=0x7f2ee8108810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:23.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.459+0000 7f2eee200640 1 -- 172.21.15.69:0/2983288877 shutdown_connections 2024-04-24T09:03:23.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:23.459+0000 7f2eee200640 1 -- 172.21.15.69:0/2983288877 wait complete. 2024-04-24T09:03:23.805 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:23 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2983288877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:23.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:23 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2983288877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:23.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:23 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2983288877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:24.091 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:03:23 smithi069 systemd[1]: Starting Ceph mds.d for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:03:24.528 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:03:24 smithi069 podman[66262]: 2024-04-24 09:03:24.0702737 +0000 UTC m=+0.034889913 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:03:24.528 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:03:24 smithi069 podman[66262]: 2024-04-24 09:03:24.190000253 +0000 UTC m=+0.154616508 container create 4e35c7e7412250d7c61d0961a4d9d857bfc58ae10f726e4405ad5d48cafff212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-d, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:03:24.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:24 smithi081 ceph-mon[30961]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:24.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:24 smithi069 ceph-mon[28170]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:24.842 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:03:24 smithi069 podman[66262]: 2024-04-24 09:03:24.536180582 +0000 UTC m=+0.500796883 container init 4e35c7e7412250d7c61d0961a4d9d857bfc58ae10f726e4405ad5d48cafff212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T09:03:24.842 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:03:24 smithi069 podman[66262]: 2024-04-24 09:03:24.546439896 +0000 UTC m=+0.511056201 container start 4e35c7e7412250d7c61d0961a4d9d857bfc58ae10f726e4405ad5d48cafff212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-24T09:03:24.842 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:03:24 smithi069 ceph-mds[66282]: -- 172.21.15.69:0/3930128376 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5601fe5c7a40 con 0x5601fe59ac00 2024-04-24T09:03:24.842 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:03:24 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-d[66277]: starting mds.d at 2024-04-24T09:03:24.842 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:03:24 smithi069 bash[66262]: 4e35c7e7412250d7c61d0961a4d9d857bfc58ae10f726e4405ad5d48cafff212 2024-04-24T09:03:24.843 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:03:24 smithi069 systemd[1]: Started Ceph mds.d for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T09:03:24.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:24 smithi130 ceph-mon[31553]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:25.477 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:25.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.678+0000 7f1632200640 1 -- 172.21.15.69:0/409114365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162c100160 msgr2=0x7f162c100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:25.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.678+0000 7f1632200640 1 --2- 172.21.15.69:0/409114365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162c100160 0x7f162c100580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1620009930 tx=0x7f162002a420 comp rx=0 tx=0).stop 2024-04-24T09:03:25.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.678+0000 7f1632200640 1 -- 172.21.15.69:0/409114365 shutdown_connections 2024-04-24T09:03:25.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.678+0000 7f1632200640 1 --2- 172.21.15.69:0/409114365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f162c103af0 0x7f162c10a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:25.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.678+0000 7f1632200640 1 --2- 172.21.15.69:0/409114365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f162c103130 0x7f162c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:25.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.678+0000 7f1632200640 1 --2- 172.21.15.69:0/409114365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162c100160 0x7f162c100580 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:25.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.678+0000 7f1632200640 1 -- 172.21.15.69:0/409114365 >> 172.21.15.69:0/409114365 conn(0x7f162c0fbab0 msgr2=0x7f162c0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:25.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.678+0000 7f1632200640 1 -- 172.21.15.69:0/409114365 shutdown_connections 2024-04-24T09:03:25.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.678+0000 7f1632200640 1 -- 172.21.15.69:0/409114365 wait complete. 2024-04-24T09:03:25.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.679+0000 7f1632200640 1 Processor -- start 2024-04-24T09:03:25.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.679+0000 7f1632200640 1 -- start start 2024-04-24T09:03:25.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.679+0000 7f1632200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f162c100160 0x7f162c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:25.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.679+0000 7f1632200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f162c103130 0x7f162c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:25.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.679+0000 7f1632200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162c103af0 0x7f162c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:25.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.679+0000 7f1632200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f162c10d9e0 con 0x7f162c103af0 2024-04-24T09:03:25.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.679+0000 7f1632200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f162c10d860 con 0x7f162c103130 2024-04-24T09:03:25.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.679+0000 7f1632200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f162c10db60 con 0x7f162c100160 2024-04-24T09:03:25.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.680+0000 7f1631800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162c103af0 0x7f162c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:25.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.680+0000 7f162be00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f162c103130 0x7f162c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:25.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.680+0000 7f162be00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f162c103130 0x7f162c198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.69:59406/0 (socket says 172.21.15.69:59406) 2024-04-24T09:03:25.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.680+0000 7f162be00640 1 -- 172.21.15.69:0/2724690495 learned_addr learned my addr 172.21.15.69:0/2724690495 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:25.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.680+0000 7f1631800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162c103af0 0x7f162c19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36346/0 (socket says 172.21.15.69:36346) 2024-04-24T09:03:25.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.680+0000 7f1630e00640 1 --2- 172.21.15.69:0/2724690495 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f162c100160 0x7f162c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:25.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.680+0000 7f162be00640 1 -- 172.21.15.69:0/2724690495 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f162c100160 msgr2=0x7f162c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:25.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.680+0000 7f162be00640 1 --2- 172.21.15.69:0/2724690495 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f162c100160 0x7f162c198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:25.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.680+0000 7f162be00640 1 -- 172.21.15.69:0/2724690495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162c103af0 msgr2=0x7f162c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:25.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.680+0000 7f162be00640 1 --2- 172.21.15.69:0/2724690495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162c103af0 0x7f162c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:25.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.680+0000 7f162be00640 1 -- 172.21.15.69:0/2724690495 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c19fd30 con 0x7f162c103130 2024-04-24T09:03:25.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.680+0000 7f162be00640 1 --2- 172.21.15.69:0/2724690495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f162c103130 0x7f162c198650 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f161800c970 tx=0x7f161800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:25.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.681+0000 7f1629600640 1 -- 172.21.15.69:0/2724690495 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618007bb0 con 0x7f162c103130 2024-04-24T09:03:25.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.681+0000 7f1632200640 1 -- 172.21.15.69:0/2724690495 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c19ffc0 con 0x7f162c103130 2024-04-24T09:03:25.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.681+0000 7f1629600640 1 -- 172.21.15.69:0/2724690495 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1618007d50 con 0x7f162c103130 2024-04-24T09:03:25.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.681+0000 7f1629600640 1 -- 172.21.15.69:0/2724690495 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161800f450 con 0x7f162c103130 2024-04-24T09:03:25.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.681+0000 7f1632200640 1 -- 172.21.15.69:0/2724690495 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c1a0490 con 0x7f162c103130 2024-04-24T09:03:25.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.682+0000 7f1632200640 1 -- 172.21.15.69:0/2724690495 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162c1093a0 con 0x7f162c103130 2024-04-24T09:03:25.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.685+0000 7f1629600640 1 -- 172.21.15.69:0/2724690495 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1618004f00 con 0x7f162c103130 2024-04-24T09:03:25.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.687+0000 7f1629600640 1 --2- 172.21.15.69:0/2724690495 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f16140a3bb0 0x7f16140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:25.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.687+0000 7f1629600640 1 -- 172.21.15.69:0/2724690495 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f16180d00b0 con 0x7f162c103130 2024-04-24T09:03:25.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.687+0000 7f1630e00640 1 --2- 172.21.15.69:0/2724690495 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f16140a3bb0 0x7f16140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:25.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.688+0000 7f1630e00640 1 --2- 172.21.15.69:0/2724690495 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f16140a3bb0 0x7f16140a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1620006010 tx=0x7f1620000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:25.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.689+0000 7f1629600640 1 -- 172.21.15.69:0/2724690495 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1618014030 con 0x7f162c103130 2024-04-24T09:03:25.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:25 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:25.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:25 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:25.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:25 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:25.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:25 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:25.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:25 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:25.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:25 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:25.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:25 smithi069 ceph-mon[28170]: Deploying daemon mds.e on smithi081 2024-04-24T09:03:25.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:25 smithi069 ceph-mon[28170]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:25.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:25 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.69:6836/2217103498,v1:172.21.15.69:6837/2217103498] up:boot 2024-04-24T09:03:25.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:25 smithi069 ceph-mon[28170]: fsmap 4 up:standby 2024-04-24T09:03:25.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:25 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T09:03:25.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.931+0000 7f1632200640 1 -- 172.21.15.69:0/2724690495 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f162c02d620 con 0x7f162c103130 2024-04-24T09:03:25.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.932+0000 7f1629600640 1 -- 172.21.15.69:0/2724690495 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7f161809ce90 con 0x7f162c103130 2024-04-24T09:03:25.932 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:25.932 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T09:03:25.933 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 5 2024-04-24T09:03:25.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.934+0000 7f1632200640 1 -- 172.21.15.69:0/2724690495 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f16140a3bb0 msgr2=0x7f16140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:25.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.934+0000 7f1632200640 1 --2- 172.21.15.69:0/2724690495 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f16140a3bb0 0x7f16140a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1620006010 tx=0x7f1620000bc0 comp rx=0 tx=0).stop 2024-04-24T09:03:25.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.934+0000 7f1632200640 1 -- 172.21.15.69:0/2724690495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f162c103130 msgr2=0x7f162c198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:25.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.934+0000 7f1632200640 1 --2- 172.21.15.69:0/2724690495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f162c103130 0x7f162c198650 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f161800c970 tx=0x7f161800ce30 comp rx=0 tx=0).stop 2024-04-24T09:03:25.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.935+0000 7f1632200640 1 -- 172.21.15.69:0/2724690495 shutdown_connections 2024-04-24T09:03:25.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.935+0000 7f1632200640 1 --2- 172.21.15.69:0/2724690495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162c103af0 0x7f162c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:25.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.935+0000 7f1632200640 1 --2- 172.21.15.69:0/2724690495 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f16140a3bb0 0x7f16140a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:25.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.935+0000 7f1632200640 1 --2- 172.21.15.69:0/2724690495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f162c103130 0x7f162c198650 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:25.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.935+0000 7f1632200640 1 --2- 172.21.15.69:0/2724690495 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f162c100160 0x7f162c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:25.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.935+0000 7f1632200640 1 -- 172.21.15.69:0/2724690495 >> 172.21.15.69:0/2724690495 conn(0x7f162c0fbab0 msgr2=0x7f162c106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:25.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.935+0000 7f1632200640 1 -- 172.21.15.69:0/2724690495 shutdown_connections 2024-04-24T09:03:25.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:25.935+0000 7f1632200640 1 -- 172.21.15.69:0/2724690495 wait complete. 2024-04-24T09:03:26.034 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:25 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:26.034 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:25 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:26.034 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:25 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:26.035 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:25 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:26.035 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:25 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:26.035 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:25 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:26.035 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:25 smithi081 ceph-mon[30961]: Deploying daemon mds.e on smithi081 2024-04-24T09:03:26.035 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:25 smithi081 ceph-mon[30961]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:26.035 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:25 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.69:6836/2217103498,v1:172.21.15.69:6837/2217103498] up:boot 2024-04-24T09:03:26.035 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:25 smithi081 ceph-mon[30961]: fsmap 4 up:standby 2024-04-24T09:03:26.035 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:25 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T09:03:26.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:25 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:26.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:25 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:26.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:25 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:26.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:25 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:26.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:25 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:26.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:25 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:26.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:25 smithi130 ceph-mon[31553]: Deploying daemon mds.e on smithi081 2024-04-24T09:03:26.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:25 smithi130 ceph-mon[31553]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:26.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:25 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.69:6836/2217103498,v1:172.21.15.69:6837/2217103498] up:boot 2024-04-24T09:03:26.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:25 smithi130 ceph-mon[31553]: fsmap 4 up:standby 2024-04-24T09:03:26.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:25 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T09:03:26.948 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:26 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2724690495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:27.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:26 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2724690495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:27.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:26 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2724690495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:27.918 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:27 smithi081 ceph-mon[30961]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:27.953 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:28.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.168+0000 7f0c65600640 1 -- 172.21.15.69:0/3023309637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c60109670 msgr2=0x7f0c60109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:28.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.168+0000 7f0c65600640 1 --2- 172.21.15.69:0/3023309637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c60109670 0x7f0c60109af0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009900 tx=0x7f0c4802a400 comp rx=0 tx=0).stop 2024-04-24T09:03:28.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.168+0000 7f0c65600640 1 -- 172.21.15.69:0/3023309637 shutdown_connections 2024-04-24T09:03:28.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.168+0000 7f0c65600640 1 --2- 172.21.15.69:0/3023309637 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c6010a030 0x7f0c601108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:28.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.168+0000 7f0c65600640 1 --2- 172.21.15.69:0/3023309637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c60109670 0x7f0c60109af0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:28.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.168+0000 7f0c65600640 1 --2- 172.21.15.69:0/3023309637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c60108480 0x7f0c601088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:28.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.168+0000 7f0c65600640 1 -- 172.21.15.69:0/3023309637 >> 172.21.15.69:0/3023309637 conn(0x7f0c60077440 msgr2=0x7f0c60079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:28.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.168+0000 7f0c65600640 1 -- 172.21.15.69:0/3023309637 shutdown_connections 2024-04-24T09:03:28.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.168+0000 7f0c65600640 1 -- 172.21.15.69:0/3023309637 wait complete. 2024-04-24T09:03:28.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.169+0000 7f0c65600640 1 Processor -- start 2024-04-24T09:03:28.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.169+0000 7f0c65600640 1 -- start start 2024-04-24T09:03:28.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.169+0000 7f0c65600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c60108480 0x7f0c6019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:28.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.169+0000 7f0c65600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c60109670 0x7f0c6019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:28.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.169+0000 7f0c65600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c6010a030 0x7f0c601a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:28.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.169+0000 7f0c65600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0c60113fc0 con 0x7f0c60108480 2024-04-24T09:03:28.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.169+0000 7f0c65600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0c60113e40 con 0x7f0c6010a030 2024-04-24T09:03:28.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.169+0000 7f0c65600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0c60114140 con 0x7f0c60109670 2024-04-24T09:03:28.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.170+0000 7f0c5fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c60108480 0x7f0c6019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:28.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.170+0000 7f0c5fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c60108480 0x7f0c6019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36356/0 (socket says 172.21.15.69:36356) 2024-04-24T09:03:28.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.170+0000 7f0c5fe00640 1 -- 172.21.15.69:0/2301021631 learned_addr learned my addr 172.21.15.69:0/2301021631 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:28.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.170+0000 7f0c64c00640 1 --2- 172.21.15.69:0/2301021631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c6010a030 0x7f0c601a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:28.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.170+0000 7f0c57e00640 1 --2- 172.21.15.69:0/2301021631 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c60109670 0x7f0c6019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:28.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.170+0000 7f0c5fe00640 1 -- 172.21.15.69:0/2301021631 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c60109670 msgr2=0x7f0c6019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:28.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.170+0000 7f0c5fe00640 1 --2- 172.21.15.69:0/2301021631 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c60109670 0x7f0c6019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:28.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.170+0000 7f0c5fe00640 1 -- 172.21.15.69:0/2301021631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c6010a030 msgr2=0x7f0c601a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:28.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.170+0000 7f0c5fe00640 1 --2- 172.21.15.69:0/2301021631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c6010a030 0x7f0c601a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:28.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.170+0000 7f0c5fe00640 1 -- 172.21.15.69:0/2301021631 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c601a62c0 con 0x7f0c60108480 2024-04-24T09:03:28.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.170+0000 7f0c5fe00640 1 --2- 172.21.15.69:0/2301021631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c60108480 0x7f0c6019e6a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000c970 tx=0x7f0c5000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:28.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.170+0000 7f0c5d600640 1 -- 172.21.15.69:0/2301021631 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50004220 con 0x7f0c60108480 2024-04-24T09:03:28.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.171+0000 7f0c65600640 1 -- 172.21.15.69:0/2301021631 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c601a65b0 con 0x7f0c60108480 2024-04-24T09:03:28.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.171+0000 7f0c5d600640 1 -- 172.21.15.69:0/2301021631 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c500043c0 con 0x7f0c60108480 2024-04-24T09:03:28.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.171+0000 7f0c5d600640 1 -- 172.21.15.69:0/2301021631 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50004a00 con 0x7f0c60108480 2024-04-24T09:03:28.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.171+0000 7f0c65600640 1 -- 172.21.15.69:0/2301021631 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c601a6ab0 con 0x7f0c60108480 2024-04-24T09:03:28.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.172+0000 7f0c65600640 1 -- 172.21.15.69:0/2301021631 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c6010f820 con 0x7f0c60108480 2024-04-24T09:03:28.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.174+0000 7f0c5d600640 1 -- 172.21.15.69:0/2301021631 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0c500079b0 con 0x7f0c60108480 2024-04-24T09:03:28.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.175+0000 7f0c5d600640 1 --2- 172.21.15.69:0/2301021631 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0c440a3bb0 0x7f0c440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:28.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.175+0000 7f0c5d600640 1 -- 172.21.15.69:0/2301021631 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0c500cfd60 con 0x7f0c60108480 2024-04-24T09:03:28.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.175+0000 7f0c57e00640 1 --2- 172.21.15.69:0/2301021631 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0c440a3bb0 0x7f0c440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:28.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.176+0000 7f0c57e00640 1 --2- 172.21.15.69:0/2301021631 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0c440a3bb0 0x7f0c440a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0c6019fbb0 tx=0x7f0c480314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:28.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.180+0000 7f0c5d600640 1 -- 172.21.15.69:0/2301021631 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c50014030 con 0x7f0c60108480 2024-04-24T09:03:28.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:27 smithi130 ceph-mon[31553]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:28.274 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:03:27 smithi081 systemd[1]: Starting Ceph mds.e for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:03:28.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:27 smithi069 ceph-mon[28170]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:28.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.421+0000 7f0c65600640 1 -- 172.21.15.69:0/2301021631 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c6002d620 con 0x7f0c60108480 2024-04-24T09:03:28.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.422+0000 7f0c5d600640 1 -- 172.21.15.69:0/2301021631 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7f0c5009cb40 con 0x7f0c60108480 2024-04-24T09:03:28.420 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:28.421 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T09:03:28.421 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 5 2024-04-24T09:03:28.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.424+0000 7f0c65600640 1 -- 172.21.15.69:0/2301021631 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0c440a3bb0 msgr2=0x7f0c440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:28.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.424+0000 7f0c65600640 1 --2- 172.21.15.69:0/2301021631 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0c440a3bb0 0x7f0c440a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0c6019fbb0 tx=0x7f0c480314a0 comp rx=0 tx=0).stop 2024-04-24T09:03:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.425+0000 7f0c65600640 1 -- 172.21.15.69:0/2301021631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c60108480 msgr2=0x7f0c6019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.425+0000 7f0c65600640 1 --2- 172.21.15.69:0/2301021631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c60108480 0x7f0c6019e6a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000c970 tx=0x7f0c5000ce30 comp rx=0 tx=0).stop 2024-04-24T09:03:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.425+0000 7f0c65600640 1 -- 172.21.15.69:0/2301021631 shutdown_connections 2024-04-24T09:03:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.425+0000 7f0c65600640 1 --2- 172.21.15.69:0/2301021631 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0c440a3bb0 0x7f0c440a6070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.425+0000 7f0c65600640 1 --2- 172.21.15.69:0/2301021631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c6010a030 0x7f0c601a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.425+0000 7f0c65600640 1 --2- 172.21.15.69:0/2301021631 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c60109670 0x7f0c6019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.425+0000 7f0c65600640 1 --2- 172.21.15.69:0/2301021631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c60108480 0x7f0c6019e6a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.425+0000 7f0c65600640 1 -- 172.21.15.69:0/2301021631 >> 172.21.15.69:0/2301021631 conn(0x7f0c60077440 msgr2=0x7f0c6010c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.425+0000 7f0c65600640 1 -- 172.21.15.69:0/2301021631 shutdown_connections 2024-04-24T09:03:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:28.425+0000 7f0c65600640 1 -- 172.21.15.69:0/2301021631 wait complete. 2024-04-24T09:03:28.568 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:03:28 smithi081 podman[60804]: 2024-04-24 09:03:28.180202427 +0000 UTC m=+0.033033875 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:03:28.568 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:03:28 smithi081 podman[60804]: 2024-04-24 09:03:28.29946752 +0000 UTC m=+0.152299002 container create ad16eaa7ced9008ba4040f4b19af68fe8490df01758e533d01028d8e05ead308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-24T09:03:28.870 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:03:28 smithi081 podman[60804]: 2024-04-24 09:03:28.57051995 +0000 UTC m=+0.423351443 container init ad16eaa7ced9008ba4040f4b19af68fe8490df01758e533d01028d8e05ead308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True) 2024-04-24T09:03:28.870 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:03:28 smithi081 podman[60804]: 2024-04-24 09:03:28.580022016 +0000 UTC m=+0.432853482 container start ad16eaa7ced9008ba4040f4b19af68fe8490df01758e533d01028d8e05ead308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-24T09:03:28.870 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:03:28 smithi081 ceph-mds[60823]: -- 172.21.15.81:0/2162683631 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5589e8dca4e0 con 0x5589e8da1000 2024-04-24T09:03:28.871 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:03:28 smithi081 ceph-mds[60823]: -- 172.21.15.81:0/2162683631 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5589e8dcda40 con 0x5589e8da1000 2024-04-24T09:03:28.871 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:03:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: starting mds.e at 2024-04-24T09:03:28.871 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:03:28 smithi081 bash[60804]: ad16eaa7ced9008ba4040f4b19af68fe8490df01758e533d01028d8e05ead308 2024-04-24T09:03:28.871 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:03:28 smithi081 systemd[1]: Started Ceph mds.e for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T09:03:29.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:28 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2301021631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:29.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:28 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:29.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:28 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:29.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:28 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:29.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:28 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:29.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:28 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:29.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:28 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:29.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:28 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2301021631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:29.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:28 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:29.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:28 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:29.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:28 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:29.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:28 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:29.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:28 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:29.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:28 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:29.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:28 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2301021631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:29.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:28 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:29.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:28 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:29.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:28 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:29.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:28 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:29.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:28 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:29.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:28 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:30.105 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:29 smithi130 ceph-mon[31553]: Deploying daemon mds.f on smithi130 2024-04-24T09:03:30.106 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:29 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] up:boot 2024-04-24T09:03:30.106 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:29 smithi130 ceph-mon[31553]: fsmap 5 up:standby 2024-04-24T09:03:30.106 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:29 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T09:03:30.106 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:29 smithi130 ceph-mon[31553]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:30.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:29 smithi081 ceph-mon[30961]: Deploying daemon mds.f on smithi130 2024-04-24T09:03:30.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:29 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] up:boot 2024-04-24T09:03:30.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:29 smithi081 ceph-mon[30961]: fsmap 5 up:standby 2024-04-24T09:03:30.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:29 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T09:03:30.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:29 smithi081 ceph-mon[30961]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:30.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:29 smithi069 ceph-mon[28170]: Deploying daemon mds.f on smithi130 2024-04-24T09:03:30.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:29 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] up:boot 2024-04-24T09:03:30.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:29 smithi069 ceph-mon[28170]: fsmap 5 up:standby 2024-04-24T09:03:30.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:29 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T09:03:30.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:29 smithi069 ceph-mon[28170]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:30.443 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:30.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.657+0000 7fa407e00640 1 -- 172.21.15.69:0/1143978205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa408107b60 msgr2=0x7fa408109fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:30.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.657+0000 7fa407e00640 1 --2- 172.21.15.69:0/1143978205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa408107b60 0x7fa408109fb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009900 tx=0x7fa3f802a410 comp rx=0 tx=0).stop 2024-04-24T09:03:30.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.657+0000 7fa407e00640 1 -- 172.21.15.69:0/1143978205 shutdown_connections 2024-04-24T09:03:30.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.657+0000 7fa407e00640 1 --2- 172.21.15.69:0/1143978205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa408107b60 0x7fa408109fb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:30.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.657+0000 7fa407e00640 1 --2- 172.21.15.69:0/1143978205 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa408105230 0x7fa408107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:30.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.657+0000 7fa407e00640 1 --2- 172.21.15.69:0/1143978205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa408102900 0x7fa408104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:30.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.657+0000 7fa407e00640 1 -- 172.21.15.69:0/1143978205 >> 172.21.15.69:0/1143978205 conn(0x7fa4080fc2d0 msgr2=0x7fa4080fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:30.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.657+0000 7fa407e00640 1 -- 172.21.15.69:0/1143978205 shutdown_connections 2024-04-24T09:03:30.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.657+0000 7fa407e00640 1 -- 172.21.15.69:0/1143978205 wait complete. 2024-04-24T09:03:30.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.658+0000 7fa407e00640 1 Processor -- start 2024-04-24T09:03:30.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.658+0000 7fa407e00640 1 -- start start 2024-04-24T09:03:30.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.658+0000 7fa407e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa408102900 0x7fa40819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:30.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.658+0000 7fa407e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa408105230 0x7fa40819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:30.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.658+0000 7fa407e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa408107b60 0x7fa4081a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:30.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.658+0000 7fa407e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa40810d9b0 con 0x7fa408107b60 2024-04-24T09:03:30.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.658+0000 7fa407e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa40810d830 con 0x7fa408105230 2024-04-24T09:03:30.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.658+0000 7fa407e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa40810db30 con 0x7fa408102900 2024-04-24T09:03:30.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.659+0000 7fa407400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa408107b60 0x7fa4081a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:30.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.659+0000 7fa407400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa408107b60 0x7fa4081a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36386/0 (socket says 172.21.15.69:36386) 2024-04-24T09:03:30.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.659+0000 7fa407400640 1 -- 172.21.15.69:0/2172411874 learned_addr learned my addr 172.21.15.69:0/2172411874 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:30.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.659+0000 7fa406a00640 1 --2- 172.21.15.69:0/2172411874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa408102900 0x7fa40819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:30.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.659+0000 7fa406000640 1 --2- 172.21.15.69:0/2172411874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa408105230 0x7fa40819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.659+0000 7fa407400640 1 -- 172.21.15.69:0/2172411874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa408102900 msgr2=0x7fa40819a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.659+0000 7fa407400640 1 --2- 172.21.15.69:0/2172411874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa408102900 0x7fa40819a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.659+0000 7fa407400640 1 -- 172.21.15.69:0/2172411874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa408105230 msgr2=0x7fa40819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.659+0000 7fa407400640 1 --2- 172.21.15.69:0/2172411874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa408105230 0x7fa40819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.659+0000 7fa407400640 1 -- 172.21.15.69:0/2172411874 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4081a1e80 con 0x7fa408107b60 2024-04-24T09:03:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.659+0000 7fa406a00640 1 --2- 172.21.15.69:0/2172411874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa408102900 0x7fa40819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:03:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.660+0000 7fa407400640 1 --2- 172.21.15.69:0/2172411874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa408107b60 0x7fa4081a1810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8005940 tx=0x7fa3f8031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.660+0000 7fa3ff400640 1 -- 172.21.15.69:0/2172411874 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f803f030 con 0x7fa408107b60 2024-04-24T09:03:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.660+0000 7fa3ff400640 1 -- 172.21.15.69:0/2172411874 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3f8003aa0 con 0x7fa408107b60 2024-04-24T09:03:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.660+0000 7fa3ff400640 1 -- 172.21.15.69:0/2172411874 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8035070 con 0x7fa408107b60 2024-04-24T09:03:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.660+0000 7fa407e00640 1 -- 172.21.15.69:0/2172411874 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4081a2110 con 0x7fa408107b60 2024-04-24T09:03:30.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.660+0000 7fa407e00640 1 -- 172.21.15.69:0/2172411874 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4081a25b0 con 0x7fa408107b60 2024-04-24T09:03:30.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.661+0000 7fa407e00640 1 -- 172.21.15.69:0/2172411874 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa408109650 con 0x7fa408107b60 2024-04-24T09:03:30.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.663+0000 7fa3ff400640 1 -- 172.21.15.69:0/2172411874 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa3f8042030 con 0x7fa408107b60 2024-04-24T09:03:30.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.665+0000 7fa3ff400640 1 --2- 172.21.15.69:0/2172411874 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3ec0a3c30 0x7fa3ec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:30.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.665+0000 7fa3ff400640 1 -- 172.21.15.69:0/2172411874 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa3f80f0650 con 0x7fa408107b60 2024-04-24T09:03:30.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.665+0000 7fa406a00640 1 --2- 172.21.15.69:0/2172411874 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3ec0a3c30 0x7fa3ec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:30.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.666+0000 7fa406a00640 1 --2- 172.21.15.69:0/2172411874 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3ec0a3c30 0x7fa3ec0a60f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa400005fd0 tx=0x7fa400009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:30.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.670+0000 7fa3ff400640 1 -- 172.21.15.69:0/2172411874 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3f80bd430 con 0x7fa408107b60 2024-04-24T09:03:30.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.914+0000 7fa407e00640 1 -- 172.21.15.69:0/2172411874 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa40802d620 con 0x7fa408107b60 2024-04-24T09:03:30.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.916+0000 7fa3ff400640 1 -- 172.21.15.69:0/2172411874 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7fa3f80bcd30 con 0x7fa408107b60 2024-04-24T09:03:30.915 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:30.915 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34460,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T09:03:30.915 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 6 2024-04-24T09:03:30.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.918+0000 7fa407e00640 1 -- 172.21.15.69:0/2172411874 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3ec0a3c30 msgr2=0x7fa3ec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:30.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.919+0000 7fa407e00640 1 --2- 172.21.15.69:0/2172411874 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3ec0a3c30 0x7fa3ec0a60f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa400005fd0 tx=0x7fa400009520 comp rx=0 tx=0).stop 2024-04-24T09:03:30.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.919+0000 7fa407e00640 1 -- 172.21.15.69:0/2172411874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa408107b60 msgr2=0x7fa4081a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:30.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.919+0000 7fa407e00640 1 --2- 172.21.15.69:0/2172411874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa408107b60 0x7fa4081a1810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8005940 tx=0x7fa3f8031a30 comp rx=0 tx=0).stop 2024-04-24T09:03:30.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.919+0000 7fa407e00640 1 -- 172.21.15.69:0/2172411874 shutdown_connections 2024-04-24T09:03:30.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.919+0000 7fa407e00640 1 --2- 172.21.15.69:0/2172411874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa408107b60 0x7fa4081a1810 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:30.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.919+0000 7fa407e00640 1 --2- 172.21.15.69:0/2172411874 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3ec0a3c30 0x7fa3ec0a60f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:30.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.919+0000 7fa407e00640 1 --2- 172.21.15.69:0/2172411874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa408105230 0x7fa40819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:30.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.919+0000 7fa407e00640 1 --2- 172.21.15.69:0/2172411874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa408102900 0x7fa40819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:30.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.919+0000 7fa407e00640 1 -- 172.21.15.69:0/2172411874 >> 172.21.15.69:0/2172411874 conn(0x7fa4080fc2d0 msgr2=0x7fa408102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:30.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.919+0000 7fa407e00640 1 -- 172.21.15.69:0/2172411874 shutdown_connections 2024-04-24T09:03:30.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:30.919+0000 7fa407e00640 1 -- 172.21.15.69:0/2172411874 wait complete. 2024-04-24T09:03:31.160 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:30 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2172411874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:31.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:30 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2172411874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:31.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:30 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2172411874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:32.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:31 smithi081 ceph-mon[30961]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:32.321 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:31 smithi130 ceph-mon[31553]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:32.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:31 smithi069 ceph-mon[28170]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:32.576 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 24 09:03:32 smithi130 podman[62200]: 2024-04-24 09:03:32.404560777 +0000 UTC m=+0.214710668 container create 47c7d8f5516209fbc94a98e44461210b3316300ae4b096198fce351786399039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-24T09:03:32.937 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:32.954 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 24 09:03:32 smithi130 podman[62200]: 2024-04-24 09:03:32.683305928 +0000 UTC m=+0.493455826 container init 47c7d8f5516209fbc94a98e44461210b3316300ae4b096198fce351786399039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-24T09:03:32.954 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 24 09:03:32 smithi130 podman[62200]: 2024-04-24 09:03:32.693296873 +0000 UTC m=+0.503446797 container start 47c7d8f5516209fbc94a98e44461210b3316300ae4b096198fce351786399039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:03:32.954 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 24 09:03:32 smithi130 ceph-mds[62219]: -- 172.21.15.130:0/3774518076 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555ced12e4e0 con 0x555ced104c00 2024-04-24T09:03:32.955 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 24 09:03:32 smithi130 ceph-mds[62219]: -- 172.21.15.130:0/3774518076 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555ced131a40 con 0x555ced104c00 2024-04-24T09:03:32.955 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 24 09:03:32 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-f[62215]: starting mds.f at 2024-04-24T09:03:32.955 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 24 09:03:32 smithi130 bash[62200]: 47c7d8f5516209fbc94a98e44461210b3316300ae4b096198fce351786399039 2024-04-24T09:03:32.955 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 24 09:03:32 smithi130 systemd[1]: Started Ceph mds.f for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T09:03:33.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.140+0000 7f0a43e00640 1 -- 172.21.15.69:0/71758153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a3c0769c0 msgr2=0x7f0a3c076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.140+0000 7f0a43e00640 1 --2- 172.21.15.69:0/71758153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a3c0769c0 0x7f0a3c076de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009900 tx=0x7f0a3402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.140+0000 7f0a43e00640 1 -- 172.21.15.69:0/71758153 shutdown_connections 2024-04-24T09:03:33.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.140+0000 7f0a43e00640 1 --2- 172.21.15.69:0/71758153 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3c0785a0 0x7f0a3c078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.140+0000 7f0a43e00640 1 --2- 172.21.15.69:0/71758153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a3c077c00 0x7f0a3c078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.140+0000 7f0a43e00640 1 --2- 172.21.15.69:0/71758153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a3c0769c0 0x7f0a3c076de0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.140+0000 7f0a43e00640 1 -- 172.21.15.69:0/71758153 >> 172.21.15.69:0/71758153 conn(0x7f0a3c0ff7f0 msgr2=0x7f0a3c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.140+0000 7f0a43e00640 1 -- 172.21.15.69:0/71758153 shutdown_connections 2024-04-24T09:03:33.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.140+0000 7f0a43e00640 1 -- 172.21.15.69:0/71758153 wait complete. 2024-04-24T09:03:33.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.141+0000 7f0a43e00640 1 Processor -- start 2024-04-24T09:03:33.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.141+0000 7f0a43e00640 1 -- start start 2024-04-24T09:03:33.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.142+0000 7f0a43e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a3c0769c0 0x7f0a3c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.142+0000 7f0a43e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3c077c00 0x7f0a3c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.142+0000 7f0a43e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a3c0785a0 0x7f0a3c1a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.142+0000 7f0a43e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1141a0 con 0x7f0a3c0785a0 2024-04-24T09:03:33.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.142+0000 7f0a43e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c114020 con 0x7f0a3c077c00 2024-04-24T09:03:33.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.142+0000 7f0a43e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c114320 con 0x7f0a3c0769c0 2024-04-24T09:03:33.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.142+0000 7f0a41e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a3c0785a0 0x7f0a3c1a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.142+0000 7f0a41400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a3c0769c0 0x7f0a3c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.142+0000 7f0a41e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a3c0785a0 0x7f0a3c1a5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34238/0 (socket says 172.21.15.69:34238) 2024-04-24T09:03:33.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.142+0000 7f0a41400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a3c0769c0 0x7f0a3c19e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.69:50592/0 (socket says 172.21.15.69:50592) 2024-04-24T09:03:33.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.142+0000 7f0a41e00640 1 -- 172.21.15.69:0/1635592843 learned_addr learned my addr 172.21.15.69:0/1635592843 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:33.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.142+0000 7f0a40a00640 1 --2- 172.21.15.69:0/1635592843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3c077c00 0x7f0a3c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.143+0000 7f0a41e00640 1 -- 172.21.15.69:0/1635592843 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a3c0769c0 msgr2=0x7f0a3c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.143+0000 7f0a41e00640 1 --2- 172.21.15.69:0/1635592843 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a3c0769c0 0x7f0a3c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.143+0000 7f0a41e00640 1 -- 172.21.15.69:0/1635592843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3c077c00 msgr2=0x7f0a3c19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.143+0000 7f0a41e00640 1 --2- 172.21.15.69:0/1635592843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3c077c00 0x7f0a3c19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.143+0000 7f0a41e00640 1 -- 172.21.15.69:0/1635592843 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3c1a64a0 con 0x7f0a3c0785a0 2024-04-24T09:03:33.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.143+0000 7f0a40a00640 1 --2- 172.21.15.69:0/1635592843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3c077c00 0x7f0a3c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:03:33.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.143+0000 7f0a41e00640 1 --2- 172.21.15.69:0/1635592843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a3c0785a0 0x7f0a3c1a5e30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00c970 tx=0x7f0a2c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.143+0000 7f0a3a000640 1 -- 172.21.15.69:0/1635592843 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c007bb0 con 0x7f0a3c0785a0 2024-04-24T09:03:33.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.143+0000 7f0a3a000640 1 -- 172.21.15.69:0/1635592843 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a2c007d50 con 0x7f0a3c0785a0 2024-04-24T09:03:33.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.143+0000 7f0a3a000640 1 -- 172.21.15.69:0/1635592843 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c018700 con 0x7f0a3c0785a0 2024-04-24T09:03:33.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.143+0000 7f0a43e00640 1 -- 172.21.15.69:0/1635592843 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3c1a6790 con 0x7f0a3c0785a0 2024-04-24T09:03:33.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.143+0000 7f0a43e00640 1 -- 172.21.15.69:0/1635592843 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a3c1a6c90 con 0x7f0a3c0785a0 2024-04-24T09:03:33.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.145+0000 7f0a43e00640 1 -- 172.21.15.69:0/1635592843 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a3c110da0 con 0x7f0a3c0785a0 2024-04-24T09:03:33.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.147+0000 7f0a3a000640 1 -- 172.21.15.69:0/1635592843 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0a2c00fa20 con 0x7f0a3c0785a0 2024-04-24T09:03:33.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.148+0000 7f0a3a000640 1 --2- 172.21.15.69:0/1635592843 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0a200a3bb0 0x7f0a200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.148+0000 7f0a3a000640 1 -- 172.21.15.69:0/1635592843 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0a2c0d00e0 con 0x7f0a3c0785a0 2024-04-24T09:03:33.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.148+0000 7f0a41400640 1 --2- 172.21.15.69:0/1635592843 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0a200a3bb0 0x7f0a200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.149+0000 7f0a41400640 1 --2- 172.21.15.69:0/1635592843 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0a200a3bb0 0x7f0a200a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a34006010 tx=0x7f0a34000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.153+0000 7f0a3a000640 1 -- 172.21.15.69:0/1635592843 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0a2c014030 con 0x7f0a3c0785a0 2024-04-24T09:03:33.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.398+0000 7f0a43e00640 1 -- 172.21.15.69:0/1635592843 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a3c02d620 con 0x7f0a3c0785a0 2024-04-24T09:03:33.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.399+0000 7f0a3a000640 1 -- 172.21.15.69:0/1635592843 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f0a2c09cec0 con 0x7f0a3c0785a0 2024-04-24T09:03:33.399 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:33.399 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34460,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T09:03:33.400 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 7 2024-04-24T09:03:33.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.402+0000 7f0a43e00640 1 -- 172.21.15.69:0/1635592843 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0a200a3bb0 msgr2=0x7f0a200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.402+0000 7f0a43e00640 1 --2- 172.21.15.69:0/1635592843 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0a200a3bb0 0x7f0a200a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a34006010 tx=0x7f0a34000bc0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.402+0000 7f0a43e00640 1 -- 172.21.15.69:0/1635592843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a3c0785a0 msgr2=0x7f0a3c1a5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.402+0000 7f0a43e00640 1 --2- 172.21.15.69:0/1635592843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a3c0785a0 0x7f0a3c1a5e30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00c970 tx=0x7f0a2c00ce30 comp rx=0 tx=0).stop 2024-04-24T09:03:33.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.402+0000 7f0a43e00640 1 -- 172.21.15.69:0/1635592843 shutdown_connections 2024-04-24T09:03:33.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.402+0000 7f0a43e00640 1 --2- 172.21.15.69:0/1635592843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a3c0785a0 0x7f0a3c1a5e30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.402+0000 7f0a43e00640 1 --2- 172.21.15.69:0/1635592843 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0a200a3bb0 0x7f0a200a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.402+0000 7f0a43e00640 1 --2- 172.21.15.69:0/1635592843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a3c077c00 0x7f0a3c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.402+0000 7f0a43e00640 1 --2- 172.21.15.69:0/1635592843 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a3c0769c0 0x7f0a3c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.402+0000 7f0a43e00640 1 -- 172.21.15.69:0/1635592843 >> 172.21.15.69:0/1635592843 conn(0x7f0a3c0ff7f0 msgr2=0x7f0a3c100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.403+0000 7f0a43e00640 1 -- 172.21.15.69:0/1635592843 shutdown_connections 2024-04-24T09:03:33.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:33.403+0000 7f0a43e00640 1 -- 172.21.15.69:0/1635592843 wait complete. 2024-04-24T09:03:34.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:33 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:34.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:33 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:34.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:33 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.130:6834/1889126439,v1:172.21.15.130:6835/1889126439] up:boot 2024-04-24T09:03:34.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:33 smithi130 ceph-mon[31553]: fsmap 6 up:standby 2024-04-24T09:03:34.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:33 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T09:03:34.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:33 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:34.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:33 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:34.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:33 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:34.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:33 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:34.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:33 smithi130 ceph-mon[31553]: Deploying daemon mds.c on smithi130 2024-04-24T09:03:34.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:33 smithi130 ceph-mon[31553]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:34.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:33 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1635592843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:34.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:33 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:34.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:33 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:34.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:33 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.130:6834/1889126439,v1:172.21.15.130:6835/1889126439] up:boot 2024-04-24T09:03:34.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:33 smithi081 ceph-mon[30961]: fsmap 6 up:standby 2024-04-24T09:03:34.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:33 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T09:03:34.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:33 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:34.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:33 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:34.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:33 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:34.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:33 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:34.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:33 smithi081 ceph-mon[30961]: Deploying daemon mds.c on smithi130 2024-04-24T09:03:34.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:33 smithi081 ceph-mon[30961]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:34.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:33 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1635592843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:34.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:33 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:34.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:33 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:34.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:33 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.130:6834/1889126439,v1:172.21.15.130:6835/1889126439] up:boot 2024-04-24T09:03:34.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:33 smithi069 ceph-mon[28170]: fsmap 6 up:standby 2024-04-24T09:03:34.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:33 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T09:03:34.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:33 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:34.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:33 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:34.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:33 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:34.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:33 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:34.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:33 smithi069 ceph-mon[28170]: Deploying daemon mds.c on smithi130 2024-04-24T09:03:34.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:33 smithi069 ceph-mon[28170]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:34.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:33 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1635592843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:35.421 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:35.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.628+0000 7f56cbe00640 1 -- 172.21.15.69:0/1238146943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56cc109670 msgr2=0x7f56cc109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.628+0000 7f56cbe00640 1 --2- 172.21.15.69:0/1238146943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56cc109670 0x7f56cc109af0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009900 tx=0x7f56b402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.628+0000 7f56cbe00640 1 -- 172.21.15.69:0/1238146943 shutdown_connections 2024-04-24T09:03:35.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.628+0000 7f56cbe00640 1 --2- 172.21.15.69:0/1238146943 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56cc10a030 0x7f56cc1108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.629+0000 7f56cbe00640 1 --2- 172.21.15.69:0/1238146943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56cc109670 0x7f56cc109af0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.629+0000 7f56cbe00640 1 --2- 172.21.15.69:0/1238146943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56cc108480 0x7f56cc1088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.629+0000 7f56cbe00640 1 -- 172.21.15.69:0/1238146943 >> 172.21.15.69:0/1238146943 conn(0x7f56cc077440 msgr2=0x7f56cc079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:35.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.629+0000 7f56cbe00640 1 -- 172.21.15.69:0/1238146943 shutdown_connections 2024-04-24T09:03:35.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.629+0000 7f56cbe00640 1 -- 172.21.15.69:0/1238146943 wait complete. 2024-04-24T09:03:35.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.630+0000 7f56cbe00640 1 Processor -- start 2024-04-24T09:03:35.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.630+0000 7f56cbe00640 1 -- start start 2024-04-24T09:03:35.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.630+0000 7f56cbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56cc108480 0x7f56cc19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.630+0000 7f56cbe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56cc109670 0x7f56cc19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.630+0000 7f56cbe00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56cc10a030 0x7f56cc1a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.630+0000 7f56cbe00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc113fc0 con 0x7f56cc108480 2024-04-24T09:03:35.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.630+0000 7f56cbe00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc113e40 con 0x7f56cc10a030 2024-04-24T09:03:35.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.630+0000 7f56cbe00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc114140 con 0x7f56cc109670 2024-04-24T09:03:35.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.630+0000 7f56caa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56cc108480 0x7f56cc19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.630+0000 7f56caa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56cc108480 0x7f56cc19e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34262/0 (socket says 172.21.15.69:34262) 2024-04-24T09:03:35.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.630+0000 7f56caa00640 1 -- 172.21.15.69:0/2344943926 learned_addr learned my addr 172.21.15.69:0/2344943926 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:35.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.631+0000 7f56cb400640 1 --2- 172.21.15.69:0/2344943926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56cc10a030 0x7f56cc1a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.631+0000 7f56c3e00640 1 --2- 172.21.15.69:0/2344943926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56cc109670 0x7f56cc19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.631+0000 7f56caa00640 1 -- 172.21.15.69:0/2344943926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56cc109670 msgr2=0x7f56cc19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.631+0000 7f56caa00640 1 --2- 172.21.15.69:0/2344943926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56cc109670 0x7f56cc19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.631+0000 7f56caa00640 1 -- 172.21.15.69:0/2344943926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56cc10a030 msgr2=0x7f56cc1a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.631+0000 7f56caa00640 1 --2- 172.21.15.69:0/2344943926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56cc10a030 0x7f56cc1a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.631+0000 7f56caa00640 1 -- 172.21.15.69:0/2344943926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc1a62c0 con 0x7f56cc108480 2024-04-24T09:03:35.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.631+0000 7f56cb400640 1 --2- 172.21.15.69:0/2344943926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56cc10a030 0x7f56cc1a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:35.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.631+0000 7f56c3e00640 1 --2- 172.21.15.69:0/2344943926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56cc109670 0x7f56cc19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:35.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.631+0000 7f56caa00640 1 --2- 172.21.15.69:0/2344943926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56cc108480 0x7f56cc19e740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00e9c0 tx=0x7f56bc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:35.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.631+0000 7f56c3400640 1 -- 172.21.15.69:0/2344943926 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00cd80 con 0x7f56cc108480 2024-04-24T09:03:35.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.631+0000 7f56c3400640 1 -- 172.21.15.69:0/2344943926 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56bc0035f0 con 0x7f56cc108480 2024-04-24T09:03:35.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.631+0000 7f56c3400640 1 -- 172.21.15.69:0/2344943926 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc010690 con 0x7f56cc108480 2024-04-24T09:03:35.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.631+0000 7f56cbe00640 1 -- 172.21.15.69:0/2344943926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56cc1a65b0 con 0x7f56cc108480 2024-04-24T09:03:35.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.631+0000 7f56cbe00640 1 -- 172.21.15.69:0/2344943926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56cc1a6ab0 con 0x7f56cc108480 2024-04-24T09:03:35.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.633+0000 7f56cbe00640 1 -- 172.21.15.69:0/2344943926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56cc10f7c0 con 0x7f56cc108480 2024-04-24T09:03:35.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.635+0000 7f56c3400640 1 -- 172.21.15.69:0/2344943926 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f56bc0037a0 con 0x7f56cc108480 2024-04-24T09:03:35.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.636+0000 7f56c3400640 1 --2- 172.21.15.69:0/2344943926 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56b00a3bb0 0x7f56b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.636+0000 7f56c3400640 1 -- 172.21.15.69:0/2344943926 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f56bc0d04f0 con 0x7f56cc108480 2024-04-24T09:03:35.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.636+0000 7f56c3e00640 1 --2- 172.21.15.69:0/2344943926 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56b00a3bb0 0x7f56b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.637+0000 7f56c3e00640 1 --2- 172.21.15.69:0/2344943926 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56b00a3bb0 0x7f56b00a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f56b4006010 tx=0x7f56b4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:35.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.640+0000 7f56c3400640 1 -- 172.21.15.69:0/2344943926 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56bc09d2d0 con 0x7f56cc108480 2024-04-24T09:03:35.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.884+0000 7f56cbe00640 1 -- 172.21.15.69:0/2344943926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56cc02d620 con 0x7f56cc108480 2024-04-24T09:03:35.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.886+0000 7f56c3400640 1 -- 172.21.15.69:0/2344943926 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f56bc09cbd0 con 0x7f56cc108480 2024-04-24T09:03:35.886 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:35.886 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34460,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T09:03:35.886 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 7 2024-04-24T09:03:35.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.888+0000 7f56cbe00640 1 -- 172.21.15.69:0/2344943926 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56b00a3bb0 msgr2=0x7f56b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.888+0000 7f56cbe00640 1 --2- 172.21.15.69:0/2344943926 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56b00a3bb0 0x7f56b00a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f56b4006010 tx=0x7f56b4000bc0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.888+0000 7f56cbe00640 1 -- 172.21.15.69:0/2344943926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56cc108480 msgr2=0x7f56cc19e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.888+0000 7f56cbe00640 1 --2- 172.21.15.69:0/2344943926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56cc108480 0x7f56cc19e740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00e9c0 tx=0x7f56bc00ee80 comp rx=0 tx=0).stop 2024-04-24T09:03:35.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.888+0000 7f56cbe00640 1 -- 172.21.15.69:0/2344943926 shutdown_connections 2024-04-24T09:03:35.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.888+0000 7f56cbe00640 1 --2- 172.21.15.69:0/2344943926 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56b00a3bb0 0x7f56b00a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.888+0000 7f56cbe00640 1 --2- 172.21.15.69:0/2344943926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56cc10a030 0x7f56cc1a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.888+0000 7f56cbe00640 1 --2- 172.21.15.69:0/2344943926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56cc109670 0x7f56cc19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.888+0000 7f56cbe00640 1 --2- 172.21.15.69:0/2344943926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56cc108480 0x7f56cc19e740 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.888+0000 7f56cbe00640 1 -- 172.21.15.69:0/2344943926 >> 172.21.15.69:0/2344943926 conn(0x7f56cc077440 msgr2=0x7f56cc10c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:35.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.889+0000 7f56cbe00640 1 -- 172.21.15.69:0/2344943926 shutdown_connections 2024-04-24T09:03:35.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:35.889+0000 7f56cbe00640 1 -- 172.21.15.69:0/2344943926 wait complete. 2024-04-24T09:03:36.691 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:36 smithi081 ceph-mon[30961]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:36.692 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:36 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2344943926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:36.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:36 smithi130 ceph-mon[31553]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:36.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:36 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2344943926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:36.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:36 smithi069 ceph-mon[28170]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:36.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:36 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2344943926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:37.034 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 24 09:03:36 smithi130 podman[62504]: 2024-04-24 09:03:36.729295945 +0000 UTC m=+0.152552835 container create 2a62d12c03c39f9e455ef2fa87d5c882afc6472f036e8bbd361440e74c64e08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-24T09:03:37.034 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 24 09:03:37 smithi130 podman[62504]: 2024-04-24 09:03:37.033705911 +0000 UTC m=+0.456962838 container init 2a62d12c03c39f9e455ef2fa87d5c882afc6472f036e8bbd361440e74c64e08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0) 2024-04-24T09:03:37.454 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 24 09:03:37 smithi130 podman[62504]: 2024-04-24 09:03:37.044184801 +0000 UTC m=+0.467441698 container start 2a62d12c03c39f9e455ef2fa87d5c882afc6472f036e8bbd361440e74c64e08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-24T09:03:37.454 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 24 09:03:37 smithi130 ceph-mds[62523]: -- 172.21.15.130:0/2311141026 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5559ea48fa40 con 0x5559ea463000 2024-04-24T09:03:37.455 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 24 09:03:37 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-c[62519]: starting mds.c at 2024-04-24T09:03:37.455 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 24 09:03:37 smithi130 bash[62504]: 2a62d12c03c39f9e455ef2fa87d5c882afc6472f036e8bbd361440e74c64e08f 2024-04-24T09:03:37.455 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 24 09:03:37 smithi130 systemd[1]: Started Ceph mds.c for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T09:03:37.910 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:38.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.121+0000 7fee4da00640 1 -- 172.21.15.69:0/1419361991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee48104260 msgr2=0x7fee48104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:38.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.121+0000 7fee4da00640 1 --2- 172.21.15.69:0/1419361991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee48104260 0x7fee48104680 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fee38009900 tx=0x7fee3802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:03:38.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.121+0000 7fee4da00640 1 -- 172.21.15.69:0/1419361991 shutdown_connections 2024-04-24T09:03:38.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.121+0000 7fee4da00640 1 --2- 172.21.15.69:0/1419361991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee48105e10 0x7fee4810c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:38.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.121+0000 7fee4da00640 1 --2- 172.21.15.69:0/1419361991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fee48105450 0x7fee481058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:38.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.121+0000 7fee4da00640 1 --2- 172.21.15.69:0/1419361991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee48104260 0x7fee48104680 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:38.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.121+0000 7fee4da00640 1 -- 172.21.15.69:0/1419361991 >> 172.21.15.69:0/1419361991 conn(0x7fee480ff7f0 msgr2=0x7fee48101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:38.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.121+0000 7fee4da00640 1 -- 172.21.15.69:0/1419361991 shutdown_connections 2024-04-24T09:03:38.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.121+0000 7fee4da00640 1 -- 172.21.15.69:0/1419361991 wait complete. 2024-04-24T09:03:38.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.122+0000 7fee4da00640 1 Processor -- start 2024-04-24T09:03:38.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.122+0000 7fee4da00640 1 -- start start 2024-04-24T09:03:38.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.122+0000 7fee4da00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee48104260 0x7fee4819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:38.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.122+0000 7fee4da00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fee48105450 0x7fee4819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:38.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.123+0000 7fee4da00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee48105e10 0x7fee481a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:38.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.123+0000 7fee4da00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fee4810fda0 con 0x7fee48105e10 2024-04-24T09:03:38.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.123+0000 7fee4da00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fee4810fc20 con 0x7fee48104260 2024-04-24T09:03:38.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.123+0000 7fee4da00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fee4810ff20 con 0x7fee48105450 2024-04-24T09:03:38.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.123+0000 7fee47400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee48105e10 0x7fee481a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:38.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.123+0000 7fee46a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee48104260 0x7fee4819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:38.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.123+0000 7fee46a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee48104260 0x7fee4819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.69:54648/0 (socket says 172.21.15.69:54648) 2024-04-24T09:03:38.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.123+0000 7fee46a00640 1 -- 172.21.15.69:0/2548760338 learned_addr learned my addr 172.21.15.69:0/2548760338 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:38.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.123+0000 7fee46000640 1 --2- 172.21.15.69:0/2548760338 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fee48105450 0x7fee4819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:38.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.123+0000 7fee46a00640 1 -- 172.21.15.69:0/2548760338 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fee48105450 msgr2=0x7fee4819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:38.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.123+0000 7fee46a00640 1 --2- 172.21.15.69:0/2548760338 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fee48105450 0x7fee4819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:38.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.123+0000 7fee46a00640 1 -- 172.21.15.69:0/2548760338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee48105e10 msgr2=0x7fee481a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:38.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.123+0000 7fee46a00640 1 --2- 172.21.15.69:0/2548760338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee48105e10 0x7fee481a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:38.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.123+0000 7fee46a00640 1 -- 172.21.15.69:0/2548760338 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee481a2130 con 0x7fee48104260 2024-04-24T09:03:38.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.123+0000 7fee47400640 1 --2- 172.21.15.69:0/2548760338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee48105e10 0x7fee481a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:03:38.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.124+0000 7fee46a00640 1 --2- 172.21.15.69:0/2548760338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee48104260 0x7fee4819a510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fee380314e0 tx=0x7fee380042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:38.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.124+0000 7fee3f400640 1 -- 172.21.15.69:0/2548760338 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3803f030 con 0x7fee48104260 2024-04-24T09:03:38.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.124+0000 7fee4da00640 1 -- 172.21.15.69:0/2548760338 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee481a23c0 con 0x7fee48104260 2024-04-24T09:03:38.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.124+0000 7fee3f400640 1 -- 172.21.15.69:0/2548760338 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee38003cb0 con 0x7fee48104260 2024-04-24T09:03:38.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.124+0000 7fee3f400640 1 -- 172.21.15.69:0/2548760338 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee38035070 con 0x7fee48104260 2024-04-24T09:03:38.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.124+0000 7fee4da00640 1 -- 172.21.15.69:0/2548760338 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee481a28b0 con 0x7fee48104260 2024-04-24T09:03:38.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.125+0000 7fee4da00640 1 -- 172.21.15.69:0/2548760338 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee4810b6a0 con 0x7fee48104260 2024-04-24T09:03:38.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.127+0000 7fee3f400640 1 -- 172.21.15.69:0/2548760338 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fee38042030 con 0x7fee48104260 2024-04-24T09:03:38.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.128+0000 7fee3f400640 1 --2- 172.21.15.69:0/2548760338 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fee2c0a3bb0 0x7fee2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:38.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.129+0000 7fee3f400640 1 -- 172.21.15.69:0/2548760338 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fee380f0a70 con 0x7fee48104260 2024-04-24T09:03:38.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.129+0000 7fee46000640 1 --2- 172.21.15.69:0/2548760338 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fee2c0a3bb0 0x7fee2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:38.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.129+0000 7fee46000640 1 --2- 172.21.15.69:0/2548760338 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fee2c0a3bb0 0x7fee2c0a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fee30005fd0 tx=0x7fee30005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:38.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.135+0000 7fee3f400640 1 -- 172.21.15.69:0/2548760338 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee380bd850 con 0x7fee48104260 2024-04-24T09:03:38.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.375+0000 7fee4da00640 1 -- 172.21.15.69:0/2548760338 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee4802d620 con 0x7fee48104260 2024-04-24T09:03:38.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.377+0000 7fee3f400640 1 -- 172.21.15.69:0/2548760338 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7fee380bd150 con 0x7fee48104260 2024-04-24T09:03:38.377 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:38.377 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34460,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T09:03:38.377 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 8 2024-04-24T09:03:38.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.379+0000 7fee4da00640 1 -- 172.21.15.69:0/2548760338 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fee2c0a3bb0 msgr2=0x7fee2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:38.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.379+0000 7fee4da00640 1 --2- 172.21.15.69:0/2548760338 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fee2c0a3bb0 0x7fee2c0a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fee30005fd0 tx=0x7fee30005e20 comp rx=0 tx=0).stop 2024-04-24T09:03:38.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.380+0000 7fee4da00640 1 -- 172.21.15.69:0/2548760338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee48104260 msgr2=0x7fee4819a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:38.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.380+0000 7fee4da00640 1 --2- 172.21.15.69:0/2548760338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee48104260 0x7fee4819a510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fee380314e0 tx=0x7fee380042e0 comp rx=0 tx=0).stop 2024-04-24T09:03:38.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.380+0000 7fee4da00640 1 -- 172.21.15.69:0/2548760338 shutdown_connections 2024-04-24T09:03:38.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.380+0000 7fee4da00640 1 --2- 172.21.15.69:0/2548760338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee48105e10 0x7fee481a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:38.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.380+0000 7fee4da00640 1 --2- 172.21.15.69:0/2548760338 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fee48105450 0x7fee4819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:38.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.380+0000 7fee4da00640 1 --2- 172.21.15.69:0/2548760338 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fee2c0a3bb0 0x7fee2c0a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:38.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.380+0000 7fee4da00640 1 --2- 172.21.15.69:0/2548760338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee48104260 0x7fee4819a510 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:38.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.380+0000 7fee4da00640 1 -- 172.21.15.69:0/2548760338 >> 172.21.15.69:0/2548760338 conn(0x7fee480ff7f0 msgr2=0x7fee48108870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:38.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.380+0000 7fee4da00640 1 -- 172.21.15.69:0/2548760338 shutdown_connections 2024-04-24T09:03:38.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:38.380+0000 7fee4da00640 1 -- 172.21.15.69:0/2548760338 wait complete. 2024-04-24T09:03:38.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:38 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:38.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:38 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:38.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:38 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:38.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:38 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:38.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:38 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:38.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:38 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:38.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:38 smithi081 ceph-mon[30961]: Deploying daemon mds.h on smithi081 2024-04-24T09:03:38.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:38 smithi081 ceph-mon[30961]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:38.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:38 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.130:6836/2790982303,v1:172.21.15.130:6837/2790982303] up:boot 2024-04-24T09:03:38.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:38 smithi081 ceph-mon[30961]: fsmap 7 up:standby 2024-04-24T09:03:38.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:38 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T09:03:38.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:38.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:38.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:38.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:38.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:38.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:38.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:38 smithi069 ceph-mon[28170]: Deploying daemon mds.h on smithi081 2024-04-24T09:03:38.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:38 smithi069 ceph-mon[28170]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:38.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:38 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.130:6836/2790982303,v1:172.21.15.130:6837/2790982303] up:boot 2024-04-24T09:03:38.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:38 smithi069 ceph-mon[28170]: fsmap 7 up:standby 2024-04-24T09:03:38.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:38 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T09:03:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:38 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:38 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:38 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:38 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:38 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:38 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:38 smithi130 ceph-mon[31553]: Deploying daemon mds.h on smithi081 2024-04-24T09:03:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:38 smithi130 ceph-mon[31553]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:38 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.130:6836/2790982303,v1:172.21.15.130:6837/2790982303] up:boot 2024-04-24T09:03:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:38 smithi130 ceph-mon[31553]: fsmap 7 up:standby 2024-04-24T09:03:38.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:38 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T09:03:39.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:39 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2548760338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:39.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:39 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2548760338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:39.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:39 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2548760338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:40.399 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:40.491 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:40 smithi081 ceph-mon[30961]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:40.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:40 smithi069 ceph-mon[28170]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:40.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.611+0000 7f1afc200640 1 -- 172.21.15.69:0/4120379155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1af4103c00 msgr2=0x7f1af410a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:40.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.611+0000 7f1afc200640 1 --2- 172.21.15.69:0/4120379155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1af4103c00 0x7f1af410a490 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1aec009ca0 tx=0x7f1aec02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:40.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.611+0000 7f1afc200640 1 -- 172.21.15.69:0/4120379155 shutdown_connections 2024-04-24T09:03:40.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.611+0000 7f1afc200640 1 --2- 172.21.15.69:0/4120379155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1af4103c00 0x7f1af410a490 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:40.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.611+0000 7f1afc200640 1 --2- 172.21.15.69:0/4120379155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af4103240 0x7f1af41036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:40.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.611+0000 7f1afc200640 1 --2- 172.21.15.69:0/4120379155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1af4102050 0x7f1af4102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:40.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.611+0000 7f1afc200640 1 -- 172.21.15.69:0/4120379155 >> 172.21.15.69:0/4120379155 conn(0x7f1af40fd5e0 msgr2=0x7f1af40ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:40.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.611+0000 7f1afc200640 1 -- 172.21.15.69:0/4120379155 shutdown_connections 2024-04-24T09:03:40.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.611+0000 7f1afc200640 1 -- 172.21.15.69:0/4120379155 wait complete. 2024-04-24T09:03:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.612+0000 7f1afc200640 1 Processor -- start 2024-04-24T09:03:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.612+0000 7f1afc200640 1 -- start start 2024-04-24T09:03:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.612+0000 7f1afc200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1af4102050 0x7f1af4198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.612+0000 7f1afc200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1af4103240 0x7f1af41987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.613+0000 7f1afc200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af4103c00 0x7f1af419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.613+0000 7f1afc200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1af410dbe0 con 0x7f1af4103240 2024-04-24T09:03:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.613+0000 7f1afc200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1af410da60 con 0x7f1af4102050 2024-04-24T09:03:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.613+0000 7f1afc200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1af410dd60 con 0x7f1af4103c00 2024-04-24T09:03:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.613+0000 7f1af8e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1af4103240 0x7f1af41987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.613+0000 7f1af8e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1af4103240 0x7f1af41987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34300/0 (socket says 172.21.15.69:34300) 2024-04-24T09:03:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.613+0000 7f1af8e00640 1 -- 172.21.15.69:0/2007105169 learned_addr learned my addr 172.21.15.69:0/2007105169 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.613+0000 7f1af9800640 1 --2- 172.21.15.69:0/2007105169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1af4102050 0x7f1af4198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.613+0000 7f1af8e00640 1 -- 172.21.15.69:0/2007105169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af4103c00 msgr2=0x7f1af419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.613+0000 7f1af8e00640 1 --2- 172.21.15.69:0/2007105169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af4103c00 0x7f1af419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.613+0000 7f1af8e00640 1 -- 172.21.15.69:0/2007105169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1af4102050 msgr2=0x7f1af4198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:40.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.613+0000 7f1af8e00640 1 --2- 172.21.15.69:0/2007105169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1af4102050 0x7f1af4198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:40.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.613+0000 7f1af8e00640 1 -- 172.21.15.69:0/2007105169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af419fe80 con 0x7f1af4103240 2024-04-24T09:03:40.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.613+0000 7f1af8e00640 1 --2- 172.21.15.69:0/2007105169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1af4103240 0x7f1af41987a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000d770 tx=0x7f1ae000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:40.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.614+0000 7f1af2000640 1 -- 172.21.15.69:0/2007105169 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0017070 con 0x7f1af4103240 2024-04-24T09:03:40.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.614+0000 7f1af2000640 1 -- 172.21.15.69:0/2007105169 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ae00054d0 con 0x7f1af4103240 2024-04-24T09:03:40.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.614+0000 7f1af2000640 1 -- 172.21.15.69:0/2007105169 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0004cf0 con 0x7f1af4103240 2024-04-24T09:03:40.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.614+0000 7f1afc200640 1 -- 172.21.15.69:0/2007105169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af41a0170 con 0x7f1af4103240 2024-04-24T09:03:40.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.614+0000 7f1afc200640 1 -- 172.21.15.69:0/2007105169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af41a0670 con 0x7f1af4103240 2024-04-24T09:03:40.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.615+0000 7f1afc200640 1 -- 172.21.15.69:0/2007105169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af4109370 con 0x7f1af4103240 2024-04-24T09:03:40.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.618+0000 7f1af2000640 1 -- 172.21.15.69:0/2007105169 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1ae0004e90 con 0x7f1af4103240 2024-04-24T09:03:40.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.619+0000 7f1af2000640 1 --2- 172.21.15.69:0/2007105169 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f1adc0a3bb0 0x7f1adc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.619+0000 7f1af2000640 1 -- 172.21.15.69:0/2007105169 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1ae00cfa70 con 0x7f1af4103240 2024-04-24T09:03:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.619+0000 7f1af9800640 1 --2- 172.21.15.69:0/2007105169 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f1adc0a3bb0 0x7f1adc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.620+0000 7f1af9800640 1 --2- 172.21.15.69:0/2007105169 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f1adc0a3bb0 0x7f1adc0a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1af41030a0 tx=0x7f1ae8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:40.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.624+0000 7f1af2000640 1 -- 172.21.15.69:0/2007105169 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ae009c850 con 0x7f1af4103240 2024-04-24T09:03:40.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:40 smithi130 ceph-mon[31553]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:40.817 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:03:40 smithi081 systemd[1]: Starting Ceph mds.h for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:03:40.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.860+0000 7f1afc200640 1 -- 172.21.15.69:0/2007105169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1af4062ca0 con 0x7f1af4103240 2024-04-24T09:03:40.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.862+0000 7f1af2000640 1 -- 172.21.15.69:0/2007105169 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f1ae009c150 con 0x7f1af4103240 2024-04-24T09:03:40.862 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:40.862 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34460,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T09:03:40.862 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 8 2024-04-24T09:03:40.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.864+0000 7f1afc200640 1 -- 172.21.15.69:0/2007105169 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f1adc0a3bb0 msgr2=0x7f1adc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:40.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.864+0000 7f1afc200640 1 --2- 172.21.15.69:0/2007105169 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f1adc0a3bb0 0x7f1adc0a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1af41030a0 tx=0x7f1ae8005f90 comp rx=0 tx=0).stop 2024-04-24T09:03:40.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.864+0000 7f1afc200640 1 -- 172.21.15.69:0/2007105169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1af4103240 msgr2=0x7f1af41987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:40.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.864+0000 7f1afc200640 1 --2- 172.21.15.69:0/2007105169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1af4103240 0x7f1af41987a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000d770 tx=0x7f1ae000dc30 comp rx=0 tx=0).stop 2024-04-24T09:03:40.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.864+0000 7f1afc200640 1 -- 172.21.15.69:0/2007105169 shutdown_connections 2024-04-24T09:03:40.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.864+0000 7f1afc200640 1 --2- 172.21.15.69:0/2007105169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af4103c00 0x7f1af419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:40.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.864+0000 7f1afc200640 1 --2- 172.21.15.69:0/2007105169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1af4103240 0x7f1af41987a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:40.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.864+0000 7f1afc200640 1 --2- 172.21.15.69:0/2007105169 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f1adc0a3bb0 0x7f1adc0a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:40.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.864+0000 7f1afc200640 1 --2- 172.21.15.69:0/2007105169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1af4102050 0x7f1af4198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:40.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.864+0000 7f1afc200640 1 -- 172.21.15.69:0/2007105169 >> 172.21.15.69:0/2007105169 conn(0x7f1af40fd5e0 msgr2=0x7f1af4106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:40.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.865+0000 7f1afc200640 1 -- 172.21.15.69:0/2007105169 shutdown_connections 2024-04-24T09:03:40.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:40.865+0000 7f1afc200640 1 -- 172.21.15.69:0/2007105169 wait complete. 2024-04-24T09:03:41.159 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:03:40 smithi081 podman[61118]: 2024-04-24 09:03:40.751193621 +0000 UTC m=+0.033235920 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:03:41.160 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:03:40 smithi081 podman[61118]: 2024-04-24 09:03:40.864559056 +0000 UTC m=+0.146601374 container create e2dbdb3a3aad4d33099296c2ed15f4b6f555f128ec8c027f8f1d7986a48243e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-24T09:03:41.160 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:03:41 smithi081 podman[61118]: 2024-04-24 09:03:41.16016241 +0000 UTC m=+0.442204735 container init e2dbdb3a3aad4d33099296c2ed15f4b6f555f128ec8c027f8f1d7986a48243e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-24T09:03:41.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:41 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2007105169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:41.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:41 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2007105169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:41.568 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:03:41 smithi081 podman[61118]: 2024-04-24 09:03:41.170432422 +0000 UTC m=+0.452474711 container start e2dbdb3a3aad4d33099296c2ed15f4b6f555f128ec8c027f8f1d7986a48243e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T09:03:41.568 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:03:41 smithi081 ceph-mds[61137]: -- 172.21.15.81:0/1992245056 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a84ebefa40 con 0x55a84ebc2c00 2024-04-24T09:03:41.568 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:03:41 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: starting mds.h at 2024-04-24T09:03:41.568 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:03:41 smithi081 bash[61118]: e2dbdb3a3aad4d33099296c2ed15f4b6f555f128ec8c027f8f1d7986a48243e6 2024-04-24T09:03:41.569 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:03:41 smithi081 systemd[1]: Started Ceph mds.h for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T09:03:41.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:41 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2007105169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:42.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:42 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] up:boot 2024-04-24T09:03:42.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:42 smithi069 ceph-mon[28170]: fsmap 8 up:standby 2024-04-24T09:03:42.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T09:03:42.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:42 smithi069 ceph-mon[28170]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:42.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:42.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:42.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:42.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:42.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:42.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:42 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:42.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:42 smithi069 ceph-mon[28170]: Deploying daemon mds.i on smithi130 2024-04-24T09:03:42.617 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:42 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] up:boot 2024-04-24T09:03:42.617 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:42 smithi130 ceph-mon[31553]: fsmap 8 up:standby 2024-04-24T09:03:42.617 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T09:03:42.617 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:42 smithi130 ceph-mon[31553]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:42.617 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:42.617 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:42.618 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:42.618 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:42.618 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:42.618 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:42 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:42.618 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:42 smithi130 ceph-mon[31553]: Deploying daemon mds.i on smithi130 2024-04-24T09:03:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:42 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] up:boot 2024-04-24T09:03:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:42 smithi081 ceph-mon[30961]: fsmap 8 up:standby 2024-04-24T09:03:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T09:03:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:42 smithi081 ceph-mon[30961]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:42.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:42.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:42.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:42.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:42.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:42.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:42 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:42.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:42 smithi081 ceph-mon[30961]: Deploying daemon mds.i on smithi130 2024-04-24T09:03:42.885 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:43.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.097+0000 7fa89be00640 1 -- 172.21.15.69:0/947398840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa89c10a030 msgr2=0x7fa89c1108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:43.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.097+0000 7fa89be00640 1 --2- 172.21.15.69:0/947398840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa89c10a030 0x7fa89c1108c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa894009ca0 tx=0x7fa89402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:43.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.097+0000 7fa89be00640 1 -- 172.21.15.69:0/947398840 shutdown_connections 2024-04-24T09:03:43.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.097+0000 7fa89be00640 1 --2- 172.21.15.69:0/947398840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa89c10a030 0x7fa89c1108c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:43.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.097+0000 7fa89be00640 1 --2- 172.21.15.69:0/947398840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa89c109670 0x7fa89c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:43.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.097+0000 7fa89be00640 1 --2- 172.21.15.69:0/947398840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa89c108480 0x7fa89c1088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:43.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.097+0000 7fa89be00640 1 -- 172.21.15.69:0/947398840 >> 172.21.15.69:0/947398840 conn(0x7fa89c077440 msgr2=0x7fa89c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:43.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.097+0000 7fa89be00640 1 -- 172.21.15.69:0/947398840 shutdown_connections 2024-04-24T09:03:43.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.097+0000 7fa89be00640 1 -- 172.21.15.69:0/947398840 wait complete. 2024-04-24T09:03:43.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.098+0000 7fa89be00640 1 Processor -- start 2024-04-24T09:03:43.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.098+0000 7fa89be00640 1 -- start start 2024-04-24T09:03:43.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.098+0000 7fa89be00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa89c108480 0x7fa89c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:43.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.098+0000 7fa89be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa89c109670 0x7fa89c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:43.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.098+0000 7fa89be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa89c10a030 0x7fa89c1a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:43.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.098+0000 7fa89be00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c114010 con 0x7fa89c109670 2024-04-24T09:03:43.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.098+0000 7fa89be00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c113e90 con 0x7fa89c108480 2024-04-24T09:03:43.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.099+0000 7fa89be00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c114190 con 0x7fa89c10a030 2024-04-24T09:03:43.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.099+0000 7fa892000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa89c109670 0x7fa89c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.099+0000 7fa892000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa89c109670 0x7fa89c19ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34790/0 (socket says 172.21.15.69:34790) 2024-04-24T09:03:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.099+0000 7fa892000640 1 -- 172.21.15.69:0/3222378747 learned_addr learned my addr 172.21.15.69:0/3222378747 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.099+0000 7fa89b400640 1 --2- 172.21.15.69:0/3222378747 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa89c10a030 0x7fa89c1a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.099+0000 7fa89aa00640 1 --2- 172.21.15.69:0/3222378747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa89c108480 0x7fa89c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.099+0000 7fa892000640 1 -- 172.21.15.69:0/3222378747 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa89c10a030 msgr2=0x7fa89c1a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.099+0000 7fa892000640 1 --2- 172.21.15.69:0/3222378747 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa89c10a030 0x7fa89c1a5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.099+0000 7fa892000640 1 -- 172.21.15.69:0/3222378747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa89c108480 msgr2=0x7fa89c19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.099+0000 7fa892000640 1 --2- 172.21.15.69:0/3222378747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa89c108480 0x7fa89c19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.099+0000 7fa892000640 1 -- 172.21.15.69:0/3222378747 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89c1a6260 con 0x7fa89c109670 2024-04-24T09:03:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.099+0000 7fa892000640 1 --2- 172.21.15.69:0/3222378747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa89c109670 0x7fa89c19ebd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa88400e9f0 tx=0x7fa88400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.100+0000 7fa893e00640 1 -- 172.21.15.69:0/3222378747 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88400cd80 con 0x7fa89c109670 2024-04-24T09:03:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.100+0000 7fa893e00640 1 -- 172.21.15.69:0/3222378747 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8840054d0 con 0x7fa89c109670 2024-04-24T09:03:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.100+0000 7fa893e00640 1 -- 172.21.15.69:0/3222378747 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa884010490 con 0x7fa89c109670 2024-04-24T09:03:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.100+0000 7fa89be00640 1 -- 172.21.15.69:0/3222378747 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89c1a6550 con 0x7fa89c109670 2024-04-24T09:03:43.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.100+0000 7fa89be00640 1 -- 172.21.15.69:0/3222378747 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa89c1a6a50 con 0x7fa89c109670 2024-04-24T09:03:43.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.101+0000 7fa89be00640 1 -- 172.21.15.69:0/3222378747 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa89c10f740 con 0x7fa89c109670 2024-04-24T09:03:43.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.103+0000 7fa893e00640 1 -- 172.21.15.69:0/3222378747 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa884005020 con 0x7fa89c109670 2024-04-24T09:03:43.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.105+0000 7fa893e00640 1 --2- 172.21.15.69:0/3222378747 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa8800a3c30 0x7fa8800a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:43.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.105+0000 7fa893e00640 1 -- 172.21.15.69:0/3222378747 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa8840d0450 con 0x7fa89c109670 2024-04-24T09:03:43.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.105+0000 7fa89aa00640 1 --2- 172.21.15.69:0/3222378747 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa8800a3c30 0x7fa8800a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:43.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.106+0000 7fa89aa00640 1 --2- 172.21.15.69:0/3222378747 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa8800a3c30 0x7fa8800a60f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa89c1094d0 tx=0x7fa88c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:43.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.109+0000 7fa893e00640 1 -- 172.21.15.69:0/3222378747 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa88409d230 con 0x7fa89c109670 2024-04-24T09:03:43.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.352+0000 7fa89be00640 1 -- 172.21.15.69:0/3222378747 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa89c02d620 con 0x7fa89c109670 2024-04-24T09:03:43.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.353+0000 7fa893e00640 1 -- 172.21.15.69:0/3222378747 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7fa88409cb30 con 0x7fa89c109670 2024-04-24T09:03:43.353 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:43.353 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34460,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34496,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T09:03:43.353 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 9 2024-04-24T09:03:43.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.356+0000 7fa89be00640 1 -- 172.21.15.69:0/3222378747 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa8800a3c30 msgr2=0x7fa8800a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:43.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.356+0000 7fa89be00640 1 --2- 172.21.15.69:0/3222378747 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa8800a3c30 0x7fa8800a60f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa89c1094d0 tx=0x7fa88c005f90 comp rx=0 tx=0).stop 2024-04-24T09:03:43.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.356+0000 7fa89be00640 1 -- 172.21.15.69:0/3222378747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa89c109670 msgr2=0x7fa89c19ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:43.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.356+0000 7fa89be00640 1 --2- 172.21.15.69:0/3222378747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa89c109670 0x7fa89c19ebd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa88400e9f0 tx=0x7fa88400edf0 comp rx=0 tx=0).stop 2024-04-24T09:03:43.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.356+0000 7fa89be00640 1 -- 172.21.15.69:0/3222378747 shutdown_connections 2024-04-24T09:03:43.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.356+0000 7fa89be00640 1 --2- 172.21.15.69:0/3222378747 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa89c10a030 0x7fa89c1a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:43.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.356+0000 7fa89be00640 1 --2- 172.21.15.69:0/3222378747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa89c109670 0x7fa89c19ebd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:43.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.356+0000 7fa89be00640 1 --2- 172.21.15.69:0/3222378747 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa8800a3c30 0x7fa8800a60f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:43.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.356+0000 7fa89be00640 1 --2- 172.21.15.69:0/3222378747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa89c108480 0x7fa89c19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:43.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.356+0000 7fa89be00640 1 -- 172.21.15.69:0/3222378747 >> 172.21.15.69:0/3222378747 conn(0x7fa89c077440 msgr2=0x7fa89c10c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:43.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.357+0000 7fa89be00640 1 -- 172.21.15.69:0/3222378747 shutdown_connections 2024-04-24T09:03:43.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:43.357+0000 7fa89be00640 1 -- 172.21.15.69:0/3222378747 wait complete. 2024-04-24T09:03:43.616 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:43 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3222378747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:43.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:43 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3222378747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:43.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:43 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3222378747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:44.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:44 smithi130 ceph-mon[31553]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:44.704 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 24 09:03:44 smithi130 systemd[1]: Starting Ceph mds.i for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:03:44.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:44 smithi081 ceph-mon[30961]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:44.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:44 smithi069 ceph-mon[28170]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:45.122 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 24 09:03:44 smithi130 podman[62811]: 2024-04-24 09:03:44.733183599 +0000 UTC m=+0.032390546 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:03:45.122 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 24 09:03:44 smithi130 podman[62811]: 2024-04-24 09:03:44.833603549 +0000 UTC m=+0.132810534 container create 4a37e99aaa24d888329c929fa96a5c880b7c48359968a64ff1d17de34496abec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-24T09:03:45.122 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 24 09:03:45 smithi130 podman[62811]: 2024-04-24 09:03:45.121848777 +0000 UTC m=+0.421055802 container init 4a37e99aaa24d888329c929fa96a5c880b7c48359968a64ff1d17de34496abec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-24T09:03:45.376 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:45.454 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 24 09:03:45 smithi130 podman[62811]: 2024-04-24 09:03:45.13219602 +0000 UTC m=+0.431403011 container start 4a37e99aaa24d888329c929fa96a5c880b7c48359968a64ff1d17de34496abec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-04-24T09:03:45.454 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 24 09:03:45 smithi130 ceph-mds[62830]: -- 172.21.15.130:0/2730432793 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5643c54e7a40 con 0x5643c54ba800 2024-04-24T09:03:45.454 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 24 09:03:45 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-i[62826]: starting mds.i at 2024-04-24T09:03:45.454 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 24 09:03:45 smithi130 bash[62811]: 4a37e99aaa24d888329c929fa96a5c880b7c48359968a64ff1d17de34496abec 2024-04-24T09:03:45.454 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 24 09:03:45 smithi130 systemd[1]: Started Ceph mds.i for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T09:03:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.575+0000 7fc167a00640 1 -- 172.21.15.69:0/3020472367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc160105400 msgr2=0x7fc1601077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.575+0000 7fc167a00640 1 --2- 172.21.15.69:0/3020472367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc160105400 0x7fc1601077f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc14c009900 tx=0x7fc14c02a410 comp rx=0 tx=0).stop 2024-04-24T09:03:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.575+0000 7fc167a00640 1 -- 172.21.15.69:0/3020472367 shutdown_connections 2024-04-24T09:03:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.575+0000 7fc167a00640 1 --2- 172.21.15.69:0/3020472367 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc160107d30 0x7fc16010a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.575+0000 7fc167a00640 1 --2- 172.21.15.69:0/3020472367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc160105400 0x7fc1601077f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.575+0000 7fc167a00640 1 --2- 172.21.15.69:0/3020472367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc160102ad0 0x7fc160104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.575+0000 7fc167a00640 1 -- 172.21.15.69:0/3020472367 >> 172.21.15.69:0/3020472367 conn(0x7fc1600fc4e0 msgr2=0x7fc1600fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.575+0000 7fc167a00640 1 -- 172.21.15.69:0/3020472367 shutdown_connections 2024-04-24T09:03:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.575+0000 7fc167a00640 1 -- 172.21.15.69:0/3020472367 wait complete. 2024-04-24T09:03:45.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.576+0000 7fc167a00640 1 Processor -- start 2024-04-24T09:03:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.576+0000 7fc167a00640 1 -- start start 2024-04-24T09:03:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.576+0000 7fc167a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc160102ad0 0x7fc16019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.576+0000 7fc167a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc160105400 0x7fc16019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.576+0000 7fc167a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc160107d30 0x7fc1601a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.576+0000 7fc167a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc16010dba0 con 0x7fc160105400 2024-04-24T09:03:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.576+0000 7fc167a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc16010da20 con 0x7fc160107d30 2024-04-24T09:03:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.577+0000 7fc167a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc16010dd20 con 0x7fc160102ad0 2024-04-24T09:03:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.577+0000 7fc15fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc160105400 0x7fc16019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.577+0000 7fc15fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc160105400 0x7fc16019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34820/0 (socket says 172.21.15.69:34820) 2024-04-24T09:03:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.577+0000 7fc15fe00640 1 -- 172.21.15.69:0/1678073110 learned_addr learned my addr 172.21.15.69:0/1678073110 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.577+0000 7fc165000640 1 --2- 172.21.15.69:0/1678073110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc160102ad0 0x7fc16019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.577+0000 7fc165a00640 1 --2- 172.21.15.69:0/1678073110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc160107d30 0x7fc1601a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.577+0000 7fc15fe00640 1 -- 172.21.15.69:0/1678073110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc160102ad0 msgr2=0x7fc16019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:45.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.577+0000 7fc15fe00640 1 --2- 172.21.15.69:0/1678073110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc160102ad0 0x7fc16019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:45.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.577+0000 7fc15fe00640 1 -- 172.21.15.69:0/1678073110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc160107d30 msgr2=0x7fc1601a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:45.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.577+0000 7fc15fe00640 1 --2- 172.21.15.69:0/1678073110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc160107d30 0x7fc1601a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:45.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.577+0000 7fc15fe00640 1 -- 172.21.15.69:0/1678073110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1601a2070 con 0x7fc160105400 2024-04-24T09:03:45.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.577+0000 7fc165a00640 1 --2- 172.21.15.69:0/1678073110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc160107d30 0x7fc1601a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:45.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.578+0000 7fc15fe00640 1 --2- 172.21.15.69:0/1678073110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc160105400 0x7fc16019a990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0314e0 tx=0x7fc14c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:45.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.578+0000 7fc15d600640 1 -- 172.21.15.69:0/1678073110 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c03f070 con 0x7fc160105400 2024-04-24T09:03:45.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.578+0000 7fc15d600640 1 -- 172.21.15.69:0/1678073110 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc14c031e40 con 0x7fc160105400 2024-04-24T09:03:45.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.578+0000 7fc15d600640 1 -- 172.21.15.69:0/1678073110 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c035030 con 0x7fc160105400 2024-04-24T09:03:45.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.578+0000 7fc167a00640 1 -- 172.21.15.69:0/1678073110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1601a2300 con 0x7fc160105400 2024-04-24T09:03:45.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.578+0000 7fc167a00640 1 -- 172.21.15.69:0/1678073110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1601a27f0 con 0x7fc160105400 2024-04-24T09:03:45.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.579+0000 7fc167a00640 1 -- 172.21.15.69:0/1678073110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc160109880 con 0x7fc160105400 2024-04-24T09:03:45.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.583+0000 7fc15d600640 1 -- 172.21.15.69:0/1678073110 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc14c042070 con 0x7fc160105400 2024-04-24T09:03:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.584+0000 7fc15d600640 1 --2- 172.21.15.69:0/1678073110 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc1480a3bb0 0x7fc1480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.584+0000 7fc15d600640 1 -- 172.21.15.69:0/1678073110 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc14c0f0a50 con 0x7fc160105400 2024-04-24T09:03:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.584+0000 7fc165000640 1 --2- 172.21.15.69:0/1678073110 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc1480a3bb0 0x7fc1480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.585+0000 7fc165000640 1 --2- 172.21.15.69:0/1678073110 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc1480a3bb0 0x7fc1480a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc154005e10 tx=0x7fc154005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:45.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.589+0000 7fc15d600640 1 -- 172.21.15.69:0/1678073110 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc14c0bd8b0 con 0x7fc160105400 2024-04-24T09:03:45.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.832+0000 7fc167a00640 1 -- 172.21.15.69:0/1678073110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc16002d870 con 0x7fc160105400 2024-04-24T09:03:45.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.833+0000 7fc15d600640 1 -- 172.21.15.69:0/1678073110 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8004 (secure 0 0 0) 0x7fc14c0bd1b0 con 0x7fc160105400 2024-04-24T09:03:45.833 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:45.833 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34460,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34496,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T09:03:45.833 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 10 2024-04-24T09:03:45.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.836+0000 7fc167a00640 1 -- 172.21.15.69:0/1678073110 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc1480a3bb0 msgr2=0x7fc1480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:45.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.836+0000 7fc167a00640 1 --2- 172.21.15.69:0/1678073110 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc1480a3bb0 0x7fc1480a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc154005e10 tx=0x7fc154005da0 comp rx=0 tx=0).stop 2024-04-24T09:03:45.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.836+0000 7fc167a00640 1 -- 172.21.15.69:0/1678073110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc160105400 msgr2=0x7fc16019a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:45.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.836+0000 7fc167a00640 1 --2- 172.21.15.69:0/1678073110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc160105400 0x7fc16019a990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0314e0 tx=0x7fc14c003d50 comp rx=0 tx=0).stop 2024-04-24T09:03:45.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.836+0000 7fc167a00640 1 -- 172.21.15.69:0/1678073110 shutdown_connections 2024-04-24T09:03:45.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.836+0000 7fc167a00640 1 --2- 172.21.15.69:0/1678073110 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc1480a3bb0 0x7fc1480a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:45.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.836+0000 7fc167a00640 1 --2- 172.21.15.69:0/1678073110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc160107d30 0x7fc1601a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:45.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.836+0000 7fc167a00640 1 --2- 172.21.15.69:0/1678073110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc160105400 0x7fc16019a990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:45.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.836+0000 7fc167a00640 1 --2- 172.21.15.69:0/1678073110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc160102ad0 0x7fc16019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:45.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.836+0000 7fc167a00640 1 -- 172.21.15.69:0/1678073110 >> 172.21.15.69:0/1678073110 conn(0x7fc1600fc4e0 msgr2=0x7fc160102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:45.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.836+0000 7fc167a00640 1 -- 172.21.15.69:0/1678073110 shutdown_connections 2024-04-24T09:03:45.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:45.836+0000 7fc167a00640 1 -- 172.21.15.69:0/1678073110 wait complete. 2024-04-24T09:03:46.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:46.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:46.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:46 smithi081 ceph-mon[30961]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:46.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:46.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:46.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:46.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:46.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:46 smithi081 ceph-mon[30961]: Deploying daemon mds.g on smithi069 2024-04-24T09:03:46.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:46 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.130:6838/4193111366,v1:172.21.15.130:6839/4193111366] up:boot 2024-04-24T09:03:46.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:46 smithi081 ceph-mon[30961]: fsmap 9 up:standby 2024-04-24T09:03:46.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:46 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T09:03:46.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:46 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1678073110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:46.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:46.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:46.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:46 smithi069 ceph-mon[28170]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:46.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:46.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:46.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:46.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:46.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:46 smithi069 ceph-mon[28170]: Deploying daemon mds.g on smithi069 2024-04-24T09:03:46.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:46 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.130:6838/4193111366,v1:172.21.15.130:6839/4193111366] up:boot 2024-04-24T09:03:46.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:46 smithi069 ceph-mon[28170]: fsmap 9 up:standby 2024-04-24T09:03:46.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:46 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T09:03:46.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:46 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1678073110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:46 smithi130 ceph-mon[31553]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:46 smithi130 ceph-mon[31553]: Deploying daemon mds.g on smithi069 2024-04-24T09:03:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:46 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.130:6838/4193111366,v1:172.21.15.130:6839/4193111366] up:boot 2024-04-24T09:03:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:46 smithi130 ceph-mon[31553]: fsmap 9 up:standby 2024-04-24T09:03:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:46 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T09:03:46.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:46 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1678073110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:47.855 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:48.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.270+0000 7f8919a00640 1 -- 172.21.15.69:0/2652773121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8914109000 msgr2=0x7f891410b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:48.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.270+0000 7f8919a00640 1 --2- 172.21.15.69:0/2652773121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8914109000 0x7f891410b430 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f890c009c30 tx=0x7f890c02a740 comp rx=0 tx=0).stop 2024-04-24T09:03:48.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.271+0000 7f8919a00640 1 -- 172.21.15.69:0/2652773121 shutdown_connections 2024-04-24T09:03:48.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.271+0000 7f8919a00640 1 --2- 172.21.15.69:0/2652773121 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f891410e340 0x7f891410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:48.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.271+0000 7f8919a00640 1 --2- 172.21.15.69:0/2652773121 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f891410b970 0x7f891410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:48.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.271+0000 7f8919a00640 1 --2- 172.21.15.69:0/2652773121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8914109000 0x7f891410b430 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:48.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.271+0000 7f8919a00640 1 -- 172.21.15.69:0/2652773121 >> 172.21.15.69:0/2652773121 conn(0x7f891406eea0 msgr2=0x7f89140712e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:48.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.271+0000 7f8919a00640 1 -- 172.21.15.69:0/2652773121 shutdown_connections 2024-04-24T09:03:48.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.271+0000 7f8919a00640 1 -- 172.21.15.69:0/2652773121 wait complete. 2024-04-24T09:03:48.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.273+0000 7f8919a00640 1 Processor -- start 2024-04-24T09:03:48.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.273+0000 7f8919a00640 1 -- start start 2024-04-24T09:03:48.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.273+0000 7f8919a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f891410b970 0x7f89141a7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:48.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.273+0000 7f8919a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f891410e340 0x7f89141a7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:48.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.273+0000 7f8919a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89141ac8c0 0x7f89141aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:48.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.273+0000 7f8919a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f891411ca80 con 0x7f891410b970 2024-04-24T09:03:48.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.273+0000 7f8919a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f891411c900 con 0x7f89141ac8c0 2024-04-24T09:03:48.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.273+0000 7f8919a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f891411cc00 con 0x7f891410e340 2024-04-24T09:03:48.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.274+0000 7f8912a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f891410b970 0x7f89141a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:48.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.274+0000 7f8912a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f891410b970 0x7f89141a7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34842/0 (socket says 172.21.15.69:34842) 2024-04-24T09:03:48.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.274+0000 7f8912a00640 1 -- 172.21.15.69:0/531449898 learned_addr learned my addr 172.21.15.69:0/531449898 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:48.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.274+0000 7f8912000640 1 --2- 172.21.15.69:0/531449898 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f891410e340 0x7f89141a7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:48.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.274+0000 7f8913400640 1 --2- 172.21.15.69:0/531449898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89141ac8c0 0x7f89141aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:48.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.274+0000 7f8912a00640 1 -- 172.21.15.69:0/531449898 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f891410e340 msgr2=0x7f89141a7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:48.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.274+0000 7f8912a00640 1 --2- 172.21.15.69:0/531449898 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f891410e340 0x7f89141a7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:48.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.274+0000 7f8912a00640 1 -- 172.21.15.69:0/531449898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89141ac8c0 msgr2=0x7f89141aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:48.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.274+0000 7f8912a00640 1 --2- 172.21.15.69:0/531449898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89141ac8c0 0x7f89141aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:48.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.274+0000 7f8912a00640 1 -- 172.21.15.69:0/531449898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89141af320 con 0x7f891410b970 2024-04-24T09:03:48.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.275+0000 7f8912a00640 1 --2- 172.21.15.69:0/531449898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f891410b970 0x7f89141a7320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f890c009b80 tx=0x7f890c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:48.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.275+0000 7f890b400640 1 -- 172.21.15.69:0/531449898 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c041020 con 0x7f891410b970 2024-04-24T09:03:48.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.276+0000 7f8919a00640 1 -- 172.21.15.69:0/531449898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89141af5b0 con 0x7f891410b970 2024-04-24T09:03:48.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.276+0000 7f890b400640 1 -- 172.21.15.69:0/531449898 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f890c037040 con 0x7f891410b970 2024-04-24T09:03:48.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.276+0000 7f890b400640 1 -- 172.21.15.69:0/531449898 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c036070 con 0x7f891410b970 2024-04-24T09:03:48.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.276+0000 7f8919a00640 1 -- 172.21.15.69:0/531449898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89141afa50 con 0x7f891410b970 2024-04-24T09:03:48.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.277+0000 7f8919a00640 1 -- 172.21.15.69:0/531449898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89140719a0 con 0x7f891410b970 2024-04-24T09:03:48.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.279+0000 7f890b400640 1 -- 172.21.15.69:0/531449898 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f890c044020 con 0x7f891410b970 2024-04-24T09:03:48.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.281+0000 7f890b400640 1 --2- 172.21.15.69:0/531449898 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f88f80a3c30 0x7f88f80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:48.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.281+0000 7f890b400640 1 -- 172.21.15.69:0/531449898 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f890c0f0c10 con 0x7f891410b970 2024-04-24T09:03:48.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.281+0000 7f8912000640 1 --2- 172.21.15.69:0/531449898 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f88f80a3c30 0x7f88f80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:48.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.282+0000 7f8912000640 1 --2- 172.21.15.69:0/531449898 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f88f80a3c30 0x7f88f80a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f89141a8830 tx=0x7f8900009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:48.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.286+0000 7f890b400640 1 -- 172.21.15.69:0/531449898 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f890c04b020 con 0x7f891410b970 2024-04-24T09:03:48.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.538+0000 7f8919a00640 1 -- 172.21.15.69:0/531449898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f891402d840 con 0x7f891410b970 2024-04-24T09:03:48.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.539+0000 7f890b400640 1 -- 172.21.15.69:0/531449898 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8004 (secure 0 0 0) 0x7f890c04d300 con 0x7f891410b970 2024-04-24T09:03:48.539 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:48.539 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34460,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34496,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T09:03:48.539 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 10 2024-04-24T09:03:48.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.542+0000 7f8919a00640 1 -- 172.21.15.69:0/531449898 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f88f80a3c30 msgr2=0x7f88f80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:48.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.542+0000 7f8919a00640 1 --2- 172.21.15.69:0/531449898 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f88f80a3c30 0x7f88f80a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f89141a8830 tx=0x7f8900009260 comp rx=0 tx=0).stop 2024-04-24T09:03:48.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.542+0000 7f8919a00640 1 -- 172.21.15.69:0/531449898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f891410b970 msgr2=0x7f89141a7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:48.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.542+0000 7f8919a00640 1 --2- 172.21.15.69:0/531449898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f891410b970 0x7f89141a7320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f890c009b80 tx=0x7f890c003f90 comp rx=0 tx=0).stop 2024-04-24T09:03:48.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.542+0000 7f8919a00640 1 -- 172.21.15.69:0/531449898 shutdown_connections 2024-04-24T09:03:48.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.542+0000 7f8919a00640 1 --2- 172.21.15.69:0/531449898 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f88f80a3c30 0x7f88f80a60f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:48.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.542+0000 7f8919a00640 1 --2- 172.21.15.69:0/531449898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89141ac8c0 0x7f89141aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:48.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.542+0000 7f8919a00640 1 --2- 172.21.15.69:0/531449898 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f891410e340 0x7f89141a7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:48.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.542+0000 7f8919a00640 1 --2- 172.21.15.69:0/531449898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f891410b970 0x7f89141a7320 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:48.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.542+0000 7f8919a00640 1 -- 172.21.15.69:0/531449898 >> 172.21.15.69:0/531449898 conn(0x7f891406eea0 msgr2=0x7f891410bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:48.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.542+0000 7f8919a00640 1 -- 172.21.15.69:0/531449898 shutdown_connections 2024-04-24T09:03:48.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:48.542+0000 7f8919a00640 1 -- 172.21.15.69:0/531449898 wait complete. 2024-04-24T09:03:48.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:48 smithi130 ceph-mon[31553]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:48.815 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:48 smithi069 ceph-mon[28170]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:48.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:48 smithi081 ceph-mon[30961]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:49.092 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 24 09:03:48 smithi069 podman[66993]: 2024-04-24 09:03:48.879377538 +0000 UTC m=+0.160046329 container create bf470faf116c17f1f68e738f8609ee41c69c870575137f43a71417e7e1bfe460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-g, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:03:49.434 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 24 09:03:49 smithi069 podman[66993]: 2024-04-24 09:03:49.150245184 +0000 UTC m=+0.430914012 container init bf470faf116c17f1f68e738f8609ee41c69c870575137f43a71417e7e1bfe460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-24T09:03:49.434 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 24 09:03:49 smithi069 podman[66993]: 2024-04-24 09:03:49.160286305 +0000 UTC m=+0.440955069 container start bf470faf116c17f1f68e738f8609ee41c69c870575137f43a71417e7e1bfe460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-24T09:03:49.434 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 24 09:03:49 smithi069 ceph-mds[67019]: -- 172.21.15.69:0/2560527514 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d607a1da40 con 0x55d6079f1000 2024-04-24T09:03:49.434 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 24 09:03:49 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-g[67013]: starting mds.g at 2024-04-24T09:03:49.434 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 24 09:03:49 smithi069 bash[66993]: bf470faf116c17f1f68e738f8609ee41c69c870575137f43a71417e7e1bfe460 2024-04-24T09:03:49.434 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 24 09:03:49 smithi069 systemd[1]: Started Ceph mds.g for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T09:03:49.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:49 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/531449898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:49.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:49.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:49 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:49.724 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:49 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/531449898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:49.724 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:49.724 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:49 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:49.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:49 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/531449898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:49.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:49.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:49 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:50.563 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:50.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:50 smithi069 ceph-mon[28170]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:50.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:50 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.69:6838/3637851934,v1:172.21.15.69:6839/3637851934] up:boot 2024-04-24T09:03:50.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:50 smithi069 ceph-mon[28170]: fsmap 10 up:standby 2024-04-24T09:03:50.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:50 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T09:03:50.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:50 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:50.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:50 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:50.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:50 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:50.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:50 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:50.680 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:50 smithi069 ceph-mon[28170]: Deploying daemon mds.a on smithi069 2024-04-24T09:03:50.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:50 smithi130 ceph-mon[31553]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:50.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:50 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.69:6838/3637851934,v1:172.21.15.69:6839/3637851934] up:boot 2024-04-24T09:03:50.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:50 smithi130 ceph-mon[31553]: fsmap 10 up:standby 2024-04-24T09:03:50.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:50 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T09:03:50.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:50 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:50.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:50 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:50.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:50 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:50.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:50 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:50.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:50 smithi130 ceph-mon[31553]: Deploying daemon mds.a on smithi069 2024-04-24T09:03:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.766+0000 7f5a5a400640 1 -- 172.21.15.69:0/918930375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a5410a210 msgr2=0x7f5a54110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.766+0000 7f5a5a400640 1 --2- 172.21.15.69:0/918930375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a5410a210 0x7f5a54110aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009900 tx=0x7f5a4c02a410 comp rx=0 tx=0).stop 2024-04-24T09:03:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.767+0000 7f5a5a400640 1 -- 172.21.15.69:0/918930375 shutdown_connections 2024-04-24T09:03:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.767+0000 7f5a5a400640 1 --2- 172.21.15.69:0/918930375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a5410a210 0x7f5a54110aa0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.767+0000 7f5a5a400640 1 --2- 172.21.15.69:0/918930375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a54109850 0x7f5a54109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.767+0000 7f5a5a400640 1 --2- 172.21.15.69:0/918930375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a54108660 0x7f5a54108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.767+0000 7f5a5a400640 1 -- 172.21.15.69:0/918930375 >> 172.21.15.69:0/918930375 conn(0x7f5a54077620 msgr2=0x7f5a54079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.767+0000 7f5a5a400640 1 -- 172.21.15.69:0/918930375 shutdown_connections 2024-04-24T09:03:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.767+0000 7f5a5a400640 1 -- 172.21.15.69:0/918930375 wait complete. 2024-04-24T09:03:50.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.768+0000 7f5a5a400640 1 Processor -- start 2024-04-24T09:03:50.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.768+0000 7f5a5a400640 1 -- start start 2024-04-24T09:03:50.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.768+0000 7f5a5a400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a54108660 0x7f5a5419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:50.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.768+0000 7f5a5a400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a54109850 0x7f5a5419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:50.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.768+0000 7f5a5a400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a5410a210 0x7f5a541a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:50.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.768+0000 7f5a5a400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541141c0 con 0x7f5a54109850 2024-04-24T09:03:50.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.768+0000 7f5a5a400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5a54114040 con 0x7f5a5410a210 2024-04-24T09:03:50.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.768+0000 7f5a5a400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5a54114340 con 0x7f5a54108660 2024-04-24T09:03:50.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.768+0000 7f5a4aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a54109850 0x7f5a5419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:50.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.768+0000 7f5a4aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a54109850 0x7f5a5419ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34882/0 (socket says 172.21.15.69:34882) 2024-04-24T09:03:50.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.768+0000 7f5a4aa00640 1 -- 172.21.15.69:0/2836981356 learned_addr learned my addr 172.21.15.69:0/2836981356 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:50.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.769+0000 7f5a53400640 1 --2- 172.21.15.69:0/2836981356 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a54108660 0x7f5a5419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:50.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.769+0000 7f5a4aa00640 1 -- 172.21.15.69:0/2836981356 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a54108660 msgr2=0x7f5a5419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:50.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.769+0000 7f5a4aa00640 1 --2- 172.21.15.69:0/2836981356 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a54108660 0x7f5a5419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:50.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.769+0000 7f5a4aa00640 1 -- 172.21.15.69:0/2836981356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a5410a210 msgr2=0x7f5a541a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:50.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.769+0000 7f5a4aa00640 1 --2- 172.21.15.69:0/2836981356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a5410a210 0x7f5a541a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:50.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.769+0000 7f5a4aa00640 1 -- 172.21.15.69:0/2836981356 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a541a6450 con 0x7f5a54109850 2024-04-24T09:03:50.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.769+0000 7f5a53400640 1 --2- 172.21.15.69:0/2836981356 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a54108660 0x7f5a5419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:50.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.769+0000 7f5a4aa00640 1 --2- 172.21.15.69:0/2836981356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a54109850 0x7f5a5419ed70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00d950 tx=0x7f5a3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:50.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.769+0000 7f5a50c00640 1 -- 172.21.15.69:0/2836981356 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c017070 con 0x7f5a54109850 2024-04-24T09:03:50.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.769+0000 7f5a50c00640 1 -- 172.21.15.69:0/2836981356 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a3c009c30 con 0x7f5a54109850 2024-04-24T09:03:50.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.769+0000 7f5a50c00640 1 -- 172.21.15.69:0/2836981356 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c00f620 con 0x7f5a54109850 2024-04-24T09:03:50.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.769+0000 7f5a5a400640 1 -- 172.21.15.69:0/2836981356 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a541a6740 con 0x7f5a54109850 2024-04-24T09:03:50.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.769+0000 7f5a5a400640 1 -- 172.21.15.69:0/2836981356 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a541a6c40 con 0x7f5a54109850 2024-04-24T09:03:50.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.771+0000 7f5a5a400640 1 -- 172.21.15.69:0/2836981356 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a5410f8b0 con 0x7f5a54109850 2024-04-24T09:03:50.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.773+0000 7f5a50c00640 1 -- 172.21.15.69:0/2836981356 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5a3c0109a0 con 0x7f5a54109850 2024-04-24T09:03:50.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.774+0000 7f5a50c00640 1 --2- 172.21.15.69:0/2836981356 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5a380a3c30 0x7f5a380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:50.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.774+0000 7f5a50c00640 1 -- 172.21.15.69:0/2836981356 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5a3c0cfbf0 con 0x7f5a54109850 2024-04-24T09:03:50.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.774+0000 7f5a53400640 1 --2- 172.21.15.69:0/2836981356 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5a380a3c30 0x7f5a380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:50.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.775+0000 7f5a53400640 1 --2- 172.21.15.69:0/2836981356 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5a380a3c30 0x7f5a380a60f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a541096b0 tx=0x7f5a44008370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:50.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:50.779+0000 7f5a50c00640 1 -- 172.21.15.69:0/2836981356 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a3c009de0 con 0x7f5a54109850 2024-04-24T09:03:50.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:50 smithi081 ceph-mon[30961]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:50.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:50 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.69:6838/3637851934,v1:172.21.15.69:6839/3637851934] up:boot 2024-04-24T09:03:50.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:50 smithi081 ceph-mon[30961]: fsmap 10 up:standby 2024-04-24T09:03:50.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:50 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T09:03:50.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:50 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:50.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:50 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:50.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:50 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:50.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:50 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:50.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:50 smithi081 ceph-mon[30961]: Deploying daemon mds.a on smithi069 2024-04-24T09:03:51.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:51.024+0000 7f5a5a400640 1 -- 172.21.15.69:0/2836981356 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a5402d620 con 0x7f5a54109850 2024-04-24T09:03:51.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:51.026+0000 7f5a50c00640 1 -- 172.21.15.69:0/2836981356 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7f5a3c09c9d0 con 0x7f5a54109850 2024-04-24T09:03:51.026 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:51.026 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34460,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34496,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T09:03:51.026 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 11 2024-04-24T09:03:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:51.029+0000 7f5a5a400640 1 -- 172.21.15.69:0/2836981356 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5a380a3c30 msgr2=0x7f5a380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:51.029+0000 7f5a5a400640 1 --2- 172.21.15.69:0/2836981356 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5a380a3c30 0x7f5a380a60f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a541096b0 tx=0x7f5a44008370 comp rx=0 tx=0).stop 2024-04-24T09:03:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:51.029+0000 7f5a5a400640 1 -- 172.21.15.69:0/2836981356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a54109850 msgr2=0x7f5a5419ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:51.029+0000 7f5a5a400640 1 --2- 172.21.15.69:0/2836981356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a54109850 0x7f5a5419ed70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00d950 tx=0x7f5a3c00de10 comp rx=0 tx=0).stop 2024-04-24T09:03:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:51.029+0000 7f5a5a400640 1 -- 172.21.15.69:0/2836981356 shutdown_connections 2024-04-24T09:03:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:51.029+0000 7f5a5a400640 1 --2- 172.21.15.69:0/2836981356 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5a380a3c30 0x7f5a380a60f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:51.029+0000 7f5a5a400640 1 --2- 172.21.15.69:0/2836981356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a5410a210 0x7f5a541a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:51.029+0000 7f5a5a400640 1 --2- 172.21.15.69:0/2836981356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a54109850 0x7f5a5419ed70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:51.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:51.029+0000 7f5a5a400640 1 --2- 172.21.15.69:0/2836981356 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a54108660 0x7f5a5419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:51.029+0000 7f5a5a400640 1 -- 172.21.15.69:0/2836981356 >> 172.21.15.69:0/2836981356 conn(0x7f5a54077620 msgr2=0x7f5a5410ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:51.029+0000 7f5a5a400640 1 -- 172.21.15.69:0/2836981356 shutdown_connections 2024-04-24T09:03:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:51.029+0000 7f5a5a400640 1 -- 172.21.15.69:0/2836981356 wait complete. 2024-04-24T09:03:51.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:51 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2836981356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:51.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:51 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2836981356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:51.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:51 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2836981356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:52.691 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:52 smithi069 ceph-mon[28170]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:52.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:52 smithi130 ceph-mon[31553]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:52.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:52 smithi081 ceph-mon[30961]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:53.042 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 24 09:03:52 smithi069 systemd[1]: Starting Ceph mds.a for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:03:53.048 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:53.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.258+0000 7fd762400640 1 -- 172.21.15.69:0/4109479706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd75c104250 msgr2=0x7fd75c104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.258+0000 7fd762400640 1 --2- 172.21.15.69:0/4109479706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd75c104250 0x7fd75c104670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009900 tx=0x7fd74c02a410 comp rx=0 tx=0).stop 2024-04-24T09:03:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.258+0000 7fd762400640 1 -- 172.21.15.69:0/4109479706 shutdown_connections 2024-04-24T09:03:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.258+0000 7fd762400640 1 --2- 172.21.15.69:0/4109479706 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd75c105e00 0x7fd75c10c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.258+0000 7fd762400640 1 --2- 172.21.15.69:0/4109479706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd75c105440 0x7fd75c1058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.258+0000 7fd762400640 1 --2- 172.21.15.69:0/4109479706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd75c104250 0x7fd75c104670 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.258+0000 7fd762400640 1 -- 172.21.15.69:0/4109479706 >> 172.21.15.69:0/4109479706 conn(0x7fd75c0ff7c0 msgr2=0x7fd75c101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.258+0000 7fd762400640 1 -- 172.21.15.69:0/4109479706 shutdown_connections 2024-04-24T09:03:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.259+0000 7fd762400640 1 -- 172.21.15.69:0/4109479706 wait complete. 2024-04-24T09:03:53.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.259+0000 7fd762400640 1 Processor -- start 2024-04-24T09:03:53.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.260+0000 7fd762400640 1 -- start start 2024-04-24T09:03:53.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.260+0000 7fd762400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd75c104250 0x7fd75c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:53.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.260+0000 7fd762400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd75c105440 0x7fd75c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.260+0000 7fd762400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd75c105e00 0x7fd75c1a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.260+0000 7fd762400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c10fd90 con 0x7fd75c104250 2024-04-24T09:03:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.260+0000 7fd762400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c10fc10 con 0x7fd75c105e00 2024-04-24T09:03:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.260+0000 7fd762400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c10ff10 con 0x7fd75c105440 2024-04-24T09:03:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.260+0000 7fd75b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd75c104250 0x7fd75c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.260+0000 7fd75b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd75c104250 0x7fd75c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48404/0 (socket says 172.21.15.69:48404) 2024-04-24T09:03:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.260+0000 7fd75b400640 1 -- 172.21.15.69:0/2231803424 learned_addr learned my addr 172.21.15.69:0/2231803424 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.261+0000 7fd753e00640 1 --2- 172.21.15.69:0/2231803424 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd75c105440 0x7fd75c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.261+0000 7fd75be00640 1 --2- 172.21.15.69:0/2231803424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd75c105e00 0x7fd75c1a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.261+0000 7fd75b400640 1 -- 172.21.15.69:0/2231803424 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd75c105440 msgr2=0x7fd75c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.261+0000 7fd75b400640 1 --2- 172.21.15.69:0/2231803424 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd75c105440 0x7fd75c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.261+0000 7fd75b400640 1 -- 172.21.15.69:0/2231803424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd75c105e00 msgr2=0x7fd75c1a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.261+0000 7fd75b400640 1 --2- 172.21.15.69:0/2231803424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd75c105e00 0x7fd75c1a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.261+0000 7fd75b400640 1 -- 172.21.15.69:0/2231803424 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75c1a2090 con 0x7fd75c104250 2024-04-24T09:03:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.261+0000 7fd753e00640 1 --2- 172.21.15.69:0/2231803424 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd75c105440 0x7fd75c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.261+0000 7fd75b400640 1 --2- 172.21.15.69:0/2231803424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd75c104250 0x7fd75c19a470 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009230 tx=0x7fd74c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.261+0000 7fd758c00640 1 -- 172.21.15.69:0/2231803424 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c03f030 con 0x7fd75c104250 2024-04-24T09:03:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.261+0000 7fd758c00640 1 -- 172.21.15.69:0/2231803424 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd74c003cb0 con 0x7fd75c104250 2024-04-24T09:03:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.261+0000 7fd758c00640 1 -- 172.21.15.69:0/2231803424 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c035070 con 0x7fd75c104250 2024-04-24T09:03:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.261+0000 7fd762400640 1 -- 172.21.15.69:0/2231803424 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75c1a2320 con 0x7fd75c104250 2024-04-24T09:03:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.262+0000 7fd762400640 1 -- 172.21.15.69:0/2231803424 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75c1a27c0 con 0x7fd75c104250 2024-04-24T09:03:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.263+0000 7fd762400640 1 -- 172.21.15.69:0/2231803424 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd75c10b5e0 con 0x7fd75c104250 2024-04-24T09:03:53.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.265+0000 7fd758c00640 1 -- 172.21.15.69:0/2231803424 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd74c042030 con 0x7fd75c104250 2024-04-24T09:03:53.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.266+0000 7fd758c00640 1 --2- 172.21.15.69:0/2231803424 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd7400a3c30 0x7fd7400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:53.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.266+0000 7fd758c00640 1 -- 172.21.15.69:0/2231803424 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd74c0f0ae0 con 0x7fd75c104250 2024-04-24T09:03:53.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.266+0000 7fd753e00640 1 --2- 172.21.15.69:0/2231803424 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd7400a3c30 0x7fd7400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:53.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.267+0000 7fd753e00640 1 --2- 172.21.15.69:0/2231803424 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd7400a3c30 0x7fd7400a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd75c19b980 tx=0x7fd744005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:53.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.269+0000 7fd758c00640 1 -- 172.21.15.69:0/2231803424 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd74c0bd8c0 con 0x7fd75c104250 2024-04-24T09:03:53.342 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 24 09:03:53 smithi069 podman[67341]: 2024-04-24 09:03:52.949324231 +0000 UTC m=+0.036910731 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:03:53.342 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 24 09:03:53 smithi069 podman[67341]: 2024-04-24 09:03:53.077480748 +0000 UTC m=+0.165067260 container create 3eefa30160f81c90203c386b030c3018aa59acf91b444aece3d10c1814c45d6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:03:53.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.532+0000 7fd762400640 1 -- 172.21.15.69:0/2231803424 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd75c02d620 con 0x7fd75c104250 2024-04-24T09:03:53.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.533+0000 7fd758c00640 1 -- 172.21.15.69:0/2231803424 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7fd74c0bd1c0 con 0x7fd75c104250 2024-04-24T09:03:53.533 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:53.533 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34460,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34496,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T09:03:53.533 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 12 2024-04-24T09:03:53.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.536+0000 7fd762400640 1 -- 172.21.15.69:0/2231803424 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd7400a3c30 msgr2=0x7fd7400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:53.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.536+0000 7fd762400640 1 --2- 172.21.15.69:0/2231803424 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd7400a3c30 0x7fd7400a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd75c19b980 tx=0x7fd744005d90 comp rx=0 tx=0).stop 2024-04-24T09:03:53.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.536+0000 7fd762400640 1 -- 172.21.15.69:0/2231803424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd75c104250 msgr2=0x7fd75c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:53.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.536+0000 7fd762400640 1 --2- 172.21.15.69:0/2231803424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd75c104250 0x7fd75c19a470 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009230 tx=0x7fd74c004340 comp rx=0 tx=0).stop 2024-04-24T09:03:53.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.536+0000 7fd762400640 1 -- 172.21.15.69:0/2231803424 shutdown_connections 2024-04-24T09:03:53.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.536+0000 7fd762400640 1 --2- 172.21.15.69:0/2231803424 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd7400a3c30 0x7fd7400a60f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:53.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.536+0000 7fd762400640 1 --2- 172.21.15.69:0/2231803424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd75c105e00 0x7fd75c1a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:53.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.536+0000 7fd762400640 1 --2- 172.21.15.69:0/2231803424 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd75c105440 0x7fd75c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:53.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.536+0000 7fd762400640 1 --2- 172.21.15.69:0/2231803424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd75c104250 0x7fd75c19a470 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:53.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.536+0000 7fd762400640 1 -- 172.21.15.69:0/2231803424 >> 172.21.15.69:0/2231803424 conn(0x7fd75c0ff7c0 msgr2=0x7fd75c1087b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:53.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.536+0000 7fd762400640 1 -- 172.21.15.69:0/2231803424 shutdown_connections 2024-04-24T09:03:53.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:53.536+0000 7fd762400640 1 -- 172.21.15.69:0/2231803424 wait complete. 2024-04-24T09:03:53.842 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 24 09:03:53 smithi069 podman[67341]: 2024-04-24 09:03:53.349881448 +0000 UTC m=+0.437467956 container init 3eefa30160f81c90203c386b030c3018aa59acf91b444aece3d10c1814c45d6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-24T09:03:53.842 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 24 09:03:53 smithi069 podman[67341]: 2024-04-24 09:03:53.358290588 +0000 UTC m=+0.445877098 container start 3eefa30160f81c90203c386b030c3018aa59acf91b444aece3d10c1814c45d6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True) 2024-04-24T09:03:53.842 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 24 09:03:53 smithi069 ceph-mds[67399]: -- 172.21.15.69:0/802864745 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa5c5d5a40 con 0x55aa5c5a9000 2024-04-24T09:03:53.842 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 24 09:03:53 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-a[67376]: starting mds.a at 2024-04-24T09:03:53.842 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 24 09:03:53 smithi069 bash[67341]: 3eefa30160f81c90203c386b030c3018aa59acf91b444aece3d10c1814c45d6b 2024-04-24T09:03:53.842 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 24 09:03:53 smithi069 systemd[1]: Started Ceph mds.a for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T09:03:54.803 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:54 smithi081 ceph-mon[30961]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:54.804 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:54 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.69:6840/1747080062,v1:172.21.15.69:6841/1747080062] up:boot 2024-04-24T09:03:54.804 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:54 smithi081 ceph-mon[30961]: fsmap 11 up:standby 2024-04-24T09:03:54.804 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:54 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T09:03:54.804 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:54 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2231803424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:54.804 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:54 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:54.804 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:54 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:54.804 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:54 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:54.804 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:54 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:54.804 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:54 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:54.804 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:54 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:54.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:54 smithi069 ceph-mon[28170]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:54.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:54 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.69:6840/1747080062,v1:172.21.15.69:6841/1747080062] up:boot 2024-04-24T09:03:54.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:54 smithi069 ceph-mon[28170]: fsmap 11 up:standby 2024-04-24T09:03:54.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:54 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T09:03:54.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:54 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2231803424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:54.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:54 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:54.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:54 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:54.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:54 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:54.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:54 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:54.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:54 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:54.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:54 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:54 smithi130 ceph-mon[31553]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:54 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.69:6840/1747080062,v1:172.21.15.69:6841/1747080062] up:boot 2024-04-24T09:03:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:54 smithi130 ceph-mon[31553]: fsmap 11 up:standby 2024-04-24T09:03:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:54 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T09:03:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:54 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2231803424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:54 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:54 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:54 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:54 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:03:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:54 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:03:54.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:54 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:55.559 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:55.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.773+0000 7fd00b200640 1 -- 172.21.15.69:0/3110847760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd004109670 msgr2=0x7fd004109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:55.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.773+0000 7fd00b200640 1 --2- 172.21.15.69:0/3110847760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd004109670 0x7fd004109af0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009900 tx=0x7fcff402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:03:55.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.773+0000 7fd00b200640 1 -- 172.21.15.69:0/3110847760 shutdown_connections 2024-04-24T09:03:55.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.773+0000 7fd00b200640 1 --2- 172.21.15.69:0/3110847760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd00410a030 0x7fd0041108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:55.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.773+0000 7fd00b200640 1 --2- 172.21.15.69:0/3110847760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd004109670 0x7fd004109af0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:55.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.773+0000 7fd00b200640 1 --2- 172.21.15.69:0/3110847760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd004108480 0x7fd0041088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:55.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.773+0000 7fd00b200640 1 -- 172.21.15.69:0/3110847760 >> 172.21.15.69:0/3110847760 conn(0x7fd004077440 msgr2=0x7fd004079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:55.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.773+0000 7fd00b200640 1 -- 172.21.15.69:0/3110847760 shutdown_connections 2024-04-24T09:03:55.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.773+0000 7fd00b200640 1 -- 172.21.15.69:0/3110847760 wait complete. 2024-04-24T09:03:55.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.774+0000 7fd00b200640 1 Processor -- start 2024-04-24T09:03:55.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.774+0000 7fd00b200640 1 -- start start 2024-04-24T09:03:55.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.774+0000 7fd00b200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd004108480 0x7fd00419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:55.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.774+0000 7fd00b200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd004109670 0x7fd00419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:55.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.775+0000 7fd00b200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd00410a030 0x7fd0041a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:55.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.775+0000 7fd00b200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd0041140a0 con 0x7fd00410a030 2024-04-24T09:03:55.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.775+0000 7fd00b200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd004113f20 con 0x7fd004109670 2024-04-24T09:03:55.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.775+0000 7fd00b200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd004114220 con 0x7fd004108480 2024-04-24T09:03:55.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.775+0000 7fd00a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd00410a030 0x7fd0041a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:55.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.775+0000 7fd00a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd00410a030 0x7fd0041a5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48454/0 (socket says 172.21.15.69:48454) 2024-04-24T09:03:55.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.775+0000 7fd00a800640 1 -- 172.21.15.69:0/3710167629 learned_addr learned my addr 172.21.15.69:0/3710167629 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:55.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.775+0000 7fd009e00640 1 --2- 172.21.15.69:0/3710167629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd004108480 0x7fd00419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:55.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.775+0000 7fd009400640 1 --2- 172.21.15.69:0/3710167629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd004109670 0x7fd00419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:55.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.775+0000 7fd00a800640 1 -- 172.21.15.69:0/3710167629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd004108480 msgr2=0x7fd00419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:55.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.775+0000 7fd00a800640 1 --2- 172.21.15.69:0/3710167629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd004108480 0x7fd00419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:55.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.775+0000 7fd00a800640 1 -- 172.21.15.69:0/3710167629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd004109670 msgr2=0x7fd00419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:55.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.775+0000 7fd00a800640 1 --2- 172.21.15.69:0/3710167629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd004109670 0x7fd00419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:55.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.775+0000 7fd00a800640 1 -- 172.21.15.69:0/3710167629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0041a6470 con 0x7fd00410a030 2024-04-24T09:03:55.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.776+0000 7fd00a800640 1 --2- 172.21.15.69:0/3710167629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd00410a030 0x7fd0041a5e00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd00000b970 tx=0x7fd00000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:55.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.776+0000 7fcffea00640 1 -- 172.21.15.69:0/3710167629 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000019070 con 0x7fd00410a030 2024-04-24T09:03:55.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.776+0000 7fcffea00640 1 -- 172.21.15.69:0/3710167629 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd000014440 con 0x7fd00410a030 2024-04-24T09:03:55.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.776+0000 7fcffea00640 1 -- 172.21.15.69:0/3710167629 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0000075d0 con 0x7fd00410a030 2024-04-24T09:03:55.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.776+0000 7fd00b200640 1 -- 172.21.15.69:0/3710167629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0041a6760 con 0x7fd00410a030 2024-04-24T09:03:55.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.776+0000 7fd00b200640 1 -- 172.21.15.69:0/3710167629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0041a6d30 con 0x7fd00410a030 2024-04-24T09:03:55.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.777+0000 7fd00b200640 1 -- 172.21.15.69:0/3710167629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd00410f8c0 con 0x7fd00410a030 2024-04-24T09:03:55.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.779+0000 7fcffea00640 1 -- 172.21.15.69:0/3710167629 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd00001c070 con 0x7fd00410a030 2024-04-24T09:03:55.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.781+0000 7fcffea00640 1 --2- 172.21.15.69:0/3710167629 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcfec0a3c30 0x7fcfec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:55.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.781+0000 7fcffea00640 1 -- 172.21.15.69:0/3710167629 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd0000cf5b0 con 0x7fd00410a030 2024-04-24T09:03:55.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.781+0000 7fd009e00640 1 --2- 172.21.15.69:0/3710167629 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcfec0a3c30 0x7fcfec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:55.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.782+0000 7fd009e00640 1 --2- 172.21.15.69:0/3710167629 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcfec0a3c30 0x7fcfec0a60f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd0041094d0 tx=0x7fcff8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:55.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:55.785+0000 7fcffea00640 1 -- 172.21.15.69:0/3710167629 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd00009c390 con 0x7fd00410a030 2024-04-24T09:03:55.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:55 smithi081 ceph-mon[30961]: Deploying daemon mds.b on smithi081 2024-04-24T09:03:55.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:55 smithi069 ceph-mon[28170]: Deploying daemon mds.b on smithi081 2024-04-24T09:03:55.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:55 smithi130 ceph-mon[31553]: Deploying daemon mds.b on smithi081 2024-04-24T09:03:56.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:56.031+0000 7fd00b200640 1 -- 172.21.15.69:0/3710167629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd00402d620 con 0x7fd00410a030 2024-04-24T09:03:56.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:56.032+0000 7fcffea00640 1 -- 172.21.15.69:0/3710167629 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7fd00009bc90 con 0x7fd00410a030 2024-04-24T09:03:56.032 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:56.032 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34460,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34496,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T09:03:56.032 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 12 2024-04-24T09:03:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:56.035+0000 7fd00b200640 1 -- 172.21.15.69:0/3710167629 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcfec0a3c30 msgr2=0x7fcfec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:56.035+0000 7fd00b200640 1 --2- 172.21.15.69:0/3710167629 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcfec0a3c30 0x7fcfec0a60f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd0041094d0 tx=0x7fcff8008040 comp rx=0 tx=0).stop 2024-04-24T09:03:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:56.035+0000 7fd00b200640 1 -- 172.21.15.69:0/3710167629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd00410a030 msgr2=0x7fd0041a5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:56.035+0000 7fd00b200640 1 --2- 172.21.15.69:0/3710167629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd00410a030 0x7fd0041a5e00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd00000b970 tx=0x7fd00000be30 comp rx=0 tx=0).stop 2024-04-24T09:03:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:56.035+0000 7fd00b200640 1 -- 172.21.15.69:0/3710167629 shutdown_connections 2024-04-24T09:03:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:56.035+0000 7fd00b200640 1 --2- 172.21.15.69:0/3710167629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd00410a030 0x7fd0041a5e00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:56.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:56.035+0000 7fd00b200640 1 --2- 172.21.15.69:0/3710167629 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcfec0a3c30 0x7fcfec0a60f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:56.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:56.035+0000 7fd00b200640 1 --2- 172.21.15.69:0/3710167629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd004109670 0x7fd00419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:56.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:56.035+0000 7fd00b200640 1 --2- 172.21.15.69:0/3710167629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd004108480 0x7fd00419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:56.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:56.035+0000 7fd00b200640 1 -- 172.21.15.69:0/3710167629 >> 172.21.15.69:0/3710167629 conn(0x7fd004077440 msgr2=0x7fd00410ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:56.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:56.035+0000 7fd00b200640 1 -- 172.21.15.69:0/3710167629 shutdown_connections 2024-04-24T09:03:56.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:56.035+0000 7fd00b200640 1 -- 172.21.15.69:0/3710167629 wait complete. 2024-04-24T09:03:56.767 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:56 smithi081 ceph-mon[30961]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:56.768 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:56 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3710167629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:56.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:56 smithi069 ceph-mon[28170]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:56.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:56 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3710167629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:56.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:56 smithi130 ceph-mon[31553]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:56.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:56 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3710167629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:57.416 INFO:journalctl@ceph.mds.b.smithi081.stdout:Apr 24 09:03:57 smithi081 podman[61425]: 2024-04-24 09:03:57.170448082 +0000 UTC m=+0.153838486 container create 3c6bdbb22db83e2799fee2d0cbdae4cd3df623626ef2363d45977f355b764d34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-24T09:03:57.416 INFO:journalctl@ceph.mds.b.smithi081.stdout:Apr 24 09:03:57 smithi081 podman[61425]: 2024-04-24 09:03:57.415815891 +0000 UTC m=+0.399206267 container init 3c6bdbb22db83e2799fee2d0cbdae4cd3df623626ef2363d45977f355b764d34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:03:57.817 INFO:journalctl@ceph.mds.b.smithi081.stdout:Apr 24 09:03:57 smithi081 podman[61425]: 2024-04-24 09:03:57.423568458 +0000 UTC m=+0.406958825 container start 3c6bdbb22db83e2799fee2d0cbdae4cd3df623626ef2363d45977f355b764d34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-24T09:03:57.817 INFO:journalctl@ceph.mds.b.smithi081.stdout:Apr 24 09:03:57 smithi081 ceph-mds[61451]: -- 172.21.15.81:0/863508716 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d3c495da40 con 0x55d3c4931000 2024-04-24T09:03:57.817 INFO:journalctl@ceph.mds.b.smithi081.stdout:Apr 24 09:03:57 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-b[61445]: starting mds.b at 2024-04-24T09:03:57.817 INFO:journalctl@ceph.mds.b.smithi081.stdout:Apr 24 09:03:57 smithi081 bash[61425]: 3c6bdbb22db83e2799fee2d0cbdae4cd3df623626ef2363d45977f355b764d34 2024-04-24T09:03:57.817 INFO:journalctl@ceph.mds.b.smithi081.stdout:Apr 24 09:03:57 smithi081 systemd[1]: Started Ceph mds.b for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T09:03:58.057 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:58.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.264+0000 7f315e400640 1 -- 172.21.15.69:0/1869693876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3158102050 msgr2=0x7f3158102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:58.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.264+0000 7f315e400640 1 --2- 172.21.15.69:0/1869693876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3158102050 0x7f3158102470 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3148009900 tx=0x7f314802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.264+0000 7f315e400640 1 -- 172.21.15.69:0/1869693876 shutdown_connections 2024-04-24T09:03:58.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.264+0000 7f315e400640 1 --2- 172.21.15.69:0/1869693876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3158103c00 0x7f315810a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.264+0000 7f315e400640 1 --2- 172.21.15.69:0/1869693876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3158103240 0x7f31581036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.264+0000 7f315e400640 1 --2- 172.21.15.69:0/1869693876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3158102050 0x7f3158102470 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.264+0000 7f315e400640 1 -- 172.21.15.69:0/1869693876 >> 172.21.15.69:0/1869693876 conn(0x7f31580fd5e0 msgr2=0x7f31580ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:58.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.264+0000 7f315e400640 1 -- 172.21.15.69:0/1869693876 shutdown_connections 2024-04-24T09:03:58.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.264+0000 7f315e400640 1 -- 172.21.15.69:0/1869693876 wait complete. 2024-04-24T09:03:58.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.265+0000 7f315e400640 1 Processor -- start 2024-04-24T09:03:58.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.265+0000 7f315e400640 1 -- start start 2024-04-24T09:03:58.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.266+0000 7f315e400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3158102050 0x7f3158075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:58.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.266+0000 7f315e400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3158103240 0x7f31580743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:58.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.266+0000 7f315e400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3158103c00 0x7f3158074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:58.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.266+0000 7f315e400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f315810dbe0 con 0x7f3158102050 2024-04-24T09:03:58.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.266+0000 7f315e400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f315810da60 con 0x7f3158103c00 2024-04-24T09:03:58.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.266+0000 7f315e400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f315810dd60 con 0x7f3158103240 2024-04-24T09:03:58.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.266+0000 7f3157400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3158102050 0x7f3158075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:58.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.266+0000 7f3157400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3158102050 0x7f3158075db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48466/0 (socket says 172.21.15.69:48466) 2024-04-24T09:03:58.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.266+0000 7f3157400640 1 -- 172.21.15.69:0/150001430 learned_addr learned my addr 172.21.15.69:0/150001430 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:58.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.266+0000 7f3157e00640 1 --2- 172.21.15.69:0/150001430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3158103c00 0x7f3158074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:58.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.266+0000 7f3156a00640 1 --2- 172.21.15.69:0/150001430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3158103240 0x7f31580743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:58.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.267+0000 7f3157400640 1 -- 172.21.15.69:0/150001430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3158103240 msgr2=0x7f31580743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:58.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.267+0000 7f3157400640 1 --2- 172.21.15.69:0/150001430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3158103240 0x7f31580743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.267+0000 7f3157400640 1 -- 172.21.15.69:0/150001430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3158103c00 msgr2=0x7f3158074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:58.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.267+0000 7f3157400640 1 --2- 172.21.15.69:0/150001430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3158103c00 0x7f3158074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.267+0000 7f3157400640 1 -- 172.21.15.69:0/150001430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31580751c0 con 0x7f3158102050 2024-04-24T09:03:58.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.267+0000 7f3157400640 1 --2- 172.21.15.69:0/150001430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3158102050 0x7f3158075db0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f31480314e0 tx=0x7f3148003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:58.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.267+0000 7f314fe00640 1 -- 172.21.15.69:0/150001430 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314803f070 con 0x7f3158102050 2024-04-24T09:03:58.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.267+0000 7f314fe00640 1 -- 172.21.15.69:0/150001430 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3148031c40 con 0x7f3158102050 2024-04-24T09:03:58.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.267+0000 7f314fe00640 1 -- 172.21.15.69:0/150001430 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148035030 con 0x7f3158102050 2024-04-24T09:03:58.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.267+0000 7f315e400640 1 -- 172.21.15.69:0/150001430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31581a2380 con 0x7f3158102050 2024-04-24T09:03:58.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.267+0000 7f315e400640 1 -- 172.21.15.69:0/150001430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31581a2880 con 0x7f3158102050 2024-04-24T09:03:58.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.269+0000 7f315e400640 1 -- 172.21.15.69:0/150001430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3158109440 con 0x7f3158102050 2024-04-24T09:03:58.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.272+0000 7f314fe00640 1 -- 172.21.15.69:0/150001430 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3148042070 con 0x7f3158102050 2024-04-24T09:03:58.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.273+0000 7f314fe00640 1 --2- 172.21.15.69:0/150001430 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f313c0a3c30 0x7f313c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:58.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.273+0000 7f314fe00640 1 -- 172.21.15.69:0/150001430 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f31480f0d60 con 0x7f3158102050 2024-04-24T09:03:58.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.274+0000 7f3156a00640 1 --2- 172.21.15.69:0/150001430 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f313c0a3c30 0x7f313c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:58.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.274+0000 7f3156a00640 1 --2- 172.21.15.69:0/150001430 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f313c0a3c30 0x7f313c0a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3144005fd0 tx=0x7f3144005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:58.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.277+0000 7f314fe00640 1 -- 172.21.15.69:0/150001430 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31480bdbe0 con 0x7f3158102050 2024-04-24T09:03:58.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.528+0000 7f315e400640 1 -- 172.21.15.69:0/150001430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f315802d620 con 0x7f3158102050 2024-04-24T09:03:58.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.529+0000 7f314fe00640 1 -- 172.21.15.69:0/150001430 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10482 (secure 0 0 0) 0x7f31480bd4e0 con 0x7f3158102050 2024-04-24T09:03:58.529 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:58.529 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34460,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34496,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":34532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-24T09:03:58.530 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 13 2024-04-24T09:03:58.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.532+0000 7f315e400640 1 -- 172.21.15.69:0/150001430 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f313c0a3c30 msgr2=0x7f313c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:58.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.532+0000 7f315e400640 1 --2- 172.21.15.69:0/150001430 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f313c0a3c30 0x7f313c0a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3144005fd0 tx=0x7f3144005850 comp rx=0 tx=0).stop 2024-04-24T09:03:58.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.532+0000 7f315e400640 1 -- 172.21.15.69:0/150001430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3158102050 msgr2=0x7f3158075db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:58.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.532+0000 7f315e400640 1 --2- 172.21.15.69:0/150001430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3158102050 0x7f3158075db0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f31480314e0 tx=0x7f3148003c10 comp rx=0 tx=0).stop 2024-04-24T09:03:58.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.533+0000 7f315e400640 1 -- 172.21.15.69:0/150001430 shutdown_connections 2024-04-24T09:03:58.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.533+0000 7f315e400640 1 --2- 172.21.15.69:0/150001430 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f313c0a3c30 0x7f313c0a60f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.533+0000 7f315e400640 1 --2- 172.21.15.69:0/150001430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3158103c00 0x7f3158074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.533+0000 7f315e400640 1 --2- 172.21.15.69:0/150001430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3158103240 0x7f31580743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.533+0000 7f315e400640 1 --2- 172.21.15.69:0/150001430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3158102050 0x7f3158075db0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.533+0000 7f315e400640 1 -- 172.21.15.69:0/150001430 >> 172.21.15.69:0/150001430 conn(0x7f31580fd5e0 msgr2=0x7f3158106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:58.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.533+0000 7f315e400640 1 -- 172.21.15.69:0/150001430 shutdown_connections 2024-04-24T09:03:58.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.533+0000 7f315e400640 1 -- 172.21.15.69:0/150001430 wait complete. 2024-04-24T09:03:58.551 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:03:58.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.739+0000 7fe02d000640 1 -- 172.21.15.69:0/143721476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe028102900 msgr2=0x7fe028104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:58.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.739+0000 7fe02d000640 1 --2- 172.21.15.69:0/143721476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe028102900 0x7fe028104cf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe018009900 tx=0x7fe01802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.739+0000 7fe02d000640 1 -- 172.21.15.69:0/143721476 shutdown_connections 2024-04-24T09:03:58.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.739+0000 7fe02d000640 1 --2- 172.21.15.69:0/143721476 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe028107b60 0x7fe028109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.739+0000 7fe02d000640 1 --2- 172.21.15.69:0/143721476 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe028105230 0x7fe028107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.739+0000 7fe02d000640 1 --2- 172.21.15.69:0/143721476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe028102900 0x7fe028104cf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.739+0000 7fe02d000640 1 -- 172.21.15.69:0/143721476 >> 172.21.15.69:0/143721476 conn(0x7fe0280fc2d0 msgr2=0x7fe0280fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:58.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.739+0000 7fe02d000640 1 -- 172.21.15.69:0/143721476 shutdown_connections 2024-04-24T09:03:58.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.739+0000 7fe02d000640 1 -- 172.21.15.69:0/143721476 wait complete. 2024-04-24T09:03:58.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.740+0000 7fe02d000640 1 Processor -- start 2024-04-24T09:03:58.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.740+0000 7fe02d000640 1 -- start start 2024-04-24T09:03:58.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.740+0000 7fe02d000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe028102900 0x7fe028198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:58.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.740+0000 7fe02d000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe028105230 0x7fe028198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:58.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.740+0000 7fe02d000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe028107b60 0x7fe02819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:58.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.740+0000 7fe02d000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe02810da30 con 0x7fe028107b60 2024-04-24T09:03:58.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.740+0000 7fe02d000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe02810d8b0 con 0x7fe028102900 2024-04-24T09:03:58.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.740+0000 7fe02d000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe02810dbb0 con 0x7fe028105230 2024-04-24T09:03:58.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.741+0000 7fe027e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe028107b60 0x7fe02819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:58.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.741+0000 7fe027e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe028107b60 0x7fe02819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48474/0 (socket says 172.21.15.69:48474) 2024-04-24T09:03:58.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.741+0000 7fe027e00640 1 -- 172.21.15.69:0/3582249883 learned_addr learned my addr 172.21.15.69:0/3582249883 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:58.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.741+0000 7fe026a00640 1 --2- 172.21.15.69:0/3582249883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe028105230 0x7fe028198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:58.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.741+0000 7fe027400640 1 --2- 172.21.15.69:0/3582249883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe028102900 0x7fe028198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:58.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.741+0000 7fe027e00640 1 -- 172.21.15.69:0/3582249883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe028105230 msgr2=0x7fe028198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:58.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.741+0000 7fe027e00640 1 --2- 172.21.15.69:0/3582249883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe028105230 0x7fe028198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.741+0000 7fe027e00640 1 -- 172.21.15.69:0/3582249883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe028102900 msgr2=0x7fe028198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:58.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.741+0000 7fe027e00640 1 --2- 172.21.15.69:0/3582249883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe028102900 0x7fe028198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.741+0000 7fe027e00640 1 -- 172.21.15.69:0/3582249883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe02819fd20 con 0x7fe028107b60 2024-04-24T09:03:58.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.741+0000 7fe027e00640 1 --2- 172.21.15.69:0/3582249883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe028107b60 0x7fe02819f6b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe02000eaa0 tx=0x7fe02000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:58.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.742+0000 7fe01fe00640 1 -- 172.21.15.69:0/3582249883 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02000cdb0 con 0x7fe028107b60 2024-04-24T09:03:58.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.742+0000 7fe01fe00640 1 -- 172.21.15.69:0/3582249883 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe020005020 con 0x7fe028107b60 2024-04-24T09:03:58.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.742+0000 7fe01fe00640 1 -- 172.21.15.69:0/3582249883 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0200116d0 con 0x7fe028107b60 2024-04-24T09:03:58.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.742+0000 7fe02d000640 1 -- 172.21.15.69:0/3582249883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe02819ffb0 con 0x7fe028107b60 2024-04-24T09:03:58.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.742+0000 7fe02d000640 1 -- 172.21.15.69:0/3582249883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0281a0480 con 0x7fe028107b60 2024-04-24T09:03:58.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.743+0000 7fe02d000640 1 -- 172.21.15.69:0/3582249883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0280fddc0 con 0x7fe028107b60 2024-04-24T09:03:58.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.745+0000 7fe01fe00640 1 -- 172.21.15.69:0/3582249883 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe0200051d0 con 0x7fe028107b60 2024-04-24T09:03:58.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.746+0000 7fe01fe00640 1 --2- 172.21.15.69:0/3582249883 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fe00c0a3bb0 0x7fe00c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:58.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.746+0000 7fe01fe00640 1 -- 172.21.15.69:0/3582249883 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe0200d01e0 con 0x7fe028107b60 2024-04-24T09:03:58.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.746+0000 7fe027400640 1 --2- 172.21.15.69:0/3582249883 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fe00c0a3bb0 0x7fe00c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:58.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.747+0000 7fe027400640 1 --2- 172.21.15.69:0/3582249883 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fe00c0a3bb0 0x7fe00c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe018000c00 tx=0x7fe0180314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:58.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.750+0000 7fe01fe00640 1 -- 172.21.15.69:0/3582249883 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe020016030 con 0x7fe028107b60 2024-04-24T09:03:58.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:58 smithi081 ceph-mon[30961]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:58.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:58 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:58 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:58 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:58 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:58 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:58 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:03:58.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:58 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:58 smithi069 ceph-mon[28170]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:58.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:58 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:58 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:58 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:58 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:58 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:58 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:03:58.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:58 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:58 smithi130 ceph-mon[31553]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:03:58.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:58 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:58 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:58 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:58 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:58 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:58 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:03:58.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:58 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:03:58.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.961+0000 7fe02d000640 1 -- 172.21.15.69:0/3582249883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe028062c00 con 0x7fe028107b60 2024-04-24T09:03:58.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.962+0000 7fe01fe00640 1 -- 172.21.15.69:0/3582249883 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18422 (secure 0 0 0) 0x7fe02009cfc0 con 0x7fe028107b60 2024-04-24T09:03:58.962 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:58.962 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":82,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:02:54.504455+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:03:58.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.965+0000 7fe02d000640 1 -- 172.21.15.69:0/3582249883 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fe00c0a3bb0 msgr2=0x7fe00c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:58.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.965+0000 7fe02d000640 1 --2- 172.21.15.69:0/3582249883 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fe00c0a3bb0 0x7fe00c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe018000c00 tx=0x7fe0180314a0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.965+0000 7fe02d000640 1 -- 172.21.15.69:0/3582249883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe028107b60 msgr2=0x7fe02819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:58.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.965+0000 7fe02d000640 1 --2- 172.21.15.69:0/3582249883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe028107b60 0x7fe02819f6b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe02000eaa0 tx=0x7fe02000ef60 comp rx=0 tx=0).stop 2024-04-24T09:03:58.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.965+0000 7fe02d000640 1 -- 172.21.15.69:0/3582249883 shutdown_connections 2024-04-24T09:03:58.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.965+0000 7fe02d000640 1 --2- 172.21.15.69:0/3582249883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe028107b60 0x7fe02819f6b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.965+0000 7fe02d000640 1 --2- 172.21.15.69:0/3582249883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe028105230 0x7fe028198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.965+0000 7fe02d000640 1 --2- 172.21.15.69:0/3582249883 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fe00c0a3bb0 0x7fe00c0a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.965+0000 7fe02d000640 1 --2- 172.21.15.69:0/3582249883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe028102900 0x7fe028198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:58.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.965+0000 7fe02d000640 1 -- 172.21.15.69:0/3582249883 >> 172.21.15.69:0/3582249883 conn(0x7fe0280fc2d0 msgr2=0x7fe0281033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:58.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.965+0000 7fe02d000640 1 -- 172.21.15.69:0/3582249883 shutdown_connections 2024-04-24T09:03:58.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:58.965+0000 7fe02d000640 1 -- 172.21.15.69:0/3582249883 wait complete. 2024-04-24T09:03:58.983 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:57:37.377599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T09:03:58.983 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:03:59.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.163+0000 7f2aebc00640 1 -- 172.21.15.69:0/1163638193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ae4109f10 msgr2=0x7f2ae410c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:59.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.163+0000 7f2aebc00640 1 --2- 172.21.15.69:0/1163638193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ae4109f10 0x7f2ae410c3a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009ca0 tx=0x7f2adc02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.164+0000 7f2aebc00640 1 -- 172.21.15.69:0/1163638193 shutdown_connections 2024-04-24T09:03:59.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.164+0000 7f2aebc00640 1 --2- 172.21.15.69:0/1163638193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ae4109f10 0x7f2ae410c3a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.164+0000 7f2aebc00640 1 --2- 172.21.15.69:0/1163638193 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ae4074a60 0x7f2ae4074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.164+0000 7f2aebc00640 1 --2- 172.21.15.69:0/1163638193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ae4076040 0x7f2ae4074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.164+0000 7f2aebc00640 1 -- 172.21.15.69:0/1163638193 >> 172.21.15.69:0/1163638193 conn(0x7f2ae40fd5e0 msgr2=0x7f2ae40ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:59.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.164+0000 7f2aebc00640 1 -- 172.21.15.69:0/1163638193 shutdown_connections 2024-04-24T09:03:59.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.164+0000 7f2aebc00640 1 -- 172.21.15.69:0/1163638193 wait complete. 2024-04-24T09:03:59.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.165+0000 7f2aebc00640 1 Processor -- start 2024-04-24T09:03:59.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.165+0000 7f2aebc00640 1 -- start start 2024-04-24T09:03:59.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.165+0000 7f2aebc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ae4074a60 0x7f2ae419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:59.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.165+0000 7f2aebc00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ae4076040 0x7f2ae419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:59.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.165+0000 7f2aebc00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ae4109f10 0x7f2ae41a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:59.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.165+0000 7f2aebc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410fe10 con 0x7f2ae4074a60 2024-04-24T09:03:59.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.165+0000 7f2aebc00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410fc90 con 0x7f2ae4076040 2024-04-24T09:03:59.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.165+0000 7f2aebc00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410ff90 con 0x7f2ae4109f10 2024-04-24T09:03:59.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.165+0000 7f2ae9200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ae4074a60 0x7f2ae419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:59.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.166+0000 7f2ae9200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ae4074a60 0x7f2ae419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48484/0 (socket says 172.21.15.69:48484) 2024-04-24T09:03:59.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.166+0000 7f2ae3e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ae4076040 0x7f2ae419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:59.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.166+0000 7f2ae9200640 1 -- 172.21.15.69:0/395476377 learned_addr learned my addr 172.21.15.69:0/395476377 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:59.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.166+0000 7f2ae9c00640 1 --2- 172.21.15.69:0/395476377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ae4109f10 0x7f2ae41a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:59.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.166+0000 7f2ae9200640 1 -- 172.21.15.69:0/395476377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ae4109f10 msgr2=0x7f2ae41a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:59.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.166+0000 7f2ae9200640 1 --2- 172.21.15.69:0/395476377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ae4109f10 0x7f2ae41a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.166+0000 7f2ae9200640 1 -- 172.21.15.69:0/395476377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ae4076040 msgr2=0x7f2ae419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:59.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.166+0000 7f2ae9200640 1 --2- 172.21.15.69:0/395476377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ae4076040 0x7f2ae419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.166+0000 7f2ae9200640 1 -- 172.21.15.69:0/395476377 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae41a20c0 con 0x7f2ae4074a60 2024-04-24T09:03:59.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.166+0000 7f2ae9200640 1 --2- 172.21.15.69:0/395476377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ae4074a60 0x7f2ae419a4a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800e9f0 tx=0x7f2ad800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:59.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.167+0000 7f2ae1600640 1 -- 172.21.15.69:0/395476377 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad800ccc0 con 0x7f2ae4074a60 2024-04-24T09:03:59.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.167+0000 7f2ae1600640 1 -- 172.21.15.69:0/395476377 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ad80108e0 con 0x7f2ae4074a60 2024-04-24T09:03:59.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.167+0000 7f2aebc00640 1 -- 172.21.15.69:0/395476377 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae41a23b0 con 0x7f2ae4074a60 2024-04-24T09:03:59.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.167+0000 7f2ae1600640 1 -- 172.21.15.69:0/395476377 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad80187c0 con 0x7f2ae4074a60 2024-04-24T09:03:59.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.167+0000 7f2aebc00640 1 -- 172.21.15.69:0/395476377 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae41a2950 con 0x7f2ae4074a60 2024-04-24T09:03:59.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.168+0000 7f2aebc00640 1 -- 172.21.15.69:0/395476377 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae4108c40 con 0x7f2ae4074a60 2024-04-24T09:03:59.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.170+0000 7f2ae1600640 1 -- 172.21.15.69:0/395476377 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2ad8010a90 con 0x7f2ae4074a60 2024-04-24T09:03:59.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.171+0000 7f2ae1600640 1 --2- 172.21.15.69:0/395476377 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2acc0a3c30 0x7f2acc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:59.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.171+0000 7f2ae1600640 1 -- 172.21.15.69:0/395476377 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2ad80d0340 con 0x7f2ae4074a60 2024-04-24T09:03:59.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.172+0000 7f2ae3e00640 1 --2- 172.21.15.69:0/395476377 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2acc0a3c30 0x7f2acc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:59.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.172+0000 7f2ae3e00640 1 --2- 172.21.15.69:0/395476377 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2acc0a3c30 0x7f2acc0a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2ae419b9b0 tx=0x7f2ad0009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:59.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.176+0000 7f2ae1600640 1 -- 172.21.15.69:0/395476377 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ad809d120 con 0x7f2ae4074a60 2024-04-24T09:03:59.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.376+0000 7f2aebc00640 1 -- 172.21.15.69:0/395476377 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2ae4062c00 con 0x7f2ae4074a60 2024-04-24T09:03:59.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.377+0000 7f2ae1600640 1 -- 172.21.15.69:0/395476377 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f2ad809ca20 con 0x7f2ae4074a60 2024-04-24T09:03:59.377 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-24T09:03:59.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.379+0000 7f2aebc00640 1 -- 172.21.15.69:0/395476377 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2acc0a3c30 msgr2=0x7f2acc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:59.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.379+0000 7f2aebc00640 1 --2- 172.21.15.69:0/395476377 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2acc0a3c30 0x7f2acc0a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2ae419b9b0 tx=0x7f2ad0009110 comp rx=0 tx=0).stop 2024-04-24T09:03:59.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.379+0000 7f2aebc00640 1 -- 172.21.15.69:0/395476377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ae4074a60 msgr2=0x7f2ae419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:59.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.379+0000 7f2aebc00640 1 --2- 172.21.15.69:0/395476377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ae4074a60 0x7f2ae419a4a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800e9f0 tx=0x7f2ad800edf0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.380+0000 7f2aebc00640 1 -- 172.21.15.69:0/395476377 shutdown_connections 2024-04-24T09:03:59.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.380+0000 7f2aebc00640 1 --2- 172.21.15.69:0/395476377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ae4109f10 0x7f2ae41a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.380+0000 7f2aebc00640 1 --2- 172.21.15.69:0/395476377 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2acc0a3c30 0x7f2acc0a60f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.380+0000 7f2aebc00640 1 --2- 172.21.15.69:0/395476377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ae4076040 0x7f2ae419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.380+0000 7f2aebc00640 1 --2- 172.21.15.69:0/395476377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ae4074a60 0x7f2ae419a4a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.380+0000 7f2aebc00640 1 -- 172.21.15.69:0/395476377 >> 172.21.15.69:0/395476377 conn(0x7f2ae40fd5e0 msgr2=0x7f2ae4105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:59.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.380+0000 7f2aebc00640 1 -- 172.21.15.69:0/395476377 shutdown_connections 2024-04-24T09:03:59.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.380+0000 7f2aebc00640 1 -- 172.21.15.69:0/395476377 wait complete. 2024-04-24T09:03:59.397 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T09:03:59.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.580+0000 7f6ed8000640 1 -- 172.21.15.69:0/3019345485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ed0107d30 msgr2=0x7f6ed010a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:59.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.580+0000 7f6ed8000640 1 --2- 172.21.15.69:0/3019345485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ed0107d30 0x7f6ed010a180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009ca0 tx=0x7f6ec802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.581+0000 7f6ed8000640 1 -- 172.21.15.69:0/3019345485 shutdown_connections 2024-04-24T09:03:59.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.581+0000 7f6ed8000640 1 --2- 172.21.15.69:0/3019345485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ed0107d30 0x7f6ed010a180 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.581+0000 7f6ed8000640 1 --2- 172.21.15.69:0/3019345485 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ed0105400 0x7f6ed01077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.581+0000 7f6ed8000640 1 --2- 172.21.15.69:0/3019345485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ed0102ad0 0x7f6ed0104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.581+0000 7f6ed8000640 1 -- 172.21.15.69:0/3019345485 >> 172.21.15.69:0/3019345485 conn(0x7f6ed00fc4e0 msgr2=0x7f6ed00fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:59.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.581+0000 7f6ed8000640 1 -- 172.21.15.69:0/3019345485 shutdown_connections 2024-04-24T09:03:59.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.581+0000 7f6ed8000640 1 -- 172.21.15.69:0/3019345485 wait complete. 2024-04-24T09:03:59.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.582+0000 7f6ed8000640 1 Processor -- start 2024-04-24T09:03:59.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.582+0000 7f6ed8000640 1 -- start start 2024-04-24T09:03:59.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.582+0000 7f6ed8000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ed0102ad0 0x7f6ed019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:59.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.582+0000 7f6ed8000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ed0105400 0x7f6ed019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:59.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.582+0000 7f6ed8000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ed0107d30 0x7f6ed01a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:59.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.582+0000 7f6ed8000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed010dbe0 con 0x7f6ed0107d30 2024-04-24T09:03:59.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.582+0000 7f6ed8000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed010da60 con 0x7f6ed0105400 2024-04-24T09:03:59.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.582+0000 7f6ed8000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed010dd60 con 0x7f6ed0102ad0 2024-04-24T09:03:59.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.582+0000 7f6ed6000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ed0107d30 0x7f6ed01a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.582+0000 7f6ed6000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ed0107d30 0x7f6ed01a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48512/0 (socket says 172.21.15.69:48512) 2024-04-24T09:03:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.582+0000 7f6ed6000640 1 -- 172.21.15.69:0/2124124565 learned_addr learned my addr 172.21.15.69:0/2124124565 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:03:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.583+0000 7f6ed5600640 1 --2- 172.21.15.69:0/2124124565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ed0102ad0 0x7f6ed019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.583+0000 7f6ed4c00640 1 --2- 172.21.15.69:0/2124124565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ed0105400 0x7f6ed019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.583+0000 7f6ed6000640 1 -- 172.21.15.69:0/2124124565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ed0102ad0 msgr2=0x7f6ed019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.583+0000 7f6ed6000640 1 --2- 172.21.15.69:0/2124124565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ed0102ad0 0x7f6ed019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.583+0000 7f6ed6000640 1 -- 172.21.15.69:0/2124124565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ed0105400 msgr2=0x7f6ed019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.583+0000 7f6ed6000640 1 --2- 172.21.15.69:0/2124124565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ed0105400 0x7f6ed019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.583+0000 7f6ed6000640 1 -- 172.21.15.69:0/2124124565 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed01a20b0 con 0x7f6ed0107d30 2024-04-24T09:03:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.583+0000 7f6ed5600640 1 --2- 172.21.15.69:0/2124124565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ed0102ad0 0x7f6ed019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:03:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.583+0000 7f6ed6000640 1 --2- 172.21.15.69:0/2124124565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ed0107d30 0x7f6ed01a1a40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8031040 tx=0x7f6ec8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.583+0000 7f6ece000640 1 -- 172.21.15.69:0/2124124565 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8032ec0 con 0x7f6ed0107d30 2024-04-24T09:03:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.583+0000 7f6ece000640 1 -- 172.21.15.69:0/2124124565 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ec80044f0 con 0x7f6ed0107d30 2024-04-24T09:03:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.583+0000 7f6ed8000640 1 -- 172.21.15.69:0/2124124565 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed01a2340 con 0x7f6ed0107d30 2024-04-24T09:03:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.583+0000 7f6ece000640 1 -- 172.21.15.69:0/2124124565 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8030d10 con 0x7f6ed0107d30 2024-04-24T09:03:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.584+0000 7f6ed8000640 1 -- 172.21.15.69:0/2124124565 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed01a28e0 con 0x7f6ed0107d30 2024-04-24T09:03:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.585+0000 7f6ed8000640 1 -- 172.21.15.69:0/2124124565 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed0109820 con 0x7f6ed0107d30 2024-04-24T09:03:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.587+0000 7f6ece000640 1 -- 172.21.15.69:0/2124124565 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6ec8004040 con 0x7f6ed0107d30 2024-04-24T09:03:59.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.588+0000 7f6ece000640 1 --2- 172.21.15.69:0/2124124565 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6eb80a3dd0 0x7f6eb80a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:59.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.588+0000 7f6ece000640 1 -- 172.21.15.69:0/2124124565 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6ec80f1830 con 0x7f6ed0107d30 2024-04-24T09:03:59.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.588+0000 7f6ed5600640 1 --2- 172.21.15.69:0/2124124565 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6eb80a3dd0 0x7f6eb80a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:59.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.589+0000 7f6ed5600640 1 --2- 172.21.15.69:0/2124124565 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6eb80a3dd0 0x7f6eb80a6290 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4005d00 tx=0x7f6ec4005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:59.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.592+0000 7f6ece000640 1 -- 172.21.15.69:0/2124124565 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ec80be610 con 0x7f6ed0107d30 2024-04-24T09:03:59.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.797+0000 7f6ed8000640 1 -- 172.21.15.69:0/2124124565 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6ed0062c00 con 0x7f6ed0107d30 2024-04-24T09:03:59.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.798+0000 7f6ece000640 1 -- 172.21.15.69:0/2124124565 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f6ec80bdf10 con 0x7f6ed0107d30 2024-04-24T09:03:59.798 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:03:59.798 INFO:teuthology.orchestra.run.smithi069.stdout:[ 2024-04-24T09:03:59.798 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:03:59.799 INFO:teuthology.orchestra.run.smithi069.stdout: "poolnum": 1, 2024-04-24T09:03:59.799 INFO:teuthology.orchestra.run.smithi069.stdout: "poolname": ".mgr" 2024-04-24T09:03:59.799 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:03:59.799 INFO:teuthology.orchestra.run.smithi069.stdout:] 2024-04-24T09:03:59.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.800+0000 7f6ed8000640 1 -- 172.21.15.69:0/2124124565 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6eb80a3dd0 msgr2=0x7f6eb80a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:59.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.800+0000 7f6ed8000640 1 --2- 172.21.15.69:0/2124124565 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6eb80a3dd0 0x7f6eb80a6290 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4005d00 tx=0x7f6ec4005c70 comp rx=0 tx=0).stop 2024-04-24T09:03:59.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.800+0000 7f6ed8000640 1 -- 172.21.15.69:0/2124124565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ed0107d30 msgr2=0x7f6ed01a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:59.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.800+0000 7f6ed8000640 1 --2- 172.21.15.69:0/2124124565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ed0107d30 0x7f6ed01a1a40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8031040 tx=0x7f6ec8003ba0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.800+0000 7f6ed8000640 1 -- 172.21.15.69:0/2124124565 shutdown_connections 2024-04-24T09:03:59.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.800+0000 7f6ed8000640 1 --2- 172.21.15.69:0/2124124565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ed0107d30 0x7f6ed01a1a40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.800+0000 7f6ed8000640 1 --2- 172.21.15.69:0/2124124565 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6eb80a3dd0 0x7f6eb80a6290 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.800+0000 7f6ed8000640 1 --2- 172.21.15.69:0/2124124565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ed0105400 0x7f6ed019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.800+0000 7f6ed8000640 1 --2- 172.21.15.69:0/2124124565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ed0102ad0 0x7f6ed019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:59.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.800+0000 7f6ed8000640 1 -- 172.21.15.69:0/2124124565 >> 172.21.15.69:0/2124124565 conn(0x7f6ed00fc4e0 msgr2=0x7f6ed0102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:59.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.800+0000 7f6ed8000640 1 -- 172.21.15.69:0/2124124565 shutdown_connections 2024-04-24T09:03:59.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:03:59.800+0000 7f6ed8000640 1 -- 172.21.15.69:0/2124124565 wait complete. 2024-04-24T09:03:59.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:59 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] up:boot 2024-04-24T09:03:59.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:59 smithi069 ceph-mon[28170]: fsmap 12 up:standby 2024-04-24T09:03:59.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:59 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T09:03:59.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:59 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/150001430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:59.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:59 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3582249883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:03:59.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:03:59 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/395476377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:03:59.818 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T09:03:59.818 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T09:03:59.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:59 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] up:boot 2024-04-24T09:03:59.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:59 smithi081 ceph-mon[30961]: fsmap 12 up:standby 2024-04-24T09:03:59.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:59 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T09:03:59.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:59 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/150001430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:59.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:59 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3582249883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:03:59.820 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:03:59 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/395476377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:03:59.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:59 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] up:boot 2024-04-24T09:03:59.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:59 smithi130 ceph-mon[31553]: fsmap 12 up:standby 2024-04-24T09:03:59.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:59 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T09:03:59.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:59 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/150001430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:59.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:59 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3582249883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:03:59.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:03:59 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/395476377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:04:00.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.037+0000 7f7577000640 1 -- 172.21.15.69:0/2097787467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7570105470 msgr2=0x7f75701058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:00.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.037+0000 7f7577000640 1 --2- 172.21.15.69:0/2097787467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7570105470 0x7f75701058f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7558009900 tx=0x7f755802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:00.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.037+0000 7f7577000640 1 -- 172.21.15.69:0/2097787467 shutdown_connections 2024-04-24T09:04:00.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.037+0000 7f7577000640 1 --2- 172.21.15.69:0/2097787467 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7570105e30 0x7f757010c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:00.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.037+0000 7f7577000640 1 --2- 172.21.15.69:0/2097787467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7570105470 0x7f75701058f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:00.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.037+0000 7f7577000640 1 --2- 172.21.15.69:0/2097787467 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7570104280 0x7f75701046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:00.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.037+0000 7f7577000640 1 -- 172.21.15.69:0/2097787467 >> 172.21.15.69:0/2097787467 conn(0x7f75700ff7f0 msgr2=0x7f7570101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:00.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.037+0000 7f7577000640 1 -- 172.21.15.69:0/2097787467 shutdown_connections 2024-04-24T09:04:00.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.037+0000 7f7577000640 1 -- 172.21.15.69:0/2097787467 wait complete. 2024-04-24T09:04:00.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.038+0000 7f7577000640 1 Processor -- start 2024-04-24T09:04:00.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.038+0000 7f7577000640 1 -- start start 2024-04-24T09:04:00.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.038+0000 7f7577000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7570104280 0x7f757019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:00.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.038+0000 7f7577000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7570105470 0x7f757019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:00.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.039+0000 7f7577000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7570105e30 0x7f75701a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:00.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.039+0000 7f7577000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f757010fe10 con 0x7f7570105e30 2024-04-24T09:04:00.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.039+0000 7f7577000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f757010fc90 con 0x7f7570104280 2024-04-24T09:04:00.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.039+0000 7f7577000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f757010ff90 con 0x7f7570105470 2024-04-24T09:04:00.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.039+0000 7f7575000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7570105e30 0x7f75701a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:00.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.039+0000 7f7575000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7570105e30 0x7f75701a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48518/0 (socket says 172.21.15.69:48518) 2024-04-24T09:04:00.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.039+0000 7f7575000640 1 -- 172.21.15.69:0/3459644461 learned_addr learned my addr 172.21.15.69:0/3459644461 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:00.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.039+0000 7f7567400640 1 --2- 172.21.15.69:0/3459644461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7570105470 0x7f757019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:00.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.039+0000 7f756fe00640 1 --2- 172.21.15.69:0/3459644461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7570104280 0x7f757019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:00.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.039+0000 7f7575000640 1 -- 172.21.15.69:0/3459644461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7570105470 msgr2=0x7f757019a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:00.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.039+0000 7f7575000640 1 --2- 172.21.15.69:0/3459644461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7570105470 0x7f757019a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:00.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.039+0000 7f7575000640 1 -- 172.21.15.69:0/3459644461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7570104280 msgr2=0x7f757019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:00.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.039+0000 7f7575000640 1 --2- 172.21.15.69:0/3459644461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7570104280 0x7f757019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:00.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.039+0000 7f7575000640 1 -- 172.21.15.69:0/3459644461 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75701a2060 con 0x7f7570105e30 2024-04-24T09:04:00.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.040+0000 7f7575000640 1 --2- 172.21.15.69:0/3459644461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7570105e30 0x7f75701a19f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7568009fd0 tx=0x7f756800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:00.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.040+0000 7f756d600640 1 -- 172.21.15.69:0/3459644461 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568017070 con 0x7f7570105e30 2024-04-24T09:04:00.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.040+0000 7f756d600640 1 -- 172.21.15.69:0/3459644461 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7568003e20 con 0x7f7570105e30 2024-04-24T09:04:00.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.040+0000 7f7577000640 1 -- 172.21.15.69:0/3459644461 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75701a2350 con 0x7f7570105e30 2024-04-24T09:04:00.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.040+0000 7f756d600640 1 -- 172.21.15.69:0/3459644461 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568010460 con 0x7f7570105e30 2024-04-24T09:04:00.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.040+0000 7f7577000640 1 -- 172.21.15.69:0/3459644461 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75701a2820 con 0x7f7570105e30 2024-04-24T09:04:00.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.041+0000 7f7577000640 1 -- 172.21.15.69:0/3459644461 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f757010b620 con 0x7f7570105e30 2024-04-24T09:04:00.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.043+0000 7f756d600640 1 -- 172.21.15.69:0/3459644461 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7568003970 con 0x7f7570105e30 2024-04-24T09:04:00.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.046+0000 7f756d600640 1 --2- 172.21.15.69:0/3459644461 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f75500a3bb0 0x7f75500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:00.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.046+0000 7f756d600640 1 -- 172.21.15.69:0/3459644461 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f75680cfda0 con 0x7f7570105e30 2024-04-24T09:04:00.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.046+0000 7f756fe00640 1 --2- 172.21.15.69:0/3459644461 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f75500a3bb0 0x7f75500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:00.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.047+0000 7f756fe00640 1 --2- 172.21.15.69:0/3459644461 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f75500a3bb0 0x7f75500a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f75701052d0 tx=0x7f7560005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:00.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.047+0000 7f756d600640 1 -- 172.21.15.69:0/3459644461 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f756809c530 con 0x7f7570105e30 2024-04-24T09:04:00.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.288+0000 7f7577000640 1 -- 172.21.15.69:0/3459644461 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f757002d620 con 0x7f7570105e30 2024-04-24T09:04:00.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.502+0000 7f756d600640 1 -- 172.21.15.69:0/3459644461 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f75680a0830 con 0x7f7570105e30 2024-04-24T09:04:00.502 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_metadata' created 2024-04-24T09:04:00.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.505+0000 7f7577000640 1 -- 172.21.15.69:0/3459644461 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f75500a3bb0 msgr2=0x7f75500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:00.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.505+0000 7f7577000640 1 --2- 172.21.15.69:0/3459644461 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f75500a3bb0 0x7f75500a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f75701052d0 tx=0x7f7560005f90 comp rx=0 tx=0).stop 2024-04-24T09:04:00.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.505+0000 7f7577000640 1 -- 172.21.15.69:0/3459644461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7570105e30 msgr2=0x7f75701a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:00.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.505+0000 7f7577000640 1 --2- 172.21.15.69:0/3459644461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7570105e30 0x7f75701a19f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7568009fd0 tx=0x7f756800b260 comp rx=0 tx=0).stop 2024-04-24T09:04:00.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.506+0000 7f7577000640 1 -- 172.21.15.69:0/3459644461 shutdown_connections 2024-04-24T09:04:00.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.506+0000 7f7577000640 1 --2- 172.21.15.69:0/3459644461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7570105e30 0x7f75701a19f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:00.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.506+0000 7f7577000640 1 --2- 172.21.15.69:0/3459644461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7570105470 0x7f757019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:00.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.506+0000 7f7577000640 1 --2- 172.21.15.69:0/3459644461 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f75500a3bb0 0x7f75500a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:00.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.506+0000 7f7577000640 1 --2- 172.21.15.69:0/3459644461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7570104280 0x7f757019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:00.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.506+0000 7f7577000640 1 -- 172.21.15.69:0/3459644461 >> 172.21.15.69:0/3459644461 conn(0x7f75700ff7f0 msgr2=0x7f75701087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:00.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.506+0000 7f7577000640 1 -- 172.21.15.69:0/3459644461 shutdown_connections 2024-04-24T09:04:00.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.506+0000 7f7577000640 1 -- 172.21.15.69:0/3459644461 wait complete. 2024-04-24T09:04:00.532 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-24T09:04:00.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.717+0000 7f3a35a00640 1 -- 172.21.15.69:0/3063206803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a30075fe0 msgr2=0x7f3a30074430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:00.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.717+0000 7f3a35a00640 1 --2- 172.21.15.69:0/3063206803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a30075fe0 0x7f3a30074430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009900 tx=0x7f3a2002a480 comp rx=0 tx=0).stop 2024-04-24T09:04:00.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.717+0000 7f3a35a00640 1 -- 172.21.15.69:0/3063206803 shutdown_connections 2024-04-24T09:04:00.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.717+0000 7f3a35a00640 1 --2- 172.21.15.69:0/3063206803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a30109ee0 0x7f3a3010c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:00.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.717+0000 7f3a35a00640 1 --2- 172.21.15.69:0/3063206803 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a30074a00 0x7f3a30074e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:00.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.717+0000 7f3a35a00640 1 --2- 172.21.15.69:0/3063206803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a30075fe0 0x7f3a30074430 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:00.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.717+0000 7f3a35a00640 1 -- 172.21.15.69:0/3063206803 >> 172.21.15.69:0/3063206803 conn(0x7f3a300fd5e0 msgr2=0x7f3a300ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:00.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.717+0000 7f3a35a00640 1 -- 172.21.15.69:0/3063206803 shutdown_connections 2024-04-24T09:04:00.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.717+0000 7f3a35a00640 1 -- 172.21.15.69:0/3063206803 wait complete. 2024-04-24T09:04:00.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.718+0000 7f3a35a00640 1 Processor -- start 2024-04-24T09:04:00.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.718+0000 7f3a35a00640 1 -- start start 2024-04-24T09:04:00.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.718+0000 7f3a35a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a30074a00 0x7f3a3019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:00.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.719+0000 7f3a35a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a30075fe0 0x7f3a3019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:00.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.719+0000 7f3a35a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a30109ee0 0x7f3a301a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:00.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.719+0000 7f3a35a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3010fde0 con 0x7f3a30074a00 2024-04-24T09:04:00.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.719+0000 7f3a35a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3010fc60 con 0x7f3a30109ee0 2024-04-24T09:04:00.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.719+0000 7f3a35a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3010ff60 con 0x7f3a30075fe0 2024-04-24T09:04:00.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.719+0000 7f3a2ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a30074a00 0x7f3a3019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:00.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.719+0000 7f3a2ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a30074a00 0x7f3a3019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48534/0 (socket says 172.21.15.69:48534) 2024-04-24T09:04:00.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.719+0000 7f3a2ea00640 1 -- 172.21.15.69:0/3859459217 learned_addr learned my addr 172.21.15.69:0/3859459217 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:00.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.719+0000 7f3a2f400640 1 --2- 172.21.15.69:0/3859459217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a30109ee0 0x7f3a301a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:00.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.719+0000 7f3a2e000640 1 --2- 172.21.15.69:0/3859459217 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a30075fe0 0x7f3a3019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:00.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.719+0000 7f3a2ea00640 1 -- 172.21.15.69:0/3859459217 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a30075fe0 msgr2=0x7f3a3019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:00.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.719+0000 7f3a2ea00640 1 --2- 172.21.15.69:0/3859459217 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a30075fe0 0x7f3a3019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:00.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.719+0000 7f3a2ea00640 1 -- 172.21.15.69:0/3859459217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a30109ee0 msgr2=0x7f3a301a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:00.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.719+0000 7f3a2ea00640 1 --2- 172.21.15.69:0/3859459217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a30109ee0 0x7f3a301a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:00.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.719+0000 7f3a2ea00640 1 -- 172.21.15.69:0/3859459217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a301a20a0 con 0x7f3a30074a00 2024-04-24T09:04:00.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.720+0000 7f3a2ea00640 1 --2- 172.21.15.69:0/3859459217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a30074a00 0x7f3a3019a4d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3a200314e0 tx=0x7f3a20003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:00.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.720+0000 7f3a27400640 1 -- 172.21.15.69:0/3859459217 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2003f070 con 0x7f3a30074a00 2024-04-24T09:04:00.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.720+0000 7f3a35a00640 1 -- 172.21.15.69:0/3859459217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a301a2330 con 0x7f3a30074a00 2024-04-24T09:04:00.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.720+0000 7f3a27400640 1 -- 172.21.15.69:0/3859459217 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a20031c40 con 0x7f3a30074a00 2024-04-24T09:04:00.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.720+0000 7f3a27400640 1 -- 172.21.15.69:0/3859459217 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a20035030 con 0x7f3a30074a00 2024-04-24T09:04:00.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.720+0000 7f3a35a00640 1 -- 172.21.15.69:0/3859459217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a301a2870 con 0x7f3a30074a00 2024-04-24T09:04:00.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.721+0000 7f3a35a00640 1 -- 172.21.15.69:0/3859459217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a30108ca0 con 0x7f3a30074a00 2024-04-24T09:04:00.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.724+0000 7f3a27400640 1 -- 172.21.15.69:0/3859459217 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3a20042070 con 0x7f3a30074a00 2024-04-24T09:04:00.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.726+0000 7f3a27400640 1 --2- 172.21.15.69:0/3859459217 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a140a3c30 0x7f3a140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:00.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.726+0000 7f3a27400640 1 -- 172.21.15.69:0/3859459217 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f3a200f0f50 con 0x7f3a30074a00 2024-04-24T09:04:00.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.727+0000 7f3a2e000640 1 --2- 172.21.15.69:0/3859459217 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a140a3c30 0x7f3a140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:00.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.727+0000 7f3a2e000640 1 --2- 172.21.15.69:0/3859459217 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a140a3c30 0x7f3a140a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c005d00 tx=0x7f3a1c005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:00.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.729+0000 7f3a27400640 1 -- 172.21.15.69:0/3859459217 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a200bdbc0 con 0x7f3a30074a00 2024-04-24T09:04:00.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:00 smithi081 ceph-mon[30961]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:04:00.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:00 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2124124565' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T09:04:00.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:00 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3459644461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T09:04:00.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:00 smithi069 ceph-mon[28170]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:04:00.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:00 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2124124565' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T09:04:00.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:00 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3459644461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T09:04:00.854 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:00 smithi130 ceph-mon[31553]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:04:00.855 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:00 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2124124565' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T09:04:00.855 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:00 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3459644461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T09:04:00.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:00.947+0000 7f3a35a00640 1 -- 172.21.15.69:0/3859459217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f3a3002d620 con 0x7f3a30074a00 2024-04-24T09:04:01.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.509+0000 7f3a27400640 1 -- 172.21.15.69:0/3859459217 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7f3a200bd4c0 con 0x7f3a30074a00 2024-04-24T09:04:01.509 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data' created 2024-04-24T09:04:01.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.513+0000 7f3a35a00640 1 -- 172.21.15.69:0/3859459217 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a140a3c30 msgr2=0x7f3a140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:01.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.513+0000 7f3a35a00640 1 --2- 172.21.15.69:0/3859459217 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a140a3c30 0x7f3a140a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c005d00 tx=0x7f3a1c005c50 comp rx=0 tx=0).stop 2024-04-24T09:04:01.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.513+0000 7f3a35a00640 1 -- 172.21.15.69:0/3859459217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a30074a00 msgr2=0x7f3a3019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:01.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.513+0000 7f3a35a00640 1 --2- 172.21.15.69:0/3859459217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a30074a00 0x7f3a3019a4d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3a200314e0 tx=0x7f3a20003c10 comp rx=0 tx=0).stop 2024-04-24T09:04:01.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.514+0000 7f3a35a00640 1 -- 172.21.15.69:0/3859459217 shutdown_connections 2024-04-24T09:04:01.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.514+0000 7f3a35a00640 1 --2- 172.21.15.69:0/3859459217 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a140a3c30 0x7f3a140a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:01.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.514+0000 7f3a35a00640 1 --2- 172.21.15.69:0/3859459217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a30109ee0 0x7f3a301a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:01.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.514+0000 7f3a35a00640 1 --2- 172.21.15.69:0/3859459217 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a30075fe0 0x7f3a3019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:01.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.514+0000 7f3a35a00640 1 --2- 172.21.15.69:0/3859459217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a30074a00 0x7f3a3019a4d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:01.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.514+0000 7f3a35a00640 1 -- 172.21.15.69:0/3859459217 >> 172.21.15.69:0/3859459217 conn(0x7f3a300fd5e0 msgr2=0x7f3a30105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:01.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.516+0000 7f3a35a00640 1 -- 172.21.15.69:0/3859459217 shutdown_connections 2024-04-24T09:04:01.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.516+0000 7f3a35a00640 1 -- 172.21.15.69:0/3859459217 wait complete. 2024-04-24T09:04:01.554 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T09:04:01.628 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:01 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3459644461' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T09:04:01.629 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:01 smithi130 ceph-mon[31553]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T09:04:01.629 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:01 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.629 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:01 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.629 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:01 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3859459217' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T09:04:01.629 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:01 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.629 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:01 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.629 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:01 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.629 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:01 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.629 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:01 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3859459217' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-24T09:04:01.630 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:01 smithi130 ceph-mon[31553]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T09:04:01.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.724+0000 7f107c000640 1 -- 172.21.15.69:0/3762257948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1074109850 msgr2=0x7f1074109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:01.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.724+0000 7f107c000640 1 --2- 172.21.15.69:0/3762257948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1074109850 0x7f1074109cd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1060009900 tx=0x7f106002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:01.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.724+0000 7f107c000640 1 -- 172.21.15.69:0/3762257948 shutdown_connections 2024-04-24T09:04:01.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.724+0000 7f107c000640 1 --2- 172.21.15.69:0/3762257948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f107410a210 0x7f1074110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:01.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.724+0000 7f107c000640 1 --2- 172.21.15.69:0/3762257948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1074109850 0x7f1074109cd0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:01.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.724+0000 7f107c000640 1 --2- 172.21.15.69:0/3762257948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1074108660 0x7f1074108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:01.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.724+0000 7f107c000640 1 -- 172.21.15.69:0/3762257948 >> 172.21.15.69:0/3762257948 conn(0x7f1074077600 msgr2=0x7f1074079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:01.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.724+0000 7f107c000640 1 -- 172.21.15.69:0/3762257948 shutdown_connections 2024-04-24T09:04:01.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.724+0000 7f107c000640 1 -- 172.21.15.69:0/3762257948 wait complete. 2024-04-24T09:04:01.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.725+0000 7f107c000640 1 Processor -- start 2024-04-24T09:04:01.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.725+0000 7f107c000640 1 -- start start 2024-04-24T09:04:01.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.725+0000 7f107c000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1074108660 0x7f107419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:01.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.725+0000 7f107c000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1074109850 0x7f107419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:01.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.725+0000 7f107c000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f107410a210 0x7f10741a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:01.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.725+0000 7f107c000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f10741141c0 con 0x7f107410a210 2024-04-24T09:04:01.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.725+0000 7f107c000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1074114040 con 0x7f1074108660 2024-04-24T09:04:01.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.725+0000 7f107c000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1074114340 con 0x7f1074109850 2024-04-24T09:04:01.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.726+0000 7f107a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f107410a210 0x7f10741a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:01.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.726+0000 7f107a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f107410a210 0x7f10741a5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48548/0 (socket says 172.21.15.69:48548) 2024-04-24T09:04:01.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.726+0000 7f107a000640 1 -- 172.21.15.69:0/3561743237 learned_addr learned my addr 172.21.15.69:0/3561743237 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:01.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.726+0000 7f1078c00640 1 --2- 172.21.15.69:0/3561743237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1074109850 0x7f107419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:01.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.726+0000 7f1079600640 1 --2- 172.21.15.69:0/3561743237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1074108660 0x7f107419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:01.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.726+0000 7f107a000640 1 -- 172.21.15.69:0/3561743237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1074109850 msgr2=0x7f107419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:01.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.726+0000 7f107a000640 1 --2- 172.21.15.69:0/3561743237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1074109850 0x7f107419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:01.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.726+0000 7f107a000640 1 -- 172.21.15.69:0/3561743237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1074108660 msgr2=0x7f107419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:01.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.726+0000 7f107a000640 1 --2- 172.21.15.69:0/3561743237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1074108660 0x7f107419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:01.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.726+0000 7f107a000640 1 -- 172.21.15.69:0/3561743237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10741a6480 con 0x7f107410a210 2024-04-24T09:04:01.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.726+0000 7f1079600640 1 --2- 172.21.15.69:0/3561743237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1074108660 0x7f107419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:04:01.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.727+0000 7f107a000640 1 --2- 172.21.15.69:0/3561743237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f107410a210 0x7f10741a5e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f106c00cce0 tx=0x7f106c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:01.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.727+0000 7f1072000640 1 -- 172.21.15.69:0/3561743237 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c00f450 con 0x7f107410a210 2024-04-24T09:04:01.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.727+0000 7f1072000640 1 -- 172.21.15.69:0/3561743237 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f106c00faa0 con 0x7f107410a210 2024-04-24T09:04:01.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.727+0000 7f107c000640 1 -- 172.21.15.69:0/3561743237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10741a6770 con 0x7f107410a210 2024-04-24T09:04:01.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.727+0000 7f1072000640 1 -- 172.21.15.69:0/3561743237 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c018b70 con 0x7f107410a210 2024-04-24T09:04:01.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.727+0000 7f107c000640 1 -- 172.21.15.69:0/3561743237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10741a6c40 con 0x7f107410a210 2024-04-24T09:04:01.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.728+0000 7f107c000640 1 -- 172.21.15.69:0/3561743237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f107410fa50 con 0x7f107410a210 2024-04-24T09:04:01.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.730+0000 7f1072000640 1 -- 172.21.15.69:0/3561743237 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f106c018d10 con 0x7f107410a210 2024-04-24T09:04:01.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.732+0000 7f1072000640 1 --2- 172.21.15.69:0/3561743237 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f105c0a3bb0 0x7f105c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:01.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.733+0000 7f1072000640 1 -- 172.21.15.69:0/3561743237 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f106c0d0720 con 0x7f107410a210 2024-04-24T09:04:01.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.733+0000 7f1079600640 1 --2- 172.21.15.69:0/3561743237 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f105c0a3bb0 0x7f105c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:01.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.733+0000 7f1079600640 1 --2- 172.21.15.69:0/3561743237 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f105c0a3bb0 0x7f105c0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f10680058c0 tx=0x7f1068005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:01.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.736+0000 7f1072000640 1 -- 172.21.15.69:0/3561743237 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f106c014030 con 0x7f107410a210 2024-04-24T09:04:01.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:01 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3459644461' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T09:04:01.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:01 smithi081 ceph-mon[30961]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T09:04:01.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:01 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:01 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:01 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3859459217' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T09:04:01.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:01 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:01 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:01 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:01 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:01 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3859459217' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-24T09:04:01.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:01 smithi081 ceph-mon[30961]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T09:04:01.840 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:01 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3459644461' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T09:04:01.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:01 smithi069 ceph-mon[28170]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T09:04:01.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:01 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:01 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:01 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3859459217' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T09:04:01.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:01 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:01 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:01 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:01 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:01.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:01 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3859459217' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-24T09:04:01.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:01 smithi069 ceph-mon[28170]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T09:04:01.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:01.989+0000 7f107c000640 1 -- 172.21.15.69:0/3561743237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1074062c00 con 0x7f107410a210 2024-04-24T09:04:02.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.534+0000 7f1072000640 1 -- 172.21.15.69:0/3561743237 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f106c09d210 con 0x7f107410a210 2024-04-24T09:04:02.534 INFO:teuthology.orchestra.run.smithi069.stderr:new fs with metadata pool 2 and data pool 3 2024-04-24T09:04:02.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.538+0000 7f107c000640 1 -- 172.21.15.69:0/3561743237 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f105c0a3bb0 msgr2=0x7f105c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:02.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.538+0000 7f107c000640 1 --2- 172.21.15.69:0/3561743237 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f105c0a3bb0 0x7f105c0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f10680058c0 tx=0x7f1068005850 comp rx=0 tx=0).stop 2024-04-24T09:04:02.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.538+0000 7f107c000640 1 -- 172.21.15.69:0/3561743237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f107410a210 msgr2=0x7f10741a5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:02.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.538+0000 7f107c000640 1 --2- 172.21.15.69:0/3561743237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f107410a210 0x7f10741a5e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f106c00cce0 tx=0x7f106c00a620 comp rx=0 tx=0).stop 2024-04-24T09:04:02.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.538+0000 7f107c000640 1 -- 172.21.15.69:0/3561743237 shutdown_connections 2024-04-24T09:04:02.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.538+0000 7f107c000640 1 --2- 172.21.15.69:0/3561743237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f107410a210 0x7f10741a5e10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:02.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.538+0000 7f107c000640 1 --2- 172.21.15.69:0/3561743237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1074109850 0x7f107419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:02.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.538+0000 7f107c000640 1 --2- 172.21.15.69:0/3561743237 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f105c0a3bb0 0x7f105c0a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:02.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.538+0000 7f107c000640 1 --2- 172.21.15.69:0/3561743237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1074108660 0x7f107419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:02.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.538+0000 7f107c000640 1 -- 172.21.15.69:0/3561743237 >> 172.21.15.69:0/3561743237 conn(0x7f1074077600 msgr2=0x7f107410cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:02.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.538+0000 7f107c000640 1 -- 172.21.15.69:0/3561743237 shutdown_connections 2024-04-24T09:04:02.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.539+0000 7f107c000640 1 -- 172.21.15.69:0/3561743237 wait complete. 2024-04-24T09:04:02.565 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-24T09:04:02.566 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-24T09:04:02.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.758+0000 7f069e600640 1 -- 172.21.15.69:0/2113826256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0698100780 msgr2=0x7f0698102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:02.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.758+0000 7f069e600640 1 --2- 172.21.15.69:0/2113826256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0698100780 0x7f0698102bb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0680009900 tx=0x7f068002a410 comp rx=0 tx=0).stop 2024-04-24T09:04:02.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.758+0000 7f069e600640 1 -- 172.21.15.69:0/2113826256 shutdown_connections 2024-04-24T09:04:02.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.758+0000 7f069e600640 1 --2- 172.21.15.69:0/2113826256 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0698105aa0 0x7f0698105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:02.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.758+0000 7f069e600640 1 --2- 172.21.15.69:0/2113826256 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06981030f0 0x7f0698105560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:02.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.758+0000 7f069e600640 1 --2- 172.21.15.69:0/2113826256 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0698100780 0x7f0698102bb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:02.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.758+0000 7f069e600640 1 -- 172.21.15.69:0/2113826256 >> 172.21.15.69:0/2113826256 conn(0x7f06980fc3a0 msgr2=0x7f06980fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:02.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.758+0000 7f069e600640 1 -- 172.21.15.69:0/2113826256 shutdown_connections 2024-04-24T09:04:02.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.758+0000 7f069e600640 1 -- 172.21.15.69:0/2113826256 wait complete. 2024-04-24T09:04:02.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.759+0000 7f069e600640 1 Processor -- start 2024-04-24T09:04:02.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.759+0000 7f069e600640 1 -- start start 2024-04-24T09:04:02.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.759+0000 7f069e600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0698100780 0x7f069819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:02.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.759+0000 7f069e600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06981030f0 0x7f069819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:02.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.759+0000 7f069e600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0698105aa0 0x7f06981a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:02.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.759+0000 7f069e600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f06981140b0 con 0x7f06981030f0 2024-04-24T09:04:02.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.759+0000 7f069e600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0698113f30 con 0x7f0698105aa0 2024-04-24T09:04:02.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.759+0000 7f069e600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0698114230 con 0x7f0698100780 2024-04-24T09:04:02.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.760+0000 7f068fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06981030f0 0x7f069819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:02.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.760+0000 7f068fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06981030f0 0x7f069819ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48572/0 (socket says 172.21.15.69:48572) 2024-04-24T09:04:02.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.760+0000 7f068fe00640 1 -- 172.21.15.69:0/3689355752 learned_addr learned my addr 172.21.15.69:0/3689355752 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:02.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.760+0000 7f069d200640 1 --2- 172.21.15.69:0/3689355752 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0698100780 0x7f069819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:02.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.760+0000 7f069dc00640 1 --2- 172.21.15.69:0/3689355752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0698105aa0 0x7f06981a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:02.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.760+0000 7f068fe00640 1 -- 172.21.15.69:0/3689355752 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0698100780 msgr2=0x7f069819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:02.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.760+0000 7f068fe00640 1 --2- 172.21.15.69:0/3689355752 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0698100780 0x7f069819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:02.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.760+0000 7f068fe00640 1 -- 172.21.15.69:0/3689355752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0698105aa0 msgr2=0x7f06981a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:02.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.760+0000 7f068fe00640 1 --2- 172.21.15.69:0/3689355752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0698105aa0 0x7f06981a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:02.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.760+0000 7f068fe00640 1 -- 172.21.15.69:0/3689355752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06981a6360 con 0x7f06981030f0 2024-04-24T09:04:02.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.761+0000 7f068fe00640 1 --2- 172.21.15.69:0/3689355752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06981030f0 0x7f069819ecd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f068800d950 tx=0x7f068800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:02.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.761+0000 7f0696000640 1 -- 172.21.15.69:0/3689355752 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688017030 con 0x7f06981030f0 2024-04-24T09:04:02.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.761+0000 7f0696000640 1 -- 172.21.15.69:0/3689355752 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0688004020 con 0x7f06981030f0 2024-04-24T09:04:02.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.761+0000 7f069e600640 1 -- 172.21.15.69:0/3689355752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06981a6650 con 0x7f06981030f0 2024-04-24T09:04:02.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.761+0000 7f0696000640 1 -- 172.21.15.69:0/3689355752 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06880047a0 con 0x7f06981030f0 2024-04-24T09:04:02.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.761+0000 7f069e600640 1 -- 172.21.15.69:0/3689355752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06981a6ba0 con 0x7f06981030f0 2024-04-24T09:04:02.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.762+0000 7f069e600640 1 -- 172.21.15.69:0/3689355752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06980fde90 con 0x7f06981030f0 2024-04-24T09:04:02.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.764+0000 7f0696000640 1 -- 172.21.15.69:0/3689355752 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f06880041d0 con 0x7f06981030f0 2024-04-24T09:04:02.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.766+0000 7f0696000640 1 --2- 172.21.15.69:0/3689355752 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f06780a3bb0 0x7f06780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:02.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.766+0000 7f0696000640 1 -- 172.21.15.69:0/3689355752 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f06880cfdc0 con 0x7f06981030f0 2024-04-24T09:04:02.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.766+0000 7f069d200640 1 --2- 172.21.15.69:0/3689355752 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f06780a3bb0 0x7f06780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:02.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.767+0000 7f069d200640 1 --2- 172.21.15.69:0/3689355752 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f06780a3bb0 0x7f06780a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f06800314e0 tx=0x7f0680005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:02.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.772+0000 7f0696000640 1 -- 172.21.15.69:0/3689355752 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f068809c850 con 0x7f06981030f0 2024-04-24T09:04:02.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:02 smithi081 ceph-mon[30961]: pgmap v362: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:04:02.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:02 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3561743237' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T09:04:02.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:02 smithi081 ceph-mon[30961]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T09:04:02.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:02 smithi081 ceph-mon[30961]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T09:04:02.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:02 smithi081 ceph-mon[30961]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T09:04:02.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:02 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:04:02.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:02 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:04:02.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:02 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:02.840 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:02 smithi069 ceph-mon[28170]: pgmap v362: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:04:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:02 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3561743237' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T09:04:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:02 smithi069 ceph-mon[28170]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T09:04:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:02 smithi069 ceph-mon[28170]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T09:04:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:02 smithi069 ceph-mon[28170]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T09:04:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:04:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:04:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:02 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:02.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:02 smithi130 ceph-mon[31553]: pgmap v362: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:04:02.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:02 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3561743237' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T09:04:02.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:02 smithi130 ceph-mon[31553]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T09:04:02.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:02 smithi130 ceph-mon[31553]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T09:04:02.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:02 smithi130 ceph-mon[31553]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T09:04:02.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:02 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:04:02.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:02 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:04:02.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:02 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:02.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:02.994+0000 7f069e600640 1 -- 172.21.15.69:0/3689355752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f069802d620 con 0x7f06981030f0 2024-04-24T09:04:03.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.072+0000 7f0696000640 1 -- 172.21.15.69:0/3689355752 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7f068809c150 con 0x7f06981030f0 2024-04-24T09:04:03.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.075+0000 7f069e600640 1 -- 172.21.15.69:0/3689355752 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f06780a3bb0 msgr2=0x7f06780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:03.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.075+0000 7f069e600640 1 --2- 172.21.15.69:0/3689355752 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f06780a3bb0 0x7f06780a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f06800314e0 tx=0x7f0680005a20 comp rx=0 tx=0).stop 2024-04-24T09:04:03.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.075+0000 7f069e600640 1 -- 172.21.15.69:0/3689355752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06981030f0 msgr2=0x7f069819ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:03.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.075+0000 7f069e600640 1 --2- 172.21.15.69:0/3689355752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06981030f0 0x7f069819ecd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f068800d950 tx=0x7f068800de10 comp rx=0 tx=0).stop 2024-04-24T09:04:03.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.075+0000 7f069e600640 1 -- 172.21.15.69:0/3689355752 shutdown_connections 2024-04-24T09:04:03.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.075+0000 7f069e600640 1 --2- 172.21.15.69:0/3689355752 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f06780a3bb0 0x7f06780a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:03.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.075+0000 7f069e600640 1 --2- 172.21.15.69:0/3689355752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0698105aa0 0x7f06981a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:03.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.075+0000 7f069e600640 1 --2- 172.21.15.69:0/3689355752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06981030f0 0x7f069819ecd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:03.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.075+0000 7f069e600640 1 --2- 172.21.15.69:0/3689355752 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0698100780 0x7f069819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:03.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.075+0000 7f069e600640 1 -- 172.21.15.69:0/3689355752 >> 172.21.15.69:0/3689355752 conn(0x7f06980fc3a0 msgr2=0x7f0698103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:03.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.076+0000 7f069e600640 1 -- 172.21.15.69:0/3689355752 shutdown_connections 2024-04-24T09:04:03.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.076+0000 7f069e600640 1 -- 172.21.15.69:0/3689355752 wait complete. 2024-04-24T09:04:03.097 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T09:04:03.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.296+0000 7f0fc7200640 1 -- 172.21.15.69:0/251279784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fc8101e80 msgr2=0x7f0fc81022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:03.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.296+0000 7f0fc7200640 1 --2- 172.21.15.69:0/251279784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fc8101e80 0x7f0fc81022a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8009900 tx=0x7f0fb802a410 comp rx=0 tx=0).stop 2024-04-24T09:04:03.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.296+0000 7f0fc7200640 1 -- 172.21.15.69:0/251279784 shutdown_connections 2024-04-24T09:04:03.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.296+0000 7f0fc7200640 1 --2- 172.21.15.69:0/251279784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fc8103990 0x7f0fc810a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:03.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.296+0000 7f0fc7200640 1 --2- 172.21.15.69:0/251279784 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0fc8102fd0 0x7f0fc8103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:03.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.296+0000 7f0fc7200640 1 --2- 172.21.15.69:0/251279784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fc8101e80 0x7f0fc81022a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:03.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.296+0000 7f0fc7200640 1 -- 172.21.15.69:0/251279784 >> 172.21.15.69:0/251279784 conn(0x7f0fc80fd410 msgr2=0x7f0fc80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:03.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.296+0000 7f0fc7200640 1 -- 172.21.15.69:0/251279784 shutdown_connections 2024-04-24T09:04:03.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.296+0000 7f0fc7200640 1 -- 172.21.15.69:0/251279784 wait complete. 2024-04-24T09:04:03.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.297+0000 7f0fc7200640 1 Processor -- start 2024-04-24T09:04:03.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.297+0000 7f0fc7200640 1 -- start start 2024-04-24T09:04:03.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.297+0000 7f0fc7200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fc8101e80 0x7f0fc819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:03.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.297+0000 7f0fc7200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fc8102fd0 0x7f0fc819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.297+0000 7f0fc7200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0fc8103990 0x7f0fc81a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.297+0000 7f0fc7200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc810da00 con 0x7f0fc8102fd0 2024-04-24T09:04:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.297+0000 7f0fc7200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc810d880 con 0x7f0fc8101e80 2024-04-24T09:04:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.297+0000 7f0fc7200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc810db80 con 0x7f0fc8103990 2024-04-24T09:04:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.297+0000 7f0fc5400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fc8102fd0 0x7f0fc819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.298+0000 7f0fc5400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fc8102fd0 0x7f0fc819a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50482/0 (socket says 172.21.15.69:50482) 2024-04-24T09:04:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.298+0000 7f0fc5400640 1 -- 172.21.15.69:0/703406488 learned_addr learned my addr 172.21.15.69:0/703406488 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.298+0000 7f0fc5e00640 1 --2- 172.21.15.69:0/703406488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fc8101e80 0x7f0fc819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.298+0000 7f0fc6800640 1 --2- 172.21.15.69:0/703406488 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0fc8103990 0x7f0fc81a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.298+0000 7f0fc5400640 1 -- 172.21.15.69:0/703406488 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0fc8103990 msgr2=0x7f0fc81a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.298+0000 7f0fc5400640 1 --2- 172.21.15.69:0/703406488 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0fc8103990 0x7f0fc81a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.298+0000 7f0fc5400640 1 -- 172.21.15.69:0/703406488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fc8101e80 msgr2=0x7f0fc819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.298+0000 7f0fc5400640 1 --2- 172.21.15.69:0/703406488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fc8101e80 0x7f0fc819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:03.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.298+0000 7f0fc5400640 1 -- 172.21.15.69:0/703406488 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc81a1e90 con 0x7f0fc8102fd0 2024-04-24T09:04:03.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.298+0000 7f0fc5400640 1 --2- 172.21.15.69:0/703406488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fc8102fd0 0x7f0fc819a7b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400c970 tx=0x7f0fb400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:03.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.298+0000 7f0fbea00640 1 -- 172.21.15.69:0/703406488 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4007c10 con 0x7f0fc8102fd0 2024-04-24T09:04:03.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.298+0000 7f0fbea00640 1 -- 172.21.15.69:0/703406488 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fb4007db0 con 0x7f0fc8102fd0 2024-04-24T09:04:03.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.298+0000 7f0fbea00640 1 -- 172.21.15.69:0/703406488 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb400f870 con 0x7f0fc8102fd0 2024-04-24T09:04:03.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.299+0000 7f0fc7200640 1 -- 172.21.15.69:0/703406488 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc81a2180 con 0x7f0fc8102fd0 2024-04-24T09:04:03.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.299+0000 7f0fc7200640 1 -- 172.21.15.69:0/703406488 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc81a2680 con 0x7f0fc8102fd0 2024-04-24T09:04:03.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.300+0000 7f0fc7200640 1 -- 172.21.15.69:0/703406488 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fc81a2940 con 0x7f0fc8102fd0 2024-04-24T09:04:03.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.305+0000 7f0fbea00640 1 -- 172.21.15.69:0/703406488 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0fb400fa10 con 0x7f0fc8102fd0 2024-04-24T09:04:03.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.306+0000 7f0fbea00640 1 --2- 172.21.15.69:0/703406488 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0fac0a3c30 0x7f0fac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:03.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.306+0000 7f0fbea00640 1 -- 172.21.15.69:0/703406488 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7f0fb40d0aa0 con 0x7f0fc8102fd0 2024-04-24T09:04:03.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.307+0000 7f0fc5e00640 1 --2- 172.21.15.69:0/703406488 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0fac0a3c30 0x7f0fac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:03.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.307+0000 7f0fc5e00640 1 --2- 172.21.15.69:0/703406488 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0fac0a3c30 0x7f0fac0a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8000c00 tx=0x7f0fb80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:03.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.308+0000 7f0fbea00640 1 -- 172.21.15.69:0/703406488 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0fb409d510 con 0x7f0fc8102fd0 2024-04-24T09:04:03.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:03.531+0000 7f0fc7200640 1 -- 172.21.15.69:0/703406488 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0fc81a2bc0 con 0x7f0fc8102fd0 2024-04-24T09:04:03.816 INFO:journalctl@ceph.mds.b.smithi081.stdout:Apr 24 09:04:03 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-b[61445]: 2024-04-24T09:04:03.507+0000 7f7a13000700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T09:04:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:03 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3561743237' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T09:04:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:03 smithi081 ceph-mon[30961]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T09:04:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:03 smithi081 ceph-mon[30961]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T09:04:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:03 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] up:active 2024-04-24T09:04:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:03 smithi081 ceph-mon[30961]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:04:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:03 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:04:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:03 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3689355752' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-24T09:04:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:03 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3689355752' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-24T09:04:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:03 smithi081 ceph-mon[30961]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T09:04:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:03 smithi081 ceph-mon[30961]: pgmap v366: 97 pgs: 18 creating+peering, 20 active+clean, 59 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:04:03.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:03 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/703406488' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T09:04:03.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:03 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3561743237' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T09:04:03.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:03 smithi069 ceph-mon[28170]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T09:04:03.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:03 smithi069 ceph-mon[28170]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T09:04:03.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:03 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] up:active 2024-04-24T09:04:03.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:03 smithi069 ceph-mon[28170]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:04:03.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:03 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:04:03.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:03 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3689355752' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-24T09:04:03.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:03 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3689355752' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-24T09:04:03.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:03 smithi069 ceph-mon[28170]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T09:04:03.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:03 smithi069 ceph-mon[28170]: pgmap v366: 97 pgs: 18 creating+peering, 20 active+clean, 59 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:04:03.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:03 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/703406488' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T09:04:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:03 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3561743237' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T09:04:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:03 smithi130 ceph-mon[31553]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T09:04:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:03 smithi130 ceph-mon[31553]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T09:04:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:03 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] up:active 2024-04-24T09:04:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:03 smithi130 ceph-mon[31553]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:04:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:03 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:04:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:03 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3689355752' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-24T09:04:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:03 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3689355752' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-24T09:04:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:03 smithi130 ceph-mon[31553]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T09:04:03.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:03 smithi130 ceph-mon[31553]: pgmap v366: 97 pgs: 18 creating+peering, 20 active+clean, 59 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:04:03.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:03 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/703406488' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T09:04:04.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:04 smithi081 ceph-mon[30961]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T09:04:04.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:04 smithi081 ceph-mon[30961]: Cluster is now healthy 2024-04-24T09:04:04.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:04 smithi081 ceph-mon[30961]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T09:04:04.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:04 smithi069 ceph-mon[28170]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T09:04:04.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:04 smithi069 ceph-mon[28170]: Cluster is now healthy 2024-04-24T09:04:04.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:04 smithi069 ceph-mon[28170]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T09:04:04.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:04 smithi130 ceph-mon[31553]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T09:04:04.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:04 smithi130 ceph-mon[31553]: Cluster is now healthy 2024-04-24T09:04:04.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:04 smithi130 ceph-mon[31553]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T09:04:05.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.081+0000 7f0fbea00640 1 -- 172.21.15.69:0/703406488 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7f0fb409ce10 con 0x7f0fc8102fd0 2024-04-24T09:04:05.082 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data_ec' created 2024-04-24T09:04:05.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.087+0000 7f0fc7200640 1 -- 172.21.15.69:0/703406488 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0fac0a3c30 msgr2=0x7f0fac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:05.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.087+0000 7f0fc7200640 1 --2- 172.21.15.69:0/703406488 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0fac0a3c30 0x7f0fac0a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8000c00 tx=0x7f0fb80314a0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.087+0000 7f0fc7200640 1 -- 172.21.15.69:0/703406488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fc8102fd0 msgr2=0x7f0fc819a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:05.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.087+0000 7f0fc7200640 1 --2- 172.21.15.69:0/703406488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fc8102fd0 0x7f0fc819a7b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400c970 tx=0x7f0fb400ce30 comp rx=0 tx=0).stop 2024-04-24T09:04:05.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.087+0000 7f0fc7200640 1 -- 172.21.15.69:0/703406488 shutdown_connections 2024-04-24T09:04:05.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.087+0000 7f0fc7200640 1 --2- 172.21.15.69:0/703406488 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0fc8103990 0x7f0fc81a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.087+0000 7f0fc7200640 1 --2- 172.21.15.69:0/703406488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fc8102fd0 0x7f0fc819a7b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.087+0000 7f0fc7200640 1 --2- 172.21.15.69:0/703406488 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0fac0a3c30 0x7f0fac0a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.087+0000 7f0fc7200640 1 --2- 172.21.15.69:0/703406488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fc8101e80 0x7f0fc819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.087+0000 7f0fc7200640 1 -- 172.21.15.69:0/703406488 >> 172.21.15.69:0/703406488 conn(0x7f0fc80fd410 msgr2=0x7f0fc8074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:05.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.088+0000 7f0fc7200640 1 -- 172.21.15.69:0/703406488 shutdown_connections 2024-04-24T09:04:05.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.088+0000 7f0fc7200640 1 -- 172.21.15.69:0/703406488 wait complete. 2024-04-24T09:04:05.117 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-24T09:04:05.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.301+0000 7f04d1e00640 1 -- 172.21.15.69:0/3183860431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04cc105e30 msgr2=0x7f04cc10c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:05.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.301+0000 7f04d1e00640 1 --2- 172.21.15.69:0/3183860431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04cc105e30 0x7f04cc10c6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009be0 tx=0x7f04c402a6d0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.301+0000 7f04d1e00640 1 -- 172.21.15.69:0/3183860431 shutdown_connections 2024-04-24T09:04:05.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.301+0000 7f04d1e00640 1 --2- 172.21.15.69:0/3183860431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04cc105e30 0x7f04cc10c6c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.301+0000 7f04d1e00640 1 --2- 172.21.15.69:0/3183860431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04cc105470 0x7f04cc1058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.301+0000 7f04d1e00640 1 --2- 172.21.15.69:0/3183860431 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04cc104280 0x7f04cc1046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.301+0000 7f04d1e00640 1 -- 172.21.15.69:0/3183860431 >> 172.21.15.69:0/3183860431 conn(0x7f04cc0ff7f0 msgr2=0x7f04cc101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:05.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.301+0000 7f04d1e00640 1 -- 172.21.15.69:0/3183860431 shutdown_connections 2024-04-24T09:04:05.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.301+0000 7f04d1e00640 1 -- 172.21.15.69:0/3183860431 wait complete. 2024-04-24T09:04:05.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.302+0000 7f04d1e00640 1 Processor -- start 2024-04-24T09:04:05.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.303+0000 7f04d1e00640 1 -- start start 2024-04-24T09:04:05.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.303+0000 7f04d1e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04cc104280 0x7f04cc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.303+0000 7f04d1e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04cc105470 0x7f04cc19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.303+0000 7f04d1e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04cc105e30 0x7f04cc1a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.303+0000 7f04d1e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc10fdc0 con 0x7f04cc105e30 2024-04-24T09:04:05.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.303+0000 7f04d1e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc10fc40 con 0x7f04cc104280 2024-04-24T09:04:05.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.303+0000 7f04d1e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc10ff40 con 0x7f04cc105470 2024-04-24T09:04:05.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.303+0000 7f04cb400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04cc105e30 0x7f04cc1a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.303+0000 7f04caa00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04cc104280 0x7f04cc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.303+0000 7f04caa00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04cc104280 0x7f04cc19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.69:36636/0 (socket says 172.21.15.69:36636) 2024-04-24T09:04:05.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.303+0000 7f04caa00640 1 -- 172.21.15.69:0/3270523954 learned_addr learned my addr 172.21.15.69:0/3270523954 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:05.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.303+0000 7f04ca000640 1 --2- 172.21.15.69:0/3270523954 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04cc105470 0x7f04cc19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.304+0000 7f04caa00640 1 -- 172.21.15.69:0/3270523954 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04cc105470 msgr2=0x7f04cc19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:05.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.304+0000 7f04caa00640 1 --2- 172.21.15.69:0/3270523954 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04cc105470 0x7f04cc19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.304+0000 7f04caa00640 1 -- 172.21.15.69:0/3270523954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04cc105e30 msgr2=0x7f04cc1a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:05.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.304+0000 7f04caa00640 1 --2- 172.21.15.69:0/3270523954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04cc105e30 0x7f04cc1a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.304+0000 7f04caa00640 1 -- 172.21.15.69:0/3270523954 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04cc1a2060 con 0x7f04cc104280 2024-04-24T09:04:05.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.304+0000 7f04ca000640 1 --2- 172.21.15.69:0/3270523954 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04cc105470 0x7f04cc19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:04:05.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.304+0000 7f04caa00640 1 --2- 172.21.15.69:0/3270523954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04cc104280 0x7f04cc19a440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00e9c0 tx=0x7f04bc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:05.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.304+0000 7f04d0a00640 1 -- 172.21.15.69:0/3270523954 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc00cd80 con 0x7f04cc104280 2024-04-24T09:04:05.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.304+0000 7f04d1e00640 1 -- 172.21.15.69:0/3270523954 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04cc1a2350 con 0x7f04cc104280 2024-04-24T09:04:05.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.304+0000 7f04d0a00640 1 -- 172.21.15.69:0/3270523954 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04bc0035f0 con 0x7f04cc104280 2024-04-24T09:04:05.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.304+0000 7f04d0a00640 1 -- 172.21.15.69:0/3270523954 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc010690 con 0x7f04cc104280 2024-04-24T09:04:05.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.305+0000 7f04d1e00640 1 -- 172.21.15.69:0/3270523954 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04cc1a28f0 con 0x7f04cc104280 2024-04-24T09:04:05.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.306+0000 7f04d1e00640 1 -- 172.21.15.69:0/3270523954 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04cc10b670 con 0x7f04cc104280 2024-04-24T09:04:05.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.308+0000 7f04d0a00640 1 -- 172.21.15.69:0/3270523954 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f04bc0037a0 con 0x7f04cc104280 2024-04-24T09:04:05.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.310+0000 7f04d0a00640 1 --2- 172.21.15.69:0/3270523954 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f04b00a3bb0 0x7f04b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.310+0000 7f04d0a00640 1 -- 172.21.15.69:0/3270523954 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f04bc0d0b30 con 0x7f04cc104280 2024-04-24T09:04:05.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.310+0000 7f04ca000640 1 --2- 172.21.15.69:0/3270523954 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f04b00a3bb0 0x7f04b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.311+0000 7f04ca000640 1 --2- 172.21.15.69:0/3270523954 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f04b00a3bb0 0x7f04b00a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f04cc19b950 tx=0x7f04b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:05.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.317+0000 7f04d0a00640 1 -- 172.21.15.69:0/3270523954 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04bc09d290 con 0x7f04cc104280 2024-04-24T09:04:05.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:05.546+0000 7f04d1e00640 1 -- 172.21.15.69:0/3270523954 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f04cc02d620 con 0x7f04cc104280 2024-04-24T09:04:06.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.106+0000 7f04d0a00640 1 -- 172.21.15.69:0/3270523954 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7f04bc09cb90 con 0x7f04cc104280 2024-04-24T09:04:06.106 INFO:teuthology.orchestra.run.smithi069.stderr:set pool 4 allow_ec_overwrites to true 2024-04-24T09:04:06.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.111+0000 7f04d1e00640 1 -- 172.21.15.69:0/3270523954 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f04b00a3bb0 msgr2=0x7f04b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.111+0000 7f04d1e00640 1 --2- 172.21.15.69:0/3270523954 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f04b00a3bb0 0x7f04b00a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f04cc19b950 tx=0x7f04b8009260 comp rx=0 tx=0).stop 2024-04-24T09:04:06.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.111+0000 7f04d1e00640 1 -- 172.21.15.69:0/3270523954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04cc104280 msgr2=0x7f04cc19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.111+0000 7f04d1e00640 1 --2- 172.21.15.69:0/3270523954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04cc104280 0x7f04cc19a440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00e9c0 tx=0x7f04bc00ee80 comp rx=0 tx=0).stop 2024-04-24T09:04:06.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.114+0000 7f04d1e00640 1 -- 172.21.15.69:0/3270523954 shutdown_connections 2024-04-24T09:04:06.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.114+0000 7f04d1e00640 1 --2- 172.21.15.69:0/3270523954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04cc105e30 0x7f04cc1a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.114+0000 7f04d1e00640 1 --2- 172.21.15.69:0/3270523954 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04cc105470 0x7f04cc19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.114+0000 7f04d1e00640 1 --2- 172.21.15.69:0/3270523954 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f04b00a3bb0 0x7f04b00a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.114+0000 7f04d1e00640 1 --2- 172.21.15.69:0/3270523954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04cc104280 0x7f04cc19a440 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.114+0000 7f04d1e00640 1 -- 172.21.15.69:0/3270523954 >> 172.21.15.69:0/3270523954 conn(0x7f04cc0ff7f0 msgr2=0x7f04cc108840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:06.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.116+0000 7f04d1e00640 1 -- 172.21.15.69:0/3270523954 shutdown_connections 2024-04-24T09:04:06.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.116+0000 7f04d1e00640 1 -- 172.21.15.69:0/3270523954 wait complete. 2024-04-24T09:04:06.152 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-24T09:04:06.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.327+0000 7f9f35600640 1 -- 172.21.15.69:0/34154127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f30103a20 msgr2=0x7f9f3010a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.327+0000 7f9f35600640 1 --2- 172.21.15.69:0/34154127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f30103a20 0x7f9f3010a340 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9f28009ca0 tx=0x7f9f2802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.327+0000 7f9f35600640 1 -- 172.21.15.69:0/34154127 shutdown_connections 2024-04-24T09:04:06.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.327+0000 7f9f35600640 1 --2- 172.21.15.69:0/34154127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f30103a20 0x7f9f3010a340 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.327+0000 7f9f35600640 1 --2- 172.21.15.69:0/34154127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f30103060 0x7f9f301034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.327+0000 7f9f35600640 1 --2- 172.21.15.69:0/34154127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f30101f10 0x7f9f30102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.327+0000 7f9f35600640 1 -- 172.21.15.69:0/34154127 >> 172.21.15.69:0/34154127 conn(0x7f9f300fd4a0 msgr2=0x7f9f300ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:06.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.327+0000 7f9f35600640 1 -- 172.21.15.69:0/34154127 shutdown_connections 2024-04-24T09:04:06.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.328+0000 7f9f35600640 1 -- 172.21.15.69:0/34154127 wait complete. 2024-04-24T09:04:06.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.328+0000 7f9f35600640 1 Processor -- start 2024-04-24T09:04:06.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.329+0000 7f9f35600640 1 -- start start 2024-04-24T09:04:06.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.329+0000 7f9f35600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f30101f10 0x7f9f3019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.329+0000 7f9f35600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f30103060 0x7f9f3019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.329+0000 7f9f35600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f30103a20 0x7f9f301a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.329+0000 7f9f35600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3010db10 con 0x7f9f30103a20 2024-04-24T09:04:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.329+0000 7f9f35600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3010d990 con 0x7f9f30103060 2024-04-24T09:04:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.329+0000 7f9f35600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3010dc90 con 0x7f9f30101f10 2024-04-24T09:04:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.329+0000 7f9f34c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f30103a20 0x7f9f301a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.329+0000 7f9f34c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f30103a20 0x7f9f301a1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50512/0 (socket says 172.21.15.69:50512) 2024-04-24T09:04:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.329+0000 7f9f34c00640 1 -- 172.21.15.69:0/2964643627 learned_addr learned my addr 172.21.15.69:0/2964643627 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.329+0000 7f9f2f400640 1 --2- 172.21.15.69:0/2964643627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f30103060 0x7f9f3019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.329+0000 7f9f34c00640 1 -- 172.21.15.69:0/2964643627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f30101f10 msgr2=0x7f9f3019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.329+0000 7f9f34c00640 1 --2- 172.21.15.69:0/2964643627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f30101f10 0x7f9f3019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.329+0000 7f9f34c00640 1 -- 172.21.15.69:0/2964643627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f30103060 msgr2=0x7f9f3019a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.330+0000 7f9f34c00640 1 --2- 172.21.15.69:0/2964643627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f30103060 0x7f9f3019a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.330+0000 7f9f34c00640 1 -- 172.21.15.69:0/2964643627 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f301a1fe0 con 0x7f9f30103a20 2024-04-24T09:04:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.330+0000 7f9f2f400640 1 --2- 172.21.15.69:0/2964643627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f30103060 0x7f9f3019a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:04:06.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.330+0000 7f9f34c00640 1 --2- 172.21.15.69:0/2964643627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f30103a20 0x7f9f301a1970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f28031040 tx=0x7f9f28003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:06.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.330+0000 7f9f2cc00640 1 -- 172.21.15.69:0/2964643627 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f28004040 con 0x7f9f30103a20 2024-04-24T09:04:06.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.330+0000 7f9f2cc00640 1 -- 172.21.15.69:0/2964643627 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f28004690 con 0x7f9f30103a20 2024-04-24T09:04:06.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.330+0000 7f9f35600640 1 -- 172.21.15.69:0/2964643627 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f301a2270 con 0x7f9f30103a20 2024-04-24T09:04:06.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.330+0000 7f9f2cc00640 1 -- 172.21.15.69:0/2964643627 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2803b410 con 0x7f9f30103a20 2024-04-24T09:04:06.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.330+0000 7f9f35600640 1 -- 172.21.15.69:0/2964643627 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f301a27c0 con 0x7f9f30103a20 2024-04-24T09:04:06.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.331+0000 7f9f35600640 1 -- 172.21.15.69:0/2964643627 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f301a2a20 con 0x7f9f30103a20 2024-04-24T09:04:06.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.333+0000 7f9f2cc00640 1 -- 172.21.15.69:0/2964643627 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9f280306e0 con 0x7f9f30103a20 2024-04-24T09:04:06.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.335+0000 7f9f2cc00640 1 --2- 172.21.15.69:0/2964643627 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9f140a3bb0 0x7f9f140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.335+0000 7f9f2cc00640 1 -- 172.21.15.69:0/2964643627 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f9f280f1f10 con 0x7f9f30103a20 2024-04-24T09:04:06.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.335+0000 7f9f2fe00640 1 --2- 172.21.15.69:0/2964643627 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9f140a3bb0 0x7f9f140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.336+0000 7f9f2fe00640 1 --2- 172.21.15.69:0/2964643627 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9f140a3bb0 0x7f9f140a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009710 tx=0x7f9f20006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:06.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.340+0000 7f9f2cc00640 1 -- 172.21.15.69:0/2964643627 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f280be6d0 con 0x7f9f30103a20 2024-04-24T09:04:06.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:06 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/703406488' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T09:04:06.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:06 smithi130 ceph-mon[31553]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T09:04:06.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:06 smithi130 ceph-mon[31553]: pgmap v369: 129 pgs: 19 creating+peering, 64 active+clean, 46 unknown; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 3 op/s 2024-04-24T09:04:06.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:06 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3270523954' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T09:04:06.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:06 smithi130 ceph-mon[31553]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T09:04:06.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.455 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:04:06.455 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:04:06.455 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:06 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/703406488' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T09:04:06.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:06 smithi081 ceph-mon[30961]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T09:04:06.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:06 smithi081 ceph-mon[30961]: pgmap v369: 129 pgs: 19 creating+peering, 64 active+clean, 46 unknown; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 3 op/s 2024-04-24T09:04:06.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:06 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3270523954' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T09:04:06.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:06 smithi081 ceph-mon[30961]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T09:04:06.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:04:06.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:04:06.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:06.588+0000 7f9f35600640 1 -- 172.21.15.69:0/2964643627 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f9f301a2d70 con 0x7f9f30103a20 2024-04-24T09:04:06.590 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:06 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/703406488' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T09:04:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:06 smithi069 ceph-mon[28170]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T09:04:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:06 smithi069 ceph-mon[28170]: pgmap v369: 129 pgs: 19 creating+peering, 64 active+clean, 46 unknown; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 3 op/s 2024-04-24T09:04:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:06 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3270523954' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T09:04:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:06 smithi069 ceph-mon[28170]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T09:04:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:04:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:04:06.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:04:07.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.098+0000 7f9f2cc00640 1 -- 172.21.15.69:0/2964643627 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7f9f280bdfd0 con 0x7f9f30103a20 2024-04-24T09:04:07.099 INFO:teuthology.orchestra.run.smithi069.stderr:added data pool 4 to fsmap 2024-04-24T09:04:07.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.102+0000 7f9f35600640 1 -- 172.21.15.69:0/2964643627 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9f140a3bb0 msgr2=0x7f9f140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.102+0000 7f9f35600640 1 --2- 172.21.15.69:0/2964643627 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9f140a3bb0 0x7f9f140a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009710 tx=0x7f9f20006c90 comp rx=0 tx=0).stop 2024-04-24T09:04:07.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.102+0000 7f9f35600640 1 -- 172.21.15.69:0/2964643627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f30103a20 msgr2=0x7f9f301a1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.102+0000 7f9f35600640 1 --2- 172.21.15.69:0/2964643627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f30103a20 0x7f9f301a1970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f28031040 tx=0x7f9f28003c10 comp rx=0 tx=0).stop 2024-04-24T09:04:07.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.103+0000 7f9f35600640 1 -- 172.21.15.69:0/2964643627 shutdown_connections 2024-04-24T09:04:07.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.103+0000 7f9f35600640 1 --2- 172.21.15.69:0/2964643627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f30103a20 0x7f9f301a1970 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.103+0000 7f9f35600640 1 --2- 172.21.15.69:0/2964643627 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9f140a3bb0 0x7f9f140a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.103+0000 7f9f35600640 1 --2- 172.21.15.69:0/2964643627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f30103060 0x7f9f3019a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.103+0000 7f9f35600640 1 --2- 172.21.15.69:0/2964643627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f30101f10 0x7f9f3019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.103+0000 7f9f35600640 1 -- 172.21.15.69:0/2964643627 >> 172.21.15.69:0/2964643627 conn(0x7f9f300fd4a0 msgr2=0x7f9f30074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.103+0000 7f9f35600640 1 -- 172.21.15.69:0/2964643627 shutdown_connections 2024-04-24T09:04:07.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.103+0000 7f9f35600640 1 -- 172.21.15.69:0/2964643627 wait complete. 2024-04-24T09:04:07.128 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:04:07.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.320+0000 7f4967e00640 1 -- 172.21.15.69:0/1522977870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f496010bc30 msgr2=0x7f496010e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.320+0000 7f4967e00640 1 --2- 172.21.15.69:0/1522977870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f496010bc30 0x7f496010e060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f494c009900 tx=0x7f494c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.320+0000 7f4967e00640 1 -- 172.21.15.69:0/1522977870 shutdown_connections 2024-04-24T09:04:07.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.320+0000 7f4967e00640 1 --2- 172.21.15.69:0/1522977870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f496010e6f0 0x7f4960110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.320+0000 7f4967e00640 1 --2- 172.21.15.69:0/1522977870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f496010bc30 0x7f496010e060 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.320+0000 7f4967e00640 1 --2- 172.21.15.69:0/1522977870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4960109270 0x7f496010b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.320+0000 7f4967e00640 1 -- 172.21.15.69:0/1522977870 >> 172.21.15.69:0/1522977870 conn(0x7f496006f240 msgr2=0x7f4960071660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.320+0000 7f4967e00640 1 -- 172.21.15.69:0/1522977870 shutdown_connections 2024-04-24T09:04:07.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.320+0000 7f4967e00640 1 -- 172.21.15.69:0/1522977870 wait complete. 2024-04-24T09:04:07.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.321+0000 7f4967e00640 1 Processor -- start 2024-04-24T09:04:07.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.321+0000 7f4967e00640 1 -- start start 2024-04-24T09:04:07.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.321+0000 7f4967e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4960109270 0x7f49601147c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.321+0000 7f4967e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f496010bc30 0x7f4960112e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.321+0000 7f4967e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f496010e6f0 0x7f4960113340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.321+0000 7f4967e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4960116490 con 0x7f496010e6f0 2024-04-24T09:04:07.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.321+0000 7f4967e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4960116310 con 0x7f496010bc30 2024-04-24T09:04:07.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.321+0000 7f4967e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4960116610 con 0x7f4960109270 2024-04-24T09:04:07.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.322+0000 7f4965e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f496010e6f0 0x7f4960113340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.322+0000 7f4965e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f496010e6f0 0x7f4960113340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50524/0 (socket says 172.21.15.69:50524) 2024-04-24T09:04:07.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.322+0000 7f4965e00640 1 -- 172.21.15.69:0/1384836302 learned_addr learned my addr 172.21.15.69:0/1384836302 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:07.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.322+0000 7f4965400640 1 --2- 172.21.15.69:0/1384836302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4960109270 0x7f49601147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.322+0000 7f4964a00640 1 --2- 172.21.15.69:0/1384836302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f496010bc30 0x7f4960112e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.322+0000 7f4965e00640 1 -- 172.21.15.69:0/1384836302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4960109270 msgr2=0x7f49601147c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.322+0000 7f4965e00640 1 --2- 172.21.15.69:0/1384836302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4960109270 0x7f49601147c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.322+0000 7f4965e00640 1 -- 172.21.15.69:0/1384836302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f496010bc30 msgr2=0x7f4960112e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.322+0000 7f4965e00640 1 --2- 172.21.15.69:0/1384836302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f496010bc30 0x7f4960112e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.322+0000 7f4965e00640 1 -- 172.21.15.69:0/1384836302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4960113880 con 0x7f496010e6f0 2024-04-24T09:04:07.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.322+0000 7f4964a00640 1 --2- 172.21.15.69:0/1384836302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f496010bc30 0x7f4960112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:07.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.322+0000 7f4965e00640 1 --2- 172.21.15.69:0/1384836302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f496010e6f0 0x7f4960113340 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f495800b970 tx=0x7f495800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.323+0000 7f495e000640 1 -- 172.21.15.69:0/1384836302 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958019070 con 0x7f496010e6f0 2024-04-24T09:04:07.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.323+0000 7f495e000640 1 -- 172.21.15.69:0/1384836302 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4958014440 con 0x7f496010e6f0 2024-04-24T09:04:07.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.323+0000 7f4967e00640 1 -- 172.21.15.69:0/1384836302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4960113b70 con 0x7f496010e6f0 2024-04-24T09:04:07.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.323+0000 7f495e000640 1 -- 172.21.15.69:0/1384836302 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495801d5d0 con 0x7f496010e6f0 2024-04-24T09:04:07.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.323+0000 7f4967e00640 1 -- 172.21.15.69:0/1384836302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49601af710 con 0x7f496010e6f0 2024-04-24T09:04:07.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.324+0000 7f4967e00640 1 -- 172.21.15.69:0/1384836302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4960070d30 con 0x7f496010e6f0 2024-04-24T09:04:07.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.326+0000 7f495e000640 1 -- 172.21.15.69:0/1384836302 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f495801c070 con 0x7f496010e6f0 2024-04-24T09:04:07.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.327+0000 7f495e000640 1 --2- 172.21.15.69:0/1384836302 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f49480a3bb0 0x7f49480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.327+0000 7f495e000640 1 -- 172.21.15.69:0/1384836302 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f49580cfc60 con 0x7f496010e6f0 2024-04-24T09:04:07.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.328+0000 7f4965400640 1 --2- 172.21.15.69:0/1384836302 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f49480a3bb0 0x7f49480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.328+0000 7f4965400640 1 --2- 172.21.15.69:0/1384836302 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f49480a3bb0 0x7f49480a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4954009890 tx=0x7f4954008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.332+0000 7f495e000640 1 -- 172.21.15.69:0/1384836302 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f495809c3a0 con 0x7f496010e6f0 2024-04-24T09:04:07.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:07 smithi130 ceph-mon[31553]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T09:04:07.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:07 smithi130 ceph-mon[31553]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-24T09:04:07.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:07 smithi130 ceph-mon[31553]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T09:04:07.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:07 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2964643627' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-24T09:04:07.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:07 smithi081 ceph-mon[30961]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T09:04:07.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:07 smithi081 ceph-mon[30961]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-24T09:04:07.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:07 smithi081 ceph-mon[30961]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T09:04:07.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:07 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2964643627' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-24T09:04:07.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.575+0000 7f4967e00640 1 -- 172.21.15.69:0/1384836302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f496002d620 con 0x7f496010e6f0 2024-04-24T09:04:07.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.576+0000 7f495e000640 1 -- 172.21.15.69:0/1384836302 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11800 (secure 0 0 0) 0x7f495809bca0 con 0x7f496010e6f0 2024-04-24T09:04:07.576 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:07.576 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34460,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34496,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T09:04:01.991559+0000","modified":"2024-04-24T09:04:06.590685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34532":{"gid":34532,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34532,"qdb_cluster":[34532]},"id":1}]} 2024-04-24T09:04:07.577 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 16 2024-04-24T09:04:07.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.580+0000 7f4967e00640 1 -- 172.21.15.69:0/1384836302 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f49480a3bb0 msgr2=0x7f49480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.580+0000 7f4967e00640 1 --2- 172.21.15.69:0/1384836302 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f49480a3bb0 0x7f49480a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4954009890 tx=0x7f4954008040 comp rx=0 tx=0).stop 2024-04-24T09:04:07.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.580+0000 7f4967e00640 1 -- 172.21.15.69:0/1384836302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f496010e6f0 msgr2=0x7f4960113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.580+0000 7f4967e00640 1 --2- 172.21.15.69:0/1384836302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f496010e6f0 0x7f4960113340 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f495800b970 tx=0x7f495800be30 comp rx=0 tx=0).stop 2024-04-24T09:04:07.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.580+0000 7f4967e00640 1 -- 172.21.15.69:0/1384836302 shutdown_connections 2024-04-24T09:04:07.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.580+0000 7f4967e00640 1 --2- 172.21.15.69:0/1384836302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f496010e6f0 0x7f4960113340 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.580+0000 7f4967e00640 1 --2- 172.21.15.69:0/1384836302 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f49480a3bb0 0x7f49480a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.580+0000 7f4967e00640 1 --2- 172.21.15.69:0/1384836302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f496010bc30 0x7f4960112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.580+0000 7f4967e00640 1 --2- 172.21.15.69:0/1384836302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4960109270 0x7f49601147c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.580+0000 7f4967e00640 1 -- 172.21.15.69:0/1384836302 >> 172.21.15.69:0/1384836302 conn(0x7f496006f240 msgr2=0x7f4960109dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.581+0000 7f4967e00640 1 -- 172.21.15.69:0/1384836302 shutdown_connections 2024-04-24T09:04:07.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.581+0000 7f4967e00640 1 -- 172.21.15.69:0/1384836302 wait complete. 2024-04-24T09:04:07.590 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:07 smithi069 ceph-mon[28170]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T09:04:07.590 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:07 smithi069 ceph-mon[28170]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-24T09:04:07.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:07 smithi069 ceph-mon[28170]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T09:04:07.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:07 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2964643627' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-24T09:04:07.600 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:04:07.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.795+0000 7f723a200640 1 -- 172.21.15.69:0/232016172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7234108480 msgr2=0x7f72341088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.795+0000 7f723a200640 1 --2- 172.21.15.69:0/232016172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7234108480 0x7f72341088a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7224009900 tx=0x7f722402a410 comp rx=0 tx=0).stop 2024-04-24T09:04:07.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.795+0000 7f723a200640 1 -- 172.21.15.69:0/232016172 shutdown_connections 2024-04-24T09:04:07.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.795+0000 7f723a200640 1 --2- 172.21.15.69:0/232016172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f723410a030 0x7f72341108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.795+0000 7f723a200640 1 --2- 172.21.15.69:0/232016172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7234109670 0x7f7234109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.795+0000 7f723a200640 1 --2- 172.21.15.69:0/232016172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7234108480 0x7f72341088a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.795+0000 7f723a200640 1 -- 172.21.15.69:0/232016172 >> 172.21.15.69:0/232016172 conn(0x7f7234077440 msgr2=0x7f7234079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.795+0000 7f723a200640 1 -- 172.21.15.69:0/232016172 shutdown_connections 2024-04-24T09:04:07.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.795+0000 7f723a200640 1 -- 172.21.15.69:0/232016172 wait complete. 2024-04-24T09:04:07.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.796+0000 7f723a200640 1 Processor -- start 2024-04-24T09:04:07.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.796+0000 7f723a200640 1 -- start start 2024-04-24T09:04:07.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.797+0000 7f723a200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7234108480 0x7f723419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.797+0000 7f723a200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7234109670 0x7f723419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.797+0000 7f723a200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f723410a030 0x7f72341a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.797+0000 7f723a200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7234113fc0 con 0x7f7234108480 2024-04-24T09:04:07.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.797+0000 7f723a200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7234113e40 con 0x7f7234109670 2024-04-24T09:04:07.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.797+0000 7f723a200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7234114140 con 0x7f723410a030 2024-04-24T09:04:07.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.797+0000 7f7238e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7234108480 0x7f723419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.797+0000 7f7238e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7234108480 0x7f723419e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50534/0 (socket says 172.21.15.69:50534) 2024-04-24T09:04:07.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.797+0000 7f7238e00640 1 -- 172.21.15.69:0/4052962215 learned_addr learned my addr 172.21.15.69:0/4052962215 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:07.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.797+0000 7f722be00640 1 --2- 172.21.15.69:0/4052962215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7234109670 0x7f723419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.797+0000 7f7238e00640 1 -- 172.21.15.69:0/4052962215 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f723410a030 msgr2=0x7f72341a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.797+0000 7f7238e00640 1 --2- 172.21.15.69:0/4052962215 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f723410a030 0x7f72341a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.797+0000 7f7238e00640 1 -- 172.21.15.69:0/4052962215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7234109670 msgr2=0x7f723419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.797+0000 7f7238e00640 1 --2- 172.21.15.69:0/4052962215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7234109670 0x7f723419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.797+0000 7f7238e00640 1 -- 172.21.15.69:0/4052962215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72341a62e0 con 0x7f7234108480 2024-04-24T09:04:07.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.797+0000 7f722be00640 1 --2- 172.21.15.69:0/4052962215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7234109670 0x7f723419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:07.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.798+0000 7f7238e00640 1 --2- 172.21.15.69:0/4052962215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7234108480 0x7f723419e6c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72240314e0 tx=0x7f72240042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.798+0000 7f7232000640 1 -- 172.21.15.69:0/4052962215 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722403f030 con 0x7f7234108480 2024-04-24T09:04:07.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.798+0000 7f7232000640 1 -- 172.21.15.69:0/4052962215 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7224003cb0 con 0x7f7234108480 2024-04-24T09:04:07.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.798+0000 7f723a200640 1 -- 172.21.15.69:0/4052962215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72341a6570 con 0x7f7234108480 2024-04-24T09:04:07.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.798+0000 7f7232000640 1 -- 172.21.15.69:0/4052962215 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224035070 con 0x7f7234108480 2024-04-24T09:04:07.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.798+0000 7f723a200640 1 -- 172.21.15.69:0/4052962215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72341a6ab0 con 0x7f7234108480 2024-04-24T09:04:07.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.799+0000 7f723a200640 1 -- 172.21.15.69:0/4052962215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f723410f870 con 0x7f7234108480 2024-04-24T09:04:07.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.801+0000 7f7232000640 1 -- 172.21.15.69:0/4052962215 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7224042030 con 0x7f7234108480 2024-04-24T09:04:07.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.802+0000 7f7232000640 1 --2- 172.21.15.69:0/4052962215 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f72180a3c30 0x7f72180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.803+0000 7f7232000640 1 -- 172.21.15.69:0/4052962215 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f72240f0ce0 con 0x7f7234108480 2024-04-24T09:04:07.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.803+0000 7f722be00640 1 --2- 172.21.15.69:0/4052962215 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f72180a3c30 0x7f72180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.803+0000 7f722be00640 1 --2- 172.21.15.69:0/4052962215 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f72180a3c30 0x7f72180a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f723419fbd0 tx=0x7f721c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:07.807+0000 7f7232000640 1 -- 172.21.15.69:0/4052962215 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f72240bd420 con 0x7f7234108480 2024-04-24T09:04:08.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.009+0000 7f723a200640 1 -- 172.21.15.69:0/4052962215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7234062ca0 con 0x7f7234108480 2024-04-24T09:04:08.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.014+0000 7f7232000640 1 -- 172.21.15.69:0/4052962215 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24094 (secure 0 0 0) 0x7f72240bcd20 con 0x7f7234108480 2024-04-24T09:04:08.014 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:08.014 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":90,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:06.590427+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:04:08.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.016+0000 7f723a200640 1 -- 172.21.15.69:0/4052962215 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f72180a3c30 msgr2=0x7f72180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.016+0000 7f723a200640 1 --2- 172.21.15.69:0/4052962215 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f72180a3c30 0x7f72180a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f723419fbd0 tx=0x7f721c005e50 comp rx=0 tx=0).stop 2024-04-24T09:04:08.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.016+0000 7f723a200640 1 -- 172.21.15.69:0/4052962215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7234108480 msgr2=0x7f723419e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.016+0000 7f723a200640 1 --2- 172.21.15.69:0/4052962215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7234108480 0x7f723419e6c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72240314e0 tx=0x7f72240042e0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.016+0000 7f723a200640 1 -- 172.21.15.69:0/4052962215 shutdown_connections 2024-04-24T09:04:08.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.016+0000 7f723a200640 1 --2- 172.21.15.69:0/4052962215 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f723410a030 0x7f72341a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.016+0000 7f723a200640 1 --2- 172.21.15.69:0/4052962215 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f72180a3c30 0x7f72180a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.016+0000 7f723a200640 1 --2- 172.21.15.69:0/4052962215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7234109670 0x7f723419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.016+0000 7f723a200640 1 --2- 172.21.15.69:0/4052962215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7234108480 0x7f723419e6c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.016+0000 7f723a200640 1 -- 172.21.15.69:0/4052962215 >> 172.21.15.69:0/4052962215 conn(0x7f7234077440 msgr2=0x7f723410ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:08.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.017+0000 7f723a200640 1 -- 172.21.15.69:0/4052962215 shutdown_connections 2024-04-24T09:04:08.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.017+0000 7f723a200640 1 -- 172.21.15.69:0/4052962215 wait complete. 2024-04-24T09:04:08.037 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:04:08.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.238+0000 7fad07200640 1 -- 172.21.15.69:0/1542340638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad08102900 msgr2=0x7fad08104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.238+0000 7fad07200640 1 --2- 172.21.15.69:0/1542340638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad08102900 0x7fad08104cf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7facf8009900 tx=0x7facf802a410 comp rx=0 tx=0).stop 2024-04-24T09:04:08.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.239+0000 7fad07200640 1 -- 172.21.15.69:0/1542340638 shutdown_connections 2024-04-24T09:04:08.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.239+0000 7fad07200640 1 --2- 172.21.15.69:0/1542340638 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad08107b60 0x7fad08109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.239+0000 7fad07200640 1 --2- 172.21.15.69:0/1542340638 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad08105230 0x7fad08107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.239+0000 7fad07200640 1 --2- 172.21.15.69:0/1542340638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad08102900 0x7fad08104cf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.239+0000 7fad07200640 1 -- 172.21.15.69:0/1542340638 >> 172.21.15.69:0/1542340638 conn(0x7fad080fc2d0 msgr2=0x7fad080fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:08.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.239+0000 7fad07200640 1 -- 172.21.15.69:0/1542340638 shutdown_connections 2024-04-24T09:04:08.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.239+0000 7fad07200640 1 -- 172.21.15.69:0/1542340638 wait complete. 2024-04-24T09:04:08.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.240+0000 7fad07200640 1 Processor -- start 2024-04-24T09:04:08.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.240+0000 7fad07200640 1 -- start start 2024-04-24T09:04:08.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.240+0000 7fad07200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad08102900 0x7fad0819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.240+0000 7fad07200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad08105230 0x7fad0819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.240+0000 7fad07200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad08107b60 0x7fad081a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.240+0000 7fad07200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fad0810da30 con 0x7fad08107b60 2024-04-24T09:04:08.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.240+0000 7fad07200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fad0810d8b0 con 0x7fad08102900 2024-04-24T09:04:08.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.240+0000 7fad07200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fad0810dbb0 con 0x7fad08105230 2024-04-24T09:04:08.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.240+0000 7fad06800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad08107b60 0x7fad081a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.240+0000 7fad06800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad08107b60 0x7fad081a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50556/0 (socket says 172.21.15.69:50556) 2024-04-24T09:04:08.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.241+0000 7fad06800640 1 -- 172.21.15.69:0/4149918737 learned_addr learned my addr 172.21.15.69:0/4149918737 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:08.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.241+0000 7fad05e00640 1 --2- 172.21.15.69:0/4149918737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad08102900 0x7fad0819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.241+0000 7fad06800640 1 -- 172.21.15.69:0/4149918737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad08105230 msgr2=0x7fad0819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.241+0000 7facfd400640 1 --2- 172.21.15.69:0/4149918737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad08105230 0x7fad0819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.241+0000 7fad06800640 1 --2- 172.21.15.69:0/4149918737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad08105230 0x7fad0819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.241+0000 7fad06800640 1 -- 172.21.15.69:0/4149918737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad08102900 msgr2=0x7fad0819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.241+0000 7fad06800640 1 --2- 172.21.15.69:0/4149918737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad08102900 0x7fad0819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.241+0000 7fad06800640 1 -- 172.21.15.69:0/4149918737 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad081a1f20 con 0x7fad08107b60 2024-04-24T09:04:08.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.241+0000 7fad05e00640 1 --2- 172.21.15.69:0/4149918737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad08102900 0x7fad0819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:08.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.241+0000 7facfd400640 1 --2- 172.21.15.69:0/4149918737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad08105230 0x7fad0819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:08.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.241+0000 7fad06800640 1 --2- 172.21.15.69:0/4149918737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad08107b60 0x7fad081a18b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fad0000cce0 tx=0x7fad0000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:08.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.241+0000 7facff400640 1 -- 172.21.15.69:0/4149918737 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0000f450 con 0x7fad08107b60 2024-04-24T09:04:08.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.242+0000 7fad07200640 1 -- 172.21.15.69:0/4149918737 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad081a2210 con 0x7fad08107b60 2024-04-24T09:04:08.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.242+0000 7facff400640 1 -- 172.21.15.69:0/4149918737 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad0000faa0 con 0x7fad08107b60 2024-04-24T09:04:08.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.242+0000 7facff400640 1 -- 172.21.15.69:0/4149918737 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00018b70 con 0x7fad08107b60 2024-04-24T09:04:08.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.242+0000 7fad07200640 1 -- 172.21.15.69:0/4149918737 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad081a2710 con 0x7fad08107b60 2024-04-24T09:04:08.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.243+0000 7fad07200640 1 -- 172.21.15.69:0/4149918737 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad081096b0 con 0x7fad08107b60 2024-04-24T09:04:08.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.245+0000 7facff400640 1 -- 172.21.15.69:0/4149918737 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fad00018d10 con 0x7fad08107b60 2024-04-24T09:04:08.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.247+0000 7facff400640 1 --2- 172.21.15.69:0/4149918737 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7facec0a3bb0 0x7facec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.247+0000 7facff400640 1 -- 172.21.15.69:0/4149918737 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fad000d0b90 con 0x7fad08107b60 2024-04-24T09:04:08.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.247+0000 7fad05e00640 1 --2- 172.21.15.69:0/4149918737 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7facec0a3bb0 0x7facec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.248+0000 7fad05e00640 1 --2- 172.21.15.69:0/4149918737 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7facec0a3bb0 0x7facec0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7facf802a930 tx=0x7facf8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:08.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.251+0000 7facff400640 1 -- 172.21.15.69:0/4149918737 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad00014030 con 0x7fad08107b60 2024-04-24T09:04:08.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:08 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2964643627' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-24T09:04:08.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:08 smithi069 ceph-mon[28170]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T09:04:08.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:08 smithi069 ceph-mon[28170]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:04:08.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:08 smithi069 ceph-mon[28170]: pgmap v372: 129 pgs: 12 creating+peering, 103 active+clean, 14 unknown; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-04-24T09:04:08.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:08 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1384836302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:04:08.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:08 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/4052962215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:08.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:08 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2964643627' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-24T09:04:08.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:08 smithi130 ceph-mon[31553]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T09:04:08.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:08 smithi130 ceph-mon[31553]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:04:08.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:08 smithi130 ceph-mon[31553]: pgmap v372: 129 pgs: 12 creating+peering, 103 active+clean, 14 unknown; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-04-24T09:04:08.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:08 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1384836302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:04:08.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:08 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/4052962215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:08.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.455+0000 7fad07200640 1 -- 172.21.15.69:0/4149918737 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad08062c00 con 0x7fad08107b60 2024-04-24T09:04:08.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.460+0000 7facff400640 1 -- 172.21.15.69:0/4149918737 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24094 (secure 0 0 0) 0x7fad0009d2d0 con 0x7fad08107b60 2024-04-24T09:04:08.460 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:08.461 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":90,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:06.590427+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:04:08.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.462+0000 7fad07200640 1 -- 172.21.15.69:0/4149918737 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7facec0a3bb0 msgr2=0x7facec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.462+0000 7fad07200640 1 --2- 172.21.15.69:0/4149918737 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7facec0a3bb0 0x7facec0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7facf802a930 tx=0x7facf8005fd0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.462+0000 7fad07200640 1 -- 172.21.15.69:0/4149918737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad08107b60 msgr2=0x7fad081a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.462+0000 7fad07200640 1 --2- 172.21.15.69:0/4149918737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad08107b60 0x7fad081a18b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fad0000cce0 tx=0x7fad0000a620 comp rx=0 tx=0).stop 2024-04-24T09:04:08.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.463+0000 7fad07200640 1 -- 172.21.15.69:0/4149918737 shutdown_connections 2024-04-24T09:04:08.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.463+0000 7fad07200640 1 --2- 172.21.15.69:0/4149918737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad08107b60 0x7fad081a18b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.463+0000 7fad07200640 1 --2- 172.21.15.69:0/4149918737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad08105230 0x7fad0819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.463+0000 7fad07200640 1 --2- 172.21.15.69:0/4149918737 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7facec0a3bb0 0x7facec0a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.463+0000 7fad07200640 1 --2- 172.21.15.69:0/4149918737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad08102900 0x7fad0819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.463+0000 7fad07200640 1 -- 172.21.15.69:0/4149918737 >> 172.21.15.69:0/4149918737 conn(0x7fad080fc2d0 msgr2=0x7fad08102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:08.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.463+0000 7fad07200640 1 -- 172.21.15.69:0/4149918737 shutdown_connections 2024-04-24T09:04:08.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.463+0000 7fad07200640 1 -- 172.21.15.69:0/4149918737 wait complete. 2024-04-24T09:04:08.498 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:04:08.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:08 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2964643627' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-24T09:04:08.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:08 smithi081 ceph-mon[30961]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T09:04:08.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:08 smithi081 ceph-mon[30961]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:04:08.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:08 smithi081 ceph-mon[30961]: pgmap v372: 129 pgs: 12 creating+peering, 103 active+clean, 14 unknown; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-04-24T09:04:08.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:08 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1384836302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:04:08.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:08 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/4052962215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.702+0000 7f3ac4a00640 1 -- 172.21.15.69:0/2967484960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0109670 msgr2=0x7f3ac0109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.702+0000 7f3ac4a00640 1 --2- 172.21.15.69:0/2967484960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0109670 0x7f3ac0109af0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009900 tx=0x7f3aa802a410 comp rx=0 tx=0).stop 2024-04-24T09:04:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.702+0000 7f3ac4a00640 1 -- 172.21.15.69:0/2967484960 shutdown_connections 2024-04-24T09:04:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.702+0000 7f3ac4a00640 1 --2- 172.21.15.69:0/2967484960 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ac010a030 0x7f3ac01108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.702+0000 7f3ac4a00640 1 --2- 172.21.15.69:0/2967484960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0109670 0x7f3ac0109af0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.702+0000 7f3ac4a00640 1 --2- 172.21.15.69:0/2967484960 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ac0108480 0x7f3ac01088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.702+0000 7f3ac4a00640 1 -- 172.21.15.69:0/2967484960 >> 172.21.15.69:0/2967484960 conn(0x7f3ac0077440 msgr2=0x7f3ac0079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.702+0000 7f3ac4a00640 1 -- 172.21.15.69:0/2967484960 shutdown_connections 2024-04-24T09:04:08.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.702+0000 7f3ac4a00640 1 -- 172.21.15.69:0/2967484960 wait complete. 2024-04-24T09:04:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.703+0000 7f3ac4a00640 1 Processor -- start 2024-04-24T09:04:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.703+0000 7f3ac4a00640 1 -- start start 2024-04-24T09:04:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.703+0000 7f3ac4a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0108480 0x7f3ac019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.703+0000 7f3ac4a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ac0109670 0x7f3ac019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.703+0000 7f3ac4a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ac010a030 0x7f3ac01a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.703+0000 7f3ac4a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac0113fc0 con 0x7f3ac0108480 2024-04-24T09:04:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.703+0000 7f3ac4a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac0113e40 con 0x7f3ac0109670 2024-04-24T09:04:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.703+0000 7f3ac4a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac0114140 con 0x7f3ac010a030 2024-04-24T09:04:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.704+0000 7f3abe800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0108480 0x7f3ac019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.704+0000 7f3abe800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0108480 0x7f3ac019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50568/0 (socket says 172.21.15.69:50568) 2024-04-24T09:04:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.704+0000 7f3abe800640 1 -- 172.21.15.69:0/1013592571 learned_addr learned my addr 172.21.15.69:0/1013592571 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.704+0000 7f3abf200640 1 --2- 172.21.15.69:0/1013592571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ac010a030 0x7f3ac01a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.704+0000 7f3ab5e00640 1 --2- 172.21.15.69:0/1013592571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ac0109670 0x7f3ac019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.704+0000 7f3abe800640 1 -- 172.21.15.69:0/1013592571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ac010a030 msgr2=0x7f3ac01a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.704+0000 7f3abe800640 1 --2- 172.21.15.69:0/1013592571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ac010a030 0x7f3ac01a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.704+0000 7f3abe800640 1 -- 172.21.15.69:0/1013592571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ac0109670 msgr2=0x7f3ac019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.704+0000 7f3abe800640 1 --2- 172.21.15.69:0/1013592571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ac0109670 0x7f3ac019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.704+0000 7f3abe800640 1 -- 172.21.15.69:0/1013592571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac01a62d0 con 0x7f3ac0108480 2024-04-24T09:04:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.704+0000 7f3ab5e00640 1 --2- 172.21.15.69:0/1013592571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ac0109670 0x7f3ac019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.704+0000 7f3abe800640 1 --2- 172.21.15.69:0/1013592571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0108480 0x7f3ac019e6b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000c970 tx=0x7f3ab000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.705+0000 7f3ab7e00640 1 -- 172.21.15.69:0/1013592571 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0004220 con 0x7f3ac0108480 2024-04-24T09:04:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.705+0000 7f3ab7e00640 1 -- 172.21.15.69:0/1013592571 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ab00043c0 con 0x7f3ac0108480 2024-04-24T09:04:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.705+0000 7f3ac4a00640 1 -- 172.21.15.69:0/1013592571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac01a65c0 con 0x7f3ac0108480 2024-04-24T09:04:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.705+0000 7f3ab7e00640 1 -- 172.21.15.69:0/1013592571 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab000f450 con 0x7f3ac0108480 2024-04-24T09:04:08.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.705+0000 7f3ac4a00640 1 -- 172.21.15.69:0/1013592571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac01a6b10 con 0x7f3ac0108480 2024-04-24T09:04:08.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.706+0000 7f3ac4a00640 1 -- 172.21.15.69:0/1013592571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ac010f810 con 0x7f3ac0108480 2024-04-24T09:04:08.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.708+0000 7f3ab7e00640 1 -- 172.21.15.69:0/1013592571 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3ab00079b0 con 0x7f3ac0108480 2024-04-24T09:04:08.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.709+0000 7f3ab7e00640 1 --2- 172.21.15.69:0/1013592571 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3aa40a3bb0 0x7f3aa40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.709+0000 7f3ab7e00640 1 -- 172.21.15.69:0/1013592571 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3ab00d0490 con 0x7f3ac0108480 2024-04-24T09:04:08.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.710+0000 7f3ab5e00640 1 --2- 172.21.15.69:0/1013592571 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3aa40a3bb0 0x7f3aa40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.710+0000 7f3ab5e00640 1 --2- 172.21.15.69:0/1013592571 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3aa40a3bb0 0x7f3aa40a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8006010 tx=0x7f3aa8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:08.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.714+0000 7f3ab7e00640 1 -- 172.21.15.69:0/1013592571 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ab0014030 con 0x7f3ac0108480 2024-04-24T09:04:08.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.962+0000 7f3ac4a00640 1 -- 172.21.15.69:0/1013592571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ac002d620 con 0x7f3ac0108480 2024-04-24T09:04:08.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.964+0000 7f3ab7e00640 1 -- 172.21.15.69:0/1013592571 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11800 (secure 0 0 0) 0x7f3ab009cc30 con 0x7f3ac0108480 2024-04-24T09:04:08.964 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:08.964 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34460,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34496,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T09:04:01.991559+0000","modified":"2024-04-24T09:04:06.590685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34532":{"gid":34532,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34532,"qdb_cluster":[34532]},"id":1}]} 2024-04-24T09:04:08.964 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 16 2024-04-24T09:04:08.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.966+0000 7f3ac4a00640 1 -- 172.21.15.69:0/1013592571 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3aa40a3bb0 msgr2=0x7f3aa40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.966+0000 7f3ac4a00640 1 --2- 172.21.15.69:0/1013592571 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3aa40a3bb0 0x7f3aa40a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8006010 tx=0x7f3aa8000bc0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.966+0000 7f3ac4a00640 1 -- 172.21.15.69:0/1013592571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0108480 msgr2=0x7f3ac019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.966+0000 7f3ac4a00640 1 --2- 172.21.15.69:0/1013592571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0108480 0x7f3ac019e6b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000c970 tx=0x7f3ab000ce30 comp rx=0 tx=0).stop 2024-04-24T09:04:08.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.967+0000 7f3ac4a00640 1 -- 172.21.15.69:0/1013592571 shutdown_connections 2024-04-24T09:04:08.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.967+0000 7f3ac4a00640 1 --2- 172.21.15.69:0/1013592571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ac010a030 0x7f3ac01a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.967+0000 7f3ac4a00640 1 --2- 172.21.15.69:0/1013592571 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3aa40a3bb0 0x7f3aa40a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.967+0000 7f3ac4a00640 1 --2- 172.21.15.69:0/1013592571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ac0109670 0x7f3ac019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.967+0000 7f3ac4a00640 1 --2- 172.21.15.69:0/1013592571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0108480 0x7f3ac019e6b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.967+0000 7f3ac4a00640 1 -- 172.21.15.69:0/1013592571 >> 172.21.15.69:0/1013592571 conn(0x7f3ac0077440 msgr2=0x7f3ac010c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:08.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.967+0000 7f3ac4a00640 1 -- 172.21.15.69:0/1013592571 shutdown_connections 2024-04-24T09:04:08.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:08.967+0000 7f3ac4a00640 1 -- 172.21.15.69:0/1013592571 wait complete. 2024-04-24T09:04:08.986 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:04:09.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.186+0000 7f1153200640 1 -- 172.21.15.69:0/3603043500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154103980 msgr2=0x7f115410a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.186+0000 7f1153200640 1 --2- 172.21.15.69:0/3603043500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154103980 0x7f115410a2a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f114c009ca0 tx=0x7f114c02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1153200640 1 -- 172.21.15.69:0/3603043500 shutdown_connections 2024-04-24T09:04:09.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1153200640 1 --2- 172.21.15.69:0/3603043500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154103980 0x7f115410a2a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1153200640 1 --2- 172.21.15.69:0/3603043500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1154102fc0 0x7f1154103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1153200640 1 --2- 172.21.15.69:0/3603043500 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1154101e70 0x7f1154102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1153200640 1 -- 172.21.15.69:0/3603043500 >> 172.21.15.69:0/3603043500 conn(0x7f11540fd3e0 msgr2=0x7f11540ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:09.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1153200640 1 -- 172.21.15.69:0/3603043500 shutdown_connections 2024-04-24T09:04:09.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1153200640 1 -- 172.21.15.69:0/3603043500 wait complete. 2024-04-24T09:04:09.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1153200640 1 Processor -- start 2024-04-24T09:04:09.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1153200640 1 -- start start 2024-04-24T09:04:09.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1153200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1154101e70 0x7f115419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1153200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1154102fc0 0x7f115419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1153200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154103980 0x7f11541a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1153200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f115410da10 con 0x7f1154103980 2024-04-24T09:04:09.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1153200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f115410d890 con 0x7f1154102fc0 2024-04-24T09:04:09.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1153200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f115410db90 con 0x7f1154101e70 2024-04-24T09:04:09.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1152800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154103980 0x7f11541a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1152800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154103980 0x7f11541a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50586/0 (socket says 172.21.15.69:50586) 2024-04-24T09:04:09.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.188+0000 7f1152800640 1 -- 172.21.15.69:0/2457079885 learned_addr learned my addr 172.21.15.69:0/2457079885 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:09.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.189+0000 7f1151400640 1 --2- 172.21.15.69:0/2457079885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1154102fc0 0x7f115419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.189+0000 7f1151e00640 1 --2- 172.21.15.69:0/2457079885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1154101e70 0x7f115419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.189+0000 7f1152800640 1 -- 172.21.15.69:0/2457079885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1154101e70 msgr2=0x7f115419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.189+0000 7f1152800640 1 --2- 172.21.15.69:0/2457079885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1154101e70 0x7f115419a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.189+0000 7f1152800640 1 -- 172.21.15.69:0/2457079885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1154102fc0 msgr2=0x7f115419a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.189+0000 7f1152800640 1 --2- 172.21.15.69:0/2457079885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1154102fc0 0x7f115419a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.189+0000 7f1152800640 1 -- 172.21.15.69:0/2457079885 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11541a1ec0 con 0x7f1154103980 2024-04-24T09:04:09.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.189+0000 7f1151400640 1 --2- 172.21.15.69:0/2457079885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1154102fc0 0x7f115419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:09.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.189+0000 7f1152800640 1 --2- 172.21.15.69:0/2457079885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154103980 0x7f11541a1850 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f114c031040 tx=0x7f114c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:09.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.189+0000 7f114aa00640 1 -- 172.21.15.69:0/2457079885 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c032ec0 con 0x7f1154103980 2024-04-24T09:04:09.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.190+0000 7f1153200640 1 -- 172.21.15.69:0/2457079885 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11541a2150 con 0x7f1154103980 2024-04-24T09:04:09.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.190+0000 7f114aa00640 1 -- 172.21.15.69:0/2457079885 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f114c0044f0 con 0x7f1154103980 2024-04-24T09:04:09.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.190+0000 7f114aa00640 1 -- 172.21.15.69:0/2457079885 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c030d10 con 0x7f1154103980 2024-04-24T09:04:09.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.190+0000 7f1153200640 1 -- 172.21.15.69:0/2457079885 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11541a26a0 con 0x7f1154103980 2024-04-24T09:04:09.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.191+0000 7f1153200640 1 -- 172.21.15.69:0/2457079885 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11541a2900 con 0x7f1154103980 2024-04-24T09:04:09.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.193+0000 7f114aa00640 1 -- 172.21.15.69:0/2457079885 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f114c004040 con 0x7f1154103980 2024-04-24T09:04:09.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.195+0000 7f114aa00640 1 --2- 172.21.15.69:0/2457079885 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f11380a3d50 0x7f11380a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.195+0000 7f114aa00640 1 -- 172.21.15.69:0/2457079885 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f114c0f1e10 con 0x7f1154103980 2024-04-24T09:04:09.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.195+0000 7f1151e00640 1 --2- 172.21.15.69:0/2457079885 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f11380a3d50 0x7f11380a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.196+0000 7f1151e00640 1 --2- 172.21.15.69:0/2457079885 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f11380a3d50 0x7f11380a6210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1144005fd0 tx=0x7f1144005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:09.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.200+0000 7f114aa00640 1 -- 172.21.15.69:0/2457079885 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f114c0f5030 con 0x7f1154103980 2024-04-24T09:04:09.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:09 smithi069 ceph-mon[28170]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T09:04:09.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:09 smithi069 ceph-mon[28170]: Cluster is now healthy 2024-04-24T09:04:09.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:09 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/4149918737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:09.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:09 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1013592571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:04:09.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.406+0000 7f1153200640 1 -- 172.21.15.69:0/2457079885 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11541a2c90 con 0x7f1154103980 2024-04-24T09:04:09.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.409+0000 7f114aa00640 1 -- 172.21.15.69:0/2457079885 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24094 (secure 0 0 0) 0x7f114c0be550 con 0x7f1154103980 2024-04-24T09:04:09.409 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:09.409 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":90,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:06.590427+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:04:09.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.411+0000 7f1153200640 1 -- 172.21.15.69:0/2457079885 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f11380a3d50 msgr2=0x7f11380a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.411+0000 7f1153200640 1 --2- 172.21.15.69:0/2457079885 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f11380a3d50 0x7f11380a6210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1144005fd0 tx=0x7f1144005850 comp rx=0 tx=0).stop 2024-04-24T09:04:09.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.411+0000 7f1153200640 1 -- 172.21.15.69:0/2457079885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154103980 msgr2=0x7f11541a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.411+0000 7f1153200640 1 --2- 172.21.15.69:0/2457079885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154103980 0x7f11541a1850 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f114c031040 tx=0x7f114c003ba0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.412+0000 7f1153200640 1 -- 172.21.15.69:0/2457079885 shutdown_connections 2024-04-24T09:04:09.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.412+0000 7f1153200640 1 --2- 172.21.15.69:0/2457079885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154103980 0x7f11541a1850 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.412+0000 7f1153200640 1 --2- 172.21.15.69:0/2457079885 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f11380a3d50 0x7f11380a6210 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.412+0000 7f1153200640 1 --2- 172.21.15.69:0/2457079885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1154102fc0 0x7f115419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.412+0000 7f1153200640 1 --2- 172.21.15.69:0/2457079885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1154101e70 0x7f115419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.412+0000 7f1153200640 1 -- 172.21.15.69:0/2457079885 >> 172.21.15.69:0/2457079885 conn(0x7f11540fd3e0 msgr2=0x7f1154074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:09.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.412+0000 7f1153200640 1 -- 172.21.15.69:0/2457079885 shutdown_connections 2024-04-24T09:04:09.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.412+0000 7f1153200640 1 -- 172.21.15.69:0/2457079885 wait complete. 2024-04-24T09:04:09.430 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T09:04:01.991559+0000', 'modified': '2024-04-24T09:04:06.590685+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34532}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34532': {'gid': 34532, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.81:6839/3185704097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6838', 'nonce': 3185704097}, {'type': 'v1', 'addr': '172.21.15.81:6839', 'nonce': 3185704097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 34532, 'qdb_cluster': [34532]} 2024-04-24T09:04:09.430 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T09:04:09.430 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T09:04:09.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:09 smithi130 ceph-mon[31553]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T09:04:09.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:09 smithi130 ceph-mon[31553]: Cluster is now healthy 2024-04-24T09:04:09.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:09 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/4149918737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:09.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:09 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1013592571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:04:09.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:09 smithi081 ceph-mon[30961]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T09:04:09.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:09 smithi081 ceph-mon[30961]: Cluster is now healthy 2024-04-24T09:04:09.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:09 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/4149918737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:09.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:09 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1013592571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:04:09.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.627+0000 7f6652800640 1 -- 172.21.15.69:0/1789341873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f664c105330 msgr2=0x7f664c1057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.627+0000 7f6652800640 1 --2- 172.21.15.69:0/1789341873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f664c105330 0x7f664c1057b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6638009900 tx=0x7f663802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.627+0000 7f6652800640 1 -- 172.21.15.69:0/1789341873 shutdown_connections 2024-04-24T09:04:09.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.627+0000 7f6652800640 1 --2- 172.21.15.69:0/1789341873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f664c105cf0 0x7f664c10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.627+0000 7f6652800640 1 --2- 172.21.15.69:0/1789341873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f664c105330 0x7f664c1057b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.627+0000 7f6652800640 1 --2- 172.21.15.69:0/1789341873 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f664c104140 0x7f664c104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.627+0000 7f6652800640 1 -- 172.21.15.69:0/1789341873 >> 172.21.15.69:0/1789341873 conn(0x7f664c0ff6b0 msgr2=0x7f664c101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:09.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.627+0000 7f6652800640 1 -- 172.21.15.69:0/1789341873 shutdown_connections 2024-04-24T09:04:09.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.628+0000 7f6652800640 1 -- 172.21.15.69:0/1789341873 wait complete. 2024-04-24T09:04:09.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.628+0000 7f6652800640 1 Processor -- start 2024-04-24T09:04:09.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.629+0000 7f6652800640 1 -- start start 2024-04-24T09:04:09.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.629+0000 7f6652800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f664c104140 0x7f664c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.629+0000 7f6652800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f664c105330 0x7f664c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.629+0000 7f6652800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f664c105cf0 0x7f664c1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.629+0000 7f6652800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f664c10fcd0 con 0x7f664c105330 2024-04-24T09:04:09.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.629+0000 7f6652800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f664c10fb50 con 0x7f664c104140 2024-04-24T09:04:09.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.629+0000 7f6652800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f664c10fe50 con 0x7f664c105cf0 2024-04-24T09:04:09.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.629+0000 7f6650a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f664c105330 0x7f664c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.629+0000 7f6650a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f664c105330 0x7f664c19a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50610/0 (socket says 172.21.15.69:50610) 2024-04-24T09:04:09.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.629+0000 7f6650a00640 1 -- 172.21.15.69:0/2319743546 learned_addr learned my addr 172.21.15.69:0/2319743546 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:09.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.629+0000 7f6650a00640 1 -- 172.21.15.69:0/2319743546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f664c105cf0 msgr2=0x7f664c1a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.629+0000 7f6650a00640 1 --2- 172.21.15.69:0/2319743546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f664c105cf0 0x7f664c1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.629+0000 7f6650a00640 1 -- 172.21.15.69:0/2319743546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f664c104140 msgr2=0x7f664c19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.629+0000 7f6650a00640 1 --2- 172.21.15.69:0/2319743546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f664c104140 0x7f664c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.629+0000 7f6650a00640 1 -- 172.21.15.69:0/2319743546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664c1a1f90 con 0x7f664c105330 2024-04-24T09:04:09.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.630+0000 7f6650a00640 1 --2- 172.21.15.69:0/2319743546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f664c105330 0x7f664c19a8b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f66380314e0 tx=0x7f6638003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:09.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.630+0000 7f664a000640 1 -- 172.21.15.69:0/2319743546 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663803f070 con 0x7f664c105330 2024-04-24T09:04:09.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.630+0000 7f664a000640 1 -- 172.21.15.69:0/2319743546 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6638031e40 con 0x7f664c105330 2024-04-24T09:04:09.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.630+0000 7f6652800640 1 -- 172.21.15.69:0/2319743546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664c1a2220 con 0x7f664c105330 2024-04-24T09:04:09.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.630+0000 7f664a000640 1 -- 172.21.15.69:0/2319743546 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638035030 con 0x7f664c105330 2024-04-24T09:04:09.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.630+0000 7f6652800640 1 -- 172.21.15.69:0/2319743546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f664c1a2760 con 0x7f664c105330 2024-04-24T09:04:09.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.633+0000 7f6656d9d740 1 Processor -- start 2024-04-24T09:04:09.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.633+0000 7f664a000640 1 -- 172.21.15.69:0/2319743546 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6638042070 con 0x7f664c105330 2024-04-24T09:04:09.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.633+0000 7f6656d9d740 1 -- start start 2024-04-24T09:04:09.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.634+0000 7f6656d9d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9f260 0x5634c3a9f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.634+0000 7f6656d9d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9fc50 0x5634c3aa4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.634+0000 7f6656d9d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3aa7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.634+0000 7f6656d9d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5634c3a04b10 con 0x5634c3aa4dc0 2024-04-24T09:04:09.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.634+0000 7f6656d9d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5634c39ab2b0 con 0x5634c3a9f260 2024-04-24T09:04:09.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.634+0000 7f6656d9d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5634c3a00ad0 con 0x5634c3a9fc50 2024-04-24T09:04:09.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.634+0000 7f6650a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3aa7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.634+0000 7f6650a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3aa7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50626/0 (socket says 172.21.15.69:50626) 2024-04-24T09:04:09.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.634+0000 7f6650a00640 1 -- 172.21.15.69:0/4067184147 learned_addr learned my addr 172.21.15.69:0/4067184147 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:09.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.634+0000 7f6651e00640 1 --2- 172.21.15.69:0/4067184147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9f260 0x5634c3a9f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.634+0000 7f6651400640 1 --2- 172.21.15.69:0/4067184147 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9fc50 0x5634c3aa4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.634+0000 7f6650a00640 1 -- 172.21.15.69:0/4067184147 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9fc50 msgr2=0x5634c3aa4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:09.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.634+0000 7f6650a00640 1 --2- 172.21.15.69:0/4067184147 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9fc50 0x5634c3aa4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.634+0000 7f6650a00640 1 -- 172.21.15.69:0/4067184147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9f260 msgr2=0x5634c3a9f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:09.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.634+0000 7f6650a00640 1 --2- 172.21.15.69:0/4067184147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9f260 0x5634c3a9f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.634+0000 7f6650a00640 1 -- 172.21.15.69:0/4067184147 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c37172d0 con 0x5634c3aa4dc0 2024-04-24T09:04:09.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.635+0000 7f6650a00640 1 --2- 172.21.15.69:0/4067184147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3aa7250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f66380f94b0 tx=0x7f66380f95a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7661be1c3cc853c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:09.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.635+0000 7f664a000640 1 --2- 172.21.15.69:0/2319743546 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f66340a3c30 0x7f66340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.635+0000 7f663e600640 1 -- 172.21.15.69:0/4067184147 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66380f32d0 con 0x5634c3aa4dc0 2024-04-24T09:04:09.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.635+0000 7f663e600640 1 -- 172.21.15.69:0/4067184147 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66380fe030 con 0x5634c3aa4dc0 2024-04-24T09:04:09.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.635+0000 7f663e600640 1 -- 172.21.15.69:0/4067184147 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66380bcab0 con 0x5634c3aa4dc0 2024-04-24T09:04:09.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.635+0000 7f664a000640 1 -- 172.21.15.69:0/2319743546 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f66380f1110 con 0x7f664c105330 2024-04-24T09:04:09.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.635+0000 7f6651e00640 1 --2- 172.21.15.69:0/2319743546 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f66340a3c30 0x7f66340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.636+0000 7f6656d9d740 1 -- 172.21.15.69:0/4067184147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 msgr2=0x5634c3aa7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.636+0000 7f6656d9d740 1 --2- 172.21.15.69:0/4067184147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3aa7250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f66380f94b0 tx=0x7f66380f95a0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.636+0000 7f6651e00640 1 --2- 172.21.15.69:0/2319743546 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f66340a3c30 0x7f66340a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x5634c3aa04b0 tx=0x7f6644007360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:09.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.636+0000 7f6656d9d740 1 -- 172.21.15.69:0/4067184147 shutdown_connections 2024-04-24T09:04:09.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.636+0000 7f6656d9d740 1 --2- 172.21.15.69:0/4067184147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3aa7250 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.636+0000 7f6656d9d740 1 --2- 172.21.15.69:0/4067184147 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9fc50 0x5634c3aa4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.636+0000 7f6656d9d740 1 --2- 172.21.15.69:0/4067184147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9f260 0x5634c3a9f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.636+0000 7f6656d9d740 1 -- 172.21.15.69:0/4067184147 >> 172.21.15.69:0/4067184147 conn(0x5634c3a9a8d0 msgr2=0x5634c3a9acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:09.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.636+0000 7f6656d9d740 1 -- 172.21.15.69:0/4067184147 shutdown_connections 2024-04-24T09:04:09.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.636+0000 7f6656d9d740 1 -- 172.21.15.69:0/4067184147 wait complete. 2024-04-24T09:04:09.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.637+0000 7f6656d9d740 1 -- start start 2024-04-24T09:04:09.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.637+0000 7f6656d9d740 1 Processor -- start 2024-04-24T09:04:09.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.637+0000 7f6656d9d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9f260 0x5634c3aa9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.637+0000 7f6656d9d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9fc50 0x5634c3aa9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.637+0000 7f6656d9d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3ab08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.637+0000 7f6656d9d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5634c3aa7b30 con 0x5634c3aa4dc0 2024-04-24T09:04:09.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.637+0000 7f6656d9d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5634c39be590 con 0x5634c3a9fc50 2024-04-24T09:04:09.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.637+0000 7f6656d9d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5634c3aa7cb0 con 0x5634c3a9f260 2024-04-24T09:04:09.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.637+0000 7f6656d9d740 10 client.0 _ll_register_callbacks cb 0x5634c39a13a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:09.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.637+0000 7f6651e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3ab08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.638+0000 7f6651e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3ab08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50642/0 (socket says 172.21.15.69:50642) 2024-04-24T09:04:09.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.638+0000 7f6651e00640 1 -- 172.21.15.69:0/3861272441 learned_addr learned my addr 172.21.15.69:0/3861272441 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:09.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.638+0000 7f6651400640 1 --2- 172.21.15.69:0/3861272441 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9f260 0x5634c3aa9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.638+0000 7f6651e00640 1 -- 172.21.15.69:0/3861272441 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9f260 msgr2=0x5634c3aa9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:09.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.638+0000 7f6651e00640 1 --2- 172.21.15.69:0/3861272441 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9f260 0x5634c3aa9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.638+0000 7f6651e00640 1 -- 172.21.15.69:0/3861272441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9fc50 msgr2=0x5634c3aa9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:09.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.638+0000 7f6650a00640 1 --2- 172.21.15.69:0/3861272441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9fc50 0x5634c3aa9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.638+0000 7f6651e00640 1 --2- 172.21.15.69:0/3861272441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9fc50 0x5634c3aa9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.638+0000 7f6651e00640 1 -- 172.21.15.69:0/3861272441 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c37e16b0 con 0x5634c3aa4dc0 2024-04-24T09:04:09.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.638+0000 7f6651e00640 1 --2- 172.21.15.69:0/3861272441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3ab08d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6644010a20 tx=0x7f6644010e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cebf4e43aa8c5e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:09.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.638+0000 7f6632a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:04:09.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.638+0000 7f6632a00640 1 -- 172.21.15.69:0/3861272441 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644021070 con 0x5634c3aa4dc0 2024-04-24T09:04:09.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.639+0000 7f6656d9d740 1 -- 172.21.15.69:0/3861272441 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66380fe030 con 0x5634c3aa4dc0 2024-04-24T09:04:09.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.639+0000 7f6632a00640 1 -- 172.21.15.69:0/3861272441 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f664401d450 con 0x5634c3aa4dc0 2024-04-24T09:04:09.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.639+0000 7f6632a00640 1 -- 172.21.15.69:0/3861272441 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f664400fe10 con 0x5634c3aa4dc0 2024-04-24T09:04:09.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.639+0000 7f6656d9d740 10 client.14757 fetch_fsmap learned FSMap version 16 2024-04-24T09:04:09.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.639+0000 7f6656d9d740 1 -- 172.21.15.69:0/3861272441 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x5634c3ab3d10 con 0x5634c3aa4dc0 2024-04-24T09:04:09.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.639+0000 7f6632a00640 1 -- 172.21.15.69:0/3861272441 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66380fe030 con 0x5634c3aa4dc0 2024-04-24T09:04:09.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.639+0000 7f6632a00640 1 -- 172.21.15.69:0/3861272441 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7f6644025710 con 0x5634c3aa4dc0 2024-04-24T09:04:09.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.639+0000 7f6656d9d740 10 client.14757 fetch_fsmap finished waiting for FSMap version 16 2024-04-24T09:04:09.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.639+0000 7f6656d9d740 10 client.14757 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T09:04:09.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.639+0000 7f6656d9d740 20 client.14757 populate_metadata read hostname 'smithi069' 2024-04-24T09:04:09.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.639+0000 7f6656d9d740 1 --2- 172.21.15.69:0/3861272441 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5634c3ab6430 0x5634c3ab78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.639+0000 7f6656d9d740 4 client.14757 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:09.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.639+0000 7f6656d9d740 1 -- 172.21.15.69:0/3861272441 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634c3a12530 con 0x5634c3ab6430 2024-04-24T09:04:09.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.640+0000 7f6651400640 1 --2- 172.21.15.69:0/3861272441 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5634c3ab6430 0x5634c3ab78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.641+0000 7f6651400640 1 --2- 172.21.15.69:0/3861272441 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5634c3ab6430 0x5634c3ab78a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=675328126b8cbf14 server_cookie=45989d043677bd24 in_seq=0 out_seq=0 2024-04-24T09:04:09.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.641+0000 7f6632a00640 10 client.14757 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:04:09.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.643+0000 7f6632a00640 1 -- 172.21.15.69:0/3861272441 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634c3a12530 con 0x5634c3ab6430 2024-04-24T09:04:09.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.643+0000 7f6632a00640 10 client.14757 handle_command_reply: tid=0 2024-04-24T09:04:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.643+0000 7f6656d9d740 1 client.14757 shutdown 2024-04-24T09:04:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.643+0000 7f6656d9d740 1 -- 172.21.15.69:0/3861272441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 msgr2=0x5634c3ab08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.643+0000 7f6656d9d740 1 --2- 172.21.15.69:0/3861272441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3ab08d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6644010a20 tx=0x7f6644010e20 comp rx=0 tx=0).stop 2024-04-24T09:04:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.644+0000 7f6656d9d740 1 -- 172.21.15.69:0/3861272441 shutdown_connections 2024-04-24T09:04:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.644+0000 7f6656d9d740 1 --2- 172.21.15.69:0/3861272441 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5634c3ab6430 0x5634c3ab78a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.644+0000 7f6656d9d740 1 --2- 172.21.15.69:0/3861272441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3ab08d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.644+0000 7f6656d9d740 1 --2- 172.21.15.69:0/3861272441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9fc50 0x5634c3aa9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.644+0000 7f6656d9d740 1 --2- 172.21.15.69:0/3861272441 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9f260 0x5634c3aa9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.644+0000 7f6632a00640 0 client.14757 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:04:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.644+0000 7f6656d9d740 1 -- 172.21.15.69:0/3861272441 >> 172.21.15.69:0/3861272441 conn(0x5634c3a9a8d0 msgr2=0x5634c3a99970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.644+0000 7f6656d9d740 1 -- 172.21.15.69:0/3861272441 shutdown_connections 2024-04-24T09:04:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.644+0000 7f6656d9d740 1 -- 172.21.15.69:0/3861272441 wait complete. 2024-04-24T09:04:09.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.644+0000 7f6656d9d740 20 client.14757 trim_cache size 0 max 16384 2024-04-24T09:04:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.663+0000 7f6656d9d740 1 Processor -- start 2024-04-24T09:04:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.663+0000 7f6656d9d740 1 -- start start 2024-04-24T09:04:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.663+0000 7f6656d9d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9f260 0x5634c3a9acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.663+0000 7f6656d9d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9fc50 0x5634c3ab0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.663+0000 7f6656d9d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3a990e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.663+0000 7f6656d9d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5634c3ab5be0 con 0x5634c3aa4dc0 2024-04-24T09:04:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.663+0000 7f6656d9d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5634c3ab62b0 con 0x5634c3a9fc50 2024-04-24T09:04:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.663+0000 7f6656d9d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5634c3ab6980 con 0x5634c3a9f260 2024-04-24T09:04:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.663+0000 7f6651e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3a990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.663+0000 7f6651e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3a990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50654/0 (socket says 172.21.15.69:50654) 2024-04-24T09:04:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.663+0000 7f6651e00640 1 -- 172.21.15.69:0/1283747395 learned_addr learned my addr 172.21.15.69:0/1283747395 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.664+0000 7f6651400640 1 --2- 172.21.15.69:0/1283747395 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9f260 0x5634c3a9acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.664+0000 7f6651e00640 1 -- 172.21.15.69:0/1283747395 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9f260 msgr2=0x5634c3a9acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.664+0000 7f6651e00640 1 --2- 172.21.15.69:0/1283747395 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9f260 0x5634c3a9acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.664+0000 7f6651e00640 1 -- 172.21.15.69:0/1283747395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9fc50 msgr2=0x5634c3ab0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.664+0000 7f6651e00640 1 --2- 172.21.15.69:0/1283747395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9fc50 0x5634c3ab0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.664+0000 7f6651e00640 1 -- 172.21.15.69:0/1283747395 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c3a9b5a0 con 0x5634c3aa4dc0 2024-04-24T09:04:09.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.664+0000 7f6651e00640 1 --2- 172.21.15.69:0/1283747395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3a990e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6644025c00 tx=0x7f664401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23de50a0a18617fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:09.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.664+0000 7f663dc00640 1 -- 172.21.15.69:0/1283747395 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644035070 con 0x5634c3aa4dc0 2024-04-24T09:04:09.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.664+0000 7f663dc00640 1 -- 172.21.15.69:0/1283747395 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6644025df0 con 0x5634c3aa4dc0 2024-04-24T09:04:09.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.664+0000 7f663dc00640 1 -- 172.21.15.69:0/1283747395 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644021050 con 0x5634c3aa4dc0 2024-04-24T09:04:09.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.665+0000 7f6656d9d740 1 -- 172.21.15.69:0/1283747395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 msgr2=0x5634c3a990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.665+0000 7f6656d9d740 1 --2- 172.21.15.69:0/1283747395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3a990e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6644025c00 tx=0x7f664401c3b0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.665+0000 7f6656d9d740 1 -- 172.21.15.69:0/1283747395 shutdown_connections 2024-04-24T09:04:09.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.665+0000 7f6656d9d740 1 --2- 172.21.15.69:0/1283747395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3a990e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.665+0000 7f6656d9d740 1 --2- 172.21.15.69:0/1283747395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9fc50 0x5634c3ab0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.665+0000 7f6656d9d740 1 --2- 172.21.15.69:0/1283747395 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9f260 0x5634c3a9acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.665+0000 7f6656d9d740 1 -- 172.21.15.69:0/1283747395 >> 172.21.15.69:0/1283747395 conn(0x5634c3a9a8d0 msgr2=0x5634c3aa1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.665+0000 7f6656d9d740 1 -- 172.21.15.69:0/1283747395 shutdown_connections 2024-04-24T09:04:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.665+0000 7f6656d9d740 1 -- 172.21.15.69:0/1283747395 wait complete. 2024-04-24T09:04:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.665+0000 7f6656d9d740 1 -- start start 2024-04-24T09:04:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.666+0000 7f6656d9d740 1 Processor -- start 2024-04-24T09:04:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.666+0000 7f6656d9d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9f260 0x5634c3aac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.666+0000 7f6656d9d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9fc50 0x5634c3aac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.666+0000 7f6656d9d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3aaf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.666+0000 7f6656d9d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5634c3ab7720 con 0x5634c3aa4dc0 2024-04-24T09:04:09.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.666+0000 7f6656d9d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5634c3ab7050 con 0x5634c3a9fc50 2024-04-24T09:04:09.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.666+0000 7f6656d9d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5634c3ab7df0 con 0x5634c3a9f260 2024-04-24T09:04:09.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.666+0000 7f6656d9d740 10 client.0 _ll_register_callbacks cb 0x7f66440106f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:09.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.666+0000 7f6651e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3aaf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.666+0000 7f6651e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3aaf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50664/0 (socket says 172.21.15.69:50664) 2024-04-24T09:04:09.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.666+0000 7f6651e00640 1 -- 172.21.15.69:0/462915212 learned_addr learned my addr 172.21.15.69:0/462915212 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:09.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6651e00640 1 -- 172.21.15.69:0/462915212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9f260 msgr2=0x5634c3aac2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:09.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6651400640 1 --2- 172.21.15.69:0/462915212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9f260 0x5634c3aac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6650a00640 1 --2- 172.21.15.69:0/462915212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9fc50 0x5634c3aac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6651e00640 1 --2- 172.21.15.69:0/462915212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9f260 0x5634c3aac2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6651e00640 1 -- 172.21.15.69:0/462915212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9fc50 msgr2=0x5634c3aac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:09.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6651e00640 1 --2- 172.21.15.69:0/462915212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9fc50 0x5634c3aac7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6651e00640 1 -- 172.21.15.69:0/462915212 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c3aafe00 con 0x5634c3aa4dc0 2024-04-24T09:04:09.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6651400640 1 --2- 172.21.15.69:0/462915212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9f260 0x5634c3aac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:09.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6651e00640 1 --2- 172.21.15.69:0/462915212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3aaf810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f664401c980 tx=0x7f664401cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfc1288b459e55f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:09.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6632a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:04:09.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6632a00640 1 -- 172.21.15.69:0/462915212 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664403e020 con 0x5634c3aa4dc0 2024-04-24T09:04:09.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6632a00640 1 -- 172.21.15.69:0/462915212 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f66440255c0 con 0x5634c3aa4dc0 2024-04-24T09:04:09.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6656d9d740 1 -- 172.21.15.69:0/462915212 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6644025df0 con 0x5634c3aa4dc0 2024-04-24T09:04:09.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6632a00640 1 -- 172.21.15.69:0/462915212 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f6644049050 con 0x5634c3aa4dc0 2024-04-24T09:04:09.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6656d9d740 10 client.14763 fetch_fsmap learned FSMap version 16 2024-04-24T09:04:09.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6656d9d740 1 -- 172.21.15.69:0/462915212 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x5634c3b1c6d0 con 0x5634c3aa4dc0 2024-04-24T09:04:09.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6632a00640 1 -- 172.21.15.69:0/462915212 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6644025df0 con 0x5634c3aa4dc0 2024-04-24T09:04:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6632a00640 1 -- 172.21.15.69:0/462915212 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7f664401dc60 con 0x5634c3aa4dc0 2024-04-24T09:04:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6656d9d740 10 client.14763 fetch_fsmap finished waiting for FSMap version 16 2024-04-24T09:04:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6656d9d740 10 client.14763 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T09:04:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6656d9d740 20 client.14763 populate_metadata read hostname 'smithi069' 2024-04-24T09:04:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6656d9d740 1 --2- 172.21.15.69:0/462915212 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5634c3b21120 0x5634c3b1fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6656d9d740 4 client.14763 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:04:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.668+0000 7f6656d9d740 1 -- 172.21.15.69:0/462915212 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "status"}) -- 0x5634c3a0c3a0 con 0x5634c3b21120 2024-04-24T09:04:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.669+0000 7f6651400640 1 --2- 172.21.15.69:0/462915212 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5634c3b21120 0x5634c3b1fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.669+0000 7f6651400640 1 --2- 172.21.15.69:0/462915212 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5634c3b21120 0x5634c3b1fe30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0f2ace7b4a323b3 server_cookie=3bbd7604b8296d06 in_seq=0 out_seq=0 2024-04-24T09:04:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.669+0000 7f6632a00640 10 client.14763 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:04:09.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.670+0000 7f6632a00640 1 -- 172.21.15.69:0/462915212 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5634c3a0c3a0 con 0x5634c3b21120 2024-04-24T09:04:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.670+0000 7f6632a00640 10 client.14763 handle_command_reply: tid=0 2024-04-24T09:04:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.670+0000 7f6656d9d740 1 client.14763 shutdown 2024-04-24T09:04:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.670+0000 7f6656d9d740 1 -- 172.21.15.69:0/462915212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 msgr2=0x5634c3aaf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.670+0000 7f6656d9d740 1 --2- 172.21.15.69:0/462915212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3aaf810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f664401c980 tx=0x7f664401cce0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.670+0000 7f6656d9d740 1 -- 172.21.15.69:0/462915212 shutdown_connections 2024-04-24T09:04:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.670+0000 7f6656d9d740 1 --2- 172.21.15.69:0/462915212 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5634c3b21120 0x5634c3b1fe30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.670+0000 7f6656d9d740 1 --2- 172.21.15.69:0/462915212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5634c3aa4dc0 0x5634c3aaf810 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.670+0000 7f6656d9d740 1 --2- 172.21.15.69:0/462915212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5634c3a9fc50 0x5634c3aac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.670+0000 7f6656d9d740 1 --2- 172.21.15.69:0/462915212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5634c3a9f260 0x5634c3aac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.670+0000 7f6656d9d740 1 -- 172.21.15.69:0/462915212 >> 172.21.15.69:0/462915212 conn(0x5634c3a9a8d0 msgr2=0x5634c3a990e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.670+0000 7f6632a00640 0 client.14763 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:04:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.670+0000 7f6656d9d740 1 -- 172.21.15.69:0/462915212 shutdown_connections 2024-04-24T09:04:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.670+0000 7f6656d9d740 1 -- 172.21.15.69:0/462915212 wait complete. 2024-04-24T09:04:09.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.671+0000 7f6656d9d740 20 client.14763 trim_cache size 0 max 16384 2024-04-24T09:04:09.677 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:04:09.677 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "ef1ec7e8-0217-11ef-bc93-c7b262605968", 2024-04-24T09:04:09.677 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-04-24T09:04:09.677 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 34532, 2024-04-24T09:04:09.677 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-24T09:04:09.677 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-24T09:04:09.677 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-24T09:04:09.677 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 7.1335766339999997, 2024-04-24T09:04:09.677 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 16, 2024-04-24T09:04:09.677 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 90, 2024-04-24T09:04:09.677 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 89, 2024-04-24T09:04:09.678 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 12.201744227000001 2024-04-24T09:04:09.678 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:04:09.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.671+0000 7f6652800640 1 -- 172.21.15.69:0/2319743546 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f66340a3c30 msgr2=0x7f66340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.671+0000 7f6652800640 1 --2- 172.21.15.69:0/2319743546 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f66340a3c30 0x7f66340a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x5634c3aa04b0 tx=0x7f6644007360 comp rx=0 tx=0).stop 2024-04-24T09:04:09.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.671+0000 7f6652800640 1 -- 172.21.15.69:0/2319743546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f664c105330 msgr2=0x7f664c19a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.671+0000 7f6652800640 1 --2- 172.21.15.69:0/2319743546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f664c105330 0x7f664c19a8b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f66380314e0 tx=0x7f6638003d50 comp rx=0 tx=0).stop 2024-04-24T09:04:09.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.671+0000 7f6652800640 1 -- 172.21.15.69:0/2319743546 shutdown_connections 2024-04-24T09:04:09.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.671+0000 7f6652800640 1 --2- 172.21.15.69:0/2319743546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f664c105cf0 0x7f664c1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.671+0000 7f6652800640 1 --2- 172.21.15.69:0/2319743546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f664c105330 0x7f664c19a8b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.671+0000 7f6652800640 1 --2- 172.21.15.69:0/2319743546 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f66340a3c30 0x7f66340a60f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.671+0000 7f6652800640 1 --2- 172.21.15.69:0/2319743546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f664c104140 0x7f664c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.671+0000 7f6652800640 1 -- 172.21.15.69:0/2319743546 >> 172.21.15.69:0/2319743546 conn(0x7f664c0ff6b0 msgr2=0x7f664c108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:09.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.671+0000 7f6652800640 1 -- 172.21.15.69:0/2319743546 shutdown_connections 2024-04-24T09:04:09.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:09.671+0000 7f6652800640 1 -- 172.21.15.69:0/2319743546 wait complete. 2024-04-24T09:04:09.688 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T09:04:09.688 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-24T09:04:09.688 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T09:04:10.109 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:04:10.442 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:10 smithi069 ceph-mon[28170]: pgmap v373: 129 pgs: 12 creating+peering, 113 active+clean, 4 unknown; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 580 B/s wr, 6 op/s 2024-04-24T09:04:10.443 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:10 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2457079885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:10.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:10 smithi130 ceph-mon[31553]: pgmap v373: 129 pgs: 12 creating+peering, 113 active+clean, 4 unknown; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 580 B/s wr, 6 op/s 2024-04-24T09:04:10.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:10 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2457079885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:10.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:10 smithi081 ceph-mon[30961]: pgmap v373: 129 pgs: 12 creating+peering, 113 active+clean, 4 unknown; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 580 B/s wr, 6 op/s 2024-04-24T09:04:10.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:10 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2457079885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:12.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.304+0000 7f12ba200700 1 -- 172.21.15.69:0/3949243245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12b406ba30 msgr2=0x7f12b410f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:12.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.304+0000 7f12ba200700 1 --2- 172.21.15.69:0/3949243245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12b406ba30 0x7f12b410f9a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00b0a0 tx=0x7f12ac00b3b0 comp rx=0 tx=0).stop 2024-04-24T09:04:12.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.304+0000 7f12ba200700 1 -- 172.21.15.69:0/3949243245 shutdown_connections 2024-04-24T09:04:12.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.304+0000 7f12ba200700 1 --2- 172.21.15.69:0/3949243245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12b406ba30 0x7f12b410f9a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:12.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.304+0000 7f12ba200700 1 --2- 172.21.15.69:0/3949243245 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12b4108b60 0x7f12b4108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:12.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.304+0000 7f12ba200700 1 --2- 172.21.15.69:0/3949243245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12b406b090 0x7f12b406b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:12.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.304+0000 7f12ba200700 1 -- 172.21.15.69:0/3949243245 >> 172.21.15.69:0/3949243245 conn(0x7f12b4078960 msgr2=0x7f12b4078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:12.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.304+0000 7f12ba200700 1 -- 172.21.15.69:0/3949243245 shutdown_connections 2024-04-24T09:04:12.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.304+0000 7f12ba200700 1 -- 172.21.15.69:0/3949243245 wait complete. 2024-04-24T09:04:12.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.305+0000 7f12ba200700 1 Processor -- start 2024-04-24T09:04:12.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.305+0000 7f12ba200700 1 -- start start 2024-04-24T09:04:12.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.306+0000 7f12ba200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12b406b090 0x7f12b419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:12.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.306+0000 7f12ba200700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12b406ba30 0x7f12b419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:12.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.306+0000 7f12ba200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12b4108b60 0x7f12b419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:12.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.306+0000 7f12ba200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f12b4112cc0 con 0x7f12b4108b60 2024-04-24T09:04:12.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.306+0000 7f12ba200700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f12b4112b40 con 0x7f12b406ba30 2024-04-24T09:04:12.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.306+0000 7f12ba200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f12b4112e40 con 0x7f12b406b090 2024-04-24T09:04:12.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.306+0000 7f12b3e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12b4108b60 0x7f12b419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:12.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.306+0000 7f12b3e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12b4108b60 0x7f12b419fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50684/0 (socket says 172.21.15.69:50684) 2024-04-24T09:04:12.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.306+0000 7f12b3e00700 1 -- 172.21.15.69:0/807994227 learned_addr learned my addr 172.21.15.69:0/807994227 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:12.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.306+0000 7f12b3400700 1 --2- 172.21.15.69:0/807994227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12b406b090 0x7f12b419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:12.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.306+0000 7f12b2a00700 1 --2- 172.21.15.69:0/807994227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12b406ba30 0x7f12b419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:12.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.307+0000 7f12b3e00700 1 -- 172.21.15.69:0/807994227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12b406b090 msgr2=0x7f12b419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:12.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.307+0000 7f12b3e00700 1 --2- 172.21.15.69:0/807994227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12b406b090 0x7f12b419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:12.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.307+0000 7f12b3e00700 1 -- 172.21.15.69:0/807994227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12b406ba30 msgr2=0x7f12b419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:12.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.307+0000 7f12b3e00700 1 --2- 172.21.15.69:0/807994227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12b406ba30 0x7f12b419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:12.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.307+0000 7f12b3e00700 1 -- 172.21.15.69:0/807994227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b41a0330 con 0x7f12b4108b60 2024-04-24T09:04:12.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.307+0000 7f12b3e00700 1 --2- 172.21.15.69:0/807994227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12b4108b60 0x7f12b419fbf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00b070 tx=0x7f12ac00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:12.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.308+0000 7f12abe00700 1 -- 172.21.15.69:0/807994227 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac021070 con 0x7f12b4108b60 2024-04-24T09:04:12.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.308+0000 7f12abe00700 1 -- 172.21.15.69:0/807994227 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12ac007cf0 con 0x7f12b4108b60 2024-04-24T09:04:12.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.308+0000 7f12ba200700 1 -- 172.21.15.69:0/807994227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b41a05c0 con 0x7f12b4108b60 2024-04-24T09:04:12.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.308+0000 7f12abe00700 1 -- 172.21.15.69:0/807994227 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac005440 con 0x7f12b4108b60 2024-04-24T09:04:12.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.308+0000 7f12ba200700 1 -- 172.21.15.69:0/807994227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b41a7ea0 con 0x7f12b4108b60 2024-04-24T09:04:12.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.309+0000 7f12ba200700 1 -- 172.21.15.69:0/807994227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b40697e0 con 0x7f12b4108b60 2024-04-24T09:04:12.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.311+0000 7f12abe00700 1 -- 172.21.15.69:0/807994227 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f12ac024070 con 0x7f12b4108b60 2024-04-24T09:04:12.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.314+0000 7f12abe00700 1 --2- 172.21.15.69:0/807994227 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f12980a3cc0 0x7f12980a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:12.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.314+0000 7f12abe00700 1 -- 172.21.15.69:0/807994227 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f12ac00f020 con 0x7f12b4108b60 2024-04-24T09:04:12.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.314+0000 7f12b3400700 1 --2- 172.21.15.69:0/807994227 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f12980a3cc0 0x7f12980a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:12.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.315+0000 7f12b3400700 1 --2- 172.21.15.69:0/807994227 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f12980a3cc0 0x7f12980a6180 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f12a4005fd0 tx=0x7f12a4005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:12.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.316+0000 7f12abe00700 1 -- 172.21.15.69:0/807994227 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f12ac02b020 con 0x7f12b4108b60 2024-04-24T09:04:12.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.598+0000 7f12ba200700 1 -- 172.21.15.69:0/807994227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f12b402dfe0 con 0x7f12b4108b60 2024-04-24T09:04:12.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.598+0000 7f12abe00700 1 -- 172.21.15.69:0/807994227 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7f12ac098e20 con 0x7f12b4108b60 2024-04-24T09:04:12.598 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T09:04:12.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.601+0000 7f12ba200700 1 -- 172.21.15.69:0/807994227 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f12980a3cc0 msgr2=0x7f12980a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:12.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.601+0000 7f12ba200700 1 --2- 172.21.15.69:0/807994227 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f12980a3cc0 0x7f12980a6180 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f12a4005fd0 tx=0x7f12a4005db0 comp rx=0 tx=0).stop 2024-04-24T09:04:12.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.601+0000 7f12ba200700 1 -- 172.21.15.69:0/807994227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12b4108b60 msgr2=0x7f12b419fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:12.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.601+0000 7f12ba200700 1 --2- 172.21.15.69:0/807994227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12b4108b60 0x7f12b419fbf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00b070 tx=0x7f12ac00cf00 comp rx=0 tx=0).stop 2024-04-24T09:04:12.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.601+0000 7f12ba200700 1 -- 172.21.15.69:0/807994227 shutdown_connections 2024-04-24T09:04:12.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.601+0000 7f12ba200700 1 --2- 172.21.15.69:0/807994227 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f12980a3cc0 0x7f12980a6180 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:12.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.601+0000 7f12ba200700 1 --2- 172.21.15.69:0/807994227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12b4108b60 0x7f12b419fbf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:12.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.601+0000 7f12ba200700 1 --2- 172.21.15.69:0/807994227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12b406b090 0x7f12b419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:12.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.601+0000 7f12ba200700 1 --2- 172.21.15.69:0/807994227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12b406ba30 0x7f12b419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:12.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.601+0000 7f12ba200700 1 -- 172.21.15.69:0/807994227 >> 172.21.15.69:0/807994227 conn(0x7f12b4078960 msgr2=0x7f12b410dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:12.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.601+0000 7f12ba200700 1 -- 172.21.15.69:0/807994227 shutdown_connections 2024-04-24T09:04:12.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:12.602+0000 7f12ba200700 1 -- 172.21.15.69:0/807994227 wait complete. 2024-04-24T09:04:12.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:12 smithi130 ceph-mon[31553]: pgmap v374: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 488 B/s wr, 5 op/s 2024-04-24T09:04:12.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:12 smithi081 ceph-mon[30961]: pgmap v374: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 488 B/s wr, 5 op/s 2024-04-24T09:04:12.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:12 smithi069 ceph-mon[28170]: pgmap v374: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 488 B/s wr, 5 op/s 2024-04-24T09:04:13.416 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T09:04:13.416 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T09:04:13.416 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi069.front.sepia.ceph.com 2024-04-24T09:04:13.416 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-24T09:04:13.416 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-24T09:04:13.416 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T09:04:13.416 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T09:04:13.416 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T09:04:13.416 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-24T09:04:13.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:04:13.417 DEBUG:teuthology.orchestra.run.smithi069:> ip addr 2024-04-24T09:04:13.469 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T09:04:13.469 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T09:04:13.469 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T09:04:13.469 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-24T09:04:13.469 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2024-04-24T09:04:13.469 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-24T09:04:13.469 INFO:teuthology.orchestra.run.smithi069.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T09:04:13.469 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c2 brd ff:ff:ff:ff:ff:ff 2024-04-24T09:04:13.469 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f0 2024-04-24T09:04:13.469 INFO:teuthology.orchestra.run.smithi069.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T09:04:13.470 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c3 brd ff:ff:ff:ff:ff:ff 2024-04-24T09:04:13.470 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f1 2024-04-24T09:04:13.470 INFO:teuthology.orchestra.run.smithi069.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T09:04:13.470 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:36 brd ff:ff:ff:ff:ff:ff 2024-04-24T09:04:13.470 INFO:teuthology.orchestra.run.smithi069.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T09:04:13.470 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:37 brd ff:ff:ff:ff:ff:ff 2024-04-24T09:04:13.470 INFO:teuthology.orchestra.run.smithi069.stdout: inet 172.21.15.69/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-24T09:04:13.470 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft 41970sec preferred_lft 41970sec 2024-04-24T09:04:13.470 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::ec4:7aff:fe8f:d737/64 scope link 2024-04-24T09:04:13.470 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-24T09:04:13.472 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T09:04:13.472 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T09:04:13.472 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-24T09:04:13.472 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add name ceph-brx type bridge 2024-04-24T09:04:13.472 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr flush dev ceph-brx 2024-04-24T09:04:13.472 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set ceph-brx up 2024-04-24T09:04:13.472 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T09:04:13.472 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-24T09:04:13.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T09:04:13.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:13 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/807994227' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T09:04:13.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:13 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/807994227' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T09:04:13.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:13 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/807994227' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T09:04:13.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:04:13.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:04:13.965 DEBUG:teuthology.orchestra.run.smithi069:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T09:04:14.039 INFO:teuthology.orchestra.run.smithi069.stdout:1 2024-04-24T09:04:14.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:04:14.039 DEBUG:teuthology.orchestra.run.smithi069:> route 2024-04-24T09:04:14.097 INFO:teuthology.orchestra.run.smithi069.stdout:Kernel IP routing table 2024-04-24T09:04:14.097 INFO:teuthology.orchestra.run.smithi069.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T09:04:14.097 INFO:teuthology.orchestra.run.smithi069.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T09:04:14.097 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T09:04:14.097 INFO:teuthology.orchestra.run.smithi069.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T09:04:14.098 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T09:04:14.098 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-24T09:04:14.098 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T09:04:14.098 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T09:04:14.098 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T09:04:14.098 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-24T09:04:14.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T09:04:14.488 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:14 smithi069 ceph-mon[28170]: pgmap v375: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 3 op/s 2024-04-24T09:04:14.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:14 smithi130 ceph-mon[31553]: pgmap v375: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 3 op/s 2024-04-24T09:04:14.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:14 smithi081 ceph-mon[30961]: pgmap v375: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 3 op/s 2024-04-24T09:04:14.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:04:14.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:04:14.866 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-04-24T09:04:14.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:04:14.918 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list-id 2024-04-24T09:04:14.968 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T09:04:14.969 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-24T09:04:14.969 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T09:04:14.969 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-24T09:04:14.969 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-24T09:04:15.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T09:04:15.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:04:15.162 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-24T09:04:15.162 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T09:04:15.162 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-24T09:04:15.162 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-24T09:04:15.162 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T09:04:15.162 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-24T09:04:15.162 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-24T09:04:15.162 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-24T09:04:15.163 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-24T09:04:15.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T09:04:15.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:04:15.470 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T09:04:15.471 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-24T09:04:15.471 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set brx.0 up 2024-04-24T09:04:15.471 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T09:04:15.471 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-24T09:04:15.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T09:04:15.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:04:15.667 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-24T09:04:15.667 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T09:04:15.667 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-24T09:04:15.710 INFO:teuthology.orchestra.run.smithi069.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-24T09:04:15.711 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T09:04:15.711 DEBUG:teuthology.orchestra.run.smithi069:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-24T09:04:15.804 DEBUG:teuthology.orchestra.run.smithi069:> sudo modprobe fuse 2024-04-24T09:04:15.878 DEBUG:teuthology.orchestra.run.smithi069:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T09:04:15.929 INFO:teuthology.orchestra.run.smithi069.stdout:/proc 2024-04-24T09:04:15.930 INFO:teuthology.orchestra.run.smithi069.stdout:/sys 2024-04-24T09:04:15.930 INFO:teuthology.orchestra.run.smithi069.stdout:/dev 2024-04-24T09:04:15.930 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/security 2024-04-24T09:04:15.930 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/shm 2024-04-24T09:04:15.930 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/pts 2024-04-24T09:04:15.930 INFO:teuthology.orchestra.run.smithi069.stdout:/run 2024-04-24T09:04:15.930 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/cgroup 2024-04-24T09:04:15.930 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/pstore 2024-04-24T09:04:15.930 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/bpf 2024-04-24T09:04:15.930 INFO:teuthology.orchestra.run.smithi069.stdout:/ 2024-04-24T09:04:15.930 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/selinux 2024-04-24T09:04:15.930 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T09:04:15.930 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/hugepages 2024-04-24T09:04:15.930 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/mqueue 2024-04-24T09:04:15.931 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/debug 2024-04-24T09:04:15.931 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/tracing 2024-04-24T09:04:15.931 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/fuse/connections 2024-04-24T09:04:15.931 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/config 2024-04-24T09:04:15.931 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T09:04:15.931 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T09:04:15.931 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T09:04:15.931 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T09:04:15.931 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/1000 2024-04-24T09:04:15.931 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/ceph 2024-04-24T09:04:15.931 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay 2024-04-24T09:04:15.931 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/4d06f77eaa5259c6bb17e8da1a8bed637ae880067f1021e3b3253ba509cd4732/merged 2024-04-24T09:04:15.931 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/7aa7cdb5c56355e95e408e531d393412286fa016cd83957227deaa009667270f/merged 2024-04-24T09:04:15.931 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/0 2024-04-24T09:04:15.931 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T09:04:15.932 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/6aecbd89fecaf0c822196acf6515275ab566f7ad98a825bfa2eb0f2477ed3ef0/merged 2024-04-24T09:04:15.932 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/fc760ee0fbe4ee78eac7c0091cf89c81988700e9a93cc7388a4946ebb780b034/merged 2024-04-24T09:04:15.932 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/443d842c61993dee7c7e6375365c522c0ec18fb322755dc7810647248857eb11/merged 2024-04-24T09:04:15.932 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/3e7dd254f74c7bf87fc37ff571c3fde92575b5d15c6ab6ef290263c26678ac0b/merged 2024-04-24T09:04:15.932 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/8197b9d0a14eb515e2c5c719db4fcdb2277f32478d09b381aad8fe9825d176ec/merged 2024-04-24T09:04:15.932 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/a111db2d886c68cdb65f91da6610327ea1f86955133a2f1d38e556a7977d08a4/merged 2024-04-24T09:04:15.932 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/fca4d0b958c67da344f2869839ad24e35e54dd2c9c941594a4fdbb065d485b47/merged 2024-04-24T09:04:15.932 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/5b8c61dc253e7f4c20a19c8e8aac7cb535181777cd68295075df6b087a11c3d1/merged 2024-04-24T09:04:15.932 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns 2024-04-24T09:04:15.932 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T09:04:15.932 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T09:04:15.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:04:15.933 DEBUG:teuthology.orchestra.run.smithi069:> ls /sys/fs/fuse/connections 2024-04-24T09:04:15.980 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T09:04:15.980 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-24T09:04:15.985 DEBUG:teuthology.orchestra.run.smithi069:> sudo modprobe fuse 2024-04-24T09:04:16.053 DEBUG:teuthology.orchestra.run.smithi069:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T09:04:16.095 INFO:teuthology.orchestra.run.smithi069.stdout:/proc 2024-04-24T09:04:16.096 INFO:teuthology.orchestra.run.smithi069.stdout:/sys 2024-04-24T09:04:16.096 INFO:teuthology.orchestra.run.smithi069.stdout:/dev 2024-04-24T09:04:16.096 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/security 2024-04-24T09:04:16.096 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/shm 2024-04-24T09:04:16.096 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/pts 2024-04-24T09:04:16.096 INFO:teuthology.orchestra.run.smithi069.stdout:/run 2024-04-24T09:04:16.096 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/cgroup 2024-04-24T09:04:16.096 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/pstore 2024-04-24T09:04:16.096 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/bpf 2024-04-24T09:04:16.096 INFO:teuthology.orchestra.run.smithi069.stdout:/ 2024-04-24T09:04:16.096 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/selinux 2024-04-24T09:04:16.097 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T09:04:16.097 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/hugepages 2024-04-24T09:04:16.097 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/mqueue 2024-04-24T09:04:16.097 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/debug 2024-04-24T09:04:16.097 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/tracing 2024-04-24T09:04:16.097 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/fuse/connections 2024-04-24T09:04:16.097 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/config 2024-04-24T09:04:16.097 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T09:04:16.097 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T09:04:16.097 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T09:04:16.097 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T09:04:16.097 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/1000 2024-04-24T09:04:16.097 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/ceph 2024-04-24T09:04:16.097 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay 2024-04-24T09:04:16.098 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/4d06f77eaa5259c6bb17e8da1a8bed637ae880067f1021e3b3253ba509cd4732/merged 2024-04-24T09:04:16.098 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/7aa7cdb5c56355e95e408e531d393412286fa016cd83957227deaa009667270f/merged 2024-04-24T09:04:16.098 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/0 2024-04-24T09:04:16.098 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T09:04:16.098 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/6aecbd89fecaf0c822196acf6515275ab566f7ad98a825bfa2eb0f2477ed3ef0/merged 2024-04-24T09:04:16.098 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/fc760ee0fbe4ee78eac7c0091cf89c81988700e9a93cc7388a4946ebb780b034/merged 2024-04-24T09:04:16.098 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/443d842c61993dee7c7e6375365c522c0ec18fb322755dc7810647248857eb11/merged 2024-04-24T09:04:16.098 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/3e7dd254f74c7bf87fc37ff571c3fde92575b5d15c6ab6ef290263c26678ac0b/merged 2024-04-24T09:04:16.098 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/8197b9d0a14eb515e2c5c719db4fcdb2277f32478d09b381aad8fe9825d176ec/merged 2024-04-24T09:04:16.098 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/a111db2d886c68cdb65f91da6610327ea1f86955133a2f1d38e556a7977d08a4/merged 2024-04-24T09:04:16.098 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/fca4d0b958c67da344f2869839ad24e35e54dd2c9c941594a4fdbb065d485b47/merged 2024-04-24T09:04:16.098 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/5b8c61dc253e7f4c20a19c8e8aac7cb535181777cd68295075df6b087a11c3d1/merged 2024-04-24T09:04:16.098 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns 2024-04-24T09:04:16.098 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T09:04:16.098 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T09:04:16.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:04:16.100 DEBUG:teuthology.orchestra.run.smithi069:> ls /sys/fs/fuse/connections 2024-04-24T09:04:16.174 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi069.stderr:2024-04-24T09:04:16.174+0000 7f43ac446480 -1 init, newargv = 0x55a5030fc200 newargc=15 2024-04-24T09:04:16.174 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi069.stderr:ceph-fuse[69613]: starting ceph client 2024-04-24T09:04:16.190 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi069.stderr:ceph-fuse[69613]: starting fuse 2024-04-24T09:04:16.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:16 smithi130 ceph-mon[31553]: pgmap v376: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 110 B/s wr, 2 op/s 2024-04-24T09:04:16.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:16 smithi081 ceph-mon[30961]: pgmap v376: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 110 B/s wr, 2 op/s 2024-04-24T09:04:16.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:16 smithi069 ceph-mon[28170]: pgmap v376: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 110 B/s wr, 2 op/s 2024-04-24T09:04:17.288 DEBUG:teuthology.orchestra.run.smithi069:> sudo modprobe fuse 2024-04-24T09:04:17.381 DEBUG:teuthology.orchestra.run.smithi069:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T09:04:17.434 INFO:teuthology.orchestra.run.smithi069.stdout:/proc 2024-04-24T09:04:17.434 INFO:teuthology.orchestra.run.smithi069.stdout:/sys 2024-04-24T09:04:17.434 INFO:teuthology.orchestra.run.smithi069.stdout:/dev 2024-04-24T09:04:17.434 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/security 2024-04-24T09:04:17.434 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/shm 2024-04-24T09:04:17.434 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/pts 2024-04-24T09:04:17.434 INFO:teuthology.orchestra.run.smithi069.stdout:/run 2024-04-24T09:04:17.434 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/cgroup 2024-04-24T09:04:17.434 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/pstore 2024-04-24T09:04:17.434 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/bpf 2024-04-24T09:04:17.434 INFO:teuthology.orchestra.run.smithi069.stdout:/ 2024-04-24T09:04:17.435 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/selinux 2024-04-24T09:04:17.435 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T09:04:17.435 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/hugepages 2024-04-24T09:04:17.435 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/mqueue 2024-04-24T09:04:17.435 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/debug 2024-04-24T09:04:17.435 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/tracing 2024-04-24T09:04:17.435 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/fuse/connections 2024-04-24T09:04:17.435 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/config 2024-04-24T09:04:17.435 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T09:04:17.435 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T09:04:17.435 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T09:04:17.435 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T09:04:17.435 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/1000 2024-04-24T09:04:17.435 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/ceph 2024-04-24T09:04:17.435 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay 2024-04-24T09:04:17.435 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/4d06f77eaa5259c6bb17e8da1a8bed637ae880067f1021e3b3253ba509cd4732/merged 2024-04-24T09:04:17.436 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/7aa7cdb5c56355e95e408e531d393412286fa016cd83957227deaa009667270f/merged 2024-04-24T09:04:17.436 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/0 2024-04-24T09:04:17.436 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T09:04:17.436 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/6aecbd89fecaf0c822196acf6515275ab566f7ad98a825bfa2eb0f2477ed3ef0/merged 2024-04-24T09:04:17.436 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/fc760ee0fbe4ee78eac7c0091cf89c81988700e9a93cc7388a4946ebb780b034/merged 2024-04-24T09:04:17.436 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/443d842c61993dee7c7e6375365c522c0ec18fb322755dc7810647248857eb11/merged 2024-04-24T09:04:17.436 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/3e7dd254f74c7bf87fc37ff571c3fde92575b5d15c6ab6ef290263c26678ac0b/merged 2024-04-24T09:04:17.436 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/8197b9d0a14eb515e2c5c719db4fcdb2277f32478d09b381aad8fe9825d176ec/merged 2024-04-24T09:04:17.436 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/a111db2d886c68cdb65f91da6610327ea1f86955133a2f1d38e556a7977d08a4/merged 2024-04-24T09:04:17.436 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/fca4d0b958c67da344f2869839ad24e35e54dd2c9c941594a4fdbb065d485b47/merged 2024-04-24T09:04:17.436 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/5b8c61dc253e7f4c20a19c8e8aac7cb535181777cd68295075df6b087a11c3d1/merged 2024-04-24T09:04:17.436 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns 2024-04-24T09:04:17.436 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T09:04:17.436 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T09:04:17.437 INFO:teuthology.orchestra.run.smithi069.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-24T09:04:17.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:04:17.437 DEBUG:teuthology.orchestra.run.smithi069:> ls /sys/fs/fuse/connections 2024-04-24T09:04:17.485 INFO:teuthology.orchestra.run.smithi069.stdout:114 2024-04-24T09:04:17.485 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-04-24T09:04:17.485 DEBUG:teuthology.orchestra.run.smithi069:> sudo stdin-killer -- python3 -c ' 2024-04-24T09:04:17.485 DEBUG:teuthology.orchestra.run.smithi069:> import glob 2024-04-24T09:04:17.485 DEBUG:teuthology.orchestra.run.smithi069:> import re 2024-04-24T09:04:17.485 DEBUG:teuthology.orchestra.run.smithi069:> import os 2024-04-24T09:04:17.486 DEBUG:teuthology.orchestra.run.smithi069:> import subprocess 2024-04-24T09:04:17.486 DEBUG:teuthology.orchestra.run.smithi069:> 2024-04-24T09:04:17.486 DEBUG:teuthology.orchestra.run.smithi069:> def _find_admin_socket(client_name): 2024-04-24T09:04:17.486 DEBUG:teuthology.orchestra.run.smithi069:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-24T09:04:17.486 DEBUG:teuthology.orchestra.run.smithi069:> files = glob.glob(asok_path) 2024-04-24T09:04:17.486 DEBUG:teuthology.orchestra.run.smithi069:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-24T09:04:17.486 DEBUG:teuthology.orchestra.run.smithi069:> 2024-04-24T09:04:17.486 DEBUG:teuthology.orchestra.run.smithi069:> # Given a non-glob path, it better be there 2024-04-24T09:04:17.486 DEBUG:teuthology.orchestra.run.smithi069:> if "*" not in asok_path: 2024-04-24T09:04:17.486 DEBUG:teuthology.orchestra.run.smithi069:> assert(len(files) == 1) 2024-04-24T09:04:17.486 DEBUG:teuthology.orchestra.run.smithi069:> return files[0] 2024-04-24T09:04:17.486 DEBUG:teuthology.orchestra.run.smithi069:> 2024-04-24T09:04:17.486 DEBUG:teuthology.orchestra.run.smithi069:> for f in files: 2024-04-24T09:04:17.486 DEBUG:teuthology.orchestra.run.smithi069:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T09:04:17.486 DEBUG:teuthology.orchestra.run.smithi069:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T09:04:17.487 DEBUG:teuthology.orchestra.run.smithi069:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T09:04:17.487 DEBUG:teuthology.orchestra.run.smithi069:> contents = proc_f.read() 2024-04-24T09:04:17.487 DEBUG:teuthology.orchestra.run.smithi069:> if mountpoint in contents: 2024-04-24T09:04:17.487 DEBUG:teuthology.orchestra.run.smithi069:> return f 2024-04-24T09:04:17.487 DEBUG:teuthology.orchestra.run.smithi069:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T09:04:17.487 DEBUG:teuthology.orchestra.run.smithi069:> 2024-04-24T09:04:17.487 DEBUG:teuthology.orchestra.run.smithi069:> print(_find_admin_socket("client.admin")) 2024-04-24T09:04:17.487 DEBUG:teuthology.orchestra.run.smithi069:> ' 2024-04-24T09:04:17.651 INFO:teuthology.orchestra.run.smithi069.stdout:/var/run/ceph/ceph-client.admin.69613.asok 2024-04-24T09:04:17.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:04:17.668 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.69613.asok 2024-04-24T09:04:17.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:04:17.669 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.69613.asok status 2024-04-24T09:04:17.895 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:04:17.895 INFO:teuthology.orchestra.run.smithi069.stdout: "metadata": { 2024-04-24T09:04:17.896 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_sha1": "1f81fb397ae98da7563d451a78c61574c8f4e6e0", 2024-04-24T09:04:17.896 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)", 2024-04-24T09:04:17.896 INFO:teuthology.orchestra.run.smithi069.stdout: "entity_id": "admin", 2024-04-24T09:04:17.896 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": "smithi069", 2024-04-24T09:04:17.896 INFO:teuthology.orchestra.run.smithi069.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-24T09:04:17.896 INFO:teuthology.orchestra.run.smithi069.stdout: "pid": "69613", 2024-04-24T09:04:17.896 INFO:teuthology.orchestra.run.smithi069.stdout: "root": "/" 2024-04-24T09:04:17.896 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:04:17.896 INFO:teuthology.orchestra.run.smithi069.stdout: "dentry_count": 0, 2024-04-24T09:04:17.896 INFO:teuthology.orchestra.run.smithi069.stdout: "dentry_pinned_count": 0, 2024-04-24T09:04:17.896 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 14775, 2024-04-24T09:04:17.897 INFO:teuthology.orchestra.run.smithi069.stdout: "inst": { 2024-04-24T09:04:17.897 INFO:teuthology.orchestra.run.smithi069.stdout: "name": { 2024-04-24T09:04:17.897 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "client", 2024-04-24T09:04:17.897 INFO:teuthology.orchestra.run.smithi069.stdout: "num": 14775 2024-04-24T09:04:17.897 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:04:17.897 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": { 2024-04-24T09:04:17.897 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-04-24T09:04:17.897 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "192.168.0.1:0", 2024-04-24T09:04:17.897 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 3608759631 2024-04-24T09:04:17.897 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:04:17.897 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:04:17.897 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": { 2024-04-24T09:04:17.897 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-04-24T09:04:17.898 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "192.168.0.1:0", 2024-04-24T09:04:17.898 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 3608759631 2024-04-24T09:04:17.898 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:04:17.898 INFO:teuthology.orchestra.run.smithi069.stdout: "inst_str": "client.14775 192.168.0.1:0/3608759631", 2024-04-24T09:04:17.898 INFO:teuthology.orchestra.run.smithi069.stdout: "addr_str": "192.168.0.1:0/3608759631", 2024-04-24T09:04:17.898 INFO:teuthology.orchestra.run.smithi069.stdout: "inode_count": 1, 2024-04-24T09:04:17.898 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_epoch": 16, 2024-04-24T09:04:17.898 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_epoch": 90, 2024-04-24T09:04:17.898 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_epoch_barrier": 0, 2024-04-24T09:04:17.898 INFO:teuthology.orchestra.run.smithi069.stdout: "blocklisted": false, 2024-04-24T09:04:17.898 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs" 2024-04-24T09:04:17.898 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:04:17.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:04:17.910 DEBUG:teuthology.orchestra.run.smithi069:> stat --file-system '--printf=%T 2024-04-24T09:04:17.910 DEBUG:teuthology.orchestra.run.smithi069:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-24T09:04:17.958 INFO:teuthology.orchestra.run.smithi069.stdout:fuseblk 2024-04-24T09:04:17.958 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-24T09:04:17.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:04:17.958 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-24T09:04:18.045 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-24T09:04:18.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T09:04:18.210 INFO:teuthology.orchestra.run.smithi069.stdout:# file: . 2024-04-24T09:04:18.210 INFO:teuthology.orchestra.run.smithi069.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-24T09:04:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:18.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:04:18.222 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T09:04:18.222 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T09:04:18.277 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:04:18.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.481+0000 7f5b5f600640 1 -- 172.21.15.69:0/1779562816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b580785a0 msgr2=0x7f5b58078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.481+0000 7f5b5f600640 1 --2- 172.21.15.69:0/1779562816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b580785a0 0x7f5b58078e40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009ca0 tx=0x7f5b5002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:04:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.481+0000 7f5b5f600640 1 -- 172.21.15.69:0/1779562816 shutdown_connections 2024-04-24T09:04:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.481+0000 7f5b5f600640 1 --2- 172.21.15.69:0/1779562816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b580785a0 0x7f5b58078e40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.481+0000 7f5b5f600640 1 --2- 172.21.15.69:0/1779562816 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5b58077c00 0x7f5b58078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.481+0000 7f5b5f600640 1 --2- 172.21.15.69:0/1779562816 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b580769c0 0x7f5b58076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.481+0000 7f5b5f600640 1 -- 172.21.15.69:0/1779562816 >> 172.21.15.69:0/1779562816 conn(0x7f5b580ff7f0 msgr2=0x7f5b58101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.481+0000 7f5b5f600640 1 -- 172.21.15.69:0/1779562816 shutdown_connections 2024-04-24T09:04:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.481+0000 7f5b5f600640 1 -- 172.21.15.69:0/1779562816 wait complete. 2024-04-24T09:04:18.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.482+0000 7f5b5f600640 1 Processor -- start 2024-04-24T09:04:18.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.483+0000 7f5b5f600640 1 -- start start 2024-04-24T09:04:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.483+0000 7f5b5f600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5b580769c0 0x7f5b5819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.483+0000 7f5b5f600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b58077c00 0x7f5b5819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.483+0000 7f5b5f600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b580785a0 0x7f5b581a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.483+0000 7f5b5f600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5b58114200 con 0x7f5b58077c00 2024-04-24T09:04:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.483+0000 7f5b5f600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5b58114080 con 0x7f5b580785a0 2024-04-24T09:04:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.483+0000 7f5b5f600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5b58114380 con 0x7f5b580769c0 2024-04-24T09:04:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.483+0000 7f5b57e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b58077c00 0x7f5b5819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.483+0000 7f5b57e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b58077c00 0x7f5b5819ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44344/0 (socket says 172.21.15.69:44344) 2024-04-24T09:04:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.483+0000 7f5b57e00640 1 -- 172.21.15.69:0/3614568595 learned_addr learned my addr 172.21.15.69:0/3614568595 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.483+0000 7f5b5cc00640 1 --2- 172.21.15.69:0/3614568595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5b580769c0 0x7f5b5819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.483+0000 7f5b5d600640 1 --2- 172.21.15.69:0/3614568595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b580785a0 0x7f5b581a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.484+0000 7f5b57e00640 1 -- 172.21.15.69:0/3614568595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5b580769c0 msgr2=0x7f5b5819e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.484+0000 7f5b57e00640 1 --2- 172.21.15.69:0/3614568595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5b580769c0 0x7f5b5819e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.484+0000 7f5b57e00640 1 -- 172.21.15.69:0/3614568595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b580785a0 msgr2=0x7f5b581a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.484+0000 7f5b57e00640 1 --2- 172.21.15.69:0/3614568595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b580785a0 0x7f5b581a5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.484+0000 7f5b57e00640 1 -- 172.21.15.69:0/3614568595 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b581a6470 con 0x7f5b58077c00 2024-04-24T09:04:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.484+0000 7f5b57e00640 1 --2- 172.21.15.69:0/3614568595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b58077c00 0x7f5b5819ed90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400d770 tx=0x7f5b4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.484+0000 7f5b55600640 1 -- 172.21.15.69:0/3614568595 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44017070 con 0x7f5b58077c00 2024-04-24T09:04:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.484+0000 7f5b55600640 1 -- 172.21.15.69:0/3614568595 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b440054d0 con 0x7f5b58077c00 2024-04-24T09:04:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.484+0000 7f5b55600640 1 -- 172.21.15.69:0/3614568595 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44004cf0 con 0x7f5b58077c00 2024-04-24T09:04:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.484+0000 7f5b5f600640 1 -- 172.21.15.69:0/3614568595 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b581a6760 con 0x7f5b58077c00 2024-04-24T09:04:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.484+0000 7f5b5f600640 1 -- 172.21.15.69:0/3614568595 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b581a6c60 con 0x7f5b58077c00 2024-04-24T09:04:18.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.486+0000 7f5b5f600640 1 -- 172.21.15.69:0/3614568595 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b58110d90 con 0x7f5b58077c00 2024-04-24T09:04:18.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.488+0000 7f5b55600640 1 -- 172.21.15.69:0/3614568595 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5b44004e90 con 0x7f5b58077c00 2024-04-24T09:04:18.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.490+0000 7f5b55600640 1 --2- 172.21.15.69:0/3614568595 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5b400a3bb0 0x7f5b400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:18.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.490+0000 7f5b55600640 1 -- 172.21.15.69:0/3614568595 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5b440d05f0 con 0x7f5b58077c00 2024-04-24T09:04:18.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.490+0000 7f5b5cc00640 1 --2- 172.21.15.69:0/3614568595 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5b400a3bb0 0x7f5b400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:18.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.491+0000 7f5b5cc00640 1 --2- 172.21.15.69:0/3614568595 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5b400a3bb0 0x7f5b400a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c005fd0 tx=0x7f5b4c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:18.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.493+0000 7f5b55600640 1 -- 172.21.15.69:0/3614568595 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b4409cd30 con 0x7f5b58077c00 2024-04-24T09:04:18.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.696+0000 7f5b5f600640 1 -- 172.21.15.69:0/3614568595 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b5802d890 con 0x7f5b58077c00 2024-04-24T09:04:18.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.698+0000 7f5b55600640 1 -- 172.21.15.69:0/3614568595 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24094 (secure 0 0 0) 0x7f5b4409c630 con 0x7f5b58077c00 2024-04-24T09:04:18.698 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:18.699 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":90,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:06.590427+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:04:18.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.701+0000 7f5b5f600640 1 -- 172.21.15.69:0/3614568595 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5b400a3bb0 msgr2=0x7f5b400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:18.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.701+0000 7f5b5f600640 1 --2- 172.21.15.69:0/3614568595 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5b400a3bb0 0x7f5b400a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c005fd0 tx=0x7f5b4c005850 comp rx=0 tx=0).stop 2024-04-24T09:04:18.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.701+0000 7f5b5f600640 1 -- 172.21.15.69:0/3614568595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b58077c00 msgr2=0x7f5b5819ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:18.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.701+0000 7f5b5f600640 1 --2- 172.21.15.69:0/3614568595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b58077c00 0x7f5b5819ed90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400d770 tx=0x7f5b4400dc30 comp rx=0 tx=0).stop 2024-04-24T09:04:18.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.701+0000 7f5b5f600640 1 -- 172.21.15.69:0/3614568595 shutdown_connections 2024-04-24T09:04:18.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.701+0000 7f5b5f600640 1 --2- 172.21.15.69:0/3614568595 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5b400a3bb0 0x7f5b400a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:18.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.701+0000 7f5b5f600640 1 --2- 172.21.15.69:0/3614568595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b580785a0 0x7f5b581a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:18.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.701+0000 7f5b5f600640 1 --2- 172.21.15.69:0/3614568595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b58077c00 0x7f5b5819ed90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:18.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.701+0000 7f5b5f600640 1 --2- 172.21.15.69:0/3614568595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5b580769c0 0x7f5b5819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:18.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.701+0000 7f5b5f600640 1 -- 172.21.15.69:0/3614568595 >> 172.21.15.69:0/3614568595 conn(0x7f5b580ff7f0 msgr2=0x7f5b58100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:18.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.701+0000 7f5b5f600640 1 -- 172.21.15.69:0/3614568595 shutdown_connections 2024-04-24T09:04:18.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.701+0000 7f5b5f600640 1 -- 172.21.15.69:0/3614568595 wait complete. 2024-04-24T09:04:18.717 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:18 smithi069 ceph-mon[28170]: pgmap v377: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-24T09:04:18.721 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:57:37.377599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:04:00.299827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:04:00.958498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T09:04:04.105314+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T09:04:18.721 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:04:18.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:18 smithi081 ceph-mon[30961]: pgmap v377: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-24T09:04:18.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.929+0000 7ff122200640 1 -- 172.21.15.69:0/3631157561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff11c105470 msgr2=0x7ff11c1058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:18.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.929+0000 7ff122200640 1 --2- 172.21.15.69:0/3631157561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff11c105470 0x7ff11c1058f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff104009900 tx=0x7ff10402a410 comp rx=0 tx=0).stop 2024-04-24T09:04:18.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.929+0000 7ff122200640 1 -- 172.21.15.69:0/3631157561 shutdown_connections 2024-04-24T09:04:18.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.929+0000 7ff122200640 1 --2- 172.21.15.69:0/3631157561 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff11c105e30 0x7ff11c10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:18.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.929+0000 7ff122200640 1 --2- 172.21.15.69:0/3631157561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff11c105470 0x7ff11c1058f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:18.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.929+0000 7ff122200640 1 --2- 172.21.15.69:0/3631157561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff11c104280 0x7ff11c1046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:18.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.929+0000 7ff122200640 1 -- 172.21.15.69:0/3631157561 >> 172.21.15.69:0/3631157561 conn(0x7ff11c0ff7f0 msgr2=0x7ff11c101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:18.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.929+0000 7ff122200640 1 -- 172.21.15.69:0/3631157561 shutdown_connections 2024-04-24T09:04:18.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.929+0000 7ff122200640 1 -- 172.21.15.69:0/3631157561 wait complete. 2024-04-24T09:04:18.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.930+0000 7ff122200640 1 Processor -- start 2024-04-24T09:04:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.931+0000 7ff122200640 1 -- start start 2024-04-24T09:04:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.931+0000 7ff122200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff11c104280 0x7ff11c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.931+0000 7ff122200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff11c105470 0x7ff11c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.931+0000 7ff122200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff11c105e30 0x7ff11c1a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.931+0000 7ff122200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c10fdc0 con 0x7ff11c105470 2024-04-24T09:04:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.931+0000 7ff122200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c10fc40 con 0x7ff11c104280 2024-04-24T09:04:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.931+0000 7ff122200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c10ff40 con 0x7ff11c105e30 2024-04-24T09:04:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.931+0000 7ff11aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff11c105470 0x7ff11c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.932+0000 7ff11aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff11c105470 0x7ff11c19aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44360/0 (socket says 172.21.15.69:44360) 2024-04-24T09:04:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.932+0000 7ff11aa00640 1 -- 172.21.15.69:0/2715026649 learned_addr learned my addr 172.21.15.69:0/2715026649 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.932+0000 7ff11b400640 1 --2- 172.21.15.69:0/2715026649 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff11c104280 0x7ff11c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.932+0000 7ff11be00640 1 --2- 172.21.15.69:0/2715026649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff11c105e30 0x7ff11c1a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.932+0000 7ff11aa00640 1 -- 172.21.15.69:0/2715026649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff11c105e30 msgr2=0x7ff11c1a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.932+0000 7ff11aa00640 1 --2- 172.21.15.69:0/2715026649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff11c105e30 0x7ff11c1a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.932+0000 7ff11aa00640 1 -- 172.21.15.69:0/2715026649 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff11c104280 msgr2=0x7ff11c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.932+0000 7ff11aa00640 1 --2- 172.21.15.69:0/2715026649 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff11c104280 0x7ff11c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.932+0000 7ff11aa00640 1 -- 172.21.15.69:0/2715026649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11c1a20f0 con 0x7ff11c105470 2024-04-24T09:04:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.932+0000 7ff11aa00640 1 --2- 172.21.15.69:0/2715026649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff11c105470 0x7ff11c19aa10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff104004770 tx=0x7ff1040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:18.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.933+0000 7ff113e00640 1 -- 172.21.15.69:0/2715026649 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10403f030 con 0x7ff11c105470 2024-04-24T09:04:18.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.933+0000 7ff122200640 1 -- 172.21.15.69:0/2715026649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11c1a2380 con 0x7ff11c105470 2024-04-24T09:04:18.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.933+0000 7ff113e00640 1 -- 172.21.15.69:0/2715026649 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff104003cb0 con 0x7ff11c105470 2024-04-24T09:04:18.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.933+0000 7ff113e00640 1 -- 172.21.15.69:0/2715026649 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104035070 con 0x7ff11c105470 2024-04-24T09:04:18.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.933+0000 7ff122200640 1 -- 172.21.15.69:0/2715026649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff11c1a28c0 con 0x7ff11c105470 2024-04-24T09:04:18.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.934+0000 7ff122200640 1 -- 172.21.15.69:0/2715026649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff11c10b610 con 0x7ff11c105470 2024-04-24T09:04:18.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.936+0000 7ff113e00640 1 -- 172.21.15.69:0/2715026649 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff104042030 con 0x7ff11c105470 2024-04-24T09:04:18.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.937+0000 7ff113e00640 1 --2- 172.21.15.69:0/2715026649 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff1000a3c30 0x7ff1000a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:18.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.937+0000 7ff113e00640 1 -- 172.21.15.69:0/2715026649 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff1040f1090 con 0x7ff11c105470 2024-04-24T09:04:18.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.938+0000 7ff11b400640 1 --2- 172.21.15.69:0/2715026649 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff1000a3c30 0x7ff1000a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:18.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.938+0000 7ff11b400640 1 --2- 172.21.15.69:0/2715026649 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff1000a3c30 0x7ff1000a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff114005fd0 tx=0x7ff114005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:18.941+0000 7ff113e00640 1 -- 172.21.15.69:0/2715026649 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1040bd7d0 con 0x7ff11c105470 2024-04-24T09:04:18.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:18 smithi130 ceph-mon[31553]: pgmap v377: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-24T09:04:19.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.144+0000 7ff122200640 1 -- 172.21.15.69:0/2715026649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff11c062c00 con 0x7ff11c105470 2024-04-24T09:04:19.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.145+0000 7ff113e00640 1 -- 172.21.15.69:0/2715026649 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7ff1040bd0d0 con 0x7ff11c105470 2024-04-24T09:04:19.147 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-24T09:04:19.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.147+0000 7ff122200640 1 -- 172.21.15.69:0/2715026649 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff1000a3c30 msgr2=0x7ff1000a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:19.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.147+0000 7ff122200640 1 --2- 172.21.15.69:0/2715026649 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff1000a3c30 0x7ff1000a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff114005fd0 tx=0x7ff114005850 comp rx=0 tx=0).stop 2024-04-24T09:04:19.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.147+0000 7ff122200640 1 -- 172.21.15.69:0/2715026649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff11c105470 msgr2=0x7ff11c19aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:19.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.147+0000 7ff122200640 1 --2- 172.21.15.69:0/2715026649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff11c105470 0x7ff11c19aa10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff104004770 tx=0x7ff1040042f0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.147+0000 7ff122200640 1 -- 172.21.15.69:0/2715026649 shutdown_connections 2024-04-24T09:04:19.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.147+0000 7ff122200640 1 --2- 172.21.15.69:0/2715026649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff11c105e30 0x7ff11c1a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.147+0000 7ff122200640 1 --2- 172.21.15.69:0/2715026649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff11c105470 0x7ff11c19aa10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.147+0000 7ff122200640 1 --2- 172.21.15.69:0/2715026649 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff1000a3c30 0x7ff1000a60f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.147+0000 7ff122200640 1 --2- 172.21.15.69:0/2715026649 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff11c104280 0x7ff11c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.147+0000 7ff122200640 1 -- 172.21.15.69:0/2715026649 >> 172.21.15.69:0/2715026649 conn(0x7ff11c0ff7f0 msgr2=0x7ff11c1087e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:19.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.148+0000 7ff122200640 1 -- 172.21.15.69:0/2715026649 shutdown_connections 2024-04-24T09:04:19.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.148+0000 7ff122200640 1 -- 172.21.15.69:0/2715026649 wait complete. 2024-04-24T09:04:19.168 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:04:19.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.373+0000 7fe24b000640 1 -- 172.21.15.69:0/3164869013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe24410bc30 msgr2=0x7fe24410e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:19.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.373+0000 7fe24b000640 1 --2- 172.21.15.69:0/3164869013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe24410bc30 0x7fe24410e060 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe230009370 tx=0x7fe23002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.373+0000 7fe24b000640 1 -- 172.21.15.69:0/3164869013 shutdown_connections 2024-04-24T09:04:19.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.373+0000 7fe24b000640 1 --2- 172.21.15.69:0/3164869013 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe24410e6f0 0x7fe244110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.373+0000 7fe24b000640 1 --2- 172.21.15.69:0/3164869013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe24410bc30 0x7fe24410e060 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.373+0000 7fe24b000640 1 --2- 172.21.15.69:0/3164869013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe244109270 0x7fe24410b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.373+0000 7fe24b000640 1 -- 172.21.15.69:0/3164869013 >> 172.21.15.69:0/3164869013 conn(0x7fe24406f230 msgr2=0x7fe244071650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:19.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.373+0000 7fe24b000640 1 -- 172.21.15.69:0/3164869013 shutdown_connections 2024-04-24T09:04:19.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.373+0000 7fe24b000640 1 -- 172.21.15.69:0/3164869013 wait complete. 2024-04-24T09:04:19.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.374+0000 7fe24b000640 1 Processor -- start 2024-04-24T09:04:19.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.375+0000 7fe24b000640 1 -- start start 2024-04-24T09:04:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.375+0000 7fe24b000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe244109270 0x7fe2441a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.375+0000 7fe24b000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe24410bc30 0x7fe2441a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.375+0000 7fe24b000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe24410e6f0 0x7fe2441a8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.375+0000 7fe24b000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441141c0 con 0x7fe24410e6f0 2024-04-24T09:04:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.375+0000 7fe24b000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe244114040 con 0x7fe24410bc30 2024-04-24T09:04:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.375+0000 7fe24b000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe244114340 con 0x7fe244109270 2024-04-24T09:04:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.375+0000 7fe249000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe24410e6f0 0x7fe2441a8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.375+0000 7fe249000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe24410e6f0 0x7fe2441a8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44386/0 (socket says 172.21.15.69:44386) 2024-04-24T09:04:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.375+0000 7fe249000640 1 -- 172.21.15.69:0/2685472363 learned_addr learned my addr 172.21.15.69:0/2685472363 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.376+0000 7fe243400640 1 --2- 172.21.15.69:0/2685472363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe24410bc30 0x7fe2441a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.376+0000 7fe243e00640 1 --2- 172.21.15.69:0/2685472363 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe244109270 0x7fe2441a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.376+0000 7fe249000640 1 -- 172.21.15.69:0/2685472363 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe244109270 msgr2=0x7fe2441a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.376+0000 7fe249000640 1 --2- 172.21.15.69:0/2685472363 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe244109270 0x7fe2441a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.376+0000 7fe249000640 1 -- 172.21.15.69:0/2685472363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe24410bc30 msgr2=0x7fe2441a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.376+0000 7fe249000640 1 --2- 172.21.15.69:0/2685472363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe24410bc30 0x7fe2441a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.376+0000 7fe249000640 1 -- 172.21.15.69:0/2685472363 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2441a86c0 con 0x7fe24410e6f0 2024-04-24T09:04:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.376+0000 7fe249000640 1 --2- 172.21.15.69:0/2685472363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe24410e6f0 0x7fe2441a8050 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00dd70 tx=0x7fe23c00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:19.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.376+0000 7fe240c00640 1 -- 172.21.15.69:0/2685472363 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c004d20 con 0x7fe24410e6f0 2024-04-24T09:04:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.376+0000 7fe240c00640 1 -- 172.21.15.69:0/2685472363 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe23c00be40 con 0x7fe24410e6f0 2024-04-24T09:04:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.376+0000 7fe24b000640 1 -- 172.21.15.69:0/2685472363 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2441a89b0 con 0x7fe24410e6f0 2024-04-24T09:04:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.376+0000 7fe240c00640 1 -- 172.21.15.69:0/2685472363 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c0055b0 con 0x7fe24410e6f0 2024-04-24T09:04:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.377+0000 7fe24b000640 1 -- 172.21.15.69:0/2685472363 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2441a8f20 con 0x7fe24410e6f0 2024-04-24T09:04:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.378+0000 7fe24b000640 1 -- 172.21.15.69:0/2685472363 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2441a9210 con 0x7fe24410e6f0 2024-04-24T09:04:19.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.381+0000 7fe240c00640 1 -- 172.21.15.69:0/2685472363 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe23c005750 con 0x7fe24410e6f0 2024-04-24T09:04:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.382+0000 7fe240c00640 1 --2- 172.21.15.69:0/2685472363 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fe2280a3c30 0x7fe2280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:19.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.382+0000 7fe240c00640 1 -- 172.21.15.69:0/2685472363 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe23c0d0ea0 con 0x7fe24410e6f0 2024-04-24T09:04:19.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.382+0000 7fe243e00640 1 --2- 172.21.15.69:0/2685472363 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fe2280a3c30 0x7fe2280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:19.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.383+0000 7fe243e00640 1 --2- 172.21.15.69:0/2685472363 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fe2280a3c30 0x7fe2280a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe234005e10 tx=0x7fe234005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:19.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.383+0000 7fe240c00640 1 -- 172.21.15.69:0/2685472363 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe23c09d5e0 con 0x7fe24410e6f0 2024-04-24T09:04:19.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.588+0000 7fe24b000640 1 -- 172.21.15.69:0/2685472363 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe2441a93e0 con 0x7fe24410e6f0 2024-04-24T09:04:19.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.589+0000 7fe240c00640 1 -- 172.21.15.69:0/2685472363 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fe23c09cee0 con 0x7fe24410e6f0 2024-04-24T09:04:19.589 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:04:19.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.592+0000 7fe24b000640 1 -- 172.21.15.69:0/2685472363 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fe2280a3c30 msgr2=0x7fe2280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.592+0000 7fe24b000640 1 --2- 172.21.15.69:0/2685472363 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fe2280a3c30 0x7fe2280a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe234005e10 tx=0x7fe234005da0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.592+0000 7fe24b000640 1 -- 172.21.15.69:0/2685472363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe24410e6f0 msgr2=0x7fe2441a8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.592+0000 7fe24b000640 1 --2- 172.21.15.69:0/2685472363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe24410e6f0 0x7fe2441a8050 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00dd70 tx=0x7fe23c00b730 comp rx=0 tx=0).stop 2024-04-24T09:04:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.592+0000 7fe24b000640 1 -- 172.21.15.69:0/2685472363 shutdown_connections 2024-04-24T09:04:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.592+0000 7fe24b000640 1 --2- 172.21.15.69:0/2685472363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe24410e6f0 0x7fe2441a8050 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.592+0000 7fe24b000640 1 --2- 172.21.15.69:0/2685472363 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fe2280a3c30 0x7fe2280a60f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.592+0000 7fe24b000640 1 --2- 172.21.15.69:0/2685472363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe24410bc30 0x7fe2441a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.592+0000 7fe24b000640 1 --2- 172.21.15.69:0/2685472363 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe244109270 0x7fe2441a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.592+0000 7fe24b000640 1 -- 172.21.15.69:0/2685472363 >> 172.21.15.69:0/2685472363 conn(0x7fe24406f230 msgr2=0x7fe244071620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.592+0000 7fe24b000640 1 -- 172.21.15.69:0/2685472363 shutdown_connections 2024-04-24T09:04:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.592+0000 7fe24b000640 1 -- 172.21.15.69:0/2685472363 wait complete. 2024-04-24T09:04:19.610 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:04:19.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.806+0000 7fc85f200640 1 -- 172.21.15.69:0/3929630833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc860108540 msgr2=0x7fc860108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:19.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.806+0000 7fc85f200640 1 --2- 172.21.15.69:0/3929630833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc860108540 0x7fc860108960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc850009900 tx=0x7fc85002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.806+0000 7fc85f200640 1 -- 172.21.15.69:0/3929630833 shutdown_connections 2024-04-24T09:04:19.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.806+0000 7fc85f200640 1 --2- 172.21.15.69:0/3929630833 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc86010a0f0 0x7fc860110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.806+0000 7fc85f200640 1 --2- 172.21.15.69:0/3929630833 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc860109730 0x7fc860109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.806+0000 7fc85f200640 1 --2- 172.21.15.69:0/3929630833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc860108540 0x7fc860108960 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.806+0000 7fc85f200640 1 -- 172.21.15.69:0/3929630833 >> 172.21.15.69:0/3929630833 conn(0x7fc8600774d0 msgr2=0x7fc860079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:19.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.806+0000 7fc85f200640 1 -- 172.21.15.69:0/3929630833 shutdown_connections 2024-04-24T09:04:19.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.806+0000 7fc85f200640 1 -- 172.21.15.69:0/3929630833 wait complete. 2024-04-24T09:04:19.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.807+0000 7fc85f200640 1 Processor -- start 2024-04-24T09:04:19.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.808+0000 7fc85f200640 1 -- start start 2024-04-24T09:04:19.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.808+0000 7fc85f200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc860108540 0x7fc86019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:19.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.808+0000 7fc85f200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc860109730 0x7fc86019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:19.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.808+0000 7fc85f200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86010a0f0 0x7fc8601a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:19.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.808+0000 7fc85f200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc860114080 con 0x7fc86010a0f0 2024-04-24T09:04:19.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.808+0000 7fc85f200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc860113f00 con 0x7fc860108540 2024-04-24T09:04:19.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.808+0000 7fc85f200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc860114200 con 0x7fc860109730 2024-04-24T09:04:19.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.808+0000 7fc85e800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86010a0f0 0x7fc8601a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:19.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.808+0000 7fc85e800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86010a0f0 0x7fc8601a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44412/0 (socket says 172.21.15.69:44412) 2024-04-24T09:04:19.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.808+0000 7fc85e800640 1 -- 172.21.15.69:0/2882829253 learned_addr learned my addr 172.21.15.69:0/2882829253 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:19.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.809+0000 7fc85de00640 1 --2- 172.21.15.69:0/2882829253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc860108540 0x7fc86019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:19.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.809+0000 7fc857e00640 1 --2- 172.21.15.69:0/2882829253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc860109730 0x7fc86019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:19.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.809+0000 7fc85e800640 1 -- 172.21.15.69:0/2882829253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc860109730 msgr2=0x7fc86019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:19.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.809+0000 7fc85e800640 1 --2- 172.21.15.69:0/2882829253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc860109730 0x7fc86019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.809+0000 7fc85e800640 1 -- 172.21.15.69:0/2882829253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc860108540 msgr2=0x7fc86019e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:19.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.809+0000 7fc85e800640 1 --2- 172.21.15.69:0/2882829253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc860108540 0x7fc86019e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:19.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.809+0000 7fc85e800640 1 -- 172.21.15.69:0/2882829253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8601a6270 con 0x7fc86010a0f0 2024-04-24T09:04:19.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.809+0000 7fc857e00640 1 --2- 172.21.15.69:0/2882829253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc860109730 0x7fc86019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:19.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.809+0000 7fc85e800640 1 --2- 172.21.15.69:0/2882829253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86010a0f0 0x7fc8601a5c00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc85800cce0 tx=0x7fc85800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:19.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.809+0000 7fc856a00640 1 -- 172.21.15.69:0/2882829253 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85800f450 con 0x7fc86010a0f0 2024-04-24T09:04:19.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.809+0000 7fc856a00640 1 -- 172.21.15.69:0/2882829253 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc85800faa0 con 0x7fc86010a0f0 2024-04-24T09:04:19.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.809+0000 7fc856a00640 1 -- 172.21.15.69:0/2882829253 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858018b70 con 0x7fc86010a0f0 2024-04-24T09:04:19.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.809+0000 7fc85f200640 1 -- 172.21.15.69:0/2882829253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8601a6560 con 0x7fc86010a0f0 2024-04-24T09:04:19.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.810+0000 7fc85f200640 1 -- 172.21.15.69:0/2882829253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8601a6a60 con 0x7fc86010a0f0 2024-04-24T09:04:19.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.811+0000 7fc85f200640 1 -- 172.21.15.69:0/2882829253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc86010f800 con 0x7fc86010a0f0 2024-04-24T09:04:19.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.813+0000 7fc856a00640 1 -- 172.21.15.69:0/2882829253 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc858018d10 con 0x7fc86010a0f0 2024-04-24T09:04:19.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.815+0000 7fc856a00640 1 --2- 172.21.15.69:0/2882829253 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc8440a3bb0 0x7fc8440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:19.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.815+0000 7fc856a00640 1 -- 172.21.15.69:0/2882829253 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc8580d0b70 con 0x7fc86010a0f0 2024-04-24T09:04:19.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.815+0000 7fc85de00640 1 --2- 172.21.15.69:0/2882829253 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc8440a3bb0 0x7fc8440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:19.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.816+0000 7fc85de00640 1 --2- 172.21.15.69:0/2882829253 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc8440a3bb0 0x7fc8440a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc850000c00 tx=0x7fc8500314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:19.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:19 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3614568595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:19.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:19 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2715026649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:04:19.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:19.819+0000 7fc856a00640 1 -- 172.21.15.69:0/2882829253 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc858014030 con 0x7fc86010a0f0 2024-04-24T09:04:19.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:19 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3614568595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:19.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:19 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2715026649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:04:19.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:19 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3614568595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:19.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:19 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2715026649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:04:20.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.006+0000 7fc85f200640 1 -- 172.21.15.69:0/2882829253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc860062c00 con 0x7fc86010a0f0 2024-04-24T09:04:20.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.007+0000 7fc856a00640 1 -- 172.21.15.69:0/2882829253 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fc85809d2b0 con 0x7fc86010a0f0 2024-04-24T09:04:20.007 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-24T09:04:20.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.009+0000 7fc85f200640 1 -- 172.21.15.69:0/2882829253 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc8440a3bb0 msgr2=0x7fc8440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:20.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.009+0000 7fc85f200640 1 --2- 172.21.15.69:0/2882829253 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc8440a3bb0 0x7fc8440a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc850000c00 tx=0x7fc8500314a0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.009+0000 7fc85f200640 1 -- 172.21.15.69:0/2882829253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86010a0f0 msgr2=0x7fc8601a5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:20.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.009+0000 7fc85f200640 1 --2- 172.21.15.69:0/2882829253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86010a0f0 0x7fc8601a5c00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc85800cce0 tx=0x7fc85800a620 comp rx=0 tx=0).stop 2024-04-24T09:04:20.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.010+0000 7fc85f200640 1 -- 172.21.15.69:0/2882829253 shutdown_connections 2024-04-24T09:04:20.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.010+0000 7fc85f200640 1 --2- 172.21.15.69:0/2882829253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86010a0f0 0x7fc8601a5c00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.010+0000 7fc85f200640 1 --2- 172.21.15.69:0/2882829253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc860109730 0x7fc86019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.010+0000 7fc85f200640 1 --2- 172.21.15.69:0/2882829253 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc8440a3bb0 0x7fc8440a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.010+0000 7fc85f200640 1 --2- 172.21.15.69:0/2882829253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc860108540 0x7fc86019e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.010+0000 7fc85f200640 1 -- 172.21.15.69:0/2882829253 >> 172.21.15.69:0/2882829253 conn(0x7fc8600774d0 msgr2=0x7fc86010c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:20.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.010+0000 7fc85f200640 1 -- 172.21.15.69:0/2882829253 shutdown_connections 2024-04-24T09:04:20.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.010+0000 7fc85f200640 1 -- 172.21.15.69:0/2882829253 wait complete. 2024-04-24T09:04:20.027 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T09:04:20.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.222+0000 7fbd62200640 1 -- 172.21.15.69:0/4197138099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd5c077c00 msgr2=0x7fbd5c078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:20.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.222+0000 7fbd62200640 1 --2- 172.21.15.69:0/4197138099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd5c077c00 0x7fbd5c078060 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009900 tx=0x7fbd4402a410 comp rx=0 tx=0).stop 2024-04-24T09:04:20.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.222+0000 7fbd62200640 1 -- 172.21.15.69:0/4197138099 shutdown_connections 2024-04-24T09:04:20.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.222+0000 7fbd62200640 1 --2- 172.21.15.69:0/4197138099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd5c0785a0 0x7fbd5c078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.222+0000 7fbd62200640 1 --2- 172.21.15.69:0/4197138099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd5c077c00 0x7fbd5c078060 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.222+0000 7fbd62200640 1 --2- 172.21.15.69:0/4197138099 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd5c0769c0 0x7fbd5c076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.222+0000 7fbd62200640 1 -- 172.21.15.69:0/4197138099 >> 172.21.15.69:0/4197138099 conn(0x7fbd5c0ff7b0 msgr2=0x7fbd5c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:20.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.222+0000 7fbd62200640 1 -- 172.21.15.69:0/4197138099 shutdown_connections 2024-04-24T09:04:20.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.222+0000 7fbd62200640 1 -- 172.21.15.69:0/4197138099 wait complete. 2024-04-24T09:04:20.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.223+0000 7fbd62200640 1 Processor -- start 2024-04-24T09:04:20.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.223+0000 7fbd62200640 1 -- start start 2024-04-24T09:04:20.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.223+0000 7fbd62200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd5c0769c0 0x7fbd5c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:20.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.223+0000 7fbd62200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd5c077c00 0x7fbd5c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:20.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.223+0000 7fbd62200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd5c0785a0 0x7fbd5c1a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:20.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.223+0000 7fbd62200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c1141e0 con 0x7fbd5c0769c0 2024-04-24T09:04:20.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.223+0000 7fbd62200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c114060 con 0x7fbd5c077c00 2024-04-24T09:04:20.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.223+0000 7fbd62200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c114360 con 0x7fbd5c0785a0 2024-04-24T09:04:20.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.224+0000 7fbd5b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd5c0769c0 0x7fbd5c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:20.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.224+0000 7fbd5b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd5c0769c0 0x7fbd5c19e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44430/0 (socket says 172.21.15.69:44430) 2024-04-24T09:04:20.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.224+0000 7fbd5b400640 1 -- 172.21.15.69:0/2536508882 learned_addr learned my addr 172.21.15.69:0/2536508882 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:20.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.224+0000 7fbd5aa00640 1 --2- 172.21.15.69:0/2536508882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd5c077c00 0x7fbd5c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:20.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.224+0000 7fbd5be00640 1 --2- 172.21.15.69:0/2536508882 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd5c0785a0 0x7fbd5c1a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:20.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.224+0000 7fbd5b400640 1 -- 172.21.15.69:0/2536508882 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd5c0785a0 msgr2=0x7fbd5c1a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:20.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.224+0000 7fbd5b400640 1 --2- 172.21.15.69:0/2536508882 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd5c0785a0 0x7fbd5c1a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.224+0000 7fbd5b400640 1 -- 172.21.15.69:0/2536508882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd5c077c00 msgr2=0x7fbd5c19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:20.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.224+0000 7fbd5b400640 1 --2- 172.21.15.69:0/2536508882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd5c077c00 0x7fbd5c19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.224+0000 7fbd5b400640 1 -- 172.21.15.69:0/2536508882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5c1a64e0 con 0x7fbd5c0769c0 2024-04-24T09:04:20.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.224+0000 7fbd5aa00640 1 --2- 172.21.15.69:0/2536508882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd5c077c00 0x7fbd5c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:20.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.224+0000 7fbd5be00640 1 --2- 172.21.15.69:0/2536508882 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd5c0785a0 0x7fbd5c1a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:20.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.225+0000 7fbd5b400640 1 --2- 172.21.15.69:0/2536508882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd5c0769c0 0x7fbd5c19e8c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00d770 tx=0x7fbd4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:20.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.225+0000 7fbd53e00640 1 -- 172.21.15.69:0/2536508882 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c017070 con 0x7fbd5c0769c0 2024-04-24T09:04:20.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.225+0000 7fbd53e00640 1 -- 172.21.15.69:0/2536508882 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd4c0054d0 con 0x7fbd5c0769c0 2024-04-24T09:04:20.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.225+0000 7fbd53e00640 1 -- 172.21.15.69:0/2536508882 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c004cf0 con 0x7fbd5c0769c0 2024-04-24T09:04:20.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.225+0000 7fbd62200640 1 -- 172.21.15.69:0/2536508882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5c1a67d0 con 0x7fbd5c0769c0 2024-04-24T09:04:20.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.225+0000 7fbd62200640 1 -- 172.21.15.69:0/2536508882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd5c1a6cd0 con 0x7fbd5c0769c0 2024-04-24T09:04:20.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.226+0000 7fbd62200640 1 -- 172.21.15.69:0/2536508882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd5c110da0 con 0x7fbd5c0769c0 2024-04-24T09:04:20.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.228+0000 7fbd53e00640 1 -- 172.21.15.69:0/2536508882 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbd4c004e90 con 0x7fbd5c0769c0 2024-04-24T09:04:20.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.229+0000 7fbd53e00640 1 --2- 172.21.15.69:0/2536508882 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbd400a3c30 0x7fbd400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:20.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.230+0000 7fbd53e00640 1 -- 172.21.15.69:0/2536508882 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbd4c0d0140 con 0x7fbd5c0769c0 2024-04-24T09:04:20.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.230+0000 7fbd5aa00640 1 --2- 172.21.15.69:0/2536508882 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbd400a3c30 0x7fbd400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:20.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.230+0000 7fbd5aa00640 1 --2- 172.21.15.69:0/2536508882 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbd400a3c30 0x7fbd400a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbd4402a930 tx=0x7fbd44005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:20.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.234+0000 7fbd53e00640 1 -- 172.21.15.69:0/2536508882 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd4c09c880 con 0x7fbd5c0769c0 2024-04-24T09:04:20.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.420+0000 7fbd62200640 1 -- 172.21.15.69:0/2536508882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbd5c062c30 con 0x7fbd5c0769c0 2024-04-24T09:04:20.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.421+0000 7fbd53e00640 1 -- 172.21.15.69:0/2536508882 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7fbd4c09c180 con 0x7fbd5c0769c0 2024-04-24T09:04:20.421 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:04:20.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.424+0000 7fbd62200640 1 -- 172.21.15.69:0/2536508882 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbd400a3c30 msgr2=0x7fbd400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:20.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.424+0000 7fbd62200640 1 --2- 172.21.15.69:0/2536508882 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbd400a3c30 0x7fbd400a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbd4402a930 tx=0x7fbd44005fd0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.424+0000 7fbd62200640 1 -- 172.21.15.69:0/2536508882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd5c0769c0 msgr2=0x7fbd5c19e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:20.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.424+0000 7fbd62200640 1 --2- 172.21.15.69:0/2536508882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd5c0769c0 0x7fbd5c19e8c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00d770 tx=0x7fbd4c00dc30 comp rx=0 tx=0).stop 2024-04-24T09:04:20.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.424+0000 7fbd62200640 1 -- 172.21.15.69:0/2536508882 shutdown_connections 2024-04-24T09:04:20.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.424+0000 7fbd62200640 1 --2- 172.21.15.69:0/2536508882 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd5c0785a0 0x7fbd5c1a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.424+0000 7fbd62200640 1 --2- 172.21.15.69:0/2536508882 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbd400a3c30 0x7fbd400a60f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.424+0000 7fbd62200640 1 --2- 172.21.15.69:0/2536508882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd5c077c00 0x7fbd5c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.424+0000 7fbd62200640 1 --2- 172.21.15.69:0/2536508882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd5c0769c0 0x7fbd5c19e8c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.424+0000 7fbd62200640 1 -- 172.21.15.69:0/2536508882 >> 172.21.15.69:0/2536508882 conn(0x7fbd5c0ff7b0 msgr2=0x7fbd5c100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:20.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.424+0000 7fbd62200640 1 -- 172.21.15.69:0/2536508882 shutdown_connections 2024-04-24T09:04:20.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.424+0000 7fbd62200640 1 -- 172.21.15.69:0/2536508882 wait complete. 2024-04-24T09:04:20.442 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T09:04:20.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.630+0000 7f4217e00640 1 -- 172.21.15.69:0/2759068131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4218108540 msgr2=0x7f4218108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:20.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.630+0000 7f4217e00640 1 --2- 172.21.15.69:0/2759068131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4218108540 0x7f4218108960 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4208009900 tx=0x7f420802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.630+0000 7f4217e00640 1 -- 172.21.15.69:0/2759068131 shutdown_connections 2024-04-24T09:04:20.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.630+0000 7f4217e00640 1 --2- 172.21.15.69:0/2759068131 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f421810a0f0 0x7f4218110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.630+0000 7f4217e00640 1 --2- 172.21.15.69:0/2759068131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4218109730 0x7f4218109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.630+0000 7f4217e00640 1 --2- 172.21.15.69:0/2759068131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4218108540 0x7f4218108960 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.630+0000 7f4217e00640 1 -- 172.21.15.69:0/2759068131 >> 172.21.15.69:0/2759068131 conn(0x7f42180774d0 msgr2=0x7f4218079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:20.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.630+0000 7f4217e00640 1 -- 172.21.15.69:0/2759068131 shutdown_connections 2024-04-24T09:04:20.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.630+0000 7f4217e00640 1 -- 172.21.15.69:0/2759068131 wait complete. 2024-04-24T09:04:20.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.631+0000 7f4217e00640 1 Processor -- start 2024-04-24T09:04:20.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.631+0000 7f4217e00640 1 -- start start 2024-04-24T09:04:20.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f4217e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4218108540 0x7f421819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:20.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f4217e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4218109730 0x7f421819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:20.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f4217e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f421810a0f0 0x7f42181a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:20.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f4217e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4218114080 con 0x7f4218109730 2024-04-24T09:04:20.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f4217e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4218113f00 con 0x7f421810a0f0 2024-04-24T09:04:20.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f4217e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4218114200 con 0x7f4218108540 2024-04-24T09:04:20.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f420e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4218109730 0x7f421819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:20.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f420e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4218109730 0x7f421819ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44446/0 (socket says 172.21.15.69:44446) 2024-04-24T09:04:20.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f420e000640 1 -- 172.21.15.69:0/3802430330 learned_addr learned my addr 172.21.15.69:0/3802430330 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:20.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f4217400640 1 --2- 172.21.15.69:0/3802430330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f421810a0f0 0x7f42181a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:20.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f4216a00640 1 --2- 172.21.15.69:0/3802430330 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4218108540 0x7f421819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:20.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f420e000640 1 -- 172.21.15.69:0/3802430330 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4218108540 msgr2=0x7f421819e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f420e000640 1 --2- 172.21.15.69:0/3802430330 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4218108540 0x7f421819e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f420e000640 1 -- 172.21.15.69:0/3802430330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f421810a0f0 msgr2=0x7f42181a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f420e000640 1 --2- 172.21.15.69:0/3802430330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f421810a0f0 0x7f42181a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f420e000640 1 -- 172.21.15.69:0/3802430330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42181a6300 con 0x7f4218109730 2024-04-24T09:04:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.632+0000 7f4216a00640 1 --2- 172.21.15.69:0/3802430330 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4218108540 0x7f421819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.633+0000 7f420e000640 1 --2- 172.21.15.69:0/3802430330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4218109730 0x7f421819ec70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f420000c920 tx=0x7f420000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.633+0000 7f420fe00640 1 -- 172.21.15.69:0/3802430330 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200007c70 con 0x7f4218109730 2024-04-24T09:04:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.633+0000 7f420fe00640 1 -- 172.21.15.69:0/3802430330 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4200007e10 con 0x7f4218109730 2024-04-24T09:04:20.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.633+0000 7f4217e00640 1 -- 172.21.15.69:0/3802430330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42181a65f0 con 0x7f4218109730 2024-04-24T09:04:20.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.633+0000 7f420fe00640 1 -- 172.21.15.69:0/3802430330 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200018820 con 0x7f4218109730 2024-04-24T09:04:20.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.633+0000 7f4217e00640 1 -- 172.21.15.69:0/3802430330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42181a6b90 con 0x7f4218109730 2024-04-24T09:04:20.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.634+0000 7f4217e00640 1 -- 172.21.15.69:0/3802430330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421810f860 con 0x7f4218109730 2024-04-24T09:04:20.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.636+0000 7f420fe00640 1 -- 172.21.15.69:0/3802430330 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f42000189c0 con 0x7f4218109730 2024-04-24T09:04:20.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.638+0000 7f420fe00640 1 --2- 172.21.15.69:0/3802430330 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f41fc0a3c30 0x7f41fc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:20.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.638+0000 7f420fe00640 1 -- 172.21.15.69:0/3802430330 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f42000d0840 con 0x7f4218109730 2024-04-24T09:04:20.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.638+0000 7f4216a00640 1 --2- 172.21.15.69:0/3802430330 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f41fc0a3c30 0x7f41fc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:20.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.639+0000 7f4216a00640 1 --2- 172.21.15.69:0/3802430330 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f41fc0a3c30 0x7f41fc0a60f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4208006010 tx=0x7f4208000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:20.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.642+0000 7f420fe00640 1 -- 172.21.15.69:0/3802430330 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4200014030 con 0x7f4218109730 2024-04-24T09:04:20.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:20 smithi081 ceph-mon[30961]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T09:04:20.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:20 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2685472363' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:04:20.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:20 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2882829253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:04:20.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:20 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2536508882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:04:20.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:20 smithi069 ceph-mon[28170]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T09:04:20.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:20 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2685472363' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:04:20.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:20 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2882829253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:04:20.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:20 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2536508882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:04:20.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.859+0000 7f4217e00640 1 -- 172.21.15.69:0/3802430330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f421802d620 con 0x7f4218109730 2024-04-24T09:04:20.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.860+0000 7f420fe00640 1 -- 172.21.15.69:0/3802430330 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+592 (secure 0 0 0) 0x7f420009cf80 con 0x7f4218109730 2024-04-24T09:04:20.860 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3826117580 2024-04-25T08:54:23.196411+0000 2024-04-24T09:04:20.860 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1006368686 2024-04-25T08:54:23.196411+0000 2024-04-24T09:04:20.860 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/3470783122 2024-04-25T08:54:23.196411+0000 2024-04-24T09:04:20.860 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2102503077 2024-04-25T08:53:48.948375+0000 2024-04-24T09:04:20.860 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/3470783122 2024-04-25T08:54:23.196411+0000 2024-04-24T09:04:20.860 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1041454348 2024-04-25T08:53:48.948375+0000 2024-04-24T09:04:20.861 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/3911821813 2024-04-25T08:53:48.948375+0000 2024-04-24T09:04:20.861 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/3911821813 2024-04-25T08:53:48.948375+0000 2024-04-24T09:04:20.861 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/4072155336 2024-04-25T08:53:48.948375+0000 2024-04-24T09:04:20.861 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1278076203 2024-04-25T08:54:23.196411+0000 2024-04-24T09:04:20.861 INFO:teuthology.orchestra.run.smithi069.stderr:listed 10 entries 2024-04-24T09:04:20.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.863+0000 7f4217e00640 1 -- 172.21.15.69:0/3802430330 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f41fc0a3c30 msgr2=0x7f41fc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:20.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.863+0000 7f4217e00640 1 --2- 172.21.15.69:0/3802430330 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f41fc0a3c30 0x7f41fc0a60f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4208006010 tx=0x7f4208000bc0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.863+0000 7f4217e00640 1 -- 172.21.15.69:0/3802430330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4218109730 msgr2=0x7f421819ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:20.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.863+0000 7f4217e00640 1 --2- 172.21.15.69:0/3802430330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4218109730 0x7f421819ec70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f420000c920 tx=0x7f420000cde0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.863+0000 7f4217e00640 1 -- 172.21.15.69:0/3802430330 shutdown_connections 2024-04-24T09:04:20.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.863+0000 7f4217e00640 1 --2- 172.21.15.69:0/3802430330 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f41fc0a3c30 0x7f41fc0a60f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.863+0000 7f4217e00640 1 --2- 172.21.15.69:0/3802430330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f421810a0f0 0x7f42181a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.863+0000 7f4217e00640 1 --2- 172.21.15.69:0/3802430330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4218109730 0x7f421819ec70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.863+0000 7f4217e00640 1 --2- 172.21.15.69:0/3802430330 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4218108540 0x7f421819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:20.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.863+0000 7f4217e00640 1 -- 172.21.15.69:0/3802430330 >> 172.21.15.69:0/3802430330 conn(0x7f42180774d0 msgr2=0x7f421810ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:20.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.863+0000 7f4217e00640 1 -- 172.21.15.69:0/3802430330 shutdown_connections 2024-04-24T09:04:20.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:20.863+0000 7f4217e00640 1 -- 172.21.15.69:0/3802430330 wait complete. 2024-04-24T09:04:20.881 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T09:04:20.881 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T09:04:20.931 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:04:20.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:20 smithi130 ceph-mon[31553]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T09:04:20.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:20 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2685472363' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:04:20.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:20 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2882829253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:04:20.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:20 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2536508882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:04:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.182+0000 7ff44e600640 1 -- 172.21.15.69:0/3129387866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff448101f70 msgr2=0x7ff4481023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.182+0000 7ff44e600640 1 --2- 172.21.15.69:0/3129387866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff448101f70 0x7ff4481023f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff434009900 tx=0x7ff43402a410 comp rx=0 tx=0).stop 2024-04-24T09:04:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.182+0000 7ff44e600640 1 -- 172.21.15.69:0/3129387866 shutdown_connections 2024-04-24T09:04:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.182+0000 7ff44e600640 1 --2- 172.21.15.69:0/3129387866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff448102930 0x7ff4481091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.182+0000 7ff44e600640 1 --2- 172.21.15.69:0/3129387866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff448101f70 0x7ff4481023f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.182+0000 7ff44e600640 1 --2- 172.21.15.69:0/3129387866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff448100d80 0x7ff4481011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.182+0000 7ff44e600640 1 -- 172.21.15.69:0/3129387866 >> 172.21.15.69:0/3129387866 conn(0x7ff4480fc2d0 msgr2=0x7ff4480fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:21.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.182+0000 7ff44e600640 1 -- 172.21.15.69:0/3129387866 shutdown_connections 2024-04-24T09:04:21.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.182+0000 7ff44e600640 1 -- 172.21.15.69:0/3129387866 wait complete. 2024-04-24T09:04:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.183+0000 7ff44e600640 1 Processor -- start 2024-04-24T09:04:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.183+0000 7ff44e600640 1 -- start start 2024-04-24T09:04:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.184+0000 7ff44e600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff448100d80 0x7ff44819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.184+0000 7ff44e600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff448101f70 0x7ff44819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.184+0000 7ff44e600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff448102930 0x7ff4481a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.184+0000 7ff44e600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff44810d9e0 con 0x7ff448101f70 2024-04-24T09:04:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.184+0000 7ff44e600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff44810d860 con 0x7ff448102930 2024-04-24T09:04:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.184+0000 7ff44e600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff44810db60 con 0x7ff448100d80 2024-04-24T09:04:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.184+0000 7ff447e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff448101f70 0x7ff44819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.184+0000 7ff447e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff448101f70 0x7ff44819a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44472/0 (socket says 172.21.15.69:44472) 2024-04-24T09:04:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.184+0000 7ff447e00640 1 -- 172.21.15.69:0/3211221429 learned_addr learned my addr 172.21.15.69:0/3211221429 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.184+0000 7ff44dc00640 1 --2- 172.21.15.69:0/3211221429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff448102930 0x7ff4481a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.184+0000 7ff44d200640 1 --2- 172.21.15.69:0/3211221429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff448100d80 0x7ff44819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.185+0000 7ff447e00640 1 -- 172.21.15.69:0/3211221429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff448100d80 msgr2=0x7ff44819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.185+0000 7ff447e00640 1 --2- 172.21.15.69:0/3211221429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff448100d80 0x7ff44819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.185+0000 7ff447e00640 1 -- 172.21.15.69:0/3211221429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff448102930 msgr2=0x7ff4481a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.185+0000 7ff447e00640 1 --2- 172.21.15.69:0/3211221429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff448102930 0x7ff4481a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.185+0000 7ff447e00640 1 -- 172.21.15.69:0/3211221429 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4481a1ed0 con 0x7ff448101f70 2024-04-24T09:04:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.185+0000 7ff447e00640 1 --2- 172.21.15.69:0/3211221429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff448101f70 0x7ff44819a7f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff434005940 tx=0x7ff4340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.185+0000 7ff445600640 1 -- 172.21.15.69:0/3211221429 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43403f030 con 0x7ff448101f70 2024-04-24T09:04:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.185+0000 7ff445600640 1 -- 172.21.15.69:0/3211221429 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4340318f0 con 0x7ff448101f70 2024-04-24T09:04:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.185+0000 7ff445600640 1 -- 172.21.15.69:0/3211221429 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434035070 con 0x7ff448101f70 2024-04-24T09:04:21.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.185+0000 7ff44e600640 1 -- 172.21.15.69:0/3211221429 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4481a2160 con 0x7ff448101f70 2024-04-24T09:04:21.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.185+0000 7ff44e600640 1 -- 172.21.15.69:0/3211221429 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4481a2600 con 0x7ff448101f70 2024-04-24T09:04:21.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.187+0000 7ff44e600640 1 -- 172.21.15.69:0/3211221429 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4481a2920 con 0x7ff448101f70 2024-04-24T09:04:21.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.189+0000 7ff445600640 1 -- 172.21.15.69:0/3211221429 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff434042030 con 0x7ff448101f70 2024-04-24T09:04:21.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.191+0000 7ff445600640 1 --2- 172.21.15.69:0/3211221429 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff4300a3c30 0x7ff4300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.191+0000 7ff445600640 1 -- 172.21.15.69:0/3211221429 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff4340f0bf0 con 0x7ff448101f70 2024-04-24T09:04:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.191+0000 7ff44d200640 1 --2- 172.21.15.69:0/3211221429 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff4300a3c30 0x7ff4300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.192+0000 7ff44d200640 1 --2- 172.21.15.69:0/3211221429 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff4300a3c30 0x7ff4300a60f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff448101dd0 tx=0x7ff43c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:21.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.195+0000 7ff445600640 1 -- 172.21.15.69:0/3211221429 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4340bd330 con 0x7ff448101f70 2024-04-24T09:04:21.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.398+0000 7ff44e600640 1 -- 172.21.15.69:0/3211221429 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4481a2af0 con 0x7ff448101f70 2024-04-24T09:04:21.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.402+0000 7ff445600640 1 -- 172.21.15.69:0/3211221429 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24094 (secure 0 0 0) 0x7ff4340bcc30 con 0x7ff448101f70 2024-04-24T09:04:21.402 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:21.402 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":90,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:06.590427+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:04:21.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.405+0000 7ff44e600640 1 -- 172.21.15.69:0/3211221429 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff4300a3c30 msgr2=0x7ff4300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:21.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.405+0000 7ff44e600640 1 --2- 172.21.15.69:0/3211221429 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff4300a3c30 0x7ff4300a60f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff448101dd0 tx=0x7ff43c005f90 comp rx=0 tx=0).stop 2024-04-24T09:04:21.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.405+0000 7ff44e600640 1 -- 172.21.15.69:0/3211221429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff448101f70 msgr2=0x7ff44819a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:21.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.405+0000 7ff44e600640 1 --2- 172.21.15.69:0/3211221429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff448101f70 0x7ff44819a7f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff434005940 tx=0x7ff4340042f0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.405+0000 7ff44e600640 1 -- 172.21.15.69:0/3211221429 shutdown_connections 2024-04-24T09:04:21.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.405+0000 7ff44e600640 1 --2- 172.21.15.69:0/3211221429 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff4300a3c30 0x7ff4300a60f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.405+0000 7ff44e600640 1 --2- 172.21.15.69:0/3211221429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff448102930 0x7ff4481a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.405+0000 7ff44e600640 1 --2- 172.21.15.69:0/3211221429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff448101f70 0x7ff44819a7f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.405+0000 7ff44e600640 1 --2- 172.21.15.69:0/3211221429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff448100d80 0x7ff44819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.405+0000 7ff44e600640 1 -- 172.21.15.69:0/3211221429 >> 172.21.15.69:0/3211221429 conn(0x7ff4480fc2d0 msgr2=0x7ff448109b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:21.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.405+0000 7ff44e600640 1 -- 172.21.15.69:0/3211221429 shutdown_connections 2024-04-24T09:04:21.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.405+0000 7ff44e600640 1 -- 172.21.15.69:0/3211221429 wait complete. 2024-04-24T09:04:21.424 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:57:37.377599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:04:00.299827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:04:00.958498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T09:04:04.105314+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T09:04:21.425 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:04:21.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.632+0000 7fd1b3e00640 1 -- 172.21.15.69:0/2721270529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1b4101f10 msgr2=0x7fd1b4102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:21.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.632+0000 7fd1b3e00640 1 --2- 172.21.15.69:0/2721270529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1b4101f10 0x7fd1b4102330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009ca0 tx=0x7fd1a402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.632+0000 7fd1b3e00640 1 -- 172.21.15.69:0/2721270529 shutdown_connections 2024-04-24T09:04:21.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.632+0000 7fd1b3e00640 1 --2- 172.21.15.69:0/2721270529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1b4103a20 0x7fd1b410a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.632+0000 7fd1b3e00640 1 --2- 172.21.15.69:0/2721270529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd1b4103060 0x7fd1b41034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.632+0000 7fd1b3e00640 1 --2- 172.21.15.69:0/2721270529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1b4101f10 0x7fd1b4102330 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.632+0000 7fd1b3e00640 1 -- 172.21.15.69:0/2721270529 >> 172.21.15.69:0/2721270529 conn(0x7fd1b40fd4a0 msgr2=0x7fd1b40ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:21.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.632+0000 7fd1b3e00640 1 -- 172.21.15.69:0/2721270529 shutdown_connections 2024-04-24T09:04:21.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.633+0000 7fd1b3e00640 1 -- 172.21.15.69:0/2721270529 wait complete. 2024-04-24T09:04:21.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.633+0000 7fd1b3e00640 1 Processor -- start 2024-04-24T09:04:21.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.634+0000 7fd1b3e00640 1 -- start start 2024-04-24T09:04:21.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.634+0000 7fd1b3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1b4101f10 0x7fd1b419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:21.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.634+0000 7fd1b3e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1b4103060 0x7fd1b419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:21.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.634+0000 7fd1b3e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd1b4103a20 0x7fd1b41a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:21.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.634+0000 7fd1b3e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b410da50 con 0x7fd1b4101f10 2024-04-24T09:04:21.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.634+0000 7fd1b3e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b410d8d0 con 0x7fd1b4103060 2024-04-24T09:04:21.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.634+0000 7fd1b3e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b410dbd0 con 0x7fd1b4103a20 2024-04-24T09:04:21.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.634+0000 7fd1b2a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1b4101f10 0x7fd1b419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:21.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.634+0000 7fd1b2a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1b4101f10 0x7fd1b419a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44494/0 (socket says 172.21.15.69:44494) 2024-04-24T09:04:21.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.634+0000 7fd1b2a00640 1 -- 172.21.15.69:0/1414783600 learned_addr learned my addr 172.21.15.69:0/1414783600 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:21.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.635+0000 7fd1aa000640 1 --2- 172.21.15.69:0/1414783600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1b4103060 0x7fd1b419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:21.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.635+0000 7fd1b3400640 1 --2- 172.21.15.69:0/1414783600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd1b4103a20 0x7fd1b41a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:21.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.635+0000 7fd1b2a00640 1 -- 172.21.15.69:0/1414783600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd1b4103a20 msgr2=0x7fd1b41a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:21.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.635+0000 7fd1b2a00640 1 --2- 172.21.15.69:0/1414783600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd1b4103a20 0x7fd1b41a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.635+0000 7fd1b2a00640 1 -- 172.21.15.69:0/1414783600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1b4103060 msgr2=0x7fd1b419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:21.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.635+0000 7fd1b2a00640 1 --2- 172.21.15.69:0/1414783600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1b4103060 0x7fd1b419a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.635+0000 7fd1b2a00640 1 -- 172.21.15.69:0/1414783600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b41a1f00 con 0x7fd1b4101f10 2024-04-24T09:04:21.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.635+0000 7fd1aa000640 1 --2- 172.21.15.69:0/1414783600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1b4103060 0x7fd1b419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:21.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.635+0000 7fd1b3400640 1 --2- 172.21.15.69:0/1414783600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd1b4103a20 0x7fd1b41a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:21.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.635+0000 7fd1b2a00640 1 --2- 172.21.15.69:0/1414783600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1b4101f10 0x7fd1b419a2e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009dc0 tx=0x7fd1a4033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:21.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.635+0000 7fd1abe00640 1 -- 172.21.15.69:0/1414783600 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4004240 con 0x7fd1b4101f10 2024-04-24T09:04:21.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.635+0000 7fd1abe00640 1 -- 172.21.15.69:0/1414783600 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1a4003e10 con 0x7fd1b4101f10 2024-04-24T09:04:21.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.635+0000 7fd1abe00640 1 -- 172.21.15.69:0/1414783600 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a402f4c0 con 0x7fd1b4101f10 2024-04-24T09:04:21.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.635+0000 7fd1b3e00640 1 -- 172.21.15.69:0/1414783600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b41a2190 con 0x7fd1b4101f10 2024-04-24T09:04:21.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.635+0000 7fd1b3e00640 1 -- 172.21.15.69:0/1414783600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b41a2730 con 0x7fd1b4101f10 2024-04-24T09:04:21.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.637+0000 7fd1b3e00640 1 -- 172.21.15.69:0/1414783600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b41a2990 con 0x7fd1b4101f10 2024-04-24T09:04:21.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.639+0000 7fd1abe00640 1 -- 172.21.15.69:0/1414783600 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd1a4030730 con 0x7fd1b4101f10 2024-04-24T09:04:21.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.640+0000 7fd1abe00640 1 --2- 172.21.15.69:0/1414783600 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd1980a3c30 0x7fd1980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:21.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.640+0000 7fd1abe00640 1 -- 172.21.15.69:0/1414783600 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd1a40f1e90 con 0x7fd1b4101f10 2024-04-24T09:04:21.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.640+0000 7fd1aa000640 1 --2- 172.21.15.69:0/1414783600 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd1980a3c30 0x7fd1980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:21.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.641+0000 7fd1aa000640 1 --2- 172.21.15.69:0/1414783600 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd1980a3c30 0x7fd1980a60f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd1b419b7f0 tx=0x7fd19c005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:21.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.644+0000 7fd1abe00640 1 -- 172.21.15.69:0/1414783600 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1a40443d0 con 0x7fd1b4101f10 2024-04-24T09:04:21.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:21 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3802430330' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:04:21.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:21 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3211221429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:21.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:21 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3802430330' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:04:21.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:21 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3211221429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:21.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.844+0000 7fd1b3e00640 1 -- 172.21.15.69:0/1414783600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd1b41a2b60 con 0x7fd1b4101f10 2024-04-24T09:04:21.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.846+0000 7fd1abe00640 1 -- 172.21.15.69:0/1414783600 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fd1a40be5d0 con 0x7fd1b4101f10 2024-04-24T09:04:21.845 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-24T09:04:21.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.848+0000 7fd1b3e00640 1 -- 172.21.15.69:0/1414783600 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd1980a3c30 msgr2=0x7fd1980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:21.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.848+0000 7fd1b3e00640 1 --2- 172.21.15.69:0/1414783600 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd1980a3c30 0x7fd1980a60f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd1b419b7f0 tx=0x7fd19c005db0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.848+0000 7fd1b3e00640 1 -- 172.21.15.69:0/1414783600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1b4101f10 msgr2=0x7fd1b419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:21.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.848+0000 7fd1b3e00640 1 --2- 172.21.15.69:0/1414783600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1b4101f10 0x7fd1b419a2e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009dc0 tx=0x7fd1a4033000 comp rx=0 tx=0).stop 2024-04-24T09:04:21.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.848+0000 7fd1b3e00640 1 -- 172.21.15.69:0/1414783600 shutdown_connections 2024-04-24T09:04:21.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.848+0000 7fd1b3e00640 1 --2- 172.21.15.69:0/1414783600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd1b4103a20 0x7fd1b41a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.848+0000 7fd1b3e00640 1 --2- 172.21.15.69:0/1414783600 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd1980a3c30 0x7fd1980a60f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.848+0000 7fd1b3e00640 1 --2- 172.21.15.69:0/1414783600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1b4103060 0x7fd1b419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.848+0000 7fd1b3e00640 1 --2- 172.21.15.69:0/1414783600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1b4101f10 0x7fd1b419a2e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:21.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.848+0000 7fd1b3e00640 1 -- 172.21.15.69:0/1414783600 >> 172.21.15.69:0/1414783600 conn(0x7fd1b40fd4a0 msgr2=0x7fd1b4074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:21.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.849+0000 7fd1b3e00640 1 -- 172.21.15.69:0/1414783600 shutdown_connections 2024-04-24T09:04:21.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:21.849+0000 7fd1b3e00640 1 -- 172.21.15.69:0/1414783600 wait complete. 2024-04-24T09:04:21.867 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:04:21.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:21 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3802430330' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:04:21.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:21 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3211221429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:22.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.075+0000 7f7e9f400640 1 -- 172.21.15.69:0/234461677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e98105470 msgr2=0x7f7e981058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:22.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.075+0000 7f7e9f400640 1 --2- 172.21.15.69:0/234461677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e98105470 0x7f7e981058f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009900 tx=0x7f7e8402a410 comp rx=0 tx=0).stop 2024-04-24T09:04:22.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.076+0000 7f7e9f400640 1 -- 172.21.15.69:0/234461677 shutdown_connections 2024-04-24T09:04:22.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.076+0000 7f7e9f400640 1 --2- 172.21.15.69:0/234461677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e98105e30 0x7f7e9810c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.076+0000 7f7e9f400640 1 --2- 172.21.15.69:0/234461677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e98105470 0x7f7e981058f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.076+0000 7f7e9f400640 1 --2- 172.21.15.69:0/234461677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e98104280 0x7f7e981046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.076+0000 7f7e9f400640 1 -- 172.21.15.69:0/234461677 >> 172.21.15.69:0/234461677 conn(0x7f7e980ff7f0 msgr2=0x7f7e98101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:22.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.076+0000 7f7e9f400640 1 -- 172.21.15.69:0/234461677 shutdown_connections 2024-04-24T09:04:22.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.076+0000 7f7e9f400640 1 -- 172.21.15.69:0/234461677 wait complete. 2024-04-24T09:04:22.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.077+0000 7f7e9f400640 1 Processor -- start 2024-04-24T09:04:22.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.077+0000 7f7e9f400640 1 -- start start 2024-04-24T09:04:22.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.077+0000 7f7e9f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e98104280 0x7f7e9819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:22.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.077+0000 7f7e9f400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e98105470 0x7f7e9819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:22.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.077+0000 7f7e9f400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e98105e30 0x7f7e981a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:22.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.077+0000 7f7e9f400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9810fe10 con 0x7f7e98104280 2024-04-24T09:04:22.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.077+0000 7f7e9f400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9810fc90 con 0x7f7e98105e30 2024-04-24T09:04:22.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.077+0000 7f7e9f400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9810ff90 con 0x7f7e98105470 2024-04-24T09:04:22.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.078+0000 7f7e9ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e98104280 0x7f7e9819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:22.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.078+0000 7f7e9ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e98104280 0x7f7e9819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44508/0 (socket says 172.21.15.69:44508) 2024-04-24T09:04:22.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.078+0000 7f7e9ca00640 1 -- 172.21.15.69:0/507595327 learned_addr learned my addr 172.21.15.69:0/507595327 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:22.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.078+0000 7f7e97e00640 1 --2- 172.21.15.69:0/507595327 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e98105470 0x7f7e9819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:22.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.078+0000 7f7e9d400640 1 --2- 172.21.15.69:0/507595327 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e98105e30 0x7f7e981a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:22.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.078+0000 7f7e9ca00640 1 -- 172.21.15.69:0/507595327 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e98105470 msgr2=0x7f7e9819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:22.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.078+0000 7f7e9ca00640 1 --2- 172.21.15.69:0/507595327 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e98105470 0x7f7e9819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.078+0000 7f7e9ca00640 1 -- 172.21.15.69:0/507595327 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e98105e30 msgr2=0x7f7e981a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:22.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.078+0000 7f7e9ca00640 1 --2- 172.21.15.69:0/507595327 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e98105e30 0x7f7e981a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.078+0000 7f7e9ca00640 1 -- 172.21.15.69:0/507595327 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e981a20c0 con 0x7f7e98104280 2024-04-24T09:04:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.078+0000 7f7e97e00640 1 --2- 172.21.15.69:0/507595327 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e98105470 0x7f7e9819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:04:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.078+0000 7f7e9ca00640 1 --2- 172.21.15.69:0/507595327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e98104280 0x7f7e9819a4a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00ea80 tx=0x7f7e8c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.079+0000 7f7e95600640 1 -- 172.21.15.69:0/507595327 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c00ce40 con 0x7f7e98104280 2024-04-24T09:04:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.079+0000 7f7e95600640 1 -- 172.21.15.69:0/507595327 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e8c011920 con 0x7f7e98104280 2024-04-24T09:04:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.079+0000 7f7e9f400640 1 -- 172.21.15.69:0/507595327 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e981a23b0 con 0x7f7e98104280 2024-04-24T09:04:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.079+0000 7f7e95600640 1 -- 172.21.15.69:0/507595327 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c01aaa0 con 0x7f7e98104280 2024-04-24T09:04:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.079+0000 7f7e9f400640 1 -- 172.21.15.69:0/507595327 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e981a28d0 con 0x7f7e98104280 2024-04-24T09:04:22.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.080+0000 7f7e9f400640 1 -- 172.21.15.69:0/507595327 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e9810b5f0 con 0x7f7e98104280 2024-04-24T09:04:22.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.082+0000 7f7e95600640 1 -- 172.21.15.69:0/507595327 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7e8c011ad0 con 0x7f7e98104280 2024-04-24T09:04:22.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.083+0000 7f7e95600640 1 --2- 172.21.15.69:0/507595327 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7e800a3bb0 0x7f7e800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:22.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.084+0000 7f7e95600640 1 -- 172.21.15.69:0/507595327 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7e8c0d0d00 con 0x7f7e98104280 2024-04-24T09:04:22.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.084+0000 7f7e97e00640 1 --2- 172.21.15.69:0/507595327 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7e800a3bb0 0x7f7e800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:22.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.084+0000 7f7e97e00640 1 --2- 172.21.15.69:0/507595327 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7e800a3bb0 0x7f7e800a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7e84000c00 tx=0x7f7e840314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:22.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.088+0000 7f7e95600640 1 -- 172.21.15.69:0/507595327 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e8c016030 con 0x7f7e98104280 2024-04-24T09:04:22.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.296+0000 7f7e9f400640 1 -- 172.21.15.69:0/507595327 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7e98062c00 con 0x7f7e98104280 2024-04-24T09:04:22.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.297+0000 7f7e95600640 1 -- 172.21.15.69:0/507595327 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f7e8c09d440 con 0x7f7e98104280 2024-04-24T09:04:22.296 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:04:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.299+0000 7f7e9f400640 1 -- 172.21.15.69:0/507595327 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7e800a3bb0 msgr2=0x7f7e800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.299+0000 7f7e9f400640 1 --2- 172.21.15.69:0/507595327 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7e800a3bb0 0x7f7e800a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7e84000c00 tx=0x7f7e840314a0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.299+0000 7f7e9f400640 1 -- 172.21.15.69:0/507595327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e98104280 msgr2=0x7f7e9819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.299+0000 7f7e9f400640 1 --2- 172.21.15.69:0/507595327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e98104280 0x7f7e9819a4a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00ea80 tx=0x7f7e8c00ef40 comp rx=0 tx=0).stop 2024-04-24T09:04:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.299+0000 7f7e9f400640 1 -- 172.21.15.69:0/507595327 shutdown_connections 2024-04-24T09:04:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.299+0000 7f7e9f400640 1 --2- 172.21.15.69:0/507595327 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7e800a3bb0 0x7f7e800a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.299+0000 7f7e9f400640 1 --2- 172.21.15.69:0/507595327 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e98105e30 0x7f7e981a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.299+0000 7f7e9f400640 1 --2- 172.21.15.69:0/507595327 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e98105470 0x7f7e9819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.299+0000 7f7e9f400640 1 --2- 172.21.15.69:0/507595327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e98104280 0x7f7e9819a4a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.299+0000 7f7e9f400640 1 -- 172.21.15.69:0/507595327 >> 172.21.15.69:0/507595327 conn(0x7f7e980ff7f0 msgr2=0x7f7e981087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.300+0000 7f7e9f400640 1 -- 172.21.15.69:0/507595327 shutdown_connections 2024-04-24T09:04:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.300+0000 7f7e9f400640 1 -- 172.21.15.69:0/507595327 wait complete. 2024-04-24T09:04:22.319 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:04:22.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.520+0000 7fc13a000640 1 -- 172.21.15.69:0/4102041258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc134101f30 msgr2=0x7fc134102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:22.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.520+0000 7fc13a000640 1 --2- 172.21.15.69:0/4102041258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc134101f30 0x7fc134102350 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc128009900 tx=0x7fc12802a410 comp rx=0 tx=0).stop 2024-04-24T09:04:22.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.521+0000 7fc13a000640 1 -- 172.21.15.69:0/4102041258 shutdown_connections 2024-04-24T09:04:22.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.521+0000 7fc13a000640 1 --2- 172.21.15.69:0/4102041258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc134103a40 0x7fc13410a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.521+0000 7fc13a000640 1 --2- 172.21.15.69:0/4102041258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc134103080 0x7fc134103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.521+0000 7fc13a000640 1 --2- 172.21.15.69:0/4102041258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc134101f30 0x7fc134102350 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.521+0000 7fc13a000640 1 -- 172.21.15.69:0/4102041258 >> 172.21.15.69:0/4102041258 conn(0x7fc1340fd4a0 msgr2=0x7fc1340ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:22.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.521+0000 7fc13a000640 1 -- 172.21.15.69:0/4102041258 shutdown_connections 2024-04-24T09:04:22.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.521+0000 7fc13a000640 1 -- 172.21.15.69:0/4102041258 wait complete. 2024-04-24T09:04:22.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.522+0000 7fc13a000640 1 Processor -- start 2024-04-24T09:04:22.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.522+0000 7fc13a000640 1 -- start start 2024-04-24T09:04:22.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.522+0000 7fc13a000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc134101f30 0x7fc13419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:22.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.522+0000 7fc13a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc134103080 0x7fc13419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:22.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.522+0000 7fc13a000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc134103a40 0x7fc1341a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:22.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.522+0000 7fc13a000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc13410da90 con 0x7fc134103080 2024-04-24T09:04:22.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.522+0000 7fc13a000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc13410d910 con 0x7fc134103a40 2024-04-24T09:04:22.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.522+0000 7fc13a000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc13410dc10 con 0x7fc134101f30 2024-04-24T09:04:22.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.522+0000 7fc133e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc134103080 0x7fc13419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:22.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.523+0000 7fc133e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc134103080 0x7fc13419a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44524/0 (socket says 172.21.15.69:44524) 2024-04-24T09:04:22.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.523+0000 7fc133e00640 1 -- 172.21.15.69:0/207770939 learned_addr learned my addr 172.21.15.69:0/207770939 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:22.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.523+0000 7fc138c00640 1 --2- 172.21.15.69:0/207770939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc134101f30 0x7fc13419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:22.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.523+0000 7fc133e00640 1 -- 172.21.15.69:0/207770939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc134101f30 msgr2=0x7fc13419a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:22.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.523+0000 7fc139600640 1 --2- 172.21.15.69:0/207770939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc134103a40 0x7fc1341a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:22.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.523+0000 7fc133e00640 1 --2- 172.21.15.69:0/207770939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc134101f30 0x7fc13419a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.523+0000 7fc133e00640 1 -- 172.21.15.69:0/207770939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc134103a40 msgr2=0x7fc1341a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:22.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.523+0000 7fc133e00640 1 --2- 172.21.15.69:0/207770939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc134103a40 0x7fc1341a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.523+0000 7fc133e00640 1 -- 172.21.15.69:0/207770939 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1341a1f60 con 0x7fc134103080 2024-04-24T09:04:22.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.523+0000 7fc138c00640 1 --2- 172.21.15.69:0/207770939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc134101f30 0x7fc13419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:22.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.523+0000 7fc133e00640 1 --2- 172.21.15.69:0/207770939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc134103080 0x7fc13419a880 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc12000d770 tx=0x7fc12000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:22.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.523+0000 7fc131600640 1 -- 172.21.15.69:0/207770939 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120017070 con 0x7fc134103080 2024-04-24T09:04:22.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.524+0000 7fc131600640 1 -- 172.21.15.69:0/207770939 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc120009c30 con 0x7fc134103080 2024-04-24T09:04:22.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.524+0000 7fc131600640 1 -- 172.21.15.69:0/207770939 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120003d00 con 0x7fc134103080 2024-04-24T09:04:22.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.524+0000 7fc13a000640 1 -- 172.21.15.69:0/207770939 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1341a2250 con 0x7fc134103080 2024-04-24T09:04:22.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.524+0000 7fc13a000640 1 -- 172.21.15.69:0/207770939 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1341a2750 con 0x7fc134103080 2024-04-24T09:04:22.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.525+0000 7fc13a000640 1 -- 172.21.15.69:0/207770939 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1341a2a70 con 0x7fc134103080 2024-04-24T09:04:22.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.528+0000 7fc131600640 1 -- 172.21.15.69:0/207770939 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc120003ea0 con 0x7fc134103080 2024-04-24T09:04:22.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.532+0000 7fc131600640 1 --2- 172.21.15.69:0/207770939 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc11c0a3c30 0x7fc11c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:22.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.532+0000 7fc131600640 1 -- 172.21.15.69:0/207770939 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc1200d0380 con 0x7fc134103080 2024-04-24T09:04:22.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.532+0000 7fc138c00640 1 --2- 172.21.15.69:0/207770939 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc11c0a3c30 0x7fc11c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:22.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.533+0000 7fc138c00640 1 --2- 172.21.15.69:0/207770939 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc11c0a3c30 0x7fc11c0a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc128006010 tx=0x7fc128000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:22.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.536+0000 7fc131600640 1 -- 172.21.15.69:0/207770939 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc12009cb20 con 0x7fc134103080 2024-04-24T09:04:22.590 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:22 smithi069 ceph-mon[28170]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T09:04:22.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:22 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1414783600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:04:22.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:22 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/507595327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:04:22.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.740+0000 7fc13a000640 1 -- 172.21.15.69:0/207770939 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc1341a2c40 con 0x7fc134103080 2024-04-24T09:04:22.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.741+0000 7fc131600640 1 -- 172.21.15.69:0/207770939 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fc120009de0 con 0x7fc134103080 2024-04-24T09:04:22.741 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-24T09:04:22.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.744+0000 7fc13a000640 1 -- 172.21.15.69:0/207770939 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc11c0a3c30 msgr2=0x7fc11c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:22.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.744+0000 7fc13a000640 1 --2- 172.21.15.69:0/207770939 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc11c0a3c30 0x7fc11c0a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc128006010 tx=0x7fc128000bc0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.744+0000 7fc13a000640 1 -- 172.21.15.69:0/207770939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc134103080 msgr2=0x7fc13419a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:22.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.744+0000 7fc13a000640 1 --2- 172.21.15.69:0/207770939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc134103080 0x7fc13419a880 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc12000d770 tx=0x7fc12000dc30 comp rx=0 tx=0).stop 2024-04-24T09:04:22.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.744+0000 7fc13a000640 1 -- 172.21.15.69:0/207770939 shutdown_connections 2024-04-24T09:04:22.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.744+0000 7fc13a000640 1 --2- 172.21.15.69:0/207770939 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc11c0a3c30 0x7fc11c0a60f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.744+0000 7fc13a000640 1 --2- 172.21.15.69:0/207770939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc134103a40 0x7fc1341a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.744+0000 7fc13a000640 1 --2- 172.21.15.69:0/207770939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc134103080 0x7fc13419a880 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.744+0000 7fc13a000640 1 --2- 172.21.15.69:0/207770939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc134101f30 0x7fc13419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.744+0000 7fc13a000640 1 -- 172.21.15.69:0/207770939 >> 172.21.15.69:0/207770939 conn(0x7fc1340fd4a0 msgr2=0x7fc134074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:22.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.744+0000 7fc13a000640 1 -- 172.21.15.69:0/207770939 shutdown_connections 2024-04-24T09:04:22.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.744+0000 7fc13a000640 1 -- 172.21.15.69:0/207770939 wait complete. 2024-04-24T09:04:22.763 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T09:04:22.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:22 smithi081 ceph-mon[30961]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T09:04:22.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:22 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1414783600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:04:22.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:22 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/507595327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:04:22.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.951+0000 7ffacac00640 1 -- 172.21.15.69:0/507766753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffac4104280 msgr2=0x7ffac41046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:22.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.951+0000 7ffacac00640 1 --2- 172.21.15.69:0/507766753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffac4104280 0x7ffac41046a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009900 tx=0x7ffab402a410 comp rx=0 tx=0).stop 2024-04-24T09:04:22.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.951+0000 7ffacac00640 1 -- 172.21.15.69:0/507766753 shutdown_connections 2024-04-24T09:04:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.951+0000 7ffacac00640 1 --2- 172.21.15.69:0/507766753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffac4105e30 0x7ffac410c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.951+0000 7ffacac00640 1 --2- 172.21.15.69:0/507766753 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffac4105470 0x7ffac41058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.951+0000 7ffacac00640 1 --2- 172.21.15.69:0/507766753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffac4104280 0x7ffac41046a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.951+0000 7ffacac00640 1 -- 172.21.15.69:0/507766753 >> 172.21.15.69:0/507766753 conn(0x7ffac40ff7f0 msgr2=0x7ffac4101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.952+0000 7ffacac00640 1 -- 172.21.15.69:0/507766753 shutdown_connections 2024-04-24T09:04:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.952+0000 7ffacac00640 1 -- 172.21.15.69:0/507766753 wait complete. 2024-04-24T09:04:22.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.952+0000 7ffacac00640 1 Processor -- start 2024-04-24T09:04:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.953+0000 7ffacac00640 1 -- start start 2024-04-24T09:04:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.953+0000 7ffacac00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffac4104280 0x7ffac419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.953+0000 7ffacac00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffac4105470 0x7ffac419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.953+0000 7ffacac00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffac4105e30 0x7ffac41a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.953+0000 7ffacac00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffac410fe10 con 0x7ffac4105470 2024-04-24T09:04:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.953+0000 7ffacac00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ffac410fc90 con 0x7ffac4105e30 2024-04-24T09:04:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.953+0000 7ffacac00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ffac410ff90 con 0x7ffac4104280 2024-04-24T09:04:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.953+0000 7ffac3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffac4105470 0x7ffac419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.953+0000 7ffac3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffac4105470 0x7ffac419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34368/0 (socket says 172.21.15.69:34368) 2024-04-24T09:04:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.953+0000 7ffac3400640 1 -- 172.21.15.69:0/429588976 learned_addr learned my addr 172.21.15.69:0/429588976 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.954+0000 7ffac3e00640 1 --2- 172.21.15.69:0/429588976 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffac4104280 0x7ffac419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.954+0000 7ffac8c00640 1 --2- 172.21.15.69:0/429588976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffac4105e30 0x7ffac41a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:22.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.954+0000 7ffac3400640 1 -- 172.21.15.69:0/429588976 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffac4104280 msgr2=0x7ffac419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:22.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.954+0000 7ffac3400640 1 --2- 172.21.15.69:0/429588976 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffac4104280 0x7ffac419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.954+0000 7ffac3400640 1 -- 172.21.15.69:0/429588976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffac4105e30 msgr2=0x7ffac41a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:22.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.954+0000 7ffac3400640 1 --2- 172.21.15.69:0/429588976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffac4105e30 0x7ffac41a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:22.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.954+0000 7ffac3400640 1 -- 172.21.15.69:0/429588976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac41a20c0 con 0x7ffac4105470 2024-04-24T09:04:22.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.954+0000 7ffac3400640 1 --2- 172.21.15.69:0/429588976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffac4105470 0x7ffac419a9e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffab000d770 tx=0x7ffab000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:22.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:22 smithi130 ceph-mon[31553]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T09:04:22.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:22 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1414783600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:04:22.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:22 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/507595327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:04:22.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.954+0000 7ffac0c00640 1 -- 172.21.15.69:0/429588976 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0017070 con 0x7ffac4105470 2024-04-24T09:04:22.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.955+0000 7ffacac00640 1 -- 172.21.15.69:0/429588976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac41a23b0 con 0x7ffac4105470 2024-04-24T09:04:22.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.955+0000 7ffac0c00640 1 -- 172.21.15.69:0/429588976 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffab00054d0 con 0x7ffac4105470 2024-04-24T09:04:22.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.955+0000 7ffac0c00640 1 -- 172.21.15.69:0/429588976 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0020060 con 0x7ffac4105470 2024-04-24T09:04:22.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.955+0000 7ffacac00640 1 -- 172.21.15.69:0/429588976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac41a2900 con 0x7ffac4105470 2024-04-24T09:04:22.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.956+0000 7ffacac00640 1 -- 172.21.15.69:0/429588976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac410b5a0 con 0x7ffac4105470 2024-04-24T09:04:22.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.958+0000 7ffac0c00640 1 -- 172.21.15.69:0/429588976 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffab0004c90 con 0x7ffac4105470 2024-04-24T09:04:22.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.960+0000 7ffac0c00640 1 --2- 172.21.15.69:0/429588976 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ffaa80a3bb0 0x7ffaa80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:22.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.960+0000 7ffac0c00640 1 -- 172.21.15.69:0/429588976 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ffab00d0520 con 0x7ffac4105470 2024-04-24T09:04:22.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.960+0000 7ffac3e00640 1 --2- 172.21.15.69:0/429588976 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ffaa80a3bb0 0x7ffaa80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:22.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.961+0000 7ffac3e00640 1 --2- 172.21.15.69:0/429588976 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ffaa80a3bb0 0x7ffaa80a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffab4006010 tx=0x7ffab40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:22.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:22.964+0000 7ffac0c00640 1 -- 172.21.15.69:0/429588976 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffab009cc60 con 0x7ffac4105470 2024-04-24T09:04:23.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.175+0000 7ffacac00640 1 -- 172.21.15.69:0/429588976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ffac4062c00 con 0x7ffac4105470 2024-04-24T09:04:23.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.176+0000 7ffac0c00640 1 -- 172.21.15.69:0/429588976 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7ffab009c560 con 0x7ffac4105470 2024-04-24T09:04:23.176 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:04:23.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.179+0000 7ffacac00640 1 -- 172.21.15.69:0/429588976 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ffaa80a3bb0 msgr2=0x7ffaa80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:23.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.179+0000 7ffacac00640 1 --2- 172.21.15.69:0/429588976 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ffaa80a3bb0 0x7ffaa80a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffab4006010 tx=0x7ffab40314a0 comp rx=0 tx=0).stop 2024-04-24T09:04:23.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.180+0000 7ffacac00640 1 -- 172.21.15.69:0/429588976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffac4105470 msgr2=0x7ffac419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:23.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.180+0000 7ffacac00640 1 --2- 172.21.15.69:0/429588976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffac4105470 0x7ffac419a9e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffab000d770 tx=0x7ffab000dc30 comp rx=0 tx=0).stop 2024-04-24T09:04:23.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.180+0000 7ffacac00640 1 -- 172.21.15.69:0/429588976 shutdown_connections 2024-04-24T09:04:23.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.180+0000 7ffacac00640 1 --2- 172.21.15.69:0/429588976 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ffaa80a3bb0 0x7ffaa80a6070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:23.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.180+0000 7ffacac00640 1 --2- 172.21.15.69:0/429588976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffac4105e30 0x7ffac41a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:23.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.180+0000 7ffacac00640 1 --2- 172.21.15.69:0/429588976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffac4105470 0x7ffac419a9e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:23.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.180+0000 7ffacac00640 1 --2- 172.21.15.69:0/429588976 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffac4104280 0x7ffac419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:23.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.180+0000 7ffacac00640 1 -- 172.21.15.69:0/429588976 >> 172.21.15.69:0/429588976 conn(0x7ffac40ff7f0 msgr2=0x7ffac4108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:23.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.180+0000 7ffacac00640 1 -- 172.21.15.69:0/429588976 shutdown_connections 2024-04-24T09:04:23.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.180+0000 7ffacac00640 1 -- 172.21.15.69:0/429588976 wait complete. 2024-04-24T09:04:23.198 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T09:04:23.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.396+0000 7fdd38a00640 1 -- 172.21.15.69:0/2041943922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd34109730 msgr2=0x7fdd34109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:23.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.396+0000 7fdd38a00640 1 --2- 172.21.15.69:0/2041943922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd34109730 0x7fdd34109bb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009900 tx=0x7fdd1c02a410 comp rx=0 tx=0).stop 2024-04-24T09:04:23.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.396+0000 7fdd38a00640 1 -- 172.21.15.69:0/2041943922 shutdown_connections 2024-04-24T09:04:23.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.396+0000 7fdd38a00640 1 --2- 172.21.15.69:0/2041943922 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd3410a0f0 0x7fdd34110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:23.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.396+0000 7fdd38a00640 1 --2- 172.21.15.69:0/2041943922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd34109730 0x7fdd34109bb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:23.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.396+0000 7fdd38a00640 1 --2- 172.21.15.69:0/2041943922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd34108540 0x7fdd34108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:23.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.396+0000 7fdd38a00640 1 -- 172.21.15.69:0/2041943922 >> 172.21.15.69:0/2041943922 conn(0x7fdd340774d0 msgr2=0x7fdd34079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:23.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.396+0000 7fdd38a00640 1 -- 172.21.15.69:0/2041943922 shutdown_connections 2024-04-24T09:04:23.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.396+0000 7fdd38a00640 1 -- 172.21.15.69:0/2041943922 wait complete. 2024-04-24T09:04:23.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.397+0000 7fdd38a00640 1 Processor -- start 2024-04-24T09:04:23.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.397+0000 7fdd38a00640 1 -- start start 2024-04-24T09:04:23.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.397+0000 7fdd38a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd34108540 0x7fdd3419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:23.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.398+0000 7fdd38a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd34109730 0x7fdd3419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:23.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.398+0000 7fdd38a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd3410a0f0 0x7fdd341a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:23.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.398+0000 7fdd38a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341140d0 con 0x7fdd34108540 2024-04-24T09:04:23.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.398+0000 7fdd38a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdd34113f50 con 0x7fdd34109730 2024-04-24T09:04:23.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.398+0000 7fdd38a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdd34114250 con 0x7fdd3410a0f0 2024-04-24T09:04:23.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.398+0000 7fdd32800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd34108540 0x7fdd3419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:23.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.398+0000 7fdd32800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd34108540 0x7fdd3419e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34382/0 (socket says 172.21.15.69:34382) 2024-04-24T09:04:23.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.398+0000 7fdd32800640 1 -- 172.21.15.69:0/1665032672 learned_addr learned my addr 172.21.15.69:0/1665032672 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:23.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.398+0000 7fdd2be00640 1 --2- 172.21.15.69:0/1665032672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd34109730 0x7fdd3419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:23.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.398+0000 7fdd33200640 1 --2- 172.21.15.69:0/1665032672 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd3410a0f0 0x7fdd341a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:23.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.398+0000 7fdd32800640 1 -- 172.21.15.69:0/1665032672 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd3410a0f0 msgr2=0x7fdd341a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:23.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.398+0000 7fdd32800640 1 --2- 172.21.15.69:0/1665032672 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd3410a0f0 0x7fdd341a5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:23.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.398+0000 7fdd32800640 1 -- 172.21.15.69:0/1665032672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd34109730 msgr2=0x7fdd3419ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:23.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.398+0000 7fdd32800640 1 --2- 172.21.15.69:0/1665032672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd34109730 0x7fdd3419ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:23.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.398+0000 7fdd32800640 1 -- 172.21.15.69:0/1665032672 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd341a6370 con 0x7fdd34108540 2024-04-24T09:04:23.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.399+0000 7fdd32800640 1 --2- 172.21.15.69:0/1665032672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd34108540 0x7fdd3419e750 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400c970 tx=0x7fdd2400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:23.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.399+0000 7fdd2b400640 1 -- 172.21.15.69:0/1665032672 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24004220 con 0x7fdd34108540 2024-04-24T09:04:23.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.399+0000 7fdd2b400640 1 -- 172.21.15.69:0/1665032672 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd240043c0 con 0x7fdd34108540 2024-04-24T09:04:23.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.399+0000 7fdd2b400640 1 -- 172.21.15.69:0/1665032672 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2400f450 con 0x7fdd34108540 2024-04-24T09:04:23.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.399+0000 7fdd38a00640 1 -- 172.21.15.69:0/1665032672 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd341a6660 con 0x7fdd34108540 2024-04-24T09:04:23.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.399+0000 7fdd38a00640 1 -- 172.21.15.69:0/1665032672 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd341a6b80 con 0x7fdd34108540 2024-04-24T09:04:23.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.400+0000 7fdd38a00640 1 -- 172.21.15.69:0/1665032672 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd3410f930 con 0x7fdd34108540 2024-04-24T09:04:23.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.403+0000 7fdd2b400640 1 -- 172.21.15.69:0/1665032672 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdd240079b0 con 0x7fdd34108540 2024-04-24T09:04:23.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.404+0000 7fdd2b400640 1 --2- 172.21.15.69:0/1665032672 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd140a3bb0 0x7fdd140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:23.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.404+0000 7fdd2b400640 1 -- 172.21.15.69:0/1665032672 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fdd240d0890 con 0x7fdd34108540 2024-04-24T09:04:23.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.404+0000 7fdd2be00640 1 --2- 172.21.15.69:0/1665032672 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd140a3bb0 0x7fdd140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:23.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.405+0000 7fdd2be00640 1 --2- 172.21.15.69:0/1665032672 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd140a3bb0 0x7fdd140a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c000c00 tx=0x7fdd1c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:23.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.407+0000 7fdd2b400640 1 -- 172.21.15.69:0/1665032672 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd24014030 con 0x7fdd34108540 2024-04-24T09:04:23.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:23 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/207770939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:04:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:23 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/429588976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:04:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:23 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-24T09:04:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:23 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-24T09:04:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:23 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-24T09:04:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:23 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-24T09:04:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:23 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-24T09:04:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:23 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-24T09:04:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:23 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-24T09:04:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:23 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-04-24T09:04:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:23 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-04-24T09:04:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:23 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-24T09:04:23.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.637+0000 7fdd38a00640 1 -- 172.21.15.69:0/1665032672 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdd3402d620 con 0x7fdd34108540 2024-04-24T09:04:23.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.638+0000 7fdd2b400640 1 -- 172.21.15.69:0/1665032672 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+592 (secure 0 0 0) 0x7fdd2409cfd0 con 0x7fdd34108540 2024-04-24T09:04:23.637 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3826117580 2024-04-25T08:54:23.196411+0000 2024-04-24T09:04:23.637 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1006368686 2024-04-25T08:54:23.196411+0000 2024-04-24T09:04:23.637 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/3470783122 2024-04-25T08:54:23.196411+0000 2024-04-24T09:04:23.637 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2102503077 2024-04-25T08:53:48.948375+0000 2024-04-24T09:04:23.637 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/3470783122 2024-04-25T08:54:23.196411+0000 2024-04-24T09:04:23.638 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1041454348 2024-04-25T08:53:48.948375+0000 2024-04-24T09:04:23.638 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/3911821813 2024-04-25T08:53:48.948375+0000 2024-04-24T09:04:23.638 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/3911821813 2024-04-25T08:53:48.948375+0000 2024-04-24T09:04:23.638 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/4072155336 2024-04-25T08:53:48.948375+0000 2024-04-24T09:04:23.638 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1278076203 2024-04-25T08:54:23.196411+0000 2024-04-24T09:04:23.638 INFO:teuthology.orchestra.run.smithi069.stderr:listed 10 entries 2024-04-24T09:04:23.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.640+0000 7fdd38a00640 1 -- 172.21.15.69:0/1665032672 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd140a3bb0 msgr2=0x7fdd140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:23.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.640+0000 7fdd38a00640 1 --2- 172.21.15.69:0/1665032672 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd140a3bb0 0x7fdd140a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c000c00 tx=0x7fdd1c0314a0 comp rx=0 tx=0).stop 2024-04-24T09:04:23.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.640+0000 7fdd38a00640 1 -- 172.21.15.69:0/1665032672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd34108540 msgr2=0x7fdd3419e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:23.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.640+0000 7fdd38a00640 1 --2- 172.21.15.69:0/1665032672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd34108540 0x7fdd3419e750 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400c970 tx=0x7fdd2400ce30 comp rx=0 tx=0).stop 2024-04-24T09:04:23.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.640+0000 7fdd38a00640 1 -- 172.21.15.69:0/1665032672 shutdown_connections 2024-04-24T09:04:23.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.640+0000 7fdd38a00640 1 --2- 172.21.15.69:0/1665032672 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd3410a0f0 0x7fdd341a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:23.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.640+0000 7fdd38a00640 1 --2- 172.21.15.69:0/1665032672 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd140a3bb0 0x7fdd140a6070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:23.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.640+0000 7fdd38a00640 1 --2- 172.21.15.69:0/1665032672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd34109730 0x7fdd3419ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:23.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.640+0000 7fdd38a00640 1 --2- 172.21.15.69:0/1665032672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd34108540 0x7fdd3419e750 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:23.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.640+0000 7fdd38a00640 1 -- 172.21.15.69:0/1665032672 >> 172.21.15.69:0/1665032672 conn(0x7fdd340774d0 msgr2=0x7fdd3410cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:23.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.640+0000 7fdd38a00640 1 -- 172.21.15.69:0/1665032672 shutdown_connections 2024-04-24T09:04:23.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:23.640+0000 7fdd38a00640 1 -- 172.21.15.69:0/1665032672 wait complete. 2024-04-24T09:04:23.657 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi069.front.sepia.ceph.com... 2024-04-24T09:04:23.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:04:23.658 DEBUG:teuthology.orchestra.run.smithi069:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-24T09:04:23.727 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi069.stderr:ceph-fuse[69613]: fuse finished with error 0 and tester_r 0 2024-04-24T09:04:23.732 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T09:04:23.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:23 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/207770939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:04:23.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:23 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/429588976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:04:23.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:23 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-24T09:04:23.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:23 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-24T09:04:23.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:23 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-24T09:04:23.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:23 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-24T09:04:23.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:23 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-24T09:04:23.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:23 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-24T09:04:23.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:23 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-24T09:04:23.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:23 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-04-24T09:04:23.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:23 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-04-24T09:04:23.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:23 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-24T09:04:23.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:23 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/207770939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:04:23.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:23 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/429588976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:04:23.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:23 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-24T09:04:23.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:23 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-24T09:04:23.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:23 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-24T09:04:23.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:23 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-24T09:04:23.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:23 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-24T09:04:23.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:23 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-24T09:04:23.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:23 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-24T09:04:23.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:23 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-04-24T09:04:23.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:23 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-04-24T09:04:23.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:23 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-24T09:04:24.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:24 smithi081 ceph-mon[30961]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T09:04:24.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-24T09:04:24.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-24T09:04:24.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-24T09:04:24.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-24T09:04:24.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-24T09:04:24.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-24T09:04:24.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-24T09:04:24.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-04-24T09:04:24.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-04-24T09:04:24.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:24 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-24T09:04:24.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:24 smithi081 ceph-mon[30961]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T09:04:24.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:24 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1665032672' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:04:24.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:24 smithi069 ceph-mon[28170]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T09:04:24.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-24T09:04:24.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-24T09:04:24.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-24T09:04:24.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-24T09:04:24.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-24T09:04:24.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-24T09:04:24.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-24T09:04:24.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-04-24T09:04:24.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-04-24T09:04:24.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:24 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-24T09:04:24.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:24 smithi069 ceph-mon[28170]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T09:04:24.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:24 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1665032672' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:04:24.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:24 smithi130 ceph-mon[31553]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T09:04:24.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-24T09:04:24.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-24T09:04:24.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-24T09:04:24.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-24T09:04:24.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-24T09:04:24.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-24T09:04:24.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-24T09:04:24.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-04-24T09:04:24.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-04-24T09:04:24.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:24 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-24T09:04:24.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:24 smithi130 ceph-mon[31553]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T09:04:24.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:24 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1665032672' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:04:25.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:25 smithi081 ceph-mon[30961]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T09:04:25.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:25 smithi069 ceph-mon[28170]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T09:04:25.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:25 smithi130 ceph-mon[31553]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T09:04:26.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:26 smithi081 ceph-mon[30961]: pgmap v383: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-04-24T09:04:26.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:26 smithi069 ceph-mon[28170]: pgmap v383: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-04-24T09:04:26.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:26 smithi130 ceph-mon[31553]: pgmap v383: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-04-24T09:04:28.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:28 smithi081 ceph-mon[30961]: pgmap v384: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-04-24T09:04:28.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:28 smithi069 ceph-mon[28170]: pgmap v384: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-04-24T09:04:28.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:28 smithi130 ceph-mon[31553]: pgmap v384: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-04-24T09:04:29.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:29 smithi081 ceph-mon[30961]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:29.833 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi069.front.sepia.ceph.com 2024-04-24T09:04:29.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:04:29.833 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-24T09:04:29.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:29 smithi069 ceph-mon[28170]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:29.900 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:04:29.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:29 smithi130 ceph-mon[31553]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:30.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.106+0000 7f5f42e00640 1 -- 172.21.15.69:0/1816572871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f3c103c00 msgr2=0x7f5f3c10a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:30.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.106+0000 7f5f42e00640 1 --2- 172.21.15.69:0/1816572871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f3c103c00 0x7f5f3c10a490 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009ca0 tx=0x7f5f3402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.106+0000 7f5f42e00640 1 -- 172.21.15.69:0/1816572871 shutdown_connections 2024-04-24T09:04:30.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.106+0000 7f5f42e00640 1 --2- 172.21.15.69:0/1816572871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f3c103c00 0x7f5f3c10a490 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.106+0000 7f5f42e00640 1 --2- 172.21.15.69:0/1816572871 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f3c103240 0x7f5f3c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.106+0000 7f5f42e00640 1 --2- 172.21.15.69:0/1816572871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f3c102050 0x7f5f3c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.106+0000 7f5f42e00640 1 -- 172.21.15.69:0/1816572871 >> 172.21.15.69:0/1816572871 conn(0x7f5f3c0fd5e0 msgr2=0x7f5f3c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:30.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.106+0000 7f5f42e00640 1 -- 172.21.15.69:0/1816572871 shutdown_connections 2024-04-24T09:04:30.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.106+0000 7f5f42e00640 1 -- 172.21.15.69:0/1816572871 wait complete. 2024-04-24T09:04:30.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.107+0000 7f5f42e00640 1 Processor -- start 2024-04-24T09:04:30.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.108+0000 7f5f42e00640 1 -- start start 2024-04-24T09:04:30.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.108+0000 7f5f42e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f3c102050 0x7f5f3c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:30.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.108+0000 7f5f42e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f3c103240 0x7f5f3c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:30.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.108+0000 7f5f42e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f3c103c00 0x7f5f3c1a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:30.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.108+0000 7f5f42e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c10dbe0 con 0x7f5f3c102050 2024-04-24T09:04:30.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.108+0000 7f5f42e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c10da60 con 0x7f5f3c103c00 2024-04-24T09:04:30.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.108+0000 7f5f42e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c10dd60 con 0x7f5f3c103240 2024-04-24T09:04:30.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.108+0000 7f5f3be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f3c102050 0x7f5f3c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:30.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.108+0000 7f5f3be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f3c102050 0x7f5f3c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34402/0 (socket says 172.21.15.69:34402) 2024-04-24T09:04:30.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.108+0000 7f5f3be00640 1 -- 172.21.15.69:0/4185616579 learned_addr learned my addr 172.21.15.69:0/4185616579 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:30.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.109+0000 7f5f33400640 1 --2- 172.21.15.69:0/4185616579 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f3c103240 0x7f5f3c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:30.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.109+0000 7f5f3be00640 1 -- 172.21.15.69:0/4185616579 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f3c103240 msgr2=0x7f5f3c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:30.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.109+0000 7f5f40e00640 1 --2- 172.21.15.69:0/4185616579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f3c103c00 0x7f5f3c1a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:30.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.109+0000 7f5f3be00640 1 --2- 172.21.15.69:0/4185616579 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f3c103240 0x7f5f3c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.109+0000 7f5f3be00640 1 -- 172.21.15.69:0/4185616579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f3c103c00 msgr2=0x7f5f3c1a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:30.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.109+0000 7f5f3be00640 1 --2- 172.21.15.69:0/4185616579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f3c103c00 0x7f5f3c1a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.109+0000 7f5f3be00640 1 -- 172.21.15.69:0/4185616579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3c1a2080 con 0x7f5f3c102050 2024-04-24T09:04:30.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.109+0000 7f5f33400640 1 --2- 172.21.15.69:0/4185616579 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f3c103240 0x7f5f3c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:30.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.109+0000 7f5f40e00640 1 --2- 172.21.15.69:0/4185616579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f3c103c00 0x7f5f3c1a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:30.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.109+0000 7f5f3be00640 1 --2- 172.21.15.69:0/4185616579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f3c102050 0x7f5f3c19a460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00d770 tx=0x7f5f2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:30.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.109+0000 7f5f39600640 1 -- 172.21.15.69:0/4185616579 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c017070 con 0x7f5f3c102050 2024-04-24T09:04:30.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.109+0000 7f5f39600640 1 -- 172.21.15.69:0/4185616579 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f2c009c30 con 0x7f5f3c102050 2024-04-24T09:04:30.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.109+0000 7f5f39600640 1 -- 172.21.15.69:0/4185616579 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c003d00 con 0x7f5f3c102050 2024-04-24T09:04:30.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.109+0000 7f5f42e00640 1 -- 172.21.15.69:0/4185616579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3c1a2370 con 0x7f5f3c102050 2024-04-24T09:04:30.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.110+0000 7f5f42e00640 1 -- 172.21.15.69:0/4185616579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f3c1a2910 con 0x7f5f3c102050 2024-04-24T09:04:30.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.111+0000 7f5f42e00640 1 -- 172.21.15.69:0/4185616579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f3c1a2b70 con 0x7f5f3c102050 2024-04-24T09:04:30.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.113+0000 7f5f39600640 1 -- 172.21.15.69:0/4185616579 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5f2c003ea0 con 0x7f5f3c102050 2024-04-24T09:04:30.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.114+0000 7f5f39600640 1 --2- 172.21.15.69:0/4185616579 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5f200a3c30 0x7f5f200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:30.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.114+0000 7f5f39600640 1 -- 172.21.15.69:0/4185616579 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5f2c0d0520 con 0x7f5f3c102050 2024-04-24T09:04:30.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.114+0000 7f5f33400640 1 --2- 172.21.15.69:0/4185616579 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5f200a3c30 0x7f5f200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:30.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.115+0000 7f5f33400640 1 --2- 172.21.15.69:0/4185616579 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5f200a3c30 0x7f5f200a60f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c19b970 tx=0x7f5f24005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:30.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.119+0000 7f5f39600640 1 -- 172.21.15.69:0/4185616579 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f2c09cb40 con 0x7f5f3c102050 2024-04-24T09:04:30.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.328+0000 7f5f42e00640 1 -- 172.21.15.69:0/4185616579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f3c1a3320 con 0x7f5f3c102050 2024-04-24T09:04:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.333+0000 7f5f39600640 1 -- 172.21.15.69:0/4185616579 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24564 (secure 0 0 0) 0x7f5f2c009de0 con 0x7f5f3c102050 2024-04-24T09:04:30.333 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:30.334 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":92,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:24.497245+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:04:30.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.335+0000 7f5f42e00640 1 -- 172.21.15.69:0/4185616579 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5f200a3c30 msgr2=0x7f5f200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:30.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.335+0000 7f5f42e00640 1 --2- 172.21.15.69:0/4185616579 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5f200a3c30 0x7f5f200a60f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c19b970 tx=0x7f5f24005d30 comp rx=0 tx=0).stop 2024-04-24T09:04:30.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.335+0000 7f5f42e00640 1 -- 172.21.15.69:0/4185616579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f3c102050 msgr2=0x7f5f3c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:30.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.335+0000 7f5f42e00640 1 --2- 172.21.15.69:0/4185616579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f3c102050 0x7f5f3c19a460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00d770 tx=0x7f5f2c00dc30 comp rx=0 tx=0).stop 2024-04-24T09:04:30.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.336+0000 7f5f42e00640 1 -- 172.21.15.69:0/4185616579 shutdown_connections 2024-04-24T09:04:30.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.336+0000 7f5f42e00640 1 --2- 172.21.15.69:0/4185616579 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5f200a3c30 0x7f5f200a60f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.336+0000 7f5f42e00640 1 --2- 172.21.15.69:0/4185616579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f3c103c00 0x7f5f3c1a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.336+0000 7f5f42e00640 1 --2- 172.21.15.69:0/4185616579 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f3c103240 0x7f5f3c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.336+0000 7f5f42e00640 1 --2- 172.21.15.69:0/4185616579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f3c102050 0x7f5f3c19a460 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.336+0000 7f5f42e00640 1 -- 172.21.15.69:0/4185616579 >> 172.21.15.69:0/4185616579 conn(0x7f5f3c0fd5e0 msgr2=0x7f5f3c074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:30.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.336+0000 7f5f42e00640 1 -- 172.21.15.69:0/4185616579 shutdown_connections 2024-04-24T09:04:30.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.336+0000 7f5f42e00640 1 -- 172.21.15.69:0/4185616579 wait complete. 2024-04-24T09:04:30.356 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:04:30.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.563+0000 7f8773e00640 1 -- 172.21.15.69:0/1958042819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8774104080 msgr2=0x7f87741044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:30.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.563+0000 7f8773e00640 1 --2- 172.21.15.69:0/1958042819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8774104080 0x7f87741044a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8764009900 tx=0x7f876402a410 comp rx=0 tx=0).stop 2024-04-24T09:04:30.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.563+0000 7f8773e00640 1 -- 172.21.15.69:0/1958042819 shutdown_connections 2024-04-24T09:04:30.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.563+0000 7f8773e00640 1 --2- 172.21.15.69:0/1958042819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8774105c30 0x7f877410c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.563+0000 7f8773e00640 1 --2- 172.21.15.69:0/1958042819 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8774105270 0x7f87741056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.563+0000 7f8773e00640 1 --2- 172.21.15.69:0/1958042819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8774104080 0x7f87741044a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.563+0000 7f8773e00640 1 -- 172.21.15.69:0/1958042819 >> 172.21.15.69:0/1958042819 conn(0x7f87740ff610 msgr2=0x7f8774101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:30.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.563+0000 7f8773e00640 1 -- 172.21.15.69:0/1958042819 shutdown_connections 2024-04-24T09:04:30.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.563+0000 7f8773e00640 1 -- 172.21.15.69:0/1958042819 wait complete. 2024-04-24T09:04:30.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.564+0000 7f8773e00640 1 Processor -- start 2024-04-24T09:04:30.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.564+0000 7f8773e00640 1 -- start start 2024-04-24T09:04:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.564+0000 7f8773e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8774104080 0x7f877419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.565+0000 7f8773e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8774105270 0x7f877419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.565+0000 7f8773e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8774105c30 0x7f87741a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.565+0000 7f8773e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f877410fc10 con 0x7f8774104080 2024-04-24T09:04:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.565+0000 7f8773e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f877410fa90 con 0x7f8774105c30 2024-04-24T09:04:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.565+0000 7f8773e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f877410fd90 con 0x7f8774105270 2024-04-24T09:04:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.565+0000 7f8772a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8774104080 0x7f877419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.565+0000 7f8772a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8774104080 0x7f877419a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34414/0 (socket says 172.21.15.69:34414) 2024-04-24T09:04:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.565+0000 7f8772a00640 1 -- 172.21.15.69:0/1750337419 learned_addr learned my addr 172.21.15.69:0/1750337419 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.565+0000 7f8773400640 1 --2- 172.21.15.69:0/1750337419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8774105c30 0x7f87741a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.565+0000 7f8772000640 1 --2- 172.21.15.69:0/1750337419 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8774105270 0x7f877419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.565+0000 7f8772a00640 1 -- 172.21.15.69:0/1750337419 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8774105270 msgr2=0x7f877419a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.565+0000 7f8772a00640 1 --2- 172.21.15.69:0/1750337419 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8774105270 0x7f877419a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.565+0000 7f8772a00640 1 -- 172.21.15.69:0/1750337419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8774105c30 msgr2=0x7f87741a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.565+0000 7f8772a00640 1 --2- 172.21.15.69:0/1750337419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8774105c30 0x7f87741a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.565+0000 7f8772a00640 1 -- 172.21.15.69:0/1750337419 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87741a1ec0 con 0x7f8774104080 2024-04-24T09:04:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.566+0000 7f8772a00640 1 --2- 172.21.15.69:0/1750337419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8774104080 0x7f877419a2a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f87640314e0 tx=0x7f8764003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.566+0000 7f876b400640 1 -- 172.21.15.69:0/1750337419 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876403f070 con 0x7f8774104080 2024-04-24T09:04:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.566+0000 7f876b400640 1 -- 172.21.15.69:0/1750337419 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8764031c40 con 0x7f8774104080 2024-04-24T09:04:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.566+0000 7f8773e00640 1 -- 172.21.15.69:0/1750337419 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87741a2150 con 0x7f8774104080 2024-04-24T09:04:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.566+0000 7f876b400640 1 -- 172.21.15.69:0/1750337419 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764035030 con 0x7f8774104080 2024-04-24T09:04:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.566+0000 7f8773e00640 1 -- 172.21.15.69:0/1750337419 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87741a2690 con 0x7f8774104080 2024-04-24T09:04:30.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.567+0000 7f8773e00640 1 -- 172.21.15.69:0/1750337419 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f877410b440 con 0x7f8774104080 2024-04-24T09:04:30.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.569+0000 7f876b400640 1 -- 172.21.15.69:0/1750337419 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8764042070 con 0x7f8774104080 2024-04-24T09:04:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.571+0000 7f876b400640 1 --2- 172.21.15.69:0/1750337419 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f87580a3c30 0x7f87580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.572+0000 7f876b400640 1 -- 172.21.15.69:0/1750337419 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f87640f1000 con 0x7f8774104080 2024-04-24T09:04:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.572+0000 7f8772000640 1 --2- 172.21.15.69:0/1750337419 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f87580a3c30 0x7f87580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.572+0000 7f8772000640 1 --2- 172.21.15.69:0/1750337419 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f87580a3c30 0x7f87580a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8760005fd0 tx=0x7f8760005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:30.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.576+0000 7f876b400640 1 -- 172.21.15.69:0/1750337419 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f87640bd620 con 0x7f8774104080 2024-04-24T09:04:30.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.778+0000 7f8773e00640 1 -- 172.21.15.69:0/1750337419 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8774062c00 con 0x7f8774104080 2024-04-24T09:04:30.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.782+0000 7f876b400640 1 -- 172.21.15.69:0/1750337419 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24564 (secure 0 0 0) 0x7f87640bcf20 con 0x7f8774104080 2024-04-24T09:04:30.782 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:30.783 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":92,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:24.497245+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:04:30.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.785+0000 7f8773e00640 1 -- 172.21.15.69:0/1750337419 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f87580a3c30 msgr2=0x7f87580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:30.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.785+0000 7f8773e00640 1 --2- 172.21.15.69:0/1750337419 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f87580a3c30 0x7f87580a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8760005fd0 tx=0x7f8760005850 comp rx=0 tx=0).stop 2024-04-24T09:04:30.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.785+0000 7f8773e00640 1 -- 172.21.15.69:0/1750337419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8774104080 msgr2=0x7f877419a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:30.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.785+0000 7f8773e00640 1 --2- 172.21.15.69:0/1750337419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8774104080 0x7f877419a2a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f87640314e0 tx=0x7f8764003c10 comp rx=0 tx=0).stop 2024-04-24T09:04:30.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.785+0000 7f8773e00640 1 -- 172.21.15.69:0/1750337419 shutdown_connections 2024-04-24T09:04:30.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.785+0000 7f8773e00640 1 --2- 172.21.15.69:0/1750337419 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f87580a3c30 0x7f87580a60f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.785+0000 7f8773e00640 1 --2- 172.21.15.69:0/1750337419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8774105c30 0x7f87741a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.785+0000 7f8773e00640 1 --2- 172.21.15.69:0/1750337419 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8774105270 0x7f877419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.785+0000 7f8773e00640 1 --2- 172.21.15.69:0/1750337419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8774104080 0x7f877419a2a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:30.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.785+0000 7f8773e00640 1 -- 172.21.15.69:0/1750337419 >> 172.21.15.69:0/1750337419 conn(0x7f87740ff610 msgr2=0x7f8774108610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:30.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.785+0000 7f8773e00640 1 -- 172.21.15.69:0/1750337419 shutdown_connections 2024-04-24T09:04:30.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:30.785+0000 7f8773e00640 1 -- 172.21.15.69:0/1750337419 wait complete. 2024-04-24T09:04:30.801 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:30 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/4185616579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:30.805 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T09:04:30.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:30 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/4185616579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:30.856 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:30 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/4185616579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:31.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.012+0000 7fb5f7e00640 1 -- 172.21.15.69:0/2219474431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f81052c0 msgr2=0x7fb5f81076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:31.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.012+0000 7fb5f7e00640 1 --2- 172.21.15.69:0/2219474431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f81052c0 0x7fb5f81076b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009900 tx=0x7fb5e002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.013+0000 7fb5f7e00640 1 -- 172.21.15.69:0/2219474431 shutdown_connections 2024-04-24T09:04:31.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.013+0000 7fb5f7e00640 1 --2- 172.21.15.69:0/2219474431 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5f8107bf0 0x7fb5f810a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.013+0000 7fb5f7e00640 1 --2- 172.21.15.69:0/2219474431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f81052c0 0x7fb5f81076b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.013+0000 7fb5f7e00640 1 --2- 172.21.15.69:0/2219474431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5f8102990 0x7fb5f8104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.013+0000 7fb5f7e00640 1 -- 172.21.15.69:0/2219474431 >> 172.21.15.69:0/2219474431 conn(0x7fb5f80fc3a0 msgr2=0x7fb5f80fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:31.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.013+0000 7fb5f7e00640 1 -- 172.21.15.69:0/2219474431 shutdown_connections 2024-04-24T09:04:31.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.013+0000 7fb5f7e00640 1 -- 172.21.15.69:0/2219474431 wait complete. 2024-04-24T09:04:31.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.014+0000 7fb5f7e00640 1 Processor -- start 2024-04-24T09:04:31.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.014+0000 7fb5f7e00640 1 -- start start 2024-04-24T09:04:31.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.014+0000 7fb5f7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8102990 0x7fb5f819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:31.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.014+0000 7fb5f7e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5f81052c0 0x7fb5f819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:31.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.014+0000 7fb5f7e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5f8107bf0 0x7fb5f81a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:31.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.014+0000 7fb5f7e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f810da80 con 0x7fb5f8102990 2024-04-24T09:04:31.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.014+0000 7fb5f7e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f810d900 con 0x7fb5f81052c0 2024-04-24T09:04:31.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.014+0000 7fb5f7e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f810dc00 con 0x7fb5f8107bf0 2024-04-24T09:04:31.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.014+0000 7fb5f6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8102990 0x7fb5f819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:31.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.014+0000 7fb5f6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8102990 0x7fb5f819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34424/0 (socket says 172.21.15.69:34424) 2024-04-24T09:04:31.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.014+0000 7fb5f6a00640 1 -- 172.21.15.69:0/2111706192 learned_addr learned my addr 172.21.15.69:0/2111706192 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:31.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.014+0000 7fb5f7400640 1 --2- 172.21.15.69:0/2111706192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5f8107bf0 0x7fb5f81a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:31.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.015+0000 7fb5ee000640 1 --2- 172.21.15.69:0/2111706192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5f81052c0 0x7fb5f819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:31.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.015+0000 7fb5f6a00640 1 -- 172.21.15.69:0/2111706192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5f8107bf0 msgr2=0x7fb5f81a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:31.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.015+0000 7fb5f6a00640 1 --2- 172.21.15.69:0/2111706192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5f8107bf0 0x7fb5f81a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.015+0000 7fb5f6a00640 1 -- 172.21.15.69:0/2111706192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5f81052c0 msgr2=0x7fb5f819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:31.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.015+0000 7fb5f6a00640 1 --2- 172.21.15.69:0/2111706192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5f81052c0 0x7fb5f819a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.015+0000 7fb5f6a00640 1 -- 172.21.15.69:0/2111706192 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f81a1ed0 con 0x7fb5f8102990 2024-04-24T09:04:31.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.015+0000 7fb5f6a00640 1 --2- 172.21.15.69:0/2111706192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8102990 0x7fb5f819a2b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800c920 tx=0x7fb5e800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:31.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.015+0000 7fb5efe00640 1 -- 172.21.15.69:0/2111706192 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8007c70 con 0x7fb5f8102990 2024-04-24T09:04:31.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.015+0000 7fb5efe00640 1 -- 172.21.15.69:0/2111706192 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5e8007e10 con 0x7fb5f8102990 2024-04-24T09:04:31.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.015+0000 7fb5efe00640 1 -- 172.21.15.69:0/2111706192 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8018820 con 0x7fb5f8102990 2024-04-24T09:04:31.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.015+0000 7fb5f7e00640 1 -- 172.21.15.69:0/2111706192 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f81a21c0 con 0x7fb5f8102990 2024-04-24T09:04:31.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.016+0000 7fb5f7e00640 1 -- 172.21.15.69:0/2111706192 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f81a2760 con 0x7fb5f8102990 2024-04-24T09:04:31.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.017+0000 7fb5f7e00640 1 -- 172.21.15.69:0/2111706192 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f8109740 con 0x7fb5f8102990 2024-04-24T09:04:31.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.019+0000 7fb5efe00640 1 -- 172.21.15.69:0/2111706192 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb5e80189c0 con 0x7fb5f8102990 2024-04-24T09:04:31.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.020+0000 7fb5efe00640 1 --2- 172.21.15.69:0/2111706192 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb5dc0a3c30 0x7fb5dc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:31.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.020+0000 7fb5efe00640 1 -- 172.21.15.69:0/2111706192 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb5e80d0860 con 0x7fb5f8102990 2024-04-24T09:04:31.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.020+0000 7fb5ee000640 1 --2- 172.21.15.69:0/2111706192 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb5dc0a3c30 0x7fb5dc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:31.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.021+0000 7fb5ee000640 1 --2- 172.21.15.69:0/2111706192 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb5dc0a3c30 0x7fb5dc0a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb5e002a910 tx=0x7fb5e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:31.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.024+0000 7fb5efe00640 1 -- 172.21.15.69:0/2111706192 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5e8014030 con 0x7fb5f8102990 2024-04-24T09:04:31.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.267+0000 7fb5f7e00640 1 -- 172.21.15.69:0/2111706192 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb5f802d620 con 0x7fb5f8102990 2024-04-24T09:04:31.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.558+0000 7fb5efe00640 1 -- 172.21.15.69:0/2111706192 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fb5e809ce80 con 0x7fb5f8102990 2024-04-24T09:04:31.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.562+0000 7fb5f7e00640 1 -- 172.21.15.69:0/2111706192 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb5dc0a3c30 msgr2=0x7fb5dc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:31.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.562+0000 7fb5f7e00640 1 --2- 172.21.15.69:0/2111706192 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb5dc0a3c30 0x7fb5dc0a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb5e002a910 tx=0x7fb5e0005fd0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.562+0000 7fb5f7e00640 1 -- 172.21.15.69:0/2111706192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8102990 msgr2=0x7fb5f819a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:31.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.562+0000 7fb5f7e00640 1 --2- 172.21.15.69:0/2111706192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8102990 0x7fb5f819a2b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800c920 tx=0x7fb5e800cde0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.562+0000 7fb5f7e00640 1 -- 172.21.15.69:0/2111706192 shutdown_connections 2024-04-24T09:04:31.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.562+0000 7fb5f7e00640 1 --2- 172.21.15.69:0/2111706192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5f8107bf0 0x7fb5f81a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.562+0000 7fb5f7e00640 1 --2- 172.21.15.69:0/2111706192 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb5dc0a3c30 0x7fb5dc0a60f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.562+0000 7fb5f7e00640 1 --2- 172.21.15.69:0/2111706192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5f81052c0 0x7fb5f819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.562+0000 7fb5f7e00640 1 --2- 172.21.15.69:0/2111706192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5f8102990 0x7fb5f819a2b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.562+0000 7fb5f7e00640 1 -- 172.21.15.69:0/2111706192 >> 172.21.15.69:0/2111706192 conn(0x7fb5f80fc3a0 msgr2=0x7fb5f81020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:31.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.563+0000 7fb5f7e00640 1 -- 172.21.15.69:0/2111706192 shutdown_connections 2024-04-24T09:04:31.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.563+0000 7fb5f7e00640 1 -- 172.21.15.69:0/2111706192 wait complete. 2024-04-24T09:04:31.584 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-24T09:04:31.584 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-24T09:04:31.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.795+0000 7ff42cc00640 1 -- 172.21.15.69:0/4000914123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff428103230 msgr2=0x7ff4281036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:31.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.795+0000 7ff42cc00640 1 --2- 172.21.15.69:0/4000914123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff428103230 0x7ff4281036b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff414009900 tx=0x7ff41402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.795+0000 7ff42cc00640 1 -- 172.21.15.69:0/4000914123 shutdown_connections 2024-04-24T09:04:31.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.795+0000 7ff42cc00640 1 --2- 172.21.15.69:0/4000914123 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff428103bf0 0x7ff42810a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.795+0000 7ff42cc00640 1 --2- 172.21.15.69:0/4000914123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff428103230 0x7ff4281036b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.795+0000 7ff42cc00640 1 --2- 172.21.15.69:0/4000914123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff428102040 0x7ff428102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.795+0000 7ff42cc00640 1 -- 172.21.15.69:0/4000914123 >> 172.21.15.69:0/4000914123 conn(0x7ff4280fd5b0 msgr2=0x7ff4280ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:31.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.795+0000 7ff42cc00640 1 -- 172.21.15.69:0/4000914123 shutdown_connections 2024-04-24T09:04:31.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.795+0000 7ff42cc00640 1 -- 172.21.15.69:0/4000914123 wait complete. 2024-04-24T09:04:31.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.796+0000 7ff42cc00640 1 Processor -- start 2024-04-24T09:04:31.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.796+0000 7ff42cc00640 1 -- start start 2024-04-24T09:04:31.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.796+0000 7ff42cc00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff428102040 0x7ff428198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:31.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.796+0000 7ff42cc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff428103230 0x7ff4281987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:31.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.796+0000 7ff42cc00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff428103bf0 0x7ff42819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:31.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.796+0000 7ff42cc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff42810dbd0 con 0x7ff428103230 2024-04-24T09:04:31.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.796+0000 7ff42cc00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff42810da50 con 0x7ff428102040 2024-04-24T09:04:31.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.796+0000 7ff42cc00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff42810dd50 con 0x7ff428103bf0 2024-04-24T09:04:31.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.797+0000 7ff425400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff428103230 0x7ff4281987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:31.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.797+0000 7ff425400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff428103230 0x7ff4281987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34440/0 (socket says 172.21.15.69:34440) 2024-04-24T09:04:31.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.797+0000 7ff425400640 1 -- 172.21.15.69:0/1513269767 learned_addr learned my addr 172.21.15.69:0/1513269767 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:31.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.797+0000 7ff426800640 1 --2- 172.21.15.69:0/1513269767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff428103bf0 0x7ff42819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:31.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.797+0000 7ff425e00640 1 --2- 172.21.15.69:0/1513269767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff428102040 0x7ff428198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:31.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.797+0000 7ff425400640 1 -- 172.21.15.69:0/1513269767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff428103bf0 msgr2=0x7ff42819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:31.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.797+0000 7ff425400640 1 --2- 172.21.15.69:0/1513269767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff428103bf0 0x7ff42819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.797+0000 7ff425400640 1 -- 172.21.15.69:0/1513269767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff428102040 msgr2=0x7ff428198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:31.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.797+0000 7ff425400640 1 --2- 172.21.15.69:0/1513269767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff428102040 0x7ff428198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:31.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.797+0000 7ff425400640 1 -- 172.21.15.69:0/1513269767 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42819fe90 con 0x7ff428103230 2024-04-24T09:04:31.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.798+0000 7ff425400640 1 --2- 172.21.15.69:0/1513269767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff428103230 0x7ff4281987b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff4140314e0 tx=0x7ff414003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:31.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.798+0000 7ff41ea00640 1 -- 172.21.15.69:0/1513269767 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41403f070 con 0x7ff428103230 2024-04-24T09:04:31.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.798+0000 7ff41ea00640 1 -- 172.21.15.69:0/1513269767 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff414031e40 con 0x7ff428103230 2024-04-24T09:04:31.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.798+0000 7ff41ea00640 1 -- 172.21.15.69:0/1513269767 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414035030 con 0x7ff428103230 2024-04-24T09:04:31.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.798+0000 7ff42cc00640 1 -- 172.21.15.69:0/1513269767 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4281a0120 con 0x7ff428103230 2024-04-24T09:04:31.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.798+0000 7ff42cc00640 1 -- 172.21.15.69:0/1513269767 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4281a0610 con 0x7ff428103230 2024-04-24T09:04:31.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.799+0000 7ff42cc00640 1 -- 172.21.15.69:0/1513269767 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff428109430 con 0x7ff428103230 2024-04-24T09:04:31.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.801+0000 7ff41ea00640 1 -- 172.21.15.69:0/1513269767 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff414042070 con 0x7ff428103230 2024-04-24T09:04:31.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.803+0000 7ff41ea00640 1 --2- 172.21.15.69:0/1513269767 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff40c0a3bb0 0x7ff40c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:31.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.803+0000 7ff41ea00640 1 -- 172.21.15.69:0/1513269767 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff4140f11a0 con 0x7ff428103230 2024-04-24T09:04:31.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.803+0000 7ff425e00640 1 --2- 172.21.15.69:0/1513269767 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff40c0a3bb0 0x7ff40c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:31.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.804+0000 7ff425e00640 1 --2- 172.21.15.69:0/1513269767 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff40c0a3bb0 0x7ff40c0a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff418005fd0 tx=0x7ff418005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:31.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:31.807+0000 7ff41ea00640 1 -- 172.21.15.69:0/1513269767 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4140bd7c0 con 0x7ff428103230 2024-04-24T09:04:31.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:31 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1750337419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:31.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:31 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2111706192' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T09:04:31.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:31 smithi081 ceph-mon[30961]: pgmap v386: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:31.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:31 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1750337419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:31.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:31 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2111706192' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T09:04:31.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:31 smithi069 ceph-mon[28170]: pgmap v386: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:31.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:31 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1750337419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:31.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:31 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2111706192' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T09:04:31.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:31 smithi130 ceph-mon[31553]: pgmap v386: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:32.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.050+0000 7ff42cc00640 1 -- 172.21.15.69:0/1513269767 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7ff428062c00 con 0x7ff428103230 2024-04-24T09:04:32.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.573+0000 7ff41ea00640 1 -- 172.21.15.69:0/1513269767 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7ff4140bd0c0 con 0x7ff428103230 2024-04-24T09:04:32.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.576+0000 7ff42cc00640 1 -- 172.21.15.69:0/1513269767 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff40c0a3bb0 msgr2=0x7ff40c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:32.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.576+0000 7ff42cc00640 1 --2- 172.21.15.69:0/1513269767 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff40c0a3bb0 0x7ff40c0a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff418005fd0 tx=0x7ff418005850 comp rx=0 tx=0).stop 2024-04-24T09:04:32.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.576+0000 7ff42cc00640 1 -- 172.21.15.69:0/1513269767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff428103230 msgr2=0x7ff4281987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:32.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.576+0000 7ff42cc00640 1 --2- 172.21.15.69:0/1513269767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff428103230 0x7ff4281987b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff4140314e0 tx=0x7ff414003c80 comp rx=0 tx=0).stop 2024-04-24T09:04:32.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.577+0000 7ff42cc00640 1 -- 172.21.15.69:0/1513269767 shutdown_connections 2024-04-24T09:04:32.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.577+0000 7ff42cc00640 1 --2- 172.21.15.69:0/1513269767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff428103bf0 0x7ff42819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:32.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.577+0000 7ff42cc00640 1 --2- 172.21.15.69:0/1513269767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff428103230 0x7ff4281987b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:32.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.577+0000 7ff42cc00640 1 --2- 172.21.15.69:0/1513269767 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff40c0a3bb0 0x7ff40c0a6070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:32.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.577+0000 7ff42cc00640 1 --2- 172.21.15.69:0/1513269767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff428102040 0x7ff428198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:32.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.577+0000 7ff42cc00640 1 -- 172.21.15.69:0/1513269767 >> 172.21.15.69:0/1513269767 conn(0x7ff4280fd5b0 msgr2=0x7ff428106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:32.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.577+0000 7ff42cc00640 1 -- 172.21.15.69:0/1513269767 shutdown_connections 2024-04-24T09:04:32.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.577+0000 7ff42cc00640 1 -- 172.21.15.69:0/1513269767 wait complete. 2024-04-24T09:04:32.599 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-24T09:04:32.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:32 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2111706192' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T09:04:32.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:32 smithi130 ceph-mon[31553]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:04:32.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:32 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1513269767' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T09:04:32.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.787+0000 7faf73e00640 1 -- 172.21.15.69:0/1858378481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf74105cd0 msgr2=0x7faf7410c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:32.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.787+0000 7faf73e00640 1 --2- 172.21.15.69:0/1858378481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf74105cd0 0x7faf7410c560 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009ca0 tx=0x7faf6c02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:04:32.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.787+0000 7faf73e00640 1 -- 172.21.15.69:0/1858378481 shutdown_connections 2024-04-24T09:04:32.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.787+0000 7faf73e00640 1 --2- 172.21.15.69:0/1858378481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf74105cd0 0x7faf7410c560 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:32.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.787+0000 7faf73e00640 1 --2- 172.21.15.69:0/1858378481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf74105310 0x7faf74105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:32.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.787+0000 7faf73e00640 1 --2- 172.21.15.69:0/1858378481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf74104120 0x7faf74104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:32.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.787+0000 7faf73e00640 1 -- 172.21.15.69:0/1858378481 >> 172.21.15.69:0/1858378481 conn(0x7faf740ff6b0 msgr2=0x7faf74101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:32.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.787+0000 7faf73e00640 1 -- 172.21.15.69:0/1858378481 shutdown_connections 2024-04-24T09:04:32.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.787+0000 7faf73e00640 1 -- 172.21.15.69:0/1858378481 wait complete. 2024-04-24T09:04:32.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.788+0000 7faf73e00640 1 Processor -- start 2024-04-24T09:04:32.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.788+0000 7faf73e00640 1 -- start start 2024-04-24T09:04:32.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.789+0000 7faf73e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf74104120 0x7faf7419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:32.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.789+0000 7faf73e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf74105310 0x7faf7419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:32.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.789+0000 7faf73e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf74105cd0 0x7faf741a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:32.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.789+0000 7faf73e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faf7410fc60 con 0x7faf74105310 2024-04-24T09:04:32.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.789+0000 7faf73e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7faf7410fae0 con 0x7faf74105cd0 2024-04-24T09:04:32.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.789+0000 7faf73e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7faf7410fde0 con 0x7faf74104120 2024-04-24T09:04:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.789+0000 7faf72000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf74105310 0x7faf7419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.789+0000 7faf72000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf74105310 0x7faf7419a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34450/0 (socket says 172.21.15.69:34450) 2024-04-24T09:04:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.789+0000 7faf72000640 1 -- 172.21.15.69:0/1255554006 learned_addr learned my addr 172.21.15.69:0/1255554006 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.789+0000 7faf73400640 1 --2- 172.21.15.69:0/1255554006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf74105cd0 0x7faf741a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.789+0000 7faf72000640 1 -- 172.21.15.69:0/1255554006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf74104120 msgr2=0x7faf7419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.789+0000 7faf72a00640 1 --2- 172.21.15.69:0/1255554006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf74104120 0x7faf7419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.790+0000 7faf72000640 1 --2- 172.21.15.69:0/1255554006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf74104120 0x7faf7419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.790+0000 7faf72000640 1 -- 172.21.15.69:0/1255554006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf74105cd0 msgr2=0x7faf741a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.790+0000 7faf72000640 1 --2- 172.21.15.69:0/1255554006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf74105cd0 0x7faf741a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.790+0000 7faf72000640 1 -- 172.21.15.69:0/1255554006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf741a1eb0 con 0x7faf74105310 2024-04-24T09:04:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.790+0000 7faf73400640 1 --2- 172.21.15.69:0/1255554006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf74105cd0 0x7faf741a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.790+0000 7faf72a00640 1 --2- 172.21.15.69:0/1255554006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf74104120 0x7faf7419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.790+0000 7faf72000640 1 --2- 172.21.15.69:0/1255554006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf74105310 0x7faf7419a7d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faf6000c920 tx=0x7faf6000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.790+0000 7faf6b400640 1 -- 172.21.15.69:0/1255554006 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60007c70 con 0x7faf74105310 2024-04-24T09:04:32.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.790+0000 7faf6b400640 1 -- 172.21.15.69:0/1255554006 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf60007e10 con 0x7faf74105310 2024-04-24T09:04:32.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.790+0000 7faf6b400640 1 -- 172.21.15.69:0/1255554006 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60018820 con 0x7faf74105310 2024-04-24T09:04:32.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.790+0000 7faf73e00640 1 -- 172.21.15.69:0/1255554006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf741a21a0 con 0x7faf74105310 2024-04-24T09:04:32.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.790+0000 7faf73e00640 1 -- 172.21.15.69:0/1255554006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf741a2740 con 0x7faf74105310 2024-04-24T09:04:32.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.792+0000 7faf73e00640 1 -- 172.21.15.69:0/1255554006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf7410b3e0 con 0x7faf74105310 2024-04-24T09:04:32.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.794+0000 7faf6b400640 1 -- 172.21.15.69:0/1255554006 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faf600189c0 con 0x7faf74105310 2024-04-24T09:04:32.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.795+0000 7faf6b400640 1 --2- 172.21.15.69:0/1255554006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7faf580a3c30 0x7faf580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:32.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.796+0000 7faf6b400640 1 -- 172.21.15.69:0/1255554006 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7faf600d0bd0 con 0x7faf74105310 2024-04-24T09:04:32.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.796+0000 7faf72a00640 1 --2- 172.21.15.69:0/1255554006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7faf580a3c30 0x7faf580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:32.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.796+0000 7faf72a00640 1 --2- 172.21.15.69:0/1255554006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7faf580a3c30 0x7faf580a60f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faf640058c0 tx=0x7faf64005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:32.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:32.799+0000 7faf6b400640 1 -- 172.21.15.69:0/1255554006 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf60014030 con 0x7faf74105310 2024-04-24T09:04:32.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:32 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2111706192' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T09:04:32.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:32 smithi081 ceph-mon[30961]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:04:32.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:32 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1513269767' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T09:04:32.840 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:32 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2111706192' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T09:04:32.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:32 smithi069 ceph-mon[28170]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:04:32.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:32 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1513269767' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T09:04:33.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.026+0000 7faf73e00640 1 -- 172.21.15.69:0/1255554006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7faf7402d620 con 0x7faf74105310 2024-04-24T09:04:33.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.595+0000 7faf6b400640 1 -- 172.21.15.69:0/1255554006 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7faf6009d300 con 0x7faf74105310 2024-04-24T09:04:33.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.598+0000 7faf73e00640 1 -- 172.21.15.69:0/1255554006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7faf580a3c30 msgr2=0x7faf580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:33.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.598+0000 7faf73e00640 1 --2- 172.21.15.69:0/1255554006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7faf580a3c30 0x7faf580a60f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faf640058c0 tx=0x7faf64005d50 comp rx=0 tx=0).stop 2024-04-24T09:04:33.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.598+0000 7faf73e00640 1 -- 172.21.15.69:0/1255554006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf74105310 msgr2=0x7faf7419a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:33.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.598+0000 7faf73e00640 1 --2- 172.21.15.69:0/1255554006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf74105310 0x7faf7419a7d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faf6000c920 tx=0x7faf6000cde0 comp rx=0 tx=0).stop 2024-04-24T09:04:33.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.599+0000 7faf73e00640 1 -- 172.21.15.69:0/1255554006 shutdown_connections 2024-04-24T09:04:33.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.599+0000 7faf73e00640 1 --2- 172.21.15.69:0/1255554006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7faf580a3c30 0x7faf580a60f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:33.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.599+0000 7faf73e00640 1 --2- 172.21.15.69:0/1255554006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf74105cd0 0x7faf741a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:33.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.599+0000 7faf73e00640 1 --2- 172.21.15.69:0/1255554006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf74105310 0x7faf7419a7d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:33.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.599+0000 7faf73e00640 1 --2- 172.21.15.69:0/1255554006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf74104120 0x7faf7419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:33.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.599+0000 7faf73e00640 1 -- 172.21.15.69:0/1255554006 >> 172.21.15.69:0/1255554006 conn(0x7faf740ff6b0 msgr2=0x7faf741085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:33.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.599+0000 7faf73e00640 1 -- 172.21.15.69:0/1255554006 shutdown_connections 2024-04-24T09:04:33.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.599+0000 7faf73e00640 1 -- 172.21.15.69:0/1255554006 wait complete. 2024-04-24T09:04:33.623 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-24T09:04:33.686 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:33 smithi069 ceph-mon[28170]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T09:04:33.686 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:33 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1513269767' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T09:04:33.686 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:33 smithi069 ceph-mon[28170]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T09:04:33.686 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:33 smithi069 ceph-mon[28170]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:04:33.686 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:33 smithi069 ceph-mon[28170]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-24T09:04:33.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:33 smithi069 ceph-mon[28170]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-24T09:04:33.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:33 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1255554006' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T09:04:33.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:33 smithi069 ceph-mon[28170]: pgmap v387: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.816+0000 7feb63c00640 1 -- 172.21.15.69:0/3615993967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb5c104260 msgr2=0x7feb5c104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.816+0000 7feb63c00640 1 --2- 172.21.15.69:0/3615993967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb5c104260 0x7feb5c104680 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feb44009900 tx=0x7feb4402a410 comp rx=0 tx=0).stop 2024-04-24T09:04:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.816+0000 7feb63c00640 1 -- 172.21.15.69:0/3615993967 shutdown_connections 2024-04-24T09:04:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.816+0000 7feb63c00640 1 --2- 172.21.15.69:0/3615993967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb5c105e10 0x7feb5c10c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.816+0000 7feb63c00640 1 --2- 172.21.15.69:0/3615993967 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb5c105450 0x7feb5c1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.816+0000 7feb63c00640 1 --2- 172.21.15.69:0/3615993967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb5c104260 0x7feb5c104680 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.816+0000 7feb63c00640 1 -- 172.21.15.69:0/3615993967 >> 172.21.15.69:0/3615993967 conn(0x7feb5c0ff7f0 msgr2=0x7feb5c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.816+0000 7feb63c00640 1 -- 172.21.15.69:0/3615993967 shutdown_connections 2024-04-24T09:04:33.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.816+0000 7feb63c00640 1 -- 172.21.15.69:0/3615993967 wait complete. 2024-04-24T09:04:33.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:33 smithi081 ceph-mon[30961]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T09:04:33.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:33 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1513269767' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T09:04:33.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:33 smithi081 ceph-mon[30961]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T09:04:33.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:33 smithi081 ceph-mon[30961]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:04:33.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:33 smithi081 ceph-mon[30961]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-24T09:04:33.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:33 smithi081 ceph-mon[30961]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-24T09:04:33.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:33 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1255554006' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T09:04:33.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:33 smithi081 ceph-mon[30961]: pgmap v387: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:33.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.817+0000 7feb63c00640 1 Processor -- start 2024-04-24T09:04:33.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.817+0000 7feb63c00640 1 -- start start 2024-04-24T09:04:33.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.818+0000 7feb63c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb5c104260 0x7feb5c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:33.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.818+0000 7feb63c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb5c105450 0x7feb5c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:33.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.818+0000 7feb63c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb5c105e10 0x7feb5c1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:33.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.818+0000 7feb63c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c10fdf0 con 0x7feb5c104260 2024-04-24T09:04:33.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.818+0000 7feb63c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c10fc70 con 0x7feb5c105e10 2024-04-24T09:04:33.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.818+0000 7feb63c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c10ff70 con 0x7feb5c105450 2024-04-24T09:04:33.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.818+0000 7feb61200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb5c104260 0x7feb5c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:33.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.818+0000 7feb61200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb5c104260 0x7feb5c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36378/0 (socket says 172.21.15.69:36378) 2024-04-24T09:04:33.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.818+0000 7feb61200640 1 -- 172.21.15.69:0/2531180576 learned_addr learned my addr 172.21.15.69:0/2531180576 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:33.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.818+0000 7feb61200640 1 -- 172.21.15.69:0/2531180576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb5c105450 msgr2=0x7feb5c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:33.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.818+0000 7feb5be00640 1 --2- 172.21.15.69:0/2531180576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb5c105450 0x7feb5c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:33.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.818+0000 7feb61c00640 1 --2- 172.21.15.69:0/2531180576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb5c105e10 0x7feb5c1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:33.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.818+0000 7feb61200640 1 --2- 172.21.15.69:0/2531180576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb5c105450 0x7feb5c19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:33.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.818+0000 7feb61200640 1 -- 172.21.15.69:0/2531180576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb5c105e10 msgr2=0x7feb5c1a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:33.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.818+0000 7feb61200640 1 --2- 172.21.15.69:0/2531180576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb5c105e10 0x7feb5c1a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:33.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.818+0000 7feb61200640 1 -- 172.21.15.69:0/2531180576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5c1a20d0 con 0x7feb5c104260 2024-04-24T09:04:33.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.819+0000 7feb61200640 1 --2- 172.21.15.69:0/2531180576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb5c104260 0x7feb5c19a4b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feb440314e0 tx=0x7feb44003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:33.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.819+0000 7feb59600640 1 -- 172.21.15.69:0/2531180576 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4403f070 con 0x7feb5c104260 2024-04-24T09:04:33.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.819+0000 7feb59600640 1 -- 172.21.15.69:0/2531180576 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb44031c40 con 0x7feb5c104260 2024-04-24T09:04:33.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.819+0000 7feb59600640 1 -- 172.21.15.69:0/2531180576 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44035030 con 0x7feb5c104260 2024-04-24T09:04:33.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.819+0000 7feb63c00640 1 -- 172.21.15.69:0/2531180576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb5c1a2360 con 0x7feb5c104260 2024-04-24T09:04:33.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.819+0000 7feb63c00640 1 -- 172.21.15.69:0/2531180576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb5c1a2800 con 0x7feb5c104260 2024-04-24T09:04:33.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.820+0000 7feb63c00640 1 -- 172.21.15.69:0/2531180576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb5c10b600 con 0x7feb5c104260 2024-04-24T09:04:33.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.822+0000 7feb59600640 1 -- 172.21.15.69:0/2531180576 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feb44042070 con 0x7feb5c104260 2024-04-24T09:04:33.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.824+0000 7feb59600640 1 --2- 172.21.15.69:0/2531180576 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7feb400a3c30 0x7feb400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:33.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.824+0000 7feb59600640 1 -- 172.21.15.69:0/2531180576 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7feb440f0ff0 con 0x7feb5c104260 2024-04-24T09:04:33.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.825+0000 7feb5be00640 1 --2- 172.21.15.69:0/2531180576 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7feb400a3c30 0x7feb400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:33.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.825+0000 7feb5be00640 1 --2- 172.21.15.69:0/2531180576 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7feb400a3c30 0x7feb400a60f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7feb4c005c40 tx=0x7feb4c005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:33.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:33.829+0000 7feb59600640 1 -- 172.21.15.69:0/2531180576 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb440bd610 con 0x7feb5c104260 2024-04-24T09:04:33.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:33 smithi130 ceph-mon[31553]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T09:04:33.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:33 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1513269767' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T09:04:33.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:33 smithi130 ceph-mon[31553]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T09:04:33.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:33 smithi130 ceph-mon[31553]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:04:33.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:33 smithi130 ceph-mon[31553]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-24T09:04:33.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:33 smithi130 ceph-mon[31553]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-24T09:04:33.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:33 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1255554006' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T09:04:33.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:33 smithi130 ceph-mon[31553]: pgmap v387: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:34.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.073+0000 7feb63c00640 1 -- 172.21.15.69:0/2531180576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7feb5c02d620 con 0x7feb5c104260 2024-04-24T09:04:34.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.610+0000 7feb59600640 1 -- 172.21.15.69:0/2531180576 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7feb440bcf10 con 0x7feb5c104260 2024-04-24T09:04:34.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.614+0000 7feb63c00640 1 -- 172.21.15.69:0/2531180576 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7feb400a3c30 msgr2=0x7feb400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:34.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.614+0000 7feb63c00640 1 --2- 172.21.15.69:0/2531180576 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7feb400a3c30 0x7feb400a60f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7feb4c005c40 tx=0x7feb4c005bd0 comp rx=0 tx=0).stop 2024-04-24T09:04:34.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.614+0000 7feb63c00640 1 -- 172.21.15.69:0/2531180576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb5c104260 msgr2=0x7feb5c19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:34.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.614+0000 7feb63c00640 1 --2- 172.21.15.69:0/2531180576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb5c104260 0x7feb5c19a4b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feb440314e0 tx=0x7feb44003c10 comp rx=0 tx=0).stop 2024-04-24T09:04:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.614+0000 7feb63c00640 1 -- 172.21.15.69:0/2531180576 shutdown_connections 2024-04-24T09:04:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.614+0000 7feb63c00640 1 --2- 172.21.15.69:0/2531180576 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7feb400a3c30 0x7feb400a60f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.614+0000 7feb63c00640 1 --2- 172.21.15.69:0/2531180576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb5c105e10 0x7feb5c1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:34.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.614+0000 7feb63c00640 1 --2- 172.21.15.69:0/2531180576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb5c105450 0x7feb5c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.614+0000 7feb63c00640 1 --2- 172.21.15.69:0/2531180576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb5c104260 0x7feb5c19a4b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.614+0000 7feb63c00640 1 -- 172.21.15.69:0/2531180576 >> 172.21.15.69:0/2531180576 conn(0x7feb5c0ff7f0 msgr2=0x7feb5c1087d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.614+0000 7feb63c00640 1 -- 172.21.15.69:0/2531180576 shutdown_connections 2024-04-24T09:04:34.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.614+0000 7feb63c00640 1 -- 172.21.15.69:0/2531180576 wait complete. 2024-04-24T09:04:34.636 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-24T09:04:34.636 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-24T09:04:34.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.840+0000 7f55c2200640 1 -- 172.21.15.69:0/1252574759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55bc1007f0 msgr2=0x7f55bc100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:34.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.840+0000 7f55c2200640 1 --2- 172.21.15.69:0/1252574759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55bc1007f0 0x7f55bc100c10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009900 tx=0x7f55b002a410 comp rx=0 tx=0).stop 2024-04-24T09:04:34.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.841+0000 7f55c2200640 1 -- 172.21.15.69:0/1252574759 shutdown_connections 2024-04-24T09:04:34.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.841+0000 7f55c2200640 1 --2- 172.21.15.69:0/1252574759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55bc101af0 0x7f55bc101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:34.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.841+0000 7f55c2200640 1 --2- 172.21.15.69:0/1252574759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55bc101150 0x7f55bc1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:34.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.841+0000 7f55c2200640 1 --2- 172.21.15.69:0/1252574759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55bc1007f0 0x7f55bc100c10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:34.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.841+0000 7f55c2200640 1 -- 172.21.15.69:0/1252574759 >> 172.21.15.69:0/1252574759 conn(0x7f55bc0fc3b0 msgr2=0x7f55bc0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:34.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.841+0000 7f55c2200640 1 -- 172.21.15.69:0/1252574759 shutdown_connections 2024-04-24T09:04:34.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.841+0000 7f55c2200640 1 -- 172.21.15.69:0/1252574759 wait complete. 2024-04-24T09:04:34.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.842+0000 7f55c2200640 1 Processor -- start 2024-04-24T09:04:34.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.842+0000 7f55c2200640 1 -- start start 2024-04-24T09:04:34.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.842+0000 7f55c2200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55bc1007f0 0x7f55bc19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:34.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.842+0000 7f55c2200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55bc101150 0x7f55bc19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:34.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.842+0000 7f55c2200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55bc101af0 0x7f55bc1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:34.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.842+0000 7f55c2200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc10fcc0 con 0x7f55bc101af0 2024-04-24T09:04:34.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.842+0000 7f55c2200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc10fb40 con 0x7f55bc1007f0 2024-04-24T09:04:34.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.842+0000 7f55c2200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc10fe40 con 0x7f55bc101150 2024-04-24T09:04:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.843+0000 7f55c1800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55bc101af0 0x7f55bc1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.843+0000 7f55c1800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55bc101af0 0x7f55bc1a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36394/0 (socket says 172.21.15.69:36394) 2024-04-24T09:04:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.843+0000 7f55c1800640 1 -- 172.21.15.69:0/3895383219 learned_addr learned my addr 172.21.15.69:0/3895383219 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.843+0000 7f55c0e00640 1 --2- 172.21.15.69:0/3895383219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55bc1007f0 0x7f55bc19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.843+0000 7f55bbe00640 1 --2- 172.21.15.69:0/3895383219 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55bc101150 0x7f55bc19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.843+0000 7f55c1800640 1 -- 172.21.15.69:0/3895383219 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55bc101150 msgr2=0x7f55bc19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.843+0000 7f55c1800640 1 --2- 172.21.15.69:0/3895383219 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55bc101150 0x7f55bc19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.843+0000 7f55c1800640 1 -- 172.21.15.69:0/3895383219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55bc1007f0 msgr2=0x7f55bc19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.843+0000 7f55c1800640 1 --2- 172.21.15.69:0/3895383219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55bc1007f0 0x7f55bc19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.843+0000 7f55c1800640 1 -- 172.21.15.69:0/3895383219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc1a1f50 con 0x7f55bc101af0 2024-04-24T09:04:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.843+0000 7f55c1800640 1 --2- 172.21.15.69:0/3895383219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55bc101af0 0x7f55bc1a18e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f55b400eb40 tx=0x7f55b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.844+0000 7f55b9600640 1 -- 172.21.15.69:0/3895383219 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4010430 con 0x7f55bc101af0 2024-04-24T09:04:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.844+0000 7f55b9600640 1 -- 172.21.15.69:0/3895383219 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55b4010a80 con 0x7f55bc101af0 2024-04-24T09:04:34.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.844+0000 7f55c2200640 1 -- 172.21.15.69:0/3895383219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55bc1a21e0 con 0x7f55bc101af0 2024-04-24T09:04:34.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.844+0000 7f55b9600640 1 -- 172.21.15.69:0/3895383219 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4018b60 con 0x7f55bc101af0 2024-04-24T09:04:34.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.844+0000 7f55c2200640 1 -- 172.21.15.69:0/3895383219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55bc1a2730 con 0x7f55bc101af0 2024-04-24T09:04:34.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.845+0000 7f55c2200640 1 -- 172.21.15.69:0/3895383219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55bc10b720 con 0x7f55bc101af0 2024-04-24T09:04:34.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.847+0000 7f55b9600640 1 -- 172.21.15.69:0/3895383219 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f55b4018d00 con 0x7f55bc101af0 2024-04-24T09:04:34.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.848+0000 7f55b9600640 1 --2- 172.21.15.69:0/3895383219 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f55a40a3bb0 0x7f55a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:34.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.849+0000 7f55b9600640 1 -- 172.21.15.69:0/3895383219 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f55b40d0ea0 con 0x7f55bc101af0 2024-04-24T09:04:34.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.849+0000 7f55c0e00640 1 --2- 172.21.15.69:0/3895383219 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f55a40a3bb0 0x7f55a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:34.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.850+0000 7f55c0e00640 1 --2- 172.21.15.69:0/3895383219 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f55a40a3bb0 0x7f55a40a6070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f55b0000c00 tx=0x7f55b00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:34.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:34.853+0000 7f55b9600640 1 -- 172.21.15.69:0/3895383219 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55b409d520 con 0x7f55bc101af0 2024-04-24T09:04:34.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:34 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1255554006' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T09:04:34.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:34 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] up:active 2024-04-24T09:04:34.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:34 smithi130 ceph-mon[31553]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T09:04:34.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:34 smithi130 ceph-mon[31553]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T09:04:34.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:34 smithi130 ceph-mon[31553]: Cluster is now healthy 2024-04-24T09:04:34.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:34 smithi130 ceph-mon[31553]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-24T09:04:34.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:34 smithi130 ceph-mon[31553]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-24T09:04:34.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:34 smithi130 ceph-mon[31553]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-24T09:04:34.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:34 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2531180576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T09:04:35.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:34 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1255554006' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T09:04:35.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:34 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] up:active 2024-04-24T09:04:35.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:34 smithi081 ceph-mon[30961]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T09:04:35.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:34 smithi081 ceph-mon[30961]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T09:04:35.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:34 smithi081 ceph-mon[30961]: Cluster is now healthy 2024-04-24T09:04:35.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:34 smithi081 ceph-mon[30961]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-24T09:04:35.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:34 smithi081 ceph-mon[30961]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-24T09:04:35.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:34 smithi081 ceph-mon[30961]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-24T09:04:35.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:34 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2531180576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T09:04:35.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.083+0000 7f55c2200640 1 -- 172.21.15.69:0/3895383219 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f55bc062440 con 0x7f55a40a3bb0 2024-04-24T09:04:35.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:34 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1255554006' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T09:04:35.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:34 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] up:active 2024-04-24T09:04:35.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:34 smithi069 ceph-mon[28170]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T09:04:35.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:34 smithi069 ceph-mon[28170]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T09:04:35.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:34 smithi069 ceph-mon[28170]: Cluster is now healthy 2024-04-24T09:04:35.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:34 smithi069 ceph-mon[28170]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-24T09:04:35.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:34 smithi069 ceph-mon[28170]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-24T09:04:35.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:34 smithi069 ceph-mon[28170]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-24T09:04:35.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:34 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2531180576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T09:04:35.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.141+0000 7f55b9600640 1 -- 172.21.15.69:0/3895383219 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f55bc062440 con 0x7f55a40a3bb0 2024-04-24T09:04:35.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.144+0000 7f55c2200640 1 -- 172.21.15.69:0/3895383219 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f55a40a3bb0 msgr2=0x7f55a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:35.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.144+0000 7f55c2200640 1 --2- 172.21.15.69:0/3895383219 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f55a40a3bb0 0x7f55a40a6070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f55b0000c00 tx=0x7f55b00314a0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.144+0000 7f55c2200640 1 -- 172.21.15.69:0/3895383219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55bc101af0 msgr2=0x7f55bc1a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:35.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.144+0000 7f55c2200640 1 --2- 172.21.15.69:0/3895383219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55bc101af0 0x7f55bc1a18e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f55b400eb40 tx=0x7f55b400c2d0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.144+0000 7f55c2200640 1 -- 172.21.15.69:0/3895383219 shutdown_connections 2024-04-24T09:04:35.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.144+0000 7f55c2200640 1 --2- 172.21.15.69:0/3895383219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55bc101af0 0x7f55bc1a18e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.144+0000 7f55c2200640 1 --2- 172.21.15.69:0/3895383219 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55bc101150 0x7f55bc19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.144+0000 7f55c2200640 1 --2- 172.21.15.69:0/3895383219 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f55a40a3bb0 0x7f55a40a6070 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.144+0000 7f55c2200640 1 --2- 172.21.15.69:0/3895383219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55bc1007f0 0x7f55bc19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.144+0000 7f55c2200640 1 -- 172.21.15.69:0/3895383219 >> 172.21.15.69:0/3895383219 conn(0x7f55bc0fc3b0 msgr2=0x7f55bc0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:35.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.144+0000 7f55c2200640 1 -- 172.21.15.69:0/3895383219 shutdown_connections 2024-04-24T09:04:35.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.145+0000 7f55c2200640 1 -- 172.21.15.69:0/3895383219 wait complete. 2024-04-24T09:04:35.169 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-24T09:04:35.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.372+0000 7fd25fe00640 1 -- 172.21.15.69:0/2438113955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd260104140 msgr2=0x7fd260104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:35.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.372+0000 7fd25fe00640 1 --2- 172.21.15.69:0/2438113955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd260104140 0x7fd260104560 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd250009900 tx=0x7fd25002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.372+0000 7fd25fe00640 1 -- 172.21.15.69:0/2438113955 shutdown_connections 2024-04-24T09:04:35.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.372+0000 7fd25fe00640 1 --2- 172.21.15.69:0/2438113955 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd260105cf0 0x7fd26010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.372+0000 7fd25fe00640 1 --2- 172.21.15.69:0/2438113955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd260105330 0x7fd2601057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.372+0000 7fd25fe00640 1 --2- 172.21.15.69:0/2438113955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd260104140 0x7fd260104560 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.372+0000 7fd25fe00640 1 -- 172.21.15.69:0/2438113955 >> 172.21.15.69:0/2438113955 conn(0x7fd2600ff6b0 msgr2=0x7fd260101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:35.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.373+0000 7fd25fe00640 1 -- 172.21.15.69:0/2438113955 shutdown_connections 2024-04-24T09:04:35.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.373+0000 7fd25fe00640 1 -- 172.21.15.69:0/2438113955 wait complete. 2024-04-24T09:04:35.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.374+0000 7fd25fe00640 1 Processor -- start 2024-04-24T09:04:35.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.374+0000 7fd25fe00640 1 -- start start 2024-04-24T09:04:35.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.374+0000 7fd25fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd260104140 0x7fd26019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:35.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.374+0000 7fd25fe00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd260105330 0x7fd26019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:35.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.374+0000 7fd25fe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd260105cf0 0x7fd2601a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:35.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.374+0000 7fd25fe00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd26010fc80 con 0x7fd260104140 2024-04-24T09:04:35.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.374+0000 7fd25fe00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd26010fb00 con 0x7fd260105330 2024-04-24T09:04:35.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.374+0000 7fd25fe00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd26010fe00 con 0x7fd260105cf0 2024-04-24T09:04:35.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.374+0000 7fd25ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd260104140 0x7fd26019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:35.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.374+0000 7fd25ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd260104140 0x7fd26019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36424/0 (socket says 172.21.15.69:36424) 2024-04-24T09:04:35.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.374+0000 7fd25ea00640 1 -- 172.21.15.69:0/2109580159 learned_addr learned my addr 172.21.15.69:0/2109580159 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:35.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.374+0000 7fd25e000640 1 --2- 172.21.15.69:0/2109580159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd260105330 0x7fd26019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:35.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.375+0000 7fd25f400640 1 --2- 172.21.15.69:0/2109580159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd260105cf0 0x7fd2601a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:35.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.375+0000 7fd25ea00640 1 -- 172.21.15.69:0/2109580159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd260105cf0 msgr2=0x7fd2601a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:35.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.375+0000 7fd25ea00640 1 --2- 172.21.15.69:0/2109580159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd260105cf0 0x7fd2601a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.375+0000 7fd25ea00640 1 -- 172.21.15.69:0/2109580159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd260105330 msgr2=0x7fd26019a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:35.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.375+0000 7fd25ea00640 1 --2- 172.21.15.69:0/2109580159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd260105330 0x7fd26019a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.375+0000 7fd25ea00640 1 -- 172.21.15.69:0/2109580159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2601a1fa0 con 0x7fd260104140 2024-04-24T09:04:35.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.375+0000 7fd25ea00640 1 --2- 172.21.15.69:0/2109580159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd260104140 0x7fd26019a380 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd2500314e0 tx=0x7fd250031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:35.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.375+0000 7fd257400640 1 -- 172.21.15.69:0/2109580159 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25003f030 con 0x7fd260104140 2024-04-24T09:04:35.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.375+0000 7fd257400640 1 -- 172.21.15.69:0/2109580159 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2500044b0 con 0x7fd260104140 2024-04-24T09:04:35.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.375+0000 7fd257400640 1 -- 172.21.15.69:0/2109580159 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250035070 con 0x7fd260104140 2024-04-24T09:04:35.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.375+0000 7fd25fe00640 1 -- 172.21.15.69:0/2109580159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2601a2230 con 0x7fd260104140 2024-04-24T09:04:35.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.376+0000 7fd25fe00640 1 -- 172.21.15.69:0/2109580159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2601a2720 con 0x7fd260104140 2024-04-24T09:04:35.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.377+0000 7fd25fe00640 1 -- 172.21.15.69:0/2109580159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd26010b580 con 0x7fd260104140 2024-04-24T09:04:35.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.379+0000 7fd257400640 1 -- 172.21.15.69:0/2109580159 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd250042030 con 0x7fd260104140 2024-04-24T09:04:35.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.380+0000 7fd257400640 1 --2- 172.21.15.69:0/2109580159 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd2440a3bb0 0x7fd2440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:35.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.380+0000 7fd257400640 1 -- 172.21.15.69:0/2109580159 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd2500f0df0 con 0x7fd260104140 2024-04-24T09:04:35.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.381+0000 7fd25e000640 1 --2- 172.21.15.69:0/2109580159 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd2440a3bb0 0x7fd2440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:35.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.381+0000 7fd25e000640 1 --2- 172.21.15.69:0/2109580159 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd2440a3bb0 0x7fd2440a6070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd26019b890 tx=0x7fd24c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:35.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.385+0000 7fd257400640 1 -- 172.21.15.69:0/2109580159 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd2500bd410 con 0x7fd260104140 2024-04-24T09:04:35.591 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:04:35 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:04:35.098+0000 7feca9000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:04:35.591 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:04:35 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:04:35.098+0000 7feca9000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:04:35.592 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:04:35 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:04:35.098+0000 7feca9000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:04:35.592 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:04:35 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:04:35.098+0000 7feca9000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:04:35.592 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:04:35 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:04:35.098+0000 7feca9000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:04:35.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.622+0000 7fd25fe00640 1 -- 172.21.15.69:0/2109580159 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd260062440 con 0x7fd2440a3bb0 2024-04-24T09:04:35.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.649+0000 7fd257400640 1 -- 172.21.15.69:0/2109580159 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd2500b9380 con 0x7fd260104140 2024-04-24T09:04:35.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.694+0000 7fd257400640 1 -- 172.21.15.69:0/2109580159 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd260062440 con 0x7fd2440a3bb0 2024-04-24T09:04:35.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.698+0000 7fd25fe00640 1 -- 172.21.15.69:0/2109580159 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd2440a3bb0 msgr2=0x7fd2440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:35.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.698+0000 7fd25fe00640 1 --2- 172.21.15.69:0/2109580159 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd2440a3bb0 0x7fd2440a6070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd26019b890 tx=0x7fd24c005f90 comp rx=0 tx=0).stop 2024-04-24T09:04:35.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.698+0000 7fd25fe00640 1 -- 172.21.15.69:0/2109580159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd260104140 msgr2=0x7fd26019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:35.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.698+0000 7fd25fe00640 1 --2- 172.21.15.69:0/2109580159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd260104140 0x7fd26019a380 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd2500314e0 tx=0x7fd250031a30 comp rx=0 tx=0).stop 2024-04-24T09:04:35.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.698+0000 7fd25fe00640 1 -- 172.21.15.69:0/2109580159 shutdown_connections 2024-04-24T09:04:35.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.698+0000 7fd25fe00640 1 --2- 172.21.15.69:0/2109580159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd260105cf0 0x7fd2601a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.698+0000 7fd25fe00640 1 --2- 172.21.15.69:0/2109580159 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd2440a3bb0 0x7fd2440a6070 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.698+0000 7fd25fe00640 1 --2- 172.21.15.69:0/2109580159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd260105330 0x7fd26019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.698+0000 7fd25fe00640 1 --2- 172.21.15.69:0/2109580159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd260104140 0x7fd26019a380 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.698+0000 7fd25fe00640 1 -- 172.21.15.69:0/2109580159 >> 172.21.15.69:0/2109580159 conn(0x7fd2600ff6b0 msgr2=0x7fd260108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:35.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.698+0000 7fd25fe00640 1 -- 172.21.15.69:0/2109580159 shutdown_connections 2024-04-24T09:04:35.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.698+0000 7fd25fe00640 1 -- 172.21.15.69:0/2109580159 wait complete. 2024-04-24T09:04:35.719 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-24T09:04:35.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.930+0000 7f8fff200640 1 -- 172.21.15.69:0/2652981907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9000105330 msgr2=0x7f90001057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:35.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.930+0000 7f8fff200640 1 --2- 172.21.15.69:0/2652981907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9000105330 0x7f90001057b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009900 tx=0x7f8fec02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.930+0000 7f8fff200640 1 -- 172.21.15.69:0/2652981907 shutdown_connections 2024-04-24T09:04:35.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.930+0000 7f8fff200640 1 --2- 172.21.15.69:0/2652981907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9000105cf0 0x7f900010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.930+0000 7f8fff200640 1 --2- 172.21.15.69:0/2652981907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9000105330 0x7f90001057b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.930+0000 7f8fff200640 1 --2- 172.21.15.69:0/2652981907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9000104140 0x7f9000104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.930+0000 7f8fff200640 1 -- 172.21.15.69:0/2652981907 >> 172.21.15.69:0/2652981907 conn(0x7f90000ff6b0 msgr2=0x7f9000101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:35.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.930+0000 7f8fff200640 1 -- 172.21.15.69:0/2652981907 shutdown_connections 2024-04-24T09:04:35.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.930+0000 7f8fff200640 1 -- 172.21.15.69:0/2652981907 wait complete. 2024-04-24T09:04:35.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.931+0000 7f8fff200640 1 Processor -- start 2024-04-24T09:04:35.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.931+0000 7f8fff200640 1 -- start start 2024-04-24T09:04:35.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.932+0000 7f8fff200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9000104140 0x7f900019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:35.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.932+0000 7f8fff200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9000105330 0x7f900019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:35.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.932+0000 7f8fff200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9000105cf0 0x7f90001a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:35.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.932+0000 7f8fff200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f900010fc80 con 0x7f9000105cf0 2024-04-24T09:04:35.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.932+0000 7f8fff200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f900010fb00 con 0x7f9000105330 2024-04-24T09:04:35.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.932+0000 7f8fff200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f900010fe00 con 0x7f9000104140 2024-04-24T09:04:35.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.932+0000 7f8ffe800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9000105cf0 0x7f90001a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:35.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.932+0000 7f8ffe800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9000105cf0 0x7f90001a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36440/0 (socket says 172.21.15.69:36440) 2024-04-24T09:04:35.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.932+0000 7f8ffe800640 1 -- 172.21.15.69:0/2898056483 learned_addr learned my addr 172.21.15.69:0/2898056483 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:35.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.932+0000 7f8ffd400640 1 --2- 172.21.15.69:0/2898056483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9000105330 0x7f900019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:35.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.932+0000 7f8ffde00640 1 --2- 172.21.15.69:0/2898056483 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9000104140 0x7f900019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:35.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.933+0000 7f8ffe800640 1 -- 172.21.15.69:0/2898056483 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9000104140 msgr2=0x7f900019a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:35.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.933+0000 7f8ffe800640 1 --2- 172.21.15.69:0/2898056483 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9000104140 0x7f900019a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.933+0000 7f8ffe800640 1 -- 172.21.15.69:0/2898056483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9000105330 msgr2=0x7f900019a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:35.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.933+0000 7f8ffe800640 1 --2- 172.21.15.69:0/2898056483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9000105330 0x7f900019a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:35.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.933+0000 7f8ffe800640 1 -- 172.21.15.69:0/2898056483 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90001a1f30 con 0x7f9000105cf0 2024-04-24T09:04:35.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.933+0000 7f8ffe800640 1 --2- 172.21.15.69:0/2898056483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9000105cf0 0x7f90001a18c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800c970 tx=0x7f8ff800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:35.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.933+0000 7f8ff6a00640 1 -- 172.21.15.69:0/2898056483 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8007bb0 con 0x7f9000105cf0 2024-04-24T09:04:35.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.933+0000 7f8ff6a00640 1 -- 172.21.15.69:0/2898056483 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ff8007d50 con 0x7f9000105cf0 2024-04-24T09:04:35.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.933+0000 7f8fff200640 1 -- 172.21.15.69:0/2898056483 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90001a2220 con 0x7f9000105cf0 2024-04-24T09:04:35.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.933+0000 7f8ff6a00640 1 -- 172.21.15.69:0/2898056483 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8018700 con 0x7f9000105cf0 2024-04-24T09:04:35.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.934+0000 7f8fff200640 1 -- 172.21.15.69:0/2898056483 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90001a2720 con 0x7f9000105cf0 2024-04-24T09:04:35.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.935+0000 7f8fff200640 1 -- 172.21.15.69:0/2898056483 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900010b530 con 0x7f9000105cf0 2024-04-24T09:04:35.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.936+0000 7f8ff6a00640 1 -- 172.21.15.69:0/2898056483 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8ff800fa20 con 0x7f9000105cf0 2024-04-24T09:04:35.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.938+0000 7f8ff6a00640 1 --2- 172.21.15.69:0/2898056483 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8fe40a3c40 0x7f8fe40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:35.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.938+0000 7f8ff6a00640 1 -- 172.21.15.69:0/2898056483 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8ff80d0b00 con 0x7f9000105cf0 2024-04-24T09:04:35.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.938+0000 7f8ffde00640 1 --2- 172.21.15.69:0/2898056483 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8fe40a3c40 0x7f8fe40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:35.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.939+0000 7f8ffde00640 1 --2- 172.21.15.69:0/2898056483 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8fe40a3c40 0x7f8fe40a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9000105190 tx=0x7f8ff0009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:35.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:35.943+0000 7f8ff6a00640 1 -- 172.21.15.69:0/2898056483 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ff8014030 con 0x7f9000105cf0 2024-04-24T09:04:35.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:35 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2531180576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T09:04:35.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:35 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] up:active 2024-04-24T09:04:35.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:35 smithi130 ceph-mon[31553]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-24T09:04:35.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:35 smithi130 ceph-mon[31553]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:04:35.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:35 smithi130 ceph-mon[31553]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:04:35.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:35 smithi130 ceph-mon[31553]: pgmap v388: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 94 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:36.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:35 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2531180576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T09:04:36.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:35 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] up:active 2024-04-24T09:04:36.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:35 smithi081 ceph-mon[30961]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-24T09:04:36.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:35 smithi081 ceph-mon[30961]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:04:36.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:35 smithi081 ceph-mon[30961]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:04:36.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:35 smithi081 ceph-mon[30961]: pgmap v388: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 94 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:36.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:35 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2531180576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T09:04:36.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:35 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] up:active 2024-04-24T09:04:36.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:35 smithi069 ceph-mon[28170]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-24T09:04:36.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:35 smithi069 ceph-mon[28170]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:04:36.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:35 smithi069 ceph-mon[28170]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:04:36.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:35 smithi069 ceph-mon[28170]: pgmap v388: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 94 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:36.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.179+0000 7f8fff200640 1 -- 172.21.15.69:0/2898056483 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f9000062440 con 0x7f8fe40a3c40 2024-04-24T09:04:36.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.192+0000 7f8ff6a00640 1 -- 172.21.15.69:0/2898056483 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9000062440 con 0x7f8fe40a3c40 2024-04-24T09:04:36.192 INFO:teuthology.orchestra.run.smithi069.stdout:/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 2024-04-24T09:04:36.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.196+0000 7f8fff200640 1 -- 172.21.15.69:0/2898056483 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8fe40a3c40 msgr2=0x7f8fe40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:36.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.196+0000 7f8fff200640 1 --2- 172.21.15.69:0/2898056483 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8fe40a3c40 0x7f8fe40a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9000105190 tx=0x7f8ff0009870 comp rx=0 tx=0).stop 2024-04-24T09:04:36.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.196+0000 7f8fff200640 1 -- 172.21.15.69:0/2898056483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9000105cf0 msgr2=0x7f90001a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:36.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.196+0000 7f8fff200640 1 --2- 172.21.15.69:0/2898056483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9000105cf0 0x7f90001a18c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800c970 tx=0x7f8ff800ce30 comp rx=0 tx=0).stop 2024-04-24T09:04:36.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.196+0000 7f8fff200640 1 -- 172.21.15.69:0/2898056483 shutdown_connections 2024-04-24T09:04:36.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.196+0000 7f8fff200640 1 --2- 172.21.15.69:0/2898056483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9000105cf0 0x7f90001a18c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.196+0000 7f8fff200640 1 --2- 172.21.15.69:0/2898056483 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8fe40a3c40 0x7f8fe40a6100 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.196+0000 7f8fff200640 1 --2- 172.21.15.69:0/2898056483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9000105330 0x7f900019a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.196+0000 7f8fff200640 1 --2- 172.21.15.69:0/2898056483 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9000104140 0x7f900019a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.196+0000 7f8fff200640 1 -- 172.21.15.69:0/2898056483 >> 172.21.15.69:0/2898056483 conn(0x7f90000ff6b0 msgr2=0x7f9000108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:36.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.196+0000 7f8fff200640 1 -- 172.21.15.69:0/2898056483 shutdown_connections 2024-04-24T09:04:36.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.196+0000 7f8fff200640 1 -- 172.21.15.69:0/2898056483 wait complete. 2024-04-24T09:04:36.215 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-24T09:04:36.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.422+0000 7fcfe7200640 1 -- 172.21.15.69:0/2727718482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfe8074a60 msgr2=0x7fcfe8074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:36.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.422+0000 7fcfe7200640 1 --2- 172.21.15.69:0/2727718482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfe8074a60 0x7fcfe8074ee0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009930 tx=0x7fcfd002a420 comp rx=0 tx=0).stop 2024-04-24T09:04:36.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.422+0000 7fcfe7200640 1 -- 172.21.15.69:0/2727718482 shutdown_connections 2024-04-24T09:04:36.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.422+0000 7fcfe7200640 1 --2- 172.21.15.69:0/2727718482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcfe810e560 0x7fcfe8110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.422+0000 7fcfe7200640 1 --2- 172.21.15.69:0/2727718482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfe8074a60 0x7fcfe8074ee0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.422+0000 7fcfe7200640 1 --2- 172.21.15.69:0/2727718482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfe8076040 0x7fcfe8074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.422+0000 7fcfe7200640 1 -- 172.21.15.69:0/2727718482 >> 172.21.15.69:0/2727718482 conn(0x7fcfe80fd500 msgr2=0x7fcfe80ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:36.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.422+0000 7fcfe7200640 1 -- 172.21.15.69:0/2727718482 shutdown_connections 2024-04-24T09:04:36.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.422+0000 7fcfe7200640 1 -- 172.21.15.69:0/2727718482 wait complete. 2024-04-24T09:04:36.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.423+0000 7fcfe7200640 1 Processor -- start 2024-04-24T09:04:36.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.423+0000 7fcfe7200640 1 -- start start 2024-04-24T09:04:36.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.424+0000 7fcfe7200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfe8074a60 0x7fcfe819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:36.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.424+0000 7fcfe7200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfe8076040 0x7fcfe819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:36.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.424+0000 7fcfe7200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcfe810e560 0x7fcfe81a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:36.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.424+0000 7fcfe7200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe8114020 con 0x7fcfe8076040 2024-04-24T09:04:36.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.424+0000 7fcfe7200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe8113ea0 con 0x7fcfe8074a60 2024-04-24T09:04:36.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.424+0000 7fcfe7200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe81141a0 con 0x7fcfe810e560 2024-04-24T09:04:36.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.424+0000 7fcfe5400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfe8076040 0x7fcfe819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:36.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.424+0000 7fcfe5400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfe8076040 0x7fcfe819ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36452/0 (socket says 172.21.15.69:36452) 2024-04-24T09:04:36.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.424+0000 7fcfe5400640 1 -- 172.21.15.69:0/2013794846 learned_addr learned my addr 172.21.15.69:0/2013794846 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:36.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.424+0000 7fcfe5e00640 1 --2- 172.21.15.69:0/2013794846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfe8074a60 0x7fcfe819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:36.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.424+0000 7fcfe6800640 1 --2- 172.21.15.69:0/2013794846 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcfe810e560 0x7fcfe81a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:36.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.425+0000 7fcfe5400640 1 -- 172.21.15.69:0/2013794846 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcfe810e560 msgr2=0x7fcfe81a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.425+0000 7fcfe5400640 1 --2- 172.21.15.69:0/2013794846 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcfe810e560 0x7fcfe81a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.425+0000 7fcfe5400640 1 -- 172.21.15.69:0/2013794846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfe8074a60 msgr2=0x7fcfe819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.425+0000 7fcfe5400640 1 --2- 172.21.15.69:0/2013794846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfe8074a60 0x7fcfe819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.425+0000 7fcfe5400640 1 -- 172.21.15.69:0/2013794846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe81a6370 con 0x7fcfe8076040 2024-04-24T09:04:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.425+0000 7fcfe5400640 1 --2- 172.21.15.69:0/2013794846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfe8076040 0x7fcfe819ec90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009900 tx=0x7fcfd0004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.425+0000 7fcfdea00640 1 -- 172.21.15.69:0/2013794846 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd003f030 con 0x7fcfe8076040 2024-04-24T09:04:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.425+0000 7fcfdea00640 1 -- 172.21.15.69:0/2013794846 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcfd0003cb0 con 0x7fcfe8076040 2024-04-24T09:04:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.425+0000 7fcfdea00640 1 -- 172.21.15.69:0/2013794846 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0035070 con 0x7fcfe8076040 2024-04-24T09:04:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.425+0000 7fcfe7200640 1 -- 172.21.15.69:0/2013794846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe81a6600 con 0x7fcfe8076040 2024-04-24T09:04:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.426+0000 7fcfe7200640 1 -- 172.21.15.69:0/2013794846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe81a6aa0 con 0x7fcfe8076040 2024-04-24T09:04:36.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.427+0000 7fcfe7200640 1 -- 172.21.15.69:0/2013794846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe8104730 con 0x7fcfe8076040 2024-04-24T09:04:36.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.429+0000 7fcfdea00640 1 -- 172.21.15.69:0/2013794846 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcfd0042030 con 0x7fcfe8076040 2024-04-24T09:04:36.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.430+0000 7fcfdea00640 1 --2- 172.21.15.69:0/2013794846 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcfc80a3cc0 0x7fcfc80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:36.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.431+0000 7fcfdea00640 1 -- 172.21.15.69:0/2013794846 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fcfd00f0fe0 con 0x7fcfe8076040 2024-04-24T09:04:36.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.431+0000 7fcfe5e00640 1 --2- 172.21.15.69:0/2013794846 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcfc80a3cc0 0x7fcfc80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:36.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.432+0000 7fcfe5e00640 1 --2- 172.21.15.69:0/2013794846 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcfc80a3cc0 0x7fcfc80a6180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8005e10 tx=0x7fcfd8005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:36.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.435+0000 7fcfdea00640 1 -- 172.21.15.69:0/2013794846 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcfd00bd4e0 con 0x7fcfe8076040 2024-04-24T09:04:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.670+0000 7fcfe7200640 1 -- 172.21.15.69:0/2013794846 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcfe80624d0 con 0x7fcfc80a3cc0 2024-04-24T09:04:36.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.736+0000 7fcfdea00640 1 -- 172.21.15.69:0/2013794846 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcfe80624d0 con 0x7fcfc80a3cc0 2024-04-24T09:04:36.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.739+0000 7fcfe7200640 1 -- 172.21.15.69:0/2013794846 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcfc80a3cc0 msgr2=0x7fcfc80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:36.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.739+0000 7fcfe7200640 1 --2- 172.21.15.69:0/2013794846 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcfc80a3cc0 0x7fcfc80a6180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8005e10 tx=0x7fcfd8005da0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.739+0000 7fcfe7200640 1 -- 172.21.15.69:0/2013794846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfe8076040 msgr2=0x7fcfe819ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:36.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.739+0000 7fcfe7200640 1 --2- 172.21.15.69:0/2013794846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfe8076040 0x7fcfe819ec90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009900 tx=0x7fcfd0004340 comp rx=0 tx=0).stop 2024-04-24T09:04:36.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.739+0000 7fcfe7200640 1 -- 172.21.15.69:0/2013794846 shutdown_connections 2024-04-24T09:04:36.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.739+0000 7fcfe7200640 1 --2- 172.21.15.69:0/2013794846 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcfe810e560 0x7fcfe81a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.739+0000 7fcfe7200640 1 --2- 172.21.15.69:0/2013794846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfe8076040 0x7fcfe819ec90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.739+0000 7fcfe7200640 1 --2- 172.21.15.69:0/2013794846 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcfc80a3cc0 0x7fcfc80a6180 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.739+0000 7fcfe7200640 1 --2- 172.21.15.69:0/2013794846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfe8074a60 0x7fcfe819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.739+0000 7fcfe7200640 1 -- 172.21.15.69:0/2013794846 >> 172.21.15.69:0/2013794846 conn(0x7fcfe80fd500 msgr2=0x7fcfe8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:36.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.739+0000 7fcfe7200640 1 -- 172.21.15.69:0/2013794846 shutdown_connections 2024-04-24T09:04:36.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.739+0000 7fcfe7200640 1 -- 172.21.15.69:0/2013794846 wait complete. 2024-04-24T09:04:36.759 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-24T09:04:36.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:36 smithi130 ceph-mon[31553]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:04:36.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:36 smithi130 ceph-mon[31553]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-04-24T09:04:36.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:36 smithi130 ceph-mon[31553]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:04:36.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.963+0000 7fbd24600640 1 -- 172.21.15.69:0/152709286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd1c104260 msgr2=0x7fbd1c104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:36.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.963+0000 7fbd24600640 1 --2- 172.21.15.69:0/152709286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd1c104260 0x7fbd1c104680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009900 tx=0x7fbd1002a410 comp rx=0 tx=0).stop 2024-04-24T09:04:36.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.964+0000 7fbd24600640 1 -- 172.21.15.69:0/152709286 shutdown_connections 2024-04-24T09:04:36.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.964+0000 7fbd24600640 1 --2- 172.21.15.69:0/152709286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd1c105e10 0x7fbd1c10c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.964+0000 7fbd24600640 1 --2- 172.21.15.69:0/152709286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd1c105450 0x7fbd1c1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.964+0000 7fbd24600640 1 --2- 172.21.15.69:0/152709286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd1c104260 0x7fbd1c104680 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.964+0000 7fbd24600640 1 -- 172.21.15.69:0/152709286 >> 172.21.15.69:0/152709286 conn(0x7fbd1c0ff7f0 msgr2=0x7fbd1c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:36.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.964+0000 7fbd24600640 1 -- 172.21.15.69:0/152709286 shutdown_connections 2024-04-24T09:04:36.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.964+0000 7fbd24600640 1 -- 172.21.15.69:0/152709286 wait complete. 2024-04-24T09:04:36.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.965+0000 7fbd24600640 1 Processor -- start 2024-04-24T09:04:36.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.965+0000 7fbd24600640 1 -- start start 2024-04-24T09:04:36.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.965+0000 7fbd24600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd1c104260 0x7fbd1c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:36.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.965+0000 7fbd24600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd1c105450 0x7fbd1c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:36.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.965+0000 7fbd24600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd1c105e10 0x7fbd1c1a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:36.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.965+0000 7fbd24600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c10fdf0 con 0x7fbd1c104260 2024-04-24T09:04:36.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.965+0000 7fbd24600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c10fc70 con 0x7fbd1c105e10 2024-04-24T09:04:36.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.965+0000 7fbd24600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c10ff70 con 0x7fbd1c105450 2024-04-24T09:04:36.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.965+0000 7fbd21c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd1c104260 0x7fbd1c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:36.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.965+0000 7fbd21c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd1c104260 0x7fbd1c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36478/0 (socket says 172.21.15.69:36478) 2024-04-24T09:04:36.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.965+0000 7fbd21c00640 1 -- 172.21.15.69:0/4072998268 learned_addr learned my addr 172.21.15.69:0/4072998268 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:36.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.966+0000 7fbd21200640 1 --2- 172.21.15.69:0/4072998268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd1c105450 0x7fbd1c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:36.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.966+0000 7fbd22600640 1 --2- 172.21.15.69:0/4072998268 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd1c105e10 0x7fbd1c1a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:36.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.966+0000 7fbd21c00640 1 -- 172.21.15.69:0/4072998268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd1c105450 msgr2=0x7fbd1c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:36.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.966+0000 7fbd21c00640 1 --2- 172.21.15.69:0/4072998268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd1c105450 0x7fbd1c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.966+0000 7fbd21c00640 1 -- 172.21.15.69:0/4072998268 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd1c105e10 msgr2=0x7fbd1c1a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:36.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.966+0000 7fbd21c00640 1 --2- 172.21.15.69:0/4072998268 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd1c105e10 0x7fbd1c1a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:36.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.966+0000 7fbd21c00640 1 -- 172.21.15.69:0/4072998268 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1c1a20b0 con 0x7fbd1c104260 2024-04-24T09:04:36.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.966+0000 7fbd21200640 1 --2- 172.21.15.69:0/4072998268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd1c105450 0x7fbd1c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:36.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.966+0000 7fbd21c00640 1 --2- 172.21.15.69:0/4072998268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd1c104260 0x7fbd1c19a490 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbd100314e0 tx=0x7fbd10003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:36.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.966+0000 7fbd1a000640 1 -- 172.21.15.69:0/4072998268 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1003f070 con 0x7fbd1c104260 2024-04-24T09:04:36.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.967+0000 7fbd1a000640 1 -- 172.21.15.69:0/4072998268 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd10031e40 con 0x7fbd1c104260 2024-04-24T09:04:36.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.967+0000 7fbd24600640 1 -- 172.21.15.69:0/4072998268 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1c1a2340 con 0x7fbd1c104260 2024-04-24T09:04:36.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.967+0000 7fbd1a000640 1 -- 172.21.15.69:0/4072998268 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10035030 con 0x7fbd1c104260 2024-04-24T09:04:36.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.967+0000 7fbd24600640 1 -- 172.21.15.69:0/4072998268 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd1c1a2830 con 0x7fbd1c104260 2024-04-24T09:04:36.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.968+0000 7fbd24600640 1 -- 172.21.15.69:0/4072998268 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd1c10b650 con 0x7fbd1c104260 2024-04-24T09:04:36.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.970+0000 7fbd1a000640 1 -- 172.21.15.69:0/4072998268 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbd10042070 con 0x7fbd1c104260 2024-04-24T09:04:36.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.972+0000 7fbd1a000640 1 --2- 172.21.15.69:0/4072998268 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbd040a3c40 0x7fbd040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:36.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.972+0000 7fbd1a000640 1 -- 172.21.15.69:0/4072998268 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fbd100f12a0 con 0x7fbd1c104260 2024-04-24T09:04:36.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.972+0000 7fbd21200640 1 --2- 172.21.15.69:0/4072998268 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbd040a3c40 0x7fbd040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:36.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.973+0000 7fbd21200640 1 --2- 172.21.15.69:0/4072998268 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbd040a3c40 0x7fbd040a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c19b9a0 tx=0x7fbd08005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:36.977+0000 7fbd1a000640 1 -- 172.21.15.69:0/4072998268 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd100bd7c0 con 0x7fbd1c104260 2024-04-24T09:04:37.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:36 smithi081 ceph-mon[30961]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:04:37.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:36 smithi081 ceph-mon[30961]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-04-24T09:04:37.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:36 smithi081 ceph-mon[30961]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:04:37.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:36 smithi069 ceph-mon[28170]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:04:37.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:36 smithi069 ceph-mon[28170]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-04-24T09:04:37.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:36 smithi069 ceph-mon[28170]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:04:37.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.209+0000 7fbd24600640 1 -- 172.21.15.69:0/4072998268 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fbd1c0624d0 con 0x7fbd040a3c40 2024-04-24T09:04:37.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.223+0000 7fbd1a000640 1 -- 172.21.15.69:0/4072998268 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fbd1c0624d0 con 0x7fbd040a3c40 2024-04-24T09:04:37.222 INFO:teuthology.orchestra.run.smithi069.stdout:/volumes/qa/sv_1/d284e29a-1752-46d6-b945-6de1a853c631 2024-04-24T09:04:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.225+0000 7fbd24600640 1 -- 172.21.15.69:0/4072998268 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbd040a3c40 msgr2=0x7fbd040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.225+0000 7fbd24600640 1 --2- 172.21.15.69:0/4072998268 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbd040a3c40 0x7fbd040a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c19b9a0 tx=0x7fbd08005e50 comp rx=0 tx=0).stop 2024-04-24T09:04:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.225+0000 7fbd24600640 1 -- 172.21.15.69:0/4072998268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd1c104260 msgr2=0x7fbd1c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.225+0000 7fbd24600640 1 --2- 172.21.15.69:0/4072998268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd1c104260 0x7fbd1c19a490 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbd100314e0 tx=0x7fbd10003c80 comp rx=0 tx=0).stop 2024-04-24T09:04:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.225+0000 7fbd24600640 1 -- 172.21.15.69:0/4072998268 shutdown_connections 2024-04-24T09:04:37.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.225+0000 7fbd24600640 1 --2- 172.21.15.69:0/4072998268 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbd040a3c40 0x7fbd040a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.225+0000 7fbd24600640 1 --2- 172.21.15.69:0/4072998268 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd1c105e10 0x7fbd1c1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.225+0000 7fbd24600640 1 --2- 172.21.15.69:0/4072998268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd1c105450 0x7fbd1c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.225+0000 7fbd24600640 1 --2- 172.21.15.69:0/4072998268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd1c104260 0x7fbd1c19a490 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.225+0000 7fbd24600640 1 -- 172.21.15.69:0/4072998268 >> 172.21.15.69:0/4072998268 conn(0x7fbd1c0ff7f0 msgr2=0x7fbd1c108820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:37.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.226+0000 7fbd24600640 1 -- 172.21.15.69:0/4072998268 shutdown_connections 2024-04-24T09:04:37.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.226+0000 7fbd24600640 1 -- 172.21.15.69:0/4072998268 wait complete. 2024-04-24T09:04:37.244 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:04:37.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.441+0000 7fa393e00640 1 -- 172.21.15.69:0/3290822908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa394101e70 msgr2=0x7fa394102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.441+0000 7fa393e00640 1 --2- 172.21.15.69:0/3290822908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa394101e70 0x7fa394102290 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa384009900 tx=0x7fa38402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.441+0000 7fa393e00640 1 -- 172.21.15.69:0/3290822908 shutdown_connections 2024-04-24T09:04:37.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.441+0000 7fa393e00640 1 --2- 172.21.15.69:0/3290822908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa394103980 0x7fa39410a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.441+0000 7fa393e00640 1 --2- 172.21.15.69:0/3290822908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa394102fc0 0x7fa394103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.441+0000 7fa393e00640 1 --2- 172.21.15.69:0/3290822908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa394101e70 0x7fa394102290 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.441+0000 7fa393e00640 1 -- 172.21.15.69:0/3290822908 >> 172.21.15.69:0/3290822908 conn(0x7fa3940fd3e0 msgr2=0x7fa3940ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:37.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.441+0000 7fa393e00640 1 -- 172.21.15.69:0/3290822908 shutdown_connections 2024-04-24T09:04:37.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.441+0000 7fa393e00640 1 -- 172.21.15.69:0/3290822908 wait complete. 2024-04-24T09:04:37.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.442+0000 7fa393e00640 1 Processor -- start 2024-04-24T09:04:37.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.443+0000 7fa393e00640 1 -- start start 2024-04-24T09:04:37.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.443+0000 7fa393e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa394101e70 0x7fa39419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.443+0000 7fa393e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa394102fc0 0x7fa39419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.443+0000 7fa393e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa394103980 0x7fa3941a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.443+0000 7fa393e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa39410da20 con 0x7fa394103980 2024-04-24T09:04:37.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.443+0000 7fa393e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa39410d8a0 con 0x7fa394102fc0 2024-04-24T09:04:37.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.443+0000 7fa393e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa39410dba0 con 0x7fa394101e70 2024-04-24T09:04:37.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.443+0000 7fa393400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa394103980 0x7fa3941a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.443+0000 7fa393400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa394103980 0x7fa3941a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36492/0 (socket says 172.21.15.69:36492) 2024-04-24T09:04:37.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.443+0000 7fa393400640 1 -- 172.21.15.69:0/1108815709 learned_addr learned my addr 172.21.15.69:0/1108815709 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:37.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.443+0000 7fa392a00640 1 --2- 172.21.15.69:0/1108815709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa394101e70 0x7fa39419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.444+0000 7fa392000640 1 --2- 172.21.15.69:0/1108815709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa394102fc0 0x7fa39419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.444+0000 7fa393400640 1 -- 172.21.15.69:0/1108815709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa394101e70 msgr2=0x7fa39419a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.444+0000 7fa393400640 1 --2- 172.21.15.69:0/1108815709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa394101e70 0x7fa39419a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.444+0000 7fa393400640 1 -- 172.21.15.69:0/1108815709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa394102fc0 msgr2=0x7fa39419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.444+0000 7fa393400640 1 --2- 172.21.15.69:0/1108815709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa394102fc0 0x7fa39419a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.444+0000 7fa393400640 1 -- 172.21.15.69:0/1108815709 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3941a1f00 con 0x7fa394103980 2024-04-24T09:04:37.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.444+0000 7fa392a00640 1 --2- 172.21.15.69:0/1108815709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa394101e70 0x7fa39419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:04:37.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.444+0000 7fa393400640 1 --2- 172.21.15.69:0/1108815709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa394103980 0x7fa3941a1890 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00b970 tx=0x7fa38c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:37.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.444+0000 7fa38b400640 1 -- 172.21.15.69:0/1108815709 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c019070 con 0x7fa394103980 2024-04-24T09:04:37.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.444+0000 7fa393e00640 1 -- 172.21.15.69:0/1108815709 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3941a21f0 con 0x7fa394103980 2024-04-24T09:04:37.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.444+0000 7fa38b400640 1 -- 172.21.15.69:0/1108815709 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa38c014440 con 0x7fa394103980 2024-04-24T09:04:37.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.444+0000 7fa38b400640 1 -- 172.21.15.69:0/1108815709 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c0075d0 con 0x7fa394103980 2024-04-24T09:04:37.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.444+0000 7fa393e00640 1 -- 172.21.15.69:0/1108815709 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3941a27c0 con 0x7fa394103980 2024-04-24T09:04:37.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.446+0000 7fa393e00640 1 -- 172.21.15.69:0/1108815709 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3941a2a80 con 0x7fa394103980 2024-04-24T09:04:37.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.448+0000 7fa38b400640 1 -- 172.21.15.69:0/1108815709 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa38c01c070 con 0x7fa394103980 2024-04-24T09:04:37.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.449+0000 7fa38b400640 1 --2- 172.21.15.69:0/1108815709 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3780a3c40 0x7fa3780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.449+0000 7fa38b400640 1 -- 172.21.15.69:0/1108815709 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa38c0cfe20 con 0x7fa394103980 2024-04-24T09:04:37.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.449+0000 7fa392a00640 1 --2- 172.21.15.69:0/1108815709 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3780a3c40 0x7fa3780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.450+0000 7fa392a00640 1 --2- 172.21.15.69:0/1108815709 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3780a3c40 0x7fa3780a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa3840314e0 tx=0x7fa384031d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:37.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.453+0000 7fa38b400640 1 -- 172.21.15.69:0/1108815709 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa38c09c3f0 con 0x7fa394103980 2024-04-24T09:04:37.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.702+0000 7fa393e00640 1 -- 172.21.15.69:0/1108815709 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3941a2c50 con 0x7fa394103980 2024-04-24T09:04:37.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.703+0000 7fa38b400640 1 -- 172.21.15.69:0/1108815709 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11873 (secure 0 0 0) 0x7fa38c09bcf0 con 0x7fa394103980 2024-04-24T09:04:37.703 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:37.703 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:04:01.991559+0000","modified":"2024-04-24T09:04:34.617681+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34532,"mds_1":34496,"mds_2":34460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34496":{"gid":34496,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34532":{"gid":34532,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34532,"qdb_cluster":[34460,34496,34532]},"id":1}]} 2024-04-24T09:04:37.704 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2024-04-24T09:04:37.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.706+0000 7fa393e00640 1 -- 172.21.15.69:0/1108815709 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3780a3c40 msgr2=0x7fa3780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.706+0000 7fa393e00640 1 --2- 172.21.15.69:0/1108815709 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3780a3c40 0x7fa3780a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa3840314e0 tx=0x7fa384031d30 comp rx=0 tx=0).stop 2024-04-24T09:04:37.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.706+0000 7fa393e00640 1 -- 172.21.15.69:0/1108815709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa394103980 msgr2=0x7fa3941a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.706+0000 7fa393e00640 1 --2- 172.21.15.69:0/1108815709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa394103980 0x7fa3941a1890 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00b970 tx=0x7fa38c00be30 comp rx=0 tx=0).stop 2024-04-24T09:04:37.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.706+0000 7fa393e00640 1 -- 172.21.15.69:0/1108815709 shutdown_connections 2024-04-24T09:04:37.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.706+0000 7fa393e00640 1 --2- 172.21.15.69:0/1108815709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa394103980 0x7fa3941a1890 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.706+0000 7fa393e00640 1 --2- 172.21.15.69:0/1108815709 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3780a3c40 0x7fa3780a6100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.706+0000 7fa393e00640 1 --2- 172.21.15.69:0/1108815709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa394102fc0 0x7fa39419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.706+0000 7fa393e00640 1 --2- 172.21.15.69:0/1108815709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa394101e70 0x7fa39419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.706+0000 7fa393e00640 1 -- 172.21.15.69:0/1108815709 >> 172.21.15.69:0/1108815709 conn(0x7fa3940fd3e0 msgr2=0x7fa394074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:37.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.706+0000 7fa393e00640 1 -- 172.21.15.69:0/1108815709 shutdown_connections 2024-04-24T09:04:37.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.706+0000 7fa393e00640 1 -- 172.21.15.69:0/1108815709 wait complete. 2024-04-24T09:04:37.725 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:04:37.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.909+0000 7f34eb200640 1 -- 172.21.15.69:0/3555918030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34ec108480 msgr2=0x7f34ec1088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.909+0000 7f34eb200640 1 --2- 172.21.15.69:0/3555918030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34ec108480 0x7f34ec1088a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009900 tx=0x7f34dc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.910+0000 7f34eb200640 1 -- 172.21.15.69:0/3555918030 shutdown_connections 2024-04-24T09:04:37.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.910+0000 7f34eb200640 1 --2- 172.21.15.69:0/3555918030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34ec10a030 0x7f34ec1108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.910+0000 7f34eb200640 1 --2- 172.21.15.69:0/3555918030 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34ec109670 0x7f34ec109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.910+0000 7f34eb200640 1 --2- 172.21.15.69:0/3555918030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34ec108480 0x7f34ec1088a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.910+0000 7f34eb200640 1 -- 172.21.15.69:0/3555918030 >> 172.21.15.69:0/3555918030 conn(0x7f34ec077440 msgr2=0x7f34ec079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:37.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.910+0000 7f34eb200640 1 -- 172.21.15.69:0/3555918030 shutdown_connections 2024-04-24T09:04:37.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.910+0000 7f34eb200640 1 -- 172.21.15.69:0/3555918030 wait complete. 2024-04-24T09:04:37.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.911+0000 7f34eb200640 1 Processor -- start 2024-04-24T09:04:37.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.911+0000 7f34eb200640 1 -- start start 2024-04-24T09:04:37.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.911+0000 7f34eb200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34ec108480 0x7f34ec19e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.911+0000 7f34eb200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34ec109670 0x7f34ec19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.911+0000 7f34eb200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34ec10a030 0x7f34ec1a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.911+0000 7f34eb200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec114010 con 0x7f34ec109670 2024-04-24T09:04:37.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.911+0000 7f34eb200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec113e90 con 0x7f34ec108480 2024-04-24T09:04:37.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.911+0000 7f34eb200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec114190 con 0x7f34ec10a030 2024-04-24T09:04:37.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.911+0000 7f34e3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34ec109670 0x7f34ec19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.911+0000 7f34e3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34ec109670 0x7f34ec19ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36508/0 (socket says 172.21.15.69:36508) 2024-04-24T09:04:37.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.911+0000 7f34e3e00640 1 -- 172.21.15.69:0/1481793712 learned_addr learned my addr 172.21.15.69:0/1481793712 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:37.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.912+0000 7f34e9e00640 1 --2- 172.21.15.69:0/1481793712 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34ec108480 0x7f34ec19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.912+0000 7f34ea800640 1 --2- 172.21.15.69:0/1481793712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34ec10a030 0x7f34ec1a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.912+0000 7f34e3e00640 1 -- 172.21.15.69:0/1481793712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34ec10a030 msgr2=0x7f34ec1a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.912+0000 7f34e3e00640 1 --2- 172.21.15.69:0/1481793712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34ec10a030 0x7f34ec1a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.912+0000 7f34e3e00640 1 -- 172.21.15.69:0/1481793712 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34ec108480 msgr2=0x7f34ec19e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.912+0000 7f34e3e00640 1 --2- 172.21.15.69:0/1481793712 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34ec108480 0x7f34ec19e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.912+0000 7f34e3e00640 1 -- 172.21.15.69:0/1481793712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ec1a62f0 con 0x7f34ec109670 2024-04-24T09:04:37.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.912+0000 7f34e3e00640 1 --2- 172.21.15.69:0/1481793712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34ec109670 0x7f34ec19ecb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f34d400d950 tx=0x7f34d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:37.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.912+0000 7f34e2a00640 1 -- 172.21.15.69:0/1481793712 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4017070 con 0x7f34ec109670 2024-04-24T09:04:37.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.912+0000 7f34eb200640 1 -- 172.21.15.69:0/1481793712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ec1a65e0 con 0x7f34ec109670 2024-04-24T09:04:37.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.912+0000 7f34e2a00640 1 -- 172.21.15.69:0/1481793712 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34d4009c30 con 0x7f34ec109670 2024-04-24T09:04:37.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.912+0000 7f34e2a00640 1 -- 172.21.15.69:0/1481793712 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d400f620 con 0x7f34ec109670 2024-04-24T09:04:37.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.913+0000 7f34eb200640 1 -- 172.21.15.69:0/1481793712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ec1a6b30 con 0x7f34ec109670 2024-04-24T09:04:37.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.914+0000 7f34eb200640 1 -- 172.21.15.69:0/1481793712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34ec10f7f0 con 0x7f34ec109670 2024-04-24T09:04:37.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.915+0000 7f34e2a00640 1 -- 172.21.15.69:0/1481793712 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f34d40109a0 con 0x7f34ec109670 2024-04-24T09:04:37.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.917+0000 7f34e2a00640 1 --2- 172.21.15.69:0/1481793712 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f34d00a3c40 0x7f34d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.918+0000 7f34e2a00640 1 -- 172.21.15.69:0/1481793712 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f34d40d05d0 con 0x7f34ec109670 2024-04-24T09:04:37.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.918+0000 7f34e9e00640 1 --2- 172.21.15.69:0/1481793712 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f34d00a3c40 0x7f34d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.918+0000 7f34e9e00640 1 --2- 172.21.15.69:0/1481793712 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f34d00a3c40 0x7f34d00a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f34dc006010 tx=0x7f34dc0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:37.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:37.923+0000 7f34e2a00640 1 -- 172.21.15.69:0/1481793712 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f34d4009de0 con 0x7f34ec109670 2024-04-24T09:04:37.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:37 smithi130 ceph-mon[31553]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:04:37.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:37 smithi130 ceph-mon[31553]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:04:37.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:37 smithi130 ceph-mon[31553]: pgmap v389: 129 pgs: 129 active+clean; 584 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1.9 KiB/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:38.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:37 smithi081 ceph-mon[30961]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:04:38.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:37 smithi081 ceph-mon[30961]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:04:38.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:37 smithi081 ceph-mon[30961]: pgmap v389: 129 pgs: 129 active+clean; 584 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1.9 KiB/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:38.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:37 smithi069 ceph-mon[28170]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:04:38.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:37 smithi069 ceph-mon[28170]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:04:38.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:37 smithi069 ceph-mon[28170]: pgmap v389: 129 pgs: 129 active+clean; 584 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1.9 KiB/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:38.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.124+0000 7f34eb200640 1 -- 172.21.15.69:0/1481793712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34ec02d870 con 0x7f34ec109670 2024-04-24T09:04:38.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.126+0000 7f34e2a00640 1 -- 172.21.15.69:0/1481793712 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24564 (secure 0 0 0) 0x7f34d409caf0 con 0x7f34ec109670 2024-04-24T09:04:38.126 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:38.127 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":92,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:24.497245+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:04:38.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.129+0000 7f34eb200640 1 -- 172.21.15.69:0/1481793712 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f34d00a3c40 msgr2=0x7f34d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:38.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.129+0000 7f34eb200640 1 --2- 172.21.15.69:0/1481793712 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f34d00a3c40 0x7f34d00a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f34dc006010 tx=0x7f34dc0314a0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.129+0000 7f34eb200640 1 -- 172.21.15.69:0/1481793712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34ec109670 msgr2=0x7f34ec19ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:38.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.129+0000 7f34eb200640 1 --2- 172.21.15.69:0/1481793712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34ec109670 0x7f34ec19ecb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f34d400d950 tx=0x7f34d400de10 comp rx=0 tx=0).stop 2024-04-24T09:04:38.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.129+0000 7f34eb200640 1 -- 172.21.15.69:0/1481793712 shutdown_connections 2024-04-24T09:04:38.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.129+0000 7f34eb200640 1 --2- 172.21.15.69:0/1481793712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34ec10a030 0x7f34ec1a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.129+0000 7f34eb200640 1 --2- 172.21.15.69:0/1481793712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34ec109670 0x7f34ec19ecb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.129+0000 7f34eb200640 1 --2- 172.21.15.69:0/1481793712 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f34d00a3c40 0x7f34d00a6100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.129+0000 7f34eb200640 1 --2- 172.21.15.69:0/1481793712 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34ec108480 0x7f34ec19e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.129+0000 7f34eb200640 1 -- 172.21.15.69:0/1481793712 >> 172.21.15.69:0/1481793712 conn(0x7f34ec077440 msgr2=0x7f34ec10c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:38.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.130+0000 7f34eb200640 1 -- 172.21.15.69:0/1481793712 shutdown_connections 2024-04-24T09:04:38.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.130+0000 7f34eb200640 1 -- 172.21.15.69:0/1481793712 wait complete. 2024-04-24T09:04:38.153 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T09:04:38.153 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T09:04:38.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.348+0000 7efd35400640 1 -- 172.21.15.69:0/3440367011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd30104260 msgr2=0x7efd30104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:38.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.348+0000 7efd35400640 1 --2- 172.21.15.69:0/3440367011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd30104260 0x7efd30104680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efd20009900 tx=0x7efd2002a410 comp rx=0 tx=0).stop 2024-04-24T09:04:38.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.348+0000 7efd35400640 1 -- 172.21.15.69:0/3440367011 shutdown_connections 2024-04-24T09:04:38.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.348+0000 7efd35400640 1 --2- 172.21.15.69:0/3440367011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd30105e10 0x7efd3010c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.348+0000 7efd35400640 1 --2- 172.21.15.69:0/3440367011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd30105450 0x7efd301058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.348+0000 7efd35400640 1 --2- 172.21.15.69:0/3440367011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd30104260 0x7efd30104680 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.348+0000 7efd35400640 1 -- 172.21.15.69:0/3440367011 >> 172.21.15.69:0/3440367011 conn(0x7efd300ff7f0 msgr2=0x7efd30101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:38.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.348+0000 7efd35400640 1 -- 172.21.15.69:0/3440367011 shutdown_connections 2024-04-24T09:04:38.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.348+0000 7efd35400640 1 -- 172.21.15.69:0/3440367011 wait complete. 2024-04-24T09:04:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.349+0000 7efd35400640 1 Processor -- start 2024-04-24T09:04:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.349+0000 7efd35400640 1 -- start start 2024-04-24T09:04:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.349+0000 7efd35400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd30104260 0x7efd3019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.350+0000 7efd35400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd30105450 0x7efd3019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.350+0000 7efd35400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd30105e10 0x7efd301a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.350+0000 7efd35400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efd3010fdf0 con 0x7efd30105450 2024-04-24T09:04:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.350+0000 7efd35400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efd3010fc70 con 0x7efd30104260 2024-04-24T09:04:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.350+0000 7efd35400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efd3010ff70 con 0x7efd30105e10 2024-04-24T09:04:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.350+0000 7efd27e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd30105450 0x7efd3019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.350+0000 7efd27e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd30105450 0x7efd3019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36532/0 (socket says 172.21.15.69:36532) 2024-04-24T09:04:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.350+0000 7efd27e00640 1 -- 172.21.15.69:0/1082675105 learned_addr learned my addr 172.21.15.69:0/1082675105 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.350+0000 7efd27e00640 1 -- 172.21.15.69:0/1082675105 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd30105e10 msgr2=0x7efd301a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.350+0000 7efd27e00640 1 --2- 172.21.15.69:0/1082675105 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd30105e10 0x7efd301a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.350+0000 7efd27e00640 1 -- 172.21.15.69:0/1082675105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd30104260 msgr2=0x7efd3019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.350+0000 7efd2e800640 1 --2- 172.21.15.69:0/1082675105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd30104260 0x7efd3019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.350+0000 7efd27e00640 1 --2- 172.21.15.69:0/1082675105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd30104260 0x7efd3019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.350+0000 7efd27e00640 1 -- 172.21.15.69:0/1082675105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd301a20a0 con 0x7efd30105450 2024-04-24T09:04:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.350+0000 7efd2e800640 1 --2- 172.21.15.69:0/1082675105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd30104260 0x7efd3019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.351+0000 7efd27e00640 1 --2- 172.21.15.69:0/1082675105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd30105450 0x7efd3019a9c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efd1800d950 tx=0x7efd1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.351+0000 7efd27400640 1 -- 172.21.15.69:0/1082675105 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18017070 con 0x7efd30105450 2024-04-24T09:04:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.351+0000 7efd27400640 1 -- 172.21.15.69:0/1082675105 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd18009c30 con 0x7efd30105450 2024-04-24T09:04:38.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.351+0000 7efd35400640 1 -- 172.21.15.69:0/1082675105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd301a2390 con 0x7efd30105450 2024-04-24T09:04:38.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.351+0000 7efd27400640 1 -- 172.21.15.69:0/1082675105 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1800f620 con 0x7efd30105450 2024-04-24T09:04:38.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.351+0000 7efd35400640 1 -- 172.21.15.69:0/1082675105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd301a28e0 con 0x7efd30105450 2024-04-24T09:04:38.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.352+0000 7efd35400640 1 -- 172.21.15.69:0/1082675105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd3010b600 con 0x7efd30105450 2024-04-24T09:04:38.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.354+0000 7efd27400640 1 -- 172.21.15.69:0/1082675105 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efd180109a0 con 0x7efd30105450 2024-04-24T09:04:38.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.355+0000 7efd27400640 1 --2- 172.21.15.69:0/1082675105 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efd140a3c40 0x7efd140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:38.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.356+0000 7efd27400640 1 -- 172.21.15.69:0/1082675105 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7efd180d0630 con 0x7efd30105450 2024-04-24T09:04:38.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.356+0000 7efd2e800640 1 --2- 172.21.15.69:0/1082675105 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efd140a3c40 0x7efd140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:38.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.356+0000 7efd2e800640 1 --2- 172.21.15.69:0/1082675105 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efd140a3c40 0x7efd140a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efd2002a930 tx=0x7efd20005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:38.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.360+0000 7efd27400640 1 -- 172.21.15.69:0/1082675105 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd18009de0 con 0x7efd30105450 2024-04-24T09:04:38.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.556+0000 7efd35400640 1 -- 172.21.15.69:0/1082675105 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efd30062440 con 0x7efd140a3c40 2024-04-24T09:04:38.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.577+0000 7efd27400640 1 -- 172.21.15.69:0/1082675105 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340426 (secure 0 0 0) 0x7efd30062440 con 0x7efd140a3c40 2024-04-24T09:04:38.581 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:38.583 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-24T09:04:38.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.583+0000 7efd35400640 1 -- 172.21.15.69:0/1082675105 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efd140a3c40 msgr2=0x7efd140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:38.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.583+0000 7efd35400640 1 --2- 172.21.15.69:0/1082675105 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efd140a3c40 0x7efd140a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efd2002a930 tx=0x7efd20005fd0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.583+0000 7efd35400640 1 -- 172.21.15.69:0/1082675105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd30105450 msgr2=0x7efd3019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:38.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.583+0000 7efd35400640 1 --2- 172.21.15.69:0/1082675105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd30105450 0x7efd3019a9c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efd1800d950 tx=0x7efd1800de10 comp rx=0 tx=0).stop 2024-04-24T09:04:38.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.583+0000 7efd35400640 1 -- 172.21.15.69:0/1082675105 shutdown_connections 2024-04-24T09:04:38.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.583+0000 7efd35400640 1 --2- 172.21.15.69:0/1082675105 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd30105e10 0x7efd301a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.583+0000 7efd35400640 1 --2- 172.21.15.69:0/1082675105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd30105450 0x7efd3019a9c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.583+0000 7efd35400640 1 --2- 172.21.15.69:0/1082675105 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efd140a3c40 0x7efd140a6100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.583+0000 7efd35400640 1 --2- 172.21.15.69:0/1082675105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd30104260 0x7efd3019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.583+0000 7efd35400640 1 -- 172.21.15.69:0/1082675105 >> 172.21.15.69:0/1082675105 conn(0x7efd300ff7f0 msgr2=0x7efd301087d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:38.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.583+0000 7efd35400640 1 -- 172.21.15.69:0/1082675105 shutdown_connections 2024-04-24T09:04:38.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.583+0000 7efd35400640 1 -- 172.21.15.69:0/1082675105 wait complete. 2024-04-24T09:04:38.602 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-04-24T09:04:37.384365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597539,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":158,"num_read_kb":137,"num_write":290,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":354848,"kb_used_data":14004,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124586976,"statfs":{"total":1151940427776,"available":1151577063424,"internally_reserved":0,"allocated":14340096,"data_stored":7834209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2531,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":22,"num_read_kb":18,"num_write":60,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005587"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533023+0000","last_change":"2024-04-24T09:04:02.107965+0000","last_active":"2024-04-24T09:04:24.533023+0000","last_peered":"2024-04-24T09:04:24.533023+0000","last_clean":"2024-04-24T09:04:24.533023+0000","last_became_active":"2024-04-24T09:04:02.106827+0000","last_became_peered":"2024-04-24T09:04:02.106827+0000","last_unstale":"2024-04-24T09:04:24.533023+0000","last_undegraded":"2024-04-24T09:04:24.533023+0000","last_fullsized":"2024-04-24T09:04:24.533023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:20:23.930599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696753+0000","last_change":"2024-04-24T09:04:02.105449+0000","last_active":"2024-04-24T09:04:24.696753+0000","last_peered":"2024-04-24T09:04:24.696753+0000","last_clean":"2024-04-24T09:04:24.696753+0000","last_became_active":"2024-04-24T09:04:02.101842+0000","last_became_peered":"2024-04-24T09:04:02.101842+0000","last_unstale":"2024-04-24T09:04:24.696753+0000","last_undegraded":"2024-04-24T09:04:24.696753+0000","last_fullsized":"2024-04-24T09:04:24.696753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:46.911768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696652+0000","last_change":"2024-04-24T09:04:02.118686+0000","last_active":"2024-04-24T09:04:24.696652+0000","last_peered":"2024-04-24T09:04:24.696652+0000","last_clean":"2024-04-24T09:04:24.696652+0000","last_became_active":"2024-04-24T09:04:02.117549+0000","last_became_peered":"2024-04-24T09:04:02.117549+0000","last_unstale":"2024-04-24T09:04:24.696652+0000","last_undegraded":"2024-04-24T09:04:24.696652+0000","last_fullsized":"2024-04-24T09:04:24.696652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:18:16.233434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.521592+0000","last_change":"2024-04-24T09:04:02.120336+0000","last_active":"2024-04-24T09:04:24.521592+0000","last_peered":"2024-04-24T09:04:24.521592+0000","last_clean":"2024-04-24T09:04:24.521592+0000","last_became_active":"2024-04-24T09:04:02.116574+0000","last_became_peered":"2024-04-24T09:04:02.116574+0000","last_unstale":"2024-04-24T09:04:24.521592+0000","last_undegraded":"2024-04-24T09:04:24.521592+0000","last_fullsized":"2024-04-24T09:04:24.521592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:10:42.366531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.516713+0000","last_change":"2024-04-24T09:04:02.113600+0000","last_active":"2024-04-24T09:04:24.516713+0000","last_peered":"2024-04-24T09:04:24.516713+0000","last_clean":"2024-04-24T09:04:24.516713+0000","last_became_active":"2024-04-24T09:04:02.113143+0000","last_became_peered":"2024-04-24T09:04:02.113143+0000","last_unstale":"2024-04-24T09:04:24.516713+0000","last_undegraded":"2024-04-24T09:04:24.516713+0000","last_fullsized":"2024-04-24T09:04:24.516713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:31:27.666661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527374+0000","last_change":"2024-04-24T09:04:02.106395+0000","last_active":"2024-04-24T09:04:24.527374+0000","last_peered":"2024-04-24T09:04:24.527374+0000","last_clean":"2024-04-24T09:04:24.527374+0000","last_became_active":"2024-04-24T09:04:02.105942+0000","last_became_peered":"2024-04-24T09:04:02.105942+0000","last_unstale":"2024-04-24T09:04:24.527374+0000","last_undegraded":"2024-04-24T09:04:24.527374+0000","last_fullsized":"2024-04-24T09:04:24.527374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:20:10.655605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531485+0000","last_change":"2024-04-24T09:04:02.100279+0000","last_active":"2024-04-24T09:04:24.531485+0000","last_peered":"2024-04-24T09:04:24.531485+0000","last_clean":"2024-04-24T09:04:24.531485+0000","last_became_active":"2024-04-24T09:04:02.099745+0000","last_became_peered":"2024-04-24T09:04:02.099745+0000","last_unstale":"2024-04-24T09:04:24.531485+0000","last_undegraded":"2024-04-24T09:04:24.531485+0000","last_fullsized":"2024-04-24T09:04:24.531485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:13.150412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.526156+0000","last_change":"2024-04-24T09:04:02.103294+0000","last_active":"2024-04-24T09:04:24.526156+0000","last_peered":"2024-04-24T09:04:24.526156+0000","last_clean":"2024-04-24T09:04:24.526156+0000","last_became_active":"2024-04-24T09:04:02.102114+0000","last_became_peered":"2024-04-24T09:04:02.102114+0000","last_unstale":"2024-04-24T09:04:24.526156+0000","last_undegraded":"2024-04-24T09:04:24.526156+0000","last_fullsized":"2024-04-24T09:04:24.526156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:26:26.916097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.698004+0000","last_change":"2024-04-24T09:04:02.104936+0000","last_active":"2024-04-24T09:04:24.698004+0000","last_peered":"2024-04-24T09:04:24.698004+0000","last_clean":"2024-04-24T09:04:24.698004+0000","last_became_active":"2024-04-24T09:04:02.104540+0000","last_became_peered":"2024-04-24T09:04:02.104540+0000","last_unstale":"2024-04-24T09:04:24.698004+0000","last_undegraded":"2024-04-24T09:04:24.698004+0000","last_fullsized":"2024-04-24T09:04:24.698004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:36.010735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.528809+0000","last_change":"2024-04-24T09:04:02.102050+0000","last_active":"2024-04-24T09:04:24.528809+0000","last_peered":"2024-04-24T09:04:24.528809+0000","last_clean":"2024-04-24T09:04:24.528809+0000","last_became_active":"2024-04-24T09:04:02.101623+0000","last_became_peered":"2024-04-24T09:04:02.101623+0000","last_unstale":"2024-04-24T09:04:24.528809+0000","last_undegraded":"2024-04-24T09:04:24.528809+0000","last_fullsized":"2024-04-24T09:04:24.528809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:13:33.606200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533131+0000","last_change":"2024-04-24T09:04:02.115804+0000","last_active":"2024-04-24T09:04:24.533131+0000","last_peered":"2024-04-24T09:04:24.533131+0000","last_clean":"2024-04-24T09:04:24.533131+0000","last_became_active":"2024-04-24T09:04:02.115093+0000","last_became_peered":"2024-04-24T09:04:02.115093+0000","last_unstale":"2024-04-24T09:04:24.533131+0000","last_undegraded":"2024-04-24T09:04:24.533131+0000","last_fullsized":"2024-04-24T09:04:24.533131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:26:56.764485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.521254+0000","last_change":"2024-04-24T09:04:02.116305+0000","last_active":"2024-04-24T09:04:24.521254+0000","last_peered":"2024-04-24T09:04:24.521254+0000","last_clean":"2024-04-24T09:04:24.521254+0000","last_became_active":"2024-04-24T09:04:02.115956+0000","last_became_peered":"2024-04-24T09:04:02.115956+0000","last_unstale":"2024-04-24T09:04:24.521254+0000","last_undegraded":"2024-04-24T09:04:24.521254+0000","last_fullsized":"2024-04-24T09:04:24.521254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:38.618317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531595+0000","last_change":"2024-04-24T09:04:02.101155+0000","last_active":"2024-04-24T09:04:24.531595+0000","last_peered":"2024-04-24T09:04:24.531595+0000","last_clean":"2024-04-24T09:04:24.531595+0000","last_became_active":"2024-04-24T09:04:02.099719+0000","last_became_peered":"2024-04-24T09:04:02.099719+0000","last_unstale":"2024-04-24T09:04:24.531595+0000","last_undegraded":"2024-04-24T09:04:24.531595+0000","last_fullsized":"2024-04-24T09:04:24.531595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:46:27.910987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696158+0000","last_change":"2024-04-24T09:04:02.114818+0000","last_active":"2024-04-24T09:04:24.696158+0000","last_peered":"2024-04-24T09:04:24.696158+0000","last_clean":"2024-04-24T09:04:24.696158+0000","last_became_active":"2024-04-24T09:04:02.112947+0000","last_became_peered":"2024-04-24T09:04:02.112947+0000","last_unstale":"2024-04-24T09:04:24.696158+0000","last_undegraded":"2024-04-24T09:04:24.696158+0000","last_fullsized":"2024-04-24T09:04:24.696158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:16.544420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.695929+0000","last_change":"2024-04-24T09:04:02.120672+0000","last_active":"2024-04-24T09:04:24.695929+0000","last_peered":"2024-04-24T09:04:24.695929+0000","last_clean":"2024-04-24T09:04:24.695929+0000","last_became_active":"2024-04-24T09:04:02.120378+0000","last_became_peered":"2024-04-24T09:04:02.120378+0000","last_unstale":"2024-04-24T09:04:24.695929+0000","last_undegraded":"2024-04-24T09:04:24.695929+0000","last_fullsized":"2024-04-24T09:04:24.695929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:17.781189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696353+0000","last_change":"2024-04-24T09:04:02.107022+0000","last_active":"2024-04-24T09:04:24.696353+0000","last_peered":"2024-04-24T09:04:24.696353+0000","last_clean":"2024-04-24T09:04:24.696353+0000","last_became_active":"2024-04-24T09:04:02.106645+0000","last_became_peered":"2024-04-24T09:04:02.106645+0000","last_unstale":"2024-04-24T09:04:24.696353+0000","last_undegraded":"2024-04-24T09:04:24.696353+0000","last_fullsized":"2024-04-24T09:04:24.696353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:47.217718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532776+0000","last_change":"2024-04-24T09:04:02.115511+0000","last_active":"2024-04-24T09:04:24.532776+0000","last_peered":"2024-04-24T09:04:24.532776+0000","last_clean":"2024-04-24T09:04:24.532776+0000","last_became_active":"2024-04-24T09:04:02.115197+0000","last_became_peered":"2024-04-24T09:04:02.115197+0000","last_unstale":"2024-04-24T09:04:24.532776+0000","last_undegraded":"2024-04-24T09:04:24.532776+0000","last_fullsized":"2024-04-24T09:04:24.532776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:18.883688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.518672+0000","last_change":"2024-04-24T09:04:02.102108+0000","last_active":"2024-04-24T09:04:24.518672+0000","last_peered":"2024-04-24T09:04:24.518672+0000","last_clean":"2024-04-24T09:04:24.518672+0000","last_became_active":"2024-04-24T09:04:02.101739+0000","last_became_peered":"2024-04-24T09:04:02.101739+0000","last_unstale":"2024-04-24T09:04:24.518672+0000","last_undegraded":"2024-04-24T09:04:24.518672+0000","last_fullsized":"2024-04-24T09:04:24.518672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:57:30.768716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697591+0000","last_change":"2024-04-24T09:04:02.108911+0000","last_active":"2024-04-24T09:04:24.697591+0000","last_peered":"2024-04-24T09:04:24.697591+0000","last_clean":"2024-04-24T09:04:24.697591+0000","last_became_active":"2024-04-24T09:04:02.105678+0000","last_became_peered":"2024-04-24T09:04:02.105678+0000","last_unstale":"2024-04-24T09:04:24.697591+0000","last_undegraded":"2024-04-24T09:04:24.697591+0000","last_fullsized":"2024-04-24T09:04:24.697591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:36:12.216271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696112+0000","last_change":"2024-04-24T09:04:06.136320+0000","last_active":"2024-04-24T09:04:24.696112+0000","last_peered":"2024-04-24T09:04:24.696112+0000","last_clean":"2024-04-24T09:04:24.696112+0000","last_became_active":"2024-04-24T09:04:06.133851+0000","last_became_peered":"2024-04-24T09:04:06.133851+0000","last_unstale":"2024-04-24T09:04:24.696112+0000","last_undegraded":"2024-04-24T09:04:24.696112+0000","last_fullsized":"2024-04-24T09:04:24.696112+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:31:39.781234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.519248+0000","last_change":"2024-04-24T09:04:01.543818+0000","last_active":"2024-04-24T09:04:24.519248+0000","last_peered":"2024-04-24T09:04:24.519248+0000","last_clean":"2024-04-24T09:04:24.519248+0000","last_became_active":"2024-04-24T09:04:01.543435+0000","last_became_peered":"2024-04-24T09:04:01.543435+0000","last_unstale":"2024-04-24T09:04:24.519248+0000","last_undegraded":"2024-04-24T09:04:24.519248+0000","last_fullsized":"2024-04-24T09:04:24.519248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:08:30.176211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.526712+0000","last_change":"2024-04-24T09:04:02.101646+0000","last_active":"2024-04-24T09:04:24.526712+0000","last_peered":"2024-04-24T09:04:24.526712+0000","last_clean":"2024-04-24T09:04:24.526712+0000","last_became_active":"2024-04-24T09:04:02.101284+0000","last_became_peered":"2024-04-24T09:04:02.101284+0000","last_unstale":"2024-04-24T09:04:24.526712+0000","last_undegraded":"2024-04-24T09:04:24.526712+0000","last_fullsized":"2024-04-24T09:04:24.526712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:36:32.827216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.519708+0000","last_change":"2024-04-24T09:04:06.130476+0000","last_active":"2024-04-24T09:04:24.519708+0000","last_peered":"2024-04-24T09:04:24.519708+0000","last_clean":"2024-04-24T09:04:24.519708+0000","last_became_active":"2024-04-24T09:04:06.129707+0000","last_became_peered":"2024-04-24T09:04:06.129707+0000","last_unstale":"2024-04-24T09:04:24.519708+0000","last_undegraded":"2024-04-24T09:04:24.519708+0000","last_fullsized":"2024-04-24T09:04:24.519708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:25:36.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527016+0000","last_change":"2024-04-24T09:04:01.551708+0000","last_active":"2024-04-24T09:04:24.527016+0000","last_peered":"2024-04-24T09:04:24.527016+0000","last_clean":"2024-04-24T09:04:24.527016+0000","last_became_active":"2024-04-24T09:04:01.551089+0000","last_became_peered":"2024-04-24T09:04:01.551089+0000","last_unstale":"2024-04-24T09:04:24.527016+0000","last_undegraded":"2024-04-24T09:04:24.527016+0000","last_fullsized":"2024-04-24T09:04:24.527016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:18:22.515729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696599+0000","last_change":"2024-04-24T09:04:02.117356+0000","last_active":"2024-04-24T09:04:24.696599+0000","last_peered":"2024-04-24T09:04:24.696599+0000","last_clean":"2024-04-24T09:04:24.696599+0000","last_became_active":"2024-04-24T09:04:02.116972+0000","last_became_peered":"2024-04-24T09:04:02.116972+0000","last_unstale":"2024-04-24T09:04:24.696599+0000","last_undegraded":"2024-04-24T09:04:24.696599+0000","last_fullsized":"2024-04-24T09:04:24.696599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:46:29.349952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.520686+0000","last_change":"2024-04-24T09:04:06.141538+0000","last_active":"2024-04-24T09:04:24.520686+0000","last_peered":"2024-04-24T09:04:24.520686+0000","last_clean":"2024-04-24T09:04:24.520686+0000","last_became_active":"2024-04-24T09:04:06.140975+0000","last_became_peered":"2024-04-24T09:04:06.140975+0000","last_unstale":"2024-04-24T09:04:24.520686+0000","last_undegraded":"2024-04-24T09:04:24.520686+0000","last_fullsized":"2024-04-24T09:04:24.520686+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:58:31.688828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.630038+0000","last_change":"2024-04-24T09:04:01.561039+0000","last_active":"2024-04-24T09:04:33.630038+0000","last_peered":"2024-04-24T09:04:33.630038+0000","last_clean":"2024-04-24T09:04:33.630038+0000","last_became_active":"2024-04-24T09:04:01.560594+0000","last_became_peered":"2024-04-24T09:04:01.560594+0000","last_unstale":"2024-04-24T09:04:33.630038+0000","last_undegraded":"2024-04-24T09:04:33.630038+0000","last_fullsized":"2024-04-24T09:04:33.630038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:59:19.707826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697386+0000","last_change":"2024-04-24T09:04:02.106007+0000","last_active":"2024-04-24T09:04:24.697386+0000","last_peered":"2024-04-24T09:04:24.697386+0000","last_clean":"2024-04-24T09:04:24.697386+0000","last_became_active":"2024-04-24T09:04:02.099786+0000","last_became_peered":"2024-04-24T09:04:02.099786+0000","last_unstale":"2024-04-24T09:04:24.697386+0000","last_undegraded":"2024-04-24T09:04:24.697386+0000","last_fullsized":"2024-04-24T09:04:24.697386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:56.288417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.520162+0000","last_change":"2024-04-24T09:04:06.143636+0000","last_active":"2024-04-24T09:04:24.520162+0000","last_peered":"2024-04-24T09:04:24.520162+0000","last_clean":"2024-04-24T09:04:24.520162+0000","last_became_active":"2024-04-24T09:04:06.143105+0000","last_became_peered":"2024-04-24T09:04:06.143105+0000","last_unstale":"2024-04-24T09:04:24.520162+0000","last_undegraded":"2024-04-24T09:04:24.520162+0000","last_fullsized":"2024-04-24T09:04:24.520162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:04:23.211067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.517222+0000","last_change":"2024-04-24T09:04:01.548403+0000","last_active":"2024-04-24T09:04:24.517222+0000","last_peered":"2024-04-24T09:04:24.517222+0000","last_clean":"2024-04-24T09:04:24.517222+0000","last_became_active":"2024-04-24T09:04:01.547447+0000","last_became_peered":"2024-04-24T09:04:01.547447+0000","last_unstale":"2024-04-24T09:04:24.517222+0000","last_undegraded":"2024-04-24T09:04:24.517222+0000","last_fullsized":"2024-04-24T09:04:24.517222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:14:08.160127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533442+0000","last_change":"2024-04-24T09:04:02.121410+0000","last_active":"2024-04-24T09:04:24.533442+0000","last_peered":"2024-04-24T09:04:24.533442+0000","last_clean":"2024-04-24T09:04:24.533442+0000","last_became_active":"2024-04-24T09:04:02.120821+0000","last_became_peered":"2024-04-24T09:04:02.120821+0000","last_unstale":"2024-04-24T09:04:24.533442+0000","last_undegraded":"2024-04-24T09:04:24.533442+0000","last_fullsized":"2024-04-24T09:04:24.533442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:39:05.525171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696856+0000","last_change":"2024-04-24T09:04:06.142349+0000","last_active":"2024-04-24T09:04:24.696856+0000","last_peered":"2024-04-24T09:04:24.696856+0000","last_clean":"2024-04-24T09:04:24.696856+0000","last_became_active":"2024-04-24T09:04:06.141911+0000","last_became_peered":"2024-04-24T09:04:06.141911+0000","last_unstale":"2024-04-24T09:04:24.696856+0000","last_undegraded":"2024-04-24T09:04:24.696856+0000","last_fullsized":"2024-04-24T09:04:24.696856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:45:30.414686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'5","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:34.634991+0000","last_change":"2024-04-24T09:04:24.537349+0000","last_active":"2024-04-24T09:04:34.634991+0000","last_peered":"2024-04-24T09:04:34.634991+0000","last_clean":"2024-04-24T09:04:34.634991+0000","last_became_active":"2024-04-24T09:04:24.536738+0000","last_became_peered":"2024-04-24T09:04:24.536738+0000","last_unstale":"2024-04-24T09:04:34.634991+0000","last_undegraded":"2024-04-24T09:04:34.634991+0000","last_fullsized":"2024-04-24T09:04:34.634991+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:33.481380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.516838+0000","last_change":"2024-04-24T09:04:02.111173+0000","last_active":"2024-04-24T09:04:24.516838+0000","last_peered":"2024-04-24T09:04:24.516838+0000","last_clean":"2024-04-24T09:04:24.516838+0000","last_became_active":"2024-04-24T09:04:02.110681+0000","last_became_peered":"2024-04-24T09:04:02.110681+0000","last_unstale":"2024-04-24T09:04:24.516838+0000","last_undegraded":"2024-04-24T09:04:24.516838+0000","last_fullsized":"2024-04-24T09:04:24.516838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:29.939116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696716+0000","last_change":"2024-04-24T09:04:06.137752+0000","last_active":"2024-04-24T09:04:24.696716+0000","last_peered":"2024-04-24T09:04:24.696716+0000","last_clean":"2024-04-24T09:04:24.696716+0000","last_became_active":"2024-04-24T09:04:06.137492+0000","last_became_peered":"2024-04-24T09:04:06.137492+0000","last_unstale":"2024-04-24T09:04:24.696716+0000","last_undegraded":"2024-04-24T09:04:24.696716+0000","last_fullsized":"2024-04-24T09:04:24.696716+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:44:00.653864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.628109+0000","last_change":"2024-04-24T09:04:01.544009+0000","last_active":"2024-04-24T09:04:33.628109+0000","last_peered":"2024-04-24T09:04:33.628109+0000","last_clean":"2024-04-24T09:04:33.628109+0000","last_became_active":"2024-04-24T09:04:01.543651+0000","last_became_peered":"2024-04-24T09:04:01.543651+0000","last_unstale":"2024-04-24T09:04:33.628109+0000","last_undegraded":"2024-04-24T09:04:33.628109+0000","last_fullsized":"2024-04-24T09:04:33.628109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:10:21.969072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531764+0000","last_change":"2024-04-24T09:04:02.115387+0000","last_active":"2024-04-24T09:04:24.531764+0000","last_peered":"2024-04-24T09:04:24.531764+0000","last_clean":"2024-04-24T09:04:24.531764+0000","last_became_active":"2024-04-24T09:04:02.115086+0000","last_became_peered":"2024-04-24T09:04:02.115086+0000","last_unstale":"2024-04-24T09:04:24.531764+0000","last_undegraded":"2024-04-24T09:04:24.531764+0000","last_fullsized":"2024-04-24T09:04:24.531764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:40:57.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532516+0000","last_change":"2024-04-24T09:04:06.146189+0000","last_active":"2024-04-24T09:04:24.532516+0000","last_peered":"2024-04-24T09:04:24.532516+0000","last_clean":"2024-04-24T09:04:24.532516+0000","last_became_active":"2024-04-24T09:04:06.145556+0000","last_became_peered":"2024-04-24T09:04:06.145556+0000","last_unstale":"2024-04-24T09:04:24.532516+0000","last_undegraded":"2024-04-24T09:04:24.532516+0000","last_fullsized":"2024-04-24T09:04:24.532516+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:41.371948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.628679+0000","last_change":"2024-04-24T09:04:01.542185+0000","last_active":"2024-04-24T09:04:33.628679+0000","last_peered":"2024-04-24T09:04:33.628679+0000","last_clean":"2024-04-24T09:04:33.628679+0000","last_became_active":"2024-04-24T09:04:01.541240+0000","last_became_peered":"2024-04-24T09:04:01.541240+0000","last_unstale":"2024-04-24T09:04:33.628679+0000","last_undegraded":"2024-04-24T09:04:33.628679+0000","last_fullsized":"2024-04-24T09:04:33.628679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:15.103276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.525028+0000","last_change":"2024-04-24T09:04:02.104054+0000","last_active":"2024-04-24T09:04:24.525028+0000","last_peered":"2024-04-24T09:04:24.525028+0000","last_clean":"2024-04-24T09:04:24.525028+0000","last_became_active":"2024-04-24T09:04:02.103673+0000","last_became_peered":"2024-04-24T09:04:02.103673+0000","last_unstale":"2024-04-24T09:04:24.525028+0000","last_undegraded":"2024-04-24T09:04:24.525028+0000","last_fullsized":"2024-04-24T09:04:24.525028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:31:58.889989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696675+0000","last_change":"2024-04-24T09:04:06.134034+0000","last_active":"2024-04-24T09:04:24.696675+0000","last_peered":"2024-04-24T09:04:24.696675+0000","last_clean":"2024-04-24T09:04:24.696675+0000","last_became_active":"2024-04-24T09:04:06.133596+0000","last_became_peered":"2024-04-24T09:04:06.133596+0000","last_unstale":"2024-04-24T09:04:24.696675+0000","last_undegraded":"2024-04-24T09:04:24.696675+0000","last_fullsized":"2024-04-24T09:04:24.696675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:17:09.572974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.528733+0000","last_change":"2024-04-24T09:04:01.543312+0000","last_active":"2024-04-24T09:04:24.528733+0000","last_peered":"2024-04-24T09:04:24.528733+0000","last_clean":"2024-04-24T09:04:24.528733+0000","last_became_active":"2024-04-24T09:04:01.542914+0000","last_became_peered":"2024-04-24T09:04:01.542914+0000","last_unstale":"2024-04-24T09:04:24.528733+0000","last_undegraded":"2024-04-24T09:04:24.528733+0000","last_fullsized":"2024-04-24T09:04:24.528733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:50.695108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.517056+0000","last_change":"2024-04-24T09:04:02.114445+0000","last_active":"2024-04-24T09:04:24.517056+0000","last_peered":"2024-04-24T09:04:24.517056+0000","last_clean":"2024-04-24T09:04:24.517056+0000","last_became_active":"2024-04-24T09:04:02.114075+0000","last_became_peered":"2024-04-24T09:04:02.114075+0000","last_unstale":"2024-04-24T09:04:24.517056+0000","last_undegraded":"2024-04-24T09:04:24.517056+0000","last_fullsized":"2024-04-24T09:04:24.517056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:35:06.933677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.525318+0000","last_change":"2024-04-24T09:04:06.142368+0000","last_active":"2024-04-24T09:04:24.525318+0000","last_peered":"2024-04-24T09:04:24.525318+0000","last_clean":"2024-04-24T09:04:24.525318+0000","last_became_active":"2024-04-24T09:04:06.141872+0000","last_became_peered":"2024-04-24T09:04:06.141872+0000","last_unstale":"2024-04-24T09:04:24.525318+0000","last_undegraded":"2024-04-24T09:04:24.525318+0000","last_fullsized":"2024-04-24T09:04:24.525318+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:13:43.733662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.612462+0000","last_change":"2024-04-24T09:04:01.556002+0000","last_active":"2024-04-24T09:04:32.612462+0000","last_peered":"2024-04-24T09:04:32.612462+0000","last_clean":"2024-04-24T09:04:32.612462+0000","last_became_active":"2024-04-24T09:04:01.555643+0000","last_became_peered":"2024-04-24T09:04:01.555643+0000","last_unstale":"2024-04-24T09:04:32.612462+0000","last_undegraded":"2024-04-24T09:04:32.612462+0000","last_fullsized":"2024-04-24T09:04:32.612462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:30:31.171184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697765+0000","last_change":"2024-04-24T09:04:02.104252+0000","last_active":"2024-04-24T09:04:24.697765+0000","last_peered":"2024-04-24T09:04:24.697765+0000","last_clean":"2024-04-24T09:04:24.697765+0000","last_became_active":"2024-04-24T09:04:02.099264+0000","last_became_peered":"2024-04-24T09:04:02.099264+0000","last_unstale":"2024-04-24T09:04:24.697765+0000","last_undegraded":"2024-04-24T09:04:24.697765+0000","last_fullsized":"2024-04-24T09:04:24.697765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:28.469814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.516701+0000","last_change":"2024-04-24T09:04:06.132407+0000","last_active":"2024-04-24T09:04:24.516701+0000","last_peered":"2024-04-24T09:04:24.516701+0000","last_clean":"2024-04-24T09:04:24.516701+0000","last_became_active":"2024-04-24T09:04:06.131997+0000","last_became_peered":"2024-04-24T09:04:06.131997+0000","last_unstale":"2024-04-24T09:04:24.516701+0000","last_undegraded":"2024-04-24T09:04:24.516701+0000","last_fullsized":"2024-04-24T09:04:24.516701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:08:07.369326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.613853+0000","last_change":"2024-04-24T09:04:01.561039+0000","last_active":"2024-04-24T09:04:32.613853+0000","last_peered":"2024-04-24T09:04:32.613853+0000","last_clean":"2024-04-24T09:04:32.613853+0000","last_became_active":"2024-04-24T09:04:01.560543+0000","last_became_peered":"2024-04-24T09:04:01.560543+0000","last_unstale":"2024-04-24T09:04:32.613853+0000","last_undegraded":"2024-04-24T09:04:32.613853+0000","last_fullsized":"2024-04-24T09:04:32.613853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:27.706982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697177+0000","last_change":"2024-04-24T09:04:02.110850+0000","last_active":"2024-04-24T09:04:24.697177+0000","last_peered":"2024-04-24T09:04:24.697177+0000","last_clean":"2024-04-24T09:04:24.697177+0000","last_became_active":"2024-04-24T09:04:02.110512+0000","last_became_peered":"2024-04-24T09:04:02.110512+0000","last_unstale":"2024-04-24T09:04:24.697177+0000","last_undegraded":"2024-04-24T09:04:24.697177+0000","last_fullsized":"2024-04-24T09:04:24.697177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:19:16.937202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.521039+0000","last_change":"2024-04-24T09:04:06.135200+0000","last_active":"2024-04-24T09:04:24.521039+0000","last_peered":"2024-04-24T09:04:24.521039+0000","last_clean":"2024-04-24T09:04:24.521039+0000","last_became_active":"2024-04-24T09:04:06.134680+0000","last_became_peered":"2024-04-24T09:04:06.134680+0000","last_unstale":"2024-04-24T09:04:24.521039+0000","last_undegraded":"2024-04-24T09:04:24.521039+0000","last_fullsized":"2024-04-24T09:04:24.521039+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:48:57.697740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.612218+0000","last_change":"2024-04-24T09:04:01.555069+0000","last_active":"2024-04-24T09:04:32.612218+0000","last_peered":"2024-04-24T09:04:32.612218+0000","last_clean":"2024-04-24T09:04:32.612218+0000","last_became_active":"2024-04-24T09:04:01.554737+0000","last_became_peered":"2024-04-24T09:04:01.554737+0000","last_unstale":"2024-04-24T09:04:32.612218+0000","last_undegraded":"2024-04-24T09:04:32.612218+0000","last_fullsized":"2024-04-24T09:04:32.612218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:23:00.272697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697205+0000","last_change":"2024-04-24T09:04:02.112126+0000","last_active":"2024-04-24T09:04:24.697205+0000","last_peered":"2024-04-24T09:04:24.697205+0000","last_clean":"2024-04-24T09:04:24.697205+0000","last_became_active":"2024-04-24T09:04:02.107995+0000","last_became_peered":"2024-04-24T09:04:02.107995+0000","last_unstale":"2024-04-24T09:04:24.697205+0000","last_undegraded":"2024-04-24T09:04:24.697205+0000","last_fullsized":"2024-04-24T09:04:24.697205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:13.189829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.542302+0000","last_change":"2024-04-24T09:04:24.542302+0000","last_active":"2024-04-24T09:04:24.542302+0000","last_peered":"2024-04-24T09:04:24.542302+0000","last_clean":"2024-04-24T09:04:24.542302+0000","last_became_active":"2024-04-24T09:04:24.539848+0000","last_became_peered":"2024-04-24T09:04:24.539848+0000","last_unstale":"2024-04-24T09:04:24.542302+0000","last_undegraded":"2024-04-24T09:04:24.542302+0000","last_fullsized":"2024-04-24T09:04:24.542302+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:01:52.055510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"92'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.613748+0000","last_change":"2024-04-24T09:04:01.548610+0000","last_active":"2024-04-24T09:04:32.613748+0000","last_peered":"2024-04-24T09:04:32.613748+0000","last_clean":"2024-04-24T09:04:32.613748+0000","last_became_active":"2024-04-24T09:04:01.548145+0000","last_became_peered":"2024-04-24T09:04:01.548145+0000","last_unstale":"2024-04-24T09:04:32.613748+0000","last_undegraded":"2024-04-24T09:04:32.613748+0000","last_fullsized":"2024-04-24T09:04:32.613748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:58:48.425187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.516908+0000","last_change":"2024-04-24T09:04:02.109965+0000","last_active":"2024-04-24T09:04:24.516908+0000","last_peered":"2024-04-24T09:04:24.516908+0000","last_clean":"2024-04-24T09:04:24.516908+0000","last_became_active":"2024-04-24T09:04:02.108143+0000","last_became_peered":"2024-04-24T09:04:02.108143+0000","last_unstale":"2024-04-24T09:04:24.516908+0000","last_undegraded":"2024-04-24T09:04:24.516908+0000","last_fullsized":"2024-04-24T09:04:24.516908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:01:21.083039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.534893+0000","last_change":"2024-04-24T09:04:06.145369+0000","last_active":"2024-04-24T09:04:24.534893+0000","last_peered":"2024-04-24T09:04:24.534893+0000","last_clean":"2024-04-24T09:04:24.534893+0000","last_became_active":"2024-04-24T09:04:06.144884+0000","last_became_peered":"2024-04-24T09:04:06.144884+0000","last_unstale":"2024-04-24T09:04:24.534893+0000","last_undegraded":"2024-04-24T09:04:24.534893+0000","last_fullsized":"2024-04-24T09:04:24.534893+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:29:22.780846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.614895+0000","last_change":"2024-04-24T09:04:01.548195+0000","last_active":"2024-04-24T09:04:32.614895+0000","last_peered":"2024-04-24T09:04:32.614895+0000","last_clean":"2024-04-24T09:04:32.614895+0000","last_became_active":"2024-04-24T09:04:01.544969+0000","last_became_peered":"2024-04-24T09:04:01.544969+0000","last_unstale":"2024-04-24T09:04:32.614895+0000","last_undegraded":"2024-04-24T09:04:32.614895+0000","last_fullsized":"2024-04-24T09:04:32.614895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:32:55.294629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.529490+0000","last_change":"2024-04-24T09:04:02.103785+0000","last_active":"2024-04-24T09:04:24.529490+0000","last_peered":"2024-04-24T09:04:24.529490+0000","last_clean":"2024-04-24T09:04:24.529490+0000","last_became_active":"2024-04-24T09:04:02.103463+0000","last_became_peered":"2024-04-24T09:04:02.103463+0000","last_unstale":"2024-04-24T09:04:24.529490+0000","last_undegraded":"2024-04-24T09:04:24.529490+0000","last_fullsized":"2024-04-24T09:04:24.529490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:56.385382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527499+0000","last_change":"2024-04-24T09:04:06.149955+0000","last_active":"2024-04-24T09:04:24.527499+0000","last_peered":"2024-04-24T09:04:24.527499+0000","last_clean":"2024-04-24T09:04:24.527499+0000","last_became_active":"2024-04-24T09:04:06.149445+0000","last_became_peered":"2024-04-24T09:04:06.149445+0000","last_unstale":"2024-04-24T09:04:24.527499+0000","last_undegraded":"2024-04-24T09:04:24.527499+0000","last_fullsized":"2024-04-24T09:04:24.527499+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:30:42.527007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.615210+0000","last_change":"2024-04-24T09:04:01.548740+0000","last_active":"2024-04-24T09:04:32.615210+0000","last_peered":"2024-04-24T09:04:32.615210+0000","last_clean":"2024-04-24T09:04:32.615210+0000","last_became_active":"2024-04-24T09:04:01.546060+0000","last_became_peered":"2024-04-24T09:04:01.546060+0000","last_unstale":"2024-04-24T09:04:32.615210+0000","last_undegraded":"2024-04-24T09:04:32.615210+0000","last_fullsized":"2024-04-24T09:04:32.615210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:09:16.301150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.538871+0000","last_change":"2024-04-24T09:04:24.538871+0000","last_active":"2024-04-24T09:04:24.538871+0000","last_peered":"2024-04-24T09:04:24.538871+0000","last_clean":"2024-04-24T09:04:24.538871+0000","last_became_active":"2024-04-24T09:04:24.537039+0000","last_became_peered":"2024-04-24T09:04:24.537039+0000","last_unstale":"2024-04-24T09:04:24.538871+0000","last_undegraded":"2024-04-24T09:04:24.538871+0000","last_fullsized":"2024-04-24T09:04:24.538871+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:18.520407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.518409+0000","last_change":"2024-04-24T09:04:06.136973+0000","last_active":"2024-04-24T09:04:24.518409+0000","last_peered":"2024-04-24T09:04:24.518409+0000","last_clean":"2024-04-24T09:04:24.518409+0000","last_became_active":"2024-04-24T09:04:06.136405+0000","last_became_peered":"2024-04-24T09:04:06.136405+0000","last_unstale":"2024-04-24T09:04:24.518409+0000","last_undegraded":"2024-04-24T09:04:24.518409+0000","last_fullsized":"2024-04-24T09:04:24.518409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:07.653162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'4","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:35.638952+0000","last_change":"2024-04-24T09:04:01.546853+0000","last_active":"2024-04-24T09:04:35.638952+0000","last_peered":"2024-04-24T09:04:35.638952+0000","last_clean":"2024-04-24T09:04:35.638952+0000","last_became_active":"2024-04-24T09:04:01.546441+0000","last_became_peered":"2024-04-24T09:04:01.546441+0000","last_unstale":"2024-04-24T09:04:35.638952+0000","last_undegraded":"2024-04-24T09:04:35.638952+0000","last_fullsized":"2024-04-24T09:04:35.638952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:23:56.333410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533833+0000","last_change":"2024-04-24T09:04:02.101158+0000","last_active":"2024-04-24T09:04:24.533833+0000","last_peered":"2024-04-24T09:04:24.533833+0000","last_clean":"2024-04-24T09:04:24.533833+0000","last_became_active":"2024-04-24T09:04:02.100832+0000","last_became_peered":"2024-04-24T09:04:02.100832+0000","last_unstale":"2024-04-24T09:04:24.533833+0000","last_undegraded":"2024-04-24T09:04:24.533833+0000","last_fullsized":"2024-04-24T09:04:24.533833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:05:34.080520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533094+0000","last_change":"2024-04-24T09:04:06.149176+0000","last_active":"2024-04-24T09:04:24.533094+0000","last_peered":"2024-04-24T09:04:24.533094+0000","last_clean":"2024-04-24T09:04:24.533094+0000","last_became_active":"2024-04-24T09:04:06.145539+0000","last_became_peered":"2024-04-24T09:04:06.145539+0000","last_unstale":"2024-04-24T09:04:24.533094+0000","last_undegraded":"2024-04-24T09:04:24.533094+0000","last_fullsized":"2024-04-24T09:04:24.533094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:38.719787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.632543+0000","last_change":"2024-04-24T09:04:01.555330+0000","last_active":"2024-04-24T09:04:33.632543+0000","last_peered":"2024-04-24T09:04:33.632543+0000","last_clean":"2024-04-24T09:04:33.632543+0000","last_became_active":"2024-04-24T09:04:01.552618+0000","last_became_peered":"2024-04-24T09:04:01.552618+0000","last_unstale":"2024-04-24T09:04:33.632543+0000","last_undegraded":"2024-04-24T09:04:33.632543+0000","last_fullsized":"2024-04-24T09:04:33.632543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:30.956657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.529355+0000","last_change":"2024-04-24T09:04:02.116301+0000","last_active":"2024-04-24T09:04:24.529355+0000","last_peered":"2024-04-24T09:04:24.529355+0000","last_clean":"2024-04-24T09:04:24.529355+0000","last_became_active":"2024-04-24T09:04:02.116001+0000","last_became_peered":"2024-04-24T09:04:02.116001+0000","last_unstale":"2024-04-24T09:04:24.529355+0000","last_undegraded":"2024-04-24T09:04:24.529355+0000","last_fullsized":"2024-04-24T09:04:24.529355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:16:18.995067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.529071+0000","last_change":"2024-04-24T09:04:06.135814+0000","last_active":"2024-04-24T09:04:24.529071+0000","last_peered":"2024-04-24T09:04:24.529071+0000","last_clean":"2024-04-24T09:04:24.529071+0000","last_became_active":"2024-04-24T09:04:06.135054+0000","last_became_peered":"2024-04-24T09:04:06.135054+0000","last_unstale":"2024-04-24T09:04:24.529071+0000","last_undegraded":"2024-04-24T09:04:24.529071+0000","last_fullsized":"2024-04-24T09:04:24.529071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:51:31.150589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:34.618324+0000","last_change":"2024-04-24T09:04:01.553144+0000","last_active":"2024-04-24T09:04:34.618324+0000","last_peered":"2024-04-24T09:04:34.618324+0000","last_clean":"2024-04-24T09:04:34.618324+0000","last_became_active":"2024-04-24T09:04:01.550599+0000","last_became_peered":"2024-04-24T09:04:01.550599+0000","last_unstale":"2024-04-24T09:04:34.618324+0000","last_undegraded":"2024-04-24T09:04:34.618324+0000","last_fullsized":"2024-04-24T09:04:34.618324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:41.950209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532070+0000","last_change":"2024-04-24T09:04:02.117064+0000","last_active":"2024-04-24T09:04:24.532070+0000","last_peered":"2024-04-24T09:04:24.532070+0000","last_clean":"2024-04-24T09:04:24.532070+0000","last_became_active":"2024-04-24T09:04:02.114528+0000","last_became_peered":"2024-04-24T09:04:02.114528+0000","last_unstale":"2024-04-24T09:04:24.532070+0000","last_undegraded":"2024-04-24T09:04:24.532070+0000","last_fullsized":"2024-04-24T09:04:24.532070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:24:53.662571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.701058+0000","last_change":"2024-04-24T09:04:24.701058+0000","last_active":"2024-04-24T09:04:24.701058+0000","last_peered":"2024-04-24T09:04:24.701058+0000","last_clean":"2024-04-24T09:04:24.701058+0000","last_became_active":"2024-04-24T09:04:24.699766+0000","last_became_peered":"2024-04-24T09:04:24.699766+0000","last_unstale":"2024-04-24T09:04:24.701058+0000","last_undegraded":"2024-04-24T09:04:24.701058+0000","last_fullsized":"2024-04-24T09:04:24.701058+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:57:48.491073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.701816+0000","last_change":"2024-04-24T09:04:24.701816+0000","last_active":"2024-04-24T09:04:24.701816+0000","last_peered":"2024-04-24T09:04:24.701816+0000","last_clean":"2024-04-24T09:04:24.701816+0000","last_became_active":"2024-04-24T09:04:24.700530+0000","last_became_peered":"2024-04-24T09:04:24.700530+0000","last_unstale":"2024-04-24T09:04:24.701816+0000","last_undegraded":"2024-04-24T09:04:24.701816+0000","last_fullsized":"2024-04-24T09:04:24.701816+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:24.614043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527276+0000","last_change":"2024-04-24T09:04:02.104602+0000","last_active":"2024-04-24T09:04:24.527276+0000","last_peered":"2024-04-24T09:04:24.527276+0000","last_clean":"2024-04-24T09:04:24.527276+0000","last_became_active":"2024-04-24T09:04:02.104188+0000","last_became_peered":"2024-04-24T09:04:02.104188+0000","last_unstale":"2024-04-24T09:04:24.527276+0000","last_undegraded":"2024-04-24T09:04:24.527276+0000","last_fullsized":"2024-04-24T09:04:24.527276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:00:50.286952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531886+0000","last_change":"2024-04-24T09:04:06.134971+0000","last_active":"2024-04-24T09:04:24.531886+0000","last_peered":"2024-04-24T09:04:24.531886+0000","last_clean":"2024-04-24T09:04:24.531886+0000","last_became_active":"2024-04-24T09:04:06.134551+0000","last_became_peered":"2024-04-24T09:04:06.134551+0000","last_unstale":"2024-04-24T09:04:24.531886+0000","last_undegraded":"2024-04-24T09:04:24.531886+0000","last_fullsized":"2024-04-24T09:04:24.531886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:40:41.766657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532337+0000","last_change":"2024-04-24T09:04:01.551117+0000","last_active":"2024-04-24T09:04:24.532337+0000","last_peered":"2024-04-24T09:04:24.532337+0000","last_clean":"2024-04-24T09:04:24.532337+0000","last_became_active":"2024-04-24T09:04:01.550646+0000","last_became_peered":"2024-04-24T09:04:01.550646+0000","last_unstale":"2024-04-24T09:04:24.532337+0000","last_undegraded":"2024-04-24T09:04:24.532337+0000","last_fullsized":"2024-04-24T09:04:24.532337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:29:03.821300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.526140+0000","last_change":"2024-04-24T09:04:02.107111+0000","last_active":"2024-04-24T09:04:24.526140+0000","last_peered":"2024-04-24T09:04:24.526140+0000","last_clean":"2024-04-24T09:04:24.526140+0000","last_became_active":"2024-04-24T09:04:02.106185+0000","last_became_peered":"2024-04-24T09:04:02.106185+0000","last_unstale":"2024-04-24T09:04:24.526140+0000","last_undegraded":"2024-04-24T09:04:24.526140+0000","last_fullsized":"2024-04-24T09:04:24.526140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:19:03.111407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532094+0000","last_change":"2024-04-24T09:04:06.143984+0000","last_active":"2024-04-24T09:04:24.532094+0000","last_peered":"2024-04-24T09:04:24.532094+0000","last_clean":"2024-04-24T09:04:24.532094+0000","last_became_active":"2024-04-24T09:04:06.143503+0000","last_became_peered":"2024-04-24T09:04:06.143503+0000","last_unstale":"2024-04-24T09:04:24.532094+0000","last_undegraded":"2024-04-24T09:04:24.532094+0000","last_fullsized":"2024-04-24T09:04:24.532094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:35:19.317771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.529007+0000","last_change":"2024-04-24T09:04:02.111560+0000","last_active":"2024-04-24T09:04:24.529007+0000","last_peered":"2024-04-24T09:04:24.529007+0000","last_clean":"2024-04-24T09:04:24.529007+0000","last_became_active":"2024-04-24T09:04:02.109885+0000","last_became_peered":"2024-04-24T09:04:02.109885+0000","last_unstale":"2024-04-24T09:04:24.529007+0000","last_undegraded":"2024-04-24T09:04:24.529007+0000","last_fullsized":"2024-04-24T09:04:24.529007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:35.837621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:34.633044+0000","last_change":"2024-04-24T09:04:01.542530+0000","last_active":"2024-04-24T09:04:34.633044+0000","last_peered":"2024-04-24T09:04:34.633044+0000","last_clean":"2024-04-24T09:04:34.633044+0000","last_became_active":"2024-04-24T09:04:01.541702+0000","last_became_peered":"2024-04-24T09:04:01.541702+0000","last_unstale":"2024-04-24T09:04:34.633044+0000","last_undegraded":"2024-04-24T09:04:34.633044+0000","last_fullsized":"2024-04-24T09:04:34.633044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:28:29.420020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532535+0000","last_change":"2024-04-24T09:04:06.143004+0000","last_active":"2024-04-24T09:04:24.532535+0000","last_peered":"2024-04-24T09:04:24.532535+0000","last_clean":"2024-04-24T09:04:24.532535+0000","last_became_active":"2024-04-24T09:04:06.142444+0000","last_became_peered":"2024-04-24T09:04:06.142444+0000","last_unstale":"2024-04-24T09:04:24.532535+0000","last_undegraded":"2024-04-24T09:04:24.532535+0000","last_fullsized":"2024-04-24T09:04:24.532535+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:13.910612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.704292+0000","last_change":"2024-04-24T09:04:24.701330+0000","last_active":"2024-04-24T09:04:24.704292+0000","last_peered":"2024-04-24T09:04:24.704292+0000","last_clean":"2024-04-24T09:04:24.704292+0000","last_became_active":"2024-04-24T09:04:24.699988+0000","last_became_peered":"2024-04-24T09:04:24.699988+0000","last_unstale":"2024-04-24T09:04:24.704292+0000","last_undegraded":"2024-04-24T09:04:24.704292+0000","last_fullsized":"2024-04-24T09:04:24.704292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:23:24.610671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:34.629794+0000","last_change":"2024-04-24T09:04:01.541206+0000","last_active":"2024-04-24T09:04:34.629794+0000","last_peered":"2024-04-24T09:04:34.629794+0000","last_clean":"2024-04-24T09:04:34.629794+0000","last_became_active":"2024-04-24T09:04:01.539888+0000","last_became_peered":"2024-04-24T09:04:01.539888+0000","last_unstale":"2024-04-24T09:04:34.629794+0000","last_undegraded":"2024-04-24T09:04:34.629794+0000","last_fullsized":"2024-04-24T09:04:34.629794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:01.550209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531165+0000","last_change":"2024-04-24T09:02:22.629840+0000","last_active":"2024-04-24T09:04:24.531165+0000","last_peered":"2024-04-24T09:04:24.531165+0000","last_clean":"2024-04-24T09:04:24.531165+0000","last_became_active":"2024-04-24T09:02:22.628700+0000","last_became_peered":"2024-04-24T09:02:22.628700+0000","last_unstale":"2024-04-24T09:04:24.531165+0000","last_undegraded":"2024-04-24T09:04:24.531165+0000","last_fullsized":"2024-04-24T09:04:24.531165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:57:38.349295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:57:38.349295+0000","last_clean_scrub_stamp":"2024-04-24T08:57:38.349295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:54:11.409492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533333+0000","last_change":"2024-04-24T09:04:06.149548+0000","last_active":"2024-04-24T09:04:24.533333+0000","last_peered":"2024-04-24T09:04:24.533333+0000","last_clean":"2024-04-24T09:04:24.533333+0000","last_became_active":"2024-04-24T09:04:06.149214+0000","last_became_peered":"2024-04-24T09:04:06.149214+0000","last_unstale":"2024-04-24T09:04:24.533333+0000","last_undegraded":"2024-04-24T09:04:24.533333+0000","last_fullsized":"2024-04-24T09:04:24.533333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:03.801242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.528440+0000","last_change":"2024-04-24T09:04:02.110242+0000","last_active":"2024-04-24T09:04:24.528440+0000","last_peered":"2024-04-24T09:04:24.528440+0000","last_clean":"2024-04-24T09:04:24.528440+0000","last_became_active":"2024-04-24T09:04:02.109879+0000","last_became_peered":"2024-04-24T09:04:02.109879+0000","last_unstale":"2024-04-24T09:04:24.528440+0000","last_undegraded":"2024-04-24T09:04:24.528440+0000","last_fullsized":"2024-04-24T09:04:24.528440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:16.353861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.628491+0000","last_change":"2024-04-24T09:04:01.550989+0000","last_active":"2024-04-24T09:04:33.628491+0000","last_peered":"2024-04-24T09:04:33.628491+0000","last_clean":"2024-04-24T09:04:33.628491+0000","last_became_active":"2024-04-24T09:04:01.550664+0000","last_became_peered":"2024-04-24T09:04:01.550664+0000","last_unstale":"2024-04-24T09:04:33.628491+0000","last_undegraded":"2024-04-24T09:04:33.628491+0000","last_fullsized":"2024-04-24T09:04:33.628491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:33:39.521951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.538207+0000","last_change":"2024-04-24T09:04:24.538207+0000","last_active":"2024-04-24T09:04:24.538207+0000","last_peered":"2024-04-24T09:04:24.538207+0000","last_clean":"2024-04-24T09:04:24.538207+0000","last_became_active":"2024-04-24T09:04:24.536894+0000","last_became_peered":"2024-04-24T09:04:24.536894+0000","last_unstale":"2024-04-24T09:04:24.538207+0000","last_undegraded":"2024-04-24T09:04:24.538207+0000","last_fullsized":"2024-04-24T09:04:24.538207+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:32:52.860371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.608599+0000","last_change":"2024-04-24T09:04:01.539415+0000","last_active":"2024-04-24T09:04:32.608599+0000","last_peered":"2024-04-24T09:04:32.608599+0000","last_clean":"2024-04-24T09:04:32.608599+0000","last_became_active":"2024-04-24T09:04:01.539020+0000","last_became_peered":"2024-04-24T09:04:01.539020+0000","last_unstale":"2024-04-24T09:04:32.608599+0000","last_undegraded":"2024-04-24T09:04:32.608599+0000","last_fullsized":"2024-04-24T09:04:32.608599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:12:48.728029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532453+0000","last_change":"2024-04-24T09:04:02.097560+0000","last_active":"2024-04-24T09:04:24.532453+0000","last_peered":"2024-04-24T09:04:24.532453+0000","last_clean":"2024-04-24T09:04:24.532453+0000","last_became_active":"2024-04-24T09:04:02.095612+0000","last_became_peered":"2024-04-24T09:04:02.095612+0000","last_unstale":"2024-04-24T09:04:24.532453+0000","last_undegraded":"2024-04-24T09:04:24.532453+0000","last_fullsized":"2024-04-24T09:04:24.532453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:55:47.747294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.524769+0000","last_change":"2024-04-24T09:04:06.137822+0000","last_active":"2024-04-24T09:04:24.524769+0000","last_peered":"2024-04-24T09:04:24.524769+0000","last_clean":"2024-04-24T09:04:24.524769+0000","last_became_active":"2024-04-24T09:04:06.137357+0000","last_became_peered":"2024-04-24T09:04:06.137357+0000","last_unstale":"2024-04-24T09:04:24.524769+0000","last_undegraded":"2024-04-24T09:04:24.524769+0000","last_fullsized":"2024-04-24T09:04:24.524769+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:55:13.119557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.544395+0000","last_change":"2024-04-24T09:04:24.544395+0000","last_active":"2024-04-24T09:04:24.544395+0000","last_peered":"2024-04-24T09:04:24.544395+0000","last_clean":"2024-04-24T09:04:24.544395+0000","last_became_active":"2024-04-24T09:04:24.535870+0000","last_became_peered":"2024-04-24T09:04:24.535870+0000","last_unstale":"2024-04-24T09:04:24.544395+0000","last_undegraded":"2024-04-24T09:04:24.544395+0000","last_fullsized":"2024-04-24T09:04:24.544395+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:29:21.271387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527426+0000","last_change":"2024-04-24T09:04:02.106107+0000","last_active":"2024-04-24T09:04:24.527426+0000","last_peered":"2024-04-24T09:04:24.527426+0000","last_clean":"2024-04-24T09:04:24.527426+0000","last_became_active":"2024-04-24T09:04:02.105734+0000","last_became_peered":"2024-04-24T09:04:02.105734+0000","last_unstale":"2024-04-24T09:04:24.527426+0000","last_undegraded":"2024-04-24T09:04:24.527426+0000","last_fullsized":"2024-04-24T09:04:24.527426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:15.372520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.695729+0000","last_change":"2024-04-24T09:04:06.139040+0000","last_active":"2024-04-24T09:04:24.695729+0000","last_peered":"2024-04-24T09:04:24.695729+0000","last_clean":"2024-04-24T09:04:24.695729+0000","last_became_active":"2024-04-24T09:04:06.138532+0000","last_became_peered":"2024-04-24T09:04:06.138532+0000","last_unstale":"2024-04-24T09:04:24.695729+0000","last_undegraded":"2024-04-24T09:04:24.695729+0000","last_fullsized":"2024-04-24T09:04:24.695729+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:28.686052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.612766+0000","last_change":"2024-04-24T09:04:01.546859+0000","last_active":"2024-04-24T09:04:32.612766+0000","last_peered":"2024-04-24T09:04:32.612766+0000","last_clean":"2024-04-24T09:04:32.612766+0000","last_became_active":"2024-04-24T09:04:01.546463+0000","last_became_peered":"2024-04-24T09:04:01.546463+0000","last_unstale":"2024-04-24T09:04:32.612766+0000","last_undegraded":"2024-04-24T09:04:32.612766+0000","last_fullsized":"2024-04-24T09:04:32.612766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:31:12.985687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697008+0000","last_change":"2024-04-24T09:04:02.112487+0000","last_active":"2024-04-24T09:04:24.697008+0000","last_peered":"2024-04-24T09:04:24.697008+0000","last_clean":"2024-04-24T09:04:24.697008+0000","last_became_active":"2024-04-24T09:04:02.112078+0000","last_became_peered":"2024-04-24T09:04:02.112078+0000","last_unstale":"2024-04-24T09:04:24.697008+0000","last_undegraded":"2024-04-24T09:04:24.697008+0000","last_fullsized":"2024-04-24T09:04:24.697008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:03:47.352110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.539120+0000","last_change":"2024-04-24T09:04:24.539120+0000","last_active":"2024-04-24T09:04:24.539120+0000","last_peered":"2024-04-24T09:04:24.539120+0000","last_clean":"2024-04-24T09:04:24.539120+0000","last_became_active":"2024-04-24T09:04:24.537048+0000","last_became_peered":"2024-04-24T09:04:24.537048+0000","last_unstale":"2024-04-24T09:04:24.539120+0000","last_undegraded":"2024-04-24T09:04:24.539120+0000","last_fullsized":"2024-04-24T09:04:24.539120+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:29:53.484131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.611836+0000","last_change":"2024-04-24T09:04:01.545444+0000","last_active":"2024-04-24T09:04:32.611836+0000","last_peered":"2024-04-24T09:04:32.611836+0000","last_clean":"2024-04-24T09:04:32.611836+0000","last_became_active":"2024-04-24T09:04:01.543816+0000","last_became_peered":"2024-04-24T09:04:01.543816+0000","last_unstale":"2024-04-24T09:04:32.611836+0000","last_undegraded":"2024-04-24T09:04:32.611836+0000","last_fullsized":"2024-04-24T09:04:32.611836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:02:36.221797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.516038+0000","last_change":"2024-04-24T09:04:02.109085+0000","last_active":"2024-04-24T09:04:24.516038+0000","last_peered":"2024-04-24T09:04:24.516038+0000","last_clean":"2024-04-24T09:04:24.516038+0000","last_became_active":"2024-04-24T09:04:02.108642+0000","last_became_peered":"2024-04-24T09:04:02.108642+0000","last_unstale":"2024-04-24T09:04:24.516038+0000","last_undegraded":"2024-04-24T09:04:24.516038+0000","last_fullsized":"2024-04-24T09:04:24.516038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:27:51.751369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.518972+0000","last_change":"2024-04-24T09:04:06.144940+0000","last_active":"2024-04-24T09:04:24.518972+0000","last_peered":"2024-04-24T09:04:24.518972+0000","last_clean":"2024-04-24T09:04:24.518972+0000","last_became_active":"2024-04-24T09:04:06.144553+0000","last_became_peered":"2024-04-24T09:04:06.144553+0000","last_unstale":"2024-04-24T09:04:24.518972+0000","last_undegraded":"2024-04-24T09:04:24.518972+0000","last_fullsized":"2024-04-24T09:04:24.518972+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:17.455644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.525606+0000","last_change":"2024-04-24T09:04:01.538939+0000","last_active":"2024-04-24T09:04:24.525606+0000","last_peered":"2024-04-24T09:04:24.525606+0000","last_clean":"2024-04-24T09:04:24.525606+0000","last_became_active":"2024-04-24T09:04:01.538541+0000","last_became_peered":"2024-04-24T09:04:01.538541+0000","last_unstale":"2024-04-24T09:04:24.525606+0000","last_undegraded":"2024-04-24T09:04:24.525606+0000","last_fullsized":"2024-04-24T09:04:24.525606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:08.185901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.530693+0000","last_change":"2024-04-24T09:04:02.103844+0000","last_active":"2024-04-24T09:04:24.530693+0000","last_peered":"2024-04-24T09:04:24.530693+0000","last_clean":"2024-04-24T09:04:24.530693+0000","last_became_active":"2024-04-24T09:04:02.103416+0000","last_became_peered":"2024-04-24T09:04:02.103416+0000","last_unstale":"2024-04-24T09:04:24.530693+0000","last_undegraded":"2024-04-24T09:04:24.530693+0000","last_fullsized":"2024-04-24T09:04:24.530693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:57:47.236384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.520023+0000","last_change":"2024-04-24T09:04:06.134838+0000","last_active":"2024-04-24T09:04:24.520023+0000","last_peered":"2024-04-24T09:04:24.520023+0000","last_clean":"2024-04-24T09:04:24.520023+0000","last_became_active":"2024-04-24T09:04:06.134343+0000","last_became_peered":"2024-04-24T09:04:06.134343+0000","last_unstale":"2024-04-24T09:04:24.520023+0000","last_undegraded":"2024-04-24T09:04:24.520023+0000","last_fullsized":"2024-04-24T09:04:24.520023+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:56:55.219012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.695997+0000","last_change":"2024-04-24T09:04:01.557194+0000","last_active":"2024-04-24T09:04:24.695997+0000","last_peered":"2024-04-24T09:04:24.695997+0000","last_clean":"2024-04-24T09:04:24.695997+0000","last_became_active":"2024-04-24T09:04:01.556441+0000","last_became_peered":"2024-04-24T09:04:01.556441+0000","last_unstale":"2024-04-24T09:04:24.695997+0000","last_undegraded":"2024-04-24T09:04:24.695997+0000","last_fullsized":"2024-04-24T09:04:24.695997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:45:50.399096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.526973+0000","last_change":"2024-04-24T09:04:02.108076+0000","last_active":"2024-04-24T09:04:24.526973+0000","last_peered":"2024-04-24T09:04:24.526973+0000","last_clean":"2024-04-24T09:04:24.526973+0000","last_became_active":"2024-04-24T09:04:02.106829+0000","last_became_peered":"2024-04-24T09:04:02.106829+0000","last_unstale":"2024-04-24T09:04:24.526973+0000","last_undegraded":"2024-04-24T09:04:24.526973+0000","last_fullsized":"2024-04-24T09:04:24.526973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:40:23.075828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.521106+0000","last_change":"2024-04-24T09:04:06.142269+0000","last_active":"2024-04-24T09:04:24.521106+0000","last_peered":"2024-04-24T09:04:24.521106+0000","last_clean":"2024-04-24T09:04:24.521106+0000","last_became_active":"2024-04-24T09:04:06.141725+0000","last_became_peered":"2024-04-24T09:04:06.141725+0000","last_unstale":"2024-04-24T09:04:24.521106+0000","last_undegraded":"2024-04-24T09:04:24.521106+0000","last_fullsized":"2024-04-24T09:04:24.521106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:28:17.934148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.615156+0000","last_change":"2024-04-24T09:04:01.553134+0000","last_active":"2024-04-24T09:04:32.615156+0000","last_peered":"2024-04-24T09:04:32.615156+0000","last_clean":"2024-04-24T09:04:32.615156+0000","last_became_active":"2024-04-24T09:04:01.552099+0000","last_became_peered":"2024-04-24T09:04:01.552099+0000","last_unstale":"2024-04-24T09:04:32.615156+0000","last_undegraded":"2024-04-24T09:04:32.615156+0000","last_fullsized":"2024-04-24T09:04:32.615156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:42:36.231405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532146+0000","last_change":"2024-04-24T09:04:02.116641+0000","last_active":"2024-04-24T09:04:24.532146+0000","last_peered":"2024-04-24T09:04:24.532146+0000","last_clean":"2024-04-24T09:04:24.532146+0000","last_became_active":"2024-04-24T09:04:02.115374+0000","last_became_peered":"2024-04-24T09:04:02.115374+0000","last_unstale":"2024-04-24T09:04:24.532146+0000","last_undegraded":"2024-04-24T09:04:24.532146+0000","last_fullsized":"2024-04-24T09:04:24.532146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:34.450990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696698+0000","last_change":"2024-04-24T09:04:06.149968+0000","last_active":"2024-04-24T09:04:24.696698+0000","last_peered":"2024-04-24T09:04:24.696698+0000","last_clean":"2024-04-24T09:04:24.696698+0000","last_became_active":"2024-04-24T09:04:06.149425+0000","last_became_peered":"2024-04-24T09:04:06.149425+0000","last_unstale":"2024-04-24T09:04:24.696698+0000","last_undegraded":"2024-04-24T09:04:24.696698+0000","last_fullsized":"2024-04-24T09:04:24.696698+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:23:53.869004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697126+0000","last_change":"2024-04-24T09:04:01.553822+0000","last_active":"2024-04-24T09:04:24.697126+0000","last_peered":"2024-04-24T09:04:24.697126+0000","last_clean":"2024-04-24T09:04:24.697126+0000","last_became_active":"2024-04-24T09:04:01.551468+0000","last_became_peered":"2024-04-24T09:04:01.551468+0000","last_unstale":"2024-04-24T09:04:24.697126+0000","last_undegraded":"2024-04-24T09:04:24.697126+0000","last_fullsized":"2024-04-24T09:04:24.697126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:56.367584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533471+0000","last_change":"2024-04-24T09:04:02.115472+0000","last_active":"2024-04-24T09:04:24.533471+0000","last_peered":"2024-04-24T09:04:24.533471+0000","last_clean":"2024-04-24T09:04:24.533471+0000","last_became_active":"2024-04-24T09:04:02.114666+0000","last_became_peered":"2024-04-24T09:04:02.114666+0000","last_unstale":"2024-04-24T09:04:24.533471+0000","last_undegraded":"2024-04-24T09:04:24.533471+0000","last_fullsized":"2024-04-24T09:04:24.533471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:28:39.145108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697174+0000","last_change":"2024-04-24T09:04:06.138786+0000","last_active":"2024-04-24T09:04:24.697174+0000","last_peered":"2024-04-24T09:04:24.697174+0000","last_clean":"2024-04-24T09:04:24.697174+0000","last_became_active":"2024-04-24T09:04:06.138366+0000","last_became_peered":"2024-04-24T09:04:06.138366+0000","last_unstale":"2024-04-24T09:04:24.697174+0000","last_undegraded":"2024-04-24T09:04:24.697174+0000","last_fullsized":"2024-04-24T09:04:24.697174+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:03:16.729928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533806+0000","last_change":"2024-04-24T09:04:01.547733+0000","last_active":"2024-04-24T09:04:24.533806+0000","last_peered":"2024-04-24T09:04:24.533806+0000","last_clean":"2024-04-24T09:04:24.533806+0000","last_became_active":"2024-04-24T09:04:01.544839+0000","last_became_peered":"2024-04-24T09:04:01.544839+0000","last_unstale":"2024-04-24T09:04:24.533806+0000","last_undegraded":"2024-04-24T09:04:24.533806+0000","last_fullsized":"2024-04-24T09:04:24.533806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:45:03.847092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531980+0000","last_change":"2024-04-24T09:04:02.108383+0000","last_active":"2024-04-24T09:04:24.531980+0000","last_peered":"2024-04-24T09:04:24.531980+0000","last_clean":"2024-04-24T09:04:24.531980+0000","last_became_active":"2024-04-24T09:04:02.107883+0000","last_became_peered":"2024-04-24T09:04:02.107883+0000","last_unstale":"2024-04-24T09:04:24.531980+0000","last_undegraded":"2024-04-24T09:04:24.531980+0000","last_fullsized":"2024-04-24T09:04:24.531980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:24:44.923708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527772+0000","last_change":"2024-04-24T09:04:06.140868+0000","last_active":"2024-04-24T09:04:24.527772+0000","last_peered":"2024-04-24T09:04:24.527772+0000","last_clean":"2024-04-24T09:04:24.527772+0000","last_became_active":"2024-04-24T09:04:06.139174+0000","last_became_peered":"2024-04-24T09:04:06.139174+0000","last_unstale":"2024-04-24T09:04:24.527772+0000","last_undegraded":"2024-04-24T09:04:24.527772+0000","last_fullsized":"2024-04-24T09:04:24.527772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:17:17.334276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533054+0000","last_change":"2024-04-24T09:04:01.534653+0000","last_active":"2024-04-24T09:04:24.533054+0000","last_peered":"2024-04-24T09:04:24.533054+0000","last_clean":"2024-04-24T09:04:24.533054+0000","last_became_active":"2024-04-24T09:04:01.534312+0000","last_became_peered":"2024-04-24T09:04:01.534312+0000","last_unstale":"2024-04-24T09:04:24.533054+0000","last_undegraded":"2024-04-24T09:04:24.533054+0000","last_fullsized":"2024-04-24T09:04:24.533054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:53:34.301007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533431+0000","last_change":"2024-04-24T09:04:02.102056+0000","last_active":"2024-04-24T09:04:24.533431+0000","last_peered":"2024-04-24T09:04:24.533431+0000","last_clean":"2024-04-24T09:04:24.533431+0000","last_became_active":"2024-04-24T09:04:02.101641+0000","last_became_peered":"2024-04-24T09:04:02.101641+0000","last_unstale":"2024-04-24T09:04:24.533431+0000","last_undegraded":"2024-04-24T09:04:24.533431+0000","last_fullsized":"2024-04-24T09:04:24.533431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:41.654024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.530927+0000","last_change":"2024-04-24T09:04:02.100327+0000","last_active":"2024-04-24T09:04:24.530927+0000","last_peered":"2024-04-24T09:04:24.530927+0000","last_clean":"2024-04-24T09:04:24.530927+0000","last_became_active":"2024-04-24T09:04:02.100028+0000","last_became_peered":"2024-04-24T09:04:02.100028+0000","last_unstale":"2024-04-24T09:04:24.530927+0000","last_undegraded":"2024-04-24T09:04:24.530927+0000","last_fullsized":"2024-04-24T09:04:24.530927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:15:18.728045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696351+0000","last_change":"2024-04-24T09:04:02.101996+0000","last_active":"2024-04-24T09:04:24.696351+0000","last_peered":"2024-04-24T09:04:24.696351+0000","last_clean":"2024-04-24T09:04:24.696351+0000","last_became_active":"2024-04-24T09:04:02.101581+0000","last_became_peered":"2024-04-24T09:04:02.101581+0000","last_unstale":"2024-04-24T09:04:24.696351+0000","last_undegraded":"2024-04-24T09:04:24.696351+0000","last_fullsized":"2024-04-24T09:04:24.696351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:55.487781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696405+0000","last_change":"2024-04-24T09:04:02.107288+0000","last_active":"2024-04-24T09:04:24.696405+0000","last_peered":"2024-04-24T09:04:24.696405+0000","last_clean":"2024-04-24T09:04:24.696405+0000","last_became_active":"2024-04-24T09:04:02.106946+0000","last_became_peered":"2024-04-24T09:04:02.106946+0000","last_unstale":"2024-04-24T09:04:24.696405+0000","last_undegraded":"2024-04-24T09:04:24.696405+0000","last_fullsized":"2024-04-24T09:04:24.696405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:38:56.821285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696457+0000","last_change":"2024-04-24T09:04:02.116829+0000","last_active":"2024-04-24T09:04:24.696457+0000","last_peered":"2024-04-24T09:04:24.696457+0000","last_clean":"2024-04-24T09:04:24.696457+0000","last_became_active":"2024-04-24T09:04:02.116520+0000","last_became_peered":"2024-04-24T09:04:02.116520+0000","last_unstale":"2024-04-24T09:04:24.696457+0000","last_undegraded":"2024-04-24T09:04:24.696457+0000","last_fullsized":"2024-04-24T09:04:24.696457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:26.873826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.700595+0000","last_change":"2024-04-24T09:04:24.700595+0000","last_active":"2024-04-24T09:04:24.700595+0000","last_peered":"2024-04-24T09:04:24.700595+0000","last_clean":"2024-04-24T09:04:24.700595+0000","last_became_active":"2024-04-24T09:04:24.699395+0000","last_became_peered":"2024-04-24T09:04:24.699395+0000","last_unstale":"2024-04-24T09:04:24.700595+0000","last_undegraded":"2024-04-24T09:04:24.700595+0000","last_fullsized":"2024-04-24T09:04:24.700595+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:28.756302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.529561+0000","last_change":"2024-04-24T09:04:02.114087+0000","last_active":"2024-04-24T09:04:24.529561+0000","last_peered":"2024-04-24T09:04:24.529561+0000","last_clean":"2024-04-24T09:04:24.529561+0000","last_became_active":"2024-04-24T09:04:02.112681+0000","last_became_peered":"2024-04-24T09:04:02.112681+0000","last_unstale":"2024-04-24T09:04:24.529561+0000","last_undegraded":"2024-04-24T09:04:24.529561+0000","last_fullsized":"2024-04-24T09:04:24.529561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:50:01.762598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697245+0000","last_change":"2024-04-24T09:04:02.102322+0000","last_active":"2024-04-24T09:04:24.697245+0000","last_peered":"2024-04-24T09:04:24.697245+0000","last_clean":"2024-04-24T09:04:24.697245+0000","last_became_active":"2024-04-24T09:04:02.101963+0000","last_became_peered":"2024-04-24T09:04:02.101963+0000","last_unstale":"2024-04-24T09:04:24.697245+0000","last_undegraded":"2024-04-24T09:04:24.697245+0000","last_fullsized":"2024-04-24T09:04:24.697245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:16.140829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527685+0000","last_change":"2024-04-24T09:04:02.110570+0000","last_active":"2024-04-24T09:04:24.527685+0000","last_peered":"2024-04-24T09:04:24.527685+0000","last_clean":"2024-04-24T09:04:24.527685+0000","last_became_active":"2024-04-24T09:04:02.110119+0000","last_became_peered":"2024-04-24T09:04:02.110119+0000","last_unstale":"2024-04-24T09:04:24.527685+0000","last_undegraded":"2024-04-24T09:04:24.527685+0000","last_fullsized":"2024-04-24T09:04:24.527685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:34.436430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.528620+0000","last_change":"2024-04-24T09:04:02.108292+0000","last_active":"2024-04-24T09:04:24.528620+0000","last_peered":"2024-04-24T09:04:24.528620+0000","last_clean":"2024-04-24T09:04:24.528620+0000","last_became_active":"2024-04-24T09:04:02.107481+0000","last_became_peered":"2024-04-24T09:04:02.107481+0000","last_unstale":"2024-04-24T09:04:24.528620+0000","last_undegraded":"2024-04-24T09:04:24.528620+0000","last_fullsized":"2024-04-24T09:04:24.528620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:49.755293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.525826+0000","last_change":"2024-04-24T09:04:02.102228+0000","last_active":"2024-04-24T09:04:24.525826+0000","last_peered":"2024-04-24T09:04:24.525826+0000","last_clean":"2024-04-24T09:04:24.525826+0000","last_became_active":"2024-04-24T09:04:02.101863+0000","last_became_peered":"2024-04-24T09:04:02.101863+0000","last_unstale":"2024-04-24T09:04:24.525826+0000","last_undegraded":"2024-04-24T09:04:24.525826+0000","last_fullsized":"2024-04-24T09:04:24.525826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:02:47.308116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.698108+0000","last_change":"2024-04-24T09:04:02.106749+0000","last_active":"2024-04-24T09:04:24.698108+0000","last_peered":"2024-04-24T09:04:24.698108+0000","last_clean":"2024-04-24T09:04:24.698108+0000","last_became_active":"2024-04-24T09:04:02.106282+0000","last_became_peered":"2024-04-24T09:04:02.106282+0000","last_unstale":"2024-04-24T09:04:24.698108+0000","last_undegraded":"2024-04-24T09:04:24.698108+0000","last_fullsized":"2024-04-24T09:04:24.698108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:35:59.765230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531528+0000","last_change":"2024-04-24T09:04:02.097825+0000","last_active":"2024-04-24T09:04:24.531528+0000","last_peered":"2024-04-24T09:04:24.531528+0000","last_clean":"2024-04-24T09:04:24.531528+0000","last_became_active":"2024-04-24T09:04:02.097488+0000","last_became_peered":"2024-04-24T09:04:02.097488+0000","last_unstale":"2024-04-24T09:04:24.531528+0000","last_undegraded":"2024-04-24T09:04:24.531528+0000","last_fullsized":"2024-04-24T09:04:24.531528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:05:59.962558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527850+0000","last_change":"2024-04-24T09:04:02.113819+0000","last_active":"2024-04-24T09:04:24.527850+0000","last_peered":"2024-04-24T09:04:24.527850+0000","last_clean":"2024-04-24T09:04:24.527850+0000","last_became_active":"2024-04-24T09:04:02.113463+0000","last_became_peered":"2024-04-24T09:04:02.113463+0000","last_unstale":"2024-04-24T09:04:24.527850+0000","last_undegraded":"2024-04-24T09:04:24.527850+0000","last_fullsized":"2024-04-24T09:04:24.527850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:47:05.109583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7171,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":22,"num_read_kb":18,"num_write":109,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":22230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350998,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":503390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612637,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32796,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712356,"statfs":{"total":95995035648,"available":95961452544,"internally_reserved":0,"allocated":1658880,"data_stored":1097974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808869,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":503592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":507642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32776,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712376,"statfs":{"total":95995035648,"available":95961473024,"internally_reserved":0,"allocated":1638400,"data_stored":1094968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":502898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":506650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":504286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":506684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":502912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":504465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32792,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712360,"statfs":{"total":95995035648,"available":95961456640,"internally_reserved":0,"allocated":1654784,"data_stored":1098748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T09:04:38.607 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T09:04:38.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.848+0000 7f9473e00640 1 -- 172.21.15.69:0/1942608132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474109670 msgr2=0x7f9474109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:38.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.848+0000 7f9473e00640 1 --2- 172.21.15.69:0/1942608132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474109670 0x7f9474109af0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f945c009900 tx=0x7f945c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.848+0000 7f9473e00640 1 -- 172.21.15.69:0/1942608132 shutdown_connections 2024-04-24T09:04:38.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.848+0000 7f9473e00640 1 --2- 172.21.15.69:0/1942608132 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f947410a030 0x7f94741108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.848+0000 7f9473e00640 1 --2- 172.21.15.69:0/1942608132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474109670 0x7f9474109af0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.848+0000 7f9473e00640 1 --2- 172.21.15.69:0/1942608132 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9474108480 0x7f94741088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.848+0000 7f9473e00640 1 -- 172.21.15.69:0/1942608132 >> 172.21.15.69:0/1942608132 conn(0x7f9474077440 msgr2=0x7f9474079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:38.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.848+0000 7f9473e00640 1 -- 172.21.15.69:0/1942608132 shutdown_connections 2024-04-24T09:04:38.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.848+0000 7f9473e00640 1 -- 172.21.15.69:0/1942608132 wait complete. 2024-04-24T09:04:38.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.849+0000 7f9473e00640 1 Processor -- start 2024-04-24T09:04:38.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.849+0000 7f9473e00640 1 -- start start 2024-04-24T09:04:38.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.850+0000 7f9473e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474108480 0x7f947419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:38.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.850+0000 7f9473e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9474109670 0x7f947419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:38.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.850+0000 7f9473e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f947410a030 0x7f94741a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:38.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.850+0000 7f9473e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9474113fc0 con 0x7f9474108480 2024-04-24T09:04:38.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.850+0000 7f9473e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9474113e40 con 0x7f947410a030 2024-04-24T09:04:38.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.850+0000 7f9473e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9474114140 con 0x7f9474109670 2024-04-24T09:04:38.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.850+0000 7f9472a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474108480 0x7f947419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:38.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.850+0000 7f9472a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474108480 0x7f947419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36560/0 (socket says 172.21.15.69:36560) 2024-04-24T09:04:38.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.850+0000 7f9472a00640 1 -- 172.21.15.69:0/406549540 learned_addr learned my addr 172.21.15.69:0/406549540 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:38.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.850+0000 7f946be00640 1 --2- 172.21.15.69:0/406549540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9474109670 0x7f947419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:38.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.850+0000 7f9473400640 1 --2- 172.21.15.69:0/406549540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f947410a030 0x7f94741a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:38.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.850+0000 7f9472a00640 1 -- 172.21.15.69:0/406549540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9474109670 msgr2=0x7f947419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:38.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.850+0000 7f9472a00640 1 --2- 172.21.15.69:0/406549540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9474109670 0x7f947419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.850+0000 7f9472a00640 1 -- 172.21.15.69:0/406549540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f947410a030 msgr2=0x7f94741a5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:38.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.850+0000 7f9472a00640 1 --2- 172.21.15.69:0/406549540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f947410a030 0x7f94741a5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:38.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.850+0000 7f9472a00640 1 -- 172.21.15.69:0/406549540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94741a6240 con 0x7f9474108480 2024-04-24T09:04:38.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.851+0000 7f9472a00640 1 --2- 172.21.15.69:0/406549540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474108480 0x7f947419e620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f946400c970 tx=0x7f946400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:38.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.851+0000 7f946b400640 1 -- 172.21.15.69:0/406549540 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9464007c10 con 0x7f9474108480 2024-04-24T09:04:38.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.851+0000 7f946b400640 1 -- 172.21.15.69:0/406549540 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9464007db0 con 0x7f9474108480 2024-04-24T09:04:38.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.851+0000 7f9473e00640 1 -- 172.21.15.69:0/406549540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94741a6530 con 0x7f9474108480 2024-04-24T09:04:38.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.851+0000 7f946b400640 1 -- 172.21.15.69:0/406549540 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946400f7d0 con 0x7f9474108480 2024-04-24T09:04:38.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.851+0000 7f9473e00640 1 -- 172.21.15.69:0/406549540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94741a6a80 con 0x7f9474108480 2024-04-24T09:04:38.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.853+0000 7f9473e00640 1 -- 172.21.15.69:0/406549540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f947410f820 con 0x7f9474108480 2024-04-24T09:04:38.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.855+0000 7f946b400640 1 -- 172.21.15.69:0/406549540 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9464018480 con 0x7f9474108480 2024-04-24T09:04:38.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.856+0000 7f946b400640 1 --2- 172.21.15.69:0/406549540 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f94580a3c40 0x7f94580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:38.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.856+0000 7f946b400640 1 -- 172.21.15.69:0/406549540 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f94640d0cf0 con 0x7f9474108480 2024-04-24T09:04:38.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.856+0000 7f946be00640 1 --2- 172.21.15.69:0/406549540 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f94580a3c40 0x7f94580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:38.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.857+0000 7f946be00640 1 --2- 172.21.15.69:0/406549540 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f94580a3c40 0x7f94580a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f945c02a910 tx=0x7f945c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:38.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:38.860+0000 7f946b400640 1 -- 172.21.15.69:0/406549540 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f946409d2a0 con 0x7f9474108480 2024-04-24T09:04:38.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:38 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1108815709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:04:38.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:38 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1481793712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:39.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:39.060+0000 7f9473e00640 1 -- 172.21.15.69:0/406549540 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9474062440 con 0x7f94580a3c40 2024-04-24T09:04:39.066 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:38 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1108815709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:04:39.066 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:38 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1481793712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:39.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:39.088+0000 7f946b400640 1 -- 172.21.15.69:0/406549540 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340426 (secure 0 0 0) 0x7f9474062440 con 0x7f94580a3c40 2024-04-24T09:04:39.090 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:39.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:38 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1108815709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:04:39.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:38 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1481793712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:39.095 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-24T09:04:39.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:39.096+0000 7f9473e00640 1 -- 172.21.15.69:0/406549540 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f94580a3c40 msgr2=0x7f94580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:39.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:39.096+0000 7f9473e00640 1 --2- 172.21.15.69:0/406549540 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f94580a3c40 0x7f94580a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f945c02a910 tx=0x7f945c000bc0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:39.096+0000 7f9473e00640 1 -- 172.21.15.69:0/406549540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474108480 msgr2=0x7f947419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:39.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:39.096+0000 7f9473e00640 1 --2- 172.21.15.69:0/406549540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474108480 0x7f947419e620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f946400c970 tx=0x7f946400ce30 comp rx=0 tx=0).stop 2024-04-24T09:04:39.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:39.096+0000 7f9473e00640 1 -- 172.21.15.69:0/406549540 shutdown_connections 2024-04-24T09:04:39.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:39.096+0000 7f9473e00640 1 --2- 172.21.15.69:0/406549540 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f94580a3c40 0x7f94580a6100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:39.096+0000 7f9473e00640 1 --2- 172.21.15.69:0/406549540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f947410a030 0x7f94741a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:39.096+0000 7f9473e00640 1 --2- 172.21.15.69:0/406549540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9474109670 0x7f947419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:39.096+0000 7f9473e00640 1 --2- 172.21.15.69:0/406549540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474108480 0x7f947419e620 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:39.096+0000 7f9473e00640 1 -- 172.21.15.69:0/406549540 >> 172.21.15.69:0/406549540 conn(0x7f9474077440 msgr2=0x7f947410c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:39.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:39.096+0000 7f9473e00640 1 -- 172.21.15.69:0/406549540 shutdown_connections 2024-04-24T09:04:39.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:39.096+0000 7f9473e00640 1 -- 172.21.15.69:0/406549540 wait complete. 2024-04-24T09:04:39.117 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-04-24T09:04:37.384365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597539,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":158,"num_read_kb":137,"num_write":290,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":354848,"kb_used_data":14004,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124586976,"statfs":{"total":1151940427776,"available":1151577063424,"internally_reserved":0,"allocated":14340096,"data_stored":7834209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2531,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":22,"num_read_kb":18,"num_write":60,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005587"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533023+0000","last_change":"2024-04-24T09:04:02.107965+0000","last_active":"2024-04-24T09:04:24.533023+0000","last_peered":"2024-04-24T09:04:24.533023+0000","last_clean":"2024-04-24T09:04:24.533023+0000","last_became_active":"2024-04-24T09:04:02.106827+0000","last_became_peered":"2024-04-24T09:04:02.106827+0000","last_unstale":"2024-04-24T09:04:24.533023+0000","last_undegraded":"2024-04-24T09:04:24.533023+0000","last_fullsized":"2024-04-24T09:04:24.533023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:20:23.930599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696753+0000","last_change":"2024-04-24T09:04:02.105449+0000","last_active":"2024-04-24T09:04:24.696753+0000","last_peered":"2024-04-24T09:04:24.696753+0000","last_clean":"2024-04-24T09:04:24.696753+0000","last_became_active":"2024-04-24T09:04:02.101842+0000","last_became_peered":"2024-04-24T09:04:02.101842+0000","last_unstale":"2024-04-24T09:04:24.696753+0000","last_undegraded":"2024-04-24T09:04:24.696753+0000","last_fullsized":"2024-04-24T09:04:24.696753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:46.911768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696652+0000","last_change":"2024-04-24T09:04:02.118686+0000","last_active":"2024-04-24T09:04:24.696652+0000","last_peered":"2024-04-24T09:04:24.696652+0000","last_clean":"2024-04-24T09:04:24.696652+0000","last_became_active":"2024-04-24T09:04:02.117549+0000","last_became_peered":"2024-04-24T09:04:02.117549+0000","last_unstale":"2024-04-24T09:04:24.696652+0000","last_undegraded":"2024-04-24T09:04:24.696652+0000","last_fullsized":"2024-04-24T09:04:24.696652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:18:16.233434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.521592+0000","last_change":"2024-04-24T09:04:02.120336+0000","last_active":"2024-04-24T09:04:24.521592+0000","last_peered":"2024-04-24T09:04:24.521592+0000","last_clean":"2024-04-24T09:04:24.521592+0000","last_became_active":"2024-04-24T09:04:02.116574+0000","last_became_peered":"2024-04-24T09:04:02.116574+0000","last_unstale":"2024-04-24T09:04:24.521592+0000","last_undegraded":"2024-04-24T09:04:24.521592+0000","last_fullsized":"2024-04-24T09:04:24.521592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:10:42.366531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.516713+0000","last_change":"2024-04-24T09:04:02.113600+0000","last_active":"2024-04-24T09:04:24.516713+0000","last_peered":"2024-04-24T09:04:24.516713+0000","last_clean":"2024-04-24T09:04:24.516713+0000","last_became_active":"2024-04-24T09:04:02.113143+0000","last_became_peered":"2024-04-24T09:04:02.113143+0000","last_unstale":"2024-04-24T09:04:24.516713+0000","last_undegraded":"2024-04-24T09:04:24.516713+0000","last_fullsized":"2024-04-24T09:04:24.516713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:31:27.666661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527374+0000","last_change":"2024-04-24T09:04:02.106395+0000","last_active":"2024-04-24T09:04:24.527374+0000","last_peered":"2024-04-24T09:04:24.527374+0000","last_clean":"2024-04-24T09:04:24.527374+0000","last_became_active":"2024-04-24T09:04:02.105942+0000","last_became_peered":"2024-04-24T09:04:02.105942+0000","last_unstale":"2024-04-24T09:04:24.527374+0000","last_undegraded":"2024-04-24T09:04:24.527374+0000","last_fullsized":"2024-04-24T09:04:24.527374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:20:10.655605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531485+0000","last_change":"2024-04-24T09:04:02.100279+0000","last_active":"2024-04-24T09:04:24.531485+0000","last_peered":"2024-04-24T09:04:24.531485+0000","last_clean":"2024-04-24T09:04:24.531485+0000","last_became_active":"2024-04-24T09:04:02.099745+0000","last_became_peered":"2024-04-24T09:04:02.099745+0000","last_unstale":"2024-04-24T09:04:24.531485+0000","last_undegraded":"2024-04-24T09:04:24.531485+0000","last_fullsized":"2024-04-24T09:04:24.531485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:13.150412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.526156+0000","last_change":"2024-04-24T09:04:02.103294+0000","last_active":"2024-04-24T09:04:24.526156+0000","last_peered":"2024-04-24T09:04:24.526156+0000","last_clean":"2024-04-24T09:04:24.526156+0000","last_became_active":"2024-04-24T09:04:02.102114+0000","last_became_peered":"2024-04-24T09:04:02.102114+0000","last_unstale":"2024-04-24T09:04:24.526156+0000","last_undegraded":"2024-04-24T09:04:24.526156+0000","last_fullsized":"2024-04-24T09:04:24.526156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:26:26.916097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.698004+0000","last_change":"2024-04-24T09:04:02.104936+0000","last_active":"2024-04-24T09:04:24.698004+0000","last_peered":"2024-04-24T09:04:24.698004+0000","last_clean":"2024-04-24T09:04:24.698004+0000","last_became_active":"2024-04-24T09:04:02.104540+0000","last_became_peered":"2024-04-24T09:04:02.104540+0000","last_unstale":"2024-04-24T09:04:24.698004+0000","last_undegraded":"2024-04-24T09:04:24.698004+0000","last_fullsized":"2024-04-24T09:04:24.698004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:36.010735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.528809+0000","last_change":"2024-04-24T09:04:02.102050+0000","last_active":"2024-04-24T09:04:24.528809+0000","last_peered":"2024-04-24T09:04:24.528809+0000","last_clean":"2024-04-24T09:04:24.528809+0000","last_became_active":"2024-04-24T09:04:02.101623+0000","last_became_peered":"2024-04-24T09:04:02.101623+0000","last_unstale":"2024-04-24T09:04:24.528809+0000","last_undegraded":"2024-04-24T09:04:24.528809+0000","last_fullsized":"2024-04-24T09:04:24.528809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:13:33.606200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533131+0000","last_change":"2024-04-24T09:04:02.115804+0000","last_active":"2024-04-24T09:04:24.533131+0000","last_peered":"2024-04-24T09:04:24.533131+0000","last_clean":"2024-04-24T09:04:24.533131+0000","last_became_active":"2024-04-24T09:04:02.115093+0000","last_became_peered":"2024-04-24T09:04:02.115093+0000","last_unstale":"2024-04-24T09:04:24.533131+0000","last_undegraded":"2024-04-24T09:04:24.533131+0000","last_fullsized":"2024-04-24T09:04:24.533131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:26:56.764485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.521254+0000","last_change":"2024-04-24T09:04:02.116305+0000","last_active":"2024-04-24T09:04:24.521254+0000","last_peered":"2024-04-24T09:04:24.521254+0000","last_clean":"2024-04-24T09:04:24.521254+0000","last_became_active":"2024-04-24T09:04:02.115956+0000","last_became_peered":"2024-04-24T09:04:02.115956+0000","last_unstale":"2024-04-24T09:04:24.521254+0000","last_undegraded":"2024-04-24T09:04:24.521254+0000","last_fullsized":"2024-04-24T09:04:24.521254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:38.618317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531595+0000","last_change":"2024-04-24T09:04:02.101155+0000","last_active":"2024-04-24T09:04:24.531595+0000","last_peered":"2024-04-24T09:04:24.531595+0000","last_clean":"2024-04-24T09:04:24.531595+0000","last_became_active":"2024-04-24T09:04:02.099719+0000","last_became_peered":"2024-04-24T09:04:02.099719+0000","last_unstale":"2024-04-24T09:04:24.531595+0000","last_undegraded":"2024-04-24T09:04:24.531595+0000","last_fullsized":"2024-04-24T09:04:24.531595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:46:27.910987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696158+0000","last_change":"2024-04-24T09:04:02.114818+0000","last_active":"2024-04-24T09:04:24.696158+0000","last_peered":"2024-04-24T09:04:24.696158+0000","last_clean":"2024-04-24T09:04:24.696158+0000","last_became_active":"2024-04-24T09:04:02.112947+0000","last_became_peered":"2024-04-24T09:04:02.112947+0000","last_unstale":"2024-04-24T09:04:24.696158+0000","last_undegraded":"2024-04-24T09:04:24.696158+0000","last_fullsized":"2024-04-24T09:04:24.696158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:16.544420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.695929+0000","last_change":"2024-04-24T09:04:02.120672+0000","last_active":"2024-04-24T09:04:24.695929+0000","last_peered":"2024-04-24T09:04:24.695929+0000","last_clean":"2024-04-24T09:04:24.695929+0000","last_became_active":"2024-04-24T09:04:02.120378+0000","last_became_peered":"2024-04-24T09:04:02.120378+0000","last_unstale":"2024-04-24T09:04:24.695929+0000","last_undegraded":"2024-04-24T09:04:24.695929+0000","last_fullsized":"2024-04-24T09:04:24.695929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:17.781189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696353+0000","last_change":"2024-04-24T09:04:02.107022+0000","last_active":"2024-04-24T09:04:24.696353+0000","last_peered":"2024-04-24T09:04:24.696353+0000","last_clean":"2024-04-24T09:04:24.696353+0000","last_became_active":"2024-04-24T09:04:02.106645+0000","last_became_peered":"2024-04-24T09:04:02.106645+0000","last_unstale":"2024-04-24T09:04:24.696353+0000","last_undegraded":"2024-04-24T09:04:24.696353+0000","last_fullsized":"2024-04-24T09:04:24.696353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:47.217718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532776+0000","last_change":"2024-04-24T09:04:02.115511+0000","last_active":"2024-04-24T09:04:24.532776+0000","last_peered":"2024-04-24T09:04:24.532776+0000","last_clean":"2024-04-24T09:04:24.532776+0000","last_became_active":"2024-04-24T09:04:02.115197+0000","last_became_peered":"2024-04-24T09:04:02.115197+0000","last_unstale":"2024-04-24T09:04:24.532776+0000","last_undegraded":"2024-04-24T09:04:24.532776+0000","last_fullsized":"2024-04-24T09:04:24.532776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:18.883688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.518672+0000","last_change":"2024-04-24T09:04:02.102108+0000","last_active":"2024-04-24T09:04:24.518672+0000","last_peered":"2024-04-24T09:04:24.518672+0000","last_clean":"2024-04-24T09:04:24.518672+0000","last_became_active":"2024-04-24T09:04:02.101739+0000","last_became_peered":"2024-04-24T09:04:02.101739+0000","last_unstale":"2024-04-24T09:04:24.518672+0000","last_undegraded":"2024-04-24T09:04:24.518672+0000","last_fullsized":"2024-04-24T09:04:24.518672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:57:30.768716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697591+0000","last_change":"2024-04-24T09:04:02.108911+0000","last_active":"2024-04-24T09:04:24.697591+0000","last_peered":"2024-04-24T09:04:24.697591+0000","last_clean":"2024-04-24T09:04:24.697591+0000","last_became_active":"2024-04-24T09:04:02.105678+0000","last_became_peered":"2024-04-24T09:04:02.105678+0000","last_unstale":"2024-04-24T09:04:24.697591+0000","last_undegraded":"2024-04-24T09:04:24.697591+0000","last_fullsized":"2024-04-24T09:04:24.697591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:36:12.216271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696112+0000","last_change":"2024-04-24T09:04:06.136320+0000","last_active":"2024-04-24T09:04:24.696112+0000","last_peered":"2024-04-24T09:04:24.696112+0000","last_clean":"2024-04-24T09:04:24.696112+0000","last_became_active":"2024-04-24T09:04:06.133851+0000","last_became_peered":"2024-04-24T09:04:06.133851+0000","last_unstale":"2024-04-24T09:04:24.696112+0000","last_undegraded":"2024-04-24T09:04:24.696112+0000","last_fullsized":"2024-04-24T09:04:24.696112+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:31:39.781234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.519248+0000","last_change":"2024-04-24T09:04:01.543818+0000","last_active":"2024-04-24T09:04:24.519248+0000","last_peered":"2024-04-24T09:04:24.519248+0000","last_clean":"2024-04-24T09:04:24.519248+0000","last_became_active":"2024-04-24T09:04:01.543435+0000","last_became_peered":"2024-04-24T09:04:01.543435+0000","last_unstale":"2024-04-24T09:04:24.519248+0000","last_undegraded":"2024-04-24T09:04:24.519248+0000","last_fullsized":"2024-04-24T09:04:24.519248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:08:30.176211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.526712+0000","last_change":"2024-04-24T09:04:02.101646+0000","last_active":"2024-04-24T09:04:24.526712+0000","last_peered":"2024-04-24T09:04:24.526712+0000","last_clean":"2024-04-24T09:04:24.526712+0000","last_became_active":"2024-04-24T09:04:02.101284+0000","last_became_peered":"2024-04-24T09:04:02.101284+0000","last_unstale":"2024-04-24T09:04:24.526712+0000","last_undegraded":"2024-04-24T09:04:24.526712+0000","last_fullsized":"2024-04-24T09:04:24.526712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:36:32.827216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.519708+0000","last_change":"2024-04-24T09:04:06.130476+0000","last_active":"2024-04-24T09:04:24.519708+0000","last_peered":"2024-04-24T09:04:24.519708+0000","last_clean":"2024-04-24T09:04:24.519708+0000","last_became_active":"2024-04-24T09:04:06.129707+0000","last_became_peered":"2024-04-24T09:04:06.129707+0000","last_unstale":"2024-04-24T09:04:24.519708+0000","last_undegraded":"2024-04-24T09:04:24.519708+0000","last_fullsized":"2024-04-24T09:04:24.519708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:25:36.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527016+0000","last_change":"2024-04-24T09:04:01.551708+0000","last_active":"2024-04-24T09:04:24.527016+0000","last_peered":"2024-04-24T09:04:24.527016+0000","last_clean":"2024-04-24T09:04:24.527016+0000","last_became_active":"2024-04-24T09:04:01.551089+0000","last_became_peered":"2024-04-24T09:04:01.551089+0000","last_unstale":"2024-04-24T09:04:24.527016+0000","last_undegraded":"2024-04-24T09:04:24.527016+0000","last_fullsized":"2024-04-24T09:04:24.527016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:18:22.515729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696599+0000","last_change":"2024-04-24T09:04:02.117356+0000","last_active":"2024-04-24T09:04:24.696599+0000","last_peered":"2024-04-24T09:04:24.696599+0000","last_clean":"2024-04-24T09:04:24.696599+0000","last_became_active":"2024-04-24T09:04:02.116972+0000","last_became_peered":"2024-04-24T09:04:02.116972+0000","last_unstale":"2024-04-24T09:04:24.696599+0000","last_undegraded":"2024-04-24T09:04:24.696599+0000","last_fullsized":"2024-04-24T09:04:24.696599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:46:29.349952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.520686+0000","last_change":"2024-04-24T09:04:06.141538+0000","last_active":"2024-04-24T09:04:24.520686+0000","last_peered":"2024-04-24T09:04:24.520686+0000","last_clean":"2024-04-24T09:04:24.520686+0000","last_became_active":"2024-04-24T09:04:06.140975+0000","last_became_peered":"2024-04-24T09:04:06.140975+0000","last_unstale":"2024-04-24T09:04:24.520686+0000","last_undegraded":"2024-04-24T09:04:24.520686+0000","last_fullsized":"2024-04-24T09:04:24.520686+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:58:31.688828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.630038+0000","last_change":"2024-04-24T09:04:01.561039+0000","last_active":"2024-04-24T09:04:33.630038+0000","last_peered":"2024-04-24T09:04:33.630038+0000","last_clean":"2024-04-24T09:04:33.630038+0000","last_became_active":"2024-04-24T09:04:01.560594+0000","last_became_peered":"2024-04-24T09:04:01.560594+0000","last_unstale":"2024-04-24T09:04:33.630038+0000","last_undegraded":"2024-04-24T09:04:33.630038+0000","last_fullsized":"2024-04-24T09:04:33.630038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:59:19.707826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697386+0000","last_change":"2024-04-24T09:04:02.106007+0000","last_active":"2024-04-24T09:04:24.697386+0000","last_peered":"2024-04-24T09:04:24.697386+0000","last_clean":"2024-04-24T09:04:24.697386+0000","last_became_active":"2024-04-24T09:04:02.099786+0000","last_became_peered":"2024-04-24T09:04:02.099786+0000","last_unstale":"2024-04-24T09:04:24.697386+0000","last_undegraded":"2024-04-24T09:04:24.697386+0000","last_fullsized":"2024-04-24T09:04:24.697386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:56.288417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.520162+0000","last_change":"2024-04-24T09:04:06.143636+0000","last_active":"2024-04-24T09:04:24.520162+0000","last_peered":"2024-04-24T09:04:24.520162+0000","last_clean":"2024-04-24T09:04:24.520162+0000","last_became_active":"2024-04-24T09:04:06.143105+0000","last_became_peered":"2024-04-24T09:04:06.143105+0000","last_unstale":"2024-04-24T09:04:24.520162+0000","last_undegraded":"2024-04-24T09:04:24.520162+0000","last_fullsized":"2024-04-24T09:04:24.520162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:04:23.211067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.517222+0000","last_change":"2024-04-24T09:04:01.548403+0000","last_active":"2024-04-24T09:04:24.517222+0000","last_peered":"2024-04-24T09:04:24.517222+0000","last_clean":"2024-04-24T09:04:24.517222+0000","last_became_active":"2024-04-24T09:04:01.547447+0000","last_became_peered":"2024-04-24T09:04:01.547447+0000","last_unstale":"2024-04-24T09:04:24.517222+0000","last_undegraded":"2024-04-24T09:04:24.517222+0000","last_fullsized":"2024-04-24T09:04:24.517222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:14:08.160127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533442+0000","last_change":"2024-04-24T09:04:02.121410+0000","last_active":"2024-04-24T09:04:24.533442+0000","last_peered":"2024-04-24T09:04:24.533442+0000","last_clean":"2024-04-24T09:04:24.533442+0000","last_became_active":"2024-04-24T09:04:02.120821+0000","last_became_peered":"2024-04-24T09:04:02.120821+0000","last_unstale":"2024-04-24T09:04:24.533442+0000","last_undegraded":"2024-04-24T09:04:24.533442+0000","last_fullsized":"2024-04-24T09:04:24.533442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:39:05.525171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696856+0000","last_change":"2024-04-24T09:04:06.142349+0000","last_active":"2024-04-24T09:04:24.696856+0000","last_peered":"2024-04-24T09:04:24.696856+0000","last_clean":"2024-04-24T09:04:24.696856+0000","last_became_active":"2024-04-24T09:04:06.141911+0000","last_became_peered":"2024-04-24T09:04:06.141911+0000","last_unstale":"2024-04-24T09:04:24.696856+0000","last_undegraded":"2024-04-24T09:04:24.696856+0000","last_fullsized":"2024-04-24T09:04:24.696856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:45:30.414686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'5","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:34.634991+0000","last_change":"2024-04-24T09:04:24.537349+0000","last_active":"2024-04-24T09:04:34.634991+0000","last_peered":"2024-04-24T09:04:34.634991+0000","last_clean":"2024-04-24T09:04:34.634991+0000","last_became_active":"2024-04-24T09:04:24.536738+0000","last_became_peered":"2024-04-24T09:04:24.536738+0000","last_unstale":"2024-04-24T09:04:34.634991+0000","last_undegraded":"2024-04-24T09:04:34.634991+0000","last_fullsized":"2024-04-24T09:04:34.634991+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:33.481380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.516838+0000","last_change":"2024-04-24T09:04:02.111173+0000","last_active":"2024-04-24T09:04:24.516838+0000","last_peered":"2024-04-24T09:04:24.516838+0000","last_clean":"2024-04-24T09:04:24.516838+0000","last_became_active":"2024-04-24T09:04:02.110681+0000","last_became_peered":"2024-04-24T09:04:02.110681+0000","last_unstale":"2024-04-24T09:04:24.516838+0000","last_undegraded":"2024-04-24T09:04:24.516838+0000","last_fullsized":"2024-04-24T09:04:24.516838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:29.939116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696716+0000","last_change":"2024-04-24T09:04:06.137752+0000","last_active":"2024-04-24T09:04:24.696716+0000","last_peered":"2024-04-24T09:04:24.696716+0000","last_clean":"2024-04-24T09:04:24.696716+0000","last_became_active":"2024-04-24T09:04:06.137492+0000","last_became_peered":"2024-04-24T09:04:06.137492+0000","last_unstale":"2024-04-24T09:04:24.696716+0000","last_undegraded":"2024-04-24T09:04:24.696716+0000","last_fullsized":"2024-04-24T09:04:24.696716+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:44:00.653864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.628109+0000","last_change":"2024-04-24T09:04:01.544009+0000","last_active":"2024-04-24T09:04:33.628109+0000","last_peered":"2024-04-24T09:04:33.628109+0000","last_clean":"2024-04-24T09:04:33.628109+0000","last_became_active":"2024-04-24T09:04:01.543651+0000","last_became_peered":"2024-04-24T09:04:01.543651+0000","last_unstale":"2024-04-24T09:04:33.628109+0000","last_undegraded":"2024-04-24T09:04:33.628109+0000","last_fullsized":"2024-04-24T09:04:33.628109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:10:21.969072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531764+0000","last_change":"2024-04-24T09:04:02.115387+0000","last_active":"2024-04-24T09:04:24.531764+0000","last_peered":"2024-04-24T09:04:24.531764+0000","last_clean":"2024-04-24T09:04:24.531764+0000","last_became_active":"2024-04-24T09:04:02.115086+0000","last_became_peered":"2024-04-24T09:04:02.115086+0000","last_unstale":"2024-04-24T09:04:24.531764+0000","last_undegraded":"2024-04-24T09:04:24.531764+0000","last_fullsized":"2024-04-24T09:04:24.531764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:40:57.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532516+0000","last_change":"2024-04-24T09:04:06.146189+0000","last_active":"2024-04-24T09:04:24.532516+0000","last_peered":"2024-04-24T09:04:24.532516+0000","last_clean":"2024-04-24T09:04:24.532516+0000","last_became_active":"2024-04-24T09:04:06.145556+0000","last_became_peered":"2024-04-24T09:04:06.145556+0000","last_unstale":"2024-04-24T09:04:24.532516+0000","last_undegraded":"2024-04-24T09:04:24.532516+0000","last_fullsized":"2024-04-24T09:04:24.532516+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:41.371948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.628679+0000","last_change":"2024-04-24T09:04:01.542185+0000","last_active":"2024-04-24T09:04:33.628679+0000","last_peered":"2024-04-24T09:04:33.628679+0000","last_clean":"2024-04-24T09:04:33.628679+0000","last_became_active":"2024-04-24T09:04:01.541240+0000","last_became_peered":"2024-04-24T09:04:01.541240+0000","last_unstale":"2024-04-24T09:04:33.628679+0000","last_undegraded":"2024-04-24T09:04:33.628679+0000","last_fullsized":"2024-04-24T09:04:33.628679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:15.103276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.525028+0000","last_change":"2024-04-24T09:04:02.104054+0000","last_active":"2024-04-24T09:04:24.525028+0000","last_peered":"2024-04-24T09:04:24.525028+0000","last_clean":"2024-04-24T09:04:24.525028+0000","last_became_active":"2024-04-24T09:04:02.103673+0000","last_became_peered":"2024-04-24T09:04:02.103673+0000","last_unstale":"2024-04-24T09:04:24.525028+0000","last_undegraded":"2024-04-24T09:04:24.525028+0000","last_fullsized":"2024-04-24T09:04:24.525028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:31:58.889989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696675+0000","last_change":"2024-04-24T09:04:06.134034+0000","last_active":"2024-04-24T09:04:24.696675+0000","last_peered":"2024-04-24T09:04:24.696675+0000","last_clean":"2024-04-24T09:04:24.696675+0000","last_became_active":"2024-04-24T09:04:06.133596+0000","last_became_peered":"2024-04-24T09:04:06.133596+0000","last_unstale":"2024-04-24T09:04:24.696675+0000","last_undegraded":"2024-04-24T09:04:24.696675+0000","last_fullsized":"2024-04-24T09:04:24.696675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:17:09.572974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.528733+0000","last_change":"2024-04-24T09:04:01.543312+0000","last_active":"2024-04-24T09:04:24.528733+0000","last_peered":"2024-04-24T09:04:24.528733+0000","last_clean":"2024-04-24T09:04:24.528733+0000","last_became_active":"2024-04-24T09:04:01.542914+0000","last_became_peered":"2024-04-24T09:04:01.542914+0000","last_unstale":"2024-04-24T09:04:24.528733+0000","last_undegraded":"2024-04-24T09:04:24.528733+0000","last_fullsized":"2024-04-24T09:04:24.528733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:50.695108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.517056+0000","last_change":"2024-04-24T09:04:02.114445+0000","last_active":"2024-04-24T09:04:24.517056+0000","last_peered":"2024-04-24T09:04:24.517056+0000","last_clean":"2024-04-24T09:04:24.517056+0000","last_became_active":"2024-04-24T09:04:02.114075+0000","last_became_peered":"2024-04-24T09:04:02.114075+0000","last_unstale":"2024-04-24T09:04:24.517056+0000","last_undegraded":"2024-04-24T09:04:24.517056+0000","last_fullsized":"2024-04-24T09:04:24.517056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:35:06.933677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.525318+0000","last_change":"2024-04-24T09:04:06.142368+0000","last_active":"2024-04-24T09:04:24.525318+0000","last_peered":"2024-04-24T09:04:24.525318+0000","last_clean":"2024-04-24T09:04:24.525318+0000","last_became_active":"2024-04-24T09:04:06.141872+0000","last_became_peered":"2024-04-24T09:04:06.141872+0000","last_unstale":"2024-04-24T09:04:24.525318+0000","last_undegraded":"2024-04-24T09:04:24.525318+0000","last_fullsized":"2024-04-24T09:04:24.525318+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:13:43.733662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.612462+0000","last_change":"2024-04-24T09:04:01.556002+0000","last_active":"2024-04-24T09:04:32.612462+0000","last_peered":"2024-04-24T09:04:32.612462+0000","last_clean":"2024-04-24T09:04:32.612462+0000","last_became_active":"2024-04-24T09:04:01.555643+0000","last_became_peered":"2024-04-24T09:04:01.555643+0000","last_unstale":"2024-04-24T09:04:32.612462+0000","last_undegraded":"2024-04-24T09:04:32.612462+0000","last_fullsized":"2024-04-24T09:04:32.612462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:30:31.171184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697765+0000","last_change":"2024-04-24T09:04:02.104252+0000","last_active":"2024-04-24T09:04:24.697765+0000","last_peered":"2024-04-24T09:04:24.697765+0000","last_clean":"2024-04-24T09:04:24.697765+0000","last_became_active":"2024-04-24T09:04:02.099264+0000","last_became_peered":"2024-04-24T09:04:02.099264+0000","last_unstale":"2024-04-24T09:04:24.697765+0000","last_undegraded":"2024-04-24T09:04:24.697765+0000","last_fullsized":"2024-04-24T09:04:24.697765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:28.469814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.516701+0000","last_change":"2024-04-24T09:04:06.132407+0000","last_active":"2024-04-24T09:04:24.516701+0000","last_peered":"2024-04-24T09:04:24.516701+0000","last_clean":"2024-04-24T09:04:24.516701+0000","last_became_active":"2024-04-24T09:04:06.131997+0000","last_became_peered":"2024-04-24T09:04:06.131997+0000","last_unstale":"2024-04-24T09:04:24.516701+0000","last_undegraded":"2024-04-24T09:04:24.516701+0000","last_fullsized":"2024-04-24T09:04:24.516701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:08:07.369326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.613853+0000","last_change":"2024-04-24T09:04:01.561039+0000","last_active":"2024-04-24T09:04:32.613853+0000","last_peered":"2024-04-24T09:04:32.613853+0000","last_clean":"2024-04-24T09:04:32.613853+0000","last_became_active":"2024-04-24T09:04:01.560543+0000","last_became_peered":"2024-04-24T09:04:01.560543+0000","last_unstale":"2024-04-24T09:04:32.613853+0000","last_undegraded":"2024-04-24T09:04:32.613853+0000","last_fullsized":"2024-04-24T09:04:32.613853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:27.706982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697177+0000","last_change":"2024-04-24T09:04:02.110850+0000","last_active":"2024-04-24T09:04:24.697177+0000","last_peered":"2024-04-24T09:04:24.697177+0000","last_clean":"2024-04-24T09:04:24.697177+0000","last_became_active":"2024-04-24T09:04:02.110512+0000","last_became_peered":"2024-04-24T09:04:02.110512+0000","last_unstale":"2024-04-24T09:04:24.697177+0000","last_undegraded":"2024-04-24T09:04:24.697177+0000","last_fullsized":"2024-04-24T09:04:24.697177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:19:16.937202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.521039+0000","last_change":"2024-04-24T09:04:06.135200+0000","last_active":"2024-04-24T09:04:24.521039+0000","last_peered":"2024-04-24T09:04:24.521039+0000","last_clean":"2024-04-24T09:04:24.521039+0000","last_became_active":"2024-04-24T09:04:06.134680+0000","last_became_peered":"2024-04-24T09:04:06.134680+0000","last_unstale":"2024-04-24T09:04:24.521039+0000","last_undegraded":"2024-04-24T09:04:24.521039+0000","last_fullsized":"2024-04-24T09:04:24.521039+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:48:57.697740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.612218+0000","last_change":"2024-04-24T09:04:01.555069+0000","last_active":"2024-04-24T09:04:32.612218+0000","last_peered":"2024-04-24T09:04:32.612218+0000","last_clean":"2024-04-24T09:04:32.612218+0000","last_became_active":"2024-04-24T09:04:01.554737+0000","last_became_peered":"2024-04-24T09:04:01.554737+0000","last_unstale":"2024-04-24T09:04:32.612218+0000","last_undegraded":"2024-04-24T09:04:32.612218+0000","last_fullsized":"2024-04-24T09:04:32.612218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:23:00.272697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697205+0000","last_change":"2024-04-24T09:04:02.112126+0000","last_active":"2024-04-24T09:04:24.697205+0000","last_peered":"2024-04-24T09:04:24.697205+0000","last_clean":"2024-04-24T09:04:24.697205+0000","last_became_active":"2024-04-24T09:04:02.107995+0000","last_became_peered":"2024-04-24T09:04:02.107995+0000","last_unstale":"2024-04-24T09:04:24.697205+0000","last_undegraded":"2024-04-24T09:04:24.697205+0000","last_fullsized":"2024-04-24T09:04:24.697205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:13.189829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.542302+0000","last_change":"2024-04-24T09:04:24.542302+0000","last_active":"2024-04-24T09:04:24.542302+0000","last_peered":"2024-04-24T09:04:24.542302+0000","last_clean":"2024-04-24T09:04:24.542302+0000","last_became_active":"2024-04-24T09:04:24.539848+0000","last_became_peered":"2024-04-24T09:04:24.539848+0000","last_unstale":"2024-04-24T09:04:24.542302+0000","last_undegraded":"2024-04-24T09:04:24.542302+0000","last_fullsized":"2024-04-24T09:04:24.542302+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:01:52.055510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"92'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.613748+0000","last_change":"2024-04-24T09:04:01.548610+0000","last_active":"2024-04-24T09:04:32.613748+0000","last_peered":"2024-04-24T09:04:32.613748+0000","last_clean":"2024-04-24T09:04:32.613748+0000","last_became_active":"2024-04-24T09:04:01.548145+0000","last_became_peered":"2024-04-24T09:04:01.548145+0000","last_unstale":"2024-04-24T09:04:32.613748+0000","last_undegraded":"2024-04-24T09:04:32.613748+0000","last_fullsized":"2024-04-24T09:04:32.613748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:58:48.425187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.516908+0000","last_change":"2024-04-24T09:04:02.109965+0000","last_active":"2024-04-24T09:04:24.516908+0000","last_peered":"2024-04-24T09:04:24.516908+0000","last_clean":"2024-04-24T09:04:24.516908+0000","last_became_active":"2024-04-24T09:04:02.108143+0000","last_became_peered":"2024-04-24T09:04:02.108143+0000","last_unstale":"2024-04-24T09:04:24.516908+0000","last_undegraded":"2024-04-24T09:04:24.516908+0000","last_fullsized":"2024-04-24T09:04:24.516908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:01:21.083039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.534893+0000","last_change":"2024-04-24T09:04:06.145369+0000","last_active":"2024-04-24T09:04:24.534893+0000","last_peered":"2024-04-24T09:04:24.534893+0000","last_clean":"2024-04-24T09:04:24.534893+0000","last_became_active":"2024-04-24T09:04:06.144884+0000","last_became_peered":"2024-04-24T09:04:06.144884+0000","last_unstale":"2024-04-24T09:04:24.534893+0000","last_undegraded":"2024-04-24T09:04:24.534893+0000","last_fullsized":"2024-04-24T09:04:24.534893+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:29:22.780846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.614895+0000","last_change":"2024-04-24T09:04:01.548195+0000","last_active":"2024-04-24T09:04:32.614895+0000","last_peered":"2024-04-24T09:04:32.614895+0000","last_clean":"2024-04-24T09:04:32.614895+0000","last_became_active":"2024-04-24T09:04:01.544969+0000","last_became_peered":"2024-04-24T09:04:01.544969+0000","last_unstale":"2024-04-24T09:04:32.614895+0000","last_undegraded":"2024-04-24T09:04:32.614895+0000","last_fullsized":"2024-04-24T09:04:32.614895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:32:55.294629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.529490+0000","last_change":"2024-04-24T09:04:02.103785+0000","last_active":"2024-04-24T09:04:24.529490+0000","last_peered":"2024-04-24T09:04:24.529490+0000","last_clean":"2024-04-24T09:04:24.529490+0000","last_became_active":"2024-04-24T09:04:02.103463+0000","last_became_peered":"2024-04-24T09:04:02.103463+0000","last_unstale":"2024-04-24T09:04:24.529490+0000","last_undegraded":"2024-04-24T09:04:24.529490+0000","last_fullsized":"2024-04-24T09:04:24.529490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:56.385382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527499+0000","last_change":"2024-04-24T09:04:06.149955+0000","last_active":"2024-04-24T09:04:24.527499+0000","last_peered":"2024-04-24T09:04:24.527499+0000","last_clean":"2024-04-24T09:04:24.527499+0000","last_became_active":"2024-04-24T09:04:06.149445+0000","last_became_peered":"2024-04-24T09:04:06.149445+0000","last_unstale":"2024-04-24T09:04:24.527499+0000","last_undegraded":"2024-04-24T09:04:24.527499+0000","last_fullsized":"2024-04-24T09:04:24.527499+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:30:42.527007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.615210+0000","last_change":"2024-04-24T09:04:01.548740+0000","last_active":"2024-04-24T09:04:32.615210+0000","last_peered":"2024-04-24T09:04:32.615210+0000","last_clean":"2024-04-24T09:04:32.615210+0000","last_became_active":"2024-04-24T09:04:01.546060+0000","last_became_peered":"2024-04-24T09:04:01.546060+0000","last_unstale":"2024-04-24T09:04:32.615210+0000","last_undegraded":"2024-04-24T09:04:32.615210+0000","last_fullsized":"2024-04-24T09:04:32.615210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:09:16.301150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.538871+0000","last_change":"2024-04-24T09:04:24.538871+0000","last_active":"2024-04-24T09:04:24.538871+0000","last_peered":"2024-04-24T09:04:24.538871+0000","last_clean":"2024-04-24T09:04:24.538871+0000","last_became_active":"2024-04-24T09:04:24.537039+0000","last_became_peered":"2024-04-24T09:04:24.537039+0000","last_unstale":"2024-04-24T09:04:24.538871+0000","last_undegraded":"2024-04-24T09:04:24.538871+0000","last_fullsized":"2024-04-24T09:04:24.538871+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:18.520407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.518409+0000","last_change":"2024-04-24T09:04:06.136973+0000","last_active":"2024-04-24T09:04:24.518409+0000","last_peered":"2024-04-24T09:04:24.518409+0000","last_clean":"2024-04-24T09:04:24.518409+0000","last_became_active":"2024-04-24T09:04:06.136405+0000","last_became_peered":"2024-04-24T09:04:06.136405+0000","last_unstale":"2024-04-24T09:04:24.518409+0000","last_undegraded":"2024-04-24T09:04:24.518409+0000","last_fullsized":"2024-04-24T09:04:24.518409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:07.653162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'4","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:35.638952+0000","last_change":"2024-04-24T09:04:01.546853+0000","last_active":"2024-04-24T09:04:35.638952+0000","last_peered":"2024-04-24T09:04:35.638952+0000","last_clean":"2024-04-24T09:04:35.638952+0000","last_became_active":"2024-04-24T09:04:01.546441+0000","last_became_peered":"2024-04-24T09:04:01.546441+0000","last_unstale":"2024-04-24T09:04:35.638952+0000","last_undegraded":"2024-04-24T09:04:35.638952+0000","last_fullsized":"2024-04-24T09:04:35.638952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:23:56.333410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533833+0000","last_change":"2024-04-24T09:04:02.101158+0000","last_active":"2024-04-24T09:04:24.533833+0000","last_peered":"2024-04-24T09:04:24.533833+0000","last_clean":"2024-04-24T09:04:24.533833+0000","last_became_active":"2024-04-24T09:04:02.100832+0000","last_became_peered":"2024-04-24T09:04:02.100832+0000","last_unstale":"2024-04-24T09:04:24.533833+0000","last_undegraded":"2024-04-24T09:04:24.533833+0000","last_fullsized":"2024-04-24T09:04:24.533833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:05:34.080520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533094+0000","last_change":"2024-04-24T09:04:06.149176+0000","last_active":"2024-04-24T09:04:24.533094+0000","last_peered":"2024-04-24T09:04:24.533094+0000","last_clean":"2024-04-24T09:04:24.533094+0000","last_became_active":"2024-04-24T09:04:06.145539+0000","last_became_peered":"2024-04-24T09:04:06.145539+0000","last_unstale":"2024-04-24T09:04:24.533094+0000","last_undegraded":"2024-04-24T09:04:24.533094+0000","last_fullsized":"2024-04-24T09:04:24.533094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:38.719787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.632543+0000","last_change":"2024-04-24T09:04:01.555330+0000","last_active":"2024-04-24T09:04:33.632543+0000","last_peered":"2024-04-24T09:04:33.632543+0000","last_clean":"2024-04-24T09:04:33.632543+0000","last_became_active":"2024-04-24T09:04:01.552618+0000","last_became_peered":"2024-04-24T09:04:01.552618+0000","last_unstale":"2024-04-24T09:04:33.632543+0000","last_undegraded":"2024-04-24T09:04:33.632543+0000","last_fullsized":"2024-04-24T09:04:33.632543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:30.956657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.529355+0000","last_change":"2024-04-24T09:04:02.116301+0000","last_active":"2024-04-24T09:04:24.529355+0000","last_peered":"2024-04-24T09:04:24.529355+0000","last_clean":"2024-04-24T09:04:24.529355+0000","last_became_active":"2024-04-24T09:04:02.116001+0000","last_became_peered":"2024-04-24T09:04:02.116001+0000","last_unstale":"2024-04-24T09:04:24.529355+0000","last_undegraded":"2024-04-24T09:04:24.529355+0000","last_fullsized":"2024-04-24T09:04:24.529355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:16:18.995067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.529071+0000","last_change":"2024-04-24T09:04:06.135814+0000","last_active":"2024-04-24T09:04:24.529071+0000","last_peered":"2024-04-24T09:04:24.529071+0000","last_clean":"2024-04-24T09:04:24.529071+0000","last_became_active":"2024-04-24T09:04:06.135054+0000","last_became_peered":"2024-04-24T09:04:06.135054+0000","last_unstale":"2024-04-24T09:04:24.529071+0000","last_undegraded":"2024-04-24T09:04:24.529071+0000","last_fullsized":"2024-04-24T09:04:24.529071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:51:31.150589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:34.618324+0000","last_change":"2024-04-24T09:04:01.553144+0000","last_active":"2024-04-24T09:04:34.618324+0000","last_peered":"2024-04-24T09:04:34.618324+0000","last_clean":"2024-04-24T09:04:34.618324+0000","last_became_active":"2024-04-24T09:04:01.550599+0000","last_became_peered":"2024-04-24T09:04:01.550599+0000","last_unstale":"2024-04-24T09:04:34.618324+0000","last_undegraded":"2024-04-24T09:04:34.618324+0000","last_fullsized":"2024-04-24T09:04:34.618324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:41.950209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532070+0000","last_change":"2024-04-24T09:04:02.117064+0000","last_active":"2024-04-24T09:04:24.532070+0000","last_peered":"2024-04-24T09:04:24.532070+0000","last_clean":"2024-04-24T09:04:24.532070+0000","last_became_active":"2024-04-24T09:04:02.114528+0000","last_became_peered":"2024-04-24T09:04:02.114528+0000","last_unstale":"2024-04-24T09:04:24.532070+0000","last_undegraded":"2024-04-24T09:04:24.532070+0000","last_fullsized":"2024-04-24T09:04:24.532070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:24:53.662571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.701058+0000","last_change":"2024-04-24T09:04:24.701058+0000","last_active":"2024-04-24T09:04:24.701058+0000","last_peered":"2024-04-24T09:04:24.701058+0000","last_clean":"2024-04-24T09:04:24.701058+0000","last_became_active":"2024-04-24T09:04:24.699766+0000","last_became_peered":"2024-04-24T09:04:24.699766+0000","last_unstale":"2024-04-24T09:04:24.701058+0000","last_undegraded":"2024-04-24T09:04:24.701058+0000","last_fullsized":"2024-04-24T09:04:24.701058+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:57:48.491073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.701816+0000","last_change":"2024-04-24T09:04:24.701816+0000","last_active":"2024-04-24T09:04:24.701816+0000","last_peered":"2024-04-24T09:04:24.701816+0000","last_clean":"2024-04-24T09:04:24.701816+0000","last_became_active":"2024-04-24T09:04:24.700530+0000","last_became_peered":"2024-04-24T09:04:24.700530+0000","last_unstale":"2024-04-24T09:04:24.701816+0000","last_undegraded":"2024-04-24T09:04:24.701816+0000","last_fullsized":"2024-04-24T09:04:24.701816+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:24.614043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527276+0000","last_change":"2024-04-24T09:04:02.104602+0000","last_active":"2024-04-24T09:04:24.527276+0000","last_peered":"2024-04-24T09:04:24.527276+0000","last_clean":"2024-04-24T09:04:24.527276+0000","last_became_active":"2024-04-24T09:04:02.104188+0000","last_became_peered":"2024-04-24T09:04:02.104188+0000","last_unstale":"2024-04-24T09:04:24.527276+0000","last_undegraded":"2024-04-24T09:04:24.527276+0000","last_fullsized":"2024-04-24T09:04:24.527276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:00:50.286952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531886+0000","last_change":"2024-04-24T09:04:06.134971+0000","last_active":"2024-04-24T09:04:24.531886+0000","last_peered":"2024-04-24T09:04:24.531886+0000","last_clean":"2024-04-24T09:04:24.531886+0000","last_became_active":"2024-04-24T09:04:06.134551+0000","last_became_peered":"2024-04-24T09:04:06.134551+0000","last_unstale":"2024-04-24T09:04:24.531886+0000","last_undegraded":"2024-04-24T09:04:24.531886+0000","last_fullsized":"2024-04-24T09:04:24.531886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:40:41.766657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532337+0000","last_change":"2024-04-24T09:04:01.551117+0000","last_active":"2024-04-24T09:04:24.532337+0000","last_peered":"2024-04-24T09:04:24.532337+0000","last_clean":"2024-04-24T09:04:24.532337+0000","last_became_active":"2024-04-24T09:04:01.550646+0000","last_became_peered":"2024-04-24T09:04:01.550646+0000","last_unstale":"2024-04-24T09:04:24.532337+0000","last_undegraded":"2024-04-24T09:04:24.532337+0000","last_fullsized":"2024-04-24T09:04:24.532337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:29:03.821300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.526140+0000","last_change":"2024-04-24T09:04:02.107111+0000","last_active":"2024-04-24T09:04:24.526140+0000","last_peered":"2024-04-24T09:04:24.526140+0000","last_clean":"2024-04-24T09:04:24.526140+0000","last_became_active":"2024-04-24T09:04:02.106185+0000","last_became_peered":"2024-04-24T09:04:02.106185+0000","last_unstale":"2024-04-24T09:04:24.526140+0000","last_undegraded":"2024-04-24T09:04:24.526140+0000","last_fullsized":"2024-04-24T09:04:24.526140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:19:03.111407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532094+0000","last_change":"2024-04-24T09:04:06.143984+0000","last_active":"2024-04-24T09:04:24.532094+0000","last_peered":"2024-04-24T09:04:24.532094+0000","last_clean":"2024-04-24T09:04:24.532094+0000","last_became_active":"2024-04-24T09:04:06.143503+0000","last_became_peered":"2024-04-24T09:04:06.143503+0000","last_unstale":"2024-04-24T09:04:24.532094+0000","last_undegraded":"2024-04-24T09:04:24.532094+0000","last_fullsized":"2024-04-24T09:04:24.532094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:35:19.317771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.529007+0000","last_change":"2024-04-24T09:04:02.111560+0000","last_active":"2024-04-24T09:04:24.529007+0000","last_peered":"2024-04-24T09:04:24.529007+0000","last_clean":"2024-04-24T09:04:24.529007+0000","last_became_active":"2024-04-24T09:04:02.109885+0000","last_became_peered":"2024-04-24T09:04:02.109885+0000","last_unstale":"2024-04-24T09:04:24.529007+0000","last_undegraded":"2024-04-24T09:04:24.529007+0000","last_fullsized":"2024-04-24T09:04:24.529007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:35.837621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:34.633044+0000","last_change":"2024-04-24T09:04:01.542530+0000","last_active":"2024-04-24T09:04:34.633044+0000","last_peered":"2024-04-24T09:04:34.633044+0000","last_clean":"2024-04-24T09:04:34.633044+0000","last_became_active":"2024-04-24T09:04:01.541702+0000","last_became_peered":"2024-04-24T09:04:01.541702+0000","last_unstale":"2024-04-24T09:04:34.633044+0000","last_undegraded":"2024-04-24T09:04:34.633044+0000","last_fullsized":"2024-04-24T09:04:34.633044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:28:29.420020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532535+0000","last_change":"2024-04-24T09:04:06.143004+0000","last_active":"2024-04-24T09:04:24.532535+0000","last_peered":"2024-04-24T09:04:24.532535+0000","last_clean":"2024-04-24T09:04:24.532535+0000","last_became_active":"2024-04-24T09:04:06.142444+0000","last_became_peered":"2024-04-24T09:04:06.142444+0000","last_unstale":"2024-04-24T09:04:24.532535+0000","last_undegraded":"2024-04-24T09:04:24.532535+0000","last_fullsized":"2024-04-24T09:04:24.532535+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:13.910612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.704292+0000","last_change":"2024-04-24T09:04:24.701330+0000","last_active":"2024-04-24T09:04:24.704292+0000","last_peered":"2024-04-24T09:04:24.704292+0000","last_clean":"2024-04-24T09:04:24.704292+0000","last_became_active":"2024-04-24T09:04:24.699988+0000","last_became_peered":"2024-04-24T09:04:24.699988+0000","last_unstale":"2024-04-24T09:04:24.704292+0000","last_undegraded":"2024-04-24T09:04:24.704292+0000","last_fullsized":"2024-04-24T09:04:24.704292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:23:24.610671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:34.629794+0000","last_change":"2024-04-24T09:04:01.541206+0000","last_active":"2024-04-24T09:04:34.629794+0000","last_peered":"2024-04-24T09:04:34.629794+0000","last_clean":"2024-04-24T09:04:34.629794+0000","last_became_active":"2024-04-24T09:04:01.539888+0000","last_became_peered":"2024-04-24T09:04:01.539888+0000","last_unstale":"2024-04-24T09:04:34.629794+0000","last_undegraded":"2024-04-24T09:04:34.629794+0000","last_fullsized":"2024-04-24T09:04:34.629794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:01.550209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531165+0000","last_change":"2024-04-24T09:02:22.629840+0000","last_active":"2024-04-24T09:04:24.531165+0000","last_peered":"2024-04-24T09:04:24.531165+0000","last_clean":"2024-04-24T09:04:24.531165+0000","last_became_active":"2024-04-24T09:02:22.628700+0000","last_became_peered":"2024-04-24T09:02:22.628700+0000","last_unstale":"2024-04-24T09:04:24.531165+0000","last_undegraded":"2024-04-24T09:04:24.531165+0000","last_fullsized":"2024-04-24T09:04:24.531165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:57:38.349295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:57:38.349295+0000","last_clean_scrub_stamp":"2024-04-24T08:57:38.349295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:54:11.409492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533333+0000","last_change":"2024-04-24T09:04:06.149548+0000","last_active":"2024-04-24T09:04:24.533333+0000","last_peered":"2024-04-24T09:04:24.533333+0000","last_clean":"2024-04-24T09:04:24.533333+0000","last_became_active":"2024-04-24T09:04:06.149214+0000","last_became_peered":"2024-04-24T09:04:06.149214+0000","last_unstale":"2024-04-24T09:04:24.533333+0000","last_undegraded":"2024-04-24T09:04:24.533333+0000","last_fullsized":"2024-04-24T09:04:24.533333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:03.801242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.528440+0000","last_change":"2024-04-24T09:04:02.110242+0000","last_active":"2024-04-24T09:04:24.528440+0000","last_peered":"2024-04-24T09:04:24.528440+0000","last_clean":"2024-04-24T09:04:24.528440+0000","last_became_active":"2024-04-24T09:04:02.109879+0000","last_became_peered":"2024-04-24T09:04:02.109879+0000","last_unstale":"2024-04-24T09:04:24.528440+0000","last_undegraded":"2024-04-24T09:04:24.528440+0000","last_fullsized":"2024-04-24T09:04:24.528440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:16.353861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.628491+0000","last_change":"2024-04-24T09:04:01.550989+0000","last_active":"2024-04-24T09:04:33.628491+0000","last_peered":"2024-04-24T09:04:33.628491+0000","last_clean":"2024-04-24T09:04:33.628491+0000","last_became_active":"2024-04-24T09:04:01.550664+0000","last_became_peered":"2024-04-24T09:04:01.550664+0000","last_unstale":"2024-04-24T09:04:33.628491+0000","last_undegraded":"2024-04-24T09:04:33.628491+0000","last_fullsized":"2024-04-24T09:04:33.628491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:33:39.521951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.538207+0000","last_change":"2024-04-24T09:04:24.538207+0000","last_active":"2024-04-24T09:04:24.538207+0000","last_peered":"2024-04-24T09:04:24.538207+0000","last_clean":"2024-04-24T09:04:24.538207+0000","last_became_active":"2024-04-24T09:04:24.536894+0000","last_became_peered":"2024-04-24T09:04:24.536894+0000","last_unstale":"2024-04-24T09:04:24.538207+0000","last_undegraded":"2024-04-24T09:04:24.538207+0000","last_fullsized":"2024-04-24T09:04:24.538207+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:32:52.860371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.608599+0000","last_change":"2024-04-24T09:04:01.539415+0000","last_active":"2024-04-24T09:04:32.608599+0000","last_peered":"2024-04-24T09:04:32.608599+0000","last_clean":"2024-04-24T09:04:32.608599+0000","last_became_active":"2024-04-24T09:04:01.539020+0000","last_became_peered":"2024-04-24T09:04:01.539020+0000","last_unstale":"2024-04-24T09:04:32.608599+0000","last_undegraded":"2024-04-24T09:04:32.608599+0000","last_fullsized":"2024-04-24T09:04:32.608599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:12:48.728029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532453+0000","last_change":"2024-04-24T09:04:02.097560+0000","last_active":"2024-04-24T09:04:24.532453+0000","last_peered":"2024-04-24T09:04:24.532453+0000","last_clean":"2024-04-24T09:04:24.532453+0000","last_became_active":"2024-04-24T09:04:02.095612+0000","last_became_peered":"2024-04-24T09:04:02.095612+0000","last_unstale":"2024-04-24T09:04:24.532453+0000","last_undegraded":"2024-04-24T09:04:24.532453+0000","last_fullsized":"2024-04-24T09:04:24.532453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:55:47.747294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.524769+0000","last_change":"2024-04-24T09:04:06.137822+0000","last_active":"2024-04-24T09:04:24.524769+0000","last_peered":"2024-04-24T09:04:24.524769+0000","last_clean":"2024-04-24T09:04:24.524769+0000","last_became_active":"2024-04-24T09:04:06.137357+0000","last_became_peered":"2024-04-24T09:04:06.137357+0000","last_unstale":"2024-04-24T09:04:24.524769+0000","last_undegraded":"2024-04-24T09:04:24.524769+0000","last_fullsized":"2024-04-24T09:04:24.524769+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:55:13.119557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.544395+0000","last_change":"2024-04-24T09:04:24.544395+0000","last_active":"2024-04-24T09:04:24.544395+0000","last_peered":"2024-04-24T09:04:24.544395+0000","last_clean":"2024-04-24T09:04:24.544395+0000","last_became_active":"2024-04-24T09:04:24.535870+0000","last_became_peered":"2024-04-24T09:04:24.535870+0000","last_unstale":"2024-04-24T09:04:24.544395+0000","last_undegraded":"2024-04-24T09:04:24.544395+0000","last_fullsized":"2024-04-24T09:04:24.544395+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:29:21.271387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527426+0000","last_change":"2024-04-24T09:04:02.106107+0000","last_active":"2024-04-24T09:04:24.527426+0000","last_peered":"2024-04-24T09:04:24.527426+0000","last_clean":"2024-04-24T09:04:24.527426+0000","last_became_active":"2024-04-24T09:04:02.105734+0000","last_became_peered":"2024-04-24T09:04:02.105734+0000","last_unstale":"2024-04-24T09:04:24.527426+0000","last_undegraded":"2024-04-24T09:04:24.527426+0000","last_fullsized":"2024-04-24T09:04:24.527426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:15.372520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.695729+0000","last_change":"2024-04-24T09:04:06.139040+0000","last_active":"2024-04-24T09:04:24.695729+0000","last_peered":"2024-04-24T09:04:24.695729+0000","last_clean":"2024-04-24T09:04:24.695729+0000","last_became_active":"2024-04-24T09:04:06.138532+0000","last_became_peered":"2024-04-24T09:04:06.138532+0000","last_unstale":"2024-04-24T09:04:24.695729+0000","last_undegraded":"2024-04-24T09:04:24.695729+0000","last_fullsized":"2024-04-24T09:04:24.695729+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:28.686052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.612766+0000","last_change":"2024-04-24T09:04:01.546859+0000","last_active":"2024-04-24T09:04:32.612766+0000","last_peered":"2024-04-24T09:04:32.612766+0000","last_clean":"2024-04-24T09:04:32.612766+0000","last_became_active":"2024-04-24T09:04:01.546463+0000","last_became_peered":"2024-04-24T09:04:01.546463+0000","last_unstale":"2024-04-24T09:04:32.612766+0000","last_undegraded":"2024-04-24T09:04:32.612766+0000","last_fullsized":"2024-04-24T09:04:32.612766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:31:12.985687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697008+0000","last_change":"2024-04-24T09:04:02.112487+0000","last_active":"2024-04-24T09:04:24.697008+0000","last_peered":"2024-04-24T09:04:24.697008+0000","last_clean":"2024-04-24T09:04:24.697008+0000","last_became_active":"2024-04-24T09:04:02.112078+0000","last_became_peered":"2024-04-24T09:04:02.112078+0000","last_unstale":"2024-04-24T09:04:24.697008+0000","last_undegraded":"2024-04-24T09:04:24.697008+0000","last_fullsized":"2024-04-24T09:04:24.697008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:03:47.352110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.539120+0000","last_change":"2024-04-24T09:04:24.539120+0000","last_active":"2024-04-24T09:04:24.539120+0000","last_peered":"2024-04-24T09:04:24.539120+0000","last_clean":"2024-04-24T09:04:24.539120+0000","last_became_active":"2024-04-24T09:04:24.537048+0000","last_became_peered":"2024-04-24T09:04:24.537048+0000","last_unstale":"2024-04-24T09:04:24.539120+0000","last_undegraded":"2024-04-24T09:04:24.539120+0000","last_fullsized":"2024-04-24T09:04:24.539120+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:29:53.484131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.611836+0000","last_change":"2024-04-24T09:04:01.545444+0000","last_active":"2024-04-24T09:04:32.611836+0000","last_peered":"2024-04-24T09:04:32.611836+0000","last_clean":"2024-04-24T09:04:32.611836+0000","last_became_active":"2024-04-24T09:04:01.543816+0000","last_became_peered":"2024-04-24T09:04:01.543816+0000","last_unstale":"2024-04-24T09:04:32.611836+0000","last_undegraded":"2024-04-24T09:04:32.611836+0000","last_fullsized":"2024-04-24T09:04:32.611836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:02:36.221797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.516038+0000","last_change":"2024-04-24T09:04:02.109085+0000","last_active":"2024-04-24T09:04:24.516038+0000","last_peered":"2024-04-24T09:04:24.516038+0000","last_clean":"2024-04-24T09:04:24.516038+0000","last_became_active":"2024-04-24T09:04:02.108642+0000","last_became_peered":"2024-04-24T09:04:02.108642+0000","last_unstale":"2024-04-24T09:04:24.516038+0000","last_undegraded":"2024-04-24T09:04:24.516038+0000","last_fullsized":"2024-04-24T09:04:24.516038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:27:51.751369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.518972+0000","last_change":"2024-04-24T09:04:06.144940+0000","last_active":"2024-04-24T09:04:24.518972+0000","last_peered":"2024-04-24T09:04:24.518972+0000","last_clean":"2024-04-24T09:04:24.518972+0000","last_became_active":"2024-04-24T09:04:06.144553+0000","last_became_peered":"2024-04-24T09:04:06.144553+0000","last_unstale":"2024-04-24T09:04:24.518972+0000","last_undegraded":"2024-04-24T09:04:24.518972+0000","last_fullsized":"2024-04-24T09:04:24.518972+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:17.455644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.525606+0000","last_change":"2024-04-24T09:04:01.538939+0000","last_active":"2024-04-24T09:04:24.525606+0000","last_peered":"2024-04-24T09:04:24.525606+0000","last_clean":"2024-04-24T09:04:24.525606+0000","last_became_active":"2024-04-24T09:04:01.538541+0000","last_became_peered":"2024-04-24T09:04:01.538541+0000","last_unstale":"2024-04-24T09:04:24.525606+0000","last_undegraded":"2024-04-24T09:04:24.525606+0000","last_fullsized":"2024-04-24T09:04:24.525606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:08.185901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.530693+0000","last_change":"2024-04-24T09:04:02.103844+0000","last_active":"2024-04-24T09:04:24.530693+0000","last_peered":"2024-04-24T09:04:24.530693+0000","last_clean":"2024-04-24T09:04:24.530693+0000","last_became_active":"2024-04-24T09:04:02.103416+0000","last_became_peered":"2024-04-24T09:04:02.103416+0000","last_unstale":"2024-04-24T09:04:24.530693+0000","last_undegraded":"2024-04-24T09:04:24.530693+0000","last_fullsized":"2024-04-24T09:04:24.530693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:57:47.236384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.520023+0000","last_change":"2024-04-24T09:04:06.134838+0000","last_active":"2024-04-24T09:04:24.520023+0000","last_peered":"2024-04-24T09:04:24.520023+0000","last_clean":"2024-04-24T09:04:24.520023+0000","last_became_active":"2024-04-24T09:04:06.134343+0000","last_became_peered":"2024-04-24T09:04:06.134343+0000","last_unstale":"2024-04-24T09:04:24.520023+0000","last_undegraded":"2024-04-24T09:04:24.520023+0000","last_fullsized":"2024-04-24T09:04:24.520023+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:56:55.219012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.695997+0000","last_change":"2024-04-24T09:04:01.557194+0000","last_active":"2024-04-24T09:04:24.695997+0000","last_peered":"2024-04-24T09:04:24.695997+0000","last_clean":"2024-04-24T09:04:24.695997+0000","last_became_active":"2024-04-24T09:04:01.556441+0000","last_became_peered":"2024-04-24T09:04:01.556441+0000","last_unstale":"2024-04-24T09:04:24.695997+0000","last_undegraded":"2024-04-24T09:04:24.695997+0000","last_fullsized":"2024-04-24T09:04:24.695997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:45:50.399096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.526973+0000","last_change":"2024-04-24T09:04:02.108076+0000","last_active":"2024-04-24T09:04:24.526973+0000","last_peered":"2024-04-24T09:04:24.526973+0000","last_clean":"2024-04-24T09:04:24.526973+0000","last_became_active":"2024-04-24T09:04:02.106829+0000","last_became_peered":"2024-04-24T09:04:02.106829+0000","last_unstale":"2024-04-24T09:04:24.526973+0000","last_undegraded":"2024-04-24T09:04:24.526973+0000","last_fullsized":"2024-04-24T09:04:24.526973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:40:23.075828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.521106+0000","last_change":"2024-04-24T09:04:06.142269+0000","last_active":"2024-04-24T09:04:24.521106+0000","last_peered":"2024-04-24T09:04:24.521106+0000","last_clean":"2024-04-24T09:04:24.521106+0000","last_became_active":"2024-04-24T09:04:06.141725+0000","last_became_peered":"2024-04-24T09:04:06.141725+0000","last_unstale":"2024-04-24T09:04:24.521106+0000","last_undegraded":"2024-04-24T09:04:24.521106+0000","last_fullsized":"2024-04-24T09:04:24.521106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:28:17.934148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.615156+0000","last_change":"2024-04-24T09:04:01.553134+0000","last_active":"2024-04-24T09:04:32.615156+0000","last_peered":"2024-04-24T09:04:32.615156+0000","last_clean":"2024-04-24T09:04:32.615156+0000","last_became_active":"2024-04-24T09:04:01.552099+0000","last_became_peered":"2024-04-24T09:04:01.552099+0000","last_unstale":"2024-04-24T09:04:32.615156+0000","last_undegraded":"2024-04-24T09:04:32.615156+0000","last_fullsized":"2024-04-24T09:04:32.615156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:42:36.231405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532146+0000","last_change":"2024-04-24T09:04:02.116641+0000","last_active":"2024-04-24T09:04:24.532146+0000","last_peered":"2024-04-24T09:04:24.532146+0000","last_clean":"2024-04-24T09:04:24.532146+0000","last_became_active":"2024-04-24T09:04:02.115374+0000","last_became_peered":"2024-04-24T09:04:02.115374+0000","last_unstale":"2024-04-24T09:04:24.532146+0000","last_undegraded":"2024-04-24T09:04:24.532146+0000","last_fullsized":"2024-04-24T09:04:24.532146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:34.450990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696698+0000","last_change":"2024-04-24T09:04:06.149968+0000","last_active":"2024-04-24T09:04:24.696698+0000","last_peered":"2024-04-24T09:04:24.696698+0000","last_clean":"2024-04-24T09:04:24.696698+0000","last_became_active":"2024-04-24T09:04:06.149425+0000","last_became_peered":"2024-04-24T09:04:06.149425+0000","last_unstale":"2024-04-24T09:04:24.696698+0000","last_undegraded":"2024-04-24T09:04:24.696698+0000","last_fullsized":"2024-04-24T09:04:24.696698+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:23:53.869004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697126+0000","last_change":"2024-04-24T09:04:01.553822+0000","last_active":"2024-04-24T09:04:24.697126+0000","last_peered":"2024-04-24T09:04:24.697126+0000","last_clean":"2024-04-24T09:04:24.697126+0000","last_became_active":"2024-04-24T09:04:01.551468+0000","last_became_peered":"2024-04-24T09:04:01.551468+0000","last_unstale":"2024-04-24T09:04:24.697126+0000","last_undegraded":"2024-04-24T09:04:24.697126+0000","last_fullsized":"2024-04-24T09:04:24.697126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:56.367584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533471+0000","last_change":"2024-04-24T09:04:02.115472+0000","last_active":"2024-04-24T09:04:24.533471+0000","last_peered":"2024-04-24T09:04:24.533471+0000","last_clean":"2024-04-24T09:04:24.533471+0000","last_became_active":"2024-04-24T09:04:02.114666+0000","last_became_peered":"2024-04-24T09:04:02.114666+0000","last_unstale":"2024-04-24T09:04:24.533471+0000","last_undegraded":"2024-04-24T09:04:24.533471+0000","last_fullsized":"2024-04-24T09:04:24.533471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:28:39.145108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697174+0000","last_change":"2024-04-24T09:04:06.138786+0000","last_active":"2024-04-24T09:04:24.697174+0000","last_peered":"2024-04-24T09:04:24.697174+0000","last_clean":"2024-04-24T09:04:24.697174+0000","last_became_active":"2024-04-24T09:04:06.138366+0000","last_became_peered":"2024-04-24T09:04:06.138366+0000","last_unstale":"2024-04-24T09:04:24.697174+0000","last_undegraded":"2024-04-24T09:04:24.697174+0000","last_fullsized":"2024-04-24T09:04:24.697174+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:03:16.729928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533806+0000","last_change":"2024-04-24T09:04:01.547733+0000","last_active":"2024-04-24T09:04:24.533806+0000","last_peered":"2024-04-24T09:04:24.533806+0000","last_clean":"2024-04-24T09:04:24.533806+0000","last_became_active":"2024-04-24T09:04:01.544839+0000","last_became_peered":"2024-04-24T09:04:01.544839+0000","last_unstale":"2024-04-24T09:04:24.533806+0000","last_undegraded":"2024-04-24T09:04:24.533806+0000","last_fullsized":"2024-04-24T09:04:24.533806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:45:03.847092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531980+0000","last_change":"2024-04-24T09:04:02.108383+0000","last_active":"2024-04-24T09:04:24.531980+0000","last_peered":"2024-04-24T09:04:24.531980+0000","last_clean":"2024-04-24T09:04:24.531980+0000","last_became_active":"2024-04-24T09:04:02.107883+0000","last_became_peered":"2024-04-24T09:04:02.107883+0000","last_unstale":"2024-04-24T09:04:24.531980+0000","last_undegraded":"2024-04-24T09:04:24.531980+0000","last_fullsized":"2024-04-24T09:04:24.531980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:24:44.923708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527772+0000","last_change":"2024-04-24T09:04:06.140868+0000","last_active":"2024-04-24T09:04:24.527772+0000","last_peered":"2024-04-24T09:04:24.527772+0000","last_clean":"2024-04-24T09:04:24.527772+0000","last_became_active":"2024-04-24T09:04:06.139174+0000","last_became_peered":"2024-04-24T09:04:06.139174+0000","last_unstale":"2024-04-24T09:04:24.527772+0000","last_undegraded":"2024-04-24T09:04:24.527772+0000","last_fullsized":"2024-04-24T09:04:24.527772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:17:17.334276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533054+0000","last_change":"2024-04-24T09:04:01.534653+0000","last_active":"2024-04-24T09:04:24.533054+0000","last_peered":"2024-04-24T09:04:24.533054+0000","last_clean":"2024-04-24T09:04:24.533054+0000","last_became_active":"2024-04-24T09:04:01.534312+0000","last_became_peered":"2024-04-24T09:04:01.534312+0000","last_unstale":"2024-04-24T09:04:24.533054+0000","last_undegraded":"2024-04-24T09:04:24.533054+0000","last_fullsized":"2024-04-24T09:04:24.533054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:53:34.301007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533431+0000","last_change":"2024-04-24T09:04:02.102056+0000","last_active":"2024-04-24T09:04:24.533431+0000","last_peered":"2024-04-24T09:04:24.533431+0000","last_clean":"2024-04-24T09:04:24.533431+0000","last_became_active":"2024-04-24T09:04:02.101641+0000","last_became_peered":"2024-04-24T09:04:02.101641+0000","last_unstale":"2024-04-24T09:04:24.533431+0000","last_undegraded":"2024-04-24T09:04:24.533431+0000","last_fullsized":"2024-04-24T09:04:24.533431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:41.654024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.530927+0000","last_change":"2024-04-24T09:04:02.100327+0000","last_active":"2024-04-24T09:04:24.530927+0000","last_peered":"2024-04-24T09:04:24.530927+0000","last_clean":"2024-04-24T09:04:24.530927+0000","last_became_active":"2024-04-24T09:04:02.100028+0000","last_became_peered":"2024-04-24T09:04:02.100028+0000","last_unstale":"2024-04-24T09:04:24.530927+0000","last_undegraded":"2024-04-24T09:04:24.530927+0000","last_fullsized":"2024-04-24T09:04:24.530927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:15:18.728045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696351+0000","last_change":"2024-04-24T09:04:02.101996+0000","last_active":"2024-04-24T09:04:24.696351+0000","last_peered":"2024-04-24T09:04:24.696351+0000","last_clean":"2024-04-24T09:04:24.696351+0000","last_became_active":"2024-04-24T09:04:02.101581+0000","last_became_peered":"2024-04-24T09:04:02.101581+0000","last_unstale":"2024-04-24T09:04:24.696351+0000","last_undegraded":"2024-04-24T09:04:24.696351+0000","last_fullsized":"2024-04-24T09:04:24.696351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:55.487781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696405+0000","last_change":"2024-04-24T09:04:02.107288+0000","last_active":"2024-04-24T09:04:24.696405+0000","last_peered":"2024-04-24T09:04:24.696405+0000","last_clean":"2024-04-24T09:04:24.696405+0000","last_became_active":"2024-04-24T09:04:02.106946+0000","last_became_peered":"2024-04-24T09:04:02.106946+0000","last_unstale":"2024-04-24T09:04:24.696405+0000","last_undegraded":"2024-04-24T09:04:24.696405+0000","last_fullsized":"2024-04-24T09:04:24.696405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:38:56.821285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696457+0000","last_change":"2024-04-24T09:04:02.116829+0000","last_active":"2024-04-24T09:04:24.696457+0000","last_peered":"2024-04-24T09:04:24.696457+0000","last_clean":"2024-04-24T09:04:24.696457+0000","last_became_active":"2024-04-24T09:04:02.116520+0000","last_became_peered":"2024-04-24T09:04:02.116520+0000","last_unstale":"2024-04-24T09:04:24.696457+0000","last_undegraded":"2024-04-24T09:04:24.696457+0000","last_fullsized":"2024-04-24T09:04:24.696457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:26.873826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.700595+0000","last_change":"2024-04-24T09:04:24.700595+0000","last_active":"2024-04-24T09:04:24.700595+0000","last_peered":"2024-04-24T09:04:24.700595+0000","last_clean":"2024-04-24T09:04:24.700595+0000","last_became_active":"2024-04-24T09:04:24.699395+0000","last_became_peered":"2024-04-24T09:04:24.699395+0000","last_unstale":"2024-04-24T09:04:24.700595+0000","last_undegraded":"2024-04-24T09:04:24.700595+0000","last_fullsized":"2024-04-24T09:04:24.700595+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:28.756302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.529561+0000","last_change":"2024-04-24T09:04:02.114087+0000","last_active":"2024-04-24T09:04:24.529561+0000","last_peered":"2024-04-24T09:04:24.529561+0000","last_clean":"2024-04-24T09:04:24.529561+0000","last_became_active":"2024-04-24T09:04:02.112681+0000","last_became_peered":"2024-04-24T09:04:02.112681+0000","last_unstale":"2024-04-24T09:04:24.529561+0000","last_undegraded":"2024-04-24T09:04:24.529561+0000","last_fullsized":"2024-04-24T09:04:24.529561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:50:01.762598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697245+0000","last_change":"2024-04-24T09:04:02.102322+0000","last_active":"2024-04-24T09:04:24.697245+0000","last_peered":"2024-04-24T09:04:24.697245+0000","last_clean":"2024-04-24T09:04:24.697245+0000","last_became_active":"2024-04-24T09:04:02.101963+0000","last_became_peered":"2024-04-24T09:04:02.101963+0000","last_unstale":"2024-04-24T09:04:24.697245+0000","last_undegraded":"2024-04-24T09:04:24.697245+0000","last_fullsized":"2024-04-24T09:04:24.697245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:16.140829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527685+0000","last_change":"2024-04-24T09:04:02.110570+0000","last_active":"2024-04-24T09:04:24.527685+0000","last_peered":"2024-04-24T09:04:24.527685+0000","last_clean":"2024-04-24T09:04:24.527685+0000","last_became_active":"2024-04-24T09:04:02.110119+0000","last_became_peered":"2024-04-24T09:04:02.110119+0000","last_unstale":"2024-04-24T09:04:24.527685+0000","last_undegraded":"2024-04-24T09:04:24.527685+0000","last_fullsized":"2024-04-24T09:04:24.527685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:34.436430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.528620+0000","last_change":"2024-04-24T09:04:02.108292+0000","last_active":"2024-04-24T09:04:24.528620+0000","last_peered":"2024-04-24T09:04:24.528620+0000","last_clean":"2024-04-24T09:04:24.528620+0000","last_became_active":"2024-04-24T09:04:02.107481+0000","last_became_peered":"2024-04-24T09:04:02.107481+0000","last_unstale":"2024-04-24T09:04:24.528620+0000","last_undegraded":"2024-04-24T09:04:24.528620+0000","last_fullsized":"2024-04-24T09:04:24.528620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:49.755293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.525826+0000","last_change":"2024-04-24T09:04:02.102228+0000","last_active":"2024-04-24T09:04:24.525826+0000","last_peered":"2024-04-24T09:04:24.525826+0000","last_clean":"2024-04-24T09:04:24.525826+0000","last_became_active":"2024-04-24T09:04:02.101863+0000","last_became_peered":"2024-04-24T09:04:02.101863+0000","last_unstale":"2024-04-24T09:04:24.525826+0000","last_undegraded":"2024-04-24T09:04:24.525826+0000","last_fullsized":"2024-04-24T09:04:24.525826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:02:47.308116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.698108+0000","last_change":"2024-04-24T09:04:02.106749+0000","last_active":"2024-04-24T09:04:24.698108+0000","last_peered":"2024-04-24T09:04:24.698108+0000","last_clean":"2024-04-24T09:04:24.698108+0000","last_became_active":"2024-04-24T09:04:02.106282+0000","last_became_peered":"2024-04-24T09:04:02.106282+0000","last_unstale":"2024-04-24T09:04:24.698108+0000","last_undegraded":"2024-04-24T09:04:24.698108+0000","last_fullsized":"2024-04-24T09:04:24.698108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:35:59.765230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531528+0000","last_change":"2024-04-24T09:04:02.097825+0000","last_active":"2024-04-24T09:04:24.531528+0000","last_peered":"2024-04-24T09:04:24.531528+0000","last_clean":"2024-04-24T09:04:24.531528+0000","last_became_active":"2024-04-24T09:04:02.097488+0000","last_became_peered":"2024-04-24T09:04:02.097488+0000","last_unstale":"2024-04-24T09:04:24.531528+0000","last_undegraded":"2024-04-24T09:04:24.531528+0000","last_fullsized":"2024-04-24T09:04:24.531528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:05:59.962558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527850+0000","last_change":"2024-04-24T09:04:02.113819+0000","last_active":"2024-04-24T09:04:24.527850+0000","last_peered":"2024-04-24T09:04:24.527850+0000","last_clean":"2024-04-24T09:04:24.527850+0000","last_became_active":"2024-04-24T09:04:02.113463+0000","last_became_peered":"2024-04-24T09:04:02.113463+0000","last_unstale":"2024-04-24T09:04:24.527850+0000","last_undegraded":"2024-04-24T09:04:24.527850+0000","last_fullsized":"2024-04-24T09:04:24.527850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:47:05.109583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7171,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":22,"num_read_kb":18,"num_write":109,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":22230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350998,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":503390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612637,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32796,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712356,"statfs":{"total":95995035648,"available":95961452544,"internally_reserved":0,"allocated":1658880,"data_stored":1097974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808869,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":503592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":507642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32776,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712376,"statfs":{"total":95995035648,"available":95961473024,"internally_reserved":0,"allocated":1638400,"data_stored":1094968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":502898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":506650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":504286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":506684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":502912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":504465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32792,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712360,"statfs":{"total":95995035648,"available":95961456640,"internally_reserved":0,"allocated":1654784,"data_stored":1098748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T09:04:39.123 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T09:04:39.123 INFO:tasks.cephadm:Setting up client nodes... 2024-04-24T09:04:39.123 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-24T09:04:39.610 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:04:39.941 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:39 smithi069 ceph-mon[28170]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:04:39.941 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:39 smithi069 ceph-mon[28170]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:04:39.941 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:39 smithi069 ceph-mon[28170]: pgmap v390: 129 pgs: 129 active+clean; 625 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:39.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:39 smithi130 ceph-mon[31553]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:04:39.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:39 smithi130 ceph-mon[31553]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:04:39.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:39 smithi130 ceph-mon[31553]: pgmap v390: 129 pgs: 129 active+clean; 625 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:40.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:39 smithi081 ceph-mon[30961]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:04:40.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:39 smithi081 ceph-mon[30961]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:04:40.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:39 smithi081 ceph-mon[30961]: pgmap v390: 129 pgs: 129 active+clean; 625 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T09:04:42.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.075+0000 7f682dc00700 1 -- 172.21.15.69:0/900126267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f682810a1c0 msgr2=0x7f682810a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:42.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.075+0000 7f682dc00700 1 --2- 172.21.15.69:0/900126267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f682810a1c0 0x7f682810a620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f68140099c0 tx=0x7f6814009cd0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.075+0000 7f682dc00700 1 -- 172.21.15.69:0/900126267 shutdown_connections 2024-04-24T09:04:42.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.075+0000 7f682dc00700 1 --2- 172.21.15.69:0/900126267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f682810ab60 0x7f6828111410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.075+0000 7f682dc00700 1 --2- 172.21.15.69:0/900126267 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6828108fc0 0x7f68281093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.075+0000 7f682dc00700 1 --2- 172.21.15.69:0/900126267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f682810a1c0 0x7f682810a620 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.075+0000 7f682dc00700 1 -- 172.21.15.69:0/900126267 >> 172.21.15.69:0/900126267 conn(0x7f6828077f90 msgr2=0x7f682807a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:42.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.075+0000 7f682dc00700 1 -- 172.21.15.69:0/900126267 shutdown_connections 2024-04-24T09:04:42.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.075+0000 7f682dc00700 1 -- 172.21.15.69:0/900126267 wait complete. 2024-04-24T09:04:42.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.076+0000 7f682dc00700 1 Processor -- start 2024-04-24T09:04:42.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.076+0000 7f682dc00700 1 -- start start 2024-04-24T09:04:42.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.077+0000 7f682dc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6828108fc0 0x7f682819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.077+0000 7f682dc00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f682810a1c0 0x7f682819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.077+0000 7f682dc00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f682810ab60 0x7f68281a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.077+0000 7f682dc00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6828114a80 con 0x7f6828108fc0 2024-04-24T09:04:42.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.077+0000 7f682dc00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6828114900 con 0x7f682810a1c0 2024-04-24T09:04:42.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.077+0000 7f682dc00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6828114c00 con 0x7f682810ab60 2024-04-24T09:04:42.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.077+0000 7f6827e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6828108fc0 0x7f682819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.077+0000 7f6827e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6828108fc0 0x7f682819f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36570/0 (socket says 172.21.15.69:36570) 2024-04-24T09:04:42.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.077+0000 7f6827e00700 1 -- 172.21.15.69:0/2433050229 learned_addr learned my addr 172.21.15.69:0/2433050229 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:42.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.078+0000 7f6827400700 1 --2- 172.21.15.69:0/2433050229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f682810a1c0 0x7f682819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.078+0000 7f682d200700 1 --2- 172.21.15.69:0/2433050229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f682810ab60 0x7f68281a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.078+0000 7f6827e00700 1 -- 172.21.15.69:0/2433050229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f682810ab60 msgr2=0x7f68281a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:42.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.078+0000 7f6827e00700 1 --2- 172.21.15.69:0/2433050229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f682810ab60 0x7f68281a65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.078+0000 7f6827e00700 1 -- 172.21.15.69:0/2433050229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f682810a1c0 msgr2=0x7f682819f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:42.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.078+0000 7f6827e00700 1 --2- 172.21.15.69:0/2433050229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f682810a1c0 0x7f682819f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.078+0000 7f6827e00700 1 -- 172.21.15.69:0/2433050229 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68281a6c90 con 0x7f6828108fc0 2024-04-24T09:04:42.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.078+0000 7f6827e00700 1 --2- 172.21.15.69:0/2433050229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6828108fc0 0x7f682819f030 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f681800c9a0 tx=0x7f681800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:42.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.079+0000 7f6824c00700 1 -- 172.21.15.69:0/2433050229 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818004220 con 0x7f6828108fc0 2024-04-24T09:04:42.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.079+0000 7f6824c00700 1 -- 172.21.15.69:0/2433050229 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6818007d10 con 0x7f6828108fc0 2024-04-24T09:04:42.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.079+0000 7f682dc00700 1 -- 172.21.15.69:0/2433050229 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68281a6f80 con 0x7f6828108fc0 2024-04-24T09:04:42.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.079+0000 7f6824c00700 1 -- 172.21.15.69:0/2433050229 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818003d00 con 0x7f6828108fc0 2024-04-24T09:04:42.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.079+0000 7f682dc00700 1 -- 172.21.15.69:0/2433050229 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68281a73e0 con 0x7f6828108fc0 2024-04-24T09:04:42.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.080+0000 7f682dc00700 1 -- 172.21.15.69:0/2433050229 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6828064150 con 0x7f6828108fc0 2024-04-24T09:04:42.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.082+0000 7f6824c00700 1 -- 172.21.15.69:0/2433050229 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6818007860 con 0x7f6828108fc0 2024-04-24T09:04:42.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.084+0000 7f6824c00700 1 --2- 172.21.15.69:0/2433050229 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f680c0a3e00 0x7f680c0a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.084+0000 7f6824c00700 1 -- 172.21.15.69:0/2433050229 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f68180d0490 con 0x7f6828108fc0 2024-04-24T09:04:42.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.084+0000 7f6827400700 1 --2- 172.21.15.69:0/2433050229 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f680c0a3e00 0x7f680c0a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.085+0000 7f6827400700 1 --2- 172.21.15.69:0/2433050229 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f680c0a3e00 0x7f680c0a62c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f68140038f0 tx=0x7f6814005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:42.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.088+0000 7f6824c00700 1 -- 172.21.15.69:0/2433050229 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6818014030 con 0x7f6828108fc0 2024-04-24T09:04:42.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.427+0000 7f682dc00700 1 -- 172.21.15.69:0/2433050229 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f682802e230 con 0x7f6828108fc0 2024-04-24T09:04:42.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.432+0000 7f6824c00700 1 -- 172.21.15.69:0/2433050229 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) ==== 170+0+59 (secure 0 0 0) 0x7f681809c950 con 0x7f6828108fc0 2024-04-24T09:04:42.432 INFO:teuthology.orchestra.run.smithi069.stdout:[client.0] 2024-04-24T09:04:42.432 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQAqyyhmDt6ZGRAA187o/7oAKNWjtFnd2+HQWw== 2024-04-24T09:04:42.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.436+0000 7f682dc00700 1 -- 172.21.15.69:0/2433050229 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f680c0a3e00 msgr2=0x7f680c0a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:42.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.436+0000 7f682dc00700 1 --2- 172.21.15.69:0/2433050229 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f680c0a3e00 0x7f680c0a62c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f68140038f0 tx=0x7f6814005fb0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.436+0000 7f682dc00700 1 -- 172.21.15.69:0/2433050229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6828108fc0 msgr2=0x7f682819f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:42.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.436+0000 7f682dc00700 1 --2- 172.21.15.69:0/2433050229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6828108fc0 0x7f682819f030 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f681800c9a0 tx=0x7f681800ccb0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.437+0000 7f682dc00700 1 -- 172.21.15.69:0/2433050229 shutdown_connections 2024-04-24T09:04:42.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.437+0000 7f682dc00700 1 --2- 172.21.15.69:0/2433050229 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f680c0a3e00 0x7f680c0a62c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.437+0000 7f682dc00700 1 --2- 172.21.15.69:0/2433050229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f682810ab60 0x7f68281a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.437+0000 7f682dc00700 1 --2- 172.21.15.69:0/2433050229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6828108fc0 0x7f682819f030 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.437+0000 7f682dc00700 1 --2- 172.21.15.69:0/2433050229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f682810a1c0 0x7f682819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.437+0000 7f682dc00700 1 -- 172.21.15.69:0/2433050229 >> 172.21.15.69:0/2433050229 conn(0x7f6828077f90 msgr2=0x7f682810d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:42.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.437+0000 7f682dc00700 1 -- 172.21.15.69:0/2433050229 shutdown_connections 2024-04-24T09:04:42.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:42.437+0000 7f682dc00700 1 -- 172.21.15.69:0/2433050229 wait complete. 2024-04-24T09:04:42.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:42 smithi130 ceph-mon[31553]: pgmap v391: 129 pgs: 129 active+clean; 625 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.4 KiB/s wr, 13 op/s 2024-04-24T09:04:42.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:42 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2433050229' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T09:04:42.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:42 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2433050229' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T09:04:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:42 smithi081 ceph-mon[30961]: pgmap v391: 129 pgs: 129 active+clean; 625 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.4 KiB/s wr, 13 op/s 2024-04-24T09:04:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:42 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2433050229' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T09:04:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:42 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2433050229' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T09:04:42.831 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:42 smithi069 ceph-mon[28170]: pgmap v391: 129 pgs: 129 active+clean; 625 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.4 KiB/s wr, 13 op/s 2024-04-24T09:04:42.831 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:42 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2433050229' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T09:04:42.831 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:42 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2433050229' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T09:04:43.399 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T09:04:43.400 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-24T09:04:43.400 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T09:04:43.515 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T09:04:43.515 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-24T09:04:43.515 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-24T09:04:43.982 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:04:44.590 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:44 smithi069 ceph-mon[28170]: pgmap v392: 129 pgs: 129 active+clean; 625 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 13 op/s 2024-04-24T09:04:44.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:44 smithi130 ceph-mon[31553]: pgmap v392: 129 pgs: 129 active+clean; 625 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 13 op/s 2024-04-24T09:04:44.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:44 smithi081 ceph-mon[30961]: pgmap v392: 129 pgs: 129 active+clean; 625 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 13 op/s 2024-04-24T09:04:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.190+0000 7fee52600700 1 -- 172.21.15.69:0/1880706872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c074d20 msgr2=0x7fee4c075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.190+0000 7fee52600700 1 --2- 172.21.15.69:0/1880706872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c074d20 0x7fee4c075100 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fee3c009990 tx=0x7fee3c009ca0 comp rx=0 tx=0).stop 2024-04-24T09:04:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.190+0000 7fee52600700 1 -- 172.21.15.69:0/1880706872 shutdown_connections 2024-04-24T09:04:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.190+0000 7fee52600700 1 --2- 172.21.15.69:0/1880706872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fee4c10f550 0x7fee4c111940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.190+0000 7fee52600700 1 --2- 172.21.15.69:0/1880706872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c074d20 0x7fee4c075100 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.190+0000 7fee52600700 1 --2- 172.21.15.69:0/1880706872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee4c075640 0x7fee4c10f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.190+0000 7fee52600700 1 -- 172.21.15.69:0/1880706872 >> 172.21.15.69:0/1880706872 conn(0x7fee4c079c50 msgr2=0x7fee4c07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.191+0000 7fee52600700 1 -- 172.21.15.69:0/1880706872 shutdown_connections 2024-04-24T09:04:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.191+0000 7fee52600700 1 -- 172.21.15.69:0/1880706872 wait complete. 2024-04-24T09:04:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.191+0000 7fee52600700 1 Processor -- start 2024-04-24T09:04:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.192+0000 7fee52600700 1 -- start start 2024-04-24T09:04:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.192+0000 7fee52600700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee4c074d20 0x7fee4c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.192+0000 7fee52600700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fee4c075640 0x7fee4c19b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.192+0000 7fee52600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c10f550 0x7fee4c19fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.192+0000 7fee52600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c114ef0 con 0x7fee4c10f550 2024-04-24T09:04:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.192+0000 7fee52600700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c114d70 con 0x7fee4c074d20 2024-04-24T09:04:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.192+0000 7fee52600700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c115070 con 0x7fee4c075640 2024-04-24T09:04:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.192+0000 7fee4be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c10f550 0x7fee4c19fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.192+0000 7fee4be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c10f550 0x7fee4c19fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48210/0 (socket says 172.21.15.69:48210) 2024-04-24T09:04:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.192+0000 7fee4be00700 1 -- 172.21.15.69:0/1466164095 learned_addr learned my addr 172.21.15.69:0/1466164095 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.193+0000 7fee4b400700 1 --2- 172.21.15.69:0/1466164095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee4c074d20 0x7fee4c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.193+0000 7fee4aa00700 1 --2- 172.21.15.69:0/1466164095 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fee4c075640 0x7fee4c19b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.193+0000 7fee4be00700 1 -- 172.21.15.69:0/1466164095 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fee4c075640 msgr2=0x7fee4c19b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.193+0000 7fee4be00700 1 --2- 172.21.15.69:0/1466164095 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fee4c075640 0x7fee4c19b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.193+0000 7fee4be00700 1 -- 172.21.15.69:0/1466164095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee4c074d20 msgr2=0x7fee4c19b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.193+0000 7fee4be00700 1 --2- 172.21.15.69:0/1466164095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee4c074d20 0x7fee4c19b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.193+0000 7fee4be00700 1 -- 172.21.15.69:0/1466164095 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4c1a02c0 con 0x7fee4c10f550 2024-04-24T09:04:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.193+0000 7fee4be00700 1 --2- 172.21.15.69:0/1466164095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c10f550 0x7fee4c19fb80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fee4400ab20 tx=0x7fee4400aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.194+0000 7fee43e00700 1 -- 172.21.15.69:0/1466164095 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44017020 con 0x7fee4c10f550 2024-04-24T09:04:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.194+0000 7fee43e00700 1 -- 172.21.15.69:0/1466164095 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee44004bb0 con 0x7fee4c10f550 2024-04-24T09:04:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.194+0000 7fee52600700 1 -- 172.21.15.69:0/1466164095 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4c1a05b0 con 0x7fee4c10f550 2024-04-24T09:04:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.194+0000 7fee43e00700 1 -- 172.21.15.69:0/1466164095 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee440052c0 con 0x7fee4c10f550 2024-04-24T09:04:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.194+0000 7fee52600700 1 -- 172.21.15.69:0/1466164095 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee4c1a7e30 con 0x7fee4c10f550 2024-04-24T09:04:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.195+0000 7fee52600700 1 -- 172.21.15.69:0/1466164095 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee4c0697e0 con 0x7fee4c10f550 2024-04-24T09:04:46.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.197+0000 7fee43e00700 1 -- 172.21.15.69:0/1466164095 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fee4400c6a0 con 0x7fee4c10f550 2024-04-24T09:04:46.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.199+0000 7fee43e00700 1 --2- 172.21.15.69:0/1466164095 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fee300a3d50 0x7fee300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:46.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.199+0000 7fee43e00700 1 -- 172.21.15.69:0/1466164095 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fee440cff10 con 0x7fee4c10f550 2024-04-24T09:04:46.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.199+0000 7fee4b400700 1 --2- 172.21.15.69:0/1466164095 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fee300a3d50 0x7fee300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:46.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.200+0000 7fee4b400700 1 --2- 172.21.15.69:0/1466164095 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fee300a3d50 0x7fee300a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fee3c006010 tx=0x7fee3c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:46.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.204+0000 7fee43e00700 1 -- 172.21.15.69:0/1466164095 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee44021020 con 0x7fee4c10f550 2024-04-24T09:04:46.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.488+0000 7fee52600700 1 -- 172.21.15.69:0/1466164095 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fee4c1a80c0 con 0x7fee4c10f550 2024-04-24T09:04:46.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.498+0000 7fee43e00700 1 -- 172.21.15.69:0/1466164095 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264996 (secure 0 0 0) 0x7fee4409c460 con 0x7fee4c10f550 2024-04-24T09:04:46.500 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:46.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.504+0000 7fee52600700 1 -- 172.21.15.69:0/1466164095 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fee300a3d50 msgr2=0x7fee300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:46.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.504+0000 7fee52600700 1 --2- 172.21.15.69:0/1466164095 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fee300a3d50 0x7fee300a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fee3c006010 tx=0x7fee3c01a040 comp rx=0 tx=0).stop 2024-04-24T09:04:46.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.504+0000 7fee52600700 1 -- 172.21.15.69:0/1466164095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c10f550 msgr2=0x7fee4c19fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:46.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.504+0000 7fee52600700 1 --2- 172.21.15.69:0/1466164095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c10f550 0x7fee4c19fb80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fee4400ab20 tx=0x7fee4400aee0 comp rx=0 tx=0).stop 2024-04-24T09:04:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.505+0000 7fee52600700 1 -- 172.21.15.69:0/1466164095 shutdown_connections 2024-04-24T09:04:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.505+0000 7fee52600700 1 --2- 172.21.15.69:0/1466164095 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fee300a3d50 0x7fee300a6210 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.505+0000 7fee52600700 1 --2- 172.21.15.69:0/1466164095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c10f550 0x7fee4c19fb80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.505+0000 7fee52600700 1 --2- 172.21.15.69:0/1466164095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee4c074d20 0x7fee4c19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.505+0000 7fee52600700 1 --2- 172.21.15.69:0/1466164095 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fee4c075640 0x7fee4c19b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.505+0000 7fee52600700 1 -- 172.21.15.69:0/1466164095 >> 172.21.15.69:0/1466164095 conn(0x7fee4c079c50 msgr2=0x7fee4c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.505+0000 7fee52600700 1 -- 172.21.15.69:0/1466164095 shutdown_connections 2024-04-24T09:04:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:46.505+0000 7fee52600700 1 -- 172.21.15.69:0/1466164095 wait complete. 2024-04-24T09:04:46.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:46 smithi081 ceph-mon[30961]: pgmap v393: 129 pgs: 129 active+clean; 633 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-04-24T09:04:46.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:46 smithi069 ceph-mon[28170]: pgmap v393: 129 pgs: 129 active+clean; 633 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-04-24T09:04:46.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:46 smithi130 ceph-mon[31553]: pgmap v393: 129 pgs: 129 active+clean; 633 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-04-24T09:04:47.445 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1098677190},{"type":"v1","addr":"172.21.15.69:6801","nonce":1098677190}]},"active_addr":"172.21.15.69:6801/1098677190","active_change":"2024-04-24T08:54:23.196573+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34109,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.69:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":3008138297}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":3145713060}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":660205566}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":2901345957}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":136471971}]}]} 2024-04-24T09:04:47.451 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-24T09:04:47.451 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-24T09:04:47.451 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T09:04:47.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:47 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1466164095' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T09:04:47.840 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:47 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1466164095' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T09:04:47.941 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:04:47.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:47 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1466164095' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T09:04:48.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:48 smithi081 ceph-mon[30961]: pgmap v394: 129 pgs: 129 active+clean; 633 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-04-24T09:04:48.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:48 smithi069 ceph-mon[28170]: pgmap v394: 129 pgs: 129 active+clean; 633 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-04-24T09:04:48.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:48 smithi130 ceph-mon[31553]: pgmap v394: 129 pgs: 129 active+clean; 633 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-04-24T09:04:50.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.190+0000 7fb734000700 1 -- 172.21.15.69:0/1268446609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb72c06a800 msgr2=0x7fb72c06abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:50.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.190+0000 7fb734000700 1 --2- 172.21.15.69:0/1268446609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb72c06a800 0x7fb72c06abe0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009990 tx=0x7fb71c009ca0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.190+0000 7fb734000700 1 -- 172.21.15.69:0/1268446609 shutdown_connections 2024-04-24T09:04:50.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.190+0000 7fb734000700 1 --2- 172.21.15.69:0/1268446609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb72c10f5a0 0x7fb72c111990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.190+0000 7fb734000700 1 --2- 172.21.15.69:0/1268446609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb72c06a800 0x7fb72c06abe0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.190+0000 7fb734000700 1 --2- 172.21.15.69:0/1268446609 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb72c06b120 0x7fb72c10f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.190+0000 7fb734000700 1 -- 172.21.15.69:0/1268446609 >> 172.21.15.69:0/1268446609 conn(0x7fb72c0789a0 msgr2=0x7fb72c078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:50.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.190+0000 7fb734000700 1 -- 172.21.15.69:0/1268446609 shutdown_connections 2024-04-24T09:04:50.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.190+0000 7fb734000700 1 -- 172.21.15.69:0/1268446609 wait complete. 2024-04-24T09:04:50.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.191+0000 7fb734000700 1 Processor -- start 2024-04-24T09:04:50.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.191+0000 7fb734000700 1 -- start start 2024-04-24T09:04:50.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.192+0000 7fb734000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb72c06a800 0x7fb72c19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.192+0000 7fb734000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb72c06b120 0x7fb72c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.192+0000 7fb734000700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb72c10f5a0 0x7fb72c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.192+0000 7fb734000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c114f00 con 0x7fb72c06a800 2024-04-24T09:04:50.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.192+0000 7fb734000700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c114d80 con 0x7fb72c10f5a0 2024-04-24T09:04:50.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.192+0000 7fb734000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c115080 con 0x7fb72c06b120 2024-04-24T09:04:50.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.192+0000 7fb731600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb72c06a800 0x7fb72c19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.192+0000 7fb731600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb72c06a800 0x7fb72c19b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48228/0 (socket says 172.21.15.69:48228) 2024-04-24T09:04:50.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.192+0000 7fb731600700 1 -- 172.21.15.69:0/2463186084 learned_addr learned my addr 172.21.15.69:0/2463186084 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:50.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.192+0000 7fb730c00700 1 --2- 172.21.15.69:0/2463186084 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb72c06b120 0x7fb72c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.192+0000 7fb732000700 1 --2- 172.21.15.69:0/2463186084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb72c10f5a0 0x7fb72c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.192+0000 7fb731600700 1 -- 172.21.15.69:0/2463186084 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb72c06b120 msgr2=0x7fb72c19b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:50.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.192+0000 7fb731600700 1 --2- 172.21.15.69:0/2463186084 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb72c06b120 0x7fb72c19b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.192+0000 7fb731600700 1 -- 172.21.15.69:0/2463186084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb72c10f5a0 msgr2=0x7fb72c19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:50.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.192+0000 7fb731600700 1 --2- 172.21.15.69:0/2463186084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb72c10f5a0 0x7fb72c19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.192+0000 7fb731600700 1 -- 172.21.15.69:0/2463186084 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c1a0460 con 0x7fb72c06a800 2024-04-24T09:04:50.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.193+0000 7fb731600700 1 --2- 172.21.15.69:0/2463186084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb72c06a800 0x7fb72c19b450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb71c000c00 tx=0x7fb71c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:50.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.193+0000 7fb72a000700 1 -- 172.21.15.69:0/2463186084 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c027070 con 0x7fb72c06a800 2024-04-24T09:04:50.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.193+0000 7fb72a000700 1 -- 172.21.15.69:0/2463186084 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb71c0040c0 con 0x7fb72c06a800 2024-04-24T09:04:50.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.193+0000 7fb734000700 1 -- 172.21.15.69:0/2463186084 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72c1a06f0 con 0x7fb72c06a800 2024-04-24T09:04:50.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.193+0000 7fb72a000700 1 -- 172.21.15.69:0/2463186084 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00f460 con 0x7fb72c06a800 2024-04-24T09:04:50.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.193+0000 7fb734000700 1 -- 172.21.15.69:0/2463186084 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72c1a7fd0 con 0x7fb72c06a800 2024-04-24T09:04:50.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.194+0000 7fb734000700 1 -- 172.21.15.69:0/2463186084 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72c102010 con 0x7fb72c06a800 2024-04-24T09:04:50.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.196+0000 7fb72a000700 1 -- 172.21.15.69:0/2463186084 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb71c02a070 con 0x7fb72c06a800 2024-04-24T09:04:50.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.198+0000 7fb72a000700 1 --2- 172.21.15.69:0/2463186084 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb7100a3cd0 0x7fb7100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.198+0000 7fb72a000700 1 -- 172.21.15.69:0/2463186084 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb71c0d2090 con 0x7fb72c06a800 2024-04-24T09:04:50.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.198+0000 7fb730c00700 1 --2- 172.21.15.69:0/2463186084 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb7100a3cd0 0x7fb7100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.199+0000 7fb730c00700 1 --2- 172.21.15.69:0/2463186084 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb7100a3cd0 0x7fb7100a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb718005fd0 tx=0x7fb71800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:50.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.201+0000 7fb72a000700 1 -- 172.21.15.69:0/2463186084 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb71c031070 con 0x7fb72c06a800 2024-04-24T09:04:50.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.428+0000 7fb734000700 1 -- 172.21.15.69:0/2463186084 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb72c02dfe0 con 0x7fb72c06a800 2024-04-24T09:04:50.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.432+0000 7fb72a000700 1 -- 172.21.15.69:0/2463186084 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24564 (secure 0 0 0) 0x7fb71c013070 con 0x7fb72c06a800 2024-04-24T09:04:50.432 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:50.433 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":92,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:24.497245+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:04:50.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.436+0000 7fb734000700 1 -- 172.21.15.69:0/2463186084 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb7100a3cd0 msgr2=0x7fb7100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:50.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.436+0000 7fb734000700 1 --2- 172.21.15.69:0/2463186084 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb7100a3cd0 0x7fb7100a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb718005fd0 tx=0x7fb71800a560 comp rx=0 tx=0).stop 2024-04-24T09:04:50.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.436+0000 7fb734000700 1 -- 172.21.15.69:0/2463186084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb72c06a800 msgr2=0x7fb72c19b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:50.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.436+0000 7fb734000700 1 --2- 172.21.15.69:0/2463186084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb72c06a800 0x7fb72c19b450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb71c000c00 tx=0x7fb71c0047b0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.437+0000 7fb734000700 1 -- 172.21.15.69:0/2463186084 shutdown_connections 2024-04-24T09:04:50.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.437+0000 7fb734000700 1 --2- 172.21.15.69:0/2463186084 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb7100a3cd0 0x7fb7100a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.437+0000 7fb734000700 1 --2- 172.21.15.69:0/2463186084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb72c10f5a0 0x7fb72c19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.437+0000 7fb734000700 1 --2- 172.21.15.69:0/2463186084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb72c06a800 0x7fb72c19b450 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.437+0000 7fb734000700 1 --2- 172.21.15.69:0/2463186084 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb72c06b120 0x7fb72c19b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.437+0000 7fb734000700 1 -- 172.21.15.69:0/2463186084 >> 172.21.15.69:0/2463186084 conn(0x7fb72c0789a0 msgr2=0x7fb72c110b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:50.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.437+0000 7fb734000700 1 -- 172.21.15.69:0/2463186084 shutdown_connections 2024-04-24T09:04:50.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:50.437+0000 7fb734000700 1 -- 172.21.15.69:0/2463186084 wait complete. 2024-04-24T09:04:50.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:50 smithi069 ceph-mon[28170]: pgmap v395: 129 pgs: 129 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-04-24T09:04:50.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:50 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2463186084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:50.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:50 smithi081 ceph-mon[30961]: pgmap v395: 129 pgs: 129 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-04-24T09:04:50.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:50 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2463186084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:50.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:50 smithi130 ceph-mon[31553]: pgmap v395: 129 pgs: 129 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-04-24T09:04:50.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:50 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2463186084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:51.361 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-24T09:04:51.362 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T09:04:51.808 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:04:52.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:52 smithi069 ceph-mon[28170]: pgmap v396: 129 pgs: 129 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-24T09:04:52.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:52 smithi081 ceph-mon[30961]: pgmap v396: 129 pgs: 129 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-24T09:04:52.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:52 smithi130 ceph-mon[31553]: pgmap v396: 129 pgs: 129 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-24T09:04:54.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.162+0000 7f15e0e00700 1 -- 172.21.15.69:0/1501378423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15dc108a20 msgr2=0x7f15dc108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:54.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.162+0000 7f15e0e00700 1 --2- 172.21.15.69:0/1501378423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15dc108a20 0x7f15dc108e00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00b0a0 tx=0x7f15cc00b3b0 comp rx=0 tx=0).stop 2024-04-24T09:04:54.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.163+0000 7f15e0e00700 1 -- 172.21.15.69:0/1501378423 shutdown_connections 2024-04-24T09:04:54.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.163+0000 7f15e0e00700 1 --2- 172.21.15.69:0/1501378423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f15dc1033e0 0x7f15dc10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:54.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.163+0000 7f15e0e00700 1 --2- 172.21.15.69:0/1501378423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15dc108a20 0x7f15dc108e00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:54.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.163+0000 7f15e0e00700 1 --2- 172.21.15.69:0/1501378423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15dc102a20 0x7f15dc102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:54.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.163+0000 7f15e0e00700 1 -- 172.21.15.69:0/1501378423 >> 172.21.15.69:0/1501378423 conn(0x7f15dc0fe570 msgr2=0x7f15dc100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:54.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.163+0000 7f15e0e00700 1 -- 172.21.15.69:0/1501378423 shutdown_connections 2024-04-24T09:04:54.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.163+0000 7f15e0e00700 1 -- 172.21.15.69:0/1501378423 wait complete. 2024-04-24T09:04:54.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.164+0000 7f15e0e00700 1 Processor -- start 2024-04-24T09:04:54.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.164+0000 7f15e0e00700 1 -- start start 2024-04-24T09:04:54.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.164+0000 7f15e0e00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15dc102a20 0x7f15dc19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:54.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.164+0000 7f15e0e00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f15dc1033e0 0x7f15dc19b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:54.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.164+0000 7f15e0e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15dc108a20 0x7f15dc19fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:54.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.164+0000 7f15e0e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc112c70 con 0x7f15dc108a20 2024-04-24T09:04:54.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.164+0000 7f15e0e00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc112af0 con 0x7f15dc102a20 2024-04-24T09:04:54.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.164+0000 7f15e0e00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc112df0 con 0x7f15dc1033e0 2024-04-24T09:04:54.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.165+0000 7f15dbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15dc108a20 0x7f15dc19fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:54.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.165+0000 7f15dbe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15dc108a20 0x7f15dc19fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44080/0 (socket says 172.21.15.69:44080) 2024-04-24T09:04:54.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.165+0000 7f15dbe00700 1 -- 172.21.15.69:0/578874135 learned_addr learned my addr 172.21.15.69:0/578874135 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:04:54.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.165+0000 7f15db400700 1 --2- 172.21.15.69:0/578874135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15dc102a20 0x7f15dc19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:54.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.165+0000 7f15d2a00700 1 --2- 172.21.15.69:0/578874135 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f15dc1033e0 0x7f15dc19b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:54.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.165+0000 7f15dbe00700 1 -- 172.21.15.69:0/578874135 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f15dc1033e0 msgr2=0x7f15dc19b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:54.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.165+0000 7f15dbe00700 1 --2- 172.21.15.69:0/578874135 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f15dc1033e0 0x7f15dc19b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:54.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.165+0000 7f15dbe00700 1 -- 172.21.15.69:0/578874135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15dc102a20 msgr2=0x7f15dc19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:54.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.165+0000 7f15dbe00700 1 --2- 172.21.15.69:0/578874135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15dc102a20 0x7f15dc19b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:54.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.165+0000 7f15dbe00700 1 -- 172.21.15.69:0/578874135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15dc1a0280 con 0x7f15dc108a20 2024-04-24T09:04:54.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.166+0000 7f15dbe00700 1 --2- 172.21.15.69:0/578874135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15dc108a20 0x7f15dc19fb40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f15d400c970 tx=0x7f15d400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:54.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.166+0000 7f15d8c00700 1 -- 172.21.15.69:0/578874135 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d400ce60 con 0x7f15dc108a20 2024-04-24T09:04:54.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.166+0000 7f15d8c00700 1 -- 172.21.15.69:0/578874135 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15d4004830 con 0x7f15dc108a20 2024-04-24T09:04:54.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.166+0000 7f15e0e00700 1 -- 172.21.15.69:0/578874135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15dc1a0570 con 0x7f15dc108a20 2024-04-24T09:04:54.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.166+0000 7f15d8c00700 1 -- 172.21.15.69:0/578874135 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d40054c0 con 0x7f15dc108a20 2024-04-24T09:04:54.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.166+0000 7f15e0e00700 1 -- 172.21.15.69:0/578874135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15dc1a7df0 con 0x7f15dc108a20 2024-04-24T09:04:54.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.167+0000 7f15e0e00700 1 -- 172.21.15.69:0/578874135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15dc0697e0 con 0x7f15dc108a20 2024-04-24T09:04:54.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.169+0000 7f15d8c00700 1 -- 172.21.15.69:0/578874135 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f15d40049e0 con 0x7f15dc108a20 2024-04-24T09:04:54.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.171+0000 7f15d8c00700 1 --2- 172.21.15.69:0/578874135 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f15c00a3cd0 0x7f15c00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:54.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.171+0000 7f15d8c00700 1 -- 172.21.15.69:0/578874135 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f15d40d0750 con 0x7f15dc108a20 2024-04-24T09:04:54.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.171+0000 7f15db400700 1 --2- 172.21.15.69:0/578874135 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f15c00a3cd0 0x7f15c00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:54.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.172+0000 7f15db400700 1 --2- 172.21.15.69:0/578874135 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f15c00a3cd0 0x7f15c00a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009e60 tx=0x7f15cc00b7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:54.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.175+0000 7f15d8c00700 1 -- 172.21.15.69:0/578874135 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15d4014030 con 0x7f15dc108a20 2024-04-24T09:04:54.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.403+0000 7f15e0e00700 1 -- 172.21.15.69:0/578874135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15dc1a8020 con 0x7f15dc108a20 2024-04-24T09:04:54.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.405+0000 7f15d8c00700 1 -- 172.21.15.69:0/578874135 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24564 (secure 0 0 0) 0x7f15d409cc10 con 0x7f15dc108a20 2024-04-24T09:04:54.405 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:04:54.405 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":92,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:24.497245+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:04:54.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.408+0000 7f15e0e00700 1 -- 172.21.15.69:0/578874135 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f15c00a3cd0 msgr2=0x7f15c00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:54.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.408+0000 7f15e0e00700 1 --2- 172.21.15.69:0/578874135 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f15c00a3cd0 0x7f15c00a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009e60 tx=0x7f15cc00b7f0 comp rx=0 tx=0).stop 2024-04-24T09:04:54.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.409+0000 7f15e0e00700 1 -- 172.21.15.69:0/578874135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15dc108a20 msgr2=0x7f15dc19fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:54.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.409+0000 7f15e0e00700 1 --2- 172.21.15.69:0/578874135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15dc108a20 0x7f15dc19fb40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f15d400c970 tx=0x7f15d400cc80 comp rx=0 tx=0).stop 2024-04-24T09:04:54.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.409+0000 7f15e0e00700 1 -- 172.21.15.69:0/578874135 shutdown_connections 2024-04-24T09:04:54.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.409+0000 7f15e0e00700 1 --2- 172.21.15.69:0/578874135 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f15c00a3cd0 0x7f15c00a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:54.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.409+0000 7f15e0e00700 1 --2- 172.21.15.69:0/578874135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15dc108a20 0x7f15dc19fb40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:54.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.409+0000 7f15e0e00700 1 --2- 172.21.15.69:0/578874135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15dc102a20 0x7f15dc19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:54.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.409+0000 7f15e0e00700 1 --2- 172.21.15.69:0/578874135 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f15dc1033e0 0x7f15dc19b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:54.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.409+0000 7f15e0e00700 1 -- 172.21.15.69:0/578874135 >> 172.21.15.69:0/578874135 conn(0x7f15dc0fe570 msgr2=0x7f15dc10dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:54.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.409+0000 7f15e0e00700 1 -- 172.21.15.69:0/578874135 shutdown_connections 2024-04-24T09:04:54.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:04:54.409+0000 7f15e0e00700 1 -- 172.21.15.69:0/578874135 wait complete. 2024-04-24T09:04:54.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:54 smithi081 ceph-mon[30961]: pgmap v397: 129 pgs: 129 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-24T09:04:54.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:54 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/578874135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:54.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:54 smithi069 ceph-mon[28170]: pgmap v397: 129 pgs: 129 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-24T09:04:54.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:54 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/578874135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:54.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:54 smithi130 ceph-mon[31553]: pgmap v397: 129 pgs: 129 active+clean; 635 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-24T09:04:54.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:54 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/578874135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:04:55.401 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-24T09:04:55.402 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-24T09:04:55.402 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-24T09:04:55.402 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-24T09:04:55.403 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-24T09:04:55.403 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-24T09:04:55.404 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-24T09:04:55.404 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-24T09:04:55.404 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-24T09:04:55.405 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-24T09:04:55.405 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-24T09:04:55.406 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-24T09:04:56.418 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:04:56.430 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:04:56.441 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:04:56.460 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:04:56.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:56 smithi081 ceph-mon[30961]: pgmap v398: 129 pgs: 129 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-24T09:04:56.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:56 smithi069 ceph-mon[28170]: pgmap v398: 129 pgs: 129 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-24T09:04:56.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:56 smithi130 ceph-mon[31553]: pgmap v398: 129 pgs: 129 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-24T09:04:57.283 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:04:57.283 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:04:58.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:04:58 smithi069 ceph-mon[28170]: pgmap v399: 129 pgs: 129 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T09:04:58.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:04:58 smithi081 ceph-mon[30961]: pgmap v399: 129 pgs: 129 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T09:04:58.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:04:58 smithi130 ceph-mon[31553]: pgmap v399: 129 pgs: 129 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T09:04:59.805 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:00.807 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:00 smithi069 ceph-mon[28170]: pgmap v400: 129 pgs: 129 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T09:05:00.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:00 smithi081 ceph-mon[30961]: pgmap v400: 129 pgs: 129 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T09:05:00.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:00 smithi130 ceph-mon[31553]: pgmap v400: 129 pgs: 129 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T09:05:01.042 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:01.204 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:01.206 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:01.318 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:01.551 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:02.656 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:02 smithi069 ceph-mon[28170]: pgmap v401: 129 pgs: 129 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-24T09:05:02.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:02 smithi081 ceph-mon[30961]: pgmap v401: 129 pgs: 129 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-24T09:05:02.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:02 smithi130 ceph-mon[31553]: pgmap v401: 129 pgs: 129 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-24T09:05:03.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.859+0000 7f5bbec00700 1 -- 172.21.15.69:0/3180234367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bb8106050 msgr2=0x7f5bb810dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:03.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.859+0000 7f5bbec00700 1 --2- 172.21.15.69:0/3180234367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bb8106050 0x7f5bb810dc20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000b0d0 tx=0x7f5bb000b3e0 comp rx=0 tx=0).stop 2024-04-24T09:05:03.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.859+0000 7f5bbec00700 1 -- 172.21.15.69:0/3180234367 shutdown_connections 2024-04-24T09:05:03.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.859+0000 7f5bbec00700 1 --2- 172.21.15.69:0/3180234367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bb8106050 0x7f5bb810dc20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:03.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.859+0000 7f5bbec00700 1 --2- 172.21.15.69:0/3180234367 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bb8100f70 0x7f5bb8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:03.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.859+0000 7f5bbec00700 1 --2- 172.21.15.69:0/3180234367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5bb8101920 0x7f5bb8105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:03.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.859+0000 7f5bbec00700 1 -- 172.21.15.69:0/3180234367 >> 172.21.15.69:0/3180234367 conn(0x7f5bb8077bf0 msgr2=0x7f5bb80fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:03.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.859+0000 7f5bbec00700 1 -- 172.21.15.69:0/3180234367 shutdown_connections 2024-04-24T09:05:03.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.859+0000 7f5bbec00700 1 -- 172.21.15.69:0/3180234367 wait complete. 2024-04-24T09:05:03.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.860+0000 7f5bbec00700 1 Processor -- start 2024-04-24T09:05:03.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.860+0000 7f5bbec00700 1 -- start start 2024-04-24T09:05:03.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.861+0000 7f5bbec00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bb8100f70 0x7f5bb819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:03.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.861+0000 7f5bbec00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bb8101920 0x7f5bb819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:03.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.861+0000 7f5bbec00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5bb8106050 0x7f5bb819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:03.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.861+0000 7f5bbec00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb8111c20 con 0x7f5bb8101920 2024-04-24T09:05:03.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.861+0000 7f5bbec00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb8111aa0 con 0x7f5bb8100f70 2024-04-24T09:05:03.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.861+0000 7f5bbec00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb8111da0 con 0x7f5bb8106050 2024-04-24T09:05:03.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.861+0000 7f5bb7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bb8101920 0x7f5bb819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:03.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.861+0000 7f5bb7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bb8101920 0x7f5bb819b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38952/0 (socket says 172.21.15.69:38952) 2024-04-24T09:05:03.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.861+0000 7f5bb7400700 1 -- 172.21.15.69:0/999629333 learned_addr learned my addr 172.21.15.69:0/999629333 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:03.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.861+0000 7f5bb7e00700 1 --2- 172.21.15.69:0/999629333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bb8100f70 0x7f5bb819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:03.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.861+0000 7f5bbcc00700 1 --2- 172.21.15.69:0/999629333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5bb8106050 0x7f5bb819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:03.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.862+0000 7f5bb7400700 1 -- 172.21.15.69:0/999629333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5bb8106050 msgr2=0x7f5bb819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:03.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.862+0000 7f5bb7400700 1 --2- 172.21.15.69:0/999629333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5bb8106050 0x7f5bb819fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:03.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.862+0000 7f5bb7400700 1 -- 172.21.15.69:0/999629333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bb8100f70 msgr2=0x7f5bb819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:03.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.862+0000 7f5bb7400700 1 --2- 172.21.15.69:0/999629333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bb8100f70 0x7f5bb819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:03.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.862+0000 7f5bb7400700 1 -- 172.21.15.69:0/999629333 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb81a0360 con 0x7f5bb8101920 2024-04-24T09:05:03.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.862+0000 7f5bb7400700 1 --2- 172.21.15.69:0/999629333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bb8101920 0x7f5bb819b890 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400b500 tx=0x7f5ba400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:03.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.862+0000 7f5bb4c00700 1 -- 172.21.15.69:0/999629333 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4015020 con 0x7f5bb8101920 2024-04-24T09:05:03.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.862+0000 7f5bb4c00700 1 -- 172.21.15.69:0/999629333 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ba4011ce0 con 0x7f5bb8101920 2024-04-24T09:05:03.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.862+0000 7f5bb4c00700 1 -- 172.21.15.69:0/999629333 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4010d40 con 0x7f5bb8101920 2024-04-24T09:05:03.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.862+0000 7f5bbec00700 1 -- 172.21.15.69:0/999629333 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb81a0650 con 0x7f5bb8101920 2024-04-24T09:05:03.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.863+0000 7f5bbec00700 1 -- 172.21.15.69:0/999629333 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb81a7ed0 con 0x7f5bb8101920 2024-04-24T09:05:03.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.863+0000 7f5bbec00700 1 -- 172.21.15.69:0/999629333 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5bb810f5e0 con 0x7f5bb8101920 2024-04-24T09:05:03.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.866+0000 7f5bb4c00700 1 -- 172.21.15.69:0/999629333 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5ba4018020 con 0x7f5bb8101920 2024-04-24T09:05:03.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.867+0000 7f5bb4c00700 1 --2- 172.21.15.69:0/999629333 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5b9c0a3cd0 0x7f5b9c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:03.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.867+0000 7f5bb4c00700 1 -- 172.21.15.69:0/999629333 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5ba40d0750 con 0x7f5bb8101920 2024-04-24T09:05:03.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.867+0000 7f5bb7e00700 1 --2- 172.21.15.69:0/999629333 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5b9c0a3cd0 0x7f5b9c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:03.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.867+0000 7f5bb4c00700 1 --2- 172.21.15.69:0/999629333 >> [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549] conn(0x7f5b9c0aee30 0x7f5b9c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:03.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.868+0000 7f5bb4c00700 1 -- 172.21.15.69:0/999629333 --> [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ba40044e0 con 0x7f5b9c0aee30 2024-04-24T09:05:03.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.868+0000 7f5bb4c00700 1 -- 172.21.15.69:0/999629333 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f5ba40d0b00 con 0x7f5bb8101920 2024-04-24T09:05:03.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.868+0000 7f5bb7e00700 1 --2- 172.21.15.69:0/999629333 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5b9c0a3cd0 0x7f5b9c0a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8005ea0 tx=0x7f5ba8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:03.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.868+0000 7f5bbcc00700 1 --2- 172.21.15.69:0/999629333 >> [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549] conn(0x7f5b9c0aee30 0x7f5b9c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:03.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.869+0000 7f5bbcc00700 1 --2- 172.21.15.69:0/999629333 >> [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549] conn(0x7f5b9c0aee30 0x7f5b9c0b12d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:03.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.871+0000 7f5bb4c00700 1 -- 172.21.15.69:0/999629333 <== osd.10 v2:172.21.15.81:6824/3563037549 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5ba40044e0 con 0x7f5b9c0aee30 2024-04-24T09:05:03.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.889+0000 7f5bbec00700 1 -- 172.21.15.69:0/999629333 --> [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5bb8102450 con 0x7f5b9c0aee30 2024-04-24T09:05:03.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.890+0000 7f5bb4c00700 1 -- 172.21.15.69:0/999629333 <== osd.10 v2:172.21.15.81:6824/3563037549 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5bb8102450 con 0x7f5b9c0aee30 2024-04-24T09:05:03.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.891+0000 7f5bbec00700 1 -- 172.21.15.69:0/999629333 >> [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549] conn(0x7f5b9c0aee30 msgr2=0x7f5b9c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:03.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.891+0000 7f5bbec00700 1 --2- 172.21.15.69:0/999629333 >> [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549] conn(0x7f5b9c0aee30 0x7f5b9c0b12d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:03.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.891+0000 7f5bbec00700 1 -- 172.21.15.69:0/999629333 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5b9c0a3cd0 msgr2=0x7f5b9c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:03.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.891+0000 7f5bbec00700 1 --2- 172.21.15.69:0/999629333 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5b9c0a3cd0 0x7f5b9c0a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8005ea0 tx=0x7f5ba8005e30 comp rx=0 tx=0).stop 2024-04-24T09:05:03.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.891+0000 7f5bbec00700 1 -- 172.21.15.69:0/999629333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bb8101920 msgr2=0x7f5bb819b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:03.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.891+0000 7f5bbec00700 1 --2- 172.21.15.69:0/999629333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bb8101920 0x7f5bb819b890 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400b500 tx=0x7f5ba400b8c0 comp rx=0 tx=0).stop 2024-04-24T09:05:03.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.891+0000 7f5bbcc00700 1 -- 172.21.15.69:0/999629333 reap_dead start 2024-04-24T09:05:03.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.891+0000 7f5bbec00700 1 -- 172.21.15.69:0/999629333 shutdown_connections 2024-04-24T09:05:03.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.891+0000 7f5bbec00700 1 -- 172.21.15.69:0/999629333 >> 172.21.15.69:0/999629333 conn(0x7f5bb8077bf0 msgr2=0x7f5bb810ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:03.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.891+0000 7f5bbec00700 1 -- 172.21.15.69:0/999629333 shutdown_connections 2024-04-24T09:05:03.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:03.891+0000 7f5bbec00700 1 -- 172.21.15.69:0/999629333 wait complete. 2024-04-24T09:05:04.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:04 smithi069 ceph-mon[28170]: pgmap v402: 129 pgs: 129 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-24T09:05:04.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:04 smithi081 ceph-mon[30961]: pgmap v402: 129 pgs: 129 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-24T09:05:04.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:04 smithi130 ceph-mon[31553]: pgmap v402: 129 pgs: 129 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-24T09:05:06.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.919+0000 7fcef3400700 1 -- 172.21.15.69:0/2627016812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fceec10e460 msgr2=0x7fceec116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:06.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.919+0000 7fcef3400700 1 --2- 172.21.15.69:0/2627016812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fceec10e460 0x7fceec116030 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcee400b0d0 tx=0x7fcee400b3e0 comp rx=0 tx=0).stop 2024-04-24T09:05:06.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.919+0000 7fcef3400700 1 -- 172.21.15.69:0/2627016812 shutdown_connections 2024-04-24T09:05:06.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.919+0000 7fcef3400700 1 --2- 172.21.15.69:0/2627016812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fceec10e460 0x7fceec116030 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:06.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.919+0000 7fcef3400700 1 --2- 172.21.15.69:0/2627016812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fceec109380 0x7fceec109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:06.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.919+0000 7fcef3400700 1 --2- 172.21.15.69:0/2627016812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fceec109d30 0x7fceec10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:06.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.919+0000 7fcef3400700 1 -- 172.21.15.69:0/2627016812 >> 172.21.15.69:0/2627016812 conn(0x7fceec0762c0 msgr2=0x7fceec107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:06.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.919+0000 7fcef3400700 1 -- 172.21.15.69:0/2627016812 shutdown_connections 2024-04-24T09:05:06.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.919+0000 7fcef3400700 1 -- 172.21.15.69:0/2627016812 wait complete. 2024-04-24T09:05:06.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.920+0000 7fcef3400700 1 Processor -- start 2024-04-24T09:05:06.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.920+0000 7fcef3400700 1 -- start start 2024-04-24T09:05:06.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.920+0000 7fcef3400700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fceec109380 0x7fceec1a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:06.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.920+0000 7fcef3400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fceec109d30 0x7fceec1a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:06.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.920+0000 7fcef3400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fceec10e460 0x7fceec19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:06.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.920+0000 7fcef3400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fceec119350 con 0x7fceec109d30 2024-04-24T09:05:06.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.920+0000 7fcef3400700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fceec1191d0 con 0x7fceec109380 2024-04-24T09:05:06.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.920+0000 7fcef3400700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fceec1194d0 con 0x7fceec10e460 2024-04-24T09:05:06.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.921+0000 7fceebe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fceec109d30 0x7fceec1a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:06.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.921+0000 7fceebe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fceec109d30 0x7fceec1a5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38976/0 (socket says 172.21.15.69:38976) 2024-04-24T09:05:06.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.921+0000 7fceebe00700 1 -- 172.21.15.69:0/1793563857 learned_addr learned my addr 172.21.15.69:0/1793563857 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:06.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.921+0000 7fcef0a00700 1 --2- 172.21.15.69:0/1793563857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fceec109380 0x7fceec1a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:06.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.921+0000 7fcef1400700 1 --2- 172.21.15.69:0/1793563857 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fceec10e460 0x7fceec19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:06.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.921+0000 7fceebe00700 1 -- 172.21.15.69:0/1793563857 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fceec10e460 msgr2=0x7fceec19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:06.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.921+0000 7fceebe00700 1 --2- 172.21.15.69:0/1793563857 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fceec10e460 0x7fceec19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:06.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.921+0000 7fceebe00700 1 -- 172.21.15.69:0/1793563857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fceec109380 msgr2=0x7fceec1a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:06.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.921+0000 7fceebe00700 1 --2- 172.21.15.69:0/1793563857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fceec109380 0x7fceec1a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:06.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.921+0000 7fceebe00700 1 -- 172.21.15.69:0/1793563857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceec19fd10 con 0x7fceec109d30 2024-04-24T09:05:06.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.921+0000 7fcef1400700 1 --2- 172.21.15.69:0/1793563857 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fceec10e460 0x7fceec19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:06.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.921+0000 7fcef0a00700 1 --2- 172.21.15.69:0/1793563857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fceec109380 0x7fceec1a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:06.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.922+0000 7fceebe00700 1 --2- 172.21.15.69:0/1793563857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fceec109d30 0x7fceec1a5930 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fced800c8f0 tx=0x7fced800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:06.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.922+0000 7fcee9600700 1 -- 172.21.15.69:0/1793563857 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8007b00 con 0x7fceec109d30 2024-04-24T09:05:06.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.922+0000 7fcee9600700 1 -- 172.21.15.69:0/1793563857 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fced8007ca0 con 0x7fceec109d30 2024-04-24T09:05:06.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.922+0000 7fcee9600700 1 -- 172.21.15.69:0/1793563857 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8018660 con 0x7fceec109d30 2024-04-24T09:05:06.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.922+0000 7fcef3400700 1 -- 172.21.15.69:0/1793563857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceec1a0000 con 0x7fceec109d30 2024-04-24T09:05:06.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.922+0000 7fcef3400700 1 -- 172.21.15.69:0/1793563857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceec1ac2c0 con 0x7fceec109d30 2024-04-24T09:05:06.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.923+0000 7fcef3400700 1 -- 172.21.15.69:0/1793563857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fceec063240 con 0x7fceec109d30 2024-04-24T09:05:06.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.925+0000 7fcee9600700 1 -- 172.21.15.69:0/1793563857 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fced800f450 con 0x7fceec109d30 2024-04-24T09:05:06.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.927+0000 7fcee9600700 1 --2- 172.21.15.69:0/1793563857 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fced40a3d50 0x7fced40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:06.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.927+0000 7fcee9600700 1 -- 172.21.15.69:0/1793563857 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fced80d0b00 con 0x7fceec109d30 2024-04-24T09:05:06.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.927+0000 7fcef0a00700 1 --2- 172.21.15.69:0/1793563857 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fced40a3d50 0x7fced40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:06.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.927+0000 7fcee9600700 1 --2- 172.21.15.69:0/1793563857 >> [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990] conn(0x7fced40aeeb0 0x7fced40b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:06.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.927+0000 7fcee9600700 1 -- 172.21.15.69:0/1793563857 --> [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fced8004520 con 0x7fced40aeeb0 2024-04-24T09:05:06.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.927+0000 7fcee9600700 1 -- 172.21.15.69:0/1793563857 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fced80d0eb0 con 0x7fceec109d30 2024-04-24T09:05:06.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.928+0000 7fcef0a00700 1 --2- 172.21.15.69:0/1793563857 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fced40a3d50 0x7fced40a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcee0005950 tx=0x7fcee00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:06.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.928+0000 7fcef1400700 1 --2- 172.21.15.69:0/1793563857 >> [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990] conn(0x7fced40aeeb0 0x7fced40b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:06.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.929+0000 7fcef1400700 1 --2- 172.21.15.69:0/1793563857 >> [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990] conn(0x7fced40aeeb0 0x7fced40b1350 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:06.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.931+0000 7fcee9600700 1 -- 172.21.15.69:0/1793563857 <== osd.4 v2:172.21.15.81:6808/575869990 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fced8004520 con 0x7fced40aeeb0 2024-04-24T09:05:06.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.950+0000 7fcef3400700 1 -- 172.21.15.69:0/1793563857 --> [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fceec063710 con 0x7fced40aeeb0 2024-04-24T09:05:06.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.951+0000 7fcee9600700 1 -- 172.21.15.69:0/1793563857 <== osd.4 v2:172.21.15.81:6808/575869990 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fceec063710 con 0x7fced40aeeb0 2024-04-24T09:05:06.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.952+0000 7fcef3400700 1 -- 172.21.15.69:0/1793563857 >> [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990] conn(0x7fced40aeeb0 msgr2=0x7fced40b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:06.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.952+0000 7fcef3400700 1 --2- 172.21.15.69:0/1793563857 >> [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990] conn(0x7fced40aeeb0 0x7fced40b1350 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:06.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.952+0000 7fcef3400700 1 -- 172.21.15.69:0/1793563857 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fced40a3d50 msgr2=0x7fced40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:06.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.952+0000 7fcef3400700 1 --2- 172.21.15.69:0/1793563857 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fced40a3d50 0x7fced40a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcee0005950 tx=0x7fcee00058e0 comp rx=0 tx=0).stop 2024-04-24T09:05:06.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.952+0000 7fcef3400700 1 -- 172.21.15.69:0/1793563857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fceec109d30 msgr2=0x7fceec1a5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:06.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.952+0000 7fcef3400700 1 --2- 172.21.15.69:0/1793563857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fceec109d30 0x7fceec1a5930 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fced800c8f0 tx=0x7fced800ccb0 comp rx=0 tx=0).stop 2024-04-24T09:05:06.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.952+0000 7fcef1400700 1 -- 172.21.15.69:0/1793563857 reap_dead start 2024-04-24T09:05:06.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.952+0000 7fcef3400700 1 -- 172.21.15.69:0/1793563857 shutdown_connections 2024-04-24T09:05:06.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.952+0000 7fcef3400700 1 -- 172.21.15.69:0/1793563857 >> 172.21.15.69:0/1793563857 conn(0x7fceec0762c0 msgr2=0x7fceec076800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:06.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.952+0000 7fcef3400700 1 -- 172.21.15.69:0/1793563857 shutdown_connections 2024-04-24T09:05:06.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:05.952+0000 7fcef3400700 1 -- 172.21.15.69:0/1793563857 wait complete. 2024-04-24T09:05:06.749 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:06 smithi069 ceph-mon[28170]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-24T09:05:06.749 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:06 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:05:06.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:06 smithi081 ceph-mon[30961]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-24T09:05:06.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:06 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:05:06.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:06 smithi130 ceph-mon[31553]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-24T09:05:06.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:06 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:05:08.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:07 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:08.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:07 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:08.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:07 smithi069 ceph-mon[28170]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-24T09:05:08.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:07 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:08.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:07 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:08.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:07 smithi130 ceph-mon[31553]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-24T09:05:08.231 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:07 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:08.231 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:07 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:08.231 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:07 smithi081 ceph-mon[30961]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-24T09:05:10.301 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:09 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:09 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:09 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:09 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:09 smithi069 ceph-mon[28170]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-24T09:05:10.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:09 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:09 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:09 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:09 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:09 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:09 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:09 smithi081 ceph-mon[30961]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-24T09:05:10.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:09 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:09 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.367 INFO:teuthology.orchestra.run.smithi069.stdout:313532612644 2024-04-24T09:05:10.368 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-24T09:05:10.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:09 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:09 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:09 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:09 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:09 smithi130 ceph-mon[31553]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-24T09:05:10.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:09 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:10.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:09 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:12.765 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:12 smithi069 ceph-mon[28170]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-24T09:05:12.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:12 smithi081 ceph-mon[30961]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-24T09:05:12.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:12 smithi130 ceph-mon[31553]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-24T09:05:13.840 INFO:teuthology.orchestra.run.smithi069.stdout:141733920846 2024-04-24T09:05:13.840 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-24T09:05:14.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.571+0000 7ff3fd800700 1 -- 172.21.15.69:0/3865974066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f8102b40 msgr2=0x7ff3f8102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.571+0000 7ff3fd800700 1 --2- 172.21.15.69:0/3865974066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f8102b40 0x7ff3f8102fc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009990 tx=0x7ff3e0009ca0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.571+0000 7ff3fd800700 1 -- 172.21.15.69:0/3865974066 shutdown_connections 2024-04-24T09:05:14.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.571+0000 7ff3fd800700 1 --2- 172.21.15.69:0/3865974066 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f8103500 0x7ff3f810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.571+0000 7ff3fd800700 1 --2- 172.21.15.69:0/3865974066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f8108b90 0x7ff3f8108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.571+0000 7ff3fd800700 1 --2- 172.21.15.69:0/3865974066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f8102b40 0x7ff3f8102fc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.571+0000 7ff3fd800700 1 -- 172.21.15.69:0/3865974066 >> 172.21.15.69:0/3865974066 conn(0x7ff3f8079c50 msgr2=0x7ff3f807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.571+0000 7ff3fd800700 1 -- 172.21.15.69:0/3865974066 shutdown_connections 2024-04-24T09:05:14.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.571+0000 7ff3fd800700 1 -- 172.21.15.69:0/3865974066 wait complete. 2024-04-24T09:05:14.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.572+0000 7ff3fd800700 1 Processor -- start 2024-04-24T09:05:14.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.573+0000 7ff3fd800700 1 -- start start 2024-04-24T09:05:14.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.573+0000 7ff3fd800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f8102b40 0x7ff3f819b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.573+0000 7ff3fd800700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f8103500 0x7ff3f819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.573+0000 7ff3fd800700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f8108b90 0x7ff3f819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.573+0000 7ff3fd800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f8112d00 con 0x7ff3f8102b40 2024-04-24T09:05:14.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.573+0000 7ff3fd800700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f8112b80 con 0x7ff3f8108b90 2024-04-24T09:05:14.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.573+0000 7ff3fd800700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f8112e80 con 0x7ff3f8103500 2024-04-24T09:05:14.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.573+0000 7ff3f6a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f8102b40 0x7ff3f819b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.574+0000 7ff3f6a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f8102b40 0x7ff3f819b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45036/0 (socket says 172.21.15.69:45036) 2024-04-24T09:05:14.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.574+0000 7ff3f6a00700 1 -- 172.21.15.69:0/1487626518 learned_addr learned my addr 172.21.15.69:0/1487626518 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:14.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.574+0000 7ff3ee000700 1 --2- 172.21.15.69:0/1487626518 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f8103500 0x7ff3f819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.574+0000 7ff3f7400700 1 --2- 172.21.15.69:0/1487626518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f8108b90 0x7ff3f819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.574+0000 7ff3f6a00700 1 -- 172.21.15.69:0/1487626518 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f8103500 msgr2=0x7ff3f819ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.574+0000 7ff3f6a00700 1 --2- 172.21.15.69:0/1487626518 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f8103500 0x7ff3f819ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.574+0000 7ff3f6a00700 1 -- 172.21.15.69:0/1487626518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f8108b90 msgr2=0x7ff3f819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.574+0000 7ff3f6a00700 1 --2- 172.21.15.69:0/1487626518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f8108b90 0x7ff3f819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.574+0000 7ff3f6a00700 1 -- 172.21.15.69:0/1487626518 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f81a04e0 con 0x7ff3f8102b40 2024-04-24T09:05:14.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.574+0000 7ff3f7400700 1 --2- 172.21.15.69:0/1487626518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f8108b90 0x7ff3f819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:14.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.575+0000 7ff3f6a00700 1 --2- 172.21.15.69:0/1487626518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f8102b40 0x7ff3f819b4d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800d980 tx=0x7ff3e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.575+0000 7ff3efe00700 1 -- 172.21.15.69:0/1487626518 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8016070 con 0x7ff3f8102b40 2024-04-24T09:05:14.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.575+0000 7ff3fd800700 1 -- 172.21.15.69:0/1487626518 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f81a07d0 con 0x7ff3f8102b40 2024-04-24T09:05:14.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.575+0000 7ff3efe00700 1 -- 172.21.15.69:0/1487626518 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3e8004830 con 0x7ff3f8102b40 2024-04-24T09:05:14.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.575+0000 7ff3efe00700 1 -- 172.21.15.69:0/1487626518 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8005450 con 0x7ff3f8102b40 2024-04-24T09:05:14.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.575+0000 7ff3fd800700 1 -- 172.21.15.69:0/1487626518 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f81a8050 con 0x7ff3f8102b40 2024-04-24T09:05:14.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.576+0000 7ff3fd800700 1 -- 172.21.15.69:0/1487626518 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff3f810f9a0 con 0x7ff3f8102b40 2024-04-24T09:05:14.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.578+0000 7ff3efe00700 1 -- 172.21.15.69:0/1487626518 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff3e80055f0 con 0x7ff3f8102b40 2024-04-24T09:05:14.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.581+0000 7ff3efe00700 1 --2- 172.21.15.69:0/1487626518 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff3dc0a3cd0 0x7ff3dc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.581+0000 7ff3efe00700 1 -- 172.21.15.69:0/1487626518 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff3e80d0d30 con 0x7ff3f8102b40 2024-04-24T09:05:14.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.581+0000 7ff3ee000700 1 --2- 172.21.15.69:0/1487626518 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff3dc0a3cd0 0x7ff3dc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.581+0000 7ff3efe00700 1 --2- 172.21.15.69:0/1487626518 >> [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143] conn(0x7ff3dc0aee30 0x7ff3dc0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.581+0000 7ff3efe00700 1 -- 172.21.15.69:0/1487626518 --> [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff3e8015ec0 con 0x7ff3dc0aee30 2024-04-24T09:05:14.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.581+0000 7ff3efe00700 1 -- 172.21.15.69:0/1487626518 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7ff3e80d1180 con 0x7ff3f8102b40 2024-04-24T09:05:14.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.582+0000 7ff3ee000700 1 --2- 172.21.15.69:0/1487626518 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff3dc0a3cd0 0x7ff3dc0a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8101300 tx=0x7ff3e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.582+0000 7ff3f7400700 1 --2- 172.21.15.69:0/1487626518 >> [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143] conn(0x7ff3dc0aee30 0x7ff3dc0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.583+0000 7ff3f7400700 1 --2- 172.21.15.69:0/1487626518 >> [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143] conn(0x7ff3dc0aee30 0x7ff3dc0b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.586+0000 7ff3efe00700 1 -- 172.21.15.69:0/1487626518 <== osd.8 v2:172.21.15.130:6816/3037039143 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff3e8015ec0 con 0x7ff3dc0aee30 2024-04-24T09:05:14.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.604+0000 7ff3fd800700 1 -- 172.21.15.69:0/1487626518 --> [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff3f81041f0 con 0x7ff3dc0aee30 2024-04-24T09:05:14.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.605+0000 7ff3efe00700 1 -- 172.21.15.69:0/1487626518 <== osd.8 v2:172.21.15.130:6816/3037039143 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff3f81041f0 con 0x7ff3dc0aee30 2024-04-24T09:05:14.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.605+0000 7ff3fd800700 1 -- 172.21.15.69:0/1487626518 >> [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143] conn(0x7ff3dc0aee30 msgr2=0x7ff3dc0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.605+0000 7ff3fd800700 1 --2- 172.21.15.69:0/1487626518 >> [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143] conn(0x7ff3dc0aee30 0x7ff3dc0b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.605+0000 7ff3fd800700 1 -- 172.21.15.69:0/1487626518 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff3dc0a3cd0 msgr2=0x7ff3dc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.605+0000 7ff3fd800700 1 --2- 172.21.15.69:0/1487626518 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff3dc0a3cd0 0x7ff3dc0a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8101300 tx=0x7ff3e0005fd0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.605+0000 7ff3fd800700 1 -- 172.21.15.69:0/1487626518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f8102b40 msgr2=0x7ff3f819b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.605+0000 7ff3fd800700 1 --2- 172.21.15.69:0/1487626518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f8102b40 0x7ff3f819b4d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800d980 tx=0x7ff3e800dc90 comp rx=0 tx=0).stop 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.605+0000 7ff3f7400700 1 -- 172.21.15.69:0/1487626518 reap_dead start 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.606+0000 7ff3fd800700 1 -- 172.21.15.69:0/1487626518 shutdown_connections 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.606+0000 7ff3fd800700 1 -- 172.21.15.69:0/1487626518 >> 172.21.15.69:0/1487626518 conn(0x7ff3f8079c50 msgr2=0x7ff3f810dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.606+0000 7ff3fd800700 1 -- 172.21.15.69:0/1487626518 shutdown_connections 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:14.606+0000 7ff3fd800700 1 -- 172.21.15.69:0/1487626518 wait complete. 2024-04-24T09:05:14.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.156+0000 7fb394e00700 1 -- 172.21.15.69:0/1276357930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb390109d30 msgr2=0x7fb39010dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.156+0000 7fb394e00700 1 --2- 172.21.15.69:0/1276357930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb390109d30 0x7fb39010dd80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0099c0 tx=0x7fb37c009cd0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.156+0000 7fb394e00700 1 -- 172.21.15.69:0/1276357930 shutdown_connections 2024-04-24T09:05:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.156+0000 7fb394e00700 1 --2- 172.21.15.69:0/1276357930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb39010e460 0x7fb390116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.156+0000 7fb394e00700 1 --2- 172.21.15.69:0/1276357930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb390109380 0x7fb390109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.156+0000 7fb394e00700 1 --2- 172.21.15.69:0/1276357930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb390109d30 0x7fb39010dd80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.156+0000 7fb394e00700 1 -- 172.21.15.69:0/1276357930 >> 172.21.15.69:0/1276357930 conn(0x7fb3900762c0 msgr2=0x7fb390107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.156+0000 7fb394e00700 1 -- 172.21.15.69:0/1276357930 shutdown_connections 2024-04-24T09:05:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.156+0000 7fb394e00700 1 -- 172.21.15.69:0/1276357930 wait complete. 2024-04-24T09:05:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.157+0000 7fb394e00700 1 Processor -- start 2024-04-24T09:05:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.157+0000 7fb394e00700 1 -- start start 2024-04-24T09:05:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.158+0000 7fb394e00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb390109380 0x7fb3901a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.158+0000 7fb394e00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb390109d30 0x7fb3901a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.158+0000 7fb394e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39010e460 0x7fb39019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.158+0000 7fb394e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb3901192f0 con 0x7fb39010e460 2024-04-24T09:05:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.158+0000 7fb394e00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb390119170 con 0x7fb390109380 2024-04-24T09:05:14.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.158+0000 7fb394e00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb390119470 con 0x7fb390109d30 2024-04-24T09:05:14.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.158+0000 7fb38e800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39010e460 0x7fb39019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.158+0000 7fb38e800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39010e460 0x7fb39019f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39016/0 (socket says 172.21.15.69:39016) 2024-04-24T09:05:14.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.158+0000 7fb38e800700 1 -- 172.21.15.69:0/2910615924 learned_addr learned my addr 172.21.15.69:0/2910615924 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:14.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.158+0000 7fb38de00700 1 --2- 172.21.15.69:0/2910615924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb390109380 0x7fb3901a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.158+0000 7fb38d400700 1 --2- 172.21.15.69:0/2910615924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb390109d30 0x7fb3901a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.159+0000 7fb38e800700 1 -- 172.21.15.69:0/2910615924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb390109d30 msgr2=0x7fb3901a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.159+0000 7fb38e800700 1 --2- 172.21.15.69:0/2910615924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb390109d30 0x7fb3901a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.159+0000 7fb38e800700 1 -- 172.21.15.69:0/2910615924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb390109380 msgr2=0x7fb3901a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.159+0000 7fb38e800700 1 --2- 172.21.15.69:0/2910615924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb390109380 0x7fb3901a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.159+0000 7fb38e800700 1 -- 172.21.15.69:0/2910615924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39019ff30 con 0x7fb39010e460 2024-04-24T09:05:14.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.159+0000 7fb38d400700 1 --2- 172.21.15.69:0/2910615924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb390109d30 0x7fb3901a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:14.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.159+0000 7fb38e800700 1 --2- 172.21.15.69:0/2910615924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39010e460 0x7fb39019f650 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb388007ee0 tx=0x7fb38800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.159+0000 7fb386a00700 1 -- 172.21.15.69:0/2910615924 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38800dbd0 con 0x7fb39010e460 2024-04-24T09:05:14.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.159+0000 7fb394e00700 1 -- 172.21.15.69:0/2910615924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3901a0220 con 0x7fb39010e460 2024-04-24T09:05:14.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.159+0000 7fb386a00700 1 -- 172.21.15.69:0/2910615924 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb38800dd70 con 0x7fb39010e460 2024-04-24T09:05:14.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.159+0000 7fb386a00700 1 -- 172.21.15.69:0/2910615924 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388005a20 con 0x7fb39010e460 2024-04-24T09:05:14.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.159+0000 7fb394e00700 1 -- 172.21.15.69:0/2910615924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3901ac500 con 0x7fb39010e460 2024-04-24T09:05:14.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.160+0000 7fb394e00700 1 -- 172.21.15.69:0/2910615924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb3900631c0 con 0x7fb39010e460 2024-04-24T09:05:14.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.163+0000 7fb386a00700 1 -- 172.21.15.69:0/2910615924 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb38801a020 con 0x7fb39010e460 2024-04-24T09:05:14.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.164+0000 7fb386a00700 1 --2- 172.21.15.69:0/2910615924 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb3740a3cd0 0x7fb3740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.164+0000 7fb386a00700 1 -- 172.21.15.69:0/2910615924 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb3880d0790 con 0x7fb39010e460 2024-04-24T09:05:14.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.164+0000 7fb38de00700 1 --2- 172.21.15.69:0/2910615924 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb3740a3cd0 0x7fb3740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.164+0000 7fb386a00700 1 --2- 172.21.15.69:0/2910615924 >> [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184] conn(0x7fb3740aee30 0x7fb3740b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.164+0000 7fb386a00700 1 -- 172.21.15.69:0/2910615924 --> [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb3880020b0 con 0x7fb3740aee30 2024-04-24T09:05:14.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.164+0000 7fb386a00700 1 -- 172.21.15.69:0/2910615924 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fb3880d0b40 con 0x7fb39010e460 2024-04-24T09:05:14.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.165+0000 7fb38de00700 1 --2- 172.21.15.69:0/2910615924 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb3740a3cd0 0x7fb3740a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb380005950 tx=0x7fb3800058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.165+0000 7fb38d400700 1 --2- 172.21.15.69:0/2910615924 >> [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184] conn(0x7fb3740aee30 0x7fb3740b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.166+0000 7fb38d400700 1 --2- 172.21.15.69:0/2910615924 >> [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184] conn(0x7fb3740aee30 0x7fb3740b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.168+0000 7fb386a00700 1 -- 172.21.15.69:0/2910615924 <== osd.5 v2:172.21.15.130:6808/1208873184 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3880020b0 con 0x7fb3740aee30 2024-04-24T09:05:14.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.187+0000 7fb394e00700 1 -- 172.21.15.69:0/2910615924 --> [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb390063640 con 0x7fb3740aee30 2024-04-24T09:05:14.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.188+0000 7fb386a00700 1 -- 172.21.15.69:0/2910615924 <== osd.5 v2:172.21.15.130:6808/1208873184 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb390063640 con 0x7fb3740aee30 2024-04-24T09:05:14.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.188+0000 7fb394e00700 1 -- 172.21.15.69:0/2910615924 >> [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184] conn(0x7fb3740aee30 msgr2=0x7fb3740b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.188+0000 7fb394e00700 1 --2- 172.21.15.69:0/2910615924 >> [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184] conn(0x7fb3740aee30 0x7fb3740b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.189+0000 7fb394e00700 1 -- 172.21.15.69:0/2910615924 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb3740a3cd0 msgr2=0x7fb3740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.189+0000 7fb394e00700 1 --2- 172.21.15.69:0/2910615924 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb3740a3cd0 0x7fb3740a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb380005950 tx=0x7fb3800058e0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.189+0000 7fb394e00700 1 -- 172.21.15.69:0/2910615924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39010e460 msgr2=0x7fb39019f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.189+0000 7fb394e00700 1 --2- 172.21.15.69:0/2910615924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39010e460 0x7fb39019f650 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb388007ee0 tx=0x7fb38800d3b0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.189+0000 7fb38e800700 1 -- 172.21.15.69:0/2910615924 reap_dead start 2024-04-24T09:05:14.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.189+0000 7fb394e00700 1 -- 172.21.15.69:0/2910615924 shutdown_connections 2024-04-24T09:05:14.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.189+0000 7fb394e00700 1 -- 172.21.15.69:0/2910615924 >> 172.21.15.69:0/2910615924 conn(0x7fb3900762c0 msgr2=0x7fb390076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.189+0000 7fb394e00700 1 -- 172.21.15.69:0/2910615924 shutdown_connections 2024-04-24T09:05:14.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:12.189+0000 7fb394e00700 1 -- 172.21.15.69:0/2910615924 wait complete. 2024-04-24T09:05:14.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:14 smithi081 ceph-mon[30961]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-24T09:05:14.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:14 smithi069 ceph-mon[28170]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-24T09:05:14.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:14 smithi130 ceph-mon[31553]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-24T09:05:14.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.017+0000 7fd075e00700 1 -- 172.21.15.69:0/1966390500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd070109d30 msgr2=0x7fd07010dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.017+0000 7fd075e00700 1 --2- 172.21.15.69:0/1966390500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd070109d30 0x7fd07010dd80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0099c0 tx=0x7fd05c009cd0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.017+0000 7fd075e00700 1 -- 172.21.15.69:0/1966390500 shutdown_connections 2024-04-24T09:05:14.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.017+0000 7fd075e00700 1 --2- 172.21.15.69:0/1966390500 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd07010e460 0x7fd070116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.017+0000 7fd075e00700 1 --2- 172.21.15.69:0/1966390500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd070109380 0x7fd070109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.017+0000 7fd075e00700 1 --2- 172.21.15.69:0/1966390500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd070109d30 0x7fd07010dd80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.017+0000 7fd075e00700 1 -- 172.21.15.69:0/1966390500 >> 172.21.15.69:0/1966390500 conn(0x7fd0700762c0 msgr2=0x7fd070107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.017+0000 7fd075e00700 1 -- 172.21.15.69:0/1966390500 shutdown_connections 2024-04-24T09:05:14.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.017+0000 7fd075e00700 1 -- 172.21.15.69:0/1966390500 wait complete. 2024-04-24T09:05:14.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.018+0000 7fd075e00700 1 Processor -- start 2024-04-24T09:05:14.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.019+0000 7fd075e00700 1 -- start start 2024-04-24T09:05:14.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.019+0000 7fd075e00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd070109380 0x7fd0701a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.019+0000 7fd075e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd070109d30 0x7fd0701a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.019+0000 7fd075e00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd07010e460 0x7fd07019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.019+0000 7fd075e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd070119340 con 0x7fd070109d30 2024-04-24T09:05:14.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.019+0000 7fd075e00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd0701191c0 con 0x7fd07010e460 2024-04-24T09:05:14.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.019+0000 7fd075e00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd0701194c0 con 0x7fd070109380 2024-04-24T09:05:14.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.019+0000 7fd06e000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd070109d30 0x7fd0701a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.019+0000 7fd06e000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd070109d30 0x7fd0701a5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38998/0 (socket says 172.21.15.69:38998) 2024-04-24T09:05:14.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.019+0000 7fd06e000700 1 -- 172.21.15.69:0/3774630436 learned_addr learned my addr 172.21.15.69:0/3774630436 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:14.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.020+0000 7fd06ea00700 1 --2- 172.21.15.69:0/3774630436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd070109380 0x7fd0701a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.020+0000 7fd06f400700 1 --2- 172.21.15.69:0/3774630436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd07010e460 0x7fd07019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.020+0000 7fd06e000700 1 -- 172.21.15.69:0/3774630436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd070109380 msgr2=0x7fd0701a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.020+0000 7fd06e000700 1 --2- 172.21.15.69:0/3774630436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd070109380 0x7fd0701a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.020+0000 7fd06e000700 1 -- 172.21.15.69:0/3774630436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd07010e460 msgr2=0x7fd07019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.020+0000 7fd06e000700 1 --2- 172.21.15.69:0/3774630436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd07010e460 0x7fd07019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.020+0000 7fd06e000700 1 -- 172.21.15.69:0/3774630436 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07019fd60 con 0x7fd070109d30 2024-04-24T09:05:14.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.020+0000 7fd06ea00700 1 --2- 172.21.15.69:0/3774630436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd070109380 0x7fd0701a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:14.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.020+0000 7fd06e000700 1 --2- 172.21.15.69:0/3774630436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd070109d30 0x7fd0701a5980 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00c010 tx=0x7fd05c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.021+0000 7fd074a00700 1 -- 172.21.15.69:0/3774630436 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c004830 con 0x7fd070109d30 2024-04-24T09:05:14.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.021+0000 7fd075e00700 1 -- 172.21.15.69:0/3774630436 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07019fff0 con 0x7fd070109d30 2024-04-24T09:05:14.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.021+0000 7fd075e00700 1 -- 172.21.15.69:0/3774630436 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0701ac310 con 0x7fd070109d30 2024-04-24T09:05:14.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.021+0000 7fd074a00700 1 -- 172.21.15.69:0/3774630436 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd05c005020 con 0x7fd070109d30 2024-04-24T09:05:14.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.021+0000 7fd074a00700 1 -- 172.21.15.69:0/3774630436 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c003e20 con 0x7fd070109d30 2024-04-24T09:05:14.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.022+0000 7fd075e00700 1 -- 172.21.15.69:0/3774630436 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd070063280 con 0x7fd070109d30 2024-04-24T09:05:14.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.024+0000 7fd074a00700 1 -- 172.21.15.69:0/3774630436 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd05c005540 con 0x7fd070109d30 2024-04-24T09:05:14.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.025+0000 7fd074a00700 1 --2- 172.21.15.69:0/3774630436 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd0540a3d50 0x7fd0540a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.025+0000 7fd074a00700 1 -- 172.21.15.69:0/3774630436 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd05c0d1be0 con 0x7fd070109d30 2024-04-24T09:05:14.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.025+0000 7fd06ea00700 1 --2- 172.21.15.69:0/3774630436 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd0540a3d50 0x7fd0540a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.025+0000 7fd074a00700 1 --2- 172.21.15.69:0/3774630436 >> [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638] conn(0x7fd0540aeeb0 0x7fd0540b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.025+0000 7fd074a00700 1 -- 172.21.15.69:0/3774630436 --> [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd0540b1a20 con 0x7fd0540aeeb0 2024-04-24T09:05:14.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.025+0000 7fd074a00700 1 -- 172.21.15.69:0/3774630436 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fd05c0d1f90 con 0x7fd070109d30 2024-04-24T09:05:14.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.026+0000 7fd06ea00700 1 --2- 172.21.15.69:0/3774630436 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd0540a3d50 0x7fd0540a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd060005950 tx=0x7fd0600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.026+0000 7fd06f400700 1 --2- 172.21.15.69:0/3774630436 >> [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638] conn(0x7fd0540aeeb0 0x7fd0540b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.027+0000 7fd06f400700 1 --2- 172.21.15.69:0/3774630436 >> [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638] conn(0x7fd0540aeeb0 0x7fd0540b1350 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.029+0000 7fd074a00700 1 -- 172.21.15.69:0/3774630436 <== osd.2 v2:172.21.15.130:6800/3109352638 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd0540b1a20 con 0x7fd0540aeeb0 2024-04-24T09:05:14.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.048+0000 7fd075e00700 1 -- 172.21.15.69:0/3774630436 --> [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd070063680 con 0x7fd0540aeeb0 2024-04-24T09:05:14.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.049+0000 7fd074a00700 1 -- 172.21.15.69:0/3774630436 <== osd.2 v2:172.21.15.130:6800/3109352638 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd070063680 con 0x7fd0540aeeb0 2024-04-24T09:05:14.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.050+0000 7fd075e00700 1 -- 172.21.15.69:0/3774630436 >> [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638] conn(0x7fd0540aeeb0 msgr2=0x7fd0540b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.050+0000 7fd075e00700 1 --2- 172.21.15.69:0/3774630436 >> [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638] conn(0x7fd0540aeeb0 0x7fd0540b1350 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.050+0000 7fd075e00700 1 -- 172.21.15.69:0/3774630436 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd0540a3d50 msgr2=0x7fd0540a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.050+0000 7fd075e00700 1 --2- 172.21.15.69:0/3774630436 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fd0540a3d50 0x7fd0540a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd060005950 tx=0x7fd0600058e0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.050+0000 7fd075e00700 1 -- 172.21.15.69:0/3774630436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd070109d30 msgr2=0x7fd0701a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.050+0000 7fd075e00700 1 --2- 172.21.15.69:0/3774630436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd070109d30 0x7fd0701a5980 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00c010 tx=0x7fd05c0047b0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.050+0000 7fd06f400700 1 -- 172.21.15.69:0/3774630436 reap_dead start 2024-04-24T09:05:14.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.050+0000 7fd075e00700 1 -- 172.21.15.69:0/3774630436 shutdown_connections 2024-04-24T09:05:14.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.050+0000 7fd075e00700 1 -- 172.21.15.69:0/3774630436 >> 172.21.15.69:0/3774630436 conn(0x7fd0700762c0 msgr2=0x7fd0700768e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.050+0000 7fd075e00700 1 -- 172.21.15.69:0/3774630436 shutdown_connections 2024-04-24T09:05:14.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:11.050+0000 7fd075e00700 1 -- 172.21.15.69:0/3774630436 wait complete. 2024-04-24T09:05:16.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:16 smithi069 ceph-mon[28170]: pgmap v408: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-24T09:05:16.773 INFO:teuthology.orchestra.run.smithi069.stdout:163208757322 2024-04-24T09:05:16.774 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-24T09:05:16.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:16 smithi081 ceph-mon[30961]: pgmap v408: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-24T09:05:16.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:16 smithi130 ceph-mon[31553]: pgmap v408: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-24T09:05:17.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.020+0000 7f8699600700 1 -- 172.21.15.69:0/4266778618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f869406a800 msgr2=0x7f869406abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.020+0000 7f8699600700 1 --2- 172.21.15.69:0/4266778618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f869406a800 0x7f869406abe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8684009990 tx=0x7f8684009ca0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.020+0000 7f8699600700 1 -- 172.21.15.69:0/4266778618 shutdown_connections 2024-04-24T09:05:17.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.020+0000 7f8699600700 1 --2- 172.21.15.69:0/4266778618 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f869410f5a0 0x7f8694111990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.020+0000 7f8699600700 1 --2- 172.21.15.69:0/4266778618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f869406a800 0x7f869406abe0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.020+0000 7f8699600700 1 --2- 172.21.15.69:0/4266778618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f869406b120 0x7f869410f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.020+0000 7f8699600700 1 -- 172.21.15.69:0/4266778618 >> 172.21.15.69:0/4266778618 conn(0x7f86940789a0 msgr2=0x7f8694078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.020+0000 7f8699600700 1 -- 172.21.15.69:0/4266778618 shutdown_connections 2024-04-24T09:05:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.020+0000 7f8699600700 1 -- 172.21.15.69:0/4266778618 wait complete. 2024-04-24T09:05:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.021+0000 7f8699600700 1 Processor -- start 2024-04-24T09:05:17.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.021+0000 7f8699600700 1 -- start start 2024-04-24T09:05:17.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.022+0000 7f8699600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f869406a800 0x7f869419b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.022+0000 7f8699600700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f869406b120 0x7f869419ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.022+0000 7f8699600700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f869410f5a0 0x7f869419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.022+0000 7f8699600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8694114f00 con 0x7f869406a800 2024-04-24T09:05:17.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.022+0000 7f8699600700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8694114d80 con 0x7f869410f5a0 2024-04-24T09:05:17.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.022+0000 7f8699600700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8694115080 con 0x7f869406b120 2024-04-24T09:05:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.022+0000 7f8692a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f869406a800 0x7f869419b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.022+0000 7f8692a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f869406a800 0x7f869419b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45048/0 (socket says 172.21.15.69:45048) 2024-04-24T09:05:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.022+0000 7f8692a00700 1 -- 172.21.15.69:0/938230205 learned_addr learned my addr 172.21.15.69:0/938230205 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.023+0000 7f8692000700 1 --2- 172.21.15.69:0/938230205 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f869406b120 0x7f869419ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.023+0000 7f8693400700 1 --2- 172.21.15.69:0/938230205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f869410f5a0 0x7f869419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.023+0000 7f8692a00700 1 -- 172.21.15.69:0/938230205 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f869406b120 msgr2=0x7f869419ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.023+0000 7f8692a00700 1 --2- 172.21.15.69:0/938230205 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f869406b120 0x7f869419ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.023+0000 7f8692a00700 1 -- 172.21.15.69:0/938230205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f869410f5a0 msgr2=0x7f869419fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.023+0000 7f8692a00700 1 --2- 172.21.15.69:0/938230205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f869410f5a0 0x7f869419fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.023+0000 7f8692a00700 1 -- 172.21.15.69:0/938230205 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86941a04e0 con 0x7f869406a800 2024-04-24T09:05:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.023+0000 7f8692a00700 1 --2- 172.21.15.69:0/938230205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f869406a800 0x7f869419b4d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8684000c00 tx=0x7f8684005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.023+0000 7f8693400700 1 --2- 172.21.15.69:0/938230205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f869410f5a0 0x7f869419fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.024+0000 7f868b400700 1 -- 172.21.15.69:0/938230205 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684027070 con 0x7f869406a800 2024-04-24T09:05:17.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.024+0000 7f8699600700 1 -- 172.21.15.69:0/938230205 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86941a0770 con 0x7f869406a800 2024-04-24T09:05:17.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.024+0000 7f868b400700 1 -- 172.21.15.69:0/938230205 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8684022470 con 0x7f869406a800 2024-04-24T09:05:17.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.024+0000 7f868b400700 1 -- 172.21.15.69:0/938230205 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684017650 con 0x7f869406a800 2024-04-24T09:05:17.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.024+0000 7f8699600700 1 -- 172.21.15.69:0/938230205 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86941a8050 con 0x7f869406a800 2024-04-24T09:05:17.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.025+0000 7f8699600700 1 -- 172.21.15.69:0/938230205 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86940631c0 con 0x7f869406a800 2024-04-24T09:05:17.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.027+0000 7f868b400700 1 -- 172.21.15.69:0/938230205 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f868402a070 con 0x7f869406a800 2024-04-24T09:05:17.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.029+0000 7f868b400700 1 --2- 172.21.15.69:0/938230205 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f86780a3cd0 0x7f86780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.029+0000 7f868b400700 1 -- 172.21.15.69:0/938230205 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f86840d1850 con 0x7f869406a800 2024-04-24T09:05:17.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.029+0000 7f8692000700 1 --2- 172.21.15.69:0/938230205 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f86780a3cd0 0x7f86780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.029+0000 7f868b400700 1 --2- 172.21.15.69:0/938230205 >> [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595] conn(0x7f86780aee30 0x7f86780b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.029+0000 7f868b400700 1 -- 172.21.15.69:0/938230205 --> [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f868402de50 con 0x7f86780aee30 2024-04-24T09:05:17.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.029+0000 7f868b400700 1 -- 172.21.15.69:0/938230205 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f86840d1c00 con 0x7f869406a800 2024-04-24T09:05:17.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.030+0000 7f8693400700 1 --2- 172.21.15.69:0/938230205 >> [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595] conn(0x7f86780aee30 0x7f86780b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.030+0000 7f8692000700 1 --2- 172.21.15.69:0/938230205 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f86780a3cd0 0x7f86780a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8680005950 tx=0x7f86800058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:17.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.030+0000 7f8693400700 1 --2- 172.21.15.69:0/938230205 >> [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595] conn(0x7f86780aee30 0x7f86780b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:17.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.031+0000 7f868b400700 1 -- 172.21.15.69:0/938230205 <== osd.0 v2:172.21.15.69:6802/340713595 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f868402de50 con 0x7f86780aee30 2024-04-24T09:05:17.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.050+0000 7f8699600700 1 -- 172.21.15.69:0/938230205 --> [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f86940635f0 con 0x7f86780aee30 2024-04-24T09:05:17.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.050+0000 7f868b400700 1 -- 172.21.15.69:0/938230205 <== osd.0 v2:172.21.15.69:6802/340713595 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f86940635f0 con 0x7f86780aee30 2024-04-24T09:05:17.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.051+0000 7f8699600700 1 -- 172.21.15.69:0/938230205 >> [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595] conn(0x7f86780aee30 msgr2=0x7f86780b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.051+0000 7f8699600700 1 --2- 172.21.15.69:0/938230205 >> [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595] conn(0x7f86780aee30 0x7f86780b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.051+0000 7f8699600700 1 -- 172.21.15.69:0/938230205 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f86780a3cd0 msgr2=0x7f86780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.051+0000 7f8699600700 1 --2- 172.21.15.69:0/938230205 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f86780a3cd0 0x7f86780a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8680005950 tx=0x7f86800058e0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.051+0000 7f8699600700 1 -- 172.21.15.69:0/938230205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f869406a800 msgr2=0x7f869419b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.051+0000 7f8699600700 1 --2- 172.21.15.69:0/938230205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f869406a800 0x7f869419b4d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8684000c00 tx=0x7f8684005ef0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.051+0000 7f8693400700 1 -- 172.21.15.69:0/938230205 reap_dead start 2024-04-24T09:05:17.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.051+0000 7f8699600700 1 -- 172.21.15.69:0/938230205 shutdown_connections 2024-04-24T09:05:17.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.051+0000 7f8699600700 1 -- 172.21.15.69:0/938230205 >> 172.21.15.69:0/938230205 conn(0x7f86940789a0 msgr2=0x7f8694110ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:17.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.051+0000 7f8699600700 1 -- 172.21.15.69:0/938230205 shutdown_connections 2024-04-24T09:05:17.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.051+0000 7f8699600700 1 -- 172.21.15.69:0/938230205 wait complete. 2024-04-24T09:05:17.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.261+0000 7fc06fe00700 1 -- 172.21.15.69:0/1140398709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc070102a20 msgr2=0x7fc070102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.261+0000 7fc06fe00700 1 --2- 172.21.15.69:0/1140398709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc070102a20 0x7fc070102ea0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc058009990 tx=0x7fc058009ca0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.261+0000 7fc06fe00700 1 -- 172.21.15.69:0/1140398709 shutdown_connections 2024-04-24T09:05:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.261+0000 7fc06fe00700 1 --2- 172.21.15.69:0/1140398709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0701033e0 0x7fc07010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.261+0000 7fc06fe00700 1 --2- 172.21.15.69:0/1140398709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc070108a20 0x7fc070108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.261+0000 7fc06fe00700 1 --2- 172.21.15.69:0/1140398709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc070102a20 0x7fc070102ea0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.261+0000 7fc06fe00700 1 -- 172.21.15.69:0/1140398709 >> 172.21.15.69:0/1140398709 conn(0x7fc0700fe570 msgr2=0x7fc070100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:17.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.261+0000 7fc06fe00700 1 -- 172.21.15.69:0/1140398709 shutdown_connections 2024-04-24T09:05:17.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.261+0000 7fc06fe00700 1 -- 172.21.15.69:0/1140398709 wait complete. 2024-04-24T09:05:17.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.262+0000 7fc06fe00700 1 Processor -- start 2024-04-24T09:05:17.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.262+0000 7fc06fe00700 1 -- start start 2024-04-24T09:05:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.263+0000 7fc06fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc070102a20 0x7fc07019b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.263+0000 7fc06fe00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0701033e0 0x7fc07019b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.263+0000 7fc06fe00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc070108a20 0x7fc07019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.263+0000 7fc06fe00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc070112c70 con 0x7fc070102a20 2024-04-24T09:05:17.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.263+0000 7fc06fe00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc070112af0 con 0x7fc0701033e0 2024-04-24T09:05:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.263+0000 7fc06fe00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc070112df0 con 0x7fc070108a20 2024-04-24T09:05:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.263+0000 7fc06ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc070102a20 0x7fc07019b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.263+0000 7fc06ea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc070102a20 0x7fc07019b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45076/0 (socket says 172.21.15.69:45076) 2024-04-24T09:05:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.263+0000 7fc06ea00700 1 -- 172.21.15.69:0/1171948091 learned_addr learned my addr 172.21.15.69:0/1171948091 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.263+0000 7fc066000700 1 --2- 172.21.15.69:0/1171948091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0701033e0 0x7fc07019b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.263+0000 7fc06f400700 1 --2- 172.21.15.69:0/1171948091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc070108a20 0x7fc07019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.264+0000 7fc06ea00700 1 -- 172.21.15.69:0/1171948091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc070108a20 msgr2=0x7fc07019fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.264+0000 7fc06ea00700 1 --2- 172.21.15.69:0/1171948091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc070108a20 0x7fc07019fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.264+0000 7fc06ea00700 1 -- 172.21.15.69:0/1171948091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0701033e0 msgr2=0x7fc07019b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.264+0000 7fc06ea00700 1 --2- 172.21.15.69:0/1171948091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0701033e0 0x7fc07019b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.264+0000 7fc06ea00700 1 -- 172.21.15.69:0/1171948091 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0701a0270 con 0x7fc070102a20 2024-04-24T09:05:17.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.264+0000 7fc066000700 1 --2- 172.21.15.69:0/1171948091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0701033e0 0x7fc07019b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:17.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.264+0000 7fc06ea00700 1 --2- 172.21.15.69:0/1171948091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc070102a20 0x7fc07019b260 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc060007e20 tx=0x7fc06000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:17.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.264+0000 7fc067e00700 1 -- 172.21.15.69:0/1171948091 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc060016020 con 0x7fc070102a20 2024-04-24T09:05:17.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.264+0000 7fc067e00700 1 -- 172.21.15.69:0/1171948091 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc06000ae00 con 0x7fc070102a20 2024-04-24T09:05:17.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.264+0000 7fc067e00700 1 -- 172.21.15.69:0/1171948091 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc060011700 con 0x7fc070102a20 2024-04-24T09:05:17.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.264+0000 7fc06fe00700 1 -- 172.21.15.69:0/1171948091 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0701a0560 con 0x7fc070102a20 2024-04-24T09:05:17.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.264+0000 7fc06fe00700 1 -- 172.21.15.69:0/1171948091 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0701a7de0 con 0x7fc070102a20 2024-04-24T09:05:17.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.265+0000 7fc06fe00700 1 -- 172.21.15.69:0/1171948091 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc07010e710 con 0x7fc070102a20 2024-04-24T09:05:17.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.268+0000 7fc067e00700 1 -- 172.21.15.69:0/1171948091 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc060019020 con 0x7fc070102a20 2024-04-24T09:05:17.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.269+0000 7fc067e00700 1 --2- 172.21.15.69:0/1171948091 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc0540a3d50 0x7fc0540a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.269+0000 7fc067e00700 1 -- 172.21.15.69:0/1171948091 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc0600d0c70 con 0x7fc070102a20 2024-04-24T09:05:17.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.269+0000 7fc066000700 1 --2- 172.21.15.69:0/1171948091 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc0540a3d50 0x7fc0540a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.269+0000 7fc067e00700 1 --2- 172.21.15.69:0/1171948091 >> [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330] conn(0x7fc0540aeeb0 0x7fc0540b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.269+0000 7fc067e00700 1 -- 172.21.15.69:0/1171948091 --> [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc060019ec0 con 0x7fc0540aeeb0 2024-04-24T09:05:17.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.269+0000 7fc067e00700 1 -- 172.21.15.69:0/1171948091 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fc0600d10c0 con 0x7fc070102a20 2024-04-24T09:05:17.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.270+0000 7fc066000700 1 --2- 172.21.15.69:0/1171948091 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc0540a3d50 0x7fc0540a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc058006010 tx=0x7fc05801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:17.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.270+0000 7fc06f400700 1 --2- 172.21.15.69:0/1171948091 >> [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330] conn(0x7fc0540aeeb0 0x7fc0540b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.271+0000 7fc06f400700 1 --2- 172.21.15.69:0/1171948091 >> [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330] conn(0x7fc0540aeeb0 0x7fc0540b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:17.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.274+0000 7fc067e00700 1 -- 172.21.15.69:0/1171948091 <== osd.1 v2:172.21.15.81:6800/3515249330 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc060019ec0 con 0x7fc0540aeeb0 2024-04-24T09:05:17.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.291+0000 7fc06fe00700 1 -- 172.21.15.69:0/1171948091 --> [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc070104160 con 0x7fc0540aeeb0 2024-04-24T09:05:17.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.292+0000 7fc067e00700 1 -- 172.21.15.69:0/1171948091 <== osd.1 v2:172.21.15.81:6800/3515249330 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc070104160 con 0x7fc0540aeeb0 2024-04-24T09:05:17.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.293+0000 7fc06fe00700 1 -- 172.21.15.69:0/1171948091 >> [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330] conn(0x7fc0540aeeb0 msgr2=0x7fc0540b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.293+0000 7fc06fe00700 1 --2- 172.21.15.69:0/1171948091 >> [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330] conn(0x7fc0540aeeb0 0x7fc0540b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.293+0000 7fc06fe00700 1 -- 172.21.15.69:0/1171948091 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc0540a3d50 msgr2=0x7fc0540a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.293+0000 7fc06fe00700 1 --2- 172.21.15.69:0/1171948091 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc0540a3d50 0x7fc0540a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc058006010 tx=0x7fc05801a040 comp rx=0 tx=0).stop 2024-04-24T09:05:17.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.293+0000 7fc06fe00700 1 -- 172.21.15.69:0/1171948091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc070102a20 msgr2=0x7fc07019b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.293+0000 7fc06fe00700 1 --2- 172.21.15.69:0/1171948091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc070102a20 0x7fc07019b260 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc060007e20 tx=0x7fc06000a340 comp rx=0 tx=0).stop 2024-04-24T09:05:17.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.293+0000 7fc06f400700 1 -- 172.21.15.69:0/1171948091 reap_dead start 2024-04-24T09:05:17.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.293+0000 7fc06fe00700 1 -- 172.21.15.69:0/1171948091 shutdown_connections 2024-04-24T09:05:17.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.293+0000 7fc06fe00700 1 -- 172.21.15.69:0/1171948091 >> 172.21.15.69:0/1171948091 conn(0x7fc0700fe570 msgr2=0x7fc07010dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:17.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.294+0000 7fc06fe00700 1 -- 172.21.15.69:0/1171948091 shutdown_connections 2024-04-24T09:05:17.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:17.294+0000 7fc06fe00700 1 -- 172.21.15.69:0/1171948091 wait complete. 2024-04-24T09:05:18.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.409+0000 7f5ae6200700 1 -- 172.21.15.69:0/580070495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ae010d010 msgr2=0x7f5ae010f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:18.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.409+0000 7f5ae6200700 1 --2- 172.21.15.69:0/580070495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ae010d010 0x7f5ae010f4d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800b0a0 tx=0x7f5ad800b3b0 comp rx=0 tx=0).stop 2024-04-24T09:05:18.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.409+0000 7f5ae6200700 1 -- 172.21.15.69:0/580070495 shutdown_connections 2024-04-24T09:05:18.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.409+0000 7f5ae6200700 1 --2- 172.21.15.69:0/580070495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ae010d010 0x7f5ae010f4d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:18.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.409+0000 7f5ae6200700 1 --2- 172.21.15.69:0/580070495 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ae0103be0 0x7f5ae0103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:18.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.409+0000 7f5ae6200700 1 --2- 172.21.15.69:0/580070495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ae0104500 0x7f5ae010c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.409+0000 7f5ae6200700 1 -- 172.21.15.69:0/580070495 >> 172.21.15.69:0/580070495 conn(0x7f5ae00fd410 msgr2=0x7f5ae00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.409+0000 7f5ae6200700 1 -- 172.21.15.69:0/580070495 shutdown_connections 2024-04-24T09:05:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.409+0000 7f5ae6200700 1 -- 172.21.15.69:0/580070495 wait complete. 2024-04-24T09:05:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.410+0000 7f5ae6200700 1 Processor -- start 2024-04-24T09:05:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.410+0000 7f5ae6200700 1 -- start start 2024-04-24T09:05:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.411+0000 7f5ae6200700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ae0103be0 0x7f5ae019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.411+0000 7f5ae6200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ae0104500 0x7f5ae019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.411+0000 7f5ae6200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ae010d010 0x7f5ae019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.411+0000 7f5ae6200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae0112cf0 con 0x7f5ae0104500 2024-04-24T09:05:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.411+0000 7f5ae6200700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae0112b70 con 0x7f5ae0103be0 2024-04-24T09:05:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.411+0000 7f5ae6200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae0112e70 con 0x7f5ae010d010 2024-04-24T09:05:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.411+0000 7f5adfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ae0104500 0x7f5ae019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.411+0000 7f5adfe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ae0104500 0x7f5ae019b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45096/0 (socket says 172.21.15.69:45096) 2024-04-24T09:05:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.411+0000 7f5adfe00700 1 -- 172.21.15.69:0/960866267 learned_addr learned my addr 172.21.15.69:0/960866267 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:18.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.412+0000 7f5ae5800700 1 --2- 172.21.15.69:0/960866267 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ae010d010 0x7f5ae019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.412+0000 7f5adfe00700 1 -- 172.21.15.69:0/960866267 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ae010d010 msgr2=0x7f5ae019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.412+0000 7f5adfe00700 1 --2- 172.21.15.69:0/960866267 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ae010d010 0x7f5ae019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.412+0000 7f5adfe00700 1 -- 172.21.15.69:0/960866267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ae0103be0 msgr2=0x7f5ae019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.412+0000 7f5ae4e00700 1 --2- 172.21.15.69:0/960866267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ae0103be0 0x7f5ae019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.412+0000 7f5adfe00700 1 --2- 172.21.15.69:0/960866267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ae0103be0 0x7f5ae019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.412+0000 7f5adfe00700 1 -- 172.21.15.69:0/960866267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae01a0370 con 0x7f5ae0104500 2024-04-24T09:05:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.412+0000 7f5ae4e00700 1 --2- 172.21.15.69:0/960866267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ae0103be0 0x7f5ae019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.412+0000 7f5adfe00700 1 --2- 172.21.15.69:0/960866267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ae0104500 0x7f5ae019b8a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00b480 tx=0x7f5acc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.412+0000 7f5add600700 1 -- 172.21.15.69:0/960866267 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc015020 con 0x7f5ae0104500 2024-04-24T09:05:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.413+0000 7f5add600700 1 -- 172.21.15.69:0/960866267 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5acc011ce0 con 0x7f5ae0104500 2024-04-24T09:05:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.413+0000 7f5ae6200700 1 -- 172.21.15.69:0/960866267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae01a0660 con 0x7f5ae0104500 2024-04-24T09:05:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.413+0000 7f5add600700 1 -- 172.21.15.69:0/960866267 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc010d60 con 0x7f5ae0104500 2024-04-24T09:05:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.413+0000 7f5ae6200700 1 -- 172.21.15.69:0/960866267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae01a7ee0 con 0x7f5ae0104500 2024-04-24T09:05:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.414+0000 7f5ae6200700 1 -- 172.21.15.69:0/960866267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ae0103320 con 0x7f5ae0104500 2024-04-24T09:05:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.416+0000 7f5add600700 1 -- 172.21.15.69:0/960866267 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5acc018020 con 0x7f5ae0104500 2024-04-24T09:05:18.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.417+0000 7f5add600700 1 --2- 172.21.15.69:0/960866267 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5ac80a3cd0 0x7f5ac80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.417+0000 7f5add600700 1 -- 172.21.15.69:0/960866267 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5acc0d06b0 con 0x7f5ae0104500 2024-04-24T09:05:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.417+0000 7f5ae4e00700 1 --2- 172.21.15.69:0/960866267 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5ac80a3cd0 0x7f5ac80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.418+0000 7f5add600700 1 --2- 172.21.15.69:0/960866267 >> [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153] conn(0x7f5ac80aee30 0x7f5ac80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.418+0000 7f5add600700 1 -- 172.21.15.69:0/960866267 --> [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5acc018ec0 con 0x7f5ac80aee30 2024-04-24T09:05:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.418+0000 7f5add600700 1 -- 172.21.15.69:0/960866267 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f5acc0d0a60 con 0x7f5ae0104500 2024-04-24T09:05:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.418+0000 7f5ae4e00700 1 --2- 172.21.15.69:0/960866267 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5ac80a3cd0 0x7f5ac80a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4005fd0 tx=0x7f5ad4005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.418+0000 7f5ae5800700 1 --2- 172.21.15.69:0/960866267 >> [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153] conn(0x7f5ac80aee30 0x7f5ac80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.420+0000 7f5ae5800700 1 --2- 172.21.15.69:0/960866267 >> [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153] conn(0x7f5ac80aee30 0x7f5ac80b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.423+0000 7f5add600700 1 -- 172.21.15.69:0/960866267 <== osd.7 v2:172.21.15.81:6816/4105801153 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5acc018ec0 con 0x7f5ac80aee30 2024-04-24T09:05:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.441+0000 7f5ae6200700 1 -- 172.21.15.69:0/960866267 --> [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ae006b0b0 con 0x7f5ac80aee30 2024-04-24T09:05:18.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.442+0000 7f5add600700 1 -- 172.21.15.69:0/960866267 <== osd.7 v2:172.21.15.81:6816/4105801153 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5ae006b0b0 con 0x7f5ac80aee30 2024-04-24T09:05:18.587 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:18 smithi069 ceph-mon[28170]: pgmap v409: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:05:18.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.443+0000 7f5ae6200700 1 -- 172.21.15.69:0/960866267 >> [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153] conn(0x7f5ac80aee30 msgr2=0x7f5ac80b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:18.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.443+0000 7f5ae6200700 1 --2- 172.21.15.69:0/960866267 >> [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153] conn(0x7f5ac80aee30 0x7f5ac80b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:18.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.443+0000 7f5ae6200700 1 -- 172.21.15.69:0/960866267 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5ac80a3cd0 msgr2=0x7f5ac80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:18.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.443+0000 7f5ae6200700 1 --2- 172.21.15.69:0/960866267 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f5ac80a3cd0 0x7f5ac80a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4005fd0 tx=0x7f5ad4005dc0 comp rx=0 tx=0).stop 2024-04-24T09:05:18.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.443+0000 7f5ae6200700 1 -- 172.21.15.69:0/960866267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ae0104500 msgr2=0x7f5ae019b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:18.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.443+0000 7f5ae6200700 1 --2- 172.21.15.69:0/960866267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ae0104500 0x7f5ae019b8a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00b480 tx=0x7f5acc00b840 comp rx=0 tx=0).stop 2024-04-24T09:05:18.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.443+0000 7f5ae5800700 1 -- 172.21.15.69:0/960866267 reap_dead start 2024-04-24T09:05:18.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.443+0000 7f5ae6200700 1 -- 172.21.15.69:0/960866267 shutdown_connections 2024-04-24T09:05:18.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.443+0000 7f5ae6200700 1 -- 172.21.15.69:0/960866267 >> 172.21.15.69:0/960866267 conn(0x7f5ae00fd410 msgr2=0x7f5ae010f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:18.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.444+0000 7f5ae6200700 1 -- 172.21.15.69:0/960866267 shutdown_connections 2024-04-24T09:05:18.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.444+0000 7f5ae6200700 1 -- 172.21.15.69:0/960866267 wait complete. 2024-04-24T09:05:18.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:18 smithi081 ceph-mon[30961]: pgmap v409: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:05:18.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:18 smithi130 ceph-mon[31553]: pgmap v409: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:05:20.469 INFO:teuthology.orchestra.run.smithi069.stdout:55834574949 2024-04-24T09:05:20.470 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-24T09:05:20.527 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:20 smithi069 ceph-mon[28170]: pgmap v410: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:05:20.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.639+0000 7f1453000700 1 -- 172.21.15.69:0/1867268046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f144c108b90 msgr2=0x7f144c108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:20.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.639+0000 7f1453000700 1 --2- 172.21.15.69:0/1867268046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f144c108b90 0x7f144c108f70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f143c009990 tx=0x7f143c009ca0 comp rx=0 tx=0).stop 2024-04-24T09:05:20.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.639+0000 7f1453000700 1 -- 172.21.15.69:0/1867268046 shutdown_connections 2024-04-24T09:05:20.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.639+0000 7f1453000700 1 --2- 172.21.15.69:0/1867268046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f144c103500 0x7f144c10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:20.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.639+0000 7f1453000700 1 --2- 172.21.15.69:0/1867268046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f144c108b90 0x7f144c108f70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:20.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.639+0000 7f1453000700 1 --2- 172.21.15.69:0/1867268046 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f144c102b40 0x7f144c102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:20.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.639+0000 7f1453000700 1 -- 172.21.15.69:0/1867268046 >> 172.21.15.69:0/1867268046 conn(0x7f144c079c50 msgr2=0x7f144c07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:20.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.639+0000 7f1453000700 1 -- 172.21.15.69:0/1867268046 shutdown_connections 2024-04-24T09:05:20.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.639+0000 7f1453000700 1 -- 172.21.15.69:0/1867268046 wait complete. 2024-04-24T09:05:20.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.640+0000 7f1453000700 1 Processor -- start 2024-04-24T09:05:20.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.641+0000 7f1453000700 1 -- start start 2024-04-24T09:05:20.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.641+0000 7f1453000700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f144c102b40 0x7f144c1a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:20.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.641+0000 7f1453000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f144c103500 0x7f144c1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:20.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.641+0000 7f1453000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f144c108b90 0x7f144c19b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:20.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.641+0000 7f1453000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f144c075b30 con 0x7f144c108b90 2024-04-24T09:05:20.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.641+0000 7f1453000700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f144c0759b0 con 0x7f144c102b40 2024-04-24T09:05:20.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.641+0000 7f1453000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f144c075cb0 con 0x7f144c103500 2024-04-24T09:05:20.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.641+0000 7f1451000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f144c108b90 0x7f144c19b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:20.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.641+0000 7f1451000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f144c108b90 0x7f144c19b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45116/0 (socket says 172.21.15.69:45116) 2024-04-24T09:05:20.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.641+0000 7f1451000700 1 -- 172.21.15.69:0/1968593960 learned_addr learned my addr 172.21.15.69:0/1968593960 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:20.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.642+0000 7f144be00700 1 --2- 172.21.15.69:0/1968593960 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f144c102b40 0x7f144c1a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:20.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.642+0000 7f144b400700 1 --2- 172.21.15.69:0/1968593960 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f144c103500 0x7f144c1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:20.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.642+0000 7f1451000700 1 -- 172.21.15.69:0/1968593960 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f144c103500 msgr2=0x7f144c1a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:20.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.642+0000 7f1451000700 1 --2- 172.21.15.69:0/1968593960 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f144c103500 0x7f144c1a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:20.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.642+0000 7f1451000700 1 -- 172.21.15.69:0/1968593960 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f144c102b40 msgr2=0x7f144c1a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:20.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.642+0000 7f1451000700 1 --2- 172.21.15.69:0/1968593960 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f144c102b40 0x7f144c1a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:20.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.642+0000 7f1451000700 1 -- 172.21.15.69:0/1968593960 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c19b9c0 con 0x7f144c108b90 2024-04-24T09:05:20.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.642+0000 7f1451000700 1 --2- 172.21.15.69:0/1968593960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f144c108b90 0x7f144c19b1f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1444007d70 tx=0x7f144400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:20.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.642+0000 7f1448c00700 1 -- 172.21.15.69:0/1968593960 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444016020 con 0x7f144c108b90 2024-04-24T09:05:20.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.642+0000 7f1448c00700 1 -- 172.21.15.69:0/1968593960 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f144400ad40 con 0x7f144c108b90 2024-04-24T09:05:20.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.642+0000 7f1448c00700 1 -- 172.21.15.69:0/1968593960 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444011620 con 0x7f144c108b90 2024-04-24T09:05:20.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.642+0000 7f1453000700 1 -- 172.21.15.69:0/1968593960 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144c19bcb0 con 0x7f144c108b90 2024-04-24T09:05:20.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.643+0000 7f1453000700 1 -- 172.21.15.69:0/1968593960 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f144c1a7ee0 con 0x7f144c108b90 2024-04-24T09:05:20.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.643+0000 7f1453000700 1 -- 172.21.15.69:0/1968593960 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f144c1079b0 con 0x7f144c108b90 2024-04-24T09:05:20.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.646+0000 7f1448c00700 1 -- 172.21.15.69:0/1968593960 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1444019020 con 0x7f144c108b90 2024-04-24T09:05:20.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.647+0000 7f1448c00700 1 --2- 172.21.15.69:0/1968593960 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f14300a3cd0 0x7f14300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:20.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.647+0000 7f1448c00700 1 -- 172.21.15.69:0/1968593960 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f14440d0dd0 con 0x7f144c108b90 2024-04-24T09:05:20.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.647+0000 7f144be00700 1 --2- 172.21.15.69:0/1968593960 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f14300a3cd0 0x7f14300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:20.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.647+0000 7f1448c00700 1 --2- 172.21.15.69:0/1968593960 >> [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387] conn(0x7f14300aee20 0x7f14300b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:20.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.647+0000 7f1448c00700 1 -- 172.21.15.69:0/1968593960 --> [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1444013e80 con 0x7f14300aee20 2024-04-24T09:05:20.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.647+0000 7f1448c00700 1 -- 172.21.15.69:0/1968593960 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f14440d1180 con 0x7f144c108b90 2024-04-24T09:05:20.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.648+0000 7f144b400700 1 --2- 172.21.15.69:0/1968593960 >> [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387] conn(0x7f14300aee20 0x7f14300b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:20.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.648+0000 7f144be00700 1 --2- 172.21.15.69:0/1968593960 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f14300a3cd0 0x7f14300a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f143c00b5c0 tx=0x7f143c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:20.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.648+0000 7f144b400700 1 --2- 172.21.15.69:0/1968593960 >> [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387] conn(0x7f14300aee20 0x7f14300b12c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:20.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.649+0000 7f1448c00700 1 -- 172.21.15.69:0/1968593960 <== osd.6 v2:172.21.15.69:6818/2606282387 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1444013e80 con 0x7f14300aee20 2024-04-24T09:05:20.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.667+0000 7f1453000700 1 -- 172.21.15.69:0/1968593960 --> [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f144c1041f0 con 0x7f14300aee20 2024-04-24T09:05:20.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.668+0000 7f1448c00700 1 -- 172.21.15.69:0/1968593960 <== osd.6 v2:172.21.15.69:6818/2606282387 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f144c1041f0 con 0x7f14300aee20 2024-04-24T09:05:20.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.668+0000 7f1453000700 1 -- 172.21.15.69:0/1968593960 >> [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387] conn(0x7f14300aee20 msgr2=0x7f14300b12c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:20.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.668+0000 7f1453000700 1 --2- 172.21.15.69:0/1968593960 >> [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387] conn(0x7f14300aee20 0x7f14300b12c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:20.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.668+0000 7f1453000700 1 -- 172.21.15.69:0/1968593960 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f14300a3cd0 msgr2=0x7f14300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:20.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.668+0000 7f1453000700 1 --2- 172.21.15.69:0/1968593960 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f14300a3cd0 0x7f14300a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f143c00b5c0 tx=0x7f143c01a040 comp rx=0 tx=0).stop 2024-04-24T09:05:20.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.668+0000 7f1453000700 1 -- 172.21.15.69:0/1968593960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f144c108b90 msgr2=0x7f144c19b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:20.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.668+0000 7f1453000700 1 --2- 172.21.15.69:0/1968593960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f144c108b90 0x7f144c19b1f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1444007d70 tx=0x7f144400a340 comp rx=0 tx=0).stop 2024-04-24T09:05:20.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.668+0000 7f1451000700 1 -- 172.21.15.69:0/1968593960 reap_dead start 2024-04-24T09:05:20.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.669+0000 7f1453000700 1 -- 172.21.15.69:0/1968593960 shutdown_connections 2024-04-24T09:05:20.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.669+0000 7f1453000700 1 -- 172.21.15.69:0/1968593960 >> 172.21.15.69:0/1968593960 conn(0x7f144c079c50 msgr2=0x7f144c101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:20.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.669+0000 7f1453000700 1 -- 172.21.15.69:0/1968593960 shutdown_connections 2024-04-24T09:05:20.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:18.669+0000 7f1453000700 1 -- 172.21.15.69:0/1968593960 wait complete. 2024-04-24T09:05:20.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:20 smithi081 ceph-mon[30961]: pgmap v410: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:05:20.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:20 smithi130 ceph-mon[31553]: pgmap v410: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:05:20.980 INFO:teuthology.orchestra.run.smithi069.stdout:34359738476 2024-04-24T09:05:20.980 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-24T09:05:21.396 INFO:teuthology.orchestra.run.smithi069.stdout:77309411422 2024-04-24T09:05:21.397 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-24T09:05:22.058 INFO:teuthology.orchestra.run.smithi069.stdout:219043332156 2024-04-24T09:05:22.059 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-24T09:05:22.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.779+0000 7f37e7400700 1 -- 172.21.15.69:0/1842523919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37e0075640 msgr2=0x7f37e0113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:22.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.779+0000 7f37e7400700 1 --2- 172.21.15.69:0/1842523919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37e0075640 0x7f37e0113780 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009990 tx=0x7f37c8009ca0 comp rx=0 tx=0).stop 2024-04-24T09:05:22.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.780+0000 7f37e7400700 1 -- 172.21.15.69:0/1842523919 shutdown_connections 2024-04-24T09:05:22.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.780+0000 7f37e7400700 1 --2- 172.21.15.69:0/1842523919 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37e0113cc0 0x7f37e01160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:22.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.780+0000 7f37e7400700 1 --2- 172.21.15.69:0/1842523919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f37e0074d20 0x7f37e0075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:22.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.780+0000 7f37e7400700 1 --2- 172.21.15.69:0/1842523919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37e0075640 0x7f37e0113780 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:22.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.780+0000 7f37e7400700 1 -- 172.21.15.69:0/1842523919 >> 172.21.15.69:0/1842523919 conn(0x7f37e0079ca0 msgr2=0x7f37e007a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:22.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.780+0000 7f37e7400700 1 -- 172.21.15.69:0/1842523919 shutdown_connections 2024-04-24T09:05:22.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.780+0000 7f37e7400700 1 -- 172.21.15.69:0/1842523919 wait complete. 2024-04-24T09:05:22.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.781+0000 7f37e7400700 1 Processor -- start 2024-04-24T09:05:22.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.781+0000 7f37e7400700 1 -- start start 2024-04-24T09:05:22.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.782+0000 7f37e7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37e0074d20 0x7f37e01a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:22.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.782+0000 7f37e7400700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37e0075640 0x7f37e01a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:22.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.782+0000 7f37e7400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f37e0113cc0 0x7f37e019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:22.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.782+0000 7f37e7400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0119310 con 0x7f37e0074d20 2024-04-24T09:05:22.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.782+0000 7f37e7400700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0119190 con 0x7f37e0075640 2024-04-24T09:05:22.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.782+0000 7f37e7400700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0119490 con 0x7f37e0113cc0 2024-04-24T09:05:22.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.782+0000 7f37e4a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37e0074d20 0x7f37e01a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:22.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.782+0000 7f37e4a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37e0074d20 0x7f37e01a5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45156/0 (socket says 172.21.15.69:45156) 2024-04-24T09:05:22.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.782+0000 7f37e4a00700 1 -- 172.21.15.69:0/432147454 learned_addr learned my addr 172.21.15.69:0/432147454 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:22.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.782+0000 7f37d7e00700 1 --2- 172.21.15.69:0/432147454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37e0075640 0x7f37e01a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:22.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.783+0000 7f37e5400700 1 --2- 172.21.15.69:0/432147454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f37e0113cc0 0x7f37e019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:22.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.783+0000 7f37e4a00700 1 -- 172.21.15.69:0/432147454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f37e0113cc0 msgr2=0x7f37e019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:22.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.783+0000 7f37e4a00700 1 --2- 172.21.15.69:0/432147454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f37e0113cc0 0x7f37e019f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:22.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.783+0000 7f37e4a00700 1 -- 172.21.15.69:0/432147454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37e0075640 msgr2=0x7f37e01a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:22.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.783+0000 7f37e4a00700 1 --2- 172.21.15.69:0/432147454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37e0075640 0x7f37e01a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:22.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.783+0000 7f37e4a00700 1 -- 172.21.15.69:0/432147454 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e019ff10 con 0x7f37e0074d20 2024-04-24T09:05:22.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.783+0000 7f37e4a00700 1 --2- 172.21.15.69:0/432147454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37e0074d20 0x7f37e01a5560 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f37d000d980 tx=0x7f37d000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:22.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.783+0000 7f37de000700 1 -- 172.21.15.69:0/432147454 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0017070 con 0x7f37e0074d20 2024-04-24T09:05:22.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.783+0000 7f37de000700 1 -- 172.21.15.69:0/432147454 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37d0004830 con 0x7f37e0074d20 2024-04-24T09:05:22.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.783+0000 7f37e7400700 1 -- 172.21.15.69:0/432147454 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e01a0200 con 0x7f37e0074d20 2024-04-24T09:05:22.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.783+0000 7f37de000700 1 -- 172.21.15.69:0/432147454 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0005440 con 0x7f37e0074d20 2024-04-24T09:05:22.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.783+0000 7f37e7400700 1 -- 172.21.15.69:0/432147454 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e01ac4e0 con 0x7f37e0074d20 2024-04-24T09:05:22.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.784+0000 7f37e7400700 1 -- 172.21.15.69:0/432147454 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f37e01134a0 con 0x7f37e0074d20 2024-04-24T09:05:22.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.786+0000 7f37de000700 1 -- 172.21.15.69:0/432147454 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f37d00055e0 con 0x7f37e0074d20 2024-04-24T09:05:22.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.788+0000 7f37de000700 1 --2- 172.21.15.69:0/432147454 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f37c40a3cd0 0x7f37c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:22.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.788+0000 7f37de000700 1 -- 172.21.15.69:0/432147454 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f37d00d0e60 con 0x7f37e0074d20 2024-04-24T09:05:22.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.788+0000 7f37d7e00700 1 --2- 172.21.15.69:0/432147454 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f37c40a3cd0 0x7f37c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:22.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.788+0000 7f37de000700 1 --2- 172.21.15.69:0/432147454 >> [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980] conn(0x7f37c40aee30 0x7f37c40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:22.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.788+0000 7f37de000700 1 -- 172.21.15.69:0/432147454 --> [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37c40b19a0 con 0x7f37c40aee30 2024-04-24T09:05:22.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.788+0000 7f37de000700 1 -- 172.21.15.69:0/432147454 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f37d00d1210 con 0x7f37e0074d20 2024-04-24T09:05:22.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.789+0000 7f37d7e00700 1 --2- 172.21.15.69:0/432147454 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f37c40a3cd0 0x7f37c40a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f37c800b5c0 tx=0x7f37c801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:22.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.789+0000 7f37e5400700 1 --2- 172.21.15.69:0/432147454 >> [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980] conn(0x7f37c40aee30 0x7f37c40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:22.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.790+0000 7f37e5400700 1 --2- 172.21.15.69:0/432147454 >> [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980] conn(0x7f37c40aee30 0x7f37c40b12d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:22.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.793+0000 7f37de000700 1 -- 172.21.15.69:0/432147454 <== osd.11 v2:172.21.15.130:6824/1208587980 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f37c40b19a0 con 0x7f37c40aee30 2024-04-24T09:05:22.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.812+0000 7f37e7400700 1 -- 172.21.15.69:0/432147454 --> [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f37e0076440 con 0x7f37c40aee30 2024-04-24T09:05:22.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.813+0000 7f37de000700 1 -- 172.21.15.69:0/432147454 <== osd.11 v2:172.21.15.130:6824/1208587980 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f37e0076440 con 0x7f37c40aee30 2024-04-24T09:05:22.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.813+0000 7f37e7400700 1 -- 172.21.15.69:0/432147454 >> [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980] conn(0x7f37c40aee30 msgr2=0x7f37c40b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:22.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.813+0000 7f37e7400700 1 --2- 172.21.15.69:0/432147454 >> [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980] conn(0x7f37c40aee30 0x7f37c40b12d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:22.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.813+0000 7f37e7400700 1 -- 172.21.15.69:0/432147454 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f37c40a3cd0 msgr2=0x7f37c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:22.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.813+0000 7f37e7400700 1 --2- 172.21.15.69:0/432147454 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f37c40a3cd0 0x7f37c40a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f37c800b5c0 tx=0x7f37c801a040 comp rx=0 tx=0).stop 2024-04-24T09:05:22.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.813+0000 7f37e7400700 1 -- 172.21.15.69:0/432147454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37e0074d20 msgr2=0x7f37e01a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:22.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.813+0000 7f37e7400700 1 --2- 172.21.15.69:0/432147454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37e0074d20 0x7f37e01a5560 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f37d000d980 tx=0x7f37d000dd40 comp rx=0 tx=0).stop 2024-04-24T09:05:22.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.814+0000 7f37e5400700 1 -- 172.21.15.69:0/432147454 reap_dead start 2024-04-24T09:05:22.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.814+0000 7f37e7400700 1 -- 172.21.15.69:0/432147454 shutdown_connections 2024-04-24T09:05:22.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.814+0000 7f37e7400700 1 -- 172.21.15.69:0/432147454 >> 172.21.15.69:0/432147454 conn(0x7f37e0079ca0 msgr2=0x7f37e0112a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:22.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.814+0000 7f37e7400700 1 -- 172.21.15.69:0/432147454 shutdown_connections 2024-04-24T09:05:22.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:21.814+0000 7f37e7400700 1 -- 172.21.15.69:0/432147454 wait complete. 2024-04-24T09:05:22.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:22 smithi081 ceph-mon[30961]: pgmap v411: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:05:22.840 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:22 smithi069 ceph-mon[28170]: pgmap v411: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:05:22.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:22 smithi130 ceph-mon[31553]: pgmap v411: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:05:22.999 INFO:teuthology.orchestra.run.smithi069.stdout:197568495684 2024-04-24T09:05:23.000 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-24T09:05:23.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.210+0000 7f36a3400700 1 -- 172.21.15.69:0/4205476387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f369c109380 msgr2=0x7f369c109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:23.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.210+0000 7f36a3400700 1 --2- 172.21.15.69:0/4205476387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f369c109380 0x7f369c109760 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f36900099c0 tx=0x7f3690009cd0 comp rx=0 tx=0).stop 2024-04-24T09:05:23.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.210+0000 7f36a3400700 1 -- 172.21.15.69:0/4205476387 shutdown_connections 2024-04-24T09:05:23.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.210+0000 7f36a3400700 1 --2- 172.21.15.69:0/4205476387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f369c10e460 0x7f369c116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:23.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.210+0000 7f36a3400700 1 --2- 172.21.15.69:0/4205476387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f369c109380 0x7f369c109760 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:23.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.210+0000 7f36a3400700 1 --2- 172.21.15.69:0/4205476387 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f369c109d30 0x7f369c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:23.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.210+0000 7f36a3400700 1 -- 172.21.15.69:0/4205476387 >> 172.21.15.69:0/4205476387 conn(0x7f369c0762c0 msgr2=0x7f369c107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:23.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.210+0000 7f36a3400700 1 -- 172.21.15.69:0/4205476387 shutdown_connections 2024-04-24T09:05:23.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.210+0000 7f36a3400700 1 -- 172.21.15.69:0/4205476387 wait complete. 2024-04-24T09:05:23.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.211+0000 7f36a3400700 1 Processor -- start 2024-04-24T09:05:23.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.212+0000 7f36a3400700 1 -- start start 2024-04-24T09:05:23.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.212+0000 7f36a3400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f369c109380 0x7f369c1a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:23.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.212+0000 7f36a3400700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f369c109d30 0x7f369c1a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:23.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.212+0000 7f36a3400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f369c10e460 0x7f369c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:23.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.212+0000 7f36a3400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f369c119360 con 0x7f369c10e460 2024-04-24T09:05:23.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.212+0000 7f36a3400700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1191e0 con 0x7f369c109d30 2024-04-24T09:05:23.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.212+0000 7f36a3400700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1194e0 con 0x7f369c109380 2024-04-24T09:05:23.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.212+0000 7f36a1400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f369c10e460 0x7f369c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:23.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.213+0000 7f36a1400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f369c10e460 0x7f369c19f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45138/0 (socket says 172.21.15.69:45138) 2024-04-24T09:05:23.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.213+0000 7f36a1400700 1 -- 172.21.15.69:0/773776925 learned_addr learned my addr 172.21.15.69:0/773776925 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:23.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.213+0000 7f36a0a00700 1 --2- 172.21.15.69:0/773776925 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f369c109380 0x7f369c1a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:23.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.213+0000 7f369be00700 1 --2- 172.21.15.69:0/773776925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f369c109d30 0x7f369c1a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:23.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.213+0000 7f36a1400700 1 -- 172.21.15.69:0/773776925 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f369c109380 msgr2=0x7f369c1a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:23.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.213+0000 7f36a1400700 1 --2- 172.21.15.69:0/773776925 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f369c109380 0x7f369c1a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:23.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.213+0000 7f36a1400700 1 -- 172.21.15.69:0/773776925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f369c109d30 msgr2=0x7f369c1a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:23.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.213+0000 7f36a1400700 1 --2- 172.21.15.69:0/773776925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f369c109d30 0x7f369c1a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:23.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.213+0000 7f36a1400700 1 -- 172.21.15.69:0/773776925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369c19fdc0 con 0x7f369c10e460 2024-04-24T09:05:23.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.213+0000 7f36a1400700 1 --2- 172.21.15.69:0/773776925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f369c10e460 0x7f369c19f660 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3694007ee0 tx=0x7f369400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:23.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.214+0000 7f3699600700 1 -- 172.21.15.69:0/773776925 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369400dbd0 con 0x7f369c10e460 2024-04-24T09:05:23.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.214+0000 7f3699600700 1 -- 172.21.15.69:0/773776925 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f369400dd70 con 0x7f369c10e460 2024-04-24T09:05:23.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.214+0000 7f36a3400700 1 -- 172.21.15.69:0/773776925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369c1a00b0 con 0x7f369c10e460 2024-04-24T09:05:23.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.214+0000 7f3699600700 1 -- 172.21.15.69:0/773776925 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694005a20 con 0x7f369c10e460 2024-04-24T09:05:23.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.214+0000 7f36a3400700 1 -- 172.21.15.69:0/773776925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f369c1ac320 con 0x7f369c10e460 2024-04-24T09:05:23.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.215+0000 7f36a3400700 1 -- 172.21.15.69:0/773776925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f369c063280 con 0x7f369c10e460 2024-04-24T09:05:23.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.217+0000 7f3699600700 1 -- 172.21.15.69:0/773776925 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f369401a020 con 0x7f369c10e460 2024-04-24T09:05:23.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.219+0000 7f3699600700 1 --2- 172.21.15.69:0/773776925 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f36840a3d50 0x7f36840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:23.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.219+0000 7f3699600700 1 -- 172.21.15.69:0/773776925 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f36940d06e0 con 0x7f369c10e460 2024-04-24T09:05:23.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.219+0000 7f36a0a00700 1 --2- 172.21.15.69:0/773776925 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f36840a3d50 0x7f36840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:23.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.219+0000 7f3699600700 1 --2- 172.21.15.69:0/773776925 >> [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529] conn(0x7f36840aeeb0 0x7f36840b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:23.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.219+0000 7f3699600700 1 -- 172.21.15.69:0/773776925 --> [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3694009390 con 0x7f36840aeeb0 2024-04-24T09:05:23.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.219+0000 7f3699600700 1 -- 172.21.15.69:0/773776925 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f36940d0a90 con 0x7f369c10e460 2024-04-24T09:05:23.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.219+0000 7f369be00700 1 --2- 172.21.15.69:0/773776925 >> [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529] conn(0x7f36840aeeb0 0x7f36840b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:23.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.220+0000 7f36a0a00700 1 --2- 172.21.15.69:0/773776925 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f36840a3d50 0x7f36840a6210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3690009f50 tx=0x7f3690005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:23.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.220+0000 7f369be00700 1 --2- 172.21.15.69:0/773776925 >> [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529] conn(0x7f36840aeeb0 0x7f36840b1350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:23.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.221+0000 7f3699600700 1 -- 172.21.15.69:0/773776925 <== osd.3 v2:172.21.15.69:6810/3549681529 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3694009390 con 0x7f36840aeeb0 2024-04-24T09:05:23.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.239+0000 7f36a3400700 1 -- 172.21.15.69:0/773776925 --> [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f369c063680 con 0x7f36840aeeb0 2024-04-24T09:05:23.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.239+0000 7f3699600700 1 -- 172.21.15.69:0/773776925 <== osd.3 v2:172.21.15.69:6810/3549681529 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f369c063680 con 0x7f36840aeeb0 2024-04-24T09:05:23.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.240+0000 7f36a3400700 1 -- 172.21.15.69:0/773776925 >> [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529] conn(0x7f36840aeeb0 msgr2=0x7f36840b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:23.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.240+0000 7f36a3400700 1 --2- 172.21.15.69:0/773776925 >> [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529] conn(0x7f36840aeeb0 0x7f36840b1350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:23.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.240+0000 7f36a3400700 1 -- 172.21.15.69:0/773776925 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f36840a3d50 msgr2=0x7f36840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:23.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.240+0000 7f36a3400700 1 --2- 172.21.15.69:0/773776925 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f36840a3d50 0x7f36840a6210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3690009f50 tx=0x7f3690005fb0 comp rx=0 tx=0).stop 2024-04-24T09:05:23.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.240+0000 7f36a3400700 1 -- 172.21.15.69:0/773776925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f369c10e460 msgr2=0x7f369c19f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:23.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.240+0000 7f36a3400700 1 --2- 172.21.15.69:0/773776925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f369c10e460 0x7f369c19f660 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3694007ee0 tx=0x7f369400d3b0 comp rx=0 tx=0).stop 2024-04-24T09:05:23.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.240+0000 7f36a1400700 1 -- 172.21.15.69:0/773776925 reap_dead start 2024-04-24T09:05:23.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.241+0000 7f36a3400700 1 -- 172.21.15.69:0/773776925 shutdown_connections 2024-04-24T09:05:23.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.241+0000 7f36a3400700 1 -- 172.21.15.69:0/773776925 >> 172.21.15.69:0/773776925 conn(0x7f369c0762c0 msgr2=0x7f369c076920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:23.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.241+0000 7f36a3400700 1 -- 172.21.15.69:0/773776925 shutdown_connections 2024-04-24T09:05:23.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:20.241+0000 7f36a3400700 1 -- 172.21.15.69:0/773776925 wait complete. 2024-04-24T09:05:24.551 INFO:teuthology.orchestra.run.smithi069.stdout:107374182489 2024-04-24T09:05:24.551 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-24T09:05:24.561 INFO:teuthology.orchestra.run.smithi069.stdout:253403070516 2024-04-24T09:05:24.562 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-24T09:05:24.789 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:24 smithi069 ceph-mon[28170]: pgmap v412: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:05:24.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:24 smithi081 ceph-mon[30961]: pgmap v412: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:05:24.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:24 smithi130 ceph-mon[31553]: pgmap v412: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:05:25.149 INFO:teuthology.orchestra.run.smithi069.stdout:347892351008 2024-04-24T09:05:25.150 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-24T09:05:25.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.425+0000 7fccb3a00700 1 -- 172.21.15.69:0/2510353806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccac107a20 msgr2=0x7fccac06a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:25.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.425+0000 7fccb3a00700 1 --2- 172.21.15.69:0/2510353806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccac107a20 0x7fccac06a800 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcca0009990 tx=0x7fcca0009ca0 comp rx=0 tx=0).stop 2024-04-24T09:05:25.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.426+0000 7fccb3a00700 1 -- 172.21.15.69:0/2510353806 shutdown_connections 2024-04-24T09:05:25.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.426+0000 7fccb3a00700 1 --2- 172.21.15.69:0/2510353806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fccac105530 0x7fccac1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:25.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.426+0000 7fccb3a00700 1 --2- 172.21.15.69:0/2510353806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccac107a20 0x7fccac06a800 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:25.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.426+0000 7fccb3a00700 1 --2- 172.21.15.69:0/2510353806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fccac06ad40 0x7fccac06b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:25.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.426+0000 7fccb3a00700 1 -- 172.21.15.69:0/2510353806 >> 172.21.15.69:0/2510353806 conn(0x7fccac0789a0 msgr2=0x7fccac078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:25.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.426+0000 7fccb3a00700 1 -- 172.21.15.69:0/2510353806 shutdown_connections 2024-04-24T09:05:25.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.426+0000 7fccb3a00700 1 -- 172.21.15.69:0/2510353806 wait complete. 2024-04-24T09:05:25.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.427+0000 7fccb3a00700 1 Processor -- start 2024-04-24T09:05:25.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.427+0000 7fccb3a00700 1 -- start start 2024-04-24T09:05:25.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.428+0000 7fccb3a00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fccac06ad40 0x7fccac1a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.428+0000 7fccb3a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccac105530 0x7fccac1a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.428+0000 7fccb3a00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fccac107a20 0x7fccac19f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.428+0000 7fccb3a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1192d0 con 0x7fccac105530 2024-04-24T09:05:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.428+0000 7fccb3a00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fccac119150 con 0x7fccac06ad40 2024-04-24T09:05:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.428+0000 7fccb3a00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fccac119450 con 0x7fccac107a20 2024-04-24T09:05:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.428+0000 7fccabe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccac105530 0x7fccac1a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.428+0000 7fccabe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccac105530 0x7fccac1a5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50956/0 (socket says 172.21.15.69:50956) 2024-04-24T09:05:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.428+0000 7fccabe00700 1 -- 172.21.15.69:0/2398568904 learned_addr learned my addr 172.21.15.69:0/2398568904 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.428+0000 7fccb1000700 1 --2- 172.21.15.69:0/2398568904 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fccac06ad40 0x7fccac1a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.428+0000 7fccb1a00700 1 --2- 172.21.15.69:0/2398568904 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fccac107a20 0x7fccac19f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.429+0000 7fccabe00700 1 -- 172.21.15.69:0/2398568904 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fccac107a20 msgr2=0x7fccac19f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.429+0000 7fccabe00700 1 --2- 172.21.15.69:0/2398568904 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fccac107a20 0x7fccac19f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.429+0000 7fccabe00700 1 -- 172.21.15.69:0/2398568904 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fccac06ad40 msgr2=0x7fccac1a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.429+0000 7fccabe00700 1 --2- 172.21.15.69:0/2398568904 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fccac06ad40 0x7fccac1a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.429+0000 7fccabe00700 1 -- 172.21.15.69:0/2398568904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccac19fcf0 con 0x7fccac105530 2024-04-24T09:05:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.429+0000 7fccabe00700 1 --2- 172.21.15.69:0/2398568904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccac105530 0x7fccac1a5910 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800c970 tx=0x7fcc9800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.430+0000 7fcca9600700 1 -- 172.21.15.69:0/2398568904 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc980078d0 con 0x7fccac105530 2024-04-24T09:05:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.430+0000 7fcca9600700 1 -- 172.21.15.69:0/2398568904 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc98018480 con 0x7fccac105530 2024-04-24T09:05:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.430+0000 7fccb3a00700 1 -- 172.21.15.69:0/2398568904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccac19ffe0 con 0x7fccac105530 2024-04-24T09:05:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.430+0000 7fccb3a00700 1 -- 172.21.15.69:0/2398568904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccac1ac2a0 con 0x7fccac105530 2024-04-24T09:05:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.430+0000 7fcca9600700 1 -- 172.21.15.69:0/2398568904 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9800f6b0 con 0x7fccac105530 2024-04-24T09:05:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.431+0000 7fccb3a00700 1 -- 172.21.15.69:0/2398568904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc8c001030 con 0x7fccac105530 2024-04-24T09:05:25.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.433+0000 7fcca9600700 1 -- 172.21.15.69:0/2398568904 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcc98007a70 con 0x7fccac105530 2024-04-24T09:05:25.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.435+0000 7fcca9600700 1 --2- 172.21.15.69:0/2398568904 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcc940a3d50 0x7fcc940a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:25.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.435+0000 7fcca9600700 1 -- 172.21.15.69:0/2398568904 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fcc980d08b0 con 0x7fccac105530 2024-04-24T09:05:25.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.435+0000 7fccb1000700 1 --2- 172.21.15.69:0/2398568904 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcc940a3d50 0x7fcc940a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:25.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.436+0000 7fcca9600700 1 --2- 172.21.15.69:0/2398568904 >> [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159] conn(0x7fcc940aeeb0 0x7fcc940b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:25.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.436+0000 7fcca9600700 1 -- 172.21.15.69:0/2398568904 --> [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc940b1a20 con 0x7fcc940aeeb0 2024-04-24T09:05:25.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.436+0000 7fcca9600700 1 -- 172.21.15.69:0/2398568904 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fcc980d52a0 con 0x7fccac105530 2024-04-24T09:05:25.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.436+0000 7fccb1a00700 1 --2- 172.21.15.69:0/2398568904 >> [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159] conn(0x7fcc940aeeb0 0x7fcc940b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:25.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.436+0000 7fccb1000700 1 --2- 172.21.15.69:0/2398568904 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcc940a3d50 0x7fcc940a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcca000b5c0 tx=0x7fcca0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:25.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.437+0000 7fccb1a00700 1 --2- 172.21.15.69:0/2398568904 >> [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159] conn(0x7fcc940aeeb0 0x7fcc940b1350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:25.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.438+0000 7fcca9600700 1 -- 172.21.15.69:0/2398568904 <== osd.9 v2:172.21.15.69:6826/4110734159 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcc940b1a20 con 0x7fcc940aeeb0 2024-04-24T09:05:25.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.458+0000 7fccb3a00700 1 -- 172.21.15.69:0/2398568904 --> [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcc8c002e60 con 0x7fcc940aeeb0 2024-04-24T09:05:25.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.459+0000 7fcca9600700 1 -- 172.21.15.69:0/2398568904 <== osd.9 v2:172.21.15.69:6826/4110734159 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcc8c002e60 con 0x7fcc940aeeb0 2024-04-24T09:05:25.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.459+0000 7fccb3a00700 1 -- 172.21.15.69:0/2398568904 >> [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159] conn(0x7fcc940aeeb0 msgr2=0x7fcc940b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:25.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.459+0000 7fccb3a00700 1 --2- 172.21.15.69:0/2398568904 >> [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159] conn(0x7fcc940aeeb0 0x7fcc940b1350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:25.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.460+0000 7fccb3a00700 1 -- 172.21.15.69:0/2398568904 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcc940a3d50 msgr2=0x7fcc940a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:25.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.460+0000 7fccb3a00700 1 --2- 172.21.15.69:0/2398568904 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fcc940a3d50 0x7fcc940a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcca000b5c0 tx=0x7fcca0005fd0 comp rx=0 tx=0).stop 2024-04-24T09:05:25.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.460+0000 7fccb3a00700 1 -- 172.21.15.69:0/2398568904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccac105530 msgr2=0x7fccac1a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:25.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.460+0000 7fccb3a00700 1 --2- 172.21.15.69:0/2398568904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccac105530 0x7fccac1a5910 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800c970 tx=0x7fcc9800cc80 comp rx=0 tx=0).stop 2024-04-24T09:05:25.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.460+0000 7fccb1a00700 1 -- 172.21.15.69:0/2398568904 reap_dead start 2024-04-24T09:05:25.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.460+0000 7fccb3a00700 1 -- 172.21.15.69:0/2398568904 shutdown_connections 2024-04-24T09:05:25.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.460+0000 7fccb3a00700 1 -- 172.21.15.69:0/2398568904 >> 172.21.15.69:0/2398568904 conn(0x7fccac0789a0 msgr2=0x7fccac1009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:25.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.460+0000 7fccb3a00700 1 -- 172.21.15.69:0/2398568904 shutdown_connections 2024-04-24T09:05:25.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:25.460+0000 7fccb3a00700 1 -- 172.21.15.69:0/2398568904 wait complete. 2024-04-24T09:05:26.279 INFO:teuthology.orchestra.run.smithi069.stdout:287762808880 2024-04-24T09:05:26.280 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-24T09:05:26.333 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:26.368 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:26.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:26 smithi069 ceph-mon[28170]: pgmap v413: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:05:26.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:26 smithi081 ceph-mon[30961]: pgmap v413: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:05:26.849 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:26.860 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:26.861 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:26.865 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:26.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:26 smithi130 ceph-mon[31553]: pgmap v413: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:05:27.497 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:28.372 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:28.373 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:28.373 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:28.373 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:28.766 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:28 smithi069 ceph-mon[28170]: pgmap v414: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:05:28.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:28 smithi081 ceph-mon[30961]: pgmap v414: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:05:28.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:28 smithi130 ceph-mon[31553]: pgmap v414: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:05:30.787 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:30 smithi130 ceph-mon[31553]: pgmap v415: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:05:30.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:30 smithi081 ceph-mon[30961]: pgmap v415: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:05:30.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:30 smithi069 ceph-mon[28170]: pgmap v415: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:05:32.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:32 smithi130 ceph-mon[31553]: pgmap v416: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-24T09:05:32.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:32 smithi081 ceph-mon[30961]: pgmap v416: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-24T09:05:32.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:32 smithi069 ceph-mon[28170]: pgmap v416: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-24T09:05:34.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:34 smithi069 ceph-mon[28170]: pgmap v417: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T09:05:34.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:34 smithi081 ceph-mon[30961]: pgmap v417: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T09:05:34.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:34 smithi130 ceph-mon[31553]: pgmap v417: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T09:05:36.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:36 smithi081 ceph-mon[30961]: pgmap v418: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:05:36.838 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:36 smithi069 ceph-mon[28170]: pgmap v418: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:05:36.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:36 smithi130 ceph-mon[31553]: pgmap v418: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:05:37.560 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:37 smithi069 ceph-mon[28170]: pgmap v419: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:05:37.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:37 smithi081 ceph-mon[30961]: pgmap v419: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:05:37.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:37 smithi130 ceph-mon[31553]: pgmap v419: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:05:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:40 smithi081 ceph-mon[30961]: pgmap v420: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:05:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:40 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/114379173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T09:05:40.833 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:40 smithi069 ceph-mon[28170]: pgmap v420: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:05:40.833 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:40 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/114379173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T09:05:40.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:40 smithi130 ceph-mon[31553]: pgmap v420: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:05:40.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:40 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/114379173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T09:05:41.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.999+0000 7f331dc00700 1 -- 172.21.15.69:0/3914094941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f331810f370 msgr2=0x7f3318111760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:41.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.999+0000 7f331dc00700 1 --2- 172.21.15.69:0/3914094941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f331810f370 0x7f3318111760 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f331000b0a0 tx=0x7f331000b3b0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.999+0000 7f331dc00700 1 -- 172.21.15.69:0/3914094941 shutdown_connections 2024-04-24T09:05:41.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.999+0000 7f331dc00700 1 --2- 172.21.15.69:0/3914094941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f331810f370 0x7f3318111760 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.999+0000 7f331dc00700 1 --2- 172.21.15.69:0/3914094941 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33181017c0 0x7f3318101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.999+0000 7f331dc00700 1 --2- 172.21.15.69:0/3914094941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3318102170 0x7f331810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.999+0000 7f331dc00700 1 -- 172.21.15.69:0/3914094941 >> 172.21.15.69:0/3914094941 conn(0x7f33180fd480 msgr2=0x7f33180ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:41.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.000+0000 7f331dc00700 1 -- 172.21.15.69:0/3914094941 shutdown_connections 2024-04-24T09:05:41.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.000+0000 7f331dc00700 1 -- 172.21.15.69:0/3914094941 wait complete. 2024-04-24T09:05:41.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.001+0000 7f331dc00700 1 Processor -- start 2024-04-24T09:05:41.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.001+0000 7f331dc00700 1 -- start start 2024-04-24T09:05:41.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.001+0000 7f331dc00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33181017c0 0x7f331819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:41.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.001+0000 7f331dc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3318102170 0x7f331819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:41.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.001+0000 7f331dc00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f331810f370 0x7f331819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:41.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.001+0000 7f331dc00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3318114ec0 con 0x7f3318102170 2024-04-24T09:05:41.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.001+0000 7f331dc00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3318114d40 con 0x7f331810f370 2024-04-24T09:05:41.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.001+0000 7f331dc00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3318115040 con 0x7f33181017c0 2024-04-24T09:05:41.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.002+0000 7f3317400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3318102170 0x7f331819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:41.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.002+0000 7f3317400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3318102170 0x7f331819b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58918/0 (socket says 172.21.15.69:58918) 2024-04-24T09:05:41.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.002+0000 7f3317400700 1 -- 172.21.15.69:0/1946842995 learned_addr learned my addr 172.21.15.69:0/1946842995 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:41.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.002+0000 7f3317e00700 1 --2- 172.21.15.69:0/1946842995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33181017c0 0x7f331819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:41.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.002+0000 7f331d200700 1 --2- 172.21.15.69:0/1946842995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f331810f370 0x7f331819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:41.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.002+0000 7f3317400700 1 -- 172.21.15.69:0/1946842995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33181017c0 msgr2=0x7f331819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:41.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.002+0000 7f3317400700 1 --2- 172.21.15.69:0/1946842995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33181017c0 0x7f331819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.002+0000 7f3317400700 1 -- 172.21.15.69:0/1946842995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f331810f370 msgr2=0x7f331819fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:41.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.002+0000 7f3317400700 1 --2- 172.21.15.69:0/1946842995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f331810f370 0x7f331819fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.002+0000 7f3317400700 1 -- 172.21.15.69:0/1946842995 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33181a02e0 con 0x7f3318102170 2024-04-24T09:05:41.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.002+0000 7f3317400700 1 --2- 172.21.15.69:0/1946842995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3318102170 0x7f331819b810 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f330400b530 tx=0x7f330400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:41.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.003+0000 7f3314c00700 1 -- 172.21.15.69:0/1946842995 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304015020 con 0x7f3318102170 2024-04-24T09:05:41.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.003+0000 7f3314c00700 1 -- 172.21.15.69:0/1946842995 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3304011ce0 con 0x7f3318102170 2024-04-24T09:05:41.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.003+0000 7f331dc00700 1 -- 172.21.15.69:0/1946842995 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33181a05d0 con 0x7f3318102170 2024-04-24T09:05:41.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.003+0000 7f3314c00700 1 -- 172.21.15.69:0/1946842995 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304010d60 con 0x7f3318102170 2024-04-24T09:05:41.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.003+0000 7f331dc00700 1 -- 172.21.15.69:0/1946842995 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33181a7e50 con 0x7f3318102170 2024-04-24T09:05:41.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.004+0000 7f331dc00700 1 -- 172.21.15.69:0/1946842995 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33180697e0 con 0x7f3318102170 2024-04-24T09:05:41.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.006+0000 7f3314c00700 1 -- 172.21.15.69:0/1946842995 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3304018020 con 0x7f3318102170 2024-04-24T09:05:41.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.007+0000 7f3314c00700 1 --2- 172.21.15.69:0/1946842995 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f32fc0a3d50 0x7f32fc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:41.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.008+0000 7f3314c00700 1 -- 172.21.15.69:0/1946842995 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f33040cfff0 con 0x7f3318102170 2024-04-24T09:05:41.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.008+0000 7f3317e00700 1 --2- 172.21.15.69:0/1946842995 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f32fc0a3d50 0x7f32fc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:41.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.009+0000 7f3317e00700 1 --2- 172.21.15.69:0/1946842995 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f32fc0a3d50 0x7f32fc0a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3308005ea0 tx=0x7f3308005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:41.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.012+0000 7f3314c00700 1 -- 172.21.15.69:0/1946842995 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f330401f020 con 0x7f3318102170 2024-04-24T09:05:41.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.250+0000 7f331dc00700 1 -- 172.21.15.69:0/1946842995 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f331819c630 con 0x7f3318102170 2024-04-24T09:05:41.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.250+0000 7f3314c00700 1 -- 172.21.15.69:0/1946842995 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f330409c5e0 con 0x7f3318102170 2024-04-24T09:05:41.250 INFO:teuthology.orchestra.run.smithi069.stdout:107374182493 2024-04-24T09:05:41.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.253+0000 7f331dc00700 1 -- 172.21.15.69:0/1946842995 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f32fc0a3d50 msgr2=0x7f32fc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:41.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.253+0000 7f331dc00700 1 --2- 172.21.15.69:0/1946842995 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f32fc0a3d50 0x7f32fc0a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3308005ea0 tx=0x7f3308005e30 comp rx=0 tx=0).stop 2024-04-24T09:05:41.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.253+0000 7f331dc00700 1 -- 172.21.15.69:0/1946842995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3318102170 msgr2=0x7f331819b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:41.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.253+0000 7f331dc00700 1 --2- 172.21.15.69:0/1946842995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3318102170 0x7f331819b810 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f330400b530 tx=0x7f330400b8f0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.253+0000 7f331dc00700 1 -- 172.21.15.69:0/1946842995 shutdown_connections 2024-04-24T09:05:41.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.253+0000 7f331dc00700 1 --2- 172.21.15.69:0/1946842995 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f32fc0a3d50 0x7f32fc0a6210 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.253+0000 7f331dc00700 1 --2- 172.21.15.69:0/1946842995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f331810f370 0x7f331819fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.253+0000 7f331dc00700 1 --2- 172.21.15.69:0/1946842995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33181017c0 0x7f331819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.253+0000 7f331dc00700 1 --2- 172.21.15.69:0/1946842995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3318102170 0x7f331819b810 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.253+0000 7f331dc00700 1 -- 172.21.15.69:0/1946842995 >> 172.21.15.69:0/1946842995 conn(0x7f33180fd480 msgr2=0x7f33181105b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:41.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.254+0000 7f331dc00700 1 -- 172.21.15.69:0/1946842995 shutdown_connections 2024-04-24T09:05:41.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.254+0000 7f331dc00700 1 -- 172.21.15.69:0/1946842995 wait complete. 2024-04-24T09:05:41.579 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:41 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1946842995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T09:05:41.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.620+0000 7f304aa00700 1 -- 172.21.15.69:0/189868790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3044102a20 msgr2=0x7f3044102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:41.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.620+0000 7f304aa00700 1 --2- 172.21.15.69:0/189868790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3044102a20 0x7f3044102ea0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3030009990 tx=0x7f3030009ca0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.620+0000 7f304aa00700 1 -- 172.21.15.69:0/189868790 shutdown_connections 2024-04-24T09:05:41.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.620+0000 7f304aa00700 1 --2- 172.21.15.69:0/189868790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30441033e0 0x7f304410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.620+0000 7f304aa00700 1 --2- 172.21.15.69:0/189868790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3044108a20 0x7f3044108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.620+0000 7f304aa00700 1 --2- 172.21.15.69:0/189868790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3044102a20 0x7f3044102ea0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.620+0000 7f304aa00700 1 -- 172.21.15.69:0/189868790 >> 172.21.15.69:0/189868790 conn(0x7f30440fe570 msgr2=0x7f3044100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:41.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.620+0000 7f304aa00700 1 -- 172.21.15.69:0/189868790 shutdown_connections 2024-04-24T09:05:41.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.620+0000 7f304aa00700 1 -- 172.21.15.69:0/189868790 wait complete. 2024-04-24T09:05:41.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.621+0000 7f304aa00700 1 Processor -- start 2024-04-24T09:05:41.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.622+0000 7f304aa00700 1 -- start start 2024-04-24T09:05:41.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.622+0000 7f304aa00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3044102a20 0x7f304419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:41.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.622+0000 7f304aa00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30441033e0 0x7f304419b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:41.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.622+0000 7f304aa00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3044108a20 0x7f304419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:41.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.622+0000 7f304aa00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3044112d10 con 0x7f3044108a20 2024-04-24T09:05:41.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.622+0000 7f304aa00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3044112b90 con 0x7f3044102a20 2024-04-24T09:05:41.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.622+0000 7f304aa00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3044112e90 con 0x7f30441033e0 2024-04-24T09:05:41.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.622+0000 7f304a000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3044108a20 0x7f304419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:41.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.622+0000 7f304a000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3044108a20 0x7f304419fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58928/0 (socket says 172.21.15.69:58928) 2024-04-24T09:05:41.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.622+0000 7f304a000700 1 -- 172.21.15.69:0/3718411880 learned_addr learned my addr 172.21.15.69:0/3718411880 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:41.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.623+0000 7f3049600700 1 --2- 172.21.15.69:0/3718411880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3044102a20 0x7f304419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:41.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.623+0000 7f3048c00700 1 --2- 172.21.15.69:0/3718411880 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30441033e0 0x7f304419b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:41.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.623+0000 7f304a000700 1 -- 172.21.15.69:0/3718411880 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30441033e0 msgr2=0x7f304419b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:41.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.623+0000 7f304a000700 1 --2- 172.21.15.69:0/3718411880 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30441033e0 0x7f304419b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.623+0000 7f304a000700 1 -- 172.21.15.69:0/3718411880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3044102a20 msgr2=0x7f304419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:41.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.623+0000 7f304a000700 1 --2- 172.21.15.69:0/3718411880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3044102a20 0x7f304419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.623+0000 7f304a000700 1 -- 172.21.15.69:0/3718411880 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30441a03a0 con 0x7f3044108a20 2024-04-24T09:05:41.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.623+0000 7f304a000700 1 --2- 172.21.15.69:0/3718411880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3044108a20 0x7f304419fc60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f303c00b970 tx=0x7f303c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:41.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.624+0000 7f3042000700 1 -- 172.21.15.69:0/3718411880 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c019070 con 0x7f3044108a20 2024-04-24T09:05:41.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.624+0000 7f304aa00700 1 -- 172.21.15.69:0/3718411880 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30441a0690 con 0x7f3044108a20 2024-04-24T09:05:41.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.624+0000 7f3042000700 1 -- 172.21.15.69:0/3718411880 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f303c014440 con 0x7f3044108a20 2024-04-24T09:05:41.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.624+0000 7f304aa00700 1 -- 172.21.15.69:0/3718411880 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30441a7f10 con 0x7f3044108a20 2024-04-24T09:05:41.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.624+0000 7f3042000700 1 -- 172.21.15.69:0/3718411880 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c00ce60 con 0x7f3044108a20 2024-04-24T09:05:41.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.625+0000 7f304aa00700 1 -- 172.21.15.69:0/3718411880 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30440697e0 con 0x7f3044108a20 2024-04-24T09:05:41.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.627+0000 7f3042000700 1 -- 172.21.15.69:0/3718411880 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f303c01c070 con 0x7f3044108a20 2024-04-24T09:05:41.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.628+0000 7f3042000700 1 --2- 172.21.15.69:0/3718411880 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f302c0a3cd0 0x7f302c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:41.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.629+0000 7f3042000700 1 -- 172.21.15.69:0/3718411880 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f303c0cfbc0 con 0x7f3044108a20 2024-04-24T09:05:41.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.629+0000 7f3049600700 1 --2- 172.21.15.69:0/3718411880 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f302c0a3cd0 0x7f302c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:41.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.629+0000 7f3049600700 1 --2- 172.21.15.69:0/3718411880 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f302c0a3cd0 0x7f302c0a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3038006fd0 tx=0x7f3038008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:41.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.633+0000 7f3042000700 1 -- 172.21.15.69:0/3718411880 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f303c09c1f0 con 0x7f3044108a20 2024-04-24T09:05:41.801 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:41 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1946842995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T09:05:41.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:41 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1946842995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T09:05:41.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.876+0000 7f304aa00700 1 -- 172.21.15.69:0/3718411880 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f30441a8140 con 0x7f3044108a20 2024-04-24T09:05:41.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.877+0000 7f3042000700 1 -- 172.21.15.69:0/3718411880 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f303c09baf0 con 0x7f3044108a20 2024-04-24T09:05:41.877 INFO:teuthology.orchestra.run.smithi069.stdout:197568495688 2024-04-24T09:05:41.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.880+0000 7f304aa00700 1 -- 172.21.15.69:0/3718411880 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f302c0a3cd0 msgr2=0x7f302c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:41.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.880+0000 7f304aa00700 1 --2- 172.21.15.69:0/3718411880 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f302c0a3cd0 0x7f302c0a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3038006fd0 tx=0x7f3038008040 comp rx=0 tx=0).stop 2024-04-24T09:05:41.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.880+0000 7f304aa00700 1 -- 172.21.15.69:0/3718411880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3044108a20 msgr2=0x7f304419fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:41.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.880+0000 7f304aa00700 1 --2- 172.21.15.69:0/3718411880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3044108a20 0x7f304419fc60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f303c00b970 tx=0x7f303c00bd30 comp rx=0 tx=0).stop 2024-04-24T09:05:41.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.880+0000 7f304aa00700 1 -- 172.21.15.69:0/3718411880 shutdown_connections 2024-04-24T09:05:41.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.880+0000 7f304aa00700 1 --2- 172.21.15.69:0/3718411880 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f302c0a3cd0 0x7f302c0a6190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.880+0000 7f304aa00700 1 --2- 172.21.15.69:0/3718411880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3044108a20 0x7f304419fc60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.880+0000 7f304aa00700 1 --2- 172.21.15.69:0/3718411880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3044102a20 0x7f304419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.880+0000 7f304aa00700 1 --2- 172.21.15.69:0/3718411880 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30441033e0 0x7f304419b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:41.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.880+0000 7f304aa00700 1 -- 172.21.15.69:0/3718411880 >> 172.21.15.69:0/3718411880 conn(0x7f30440fe570 msgr2=0x7f304410f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:41.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.880+0000 7f304aa00700 1 -- 172.21.15.69:0/3718411880 shutdown_connections 2024-04-24T09:05:41.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:41.880+0000 7f304aa00700 1 -- 172.21.15.69:0/3718411880 wait complete. 2024-04-24T09:05:42.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.215+0000 7f3aa5000700 1 -- 172.21.15.69:0/868774758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa0103500 msgr2=0x7f3aa010f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.215+0000 7f3aa5000700 1 --2- 172.21.15.69:0/868774758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa0103500 0x7f3aa010f960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800b0a0 tx=0x7f3a9800b3b0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.215+0000 7f3aa5000700 1 -- 172.21.15.69:0/868774758 shutdown_connections 2024-04-24T09:05:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.215+0000 7f3aa5000700 1 --2- 172.21.15.69:0/868774758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa0103500 0x7f3aa010f960 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.215+0000 7f3aa5000700 1 --2- 172.21.15.69:0/868774758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa0108b90 0x7f3aa0108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.215+0000 7f3aa5000700 1 --2- 172.21.15.69:0/868774758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3aa0102b40 0x7f3aa0102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.215+0000 7f3aa5000700 1 -- 172.21.15.69:0/868774758 >> 172.21.15.69:0/868774758 conn(0x7f3aa0079c50 msgr2=0x7f3aa007a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.215+0000 7f3aa5000700 1 -- 172.21.15.69:0/868774758 shutdown_connections 2024-04-24T09:05:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.216+0000 7f3aa5000700 1 -- 172.21.15.69:0/868774758 wait complete. 2024-04-24T09:05:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.217+0000 7f3aa5000700 1 Processor -- start 2024-04-24T09:05:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.217+0000 7f3aa5000700 1 -- start start 2024-04-24T09:05:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.218+0000 7f3aa5000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa0102b40 0x7f3aa0074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:42.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.218+0000 7f3aa5000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa0103500 0x7f3aa0075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.218+0000 7f3aa5000700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3aa0108b90 0x7f3aa0075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.218+0000 7f3aa5000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0112d10 con 0x7f3aa0102b40 2024-04-24T09:05:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.218+0000 7f3aa5000700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0112b90 con 0x7f3aa0108b90 2024-04-24T09:05:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.218+0000 7f3aa5000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0112e90 con 0x7f3aa0103500 2024-04-24T09:05:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.218+0000 7f3a9de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa0102b40 0x7f3aa0074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.218+0000 7f3a9de00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa0102b40 0x7f3aa0074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58952/0 (socket says 172.21.15.69:58952) 2024-04-24T09:05:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.218+0000 7f3a9de00700 1 -- 172.21.15.69:0/913568520 learned_addr learned my addr 172.21.15.69:0/913568520 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.218+0000 7f3a9d400700 1 --2- 172.21.15.69:0/913568520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa0103500 0x7f3aa0075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.218+0000 7f3a9e800700 1 --2- 172.21.15.69:0/913568520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3aa0108b90 0x7f3aa0075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.218+0000 7f3a9de00700 1 -- 172.21.15.69:0/913568520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa0103500 msgr2=0x7f3aa0075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.218+0000 7f3a9de00700 1 --2- 172.21.15.69:0/913568520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa0103500 0x7f3aa0075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.218+0000 7f3a9de00700 1 -- 172.21.15.69:0/913568520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3aa0108b90 msgr2=0x7f3aa0075870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.218+0000 7f3a9de00700 1 --2- 172.21.15.69:0/913568520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3aa0108b90 0x7f3aa0075870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.218+0000 7f3a9de00700 1 -- 172.21.15.69:0/913568520 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa01a7710 con 0x7f3aa0102b40 2024-04-24T09:05:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.219+0000 7f3a9de00700 1 --2- 172.21.15.69:0/913568520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa0102b40 0x7f3aa0074df0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000d980 tx=0x7f3a9000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.219+0000 7f3a96a00700 1 -- 172.21.15.69:0/913568520 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90017070 con 0x7f3aa0102b40 2024-04-24T09:05:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.219+0000 7f3a96a00700 1 -- 172.21.15.69:0/913568520 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a90004830 con 0x7f3aa0102b40 2024-04-24T09:05:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.219+0000 7f3aa5000700 1 -- 172.21.15.69:0/913568520 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa01a7a00 con 0x7f3aa0102b40 2024-04-24T09:05:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.219+0000 7f3a96a00700 1 -- 172.21.15.69:0/913568520 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90005440 con 0x7f3aa0102b40 2024-04-24T09:05:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.219+0000 7f3aa5000700 1 -- 172.21.15.69:0/913568520 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa01a7f40 con 0x7f3aa0102b40 2024-04-24T09:05:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.220+0000 7f3aa5000700 1 -- 172.21.15.69:0/913568520 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa00697e0 con 0x7f3aa0102b40 2024-04-24T09:05:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.223+0000 7f3a96a00700 1 -- 172.21.15.69:0/913568520 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3a900055e0 con 0x7f3aa0102b40 2024-04-24T09:05:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.224+0000 7f3a96a00700 1 --2- 172.21.15.69:0/913568520 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a840a3d50 0x7f3a840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.224+0000 7f3a96a00700 1 -- 172.21.15.69:0/913568520 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3a900d0b20 con 0x7f3aa0102b40 2024-04-24T09:05:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.224+0000 7f3a9d400700 1 --2- 172.21.15.69:0/913568520 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a840a3d50 0x7f3a840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.225+0000 7f3a9d400700 1 --2- 172.21.15.69:0/913568520 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a840a3d50 0x7f3a840a6210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c005950 tx=0x7f3a8c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:42.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.231+0000 7f3a96a00700 1 -- 172.21.15.69:0/913568520 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a90020020 con 0x7f3aa0102b40 2024-04-24T09:05:42.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.324+0000 7f411d000700 1 -- 172.21.15.69:0/3484346425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f411810f550 msgr2=0x7f4118111940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:42.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.324+0000 7f411d000700 1 --2- 172.21.15.69:0/3484346425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f411810f550 0x7f4118111940 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f411000b0a0 tx=0x7f411000b3b0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.324+0000 7f411d000700 1 -- 172.21.15.69:0/3484346425 shutdown_connections 2024-04-24T09:05:42.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.324+0000 7f411d000700 1 --2- 172.21.15.69:0/3484346425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f411810f550 0x7f4118111940 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.324+0000 7f411d000700 1 --2- 172.21.15.69:0/3484346425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4118074d20 0x7f4118075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.324+0000 7f411d000700 1 --2- 172.21.15.69:0/3484346425 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4118075640 0x7f411810f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.324+0000 7f411d000700 1 -- 172.21.15.69:0/3484346425 >> 172.21.15.69:0/3484346425 conn(0x7f4118079c50 msgr2=0x7f411807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:42.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.324+0000 7f411d000700 1 -- 172.21.15.69:0/3484346425 shutdown_connections 2024-04-24T09:05:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.324+0000 7f411d000700 1 -- 172.21.15.69:0/3484346425 wait complete. 2024-04-24T09:05:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.325+0000 7f411d000700 1 Processor -- start 2024-04-24T09:05:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.326+0000 7f411d000700 1 -- start start 2024-04-24T09:05:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.326+0000 7f411d000700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4118074d20 0x7f411819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.326+0000 7f411d000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4118075640 0x7f411819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.326+0000 7f411d000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f411810f550 0x7f411819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.326+0000 7f411d000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4118114ee0 con 0x7f4118075640 2024-04-24T09:05:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.326+0000 7f411d000700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4118114d60 con 0x7f4118074d20 2024-04-24T09:05:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.326+0000 7f411d000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4118115060 con 0x7f411810f550 2024-04-24T09:05:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.326+0000 7f4115400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4118075640 0x7f411819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.326+0000 7f4115400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4118075640 0x7f411819b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58982/0 (socket says 172.21.15.69:58982) 2024-04-24T09:05:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.326+0000 7f4115400700 1 -- 172.21.15.69:0/3843090862 learned_addr learned my addr 172.21.15.69:0/3843090862 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.326+0000 7f4115e00700 1 --2- 172.21.15.69:0/3843090862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4118074d20 0x7f411819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:42.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.327+0000 7f4116800700 1 --2- 172.21.15.69:0/3843090862 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f411810f550 0x7f411819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.327+0000 7f4115400700 1 -- 172.21.15.69:0/3843090862 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f411810f550 msgr2=0x7f411819fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.327+0000 7f4115400700 1 --2- 172.21.15.69:0/3843090862 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f411810f550 0x7f411819fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.327+0000 7f4115400700 1 -- 172.21.15.69:0/3843090862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4118074d20 msgr2=0x7f411819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.327+0000 7f4115400700 1 --2- 172.21.15.69:0/3843090862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4118074d20 0x7f411819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.327+0000 7f4115400700 1 -- 172.21.15.69:0/3843090862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41181a02d0 con 0x7f4118075640 2024-04-24T09:05:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.327+0000 7f4115400700 1 --2- 172.21.15.69:0/3843090862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4118075640 0x7f411819b850 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f410400d980 tx=0x7f410400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.327+0000 7f410ea00700 1 -- 172.21.15.69:0/3843090862 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4104017070 con 0x7f4118075640 2024-04-24T09:05:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.327+0000 7f410ea00700 1 -- 172.21.15.69:0/3843090862 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4104004830 con 0x7f4118075640 2024-04-24T09:05:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.327+0000 7f411d000700 1 -- 172.21.15.69:0/3843090862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41181a05c0 con 0x7f4118075640 2024-04-24T09:05:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.327+0000 7f410ea00700 1 -- 172.21.15.69:0/3843090862 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4104005440 con 0x7f4118075640 2024-04-24T09:05:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.328+0000 7f411d000700 1 -- 172.21.15.69:0/3843090862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41181a7e40 con 0x7f4118075640 2024-04-24T09:05:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.329+0000 7f411d000700 1 -- 172.21.15.69:0/3843090862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41180697e0 con 0x7f4118075640 2024-04-24T09:05:42.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.331+0000 7f410ea00700 1 -- 172.21.15.69:0/3843090862 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f41040055e0 con 0x7f4118075640 2024-04-24T09:05:42.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.332+0000 7f410ea00700 1 --2- 172.21.15.69:0/3843090862 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f40fc0a3cd0 0x7f40fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:42.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.332+0000 7f410ea00700 1 -- 172.21.15.69:0/3843090862 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f41040d0840 con 0x7f4118075640 2024-04-24T09:05:42.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.332+0000 7f4115e00700 1 --2- 172.21.15.69:0/3843090862 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f40fc0a3cd0 0x7f40fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:42.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.333+0000 7f4115e00700 1 --2- 172.21.15.69:0/3843090862 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f40fc0a3cd0 0x7f40fc0a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4108005ea0 tx=0x7f4108005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:42.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.336+0000 7f410ea00700 1 -- 172.21.15.69:0/3843090862 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4104020020 con 0x7f4118075640 2024-04-24T09:05:42.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.466+0000 7f3aa5000700 1 -- 172.21.15.69:0/913568520 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3aa002e1a0 con 0x7f3aa0102b40 2024-04-24T09:05:42.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.467+0000 7f3a96a00700 1 -- 172.21.15.69:0/913568520 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3a9009d0c0 con 0x7f3aa0102b40 2024-04-24T09:05:42.467 INFO:teuthology.orchestra.run.smithi069.stdout:77309411428 2024-04-24T09:05:42.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.470+0000 7f3aa5000700 1 -- 172.21.15.69:0/913568520 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a840a3d50 msgr2=0x7f3a840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:42.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.470+0000 7f3aa5000700 1 --2- 172.21.15.69:0/913568520 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a840a3d50 0x7f3a840a6210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c005950 tx=0x7f3a8c0058e0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.470+0000 7f3aa5000700 1 -- 172.21.15.69:0/913568520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa0102b40 msgr2=0x7f3aa0074df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:42.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.470+0000 7f3aa5000700 1 --2- 172.21.15.69:0/913568520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa0102b40 0x7f3aa0074df0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000d980 tx=0x7f3a9000dd40 comp rx=0 tx=0).stop 2024-04-24T09:05:42.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.470+0000 7f3aa5000700 1 -- 172.21.15.69:0/913568520 shutdown_connections 2024-04-24T09:05:42.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.470+0000 7f3aa5000700 1 --2- 172.21.15.69:0/913568520 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a840a3d50 0x7f3a840a6210 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.470+0000 7f3aa5000700 1 --2- 172.21.15.69:0/913568520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3aa0108b90 0x7f3aa0075870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.470+0000 7f3aa5000700 1 --2- 172.21.15.69:0/913568520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa0102b40 0x7f3aa0074df0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.470+0000 7f3aa5000700 1 --2- 172.21.15.69:0/913568520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3aa0103500 0x7f3aa0075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.470+0000 7f3aa5000700 1 -- 172.21.15.69:0/913568520 >> 172.21.15.69:0/913568520 conn(0x7f3aa0079c50 msgr2=0x7f3aa0101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:42.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.471+0000 7f3aa5000700 1 -- 172.21.15.69:0/913568520 shutdown_connections 2024-04-24T09:05:42.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.471+0000 7f3aa5000700 1 -- 172.21.15.69:0/913568520 wait complete. 2024-04-24T09:05:42.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.590+0000 7f411d000700 1 -- 172.21.15.69:0/3843090862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f41181a8010 con 0x7f4118075640 2024-04-24T09:05:42.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.590+0000 7f410ea00700 1 -- 172.21.15.69:0/3843090862 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f410409cd00 con 0x7f4118075640 2024-04-24T09:05:42.590 INFO:teuthology.orchestra.run.smithi069.stdout:55834574954 2024-04-24T09:05:42.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.593+0000 7f411d000700 1 -- 172.21.15.69:0/3843090862 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f40fc0a3cd0 msgr2=0x7f40fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:42.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.593+0000 7f411d000700 1 --2- 172.21.15.69:0/3843090862 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f40fc0a3cd0 0x7f40fc0a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4108005ea0 tx=0x7f4108005e30 comp rx=0 tx=0).stop 2024-04-24T09:05:42.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.593+0000 7f411d000700 1 -- 172.21.15.69:0/3843090862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4118075640 msgr2=0x7f411819b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:42.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.593+0000 7f411d000700 1 --2- 172.21.15.69:0/3843090862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4118075640 0x7f411819b850 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f410400d980 tx=0x7f410400dd40 comp rx=0 tx=0).stop 2024-04-24T09:05:42.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.593+0000 7f411d000700 1 -- 172.21.15.69:0/3843090862 shutdown_connections 2024-04-24T09:05:42.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.593+0000 7f411d000700 1 --2- 172.21.15.69:0/3843090862 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f40fc0a3cd0 0x7f40fc0a6190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.593+0000 7f411d000700 1 --2- 172.21.15.69:0/3843090862 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f411810f550 0x7f411819fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.593+0000 7f411d000700 1 --2- 172.21.15.69:0/3843090862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4118074d20 0x7f411819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.593+0000 7f411d000700 1 --2- 172.21.15.69:0/3843090862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4118075640 0x7f411819b850 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.593+0000 7f411d000700 1 -- 172.21.15.69:0/3843090862 >> 172.21.15.69:0/3843090862 conn(0x7f4118079c50 msgr2=0x7f4118100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:42.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.593+0000 7f411d000700 1 -- 172.21.15.69:0/3843090862 shutdown_connections 2024-04-24T09:05:42.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:42.593+0000 7f411d000700 1 -- 172.21.15.69:0/3843090862 wait complete. 2024-04-24T09:05:42.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:42 smithi130 ceph-mon[31553]: pgmap v421: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:05:42.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:42 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3718411880' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T09:05:42.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:42 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/913568520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T09:05:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:42 smithi081 ceph-mon[30961]: pgmap v421: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:05:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:42 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3718411880' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T09:05:42.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:42 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/913568520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T09:05:42.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:42 smithi069 ceph-mon[28170]: pgmap v421: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:05:42.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:42 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3718411880' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T09:05:42.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:42 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/913568520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T09:05:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.806+0000 7f8d71000700 1 -- 172.21.15.69:0/3042685797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d6c10a040 msgr2=0x7f8d6c111c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.806+0000 7f8d71000700 1 --2- 172.21.15.69:0/3042685797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d6c10a040 0x7f8d6c111c10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400b0d0 tx=0x7f8d6400b3e0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.807+0000 7f8d71000700 1 -- 172.21.15.69:0/3042685797 shutdown_connections 2024-04-24T09:05:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.807+0000 7f8d71000700 1 --2- 172.21.15.69:0/3042685797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d6c10a040 0x7f8d6c111c10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.807+0000 7f8d71000700 1 --2- 172.21.15.69:0/3042685797 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d6c104f60 0x7f8d6c105340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.807+0000 7f8d71000700 1 --2- 172.21.15.69:0/3042685797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d6c105910 0x7f8d6c109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.807+0000 7f8d71000700 1 -- 172.21.15.69:0/3042685797 >> 172.21.15.69:0/3042685797 conn(0x7f8d6c1007f0 msgr2=0x7f8d6c102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.808+0000 7f8d71000700 1 -- 172.21.15.69:0/3042685797 shutdown_connections 2024-04-24T09:05:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.808+0000 7f8d71000700 1 -- 172.21.15.69:0/3042685797 wait complete. 2024-04-24T09:05:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.809+0000 7f8d71000700 1 Processor -- start 2024-04-24T09:05:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.809+0000 7f8d71000700 1 -- start start 2024-04-24T09:05:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.810+0000 7f8d71000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d6c104f60 0x7f8d6c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.810+0000 7f8d71000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d6c105910 0x7f8d6c19b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.810+0000 7f8d71000700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d6c10a040 0x7f8d6c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.810+0000 7f8d71000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c114f30 con 0x7f8d6c105910 2024-04-24T09:05:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.810+0000 7f8d71000700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c114db0 con 0x7f8d6c10a040 2024-04-24T09:05:42.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.810+0000 7f8d71000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c1150b0 con 0x7f8d6c104f60 2024-04-24T09:05:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.810+0000 7f8d69400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d6c105910 0x7f8d6c19b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.810+0000 7f8d69400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d6c105910 0x7f8d6c19b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58894/0 (socket says 172.21.15.69:58894) 2024-04-24T09:05:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.810+0000 7f8d69400700 1 -- 172.21.15.69:0/114379173 learned_addr learned my addr 172.21.15.69:0/114379173 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.811+0000 7f8d69e00700 1 --2- 172.21.15.69:0/114379173 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d6c104f60 0x7f8d6c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.811+0000 7f8d69400700 1 -- 172.21.15.69:0/114379173 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d6c104f60 msgr2=0x7f8d6c19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.811+0000 7f8d69400700 1 --2- 172.21.15.69:0/114379173 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d6c104f60 0x7f8d6c19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.811+0000 7f8d69400700 1 -- 172.21.15.69:0/114379173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d6c10a040 msgr2=0x7f8d6c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.811+0000 7f8d6a800700 1 --2- 172.21.15.69:0/114379173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d6c10a040 0x7f8d6c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.811+0000 7f8d69400700 1 --2- 172.21.15.69:0/114379173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d6c10a040 0x7f8d6c19fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.811+0000 7f8d69400700 1 -- 172.21.15.69:0/114379173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6c1a03a0 con 0x7f8d6c105910 2024-04-24T09:05:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.811+0000 7f8d69400700 1 --2- 172.21.15.69:0/114379173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d6c105910 0x7f8d6c19b920 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800c8f0 tx=0x7f8d5800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.812+0000 7f8d62a00700 1 -- 172.21.15.69:0/114379173 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58007b00 con 0x7f8d6c105910 2024-04-24T09:05:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.812+0000 7f8d62a00700 1 -- 172.21.15.69:0/114379173 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d58007ca0 con 0x7f8d6c105910 2024-04-24T09:05:42.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.812+0000 7f8d71000700 1 -- 172.21.15.69:0/114379173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d6c1a0690 con 0x7f8d6c105910 2024-04-24T09:05:42.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.812+0000 7f8d62a00700 1 -- 172.21.15.69:0/114379173 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58018660 con 0x7f8d6c105910 2024-04-24T09:05:42.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.812+0000 7f8d71000700 1 -- 172.21.15.69:0/114379173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d6c1a7f10 con 0x7f8d6c105910 2024-04-24T09:05:42.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.813+0000 7f8d71000700 1 -- 172.21.15.69:0/114379173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d6c0697e0 con 0x7f8d6c105910 2024-04-24T09:05:42.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.815+0000 7f8d62a00700 1 -- 172.21.15.69:0/114379173 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8d5800f450 con 0x7f8d6c105910 2024-04-24T09:05:42.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.816+0000 7f8d62a00700 1 --2- 172.21.15.69:0/114379173 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8d500a3cd0 0x7f8d500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:42.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.817+0000 7f8d62a00700 1 -- 172.21.15.69:0/114379173 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8d580d08e0 con 0x7f8d6c105910 2024-04-24T09:05:42.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.818+0000 7f8d69e00700 1 --2- 172.21.15.69:0/114379173 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8d500a3cd0 0x7f8d500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:42.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.818+0000 7f8d69e00700 1 --2- 172.21.15.69:0/114379173 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8d500a3cd0 0x7f8d500a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c005fd0 tx=0x7f8d5c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:42.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:39.823+0000 7f8d62a00700 1 -- 172.21.15.69:0/114379173 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d58014030 con 0x7f8d6c105910 2024-04-24T09:05:42.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.051+0000 7f8d71000700 1 -- 172.21.15.69:0/114379173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f8d6c02dfe0 con 0x7f8d6c105910 2024-04-24T09:05:42.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.051+0000 7f8d62a00700 1 -- 172.21.15.69:0/114379173 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f8d5809cda0 con 0x7f8d6c105910 2024-04-24T09:05:42.991 INFO:teuthology.orchestra.run.smithi069.stdout:313532612651 2024-04-24T09:05:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.054+0000 7f8d71000700 1 -- 172.21.15.69:0/114379173 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8d500a3cd0 msgr2=0x7f8d500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.054+0000 7f8d71000700 1 --2- 172.21.15.69:0/114379173 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8d500a3cd0 0x7f8d500a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c005fd0 tx=0x7f8d5c00a560 comp rx=0 tx=0).stop 2024-04-24T09:05:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.054+0000 7f8d71000700 1 -- 172.21.15.69:0/114379173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d6c105910 msgr2=0x7f8d6c19b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.054+0000 7f8d71000700 1 --2- 172.21.15.69:0/114379173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d6c105910 0x7f8d6c19b920 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800c8f0 tx=0x7f8d5800ccb0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.054+0000 7f8d71000700 1 -- 172.21.15.69:0/114379173 shutdown_connections 2024-04-24T09:05:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.054+0000 7f8d71000700 1 --2- 172.21.15.69:0/114379173 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8d500a3cd0 0x7f8d500a6190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.054+0000 7f8d71000700 1 --2- 172.21.15.69:0/114379173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8d6c10a040 0x7f8d6c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.054+0000 7f8d71000700 1 --2- 172.21.15.69:0/114379173 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8d6c104f60 0x7f8d6c19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.054+0000 7f8d71000700 1 --2- 172.21.15.69:0/114379173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d6c105910 0x7f8d6c19b920 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:42.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.054+0000 7f8d71000700 1 -- 172.21.15.69:0/114379173 >> 172.21.15.69:0/114379173 conn(0x7f8d6c1007f0 msgr2=0x7f8d6c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:42.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.054+0000 7f8d71000700 1 -- 172.21.15.69:0/114379173 shutdown_connections 2024-04-24T09:05:42.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:40.055+0000 7f8d71000700 1 -- 172.21.15.69:0/114379173 wait complete. 2024-04-24T09:05:43.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:43 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3843090862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T09:05:43.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:43 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3843090862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T09:05:43.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:43 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3843090862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T09:05:44.641 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:44 smithi069 ceph-mon[28170]: pgmap v422: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:05:44.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:44 smithi081 ceph-mon[30961]: pgmap v422: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:05:44.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:44 smithi130 ceph-mon[31553]: pgmap v422: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:05:45.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.468+0000 7f9116800700 1 -- 172.21.15.69:0/844306582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9110107a40 msgr2=0x7f9110107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:45.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.468+0000 7f9116800700 1 --2- 172.21.15.69:0/844306582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9110107a40 0x7f9110107e20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9100009990 tx=0x7f9100009ca0 comp rx=0 tx=0).stop 2024-04-24T09:05:45.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.468+0000 7f9116800700 1 -- 172.21.15.69:0/844306582 shutdown_connections 2024-04-24T09:05:45.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.468+0000 7f9116800700 1 --2- 172.21.15.69:0/844306582 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f91101023b0 0x7f911010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:45.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.468+0000 7f9116800700 1 --2- 172.21.15.69:0/844306582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9110107a40 0x7f9110107e20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:45.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.468+0000 7f9116800700 1 --2- 172.21.15.69:0/844306582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91101019f0 0x7f9110101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:45.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.468+0000 7f9116800700 1 -- 172.21.15.69:0/844306582 >> 172.21.15.69:0/844306582 conn(0x7f91100789a0 msgr2=0x7f9110078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:45.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.468+0000 7f9116800700 1 -- 172.21.15.69:0/844306582 shutdown_connections 2024-04-24T09:05:45.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.468+0000 7f9116800700 1 -- 172.21.15.69:0/844306582 wait complete. 2024-04-24T09:05:45.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.469+0000 7f9116800700 1 Processor -- start 2024-04-24T09:05:45.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.469+0000 7f9116800700 1 -- start start 2024-04-24T09:05:45.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.469+0000 7f9116800700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91101019f0 0x7f9110199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:45.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.470+0000 7f9116800700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f91101023b0 0x7f9110199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:45.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.470+0000 7f9116800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9110107a40 0x7f911019d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:45.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.470+0000 7f9116800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9110111ba0 con 0x7f9110107a40 2024-04-24T09:05:45.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.470+0000 7f9116800700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9110111a20 con 0x7f91101019f0 2024-04-24T09:05:45.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.470+0000 7f9116800700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9110111d20 con 0x7f91101023b0 2024-04-24T09:05:45.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.470+0000 7f910fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9110107a40 0x7f911019d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:45.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.470+0000 7f910fe00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9110107a40 0x7f911019d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34000/0 (socket says 172.21.15.69:34000) 2024-04-24T09:05:45.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.470+0000 7f910fe00700 1 -- 172.21.15.69:0/2363580751 learned_addr learned my addr 172.21.15.69:0/2363580751 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:45.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.470+0000 7f910f400700 1 --2- 172.21.15.69:0/2363580751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91101019f0 0x7f9110199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:45.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.470+0000 7f910ea00700 1 --2- 172.21.15.69:0/2363580751 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f91101023b0 0x7f9110199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:45.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.470+0000 7f910fe00700 1 -- 172.21.15.69:0/2363580751 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f91101023b0 msgr2=0x7f9110199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:45.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.470+0000 7f910fe00700 1 --2- 172.21.15.69:0/2363580751 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f91101023b0 0x7f9110199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:45.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.470+0000 7f910fe00700 1 -- 172.21.15.69:0/2363580751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91101019f0 msgr2=0x7f9110199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:45.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.470+0000 7f910fe00700 1 --2- 172.21.15.69:0/2363580751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91101019f0 0x7f9110199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:45.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.470+0000 7f910fe00700 1 -- 172.21.15.69:0/2363580751 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911019e130 con 0x7f9110107a40 2024-04-24T09:05:45.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.471+0000 7f910fe00700 1 --2- 172.21.15.69:0/2363580751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9110107a40 0x7f911019d9f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9108007ee0 tx=0x7f910800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:45.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.471+0000 7f9107e00700 1 -- 172.21.15.69:0/2363580751 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910800dbd0 con 0x7f9110107a40 2024-04-24T09:05:45.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.471+0000 7f9107e00700 1 -- 172.21.15.69:0/2363580751 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f910800dd70 con 0x7f9110107a40 2024-04-24T09:05:45.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.471+0000 7f9116800700 1 -- 172.21.15.69:0/2363580751 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911019e420 con 0x7f9110107a40 2024-04-24T09:05:45.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.471+0000 7f9107e00700 1 -- 172.21.15.69:0/2363580751 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108005a20 con 0x7f9110107a40 2024-04-24T09:05:45.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.471+0000 7f9116800700 1 -- 172.21.15.69:0/2363580751 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91101a5ca0 con 0x7f9110107a40 2024-04-24T09:05:45.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.473+0000 7f9116800700 1 -- 172.21.15.69:0/2363580751 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91100697e0 con 0x7f9110107a40 2024-04-24T09:05:45.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.474+0000 7f9107e00700 1 -- 172.21.15.69:0/2363580751 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f910801a020 con 0x7f9110107a40 2024-04-24T09:05:45.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.476+0000 7f9107e00700 1 --2- 172.21.15.69:0/2363580751 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f90f40a3cd0 0x7f90f40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:45.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.476+0000 7f9107e00700 1 -- 172.21.15.69:0/2363580751 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f91080d00b0 con 0x7f9110107a40 2024-04-24T09:05:45.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.476+0000 7f910f400700 1 --2- 172.21.15.69:0/2363580751 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f90f40a3cd0 0x7f90f40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:45.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.477+0000 7f910f400700 1 --2- 172.21.15.69:0/2363580751 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f90f40a3cd0 0x7f90f40a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9100006010 tx=0x7f910001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:45.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.481+0000 7f9107e00700 1 -- 172.21.15.69:0/2363580751 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9108010070 con 0x7f9110107a40 2024-04-24T09:05:45.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.719+0000 7f9116800700 1 -- 172.21.15.69:0/2363580751 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f91101a5ea0 con 0x7f9110107a40 2024-04-24T09:05:45.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.720+0000 7f9107e00700 1 -- 172.21.15.69:0/2363580751 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f910809c570 con 0x7f9110107a40 2024-04-24T09:05:45.720 INFO:teuthology.orchestra.run.smithi069.stdout:163208757328 2024-04-24T09:05:45.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.723+0000 7f9116800700 1 -- 172.21.15.69:0/2363580751 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f90f40a3cd0 msgr2=0x7f90f40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:45.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.723+0000 7f9116800700 1 --2- 172.21.15.69:0/2363580751 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f90f40a3cd0 0x7f90f40a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9100006010 tx=0x7f910001a040 comp rx=0 tx=0).stop 2024-04-24T09:05:45.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.723+0000 7f9116800700 1 -- 172.21.15.69:0/2363580751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9110107a40 msgr2=0x7f911019d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:45.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.723+0000 7f9116800700 1 --2- 172.21.15.69:0/2363580751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9110107a40 0x7f911019d9f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9108007ee0 tx=0x7f910800d3b0 comp rx=0 tx=0).stop 2024-04-24T09:05:45.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.723+0000 7f9116800700 1 -- 172.21.15.69:0/2363580751 shutdown_connections 2024-04-24T09:05:45.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.723+0000 7f9116800700 1 --2- 172.21.15.69:0/2363580751 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f90f40a3cd0 0x7f90f40a6190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:45.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.723+0000 7f9116800700 1 --2- 172.21.15.69:0/2363580751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9110107a40 0x7f911019d9f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:45.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.724+0000 7f9116800700 1 --2- 172.21.15.69:0/2363580751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91101019f0 0x7f9110199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:45.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.724+0000 7f9116800700 1 --2- 172.21.15.69:0/2363580751 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f91101023b0 0x7f9110199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:45.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.724+0000 7f9116800700 1 -- 172.21.15.69:0/2363580751 >> 172.21.15.69:0/2363580751 conn(0x7f91100789a0 msgr2=0x7f9110100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:45.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.724+0000 7f9116800700 1 -- 172.21.15.69:0/2363580751 shutdown_connections 2024-04-24T09:05:45.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:45.724+0000 7f9116800700 1 -- 172.21.15.69:0/2363580751 wait complete. 2024-04-24T09:05:45.936 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495684 got 197568495688 for osd.6 2024-04-24T09:05:45.936 DEBUG:teuthology.parallel:result is None 2024-04-24T09:05:46.642 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612644 got 313532612651 for osd.10 2024-04-24T09:05:46.643 DEBUG:teuthology.parallel:result is None 2024-04-24T09:05:46.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.666+0000 7fb8d4e00700 1 -- 172.21.15.69:0/1101563393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8d0101780 msgr2=0x7fb8d0101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:46.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.666+0000 7fb8d4e00700 1 --2- 172.21.15.69:0/1101563393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8d0101780 0x7fb8d0101b60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009990 tx=0x7fb8c0009ca0 comp rx=0 tx=0).stop 2024-04-24T09:05:46.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.666+0000 7fb8d4e00700 1 -- 172.21.15.69:0/1101563393 shutdown_connections 2024-04-24T09:05:46.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.666+0000 7fb8d4e00700 1 --2- 172.21.15.69:0/1101563393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8d010f300 0x7fb8d01117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:46.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.666+0000 7fb8d4e00700 1 --2- 172.21.15.69:0/1101563393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8d0101780 0x7fb8d0101b60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.666+0000 7fb8d4e00700 1 --2- 172.21.15.69:0/1101563393 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8d0102130 0x7fb8d010ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.666+0000 7fb8d4e00700 1 -- 172.21.15.69:0/1101563393 >> 172.21.15.69:0/1101563393 conn(0x7fb8d0076c60 msgr2=0x7fb8d0077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.666+0000 7fb8d4e00700 1 -- 172.21.15.69:0/1101563393 shutdown_connections 2024-04-24T09:05:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.666+0000 7fb8d4e00700 1 -- 172.21.15.69:0/1101563393 wait complete. 2024-04-24T09:05:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.667+0000 7fb8d4e00700 1 Processor -- start 2024-04-24T09:05:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.668+0000 7fb8d4e00700 1 -- start start 2024-04-24T09:05:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.668+0000 7fb8d4e00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8d0101780 0x7fb8d019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.668+0000 7fb8d4e00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8d0102130 0x7fb8d019b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.668+0000 7fb8d4e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8d010f300 0x7fb8d019fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.668+0000 7fb8d4e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0114ed0 con 0x7fb8d010f300 2024-04-24T09:05:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.668+0000 7fb8d4e00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0114d50 con 0x7fb8d0101780 2024-04-24T09:05:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.668+0000 7fb8d4e00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0115050 con 0x7fb8d0102130 2024-04-24T09:05:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.668+0000 7fb8ce800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8d010f300 0x7fb8d019fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.668+0000 7fb8ce800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8d010f300 0x7fb8d019fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34018/0 (socket says 172.21.15.69:34018) 2024-04-24T09:05:46.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.668+0000 7fb8ce800700 1 -- 172.21.15.69:0/3667885757 learned_addr learned my addr 172.21.15.69:0/3667885757 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:46.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.669+0000 7fb8cd400700 1 --2- 172.21.15.69:0/3667885757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8d0102130 0x7fb8d019b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:46.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.669+0000 7fb8cde00700 1 --2- 172.21.15.69:0/3667885757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8d0101780 0x7fb8d019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:46.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.669+0000 7fb8ce800700 1 -- 172.21.15.69:0/3667885757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8d0102130 msgr2=0x7fb8d019b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:46.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.669+0000 7fb8ce800700 1 --2- 172.21.15.69:0/3667885757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8d0102130 0x7fb8d019b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:46.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.669+0000 7fb8ce800700 1 -- 172.21.15.69:0/3667885757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8d0101780 msgr2=0x7fb8d019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:46.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.669+0000 7fb8ce800700 1 --2- 172.21.15.69:0/3667885757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8d0101780 0x7fb8d019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:46.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.669+0000 7fb8ce800700 1 -- 172.21.15.69:0/3667885757 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d01a0310 con 0x7fb8d010f300 2024-04-24T09:05:46.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.669+0000 7fb8ce800700 1 --2- 172.21.15.69:0/3667885757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8d010f300 0x7fb8d019fbd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8007d70 tx=0x7fb8c800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:46.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.669+0000 7fb8c6a00700 1 -- 172.21.15.69:0/3667885757 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8016020 con 0x7fb8d010f300 2024-04-24T09:05:46.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.669+0000 7fb8c6a00700 1 -- 172.21.15.69:0/3667885757 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8c800ad40 con 0x7fb8d010f300 2024-04-24T09:05:46.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.669+0000 7fb8d4e00700 1 -- 172.21.15.69:0/3667885757 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d01a0600 con 0x7fb8d010f300 2024-04-24T09:05:46.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.669+0000 7fb8c6a00700 1 -- 172.21.15.69:0/3667885757 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8011620 con 0x7fb8d010f300 2024-04-24T09:05:46.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.670+0000 7fb8d4e00700 1 -- 172.21.15.69:0/3667885757 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d01a7e80 con 0x7fb8d010f300 2024-04-24T09:05:46.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.671+0000 7fb8d4e00700 1 -- 172.21.15.69:0/3667885757 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d00697e0 con 0x7fb8d010f300 2024-04-24T09:05:46.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.673+0000 7fb8c6a00700 1 -- 172.21.15.69:0/3667885757 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb8c8019020 con 0x7fb8d010f300 2024-04-24T09:05:46.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.674+0000 7fb8c6a00700 1 --2- 172.21.15.69:0/3667885757 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb8b40a3d50 0x7fb8b40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:46.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.674+0000 7fb8c6a00700 1 -- 172.21.15.69:0/3667885757 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb8c80d0bc0 con 0x7fb8d010f300 2024-04-24T09:05:46.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.675+0000 7fb8cde00700 1 --2- 172.21.15.69:0/3667885757 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb8b40a3d50 0x7fb8b40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:46.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.675+0000 7fb8cde00700 1 --2- 172.21.15.69:0/3667885757 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb8b40a3d50 0x7fb8b40a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000b5c0 tx=0x7fb8c0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:46.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.676+0000 7fb8c6a00700 1 -- 172.21.15.69:0/3667885757 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8c8020020 con 0x7fb8d010f300 2024-04-24T09:05:46.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:46 smithi081 ceph-mon[30961]: pgmap v423: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:05:46.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:46 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2363580751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T09:05:46.840 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:46 smithi069 ceph-mon[28170]: pgmap v423: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:05:46.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:46 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2363580751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T09:05:46.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.911+0000 7fb8d4e00700 1 -- 172.21.15.69:0/3667885757 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb8d002dfe0 con 0x7fb8d010f300 2024-04-24T09:05:46.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.911+0000 7fb8c6a00700 1 -- 172.21.15.69:0/3667885757 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb8c809d190 con 0x7fb8d010f300 2024-04-24T09:05:46.911 INFO:teuthology.orchestra.run.smithi069.stdout:219043332162 2024-04-24T09:05:46.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.911+0000 7fb8d4e00700 1 -- 172.21.15.69:0/3667885757 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb8b40a3d50 msgr2=0x7fb8b40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:46.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.911+0000 7fb8d4e00700 1 --2- 172.21.15.69:0/3667885757 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb8b40a3d50 0x7fb8b40a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000b5c0 tx=0x7fb8c0005fb0 comp rx=0 tx=0).stop 2024-04-24T09:05:46.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.911+0000 7fb8d4e00700 1 -- 172.21.15.69:0/3667885757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8d010f300 msgr2=0x7fb8d019fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:46.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.911+0000 7fb8d4e00700 1 --2- 172.21.15.69:0/3667885757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8d010f300 0x7fb8d019fbd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8007d70 tx=0x7fb8c800a340 comp rx=0 tx=0).stop 2024-04-24T09:05:46.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.911+0000 7fb8d4e00700 1 -- 172.21.15.69:0/3667885757 shutdown_connections 2024-04-24T09:05:46.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.911+0000 7fb8d4e00700 1 --2- 172.21.15.69:0/3667885757 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb8b40a3d50 0x7fb8b40a6210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:46.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.911+0000 7fb8d4e00700 1 --2- 172.21.15.69:0/3667885757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8d010f300 0x7fb8d019fbd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:46.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.911+0000 7fb8d4e00700 1 --2- 172.21.15.69:0/3667885757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8d0101780 0x7fb8d019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:46.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.911+0000 7fb8d4e00700 1 --2- 172.21.15.69:0/3667885757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8d0102130 0x7fb8d019b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:46.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.911+0000 7fb8d4e00700 1 -- 172.21.15.69:0/3667885757 >> 172.21.15.69:0/3667885757 conn(0x7fb8d0076c60 msgr2=0x7fb8d010f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:46.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.911+0000 7fb8d4e00700 1 -- 172.21.15.69:0/3667885757 shutdown_connections 2024-04-24T09:05:46.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:46.911+0000 7fb8d4e00700 1 -- 172.21.15.69:0/3667885757 wait complete. 2024-04-24T09:05:46.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:46 smithi130 ceph-mon[31553]: pgmap v423: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:05:46.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:46 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2363580751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T09:05:47.259 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574949 got 55834574954 for osd.1 2024-04-24T09:05:47.259 DEBUG:teuthology.parallel:result is None 2024-04-24T09:05:47.812 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:47 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3667885757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T09:05:47.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:47 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3667885757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T09:05:47.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:47 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3667885757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T09:05:48.523 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411422 got 77309411428 for osd.2 2024-04-24T09:05:48.524 DEBUG:teuthology.parallel:result is None 2024-04-24T09:05:48.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:48 smithi081 ceph-mon[30961]: pgmap v424: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:05:48.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:48 smithi069 ceph-mon[28170]: pgmap v424: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:05:48.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:48 smithi130 ceph-mon[31553]: pgmap v424: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:05:49.551 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332156 got 219043332162 for osd.7 2024-04-24T09:05:49.552 DEBUG:teuthology.parallel:result is None 2024-04-24T09:05:49.898 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757322 got 163208757328 for osd.5 2024-04-24T09:05:49.899 DEBUG:teuthology.parallel:result is None 2024-04-24T09:05:50.739 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:50 smithi069 ceph-mon[28170]: pgmap v425: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-24T09:05:50.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.801+0000 7f9ff2600700 1 -- 172.21.15.69:0/3643803424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fec102b40 msgr2=0x7f9fec102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:50.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.801+0000 7f9ff2600700 1 --2- 172.21.15.69:0/3643803424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fec102b40 0x7f9fec102fc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009990 tx=0x7f9fd4009ca0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.801+0000 7f9ff2600700 1 -- 172.21.15.69:0/3643803424 shutdown_connections 2024-04-24T09:05:50.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.801+0000 7f9ff2600700 1 --2- 172.21.15.69:0/3643803424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fec103500 0x7f9fec10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.801+0000 7f9ff2600700 1 --2- 172.21.15.69:0/3643803424 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9fec108b90 0x7f9fec108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.801+0000 7f9ff2600700 1 --2- 172.21.15.69:0/3643803424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fec102b40 0x7f9fec102fc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.801+0000 7f9ff2600700 1 -- 172.21.15.69:0/3643803424 >> 172.21.15.69:0/3643803424 conn(0x7f9fec079c50 msgr2=0x7f9fec07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:50.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.802+0000 7f9ff2600700 1 -- 172.21.15.69:0/3643803424 shutdown_connections 2024-04-24T09:05:50.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.802+0000 7f9ff2600700 1 -- 172.21.15.69:0/3643803424 wait complete. 2024-04-24T09:05:50.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.803+0000 7f9ff2600700 1 Processor -- start 2024-04-24T09:05:50.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.803+0000 7f9ff2600700 1 -- start start 2024-04-24T09:05:50.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.803+0000 7f9ff2600700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9fec102b40 0x7f9fec074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:50.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.803+0000 7f9ff2600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fec103500 0x7f9fec075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:50.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.803+0000 7f9ff2600700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fec108b90 0x7f9fec075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:50.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.803+0000 7f9ff2600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec112ca0 con 0x7f9fec103500 2024-04-24T09:05:50.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.803+0000 7f9ff2600700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec112b20 con 0x7f9fec108b90 2024-04-24T09:05:50.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.803+0000 7f9ff2600700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec112e20 con 0x7f9fec102b40 2024-04-24T09:05:50.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.803+0000 7f9fe2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fec103500 0x7f9fec075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:50.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.803+0000 7f9fe2a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fec103500 0x7f9fec075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34042/0 (socket says 172.21.15.69:34042) 2024-04-24T09:05:50.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.803+0000 7f9fe2a00700 1 -- 172.21.15.69:0/4050099496 learned_addr learned my addr 172.21.15.69:0/4050099496 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:50.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.804+0000 7f9feb400700 1 --2- 172.21.15.69:0/4050099496 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9fec102b40 0x7f9fec074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:50.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.804+0000 7f9fe2a00700 1 -- 172.21.15.69:0/4050099496 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9fec102b40 msgr2=0x7f9fec074df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:50.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.804+0000 7f9febe00700 1 --2- 172.21.15.69:0/4050099496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fec108b90 0x7f9fec075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:50.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.804+0000 7f9fe2a00700 1 --2- 172.21.15.69:0/4050099496 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9fec102b40 0x7f9fec074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.804+0000 7f9fe2a00700 1 -- 172.21.15.69:0/4050099496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fec108b90 msgr2=0x7f9fec075870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:50.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.804+0000 7f9fe2a00700 1 --2- 172.21.15.69:0/4050099496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fec108b90 0x7f9fec075870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.804+0000 7f9fe2a00700 1 -- 172.21.15.69:0/4050099496 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fec1a7640 con 0x7f9fec103500 2024-04-24T09:05:50.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.804+0000 7f9febe00700 1 --2- 172.21.15.69:0/4050099496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fec108b90 0x7f9fec075870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:50.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.804+0000 7f9fe2a00700 1 --2- 172.21.15.69:0/4050099496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fec103500 0x7f9fec075330 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000c00 tx=0x7f9fd4003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:50.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.804+0000 7f9fe8c00700 1 -- 172.21.15.69:0/4050099496 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4027070 con 0x7f9fec103500 2024-04-24T09:05:50.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.805+0000 7f9fe8c00700 1 -- 172.21.15.69:0/4050099496 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fd400bdc0 con 0x7f9fec103500 2024-04-24T09:05:50.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.805+0000 7f9fe8c00700 1 -- 172.21.15.69:0/4050099496 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd400f6c0 con 0x7f9fec103500 2024-04-24T09:05:50.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.805+0000 7f9ff2600700 1 -- 172.21.15.69:0/4050099496 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fec1a78d0 con 0x7f9fec103500 2024-04-24T09:05:50.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.805+0000 7f9ff2600700 1 -- 172.21.15.69:0/4050099496 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fec1a7d60 con 0x7f9fec103500 2024-04-24T09:05:50.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.806+0000 7f9ff2600700 1 -- 172.21.15.69:0/4050099496 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc8005330 con 0x7f9fec103500 2024-04-24T09:05:50.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.808+0000 7f9fe8c00700 1 -- 172.21.15.69:0/4050099496 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9fd402a070 con 0x7f9fec103500 2024-04-24T09:05:50.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.809+0000 7f9fe8c00700 1 --2- 172.21.15.69:0/4050099496 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9fd00a3d50 0x7f9fd00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:50.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.809+0000 7f9fe8c00700 1 -- 172.21.15.69:0/4050099496 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9fd40d1950 con 0x7f9fec103500 2024-04-24T09:05:50.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.809+0000 7f9feb400700 1 --2- 172.21.15.69:0/4050099496 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9fd00a3d50 0x7f9fd00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:50.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.810+0000 7f9feb400700 1 --2- 172.21.15.69:0/4050099496 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9fd00a3d50 0x7f9fd00a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc005fd0 tx=0x7f9fdc005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:50.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:50.814+0000 7f9fe8c00700 1 -- 172.21.15.69:0/4050099496 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9fd405eff0 con 0x7f9fec103500 2024-04-24T09:05:50.816 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:50 smithi081 ceph-mon[30961]: pgmap v425: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-24T09:05:50.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:50 smithi130 ceph-mon[31553]: pgmap v425: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-24T09:05:51.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.049+0000 7f9ff2600700 1 -- 172.21.15.69:0/4050099496 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9fc80058f0 con 0x7f9fec103500 2024-04-24T09:05:51.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.049+0000 7f9fe8c00700 1 -- 172.21.15.69:0/4050099496 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9fd4013070 con 0x7f9fec103500 2024-04-24T09:05:51.049 INFO:teuthology.orchestra.run.smithi069.stdout:34359738482 2024-04-24T09:05:51.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.052+0000 7f9ff2600700 1 -- 172.21.15.69:0/4050099496 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9fd00a3d50 msgr2=0x7f9fd00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.052+0000 7f9ff2600700 1 --2- 172.21.15.69:0/4050099496 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9fd00a3d50 0x7f9fd00a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc005fd0 tx=0x7f9fdc005ce0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.052+0000 7f9ff2600700 1 -- 172.21.15.69:0/4050099496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fec103500 msgr2=0x7f9fec075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.052+0000 7f9ff2600700 1 --2- 172.21.15.69:0/4050099496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fec103500 0x7f9fec075330 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000c00 tx=0x7f9fd4003b30 comp rx=0 tx=0).stop 2024-04-24T09:05:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.053+0000 7f9ff2600700 1 -- 172.21.15.69:0/4050099496 shutdown_connections 2024-04-24T09:05:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.053+0000 7f9ff2600700 1 --2- 172.21.15.69:0/4050099496 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9fd00a3d50 0x7f9fd00a6210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.053+0000 7f9ff2600700 1 --2- 172.21.15.69:0/4050099496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fec108b90 0x7f9fec075870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.053+0000 7f9ff2600700 1 --2- 172.21.15.69:0/4050099496 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9fec102b40 0x7f9fec074df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.053+0000 7f9ff2600700 1 --2- 172.21.15.69:0/4050099496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fec103500 0x7f9fec075330 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.053+0000 7f9ff2600700 1 -- 172.21.15.69:0/4050099496 >> 172.21.15.69:0/4050099496 conn(0x7f9fec079c50 msgr2=0x7f9fec101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.053+0000 7f9ff2600700 1 -- 172.21.15.69:0/4050099496 shutdown_connections 2024-04-24T09:05:51.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.053+0000 7f9ff2600700 1 -- 172.21.15.69:0/4050099496 wait complete. 2024-04-24T09:05:51.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.377+0000 7f8e94000700 1 -- 172.21.15.69:0/3765173494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8e8c102b40 msgr2=0x7f8e8c102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.377+0000 7f8e94000700 1 --2- 172.21.15.69:0/3765173494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8e8c102b40 0x7f8e8c102fc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009990 tx=0x7f8e78009ca0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.377+0000 7f8e94000700 1 -- 172.21.15.69:0/3765173494 shutdown_connections 2024-04-24T09:05:51.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.377+0000 7f8e94000700 1 --2- 172.21.15.69:0/3765173494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e8c103500 0x7f8e8c10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.377+0000 7f8e94000700 1 --2- 172.21.15.69:0/3765173494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e8c108b90 0x7f8e8c108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.377+0000 7f8e94000700 1 --2- 172.21.15.69:0/3765173494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8e8c102b40 0x7f8e8c102fc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.377+0000 7f8e94000700 1 -- 172.21.15.69:0/3765173494 >> 172.21.15.69:0/3765173494 conn(0x7f8e8c079c50 msgr2=0x7f8e8c07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:51.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.377+0000 7f8e94000700 1 -- 172.21.15.69:0/3765173494 shutdown_connections 2024-04-24T09:05:51.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.377+0000 7f8e94000700 1 -- 172.21.15.69:0/3765173494 wait complete. 2024-04-24T09:05:51.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.378+0000 7f8e94000700 1 Processor -- start 2024-04-24T09:05:51.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.378+0000 7f8e94000700 1 -- start start 2024-04-24T09:05:51.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.379+0000 7f8e94000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e8c102b40 0x7f8e8c074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.379+0000 7f8e94000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8e8c103500 0x7f8e8c075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.379+0000 7f8e94000700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e8c108b90 0x7f8e8c075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.379+0000 7f8e94000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c112cc0 con 0x7f8e8c103500 2024-04-24T09:05:51.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.379+0000 7f8e94000700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c112b40 con 0x7f8e8c108b90 2024-04-24T09:05:51.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.379+0000 7f8e94000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c112e40 con 0x7f8e8c102b40 2024-04-24T09:05:51.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.379+0000 7f8e90c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8e8c103500 0x7f8e8c075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.379+0000 7f8e90c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8e8c103500 0x7f8e8c075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34082/0 (socket says 172.21.15.69:34082) 2024-04-24T09:05:51.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.379+0000 7f8e90c00700 1 -- 172.21.15.69:0/1594314326 learned_addr learned my addr 172.21.15.69:0/1594314326 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:51.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.379+0000 7f8e92000700 1 --2- 172.21.15.69:0/1594314326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e8c108b90 0x7f8e8c075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.380+0000 7f8e90c00700 1 -- 172.21.15.69:0/1594314326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e8c102b40 msgr2=0x7f8e8c074df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.380+0000 7f8e91600700 1 --2- 172.21.15.69:0/1594314326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e8c102b40 0x7f8e8c074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.380+0000 7f8e90c00700 1 --2- 172.21.15.69:0/1594314326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e8c102b40 0x7f8e8c074df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.380+0000 7f8e90c00700 1 -- 172.21.15.69:0/1594314326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e8c108b90 msgr2=0x7f8e8c075870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.380+0000 7f8e90c00700 1 --2- 172.21.15.69:0/1594314326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e8c108b90 0x7f8e8c075870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.380+0000 7f8e90c00700 1 -- 172.21.15.69:0/1594314326 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8c1a7760 con 0x7f8e8c103500 2024-04-24T09:05:51.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.380+0000 7f8e92000700 1 --2- 172.21.15.69:0/1594314326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e8c108b90 0x7f8e8c075870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:51.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.380+0000 7f8e91600700 1 --2- 172.21.15.69:0/1594314326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e8c102b40 0x7f8e8c074df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:51.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.380+0000 7f8e90c00700 1 --2- 172.21.15.69:0/1594314326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8e8c103500 0x7f8e8c075330 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8e78003ab0 tx=0x7f8e78003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:51.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.380+0000 7f8e8a000700 1 -- 172.21.15.69:0/1594314326 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78027070 con 0x7f8e8c103500 2024-04-24T09:05:51.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.380+0000 7f8e8a000700 1 -- 172.21.15.69:0/1594314326 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e7800bdc0 con 0x7f8e8c103500 2024-04-24T09:05:51.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.380+0000 7f8e94000700 1 -- 172.21.15.69:0/1594314326 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8c1a79f0 con 0x7f8e8c103500 2024-04-24T09:05:51.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.381+0000 7f8e8a000700 1 -- 172.21.15.69:0/1594314326 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78022db0 con 0x7f8e8c103500 2024-04-24T09:05:51.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.381+0000 7f8e94000700 1 -- 172.21.15.69:0/1594314326 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8c1a7ed0 con 0x7f8e8c103500 2024-04-24T09:05:51.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.382+0000 7f8e94000700 1 -- 172.21.15.69:0/1594314326 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e8c0697e0 con 0x7f8e8c103500 2024-04-24T09:05:51.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.384+0000 7f8e8a000700 1 -- 172.21.15.69:0/1594314326 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8e7802a070 con 0x7f8e8c103500 2024-04-24T09:05:51.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.386+0000 7f8e8a000700 1 --2- 172.21.15.69:0/1594314326 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8e740a3cd0 0x7f8e740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.386+0000 7f8e8a000700 1 -- 172.21.15.69:0/1594314326 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8e780d1610 con 0x7f8e8c103500 2024-04-24T09:05:51.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.386+0000 7f8e91600700 1 --2- 172.21.15.69:0/1594314326 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8e740a3cd0 0x7f8e740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.387+0000 7f8e91600700 1 --2- 172.21.15.69:0/1594314326 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8e740a3cd0 0x7f8e740a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8e80005fd0 tx=0x7f8e80005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:51.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.389+0000 7f8e8a000700 1 -- 172.21.15.69:0/1594314326 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e78013070 con 0x7f8e8c103500 2024-04-24T09:05:51.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.253+0000 7faecc000700 1 -- 172.21.15.69:0/1450951254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faec410a070 msgr2=0x7faec4111c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.253+0000 7faecc000700 1 --2- 172.21.15.69:0/1450951254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faec410a070 0x7faec4111c40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7faebc00b0d0 tx=0x7faebc00b3e0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.253+0000 7faecc000700 1 -- 172.21.15.69:0/1450951254 shutdown_connections 2024-04-24T09:05:51.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.253+0000 7faecc000700 1 --2- 172.21.15.69:0/1450951254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faec410a070 0x7faec4111c40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.253+0000 7faecc000700 1 --2- 172.21.15.69:0/1450951254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faec4104f90 0x7faec4105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.253+0000 7faecc000700 1 --2- 172.21.15.69:0/1450951254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faec4105940 0x7faec4109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.253+0000 7faecc000700 1 -- 172.21.15.69:0/1450951254 >> 172.21.15.69:0/1450951254 conn(0x7faec4100820 msgr2=0x7faec4102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:51.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.253+0000 7faecc000700 1 -- 172.21.15.69:0/1450951254 shutdown_connections 2024-04-24T09:05:51.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.253+0000 7faecc000700 1 -- 172.21.15.69:0/1450951254 wait complete. 2024-04-24T09:05:51.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.254+0000 7faecc000700 1 Processor -- start 2024-04-24T09:05:51.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.254+0000 7faecc000700 1 -- start start 2024-04-24T09:05:51.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.255+0000 7faecc000700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faec4104f90 0x7faec419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.255+0000 7faecc000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faec4105940 0x7faec419b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.255+0000 7faecc000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faec410a070 0x7faec419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.255+0000 7faecc000700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faec4114f10 con 0x7faec410a070 2024-04-24T09:05:51.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.255+0000 7faecc000700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7faec4114d90 con 0x7faec4104f90 2024-04-24T09:05:51.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.255+0000 7faecc000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7faec4115090 con 0x7faec4105940 2024-04-24T09:05:51.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.255+0000 7faeca000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faec410a070 0x7faec419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.255+0000 7faeca000700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faec410a070 0x7faec419fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34072/0 (socket says 172.21.15.69:34072) 2024-04-24T09:05:51.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.255+0000 7faeca000700 1 -- 172.21.15.69:0/3223676887 learned_addr learned my addr 172.21.15.69:0/3223676887 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:51.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.255+0000 7faec9600700 1 --2- 172.21.15.69:0/3223676887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faec4104f90 0x7faec419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.256+0000 7faec8c00700 1 --2- 172.21.15.69:0/3223676887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faec4105940 0x7faec419b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.256+0000 7faeca000700 1 -- 172.21.15.69:0/3223676887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faec4105940 msgr2=0x7faec419b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.256+0000 7faeca000700 1 --2- 172.21.15.69:0/3223676887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faec4105940 0x7faec419b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.256+0000 7faeca000700 1 -- 172.21.15.69:0/3223676887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faec4104f90 msgr2=0x7faec419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.256+0000 7faeca000700 1 --2- 172.21.15.69:0/3223676887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faec4104f90 0x7faec419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.256+0000 7faeca000700 1 -- 172.21.15.69:0/3223676887 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec41a0300 con 0x7faec410a070 2024-04-24T09:05:51.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.256+0000 7faeca000700 1 --2- 172.21.15.69:0/3223676887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faec410a070 0x7faec419fbc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faebc009fd0 tx=0x7faebc007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:51.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.256+0000 7faec2000700 1 -- 172.21.15.69:0/3223676887 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc01f070 con 0x7faec410a070 2024-04-24T09:05:51.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.257+0000 7faec2000700 1 -- 172.21.15.69:0/3223676887 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faebc003c00 con 0x7faec410a070 2024-04-24T09:05:51.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.257+0000 7faecc000700 1 -- 172.21.15.69:0/3223676887 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec41a0590 con 0x7faec410a070 2024-04-24T09:05:51.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.257+0000 7faec2000700 1 -- 172.21.15.69:0/3223676887 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc01b820 con 0x7faec410a070 2024-04-24T09:05:51.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.257+0000 7faecc000700 1 -- 172.21.15.69:0/3223676887 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec41a7e70 con 0x7faec410a070 2024-04-24T09:05:51.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.258+0000 7faecc000700 1 -- 172.21.15.69:0/3223676887 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec40697e0 con 0x7faec410a070 2024-04-24T09:05:51.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.260+0000 7faec2000700 1 -- 172.21.15.69:0/3223676887 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faebc022070 con 0x7faec410a070 2024-04-24T09:05:51.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.261+0000 7faec2000700 1 --2- 172.21.15.69:0/3223676887 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7faeac0a3d50 0x7faeac0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.261+0000 7faec2000700 1 -- 172.21.15.69:0/3223676887 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7faebc00f020 con 0x7faec410a070 2024-04-24T09:05:51.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.262+0000 7faec9600700 1 --2- 172.21.15.69:0/3223676887 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7faeac0a3d50 0x7faeac0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.262+0000 7faec9600700 1 --2- 172.21.15.69:0/3223676887 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7faeac0a3d50 0x7faeac0a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faeb8005fd0 tx=0x7faeb800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:51.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.264+0000 7faec2000700 1 -- 172.21.15.69:0/3223676887 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faebc029020 con 0x7faec410a070 2024-04-24T09:05:51.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.501+0000 7faecc000700 1 -- 172.21.15.69:0/3223676887 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7faec41a8040 con 0x7faec410a070 2024-04-24T09:05:51.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.502+0000 7faec2000700 1 -- 172.21.15.69:0/3223676887 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faebc09e500 con 0x7faec410a070 2024-04-24T09:05:51.502 INFO:teuthology.orchestra.run.smithi069.stdout:253403070523 2024-04-24T09:05:51.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.505+0000 7faecc000700 1 -- 172.21.15.69:0/3223676887 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7faeac0a3d50 msgr2=0x7faeac0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.505+0000 7faecc000700 1 --2- 172.21.15.69:0/3223676887 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7faeac0a3d50 0x7faeac0a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faeb8005fd0 tx=0x7faeb800a560 comp rx=0 tx=0).stop 2024-04-24T09:05:51.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.505+0000 7faecc000700 1 -- 172.21.15.69:0/3223676887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faec410a070 msgr2=0x7faec419fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.505+0000 7faecc000700 1 --2- 172.21.15.69:0/3223676887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faec410a070 0x7faec419fbc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faebc009fd0 tx=0x7faebc007ad0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.505+0000 7faecc000700 1 -- 172.21.15.69:0/3223676887 shutdown_connections 2024-04-24T09:05:51.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.505+0000 7faecc000700 1 --2- 172.21.15.69:0/3223676887 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7faeac0a3d50 0x7faeac0a6210 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.505+0000 7faecc000700 1 --2- 172.21.15.69:0/3223676887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faec410a070 0x7faec419fbc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.505+0000 7faecc000700 1 --2- 172.21.15.69:0/3223676887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faec4104f90 0x7faec419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.505+0000 7faecc000700 1 --2- 172.21.15.69:0/3223676887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faec4105940 0x7faec419b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.505+0000 7faecc000700 1 -- 172.21.15.69:0/3223676887 >> 172.21.15.69:0/3223676887 conn(0x7faec4100820 msgr2=0x7faec4101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:51.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.505+0000 7faecc000700 1 -- 172.21.15.69:0/3223676887 shutdown_connections 2024-04-24T09:05:51.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.505+0000 7faecc000700 1 -- 172.21.15.69:0/3223676887 wait complete. 2024-04-24T09:05:51.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:51 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/4050099496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T09:05:51.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:51 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3223676887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T09:05:51.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.620+0000 7f8e94000700 1 -- 172.21.15.69:0/1594314326 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f8e8c02e1a0 con 0x7f8e8c103500 2024-04-24T09:05:51.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.621+0000 7f8e8a000700 1 -- 172.21.15.69:0/1594314326 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8e7809db60 con 0x7f8e8c103500 2024-04-24T09:05:51.621 INFO:teuthology.orchestra.run.smithi069.stdout:141733920856 2024-04-24T09:05:51.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.624+0000 7f8e94000700 1 -- 172.21.15.69:0/1594314326 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8e740a3cd0 msgr2=0x7f8e740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.624+0000 7f8e94000700 1 --2- 172.21.15.69:0/1594314326 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8e740a3cd0 0x7f8e740a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8e80005fd0 tx=0x7f8e80005d40 comp rx=0 tx=0).stop 2024-04-24T09:05:51.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.624+0000 7f8e94000700 1 -- 172.21.15.69:0/1594314326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8e8c103500 msgr2=0x7f8e8c075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.624+0000 7f8e94000700 1 --2- 172.21.15.69:0/1594314326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8e8c103500 0x7f8e8c075330 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8e78003ab0 tx=0x7f8e78003b90 comp rx=0 tx=0).stop 2024-04-24T09:05:51.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.624+0000 7f8e94000700 1 -- 172.21.15.69:0/1594314326 shutdown_connections 2024-04-24T09:05:51.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.624+0000 7f8e94000700 1 --2- 172.21.15.69:0/1594314326 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8e740a3cd0 0x7f8e740a6190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.624+0000 7f8e94000700 1 --2- 172.21.15.69:0/1594314326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e8c108b90 0x7f8e8c075870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.624+0000 7f8e94000700 1 --2- 172.21.15.69:0/1594314326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e8c102b40 0x7f8e8c074df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.624+0000 7f8e94000700 1 --2- 172.21.15.69:0/1594314326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8e8c103500 0x7f8e8c075330 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.624+0000 7f8e94000700 1 -- 172.21.15.69:0/1594314326 >> 172.21.15.69:0/1594314326 conn(0x7f8e8c079c50 msgr2=0x7f8e8c1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:51.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.624+0000 7f8e94000700 1 -- 172.21.15.69:0/1594314326 shutdown_connections 2024-04-24T09:05:51.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:51.624+0000 7f8e94000700 1 -- 172.21.15.69:0/1594314326 wait complete. 2024-04-24T09:05:51.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:51 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/4050099496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T09:05:51.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:51 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3223676887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T09:05:51.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:51 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/4050099496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T09:05:51.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:51 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3223676887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T09:05:52.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:52 smithi081 ceph-mon[30961]: pgmap v426: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-24T09:05:52.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:52 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1594314326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T09:05:52.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:52 smithi069 ceph-mon[28170]: pgmap v426: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-24T09:05:52.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:52 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1594314326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T09:05:52.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:52 smithi130 ceph-mon[31553]: pgmap v426: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-24T09:05:52.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:52 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1594314326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T09:05:53.063 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182489 got 107374182493 for osd.3 2024-04-24T09:05:53.063 DEBUG:teuthology.parallel:result is None 2024-04-24T09:05:53.502 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920846 got 141733920856 for osd.4 2024-04-24T09:05:53.502 DEBUG:teuthology.parallel:result is None 2024-04-24T09:05:53.711 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:53 smithi069 ceph-mon[28170]: pgmap v427: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T09:05:53.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:53 smithi130 ceph-mon[31553]: pgmap v427: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T09:05:54.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:53 smithi081 ceph-mon[30961]: pgmap v427: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T09:05:54.155 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738476 got 34359738482 for osd.0 2024-04-24T09:05:54.156 DEBUG:teuthology.parallel:result is None 2024-04-24T09:05:54.561 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:05:54.610 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070516 got 253403070523 for osd.8 2024-04-24T09:05:54.610 DEBUG:teuthology.parallel:result is None 2024-04-24T09:05:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.394+0000 7f3286400700 1 -- 172.21.15.69:0/933904374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f328006b1c0 msgr2=0x7f328006b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.394+0000 7f3286400700 1 --2- 172.21.15.69:0/933904374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f328006b1c0 0x7f328006b620 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3268009990 tx=0x7f3268009ca0 comp rx=0 tx=0).stop 2024-04-24T09:05:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.394+0000 7f3286400700 1 -- 172.21.15.69:0/933904374 shutdown_connections 2024-04-24T09:05:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.394+0000 7f3286400700 1 --2- 172.21.15.69:0/933904374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f328006bb60 0x7f328010f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.394+0000 7f3286400700 1 --2- 172.21.15.69:0/933904374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3280108b90 0x7f3280108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.394+0000 7f3286400700 1 --2- 172.21.15.69:0/933904374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f328006b1c0 0x7f328006b620 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.394+0000 7f3286400700 1 -- 172.21.15.69:0/933904374 >> 172.21.15.69:0/933904374 conn(0x7f32800789a0 msgr2=0x7f3280078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.394+0000 7f3286400700 1 -- 172.21.15.69:0/933904374 shutdown_connections 2024-04-24T09:05:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.394+0000 7f3286400700 1 -- 172.21.15.69:0/933904374 wait complete. 2024-04-24T09:05:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.396+0000 7f3286400700 1 Processor -- start 2024-04-24T09:05:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.397+0000 7f3286400700 1 -- start start 2024-04-24T09:05:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.397+0000 7f3286400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f328006b1c0 0x7f3280101ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.397+0000 7f3286400700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f328006bb60 0x7f32801023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.397+0000 7f3286400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3280108b90 0x7f3280102920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.397+0000 7f3286400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3280112d00 con 0x7f328006b1c0 2024-04-24T09:05:55.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.397+0000 7f3286400700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3280112b80 con 0x7f328006bb60 2024-04-24T09:05:55.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.397+0000 7f3286400700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3280112e80 con 0x7f3280108b90 2024-04-24T09:05:55.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.398+0000 7f327f400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f328006b1c0 0x7f3280101ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:55.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.398+0000 7f327f400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f328006b1c0 0x7f3280101ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58452/0 (socket says 172.21.15.69:58452) 2024-04-24T09:05:55.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.398+0000 7f327f400700 1 -- 172.21.15.69:0/1894056695 learned_addr learned my addr 172.21.15.69:0/1894056695 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:55.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.398+0000 7f3277e00700 1 --2- 172.21.15.69:0/1894056695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f328006bb60 0x7f32801023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.398+0000 7f327fe00700 1 --2- 172.21.15.69:0/1894056695 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3280108b90 0x7f3280102920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.398+0000 7f327f400700 1 -- 172.21.15.69:0/1894056695 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3280108b90 msgr2=0x7f3280102920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.398+0000 7f327f400700 1 --2- 172.21.15.69:0/1894056695 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3280108b90 0x7f3280102920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.398+0000 7f327f400700 1 -- 172.21.15.69:0/1894056695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f328006bb60 msgr2=0x7f32801023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.398+0000 7f327f400700 1 --2- 172.21.15.69:0/1894056695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f328006bb60 0x7f32801023e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.398+0000 7f327f400700 1 -- 172.21.15.69:0/1894056695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32801a7780 con 0x7f328006b1c0 2024-04-24T09:05:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.399+0000 7f327f400700 1 --2- 172.21.15.69:0/1894056695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f328006b1c0 0x7f3280101ea0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f327000d980 tx=0x7f327000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.399+0000 7f327cc00700 1 -- 172.21.15.69:0/1894056695 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270017070 con 0x7f328006b1c0 2024-04-24T09:05:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.399+0000 7f327cc00700 1 -- 172.21.15.69:0/1894056695 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3270005020 con 0x7f328006b1c0 2024-04-24T09:05:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.399+0000 7f3286400700 1 -- 172.21.15.69:0/1894056695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32801a7a70 con 0x7f328006b1c0 2024-04-24T09:05:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.399+0000 7f327cc00700 1 -- 172.21.15.69:0/1894056695 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270009b50 con 0x7f328006b1c0 2024-04-24T09:05:55.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.399+0000 7f3286400700 1 -- 172.21.15.69:0/1894056695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32801a7fb0 con 0x7f328006b1c0 2024-04-24T09:05:55.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.401+0000 7f3286400700 1 -- 172.21.15.69:0/1894056695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3280104240 con 0x7f328006b1c0 2024-04-24T09:05:55.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.403+0000 7f327cc00700 1 -- 172.21.15.69:0/1894056695 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3270009cf0 con 0x7f328006b1c0 2024-04-24T09:05:55.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.404+0000 7f327cc00700 1 --2- 172.21.15.69:0/1894056695 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f32640a3cc0 0x7f32640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:55.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.404+0000 7f327cc00700 1 -- 172.21.15.69:0/1894056695 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f32700d05c0 con 0x7f328006b1c0 2024-04-24T09:05:55.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.405+0000 7f3277e00700 1 --2- 172.21.15.69:0/1894056695 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f32640a3cc0 0x7f32640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:55.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.405+0000 7f3277e00700 1 --2- 172.21.15.69:0/1894056695 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f32640a3cc0 0x7f32640a6180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3280079350 tx=0x7f326800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:55.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.409+0000 7f327cc00700 1 -- 172.21.15.69:0/1894056695 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3270020020 con 0x7f328006b1c0 2024-04-24T09:05:55.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.637+0000 7f3286400700 1 -- 172.21.15.69:0/1894056695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f328002e1a0 con 0x7f328006b1c0 2024-04-24T09:05:55.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.637+0000 7f327cc00700 1 -- 172.21.15.69:0/1894056695 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f327009cbe0 con 0x7f328006b1c0 2024-04-24T09:05:55.637 INFO:teuthology.orchestra.run.smithi069.stdout:347892351015 2024-04-24T09:05:55.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.640+0000 7f3286400700 1 -- 172.21.15.69:0/1894056695 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f32640a3cc0 msgr2=0x7f32640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:55.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.640+0000 7f3286400700 1 --2- 172.21.15.69:0/1894056695 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f32640a3cc0 0x7f32640a6180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3280079350 tx=0x7f326800b540 comp rx=0 tx=0).stop 2024-04-24T09:05:55.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.640+0000 7f3286400700 1 -- 172.21.15.69:0/1894056695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f328006b1c0 msgr2=0x7f3280101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:55.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.640+0000 7f3286400700 1 --2- 172.21.15.69:0/1894056695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f328006b1c0 0x7f3280101ea0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f327000d980 tx=0x7f327000dc90 comp rx=0 tx=0).stop 2024-04-24T09:05:55.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.640+0000 7f3286400700 1 -- 172.21.15.69:0/1894056695 shutdown_connections 2024-04-24T09:05:55.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.640+0000 7f3286400700 1 --2- 172.21.15.69:0/1894056695 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f32640a3cc0 0x7f32640a6180 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:55.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.640+0000 7f3286400700 1 --2- 172.21.15.69:0/1894056695 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3280108b90 0x7f3280102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:55.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.640+0000 7f3286400700 1 --2- 172.21.15.69:0/1894056695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f328006b1c0 0x7f3280101ea0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:55.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.640+0000 7f3286400700 1 --2- 172.21.15.69:0/1894056695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f328006bb60 0x7f32801023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:55.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.640+0000 7f3286400700 1 -- 172.21.15.69:0/1894056695 >> 172.21.15.69:0/1894056695 conn(0x7f32800789a0 msgr2=0x7f3280106b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:55.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.641+0000 7f3286400700 1 -- 172.21.15.69:0/1894056695 shutdown_connections 2024-04-24T09:05:55.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:55.641+0000 7f3286400700 1 -- 172.21.15.69:0/1894056695 wait complete. 2024-04-24T09:05:55.657 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:55 smithi069 ceph-mon[28170]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-04-24T09:05:55.657 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:55 smithi069 ceph-mon[28170]: pgmap v428: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:05:55.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:55 smithi130 ceph-mon[31553]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-04-24T09:05:55.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:55 smithi130 ceph-mon[31553]: pgmap v428: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:05:56.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:55 smithi081 ceph-mon[30961]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-04-24T09:05:56.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:55 smithi081 ceph-mon[30961]: pgmap v428: 129 pgs: 129 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:05:56.671 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351008 got 347892351015 for osd.11 2024-04-24T09:05:56.671 DEBUG:teuthology.parallel:result is None 2024-04-24T09:05:56.832 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:56 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1894056695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T09:05:56.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:56 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1894056695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T09:05:57.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:56 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1894056695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T09:05:58.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:57 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:58.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:57 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:58.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:57 smithi081 ceph-mon[30961]: pgmap v429: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T09:05:58.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.089+0000 7fdd9fc00700 1 -- 172.21.15.69:0/3828161992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd981019e0 msgr2=0x7fdd98101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:58.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.089+0000 7fdd9fc00700 1 --2- 172.21.15.69:0/3828161992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd981019e0 0x7fdd98101e60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009990 tx=0x7fdd84009ca0 comp rx=0 tx=0).stop 2024-04-24T09:05:58.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.090+0000 7fdd9fc00700 1 -- 172.21.15.69:0/3828161992 shutdown_connections 2024-04-24T09:05:58.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.090+0000 7fdd9fc00700 1 --2- 172.21.15.69:0/3828161992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd981023a0 0x7fdd9810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:58.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.090+0000 7fdd9fc00700 1 --2- 172.21.15.69:0/3828161992 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd98107a30 0x7fdd98107e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:58.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.090+0000 7fdd9fc00700 1 --2- 172.21.15.69:0/3828161992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd981019e0 0x7fdd98101e60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:58.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.090+0000 7fdd9fc00700 1 -- 172.21.15.69:0/3828161992 >> 172.21.15.69:0/3828161992 conn(0x7fdd98078960 msgr2=0x7fdd98078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:58.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.090+0000 7fdd9fc00700 1 -- 172.21.15.69:0/3828161992 shutdown_connections 2024-04-24T09:05:58.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.090+0000 7fdd9fc00700 1 -- 172.21.15.69:0/3828161992 wait complete. 2024-04-24T09:05:58.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:57 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:58.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:57 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:58.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:57 smithi069 ceph-mon[28170]: pgmap v429: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T09:05:58.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.091+0000 7fdd9fc00700 1 Processor -- start 2024-04-24T09:05:58.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.091+0000 7fdd9fc00700 1 -- start start 2024-04-24T09:05:58.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.091+0000 7fdd9fc00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd981019e0 0x7fdd98103590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:58.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.091+0000 7fdd9fc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd981023a0 0x7fdd98103ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:58.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.092+0000 7fdd9fc00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd98107a30 0x7fdd98104010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:58.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.092+0000 7fdd9fc00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98111bf0 con 0x7fdd981023a0 2024-04-24T09:05:58.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.092+0000 7fdd9fc00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98111a70 con 0x7fdd981019e0 2024-04-24T09:05:58.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.092+0000 7fdd9fc00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98111d70 con 0x7fdd98107a30 2024-04-24T09:05:58.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.092+0000 7fdd97e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd981023a0 0x7fdd98103ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:58.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.092+0000 7fdd97e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd981023a0 0x7fdd98103ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58466/0 (socket says 172.21.15.69:58466) 2024-04-24T09:05:58.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.092+0000 7fdd97e00700 1 -- 172.21.15.69:0/767766055 learned_addr learned my addr 172.21.15.69:0/767766055 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:05:58.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.092+0000 7fdd9d200700 1 --2- 172.21.15.69:0/767766055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd981019e0 0x7fdd98103590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:58.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.092+0000 7fdd9dc00700 1 --2- 172.21.15.69:0/767766055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd98107a30 0x7fdd98104010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:58.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.092+0000 7fdd97e00700 1 -- 172.21.15.69:0/767766055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd98107a30 msgr2=0x7fdd98104010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:58.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.092+0000 7fdd97e00700 1 --2- 172.21.15.69:0/767766055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd98107a30 0x7fdd98104010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:58.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.092+0000 7fdd97e00700 1 -- 172.21.15.69:0/767766055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd981019e0 msgr2=0x7fdd98103590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:58.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.092+0000 7fdd97e00700 1 --2- 172.21.15.69:0/767766055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd981019e0 0x7fdd98103590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:58.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.092+0000 7fdd97e00700 1 -- 172.21.15.69:0/767766055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd981a7760 con 0x7fdd981023a0 2024-04-24T09:05:58.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.093+0000 7fdd9d200700 1 --2- 172.21.15.69:0/767766055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd981019e0 0x7fdd98103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:58.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.093+0000 7fdd97e00700 1 --2- 172.21.15.69:0/767766055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd981023a0 0x7fdd98103ad0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdd84000c00 tx=0x7fdd840036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:58.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.093+0000 7fdd95600700 1 -- 172.21.15.69:0/767766055 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84027070 con 0x7fdd981023a0 2024-04-24T09:05:58.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.093+0000 7fdd95600700 1 -- 172.21.15.69:0/767766055 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd84004090 con 0x7fdd981023a0 2024-04-24T09:05:58.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.093+0000 7fdd9fc00700 1 -- 172.21.15.69:0/767766055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd981a79f0 con 0x7fdd981023a0 2024-04-24T09:05:58.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.093+0000 7fdd95600700 1 -- 172.21.15.69:0/767766055 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84022e20 con 0x7fdd981023a0 2024-04-24T09:05:58.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.093+0000 7fdd9fc00700 1 -- 172.21.15.69:0/767766055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd981a7ea0 con 0x7fdd981023a0 2024-04-24T09:05:58.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.095+0000 7fdd8aa00700 1 -- 172.21.15.69:0/767766055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd74005330 con 0x7fdd981023a0 2024-04-24T09:05:58.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.096+0000 7fdd95600700 1 -- 172.21.15.69:0/767766055 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdd8402a070 con 0x7fdd981023a0 2024-04-24T09:05:58.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.099+0000 7fdd95600700 1 --2- 172.21.15.69:0/767766055 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd800a3cc0 0x7fdd800a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:58.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.099+0000 7fdd95600700 1 -- 172.21.15.69:0/767766055 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdd840d1470 con 0x7fdd981023a0 2024-04-24T09:05:58.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.099+0000 7fdd9d200700 1 --2- 172.21.15.69:0/767766055 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd800a3cc0 0x7fdd800a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:58.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.100+0000 7fdd9d200700 1 --2- 172.21.15.69:0/767766055 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd800a3cc0 0x7fdd800a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c005fd0 tx=0x7fdd8c005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:58.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.103+0000 7fdd95600700 1 -- 172.21.15.69:0/767766055 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd84013070 con 0x7fdd981023a0 2024-04-24T09:05:58.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:57 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:58.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:57 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:58.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:57 smithi130 ceph-mon[31553]: pgmap v429: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T09:05:58.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.328+0000 7fdd8aa00700 1 -- 172.21.15.69:0/767766055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fdd74005620 con 0x7fdd981023a0 2024-04-24T09:05:58.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.329+0000 7fdd95600700 1 -- 172.21.15.69:0/767766055 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdd8409db20 con 0x7fdd981023a0 2024-04-24T09:05:58.361 INFO:teuthology.orchestra.run.smithi069.stdout:287762808886 2024-04-24T09:05:58.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.331+0000 7fdd8aa00700 1 -- 172.21.15.69:0/767766055 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd800a3cc0 msgr2=0x7fdd800a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:58.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.331+0000 7fdd8aa00700 1 --2- 172.21.15.69:0/767766055 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd800a3cc0 0x7fdd800a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c005fd0 tx=0x7fdd8c005d90 comp rx=0 tx=0).stop 2024-04-24T09:05:58.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.331+0000 7fdd8aa00700 1 -- 172.21.15.69:0/767766055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd981023a0 msgr2=0x7fdd98103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:58.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.331+0000 7fdd8aa00700 1 --2- 172.21.15.69:0/767766055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd981023a0 0x7fdd98103ad0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdd84000c00 tx=0x7fdd840036f0 comp rx=0 tx=0).stop 2024-04-24T09:05:58.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.332+0000 7fdd8aa00700 1 -- 172.21.15.69:0/767766055 shutdown_connections 2024-04-24T09:05:58.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.332+0000 7fdd8aa00700 1 --2- 172.21.15.69:0/767766055 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdd800a3cc0 0x7fdd800a6180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:58.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.332+0000 7fdd8aa00700 1 --2- 172.21.15.69:0/767766055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd98107a30 0x7fdd98104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:58.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.332+0000 7fdd8aa00700 1 --2- 172.21.15.69:0/767766055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd981019e0 0x7fdd98103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:58.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.332+0000 7fdd8aa00700 1 --2- 172.21.15.69:0/767766055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd981023a0 0x7fdd98103ad0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:58.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.332+0000 7fdd8aa00700 1 -- 172.21.15.69:0/767766055 >> 172.21.15.69:0/767766055 conn(0x7fdd98078960 msgr2=0x7fdd9806b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:58.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.332+0000 7fdd8aa00700 1 -- 172.21.15.69:0/767766055 shutdown_connections 2024-04-24T09:05:58.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:05:58.332+0000 7fdd8aa00700 1 -- 172.21.15.69:0/767766055 wait complete. 2024-04-24T09:05:59.085 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:58 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:05:59.085 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:58 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:05:59.085 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:58 smithi069 ceph-mon[28170]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:59.085 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:58 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/767766055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T09:05:59.151 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808880 got 287762808886 for osd.9 2024-04-24T09:05:59.151 DEBUG:teuthology.parallel:result is None 2024-04-24T09:05:59.151 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-24T09:05:59.152 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T09:05:59.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:58 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:05:59.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:58 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:05:59.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:58 smithi130 ceph-mon[31553]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:59.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:58 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/767766055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T09:05:59.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:58 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:05:59.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:58 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:05:59.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:58 smithi081 ceph-mon[30961]: from='mgr.14152 172.21.15.69:0/2710363019' entity='mgr.x' 2024-04-24T09:05:59.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:58 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/767766055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T09:05:59.602 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:06:00.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:05:59 smithi069 ceph-mon[28170]: pgmap v430: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T09:06:00.105 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:05:59 smithi130 ceph-mon[31553]: pgmap v430: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T09:06:00.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:05:59 smithi081 ceph-mon[30961]: pgmap v430: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T09:06:01.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.853+0000 7f2bd5a00700 1 -- 172.21.15.69:0/2305100599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd010f520 msgr2=0x7f2bd0111910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:01.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.853+0000 7f2bd5a00700 1 --2- 172.21.15.69:0/2305100599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd010f520 0x7f2bd0111910 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800b0a0 tx=0x7f2bc800b3b0 comp rx=0 tx=0).stop 2024-04-24T09:06:01.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.853+0000 7f2bd5a00700 1 -- 172.21.15.69:0/2305100599 shutdown_connections 2024-04-24T09:06:01.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.853+0000 7f2bd5a00700 1 --2- 172.21.15.69:0/2305100599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd010f520 0x7f2bd0111910 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:01.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.853+0000 7f2bd5a00700 1 --2- 172.21.15.69:0/2305100599 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bd0074d20 0x7f2bd0075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:01.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.853+0000 7f2bd5a00700 1 --2- 172.21.15.69:0/2305100599 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bd0075640 0x7f2bd010efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:01.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.853+0000 7f2bd5a00700 1 -- 172.21.15.69:0/2305100599 >> 172.21.15.69:0/2305100599 conn(0x7f2bd0079bc0 msgr2=0x7f2bd0079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:01.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.854+0000 7f2bd5a00700 1 -- 172.21.15.69:0/2305100599 shutdown_connections 2024-04-24T09:06:01.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.854+0000 7f2bd5a00700 1 -- 172.21.15.69:0/2305100599 wait complete. 2024-04-24T09:06:01.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.855+0000 7f2bd5a00700 1 Processor -- start 2024-04-24T09:06:01.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.855+0000 7f2bd5a00700 1 -- start start 2024-04-24T09:06:01.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.856+0000 7f2bd5a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd0074d20 0x7f2bd019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:01.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.856+0000 7f2bd5a00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bd0075640 0x7f2bd019b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:01.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.856+0000 7f2bd5a00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bd010f520 0x7f2bd019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:01.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.856+0000 7f2bd5a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0114ee0 con 0x7f2bd0074d20 2024-04-24T09:06:01.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.856+0000 7f2bd5a00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0114d60 con 0x7f2bd0075640 2024-04-24T09:06:01.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.856+0000 7f2bd5a00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0115060 con 0x7f2bd010f520 2024-04-24T09:06:01.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.856+0000 7f2bcea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd0074d20 0x7f2bd019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:01.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.856+0000 7f2bcea00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd0074d20 0x7f2bd019b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58484/0 (socket says 172.21.15.69:58484) 2024-04-24T09:06:01.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.856+0000 7f2bcea00700 1 -- 172.21.15.69:0/1095478305 learned_addr learned my addr 172.21.15.69:0/1095478305 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:01.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.856+0000 7f2bc7e00700 1 --2- 172.21.15.69:0/1095478305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bd0075640 0x7f2bd019b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:01.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.856+0000 7f2bcea00700 1 -- 172.21.15.69:0/1095478305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bd010f520 msgr2=0x7f2bd019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:01.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.857+0000 7f2bcea00700 1 --2- 172.21.15.69:0/1095478305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bd010f520 0x7f2bd019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:01.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.857+0000 7f2bcea00700 1 -- 172.21.15.69:0/1095478305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bd0075640 msgr2=0x7f2bd019b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:01.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.857+0000 7f2bcea00700 1 --2- 172.21.15.69:0/1095478305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bd0075640 0x7f2bd019b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:01.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.857+0000 7f2bcea00700 1 -- 172.21.15.69:0/1095478305 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd01a0390 con 0x7f2bd0074d20 2024-04-24T09:06:01.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.857+0000 7f2bc7e00700 1 --2- 172.21.15.69:0/1095478305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bd0075640 0x7f2bd019b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:01.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.857+0000 7f2bcea00700 1 --2- 172.21.15.69:0/1095478305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd0074d20 0x7f2bd019b3d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000d950 tx=0x7f2bc000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:01.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.857+0000 7f2bc7400700 1 -- 172.21.15.69:0/1095478305 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0016070 con 0x7f2bd0074d20 2024-04-24T09:06:01.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.857+0000 7f2bc7400700 1 -- 172.21.15.69:0/1095478305 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bc0010460 con 0x7f2bd0074d20 2024-04-24T09:06:01.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.857+0000 7f2bc7400700 1 -- 172.21.15.69:0/1095478305 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000f620 con 0x7f2bd0074d20 2024-04-24T09:06:01.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.857+0000 7f2bd5a00700 1 -- 172.21.15.69:0/1095478305 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd01a0680 con 0x7f2bd0074d20 2024-04-24T09:06:01.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.857+0000 7f2bd5a00700 1 -- 172.21.15.69:0/1095478305 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd01a7f00 con 0x7f2bd0074d20 2024-04-24T09:06:01.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.858+0000 7f2bd5a00700 1 -- 172.21.15.69:0/1095478305 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd00697e0 con 0x7f2bd0074d20 2024-04-24T09:06:01.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.860+0000 7f2bc7400700 1 -- 172.21.15.69:0/1095478305 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2bc0019070 con 0x7f2bd0074d20 2024-04-24T09:06:01.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.864+0000 7f2bc7400700 1 --2- 172.21.15.69:0/1095478305 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2bb40a3c40 0x7f2bb40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:01.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.864+0000 7f2bc7400700 1 -- 172.21.15.69:0/1095478305 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f2bc00d0400 con 0x7f2bd0074d20 2024-04-24T09:06:01.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.864+0000 7f2bc7e00700 1 --2- 172.21.15.69:0/1095478305 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2bb40a3c40 0x7f2bb40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:01.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.865+0000 7f2bc7e00700 1 --2- 172.21.15.69:0/1095478305 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2bb40a3c40 0x7f2bb40a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8005e10 tx=0x7f2bb8005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:01.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:01.871+0000 7f2bc7400700 1 -- 172.21.15.69:0/1095478305 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2bc009ca20 con 0x7f2bd0074d20 2024-04-24T09:06:02.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:02.099+0000 7f2bd5a00700 1 -- 172.21.15.69:0/1095478305 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2bd0101610 con 0x7f2bb40a3c40 2024-04-24T09:06:02.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:02.138+0000 7f2bc7400700 1 -- 172.21.15.69:0/1095478305 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340519 (secure 0 0 0) 0x7f2bd0101610 con 0x7f2bb40a3c40 2024-04-24T09:06:02.145 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:02.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:02.145+0000 7f2bd5a00700 1 -- 172.21.15.69:0/1095478305 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2bb40a3c40 msgr2=0x7f2bb40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:02.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:02.146+0000 7f2bd5a00700 1 --2- 172.21.15.69:0/1095478305 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2bb40a3c40 0x7f2bb40a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8005e10 tx=0x7f2bb8005d20 comp rx=0 tx=0).stop 2024-04-24T09:06:02.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:02.146+0000 7f2bd5a00700 1 -- 172.21.15.69:0/1095478305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd0074d20 msgr2=0x7f2bd019b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:02.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:02.146+0000 7f2bd5a00700 1 --2- 172.21.15.69:0/1095478305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd0074d20 0x7f2bd019b3d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000d950 tx=0x7f2bc000dd10 comp rx=0 tx=0).stop 2024-04-24T09:06:02.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:02.146+0000 7f2bd5a00700 1 -- 172.21.15.69:0/1095478305 shutdown_connections 2024-04-24T09:06:02.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:02.146+0000 7f2bd5a00700 1 --2- 172.21.15.69:0/1095478305 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f2bb40a3c40 0x7f2bb40a6100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:02.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:02.146+0000 7f2bd5a00700 1 --2- 172.21.15.69:0/1095478305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bd010f520 0x7f2bd019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:02.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:02.146+0000 7f2bd5a00700 1 --2- 172.21.15.69:0/1095478305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd0074d20 0x7f2bd019b3d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:02.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:02.146+0000 7f2bd5a00700 1 --2- 172.21.15.69:0/1095478305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bd0075640 0x7f2bd019b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:02.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:02.146+0000 7f2bd5a00700 1 -- 172.21.15.69:0/1095478305 >> 172.21.15.69:0/1095478305 conn(0x7f2bd0079bc0 msgr2=0x7f2bd0100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:02.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:02.147+0000 7f2bd5a00700 1 -- 172.21.15.69:0/1095478305 shutdown_connections 2024-04-24T09:06:02.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:02.147+0000 7f2bd5a00700 1 -- 172.21.15.69:0/1095478305 wait complete. 2024-04-24T09:06:02.148 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-24T09:06:02.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:02 smithi081 ceph-mon[30961]: pgmap v431: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T09:06:02.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:02 smithi069 ceph-mon[28170]: pgmap v431: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T09:06:02.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:02 smithi130 ceph-mon[31553]: pgmap v431: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T09:06:03.026 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-04-24T09:06:01.425113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661031,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":675,"num_read_kb":460,"num_write":358,"num_write_kb":3066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":791,"ondisk_log_size":791,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355084,"kb_used_data":14252,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124586740,"statfs":{"total":1151940427776,"available":1151576821760,"internally_reserved":0,"allocated":14594048,"data_stored":8048855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":32,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004760"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.378272+0000","last_change":"2024-04-24T09:04:02.107965+0000","last_active":"2024-04-24T09:05:53.378272+0000","last_peered":"2024-04-24T09:05:53.378272+0000","last_clean":"2024-04-24T09:05:53.378272+0000","last_became_active":"2024-04-24T09:04:02.106827+0000","last_became_peered":"2024-04-24T09:04:02.106827+0000","last_unstale":"2024-04-24T09:05:53.378272+0000","last_undegraded":"2024-04-24T09:05:53.378272+0000","last_fullsized":"2024-04-24T09:05:53.378272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:20:23.930599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.361508+0000","last_change":"2024-04-24T09:04:02.105449+0000","last_active":"2024-04-24T09:05:53.361508+0000","last_peered":"2024-04-24T09:05:53.361508+0000","last_clean":"2024-04-24T09:05:53.361508+0000","last_became_active":"2024-04-24T09:04:02.101842+0000","last_became_peered":"2024-04-24T09:04:02.101842+0000","last_unstale":"2024-04-24T09:05:53.361508+0000","last_undegraded":"2024-04-24T09:05:53.361508+0000","last_fullsized":"2024-04-24T09:05:53.361508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:46.911768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.377788+0000","last_change":"2024-04-24T09:04:02.118686+0000","last_active":"2024-04-24T09:05:53.377788+0000","last_peered":"2024-04-24T09:05:53.377788+0000","last_clean":"2024-04-24T09:05:53.377788+0000","last_became_active":"2024-04-24T09:04:02.117549+0000","last_became_peered":"2024-04-24T09:04:02.117549+0000","last_unstale":"2024-04-24T09:05:53.377788+0000","last_undegraded":"2024-04-24T09:05:53.377788+0000","last_fullsized":"2024-04-24T09:05:53.377788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:18:16.233434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.350437+0000","last_change":"2024-04-24T09:04:02.120336+0000","last_active":"2024-04-24T09:05:53.350437+0000","last_peered":"2024-04-24T09:05:53.350437+0000","last_clean":"2024-04-24T09:05:53.350437+0000","last_became_active":"2024-04-24T09:04:02.116574+0000","last_became_peered":"2024-04-24T09:04:02.116574+0000","last_unstale":"2024-04-24T09:05:53.350437+0000","last_undegraded":"2024-04-24T09:05:53.350437+0000","last_fullsized":"2024-04-24T09:05:53.350437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:10:42.366531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.359791+0000","last_change":"2024-04-24T09:04:02.113600+0000","last_active":"2024-04-24T09:05:53.359791+0000","last_peered":"2024-04-24T09:05:53.359791+0000","last_clean":"2024-04-24T09:05:53.359791+0000","last_became_active":"2024-04-24T09:04:02.113143+0000","last_became_peered":"2024-04-24T09:04:02.113143+0000","last_unstale":"2024-04-24T09:05:53.359791+0000","last_undegraded":"2024-04-24T09:05:53.359791+0000","last_fullsized":"2024-04-24T09:05:53.359791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:31:27.666661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.366082+0000","last_change":"2024-04-24T09:04:02.106395+0000","last_active":"2024-04-24T09:05:53.366082+0000","last_peered":"2024-04-24T09:05:53.366082+0000","last_clean":"2024-04-24T09:05:53.366082+0000","last_became_active":"2024-04-24T09:04:02.105942+0000","last_became_peered":"2024-04-24T09:04:02.105942+0000","last_unstale":"2024-04-24T09:05:53.366082+0000","last_undegraded":"2024-04-24T09:05:53.366082+0000","last_fullsized":"2024-04-24T09:05:53.366082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:20:10.655605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.361146+0000","last_change":"2024-04-24T09:04:02.100279+0000","last_active":"2024-04-24T09:05:53.361146+0000","last_peered":"2024-04-24T09:05:53.361146+0000","last_clean":"2024-04-24T09:05:53.361146+0000","last_became_active":"2024-04-24T09:04:02.099745+0000","last_became_peered":"2024-04-24T09:04:02.099745+0000","last_unstale":"2024-04-24T09:05:53.361146+0000","last_undegraded":"2024-04-24T09:05:53.361146+0000","last_fullsized":"2024-04-24T09:05:53.361146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:13.150412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.374378+0000","last_change":"2024-04-24T09:04:02.103294+0000","last_active":"2024-04-24T09:05:53.374378+0000","last_peered":"2024-04-24T09:05:53.374378+0000","last_clean":"2024-04-24T09:05:53.374378+0000","last_became_active":"2024-04-24T09:04:02.102114+0000","last_became_peered":"2024-04-24T09:04:02.102114+0000","last_unstale":"2024-04-24T09:05:53.374378+0000","last_undegraded":"2024-04-24T09:05:53.374378+0000","last_fullsized":"2024-04-24T09:05:53.374378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:26:26.916097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.377067+0000","last_change":"2024-04-24T09:04:02.104936+0000","last_active":"2024-04-24T09:05:53.377067+0000","last_peered":"2024-04-24T09:05:53.377067+0000","last_clean":"2024-04-24T09:05:53.377067+0000","last_became_active":"2024-04-24T09:04:02.104540+0000","last_became_peered":"2024-04-24T09:04:02.104540+0000","last_unstale":"2024-04-24T09:05:53.377067+0000","last_undegraded":"2024-04-24T09:05:53.377067+0000","last_fullsized":"2024-04-24T09:05:53.377067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:36.010735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.367628+0000","last_change":"2024-04-24T09:04:02.102050+0000","last_active":"2024-04-24T09:05:53.367628+0000","last_peered":"2024-04-24T09:05:53.367628+0000","last_clean":"2024-04-24T09:05:53.367628+0000","last_became_active":"2024-04-24T09:04:02.101623+0000","last_became_peered":"2024-04-24T09:04:02.101623+0000","last_unstale":"2024-04-24T09:05:53.367628+0000","last_undegraded":"2024-04-24T09:05:53.367628+0000","last_fullsized":"2024-04-24T09:05:53.367628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:13:33.606200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.378933+0000","last_change":"2024-04-24T09:04:02.115804+0000","last_active":"2024-04-24T09:05:53.378933+0000","last_peered":"2024-04-24T09:05:53.378933+0000","last_clean":"2024-04-24T09:05:53.378933+0000","last_became_active":"2024-04-24T09:04:02.115093+0000","last_became_peered":"2024-04-24T09:04:02.115093+0000","last_unstale":"2024-04-24T09:05:53.378933+0000","last_undegraded":"2024-04-24T09:05:53.378933+0000","last_fullsized":"2024-04-24T09:05:53.378933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:26:56.764485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.358804+0000","last_change":"2024-04-24T09:04:02.116305+0000","last_active":"2024-04-24T09:05:53.358804+0000","last_peered":"2024-04-24T09:05:53.358804+0000","last_clean":"2024-04-24T09:05:53.358804+0000","last_became_active":"2024-04-24T09:04:02.115956+0000","last_became_peered":"2024-04-24T09:04:02.115956+0000","last_unstale":"2024-04-24T09:05:53.358804+0000","last_undegraded":"2024-04-24T09:05:53.358804+0000","last_fullsized":"2024-04-24T09:05:53.358804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:38.618317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.362695+0000","last_change":"2024-04-24T09:04:02.101155+0000","last_active":"2024-04-24T09:05:53.362695+0000","last_peered":"2024-04-24T09:05:53.362695+0000","last_clean":"2024-04-24T09:05:53.362695+0000","last_became_active":"2024-04-24T09:04:02.099719+0000","last_became_peered":"2024-04-24T09:04:02.099719+0000","last_unstale":"2024-04-24T09:05:53.362695+0000","last_undegraded":"2024-04-24T09:05:53.362695+0000","last_fullsized":"2024-04-24T09:05:53.362695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:46:27.910987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.362194+0000","last_change":"2024-04-24T09:04:02.114818+0000","last_active":"2024-04-24T09:05:53.362194+0000","last_peered":"2024-04-24T09:05:53.362194+0000","last_clean":"2024-04-24T09:05:53.362194+0000","last_became_active":"2024-04-24T09:04:02.112947+0000","last_became_peered":"2024-04-24T09:04:02.112947+0000","last_unstale":"2024-04-24T09:05:53.362194+0000","last_undegraded":"2024-04-24T09:05:53.362194+0000","last_fullsized":"2024-04-24T09:05:53.362194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:16.544420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.355959+0000","last_change":"2024-04-24T09:04:02.120672+0000","last_active":"2024-04-24T09:05:53.355959+0000","last_peered":"2024-04-24T09:05:53.355959+0000","last_clean":"2024-04-24T09:05:53.355959+0000","last_became_active":"2024-04-24T09:04:02.120378+0000","last_became_peered":"2024-04-24T09:04:02.120378+0000","last_unstale":"2024-04-24T09:05:53.355959+0000","last_undegraded":"2024-04-24T09:05:53.355959+0000","last_fullsized":"2024-04-24T09:05:53.355959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:17.781189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.376999+0000","last_change":"2024-04-24T09:04:02.107022+0000","last_active":"2024-04-24T09:05:53.376999+0000","last_peered":"2024-04-24T09:05:53.376999+0000","last_clean":"2024-04-24T09:05:53.376999+0000","last_became_active":"2024-04-24T09:04:02.106645+0000","last_became_peered":"2024-04-24T09:04:02.106645+0000","last_unstale":"2024-04-24T09:05:53.376999+0000","last_undegraded":"2024-04-24T09:05:53.376999+0000","last_fullsized":"2024-04-24T09:05:53.376999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:47.217718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.379883+0000","last_change":"2024-04-24T09:04:02.115511+0000","last_active":"2024-04-24T09:05:53.379883+0000","last_peered":"2024-04-24T09:05:53.379883+0000","last_clean":"2024-04-24T09:05:53.379883+0000","last_became_active":"2024-04-24T09:04:02.115197+0000","last_became_peered":"2024-04-24T09:04:02.115197+0000","last_unstale":"2024-04-24T09:05:53.379883+0000","last_undegraded":"2024-04-24T09:05:53.379883+0000","last_fullsized":"2024-04-24T09:05:53.379883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:18.883688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.378708+0000","last_change":"2024-04-24T09:04:02.102108+0000","last_active":"2024-04-24T09:05:53.378708+0000","last_peered":"2024-04-24T09:05:53.378708+0000","last_clean":"2024-04-24T09:05:53.378708+0000","last_became_active":"2024-04-24T09:04:02.101739+0000","last_became_peered":"2024-04-24T09:04:02.101739+0000","last_unstale":"2024-04-24T09:05:53.378708+0000","last_undegraded":"2024-04-24T09:05:53.378708+0000","last_fullsized":"2024-04-24T09:05:53.378708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:57:30.768716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.367483+0000","last_change":"2024-04-24T09:04:02.108911+0000","last_active":"2024-04-24T09:05:53.367483+0000","last_peered":"2024-04-24T09:05:53.367483+0000","last_clean":"2024-04-24T09:05:53.367483+0000","last_became_active":"2024-04-24T09:04:02.105678+0000","last_became_peered":"2024-04-24T09:04:02.105678+0000","last_unstale":"2024-04-24T09:05:53.367483+0000","last_undegraded":"2024-04-24T09:05:53.367483+0000","last_fullsized":"2024-04-24T09:05:53.367483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:36:12.216271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696112+0000","last_change":"2024-04-24T09:04:06.136320+0000","last_active":"2024-04-24T09:04:24.696112+0000","last_peered":"2024-04-24T09:04:24.696112+0000","last_clean":"2024-04-24T09:04:24.696112+0000","last_became_active":"2024-04-24T09:04:06.133851+0000","last_became_peered":"2024-04-24T09:04:06.133851+0000","last_unstale":"2024-04-24T09:04:24.696112+0000","last_undegraded":"2024-04-24T09:04:24.696112+0000","last_fullsized":"2024-04-24T09:04:24.696112+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:31:39.781234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.630633+0000","last_change":"2024-04-24T09:04:01.543818+0000","last_active":"2024-04-24T09:04:33.630633+0000","last_peered":"2024-04-24T09:04:33.630633+0000","last_clean":"2024-04-24T09:04:33.630633+0000","last_became_active":"2024-04-24T09:04:01.543435+0000","last_became_peered":"2024-04-24T09:04:01.543435+0000","last_unstale":"2024-04-24T09:04:33.630633+0000","last_undegraded":"2024-04-24T09:04:33.630633+0000","last_fullsized":"2024-04-24T09:04:33.630633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:08:30.176211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.363043+0000","last_change":"2024-04-24T09:04:02.101646+0000","last_active":"2024-04-24T09:05:53.363043+0000","last_peered":"2024-04-24T09:05:53.363043+0000","last_clean":"2024-04-24T09:05:53.363043+0000","last_became_active":"2024-04-24T09:04:02.101284+0000","last_became_peered":"2024-04-24T09:04:02.101284+0000","last_unstale":"2024-04-24T09:05:53.363043+0000","last_undegraded":"2024-04-24T09:05:53.363043+0000","last_fullsized":"2024-04-24T09:05:53.363043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:36:32.827216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.519708+0000","last_change":"2024-04-24T09:04:06.130476+0000","last_active":"2024-04-24T09:04:24.519708+0000","last_peered":"2024-04-24T09:04:24.519708+0000","last_clean":"2024-04-24T09:04:24.519708+0000","last_became_active":"2024-04-24T09:04:06.129707+0000","last_became_peered":"2024-04-24T09:04:06.129707+0000","last_unstale":"2024-04-24T09:04:24.519708+0000","last_undegraded":"2024-04-24T09:04:24.519708+0000","last_fullsized":"2024-04-24T09:04:24.519708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:25:36.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527232+0000","last_change":"2024-04-24T09:04:01.551708+0000","last_active":"2024-04-24T09:04:24.527232+0000","last_peered":"2024-04-24T09:04:24.527232+0000","last_clean":"2024-04-24T09:04:24.527232+0000","last_became_active":"2024-04-24T09:04:01.551089+0000","last_became_peered":"2024-04-24T09:04:01.551089+0000","last_unstale":"2024-04-24T09:04:24.527232+0000","last_undegraded":"2024-04-24T09:04:24.527232+0000","last_fullsized":"2024-04-24T09:04:24.527232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:18:22.515729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.378011+0000","last_change":"2024-04-24T09:04:02.117356+0000","last_active":"2024-04-24T09:05:53.378011+0000","last_peered":"2024-04-24T09:05:53.378011+0000","last_clean":"2024-04-24T09:05:53.378011+0000","last_became_active":"2024-04-24T09:04:02.116972+0000","last_became_peered":"2024-04-24T09:04:02.116972+0000","last_unstale":"2024-04-24T09:05:53.378011+0000","last_undegraded":"2024-04-24T09:05:53.378011+0000","last_fullsized":"2024-04-24T09:05:53.378011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:46:29.349952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.520686+0000","last_change":"2024-04-24T09:04:06.141538+0000","last_active":"2024-04-24T09:04:24.520686+0000","last_peered":"2024-04-24T09:04:24.520686+0000","last_clean":"2024-04-24T09:04:24.520686+0000","last_became_active":"2024-04-24T09:04:06.140975+0000","last_became_peered":"2024-04-24T09:04:06.140975+0000","last_unstale":"2024-04-24T09:04:24.520686+0000","last_undegraded":"2024-04-24T09:04:24.520686+0000","last_fullsized":"2024-04-24T09:04:24.520686+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:58:31.688828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.630038+0000","last_change":"2024-04-24T09:04:01.561039+0000","last_active":"2024-04-24T09:04:33.630038+0000","last_peered":"2024-04-24T09:04:33.630038+0000","last_clean":"2024-04-24T09:04:33.630038+0000","last_became_active":"2024-04-24T09:04:01.560594+0000","last_became_peered":"2024-04-24T09:04:01.560594+0000","last_unstale":"2024-04-24T09:04:33.630038+0000","last_undegraded":"2024-04-24T09:04:33.630038+0000","last_fullsized":"2024-04-24T09:04:33.630038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:59:19.707826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.378030+0000","last_change":"2024-04-24T09:04:02.106007+0000","last_active":"2024-04-24T09:05:53.378030+0000","last_peered":"2024-04-24T09:05:53.378030+0000","last_clean":"2024-04-24T09:05:53.378030+0000","last_became_active":"2024-04-24T09:04:02.099786+0000","last_became_peered":"2024-04-24T09:04:02.099786+0000","last_unstale":"2024-04-24T09:05:53.378030+0000","last_undegraded":"2024-04-24T09:05:53.378030+0000","last_fullsized":"2024-04-24T09:05:53.378030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:56.288417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.520162+0000","last_change":"2024-04-24T09:04:06.143636+0000","last_active":"2024-04-24T09:04:24.520162+0000","last_peered":"2024-04-24T09:04:24.520162+0000","last_clean":"2024-04-24T09:04:24.520162+0000","last_became_active":"2024-04-24T09:04:06.143105+0000","last_became_peered":"2024-04-24T09:04:06.143105+0000","last_unstale":"2024-04-24T09:04:24.520162+0000","last_undegraded":"2024-04-24T09:04:24.520162+0000","last_fullsized":"2024-04-24T09:04:24.520162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:04:23.211067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.517222+0000","last_change":"2024-04-24T09:04:01.548403+0000","last_active":"2024-04-24T09:04:24.517222+0000","last_peered":"2024-04-24T09:04:24.517222+0000","last_clean":"2024-04-24T09:04:24.517222+0000","last_became_active":"2024-04-24T09:04:01.547447+0000","last_became_peered":"2024-04-24T09:04:01.547447+0000","last_unstale":"2024-04-24T09:04:24.517222+0000","last_undegraded":"2024-04-24T09:04:24.517222+0000","last_fullsized":"2024-04-24T09:04:24.517222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:14:08.160127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.378912+0000","last_change":"2024-04-24T09:04:02.121410+0000","last_active":"2024-04-24T09:05:53.378912+0000","last_peered":"2024-04-24T09:05:53.378912+0000","last_clean":"2024-04-24T09:05:53.378912+0000","last_became_active":"2024-04-24T09:04:02.120821+0000","last_became_peered":"2024-04-24T09:04:02.120821+0000","last_unstale":"2024-04-24T09:05:53.378912+0000","last_undegraded":"2024-04-24T09:05:53.378912+0000","last_fullsized":"2024-04-24T09:05:53.378912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:39:05.525171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696856+0000","last_change":"2024-04-24T09:04:06.142349+0000","last_active":"2024-04-24T09:04:24.696856+0000","last_peered":"2024-04-24T09:04:24.696856+0000","last_clean":"2024-04-24T09:04:24.696856+0000","last_became_active":"2024-04-24T09:04:06.141911+0000","last_became_peered":"2024-04-24T09:04:06.141911+0000","last_unstale":"2024-04-24T09:04:24.696856+0000","last_undegraded":"2024-04-24T09:04:24.696856+0000","last_fullsized":"2024-04-24T09:04:24.696856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:45:30.414686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'5","reported_seq":126,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:59.886677+0000","last_change":"2024-04-24T09:04:24.537349+0000","last_active":"2024-04-24T09:05:59.886677+0000","last_peered":"2024-04-24T09:05:59.886677+0000","last_clean":"2024-04-24T09:05:59.886677+0000","last_became_active":"2024-04-24T09:04:24.536738+0000","last_became_peered":"2024-04-24T09:04:24.536738+0000","last_unstale":"2024-04-24T09:05:59.886677+0000","last_undegraded":"2024-04-24T09:05:59.886677+0000","last_fullsized":"2024-04-24T09:05:59.886677+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:33.481380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.351533+0000","last_change":"2024-04-24T09:04:02.111173+0000","last_active":"2024-04-24T09:05:53.351533+0000","last_peered":"2024-04-24T09:05:53.351533+0000","last_clean":"2024-04-24T09:05:53.351533+0000","last_became_active":"2024-04-24T09:04:02.110681+0000","last_became_peered":"2024-04-24T09:04:02.110681+0000","last_unstale":"2024-04-24T09:05:53.351533+0000","last_undegraded":"2024-04-24T09:05:53.351533+0000","last_fullsized":"2024-04-24T09:05:53.351533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:29.939116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696716+0000","last_change":"2024-04-24T09:04:06.137752+0000","last_active":"2024-04-24T09:04:24.696716+0000","last_peered":"2024-04-24T09:04:24.696716+0000","last_clean":"2024-04-24T09:04:24.696716+0000","last_became_active":"2024-04-24T09:04:06.137492+0000","last_became_peered":"2024-04-24T09:04:06.137492+0000","last_unstale":"2024-04-24T09:04:24.696716+0000","last_undegraded":"2024-04-24T09:04:24.696716+0000","last_fullsized":"2024-04-24T09:04:24.696716+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:44:00.653864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.628109+0000","last_change":"2024-04-24T09:04:01.544009+0000","last_active":"2024-04-24T09:04:33.628109+0000","last_peered":"2024-04-24T09:04:33.628109+0000","last_clean":"2024-04-24T09:04:33.628109+0000","last_became_active":"2024-04-24T09:04:01.543651+0000","last_became_peered":"2024-04-24T09:04:01.543651+0000","last_unstale":"2024-04-24T09:04:33.628109+0000","last_undegraded":"2024-04-24T09:04:33.628109+0000","last_fullsized":"2024-04-24T09:04:33.628109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:10:21.969072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.373547+0000","last_change":"2024-04-24T09:04:02.115387+0000","last_active":"2024-04-24T09:05:53.373547+0000","last_peered":"2024-04-24T09:05:53.373547+0000","last_clean":"2024-04-24T09:05:53.373547+0000","last_became_active":"2024-04-24T09:04:02.115086+0000","last_became_peered":"2024-04-24T09:04:02.115086+0000","last_unstale":"2024-04-24T09:05:53.373547+0000","last_undegraded":"2024-04-24T09:05:53.373547+0000","last_fullsized":"2024-04-24T09:05:53.373547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:40:57.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532516+0000","last_change":"2024-04-24T09:04:06.146189+0000","last_active":"2024-04-24T09:04:24.532516+0000","last_peered":"2024-04-24T09:04:24.532516+0000","last_clean":"2024-04-24T09:04:24.532516+0000","last_became_active":"2024-04-24T09:04:06.145556+0000","last_became_peered":"2024-04-24T09:04:06.145556+0000","last_unstale":"2024-04-24T09:04:24.532516+0000","last_undegraded":"2024-04-24T09:04:24.532516+0000","last_fullsized":"2024-04-24T09:04:24.532516+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:41.371948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.628679+0000","last_change":"2024-04-24T09:04:01.542185+0000","last_active":"2024-04-24T09:04:33.628679+0000","last_peered":"2024-04-24T09:04:33.628679+0000","last_clean":"2024-04-24T09:04:33.628679+0000","last_became_active":"2024-04-24T09:04:01.541240+0000","last_became_peered":"2024-04-24T09:04:01.541240+0000","last_unstale":"2024-04-24T09:04:33.628679+0000","last_undegraded":"2024-04-24T09:04:33.628679+0000","last_fullsized":"2024-04-24T09:04:33.628679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:15.103276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.372576+0000","last_change":"2024-04-24T09:04:02.104054+0000","last_active":"2024-04-24T09:05:53.372576+0000","last_peered":"2024-04-24T09:05:53.372576+0000","last_clean":"2024-04-24T09:05:53.372576+0000","last_became_active":"2024-04-24T09:04:02.103673+0000","last_became_peered":"2024-04-24T09:04:02.103673+0000","last_unstale":"2024-04-24T09:05:53.372576+0000","last_undegraded":"2024-04-24T09:05:53.372576+0000","last_fullsized":"2024-04-24T09:05:53.372576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:31:58.889989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696675+0000","last_change":"2024-04-24T09:04:06.134034+0000","last_active":"2024-04-24T09:04:24.696675+0000","last_peered":"2024-04-24T09:04:24.696675+0000","last_clean":"2024-04-24T09:04:24.696675+0000","last_became_active":"2024-04-24T09:04:06.133596+0000","last_became_peered":"2024-04-24T09:04:06.133596+0000","last_unstale":"2024-04-24T09:04:24.696675+0000","last_undegraded":"2024-04-24T09:04:24.696675+0000","last_fullsized":"2024-04-24T09:04:24.696675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:17:09.572974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.528733+0000","last_change":"2024-04-24T09:04:01.543312+0000","last_active":"2024-04-24T09:04:24.528733+0000","last_peered":"2024-04-24T09:04:24.528733+0000","last_clean":"2024-04-24T09:04:24.528733+0000","last_became_active":"2024-04-24T09:04:01.542914+0000","last_became_peered":"2024-04-24T09:04:01.542914+0000","last_unstale":"2024-04-24T09:04:24.528733+0000","last_undegraded":"2024-04-24T09:04:24.528733+0000","last_fullsized":"2024-04-24T09:04:24.528733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:50.695108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.367677+0000","last_change":"2024-04-24T09:04:02.114445+0000","last_active":"2024-04-24T09:05:53.367677+0000","last_peered":"2024-04-24T09:05:53.367677+0000","last_clean":"2024-04-24T09:05:53.367677+0000","last_became_active":"2024-04-24T09:04:02.114075+0000","last_became_peered":"2024-04-24T09:04:02.114075+0000","last_unstale":"2024-04-24T09:05:53.367677+0000","last_undegraded":"2024-04-24T09:05:53.367677+0000","last_fullsized":"2024-04-24T09:05:53.367677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:35:06.933677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.525318+0000","last_change":"2024-04-24T09:04:06.142368+0000","last_active":"2024-04-24T09:04:24.525318+0000","last_peered":"2024-04-24T09:04:24.525318+0000","last_clean":"2024-04-24T09:04:24.525318+0000","last_became_active":"2024-04-24T09:04:06.141872+0000","last_became_peered":"2024-04-24T09:04:06.141872+0000","last_unstale":"2024-04-24T09:04:24.525318+0000","last_undegraded":"2024-04-24T09:04:24.525318+0000","last_fullsized":"2024-04-24T09:04:24.525318+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:13:43.733662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":114,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:56.875692+0000","last_change":"2024-04-24T09:04:01.556002+0000","last_active":"2024-04-24T09:05:56.875692+0000","last_peered":"2024-04-24T09:05:56.875692+0000","last_clean":"2024-04-24T09:05:56.875692+0000","last_became_active":"2024-04-24T09:04:01.555643+0000","last_became_peered":"2024-04-24T09:04:01.555643+0000","last_unstale":"2024-04-24T09:05:56.875692+0000","last_undegraded":"2024-04-24T09:05:56.875692+0000","last_fullsized":"2024-04-24T09:05:56.875692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:30:31.171184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.377508+0000","last_change":"2024-04-24T09:04:02.104252+0000","last_active":"2024-04-24T09:05:53.377508+0000","last_peered":"2024-04-24T09:05:53.377508+0000","last_clean":"2024-04-24T09:05:53.377508+0000","last_became_active":"2024-04-24T09:04:02.099264+0000","last_became_peered":"2024-04-24T09:04:02.099264+0000","last_unstale":"2024-04-24T09:05:53.377508+0000","last_undegraded":"2024-04-24T09:05:53.377508+0000","last_fullsized":"2024-04-24T09:05:53.377508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:28.469814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.516701+0000","last_change":"2024-04-24T09:04:06.132407+0000","last_active":"2024-04-24T09:04:24.516701+0000","last_peered":"2024-04-24T09:04:24.516701+0000","last_clean":"2024-04-24T09:04:24.516701+0000","last_became_active":"2024-04-24T09:04:06.131997+0000","last_became_peered":"2024-04-24T09:04:06.131997+0000","last_unstale":"2024-04-24T09:04:24.516701+0000","last_undegraded":"2024-04-24T09:04:24.516701+0000","last_fullsized":"2024-04-24T09:04:24.516701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:08:07.369326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.613853+0000","last_change":"2024-04-24T09:04:01.561039+0000","last_active":"2024-04-24T09:04:32.613853+0000","last_peered":"2024-04-24T09:04:32.613853+0000","last_clean":"2024-04-24T09:04:32.613853+0000","last_became_active":"2024-04-24T09:04:01.560543+0000","last_became_peered":"2024-04-24T09:04:01.560543+0000","last_unstale":"2024-04-24T09:04:32.613853+0000","last_undegraded":"2024-04-24T09:04:32.613853+0000","last_fullsized":"2024-04-24T09:04:32.613853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:27.706982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.370657+0000","last_change":"2024-04-24T09:04:02.110850+0000","last_active":"2024-04-24T09:05:53.370657+0000","last_peered":"2024-04-24T09:05:53.370657+0000","last_clean":"2024-04-24T09:05:53.370657+0000","last_became_active":"2024-04-24T09:04:02.110512+0000","last_became_peered":"2024-04-24T09:04:02.110512+0000","last_unstale":"2024-04-24T09:05:53.370657+0000","last_undegraded":"2024-04-24T09:05:53.370657+0000","last_fullsized":"2024-04-24T09:05:53.370657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:19:16.937202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.521039+0000","last_change":"2024-04-24T09:04:06.135200+0000","last_active":"2024-04-24T09:04:24.521039+0000","last_peered":"2024-04-24T09:04:24.521039+0000","last_clean":"2024-04-24T09:04:24.521039+0000","last_became_active":"2024-04-24T09:04:06.134680+0000","last_became_peered":"2024-04-24T09:04:06.134680+0000","last_unstale":"2024-04-24T09:04:24.521039+0000","last_undegraded":"2024-04-24T09:04:24.521039+0000","last_fullsized":"2024-04-24T09:04:24.521039+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:48:57.697740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.612218+0000","last_change":"2024-04-24T09:04:01.555069+0000","last_active":"2024-04-24T09:04:32.612218+0000","last_peered":"2024-04-24T09:04:32.612218+0000","last_clean":"2024-04-24T09:04:32.612218+0000","last_became_active":"2024-04-24T09:04:01.554737+0000","last_became_peered":"2024-04-24T09:04:01.554737+0000","last_unstale":"2024-04-24T09:04:32.612218+0000","last_undegraded":"2024-04-24T09:04:32.612218+0000","last_fullsized":"2024-04-24T09:04:32.612218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:23:00.272697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.379647+0000","last_change":"2024-04-24T09:04:02.112126+0000","last_active":"2024-04-24T09:05:53.379647+0000","last_peered":"2024-04-24T09:05:53.379647+0000","last_clean":"2024-04-24T09:05:53.379647+0000","last_became_active":"2024-04-24T09:04:02.107995+0000","last_became_peered":"2024-04-24T09:04:02.107995+0000","last_unstale":"2024-04-24T09:05:53.379647+0000","last_undegraded":"2024-04-24T09:05:53.379647+0000","last_fullsized":"2024-04-24T09:05:53.379647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:13.189829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.542302+0000","last_change":"2024-04-24T09:04:24.542302+0000","last_active":"2024-04-24T09:04:24.542302+0000","last_peered":"2024-04-24T09:04:24.542302+0000","last_clean":"2024-04-24T09:04:24.542302+0000","last_became_active":"2024-04-24T09:04:24.539848+0000","last_became_peered":"2024-04-24T09:04:24.539848+0000","last_unstale":"2024-04-24T09:04:24.542302+0000","last_undegraded":"2024-04-24T09:04:24.542302+0000","last_fullsized":"2024-04-24T09:04:24.542302+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:01:52.055510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"92'41","reported_seq":247,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:57.879148+0000","last_change":"2024-04-24T09:04:01.548610+0000","last_active":"2024-04-24T09:05:57.879148+0000","last_peered":"2024-04-24T09:05:57.879148+0000","last_clean":"2024-04-24T09:05:57.879148+0000","last_became_active":"2024-04-24T09:04:01.548145+0000","last_became_peered":"2024-04-24T09:04:01.548145+0000","last_unstale":"2024-04-24T09:05:57.879148+0000","last_undegraded":"2024-04-24T09:05:57.879148+0000","last_fullsized":"2024-04-24T09:05:57.879148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:58:48.425187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66287,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":179,"num_read_kb":155,"num_write":42,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.361962+0000","last_change":"2024-04-24T09:04:02.109965+0000","last_active":"2024-04-24T09:05:53.361962+0000","last_peered":"2024-04-24T09:05:53.361962+0000","last_clean":"2024-04-24T09:05:53.361962+0000","last_became_active":"2024-04-24T09:04:02.108143+0000","last_became_peered":"2024-04-24T09:04:02.108143+0000","last_unstale":"2024-04-24T09:05:53.361962+0000","last_undegraded":"2024-04-24T09:05:53.361962+0000","last_fullsized":"2024-04-24T09:05:53.361962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:01:21.083039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.534893+0000","last_change":"2024-04-24T09:04:06.145369+0000","last_active":"2024-04-24T09:04:24.534893+0000","last_peered":"2024-04-24T09:04:24.534893+0000","last_clean":"2024-04-24T09:04:24.534893+0000","last_became_active":"2024-04-24T09:04:06.144884+0000","last_became_peered":"2024-04-24T09:04:06.144884+0000","last_unstale":"2024-04-24T09:04:24.534893+0000","last_undegraded":"2024-04-24T09:04:24.534893+0000","last_fullsized":"2024-04-24T09:04:24.534893+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:29:22.780846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.629531+0000","last_change":"2024-04-24T09:04:01.548195+0000","last_active":"2024-04-24T09:04:33.629531+0000","last_peered":"2024-04-24T09:04:33.629531+0000","last_clean":"2024-04-24T09:04:33.629531+0000","last_became_active":"2024-04-24T09:04:01.544969+0000","last_became_peered":"2024-04-24T09:04:01.544969+0000","last_unstale":"2024-04-24T09:04:33.629531+0000","last_undegraded":"2024-04-24T09:04:33.629531+0000","last_fullsized":"2024-04-24T09:04:33.629531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:32:55.294629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.367195+0000","last_change":"2024-04-24T09:04:02.103785+0000","last_active":"2024-04-24T09:05:53.367195+0000","last_peered":"2024-04-24T09:05:53.367195+0000","last_clean":"2024-04-24T09:05:53.367195+0000","last_became_active":"2024-04-24T09:04:02.103463+0000","last_became_peered":"2024-04-24T09:04:02.103463+0000","last_unstale":"2024-04-24T09:05:53.367195+0000","last_undegraded":"2024-04-24T09:05:53.367195+0000","last_fullsized":"2024-04-24T09:05:53.367195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:56.385382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527499+0000","last_change":"2024-04-24T09:04:06.149955+0000","last_active":"2024-04-24T09:04:24.527499+0000","last_peered":"2024-04-24T09:04:24.527499+0000","last_clean":"2024-04-24T09:04:24.527499+0000","last_became_active":"2024-04-24T09:04:06.149445+0000","last_became_peered":"2024-04-24T09:04:06.149445+0000","last_unstale":"2024-04-24T09:04:24.527499+0000","last_undegraded":"2024-04-24T09:04:24.527499+0000","last_fullsized":"2024-04-24T09:04:24.527499+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:30:42.527007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.615210+0000","last_change":"2024-04-24T09:04:01.548740+0000","last_active":"2024-04-24T09:04:32.615210+0000","last_peered":"2024-04-24T09:04:32.615210+0000","last_clean":"2024-04-24T09:04:32.615210+0000","last_became_active":"2024-04-24T09:04:01.546060+0000","last_became_peered":"2024-04-24T09:04:01.546060+0000","last_unstale":"2024-04-24T09:04:32.615210+0000","last_undegraded":"2024-04-24T09:04:32.615210+0000","last_fullsized":"2024-04-24T09:04:32.615210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:09:16.301150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.367883+0000","last_change":"2024-04-24T09:04:24.538871+0000","last_active":"2024-04-24T09:05:53.367883+0000","last_peered":"2024-04-24T09:05:53.367883+0000","last_clean":"2024-04-24T09:05:53.367883+0000","last_became_active":"2024-04-24T09:04:24.537039+0000","last_became_peered":"2024-04-24T09:04:24.537039+0000","last_unstale":"2024-04-24T09:05:53.367883+0000","last_undegraded":"2024-04-24T09:05:53.367883+0000","last_fullsized":"2024-04-24T09:05:53.367883+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:18.520407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.518409+0000","last_change":"2024-04-24T09:04:06.136973+0000","last_active":"2024-04-24T09:04:24.518409+0000","last_peered":"2024-04-24T09:04:24.518409+0000","last_clean":"2024-04-24T09:04:24.518409+0000","last_became_active":"2024-04-24T09:04:06.136405+0000","last_became_peered":"2024-04-24T09:04:06.136405+0000","last_unstale":"2024-04-24T09:04:24.518409+0000","last_undegraded":"2024-04-24T09:04:24.518409+0000","last_fullsized":"2024-04-24T09:04:24.518409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:07.653162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":123,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:59.894114+0000","last_change":"2024-04-24T09:04:01.546853+0000","last_active":"2024-04-24T09:05:59.894114+0000","last_peered":"2024-04-24T09:05:59.894114+0000","last_clean":"2024-04-24T09:05:59.894114+0000","last_became_active":"2024-04-24T09:04:01.546441+0000","last_became_peered":"2024-04-24T09:04:01.546441+0000","last_unstale":"2024-04-24T09:05:59.894114+0000","last_undegraded":"2024-04-24T09:05:59.894114+0000","last_fullsized":"2024-04-24T09:05:59.894114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:23:56.333410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.361044+0000","last_change":"2024-04-24T09:04:02.101158+0000","last_active":"2024-04-24T09:05:53.361044+0000","last_peered":"2024-04-24T09:05:53.361044+0000","last_clean":"2024-04-24T09:05:53.361044+0000","last_became_active":"2024-04-24T09:04:02.100832+0000","last_became_peered":"2024-04-24T09:04:02.100832+0000","last_unstale":"2024-04-24T09:05:53.361044+0000","last_undegraded":"2024-04-24T09:05:53.361044+0000","last_fullsized":"2024-04-24T09:05:53.361044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:05:34.080520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533094+0000","last_change":"2024-04-24T09:04:06.149176+0000","last_active":"2024-04-24T09:04:24.533094+0000","last_peered":"2024-04-24T09:04:24.533094+0000","last_clean":"2024-04-24T09:04:24.533094+0000","last_became_active":"2024-04-24T09:04:06.145539+0000","last_became_peered":"2024-04-24T09:04:06.145539+0000","last_unstale":"2024-04-24T09:04:24.533094+0000","last_undegraded":"2024-04-24T09:04:24.533094+0000","last_fullsized":"2024-04-24T09:04:24.533094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:38.719787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.632543+0000","last_change":"2024-04-24T09:04:01.555330+0000","last_active":"2024-04-24T09:04:33.632543+0000","last_peered":"2024-04-24T09:04:33.632543+0000","last_clean":"2024-04-24T09:04:33.632543+0000","last_became_active":"2024-04-24T09:04:01.552618+0000","last_became_peered":"2024-04-24T09:04:01.552618+0000","last_unstale":"2024-04-24T09:04:33.632543+0000","last_undegraded":"2024-04-24T09:04:33.632543+0000","last_fullsized":"2024-04-24T09:04:33.632543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:30.956657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.374740+0000","last_change":"2024-04-24T09:04:02.116301+0000","last_active":"2024-04-24T09:05:53.374740+0000","last_peered":"2024-04-24T09:05:53.374740+0000","last_clean":"2024-04-24T09:05:53.374740+0000","last_became_active":"2024-04-24T09:04:02.116001+0000","last_became_peered":"2024-04-24T09:04:02.116001+0000","last_unstale":"2024-04-24T09:05:53.374740+0000","last_undegraded":"2024-04-24T09:05:53.374740+0000","last_fullsized":"2024-04-24T09:05:53.374740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:16:18.995067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.529071+0000","last_change":"2024-04-24T09:04:06.135814+0000","last_active":"2024-04-24T09:04:24.529071+0000","last_peered":"2024-04-24T09:04:24.529071+0000","last_clean":"2024-04-24T09:04:24.529071+0000","last_became_active":"2024-04-24T09:04:06.135054+0000","last_became_peered":"2024-04-24T09:04:06.135054+0000","last_unstale":"2024-04-24T09:04:24.529071+0000","last_undegraded":"2024-04-24T09:04:24.529071+0000","last_fullsized":"2024-04-24T09:04:24.529071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:51:31.150589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:34.618324+0000","last_change":"2024-04-24T09:04:01.553144+0000","last_active":"2024-04-24T09:04:34.618324+0000","last_peered":"2024-04-24T09:04:34.618324+0000","last_clean":"2024-04-24T09:04:34.618324+0000","last_became_active":"2024-04-24T09:04:01.550599+0000","last_became_peered":"2024-04-24T09:04:01.550599+0000","last_unstale":"2024-04-24T09:04:34.618324+0000","last_undegraded":"2024-04-24T09:04:34.618324+0000","last_fullsized":"2024-04-24T09:04:34.618324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:41.950209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.363246+0000","last_change":"2024-04-24T09:04:02.117064+0000","last_active":"2024-04-24T09:05:53.363246+0000","last_peered":"2024-04-24T09:05:53.363246+0000","last_clean":"2024-04-24T09:05:53.363246+0000","last_became_active":"2024-04-24T09:04:02.114528+0000","last_became_peered":"2024-04-24T09:04:02.114528+0000","last_unstale":"2024-04-24T09:05:53.363246+0000","last_undegraded":"2024-04-24T09:05:53.363246+0000","last_fullsized":"2024-04-24T09:05:53.363246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:24:53.662571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.701058+0000","last_change":"2024-04-24T09:04:24.701058+0000","last_active":"2024-04-24T09:04:24.701058+0000","last_peered":"2024-04-24T09:04:24.701058+0000","last_clean":"2024-04-24T09:04:24.701058+0000","last_became_active":"2024-04-24T09:04:24.699766+0000","last_became_peered":"2024-04-24T09:04:24.699766+0000","last_unstale":"2024-04-24T09:04:24.701058+0000","last_undegraded":"2024-04-24T09:04:24.701058+0000","last_fullsized":"2024-04-24T09:04:24.701058+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:57:48.491073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.701816+0000","last_change":"2024-04-24T09:04:24.701816+0000","last_active":"2024-04-24T09:04:24.701816+0000","last_peered":"2024-04-24T09:04:24.701816+0000","last_clean":"2024-04-24T09:04:24.701816+0000","last_became_active":"2024-04-24T09:04:24.700530+0000","last_became_peered":"2024-04-24T09:04:24.700530+0000","last_unstale":"2024-04-24T09:04:24.701816+0000","last_undegraded":"2024-04-24T09:04:24.701816+0000","last_fullsized":"2024-04-24T09:04:24.701816+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:24.614043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.367605+0000","last_change":"2024-04-24T09:04:02.104602+0000","last_active":"2024-04-24T09:05:53.367605+0000","last_peered":"2024-04-24T09:05:53.367605+0000","last_clean":"2024-04-24T09:05:53.367605+0000","last_became_active":"2024-04-24T09:04:02.104188+0000","last_became_peered":"2024-04-24T09:04:02.104188+0000","last_unstale":"2024-04-24T09:05:53.367605+0000","last_undegraded":"2024-04-24T09:05:53.367605+0000","last_fullsized":"2024-04-24T09:05:53.367605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:00:50.286952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531886+0000","last_change":"2024-04-24T09:04:06.134971+0000","last_active":"2024-04-24T09:04:24.531886+0000","last_peered":"2024-04-24T09:04:24.531886+0000","last_clean":"2024-04-24T09:04:24.531886+0000","last_became_active":"2024-04-24T09:04:06.134551+0000","last_became_peered":"2024-04-24T09:04:06.134551+0000","last_unstale":"2024-04-24T09:04:24.531886+0000","last_undegraded":"2024-04-24T09:04:24.531886+0000","last_fullsized":"2024-04-24T09:04:24.531886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:40:41.766657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532337+0000","last_change":"2024-04-24T09:04:01.551117+0000","last_active":"2024-04-24T09:04:24.532337+0000","last_peered":"2024-04-24T09:04:24.532337+0000","last_clean":"2024-04-24T09:04:24.532337+0000","last_became_active":"2024-04-24T09:04:01.550646+0000","last_became_peered":"2024-04-24T09:04:01.550646+0000","last_unstale":"2024-04-24T09:04:24.532337+0000","last_undegraded":"2024-04-24T09:04:24.532337+0000","last_fullsized":"2024-04-24T09:04:24.532337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:29:03.821300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.366569+0000","last_change":"2024-04-24T09:04:02.107111+0000","last_active":"2024-04-24T09:05:53.366569+0000","last_peered":"2024-04-24T09:05:53.366569+0000","last_clean":"2024-04-24T09:05:53.366569+0000","last_became_active":"2024-04-24T09:04:02.106185+0000","last_became_peered":"2024-04-24T09:04:02.106185+0000","last_unstale":"2024-04-24T09:05:53.366569+0000","last_undegraded":"2024-04-24T09:05:53.366569+0000","last_fullsized":"2024-04-24T09:05:53.366569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:19:03.111407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532094+0000","last_change":"2024-04-24T09:04:06.143984+0000","last_active":"2024-04-24T09:04:24.532094+0000","last_peered":"2024-04-24T09:04:24.532094+0000","last_clean":"2024-04-24T09:04:24.532094+0000","last_became_active":"2024-04-24T09:04:06.143503+0000","last_became_peered":"2024-04-24T09:04:06.143503+0000","last_unstale":"2024-04-24T09:04:24.532094+0000","last_undegraded":"2024-04-24T09:04:24.532094+0000","last_fullsized":"2024-04-24T09:04:24.532094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:35:19.317771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.352379+0000","last_change":"2024-04-24T09:04:02.111560+0000","last_active":"2024-04-24T09:05:53.352379+0000","last_peered":"2024-04-24T09:05:53.352379+0000","last_clean":"2024-04-24T09:05:53.352379+0000","last_became_active":"2024-04-24T09:04:02.109885+0000","last_became_peered":"2024-04-24T09:04:02.109885+0000","last_unstale":"2024-04-24T09:05:53.352379+0000","last_undegraded":"2024-04-24T09:05:53.352379+0000","last_fullsized":"2024-04-24T09:05:53.352379+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:35.837621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":114,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:54.878157+0000","last_change":"2024-04-24T09:04:01.542530+0000","last_active":"2024-04-24T09:05:54.878157+0000","last_peered":"2024-04-24T09:05:54.878157+0000","last_clean":"2024-04-24T09:05:54.878157+0000","last_became_active":"2024-04-24T09:04:01.541702+0000","last_became_peered":"2024-04-24T09:04:01.541702+0000","last_unstale":"2024-04-24T09:05:54.878157+0000","last_undegraded":"2024-04-24T09:05:54.878157+0000","last_fullsized":"2024-04-24T09:05:54.878157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:28:29.420020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532686+0000","last_change":"2024-04-24T09:04:06.143004+0000","last_active":"2024-04-24T09:04:24.532686+0000","last_peered":"2024-04-24T09:04:24.532686+0000","last_clean":"2024-04-24T09:04:24.532686+0000","last_became_active":"2024-04-24T09:04:06.142444+0000","last_became_peered":"2024-04-24T09:04:06.142444+0000","last_unstale":"2024-04-24T09:04:24.532686+0000","last_undegraded":"2024-04-24T09:04:24.532686+0000","last_fullsized":"2024-04-24T09:04:24.532686+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:13.910612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.375464+0000","last_change":"2024-04-24T09:04:24.701330+0000","last_active":"2024-04-24T09:05:53.375464+0000","last_peered":"2024-04-24T09:05:53.375464+0000","last_clean":"2024-04-24T09:05:53.375464+0000","last_became_active":"2024-04-24T09:04:24.699988+0000","last_became_peered":"2024-04-24T09:04:24.699988+0000","last_unstale":"2024-04-24T09:05:53.375464+0000","last_undegraded":"2024-04-24T09:05:53.375464+0000","last_fullsized":"2024-04-24T09:05:53.375464+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:23:24.610671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:34.629794+0000","last_change":"2024-04-24T09:04:01.541206+0000","last_active":"2024-04-24T09:04:34.629794+0000","last_peered":"2024-04-24T09:04:34.629794+0000","last_clean":"2024-04-24T09:04:34.629794+0000","last_became_active":"2024-04-24T09:04:01.539888+0000","last_became_peered":"2024-04-24T09:04:01.539888+0000","last_unstale":"2024-04-24T09:04:34.629794+0000","last_undegraded":"2024-04-24T09:04:34.629794+0000","last_fullsized":"2024-04-24T09:04:34.629794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:01.550209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531165+0000","last_change":"2024-04-24T09:02:22.629840+0000","last_active":"2024-04-24T09:04:24.531165+0000","last_peered":"2024-04-24T09:04:24.531165+0000","last_clean":"2024-04-24T09:04:24.531165+0000","last_became_active":"2024-04-24T09:02:22.628700+0000","last_became_peered":"2024-04-24T09:02:22.628700+0000","last_unstale":"2024-04-24T09:04:24.531165+0000","last_undegraded":"2024-04-24T09:04:24.531165+0000","last_fullsized":"2024-04-24T09:04:24.531165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:57:38.349295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:57:38.349295+0000","last_clean_scrub_stamp":"2024-04-24T08:57:38.349295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:54:11.409492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533333+0000","last_change":"2024-04-24T09:04:06.149548+0000","last_active":"2024-04-24T09:04:24.533333+0000","last_peered":"2024-04-24T09:04:24.533333+0000","last_clean":"2024-04-24T09:04:24.533333+0000","last_became_active":"2024-04-24T09:04:06.149214+0000","last_became_peered":"2024-04-24T09:04:06.149214+0000","last_unstale":"2024-04-24T09:04:24.533333+0000","last_undegraded":"2024-04-24T09:04:24.533333+0000","last_fullsized":"2024-04-24T09:04:24.533333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:03.801242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.352320+0000","last_change":"2024-04-24T09:04:02.110242+0000","last_active":"2024-04-24T09:05:53.352320+0000","last_peered":"2024-04-24T09:05:53.352320+0000","last_clean":"2024-04-24T09:05:53.352320+0000","last_became_active":"2024-04-24T09:04:02.109879+0000","last_became_peered":"2024-04-24T09:04:02.109879+0000","last_unstale":"2024-04-24T09:05:53.352320+0000","last_undegraded":"2024-04-24T09:05:53.352320+0000","last_fullsized":"2024-04-24T09:05:53.352320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:16.353861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.628491+0000","last_change":"2024-04-24T09:04:01.550989+0000","last_active":"2024-04-24T09:04:33.628491+0000","last_peered":"2024-04-24T09:04:33.628491+0000","last_clean":"2024-04-24T09:04:33.628491+0000","last_became_active":"2024-04-24T09:04:01.550664+0000","last_became_peered":"2024-04-24T09:04:01.550664+0000","last_unstale":"2024-04-24T09:04:33.628491+0000","last_undegraded":"2024-04-24T09:04:33.628491+0000","last_fullsized":"2024-04-24T09:04:33.628491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:33:39.521951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.538207+0000","last_change":"2024-04-24T09:04:24.538207+0000","last_active":"2024-04-24T09:04:24.538207+0000","last_peered":"2024-04-24T09:04:24.538207+0000","last_clean":"2024-04-24T09:04:24.538207+0000","last_became_active":"2024-04-24T09:04:24.536894+0000","last_became_peered":"2024-04-24T09:04:24.536894+0000","last_unstale":"2024-04-24T09:04:24.538207+0000","last_undegraded":"2024-04-24T09:04:24.538207+0000","last_fullsized":"2024-04-24T09:04:24.538207+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:32:52.860371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.608599+0000","last_change":"2024-04-24T09:04:01.539415+0000","last_active":"2024-04-24T09:04:32.608599+0000","last_peered":"2024-04-24T09:04:32.608599+0000","last_clean":"2024-04-24T09:04:32.608599+0000","last_became_active":"2024-04-24T09:04:01.539020+0000","last_became_peered":"2024-04-24T09:04:01.539020+0000","last_unstale":"2024-04-24T09:04:32.608599+0000","last_undegraded":"2024-04-24T09:04:32.608599+0000","last_fullsized":"2024-04-24T09:04:32.608599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:12:48.728029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.353539+0000","last_change":"2024-04-24T09:04:02.097560+0000","last_active":"2024-04-24T09:05:53.353539+0000","last_peered":"2024-04-24T09:05:53.353539+0000","last_clean":"2024-04-24T09:05:53.353539+0000","last_became_active":"2024-04-24T09:04:02.095612+0000","last_became_peered":"2024-04-24T09:04:02.095612+0000","last_unstale":"2024-04-24T09:05:53.353539+0000","last_undegraded":"2024-04-24T09:05:53.353539+0000","last_fullsized":"2024-04-24T09:05:53.353539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:55:47.747294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.524769+0000","last_change":"2024-04-24T09:04:06.137822+0000","last_active":"2024-04-24T09:04:24.524769+0000","last_peered":"2024-04-24T09:04:24.524769+0000","last_clean":"2024-04-24T09:04:24.524769+0000","last_became_active":"2024-04-24T09:04:06.137357+0000","last_became_peered":"2024-04-24T09:04:06.137357+0000","last_unstale":"2024-04-24T09:04:24.524769+0000","last_undegraded":"2024-04-24T09:04:24.524769+0000","last_fullsized":"2024-04-24T09:04:24.524769+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:55:13.119557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'10","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:43.555942+0000","last_change":"2024-04-24T09:04:24.544395+0000","last_active":"2024-04-24T09:04:43.555942+0000","last_peered":"2024-04-24T09:04:43.555942+0000","last_clean":"2024-04-24T09:04:43.555942+0000","last_became_active":"2024-04-24T09:04:24.535870+0000","last_became_peered":"2024-04-24T09:04:24.535870+0000","last_unstale":"2024-04-24T09:04:43.555942+0000","last_undegraded":"2024-04-24T09:04:43.555942+0000","last_fullsized":"2024-04-24T09:04:43.555942+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:29:21.271387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.374330+0000","last_change":"2024-04-24T09:04:02.106107+0000","last_active":"2024-04-24T09:05:53.374330+0000","last_peered":"2024-04-24T09:05:53.374330+0000","last_clean":"2024-04-24T09:05:53.374330+0000","last_became_active":"2024-04-24T09:04:02.105734+0000","last_became_peered":"2024-04-24T09:04:02.105734+0000","last_unstale":"2024-04-24T09:05:53.374330+0000","last_undegraded":"2024-04-24T09:05:53.374330+0000","last_fullsized":"2024-04-24T09:05:53.374330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:15.372520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.695729+0000","last_change":"2024-04-24T09:04:06.139040+0000","last_active":"2024-04-24T09:04:24.695729+0000","last_peered":"2024-04-24T09:04:24.695729+0000","last_clean":"2024-04-24T09:04:24.695729+0000","last_became_active":"2024-04-24T09:04:06.138532+0000","last_became_peered":"2024-04-24T09:04:06.138532+0000","last_unstale":"2024-04-24T09:04:24.695729+0000","last_undegraded":"2024-04-24T09:04:24.695729+0000","last_fullsized":"2024-04-24T09:04:24.695729+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:28.686052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.612766+0000","last_change":"2024-04-24T09:04:01.546859+0000","last_active":"2024-04-24T09:04:32.612766+0000","last_peered":"2024-04-24T09:04:32.612766+0000","last_clean":"2024-04-24T09:04:32.612766+0000","last_became_active":"2024-04-24T09:04:01.546463+0000","last_became_peered":"2024-04-24T09:04:01.546463+0000","last_unstale":"2024-04-24T09:04:32.612766+0000","last_undegraded":"2024-04-24T09:04:32.612766+0000","last_fullsized":"2024-04-24T09:04:32.612766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:31:12.985687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.370925+0000","last_change":"2024-04-24T09:04:02.112487+0000","last_active":"2024-04-24T09:05:53.370925+0000","last_peered":"2024-04-24T09:05:53.370925+0000","last_clean":"2024-04-24T09:05:53.370925+0000","last_became_active":"2024-04-24T09:04:02.112078+0000","last_became_peered":"2024-04-24T09:04:02.112078+0000","last_unstale":"2024-04-24T09:05:53.370925+0000","last_undegraded":"2024-04-24T09:05:53.370925+0000","last_fullsized":"2024-04-24T09:05:53.370925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:03:47.352110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.539120+0000","last_change":"2024-04-24T09:04:24.539120+0000","last_active":"2024-04-24T09:04:24.539120+0000","last_peered":"2024-04-24T09:04:24.539120+0000","last_clean":"2024-04-24T09:04:24.539120+0000","last_became_active":"2024-04-24T09:04:24.537048+0000","last_became_peered":"2024-04-24T09:04:24.537048+0000","last_unstale":"2024-04-24T09:04:24.539120+0000","last_undegraded":"2024-04-24T09:04:24.539120+0000","last_fullsized":"2024-04-24T09:04:24.539120+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:29:53.484131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.611836+0000","last_change":"2024-04-24T09:04:01.545444+0000","last_active":"2024-04-24T09:04:32.611836+0000","last_peered":"2024-04-24T09:04:32.611836+0000","last_clean":"2024-04-24T09:04:32.611836+0000","last_became_active":"2024-04-24T09:04:01.543816+0000","last_became_peered":"2024-04-24T09:04:01.543816+0000","last_unstale":"2024-04-24T09:04:32.611836+0000","last_undegraded":"2024-04-24T09:04:32.611836+0000","last_fullsized":"2024-04-24T09:04:32.611836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:02:36.221797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.306910+0000","last_change":"2024-04-24T09:04:02.109085+0000","last_active":"2024-04-24T09:05:53.306910+0000","last_peered":"2024-04-24T09:05:53.306910+0000","last_clean":"2024-04-24T09:05:53.306910+0000","last_became_active":"2024-04-24T09:04:02.108642+0000","last_became_peered":"2024-04-24T09:04:02.108642+0000","last_unstale":"2024-04-24T09:05:53.306910+0000","last_undegraded":"2024-04-24T09:05:53.306910+0000","last_fullsized":"2024-04-24T09:05:53.306910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:27:51.751369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.518972+0000","last_change":"2024-04-24T09:04:06.144940+0000","last_active":"2024-04-24T09:04:24.518972+0000","last_peered":"2024-04-24T09:04:24.518972+0000","last_clean":"2024-04-24T09:04:24.518972+0000","last_became_active":"2024-04-24T09:04:06.144553+0000","last_became_peered":"2024-04-24T09:04:06.144553+0000","last_unstale":"2024-04-24T09:04:24.518972+0000","last_undegraded":"2024-04-24T09:04:24.518972+0000","last_fullsized":"2024-04-24T09:04:24.518972+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:17.455644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.610185+0000","last_change":"2024-04-24T09:04:01.538939+0000","last_active":"2024-04-24T09:04:32.610185+0000","last_peered":"2024-04-24T09:04:32.610185+0000","last_clean":"2024-04-24T09:04:32.610185+0000","last_became_active":"2024-04-24T09:04:01.538541+0000","last_became_peered":"2024-04-24T09:04:01.538541+0000","last_unstale":"2024-04-24T09:04:32.610185+0000","last_undegraded":"2024-04-24T09:04:32.610185+0000","last_fullsized":"2024-04-24T09:04:32.610185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:08.185901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.359806+0000","last_change":"2024-04-24T09:04:02.103844+0000","last_active":"2024-04-24T09:05:53.359806+0000","last_peered":"2024-04-24T09:05:53.359806+0000","last_clean":"2024-04-24T09:05:53.359806+0000","last_became_active":"2024-04-24T09:04:02.103416+0000","last_became_peered":"2024-04-24T09:04:02.103416+0000","last_unstale":"2024-04-24T09:05:53.359806+0000","last_undegraded":"2024-04-24T09:05:53.359806+0000","last_fullsized":"2024-04-24T09:05:53.359806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:57:47.236384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.520182+0000","last_change":"2024-04-24T09:04:06.134838+0000","last_active":"2024-04-24T09:04:24.520182+0000","last_peered":"2024-04-24T09:04:24.520182+0000","last_clean":"2024-04-24T09:04:24.520182+0000","last_became_active":"2024-04-24T09:04:06.134343+0000","last_became_peered":"2024-04-24T09:04:06.134343+0000","last_unstale":"2024-04-24T09:04:24.520182+0000","last_undegraded":"2024-04-24T09:04:24.520182+0000","last_fullsized":"2024-04-24T09:04:24.520182+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:56:55.219012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696139+0000","last_change":"2024-04-24T09:04:01.557194+0000","last_active":"2024-04-24T09:04:24.696139+0000","last_peered":"2024-04-24T09:04:24.696139+0000","last_clean":"2024-04-24T09:04:24.696139+0000","last_became_active":"2024-04-24T09:04:01.556441+0000","last_became_peered":"2024-04-24T09:04:01.556441+0000","last_unstale":"2024-04-24T09:04:24.696139+0000","last_undegraded":"2024-04-24T09:04:24.696139+0000","last_fullsized":"2024-04-24T09:04:24.696139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:45:50.399096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.372082+0000","last_change":"2024-04-24T09:04:02.108076+0000","last_active":"2024-04-24T09:05:53.372082+0000","last_peered":"2024-04-24T09:05:53.372082+0000","last_clean":"2024-04-24T09:05:53.372082+0000","last_became_active":"2024-04-24T09:04:02.106829+0000","last_became_peered":"2024-04-24T09:04:02.106829+0000","last_unstale":"2024-04-24T09:05:53.372082+0000","last_undegraded":"2024-04-24T09:05:53.372082+0000","last_fullsized":"2024-04-24T09:05:53.372082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:40:23.075828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.521106+0000","last_change":"2024-04-24T09:04:06.142269+0000","last_active":"2024-04-24T09:04:24.521106+0000","last_peered":"2024-04-24T09:04:24.521106+0000","last_clean":"2024-04-24T09:04:24.521106+0000","last_became_active":"2024-04-24T09:04:06.141725+0000","last_became_peered":"2024-04-24T09:04:06.141725+0000","last_unstale":"2024-04-24T09:04:24.521106+0000","last_undegraded":"2024-04-24T09:04:24.521106+0000","last_fullsized":"2024-04-24T09:04:24.521106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:28:17.934148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.615156+0000","last_change":"2024-04-24T09:04:01.553134+0000","last_active":"2024-04-24T09:04:32.615156+0000","last_peered":"2024-04-24T09:04:32.615156+0000","last_clean":"2024-04-24T09:04:32.615156+0000","last_became_active":"2024-04-24T09:04:01.552099+0000","last_became_peered":"2024-04-24T09:04:01.552099+0000","last_unstale":"2024-04-24T09:04:32.615156+0000","last_undegraded":"2024-04-24T09:04:32.615156+0000","last_fullsized":"2024-04-24T09:04:32.615156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:42:36.231405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.379961+0000","last_change":"2024-04-24T09:04:02.116641+0000","last_active":"2024-04-24T09:05:53.379961+0000","last_peered":"2024-04-24T09:05:53.379961+0000","last_clean":"2024-04-24T09:05:53.379961+0000","last_became_active":"2024-04-24T09:04:02.115374+0000","last_became_peered":"2024-04-24T09:04:02.115374+0000","last_unstale":"2024-04-24T09:05:53.379961+0000","last_undegraded":"2024-04-24T09:05:53.379961+0000","last_fullsized":"2024-04-24T09:05:53.379961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:34.450990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696698+0000","last_change":"2024-04-24T09:04:06.149968+0000","last_active":"2024-04-24T09:04:24.696698+0000","last_peered":"2024-04-24T09:04:24.696698+0000","last_clean":"2024-04-24T09:04:24.696698+0000","last_became_active":"2024-04-24T09:04:06.149425+0000","last_became_peered":"2024-04-24T09:04:06.149425+0000","last_unstale":"2024-04-24T09:04:24.696698+0000","last_undegraded":"2024-04-24T09:04:24.696698+0000","last_fullsized":"2024-04-24T09:04:24.696698+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:23:53.869004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.633039+0000","last_change":"2024-04-24T09:04:01.553822+0000","last_active":"2024-04-24T09:04:33.633039+0000","last_peered":"2024-04-24T09:04:33.633039+0000","last_clean":"2024-04-24T09:04:33.633039+0000","last_became_active":"2024-04-24T09:04:01.551468+0000","last_became_peered":"2024-04-24T09:04:01.551468+0000","last_unstale":"2024-04-24T09:04:33.633039+0000","last_undegraded":"2024-04-24T09:04:33.633039+0000","last_fullsized":"2024-04-24T09:04:33.633039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:56.367584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.378945+0000","last_change":"2024-04-24T09:04:02.115472+0000","last_active":"2024-04-24T09:05:53.378945+0000","last_peered":"2024-04-24T09:05:53.378945+0000","last_clean":"2024-04-24T09:05:53.378945+0000","last_became_active":"2024-04-24T09:04:02.114666+0000","last_became_peered":"2024-04-24T09:04:02.114666+0000","last_unstale":"2024-04-24T09:05:53.378945+0000","last_undegraded":"2024-04-24T09:05:53.378945+0000","last_fullsized":"2024-04-24T09:05:53.378945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:28:39.145108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697174+0000","last_change":"2024-04-24T09:04:06.138786+0000","last_active":"2024-04-24T09:04:24.697174+0000","last_peered":"2024-04-24T09:04:24.697174+0000","last_clean":"2024-04-24T09:04:24.697174+0000","last_became_active":"2024-04-24T09:04:06.138366+0000","last_became_peered":"2024-04-24T09:04:06.138366+0000","last_unstale":"2024-04-24T09:04:24.697174+0000","last_undegraded":"2024-04-24T09:04:24.697174+0000","last_fullsized":"2024-04-24T09:04:24.697174+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:03:16.729928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533806+0000","last_change":"2024-04-24T09:04:01.547733+0000","last_active":"2024-04-24T09:04:24.533806+0000","last_peered":"2024-04-24T09:04:24.533806+0000","last_clean":"2024-04-24T09:04:24.533806+0000","last_became_active":"2024-04-24T09:04:01.544839+0000","last_became_peered":"2024-04-24T09:04:01.544839+0000","last_unstale":"2024-04-24T09:04:24.533806+0000","last_undegraded":"2024-04-24T09:04:24.533806+0000","last_fullsized":"2024-04-24T09:04:24.533806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:45:03.847092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.373123+0000","last_change":"2024-04-24T09:04:02.108383+0000","last_active":"2024-04-24T09:05:53.373123+0000","last_peered":"2024-04-24T09:05:53.373123+0000","last_clean":"2024-04-24T09:05:53.373123+0000","last_became_active":"2024-04-24T09:04:02.107883+0000","last_became_peered":"2024-04-24T09:04:02.107883+0000","last_unstale":"2024-04-24T09:05:53.373123+0000","last_undegraded":"2024-04-24T09:05:53.373123+0000","last_fullsized":"2024-04-24T09:05:53.373123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:24:44.923708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527772+0000","last_change":"2024-04-24T09:04:06.140868+0000","last_active":"2024-04-24T09:04:24.527772+0000","last_peered":"2024-04-24T09:04:24.527772+0000","last_clean":"2024-04-24T09:04:24.527772+0000","last_became_active":"2024-04-24T09:04:06.139174+0000","last_became_peered":"2024-04-24T09:04:06.139174+0000","last_unstale":"2024-04-24T09:04:24.527772+0000","last_undegraded":"2024-04-24T09:04:24.527772+0000","last_fullsized":"2024-04-24T09:04:24.527772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:17:17.334276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533054+0000","last_change":"2024-04-24T09:04:01.534653+0000","last_active":"2024-04-24T09:04:24.533054+0000","last_peered":"2024-04-24T09:04:24.533054+0000","last_clean":"2024-04-24T09:04:24.533054+0000","last_became_active":"2024-04-24T09:04:01.534312+0000","last_became_peered":"2024-04-24T09:04:01.534312+0000","last_unstale":"2024-04-24T09:04:24.533054+0000","last_undegraded":"2024-04-24T09:04:24.533054+0000","last_fullsized":"2024-04-24T09:04:24.533054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:53:34.301007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.365862+0000","last_change":"2024-04-24T09:04:02.102056+0000","last_active":"2024-04-24T09:05:53.365862+0000","last_peered":"2024-04-24T09:05:53.365862+0000","last_clean":"2024-04-24T09:05:53.365862+0000","last_became_active":"2024-04-24T09:04:02.101641+0000","last_became_peered":"2024-04-24T09:04:02.101641+0000","last_unstale":"2024-04-24T09:05:53.365862+0000","last_undegraded":"2024-04-24T09:05:53.365862+0000","last_fullsized":"2024-04-24T09:05:53.365862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:41.654024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.362819+0000","last_change":"2024-04-24T09:04:02.100327+0000","last_active":"2024-04-24T09:05:53.362819+0000","last_peered":"2024-04-24T09:05:53.362819+0000","last_clean":"2024-04-24T09:05:53.362819+0000","last_became_active":"2024-04-24T09:04:02.100028+0000","last_became_peered":"2024-04-24T09:04:02.100028+0000","last_unstale":"2024-04-24T09:05:53.362819+0000","last_undegraded":"2024-04-24T09:05:53.362819+0000","last_fullsized":"2024-04-24T09:05:53.362819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:15:18.728045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.355093+0000","last_change":"2024-04-24T09:04:02.101996+0000","last_active":"2024-04-24T09:05:53.355093+0000","last_peered":"2024-04-24T09:05:53.355093+0000","last_clean":"2024-04-24T09:05:53.355093+0000","last_became_active":"2024-04-24T09:04:02.101581+0000","last_became_peered":"2024-04-24T09:04:02.101581+0000","last_unstale":"2024-04-24T09:05:53.355093+0000","last_undegraded":"2024-04-24T09:05:53.355093+0000","last_fullsized":"2024-04-24T09:05:53.355093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:55.487781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.362367+0000","last_change":"2024-04-24T09:04:02.107288+0000","last_active":"2024-04-24T09:05:53.362367+0000","last_peered":"2024-04-24T09:05:53.362367+0000","last_clean":"2024-04-24T09:05:53.362367+0000","last_became_active":"2024-04-24T09:04:02.106946+0000","last_became_peered":"2024-04-24T09:04:02.106946+0000","last_unstale":"2024-04-24T09:05:53.362367+0000","last_undegraded":"2024-04-24T09:05:53.362367+0000","last_fullsized":"2024-04-24T09:05:53.362367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:38:56.821285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.366560+0000","last_change":"2024-04-24T09:04:02.116829+0000","last_active":"2024-04-24T09:05:53.366560+0000","last_peered":"2024-04-24T09:05:53.366560+0000","last_clean":"2024-04-24T09:05:53.366560+0000","last_became_active":"2024-04-24T09:04:02.116520+0000","last_became_peered":"2024-04-24T09:04:02.116520+0000","last_unstale":"2024-04-24T09:05:53.366560+0000","last_undegraded":"2024-04-24T09:05:53.366560+0000","last_fullsized":"2024-04-24T09:05:53.366560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:26.873826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.369610+0000","last_change":"2024-04-24T09:04:24.700595+0000","last_active":"2024-04-24T09:05:53.369610+0000","last_peered":"2024-04-24T09:05:53.369610+0000","last_clean":"2024-04-24T09:05:53.369610+0000","last_became_active":"2024-04-24T09:04:24.699395+0000","last_became_peered":"2024-04-24T09:04:24.699395+0000","last_unstale":"2024-04-24T09:05:53.369610+0000","last_undegraded":"2024-04-24T09:05:53.369610+0000","last_fullsized":"2024-04-24T09:05:53.369610+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:28.756302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.362263+0000","last_change":"2024-04-24T09:04:02.114087+0000","last_active":"2024-04-24T09:05:53.362263+0000","last_peered":"2024-04-24T09:05:53.362263+0000","last_clean":"2024-04-24T09:05:53.362263+0000","last_became_active":"2024-04-24T09:04:02.112681+0000","last_became_peered":"2024-04-24T09:04:02.112681+0000","last_unstale":"2024-04-24T09:05:53.362263+0000","last_undegraded":"2024-04-24T09:05:53.362263+0000","last_fullsized":"2024-04-24T09:05:53.362263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:50:01.762598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.379473+0000","last_change":"2024-04-24T09:04:02.102322+0000","last_active":"2024-04-24T09:05:53.379473+0000","last_peered":"2024-04-24T09:05:53.379473+0000","last_clean":"2024-04-24T09:05:53.379473+0000","last_became_active":"2024-04-24T09:04:02.101963+0000","last_became_peered":"2024-04-24T09:04:02.101963+0000","last_unstale":"2024-04-24T09:05:53.379473+0000","last_undegraded":"2024-04-24T09:05:53.379473+0000","last_fullsized":"2024-04-24T09:05:53.379473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:16.140829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.356404+0000","last_change":"2024-04-24T09:04:02.110570+0000","last_active":"2024-04-24T09:05:53.356404+0000","last_peered":"2024-04-24T09:05:53.356404+0000","last_clean":"2024-04-24T09:05:53.356404+0000","last_became_active":"2024-04-24T09:04:02.110119+0000","last_became_peered":"2024-04-24T09:04:02.110119+0000","last_unstale":"2024-04-24T09:05:53.356404+0000","last_undegraded":"2024-04-24T09:05:53.356404+0000","last_fullsized":"2024-04-24T09:05:53.356404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:34.436430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.373511+0000","last_change":"2024-04-24T09:04:02.108292+0000","last_active":"2024-04-24T09:05:53.373511+0000","last_peered":"2024-04-24T09:05:53.373511+0000","last_clean":"2024-04-24T09:05:53.373511+0000","last_became_active":"2024-04-24T09:04:02.107481+0000","last_became_peered":"2024-04-24T09:04:02.107481+0000","last_unstale":"2024-04-24T09:05:53.373511+0000","last_undegraded":"2024-04-24T09:05:53.373511+0000","last_fullsized":"2024-04-24T09:05:53.373511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:49.755293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.371257+0000","last_change":"2024-04-24T09:04:02.102228+0000","last_active":"2024-04-24T09:05:53.371257+0000","last_peered":"2024-04-24T09:05:53.371257+0000","last_clean":"2024-04-24T09:05:53.371257+0000","last_became_active":"2024-04-24T09:04:02.101863+0000","last_became_peered":"2024-04-24T09:04:02.101863+0000","last_unstale":"2024-04-24T09:05:53.371257+0000","last_undegraded":"2024-04-24T09:05:53.371257+0000","last_fullsized":"2024-04-24T09:05:53.371257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:02:47.308116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.379324+0000","last_change":"2024-04-24T09:04:02.106749+0000","last_active":"2024-04-24T09:05:53.379324+0000","last_peered":"2024-04-24T09:05:53.379324+0000","last_clean":"2024-04-24T09:05:53.379324+0000","last_became_active":"2024-04-24T09:04:02.106282+0000","last_became_peered":"2024-04-24T09:04:02.106282+0000","last_unstale":"2024-04-24T09:05:53.379324+0000","last_undegraded":"2024-04-24T09:05:53.379324+0000","last_fullsized":"2024-04-24T09:05:53.379324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:35:59.765230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.369805+0000","last_change":"2024-04-24T09:04:02.097825+0000","last_active":"2024-04-24T09:05:53.369805+0000","last_peered":"2024-04-24T09:05:53.369805+0000","last_clean":"2024-04-24T09:05:53.369805+0000","last_became_active":"2024-04-24T09:04:02.097488+0000","last_became_peered":"2024-04-24T09:04:02.097488+0000","last_unstale":"2024-04-24T09:05:53.369805+0000","last_undegraded":"2024-04-24T09:05:53.369805+0000","last_fullsized":"2024-04-24T09:05:53.369805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:05:59.962558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.357499+0000","last_change":"2024-04-24T09:04:02.113819+0000","last_active":"2024-04-24T09:05:53.357499+0000","last_peered":"2024-04-24T09:05:53.357499+0000","last_clean":"2024-04-24T09:05:53.357499+0000","last_became_active":"2024-04-24T09:04:02.113463+0000","last_became_peered":"2024-04-24T09:04:02.113463+0000","last_unstale":"2024-04-24T09:05:53.357499+0000","last_undegraded":"2024-04-24T09:05:53.357499+0000","last_fullsized":"2024-04-24T09:05:53.357499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:47:05.109583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70429,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":539,"num_read_kb":341,"num_write":175,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":211287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":507486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32868,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712284,"statfs":{"total":95995035648,"available":95961378816,"internally_reserved":0,"allocated":1732608,"data_stored":1161573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808887,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":511818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070525,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":507642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332165,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32784,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712368,"statfs":{"total":95995035648,"available":95961464832,"internally_reserved":0,"allocated":1646592,"data_stored":1103692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495692,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":502898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":569095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738484,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":504286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":569823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182497,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":503092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920858,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":508702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757332,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32792,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712360,"statfs":{"total":95995035648,"available":95961456640,"internally_reserved":0,"allocated":1654784,"data_stored":1098748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T09:06:03.032 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T09:06:03.519 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:06:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:03 smithi081 ceph-mon[30961]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:06:03.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:03 smithi069 ceph-mon[28170]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:06:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:03 smithi130 ceph-mon[31553]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:06:04.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:04 smithi081 ceph-mon[30961]: pgmap v432: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T09:06:04.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:04 smithi069 ceph-mon[28170]: pgmap v432: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T09:06:04.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:04 smithi130 ceph-mon[31553]: pgmap v432: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T09:06:05.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.749+0000 7f8b65c00700 1 -- 172.21.15.69:0/492531779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b60077020 msgr2=0x7f8b60077400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:05.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.749+0000 7f8b65c00700 1 --2- 172.21.15.69:0/492531779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b60077020 0x7f8b60077400 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009990 tx=0x7f8b50009ca0 comp rx=0 tx=0).stop 2024-04-24T09:06:05.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.749+0000 7f8b65c00700 1 -- 172.21.15.69:0/492531779 shutdown_connections 2024-04-24T09:06:05.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.749+0000 7f8b65c00700 1 --2- 172.21.15.69:0/492531779 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b601139a0 0x7f8b60115d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:05.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.749+0000 7f8b65c00700 1 --2- 172.21.15.69:0/492531779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b60077020 0x7f8b60077400 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:05.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.749+0000 7f8b65c00700 1 --2- 172.21.15.69:0/492531779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b60077940 0x7f8b60113460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:05.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.749+0000 7f8b65c00700 1 -- 172.21.15.69:0/492531779 >> 172.21.15.69:0/492531779 conn(0x7f8b601007f0 msgr2=0x7f8b60102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:05.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.749+0000 7f8b65c00700 1 -- 172.21.15.69:0/492531779 shutdown_connections 2024-04-24T09:06:05.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.750+0000 7f8b65c00700 1 -- 172.21.15.69:0/492531779 wait complete. 2024-04-24T09:06:05.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.751+0000 7f8b65c00700 1 Processor -- start 2024-04-24T09:06:05.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.751+0000 7f8b65c00700 1 -- start start 2024-04-24T09:06:05.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.751+0000 7f8b65c00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b60077020 0x7f8b601a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:05.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.751+0000 7f8b65c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b60077940 0x7f8b601a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:05.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.751+0000 7f8b65c00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b601139a0 0x7f8b6019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:05.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.751+0000 7f8b65c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8b60119380 con 0x7f8b60077940 2024-04-24T09:06:05.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.751+0000 7f8b65c00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8b60119200 con 0x7f8b601139a0 2024-04-24T09:06:05.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.751+0000 7f8b65c00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8b60119500 con 0x7f8b60077020 2024-04-24T09:06:05.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.752+0000 7f8b57e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b60077940 0x7f8b601a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:05.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.752+0000 7f8b57e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b60077940 0x7f8b601a5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35340/0 (socket says 172.21.15.69:35340) 2024-04-24T09:06:05.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.752+0000 7f8b57e00700 1 -- 172.21.15.69:0/1949611701 learned_addr learned my addr 172.21.15.69:0/1949611701 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:05.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.752+0000 7f8b5ea00700 1 --2- 172.21.15.69:0/1949611701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b60077020 0x7f8b601a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:05.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.752+0000 7f8b5f400700 1 --2- 172.21.15.69:0/1949611701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b601139a0 0x7f8b6019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:05.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.752+0000 7f8b57e00700 1 -- 172.21.15.69:0/1949611701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b60077020 msgr2=0x7f8b601a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:05.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.752+0000 7f8b57e00700 1 --2- 172.21.15.69:0/1949611701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b60077020 0x7f8b601a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:05.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.752+0000 7f8b57e00700 1 -- 172.21.15.69:0/1949611701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b601139a0 msgr2=0x7f8b6019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:05.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.752+0000 7f8b57e00700 1 --2- 172.21.15.69:0/1949611701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b601139a0 0x7f8b6019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:05.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.752+0000 7f8b57e00700 1 -- 172.21.15.69:0/1949611701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6019ff80 con 0x7f8b60077940 2024-04-24T09:06:05.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.752+0000 7f8b5ea00700 1 --2- 172.21.15.69:0/1949611701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b60077020 0x7f8b601a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:05.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.753+0000 7f8b57e00700 1 --2- 172.21.15.69:0/1949611701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b60077940 0x7f8b601a5b10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800d950 tx=0x7f8b4800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:05.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.753+0000 7f8b57400700 1 -- 172.21.15.69:0/1949611701 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48016070 con 0x7f8b60077940 2024-04-24T09:06:05.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.753+0000 7f8b57400700 1 -- 172.21.15.69:0/1949611701 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b48010460 con 0x7f8b60077940 2024-04-24T09:06:05.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.753+0000 7f8b65c00700 1 -- 172.21.15.69:0/1949611701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b601a0270 con 0x7f8b60077940 2024-04-24T09:06:05.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.753+0000 7f8b57400700 1 -- 172.21.15.69:0/1949611701 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4800f620 con 0x7f8b60077940 2024-04-24T09:06:05.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.753+0000 7f8b65c00700 1 -- 172.21.15.69:0/1949611701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b601ac550 con 0x7f8b60077940 2024-04-24T09:06:05.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.754+0000 7f8b65c00700 1 -- 172.21.15.69:0/1949611701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b60078760 con 0x7f8b60077940 2024-04-24T09:06:05.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.756+0000 7f8b57400700 1 -- 172.21.15.69:0/1949611701 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8b48019070 con 0x7f8b60077940 2024-04-24T09:06:05.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.758+0000 7f8b57400700 1 --2- 172.21.15.69:0/1949611701 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8b440a3cc0 0x7f8b440a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:05.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.758+0000 7f8b57400700 1 -- 172.21.15.69:0/1949611701 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8b480d0310 con 0x7f8b60077940 2024-04-24T09:06:05.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.758+0000 7f8b5ea00700 1 --2- 172.21.15.69:0/1949611701 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8b440a3cc0 0x7f8b440a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:05.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.759+0000 7f8b5ea00700 1 --2- 172.21.15.69:0/1949611701 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8b440a3cc0 0x7f8b440a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000b5c0 tx=0x7f8b50005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:05.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.761+0000 7f8b57400700 1 -- 172.21.15.69:0/1949611701 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b4809c990 con 0x7f8b60077940 2024-04-24T09:06:05.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:05.986+0000 7f8b65c00700 1 -- 172.21.15.69:0/1949611701 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b601a0bd0 con 0x7f8b440a3cc0 2024-04-24T09:06:05.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:06.000+0000 7f8b57400700 1 -- 172.21.15.69:0/1949611701 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340455 (secure 0 0 0) 0x7f8b601a0bd0 con 0x7f8b440a3cc0 2024-04-24T09:06:06.003 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:06.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:06.010+0000 7f8b65c00700 1 -- 172.21.15.69:0/1949611701 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8b440a3cc0 msgr2=0x7f8b440a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:06.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:06.010+0000 7f8b65c00700 1 --2- 172.21.15.69:0/1949611701 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8b440a3cc0 0x7f8b440a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000b5c0 tx=0x7f8b50005fd0 comp rx=0 tx=0).stop 2024-04-24T09:06:06.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:06.010+0000 7f8b65c00700 1 -- 172.21.15.69:0/1949611701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b60077940 msgr2=0x7f8b601a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:06.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:06.010+0000 7f8b65c00700 1 --2- 172.21.15.69:0/1949611701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b60077940 0x7f8b601a5b10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800d950 tx=0x7f8b4800dd10 comp rx=0 tx=0).stop 2024-04-24T09:06:06.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:06.010+0000 7f8b65c00700 1 -- 172.21.15.69:0/1949611701 shutdown_connections 2024-04-24T09:06:06.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:06.010+0000 7f8b65c00700 1 --2- 172.21.15.69:0/1949611701 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f8b440a3cc0 0x7f8b440a6180 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:06.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:06.010+0000 7f8b65c00700 1 --2- 172.21.15.69:0/1949611701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b601139a0 0x7f8b6019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:06.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:06.010+0000 7f8b65c00700 1 --2- 172.21.15.69:0/1949611701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b60077020 0x7f8b601a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:06.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:06.010+0000 7f8b65c00700 1 --2- 172.21.15.69:0/1949611701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b60077940 0x7f8b601a5b10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:06.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:06.010+0000 7f8b65c00700 1 -- 172.21.15.69:0/1949611701 >> 172.21.15.69:0/1949611701 conn(0x7f8b601007f0 msgr2=0x7f8b60102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:06.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:06.010+0000 7f8b65c00700 1 -- 172.21.15.69:0/1949611701 shutdown_connections 2024-04-24T09:06:06.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:06.010+0000 7f8b65c00700 1 -- 172.21.15.69:0/1949611701 wait complete. 2024-04-24T09:06:06.011 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-24T09:06:06.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:06 smithi081 ceph-mon[30961]: pgmap v433: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T09:06:06.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:06 smithi069 ceph-mon[28170]: pgmap v433: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T09:06:06.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:06 smithi130 ceph-mon[31553]: pgmap v433: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T09:06:07.017 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-04-24T09:06:05.426846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661031,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":695,"num_read_kb":475,"num_write":358,"num_write_kb":3066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":791,"ondisk_log_size":791,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355084,"kb_used_data":14252,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124586740,"statfs":{"total":1151940427776,"available":1151576821760,"internally_reserved":0,"allocated":14594048,"data_stored":8048855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":42,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004658"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.378272+0000","last_change":"2024-04-24T09:04:02.107965+0000","last_active":"2024-04-24T09:05:53.378272+0000","last_peered":"2024-04-24T09:05:53.378272+0000","last_clean":"2024-04-24T09:05:53.378272+0000","last_became_active":"2024-04-24T09:04:02.106827+0000","last_became_peered":"2024-04-24T09:04:02.106827+0000","last_unstale":"2024-04-24T09:05:53.378272+0000","last_undegraded":"2024-04-24T09:05:53.378272+0000","last_fullsized":"2024-04-24T09:05:53.378272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:20:23.930599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.361508+0000","last_change":"2024-04-24T09:04:02.105449+0000","last_active":"2024-04-24T09:05:53.361508+0000","last_peered":"2024-04-24T09:05:53.361508+0000","last_clean":"2024-04-24T09:05:53.361508+0000","last_became_active":"2024-04-24T09:04:02.101842+0000","last_became_peered":"2024-04-24T09:04:02.101842+0000","last_unstale":"2024-04-24T09:05:53.361508+0000","last_undegraded":"2024-04-24T09:05:53.361508+0000","last_fullsized":"2024-04-24T09:05:53.361508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:46.911768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.377788+0000","last_change":"2024-04-24T09:04:02.118686+0000","last_active":"2024-04-24T09:05:53.377788+0000","last_peered":"2024-04-24T09:05:53.377788+0000","last_clean":"2024-04-24T09:05:53.377788+0000","last_became_active":"2024-04-24T09:04:02.117549+0000","last_became_peered":"2024-04-24T09:04:02.117549+0000","last_unstale":"2024-04-24T09:05:53.377788+0000","last_undegraded":"2024-04-24T09:05:53.377788+0000","last_fullsized":"2024-04-24T09:05:53.377788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:18:16.233434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.350437+0000","last_change":"2024-04-24T09:04:02.120336+0000","last_active":"2024-04-24T09:05:53.350437+0000","last_peered":"2024-04-24T09:05:53.350437+0000","last_clean":"2024-04-24T09:05:53.350437+0000","last_became_active":"2024-04-24T09:04:02.116574+0000","last_became_peered":"2024-04-24T09:04:02.116574+0000","last_unstale":"2024-04-24T09:05:53.350437+0000","last_undegraded":"2024-04-24T09:05:53.350437+0000","last_fullsized":"2024-04-24T09:05:53.350437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:10:42.366531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.359791+0000","last_change":"2024-04-24T09:04:02.113600+0000","last_active":"2024-04-24T09:05:53.359791+0000","last_peered":"2024-04-24T09:05:53.359791+0000","last_clean":"2024-04-24T09:05:53.359791+0000","last_became_active":"2024-04-24T09:04:02.113143+0000","last_became_peered":"2024-04-24T09:04:02.113143+0000","last_unstale":"2024-04-24T09:05:53.359791+0000","last_undegraded":"2024-04-24T09:05:53.359791+0000","last_fullsized":"2024-04-24T09:05:53.359791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:31:27.666661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.366082+0000","last_change":"2024-04-24T09:04:02.106395+0000","last_active":"2024-04-24T09:05:53.366082+0000","last_peered":"2024-04-24T09:05:53.366082+0000","last_clean":"2024-04-24T09:05:53.366082+0000","last_became_active":"2024-04-24T09:04:02.105942+0000","last_became_peered":"2024-04-24T09:04:02.105942+0000","last_unstale":"2024-04-24T09:05:53.366082+0000","last_undegraded":"2024-04-24T09:05:53.366082+0000","last_fullsized":"2024-04-24T09:05:53.366082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:20:10.655605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.361146+0000","last_change":"2024-04-24T09:04:02.100279+0000","last_active":"2024-04-24T09:05:53.361146+0000","last_peered":"2024-04-24T09:05:53.361146+0000","last_clean":"2024-04-24T09:05:53.361146+0000","last_became_active":"2024-04-24T09:04:02.099745+0000","last_became_peered":"2024-04-24T09:04:02.099745+0000","last_unstale":"2024-04-24T09:05:53.361146+0000","last_undegraded":"2024-04-24T09:05:53.361146+0000","last_fullsized":"2024-04-24T09:05:53.361146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:13.150412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.374378+0000","last_change":"2024-04-24T09:04:02.103294+0000","last_active":"2024-04-24T09:05:53.374378+0000","last_peered":"2024-04-24T09:05:53.374378+0000","last_clean":"2024-04-24T09:05:53.374378+0000","last_became_active":"2024-04-24T09:04:02.102114+0000","last_became_peered":"2024-04-24T09:04:02.102114+0000","last_unstale":"2024-04-24T09:05:53.374378+0000","last_undegraded":"2024-04-24T09:05:53.374378+0000","last_fullsized":"2024-04-24T09:05:53.374378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:26:26.916097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.377067+0000","last_change":"2024-04-24T09:04:02.104936+0000","last_active":"2024-04-24T09:05:53.377067+0000","last_peered":"2024-04-24T09:05:53.377067+0000","last_clean":"2024-04-24T09:05:53.377067+0000","last_became_active":"2024-04-24T09:04:02.104540+0000","last_became_peered":"2024-04-24T09:04:02.104540+0000","last_unstale":"2024-04-24T09:05:53.377067+0000","last_undegraded":"2024-04-24T09:05:53.377067+0000","last_fullsized":"2024-04-24T09:05:53.377067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:36.010735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.367628+0000","last_change":"2024-04-24T09:04:02.102050+0000","last_active":"2024-04-24T09:05:53.367628+0000","last_peered":"2024-04-24T09:05:53.367628+0000","last_clean":"2024-04-24T09:05:53.367628+0000","last_became_active":"2024-04-24T09:04:02.101623+0000","last_became_peered":"2024-04-24T09:04:02.101623+0000","last_unstale":"2024-04-24T09:05:53.367628+0000","last_undegraded":"2024-04-24T09:05:53.367628+0000","last_fullsized":"2024-04-24T09:05:53.367628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:13:33.606200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.378933+0000","last_change":"2024-04-24T09:04:02.115804+0000","last_active":"2024-04-24T09:05:53.378933+0000","last_peered":"2024-04-24T09:05:53.378933+0000","last_clean":"2024-04-24T09:05:53.378933+0000","last_became_active":"2024-04-24T09:04:02.115093+0000","last_became_peered":"2024-04-24T09:04:02.115093+0000","last_unstale":"2024-04-24T09:05:53.378933+0000","last_undegraded":"2024-04-24T09:05:53.378933+0000","last_fullsized":"2024-04-24T09:05:53.378933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:26:56.764485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.358804+0000","last_change":"2024-04-24T09:04:02.116305+0000","last_active":"2024-04-24T09:05:53.358804+0000","last_peered":"2024-04-24T09:05:53.358804+0000","last_clean":"2024-04-24T09:05:53.358804+0000","last_became_active":"2024-04-24T09:04:02.115956+0000","last_became_peered":"2024-04-24T09:04:02.115956+0000","last_unstale":"2024-04-24T09:05:53.358804+0000","last_undegraded":"2024-04-24T09:05:53.358804+0000","last_fullsized":"2024-04-24T09:05:53.358804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:38.618317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.362695+0000","last_change":"2024-04-24T09:04:02.101155+0000","last_active":"2024-04-24T09:05:53.362695+0000","last_peered":"2024-04-24T09:05:53.362695+0000","last_clean":"2024-04-24T09:05:53.362695+0000","last_became_active":"2024-04-24T09:04:02.099719+0000","last_became_peered":"2024-04-24T09:04:02.099719+0000","last_unstale":"2024-04-24T09:05:53.362695+0000","last_undegraded":"2024-04-24T09:05:53.362695+0000","last_fullsized":"2024-04-24T09:05:53.362695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:46:27.910987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.362194+0000","last_change":"2024-04-24T09:04:02.114818+0000","last_active":"2024-04-24T09:05:53.362194+0000","last_peered":"2024-04-24T09:05:53.362194+0000","last_clean":"2024-04-24T09:05:53.362194+0000","last_became_active":"2024-04-24T09:04:02.112947+0000","last_became_peered":"2024-04-24T09:04:02.112947+0000","last_unstale":"2024-04-24T09:05:53.362194+0000","last_undegraded":"2024-04-24T09:05:53.362194+0000","last_fullsized":"2024-04-24T09:05:53.362194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:16.544420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.355959+0000","last_change":"2024-04-24T09:04:02.120672+0000","last_active":"2024-04-24T09:05:53.355959+0000","last_peered":"2024-04-24T09:05:53.355959+0000","last_clean":"2024-04-24T09:05:53.355959+0000","last_became_active":"2024-04-24T09:04:02.120378+0000","last_became_peered":"2024-04-24T09:04:02.120378+0000","last_unstale":"2024-04-24T09:05:53.355959+0000","last_undegraded":"2024-04-24T09:05:53.355959+0000","last_fullsized":"2024-04-24T09:05:53.355959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:17.781189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.376999+0000","last_change":"2024-04-24T09:04:02.107022+0000","last_active":"2024-04-24T09:05:53.376999+0000","last_peered":"2024-04-24T09:05:53.376999+0000","last_clean":"2024-04-24T09:05:53.376999+0000","last_became_active":"2024-04-24T09:04:02.106645+0000","last_became_peered":"2024-04-24T09:04:02.106645+0000","last_unstale":"2024-04-24T09:05:53.376999+0000","last_undegraded":"2024-04-24T09:05:53.376999+0000","last_fullsized":"2024-04-24T09:05:53.376999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:47.217718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.379883+0000","last_change":"2024-04-24T09:04:02.115511+0000","last_active":"2024-04-24T09:05:53.379883+0000","last_peered":"2024-04-24T09:05:53.379883+0000","last_clean":"2024-04-24T09:05:53.379883+0000","last_became_active":"2024-04-24T09:04:02.115197+0000","last_became_peered":"2024-04-24T09:04:02.115197+0000","last_unstale":"2024-04-24T09:05:53.379883+0000","last_undegraded":"2024-04-24T09:05:53.379883+0000","last_fullsized":"2024-04-24T09:05:53.379883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:18.883688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.378708+0000","last_change":"2024-04-24T09:04:02.102108+0000","last_active":"2024-04-24T09:05:53.378708+0000","last_peered":"2024-04-24T09:05:53.378708+0000","last_clean":"2024-04-24T09:05:53.378708+0000","last_became_active":"2024-04-24T09:04:02.101739+0000","last_became_peered":"2024-04-24T09:04:02.101739+0000","last_unstale":"2024-04-24T09:05:53.378708+0000","last_undegraded":"2024-04-24T09:05:53.378708+0000","last_fullsized":"2024-04-24T09:05:53.378708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:57:30.768716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.367483+0000","last_change":"2024-04-24T09:04:02.108911+0000","last_active":"2024-04-24T09:05:53.367483+0000","last_peered":"2024-04-24T09:05:53.367483+0000","last_clean":"2024-04-24T09:05:53.367483+0000","last_became_active":"2024-04-24T09:04:02.105678+0000","last_became_peered":"2024-04-24T09:04:02.105678+0000","last_unstale":"2024-04-24T09:05:53.367483+0000","last_undegraded":"2024-04-24T09:05:53.367483+0000","last_fullsized":"2024-04-24T09:05:53.367483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:36:12.216271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696112+0000","last_change":"2024-04-24T09:04:06.136320+0000","last_active":"2024-04-24T09:04:24.696112+0000","last_peered":"2024-04-24T09:04:24.696112+0000","last_clean":"2024-04-24T09:04:24.696112+0000","last_became_active":"2024-04-24T09:04:06.133851+0000","last_became_peered":"2024-04-24T09:04:06.133851+0000","last_unstale":"2024-04-24T09:04:24.696112+0000","last_undegraded":"2024-04-24T09:04:24.696112+0000","last_fullsized":"2024-04-24T09:04:24.696112+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:31:39.781234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.630633+0000","last_change":"2024-04-24T09:04:01.543818+0000","last_active":"2024-04-24T09:04:33.630633+0000","last_peered":"2024-04-24T09:04:33.630633+0000","last_clean":"2024-04-24T09:04:33.630633+0000","last_became_active":"2024-04-24T09:04:01.543435+0000","last_became_peered":"2024-04-24T09:04:01.543435+0000","last_unstale":"2024-04-24T09:04:33.630633+0000","last_undegraded":"2024-04-24T09:04:33.630633+0000","last_fullsized":"2024-04-24T09:04:33.630633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:08:30.176211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.370710+0000","last_change":"2024-04-24T09:04:02.101646+0000","last_active":"2024-04-24T09:05:53.370710+0000","last_peered":"2024-04-24T09:05:53.370710+0000","last_clean":"2024-04-24T09:05:53.370710+0000","last_became_active":"2024-04-24T09:04:02.101284+0000","last_became_peered":"2024-04-24T09:04:02.101284+0000","last_unstale":"2024-04-24T09:05:53.370710+0000","last_undegraded":"2024-04-24T09:05:53.370710+0000","last_fullsized":"2024-04-24T09:05:53.370710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:36:32.827216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.519708+0000","last_change":"2024-04-24T09:04:06.130476+0000","last_active":"2024-04-24T09:04:24.519708+0000","last_peered":"2024-04-24T09:04:24.519708+0000","last_clean":"2024-04-24T09:04:24.519708+0000","last_became_active":"2024-04-24T09:04:06.129707+0000","last_became_peered":"2024-04-24T09:04:06.129707+0000","last_unstale":"2024-04-24T09:04:24.519708+0000","last_undegraded":"2024-04-24T09:04:24.519708+0000","last_fullsized":"2024-04-24T09:04:24.519708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:25:36.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.631003+0000","last_change":"2024-04-24T09:04:01.551708+0000","last_active":"2024-04-24T09:04:33.631003+0000","last_peered":"2024-04-24T09:04:33.631003+0000","last_clean":"2024-04-24T09:04:33.631003+0000","last_became_active":"2024-04-24T09:04:01.551089+0000","last_became_peered":"2024-04-24T09:04:01.551089+0000","last_unstale":"2024-04-24T09:04:33.631003+0000","last_undegraded":"2024-04-24T09:04:33.631003+0000","last_fullsized":"2024-04-24T09:04:33.631003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:18:22.515729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.378011+0000","last_change":"2024-04-24T09:04:02.117356+0000","last_active":"2024-04-24T09:05:53.378011+0000","last_peered":"2024-04-24T09:05:53.378011+0000","last_clean":"2024-04-24T09:05:53.378011+0000","last_became_active":"2024-04-24T09:04:02.116972+0000","last_became_peered":"2024-04-24T09:04:02.116972+0000","last_unstale":"2024-04-24T09:05:53.378011+0000","last_undegraded":"2024-04-24T09:05:53.378011+0000","last_fullsized":"2024-04-24T09:05:53.378011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:46:29.349952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.520686+0000","last_change":"2024-04-24T09:04:06.141538+0000","last_active":"2024-04-24T09:04:24.520686+0000","last_peered":"2024-04-24T09:04:24.520686+0000","last_clean":"2024-04-24T09:04:24.520686+0000","last_became_active":"2024-04-24T09:04:06.140975+0000","last_became_peered":"2024-04-24T09:04:06.140975+0000","last_unstale":"2024-04-24T09:04:24.520686+0000","last_undegraded":"2024-04-24T09:04:24.520686+0000","last_fullsized":"2024-04-24T09:04:24.520686+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:58:31.688828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.630038+0000","last_change":"2024-04-24T09:04:01.561039+0000","last_active":"2024-04-24T09:04:33.630038+0000","last_peered":"2024-04-24T09:04:33.630038+0000","last_clean":"2024-04-24T09:04:33.630038+0000","last_became_active":"2024-04-24T09:04:01.560594+0000","last_became_peered":"2024-04-24T09:04:01.560594+0000","last_unstale":"2024-04-24T09:04:33.630038+0000","last_undegraded":"2024-04-24T09:04:33.630038+0000","last_fullsized":"2024-04-24T09:04:33.630038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:59:19.707826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.378030+0000","last_change":"2024-04-24T09:04:02.106007+0000","last_active":"2024-04-24T09:05:53.378030+0000","last_peered":"2024-04-24T09:05:53.378030+0000","last_clean":"2024-04-24T09:05:53.378030+0000","last_became_active":"2024-04-24T09:04:02.099786+0000","last_became_peered":"2024-04-24T09:04:02.099786+0000","last_unstale":"2024-04-24T09:05:53.378030+0000","last_undegraded":"2024-04-24T09:05:53.378030+0000","last_fullsized":"2024-04-24T09:05:53.378030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:56.288417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.520162+0000","last_change":"2024-04-24T09:04:06.143636+0000","last_active":"2024-04-24T09:04:24.520162+0000","last_peered":"2024-04-24T09:04:24.520162+0000","last_clean":"2024-04-24T09:04:24.520162+0000","last_became_active":"2024-04-24T09:04:06.143105+0000","last_became_peered":"2024-04-24T09:04:06.143105+0000","last_unstale":"2024-04-24T09:04:24.520162+0000","last_undegraded":"2024-04-24T09:04:24.520162+0000","last_fullsized":"2024-04-24T09:04:24.520162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:04:23.211067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.517222+0000","last_change":"2024-04-24T09:04:01.548403+0000","last_active":"2024-04-24T09:04:24.517222+0000","last_peered":"2024-04-24T09:04:24.517222+0000","last_clean":"2024-04-24T09:04:24.517222+0000","last_became_active":"2024-04-24T09:04:01.547447+0000","last_became_peered":"2024-04-24T09:04:01.547447+0000","last_unstale":"2024-04-24T09:04:24.517222+0000","last_undegraded":"2024-04-24T09:04:24.517222+0000","last_fullsized":"2024-04-24T09:04:24.517222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:14:08.160127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.378912+0000","last_change":"2024-04-24T09:04:02.121410+0000","last_active":"2024-04-24T09:05:53.378912+0000","last_peered":"2024-04-24T09:05:53.378912+0000","last_clean":"2024-04-24T09:05:53.378912+0000","last_became_active":"2024-04-24T09:04:02.120821+0000","last_became_peered":"2024-04-24T09:04:02.120821+0000","last_unstale":"2024-04-24T09:05:53.378912+0000","last_undegraded":"2024-04-24T09:05:53.378912+0000","last_fullsized":"2024-04-24T09:05:53.378912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:39:05.525171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696856+0000","last_change":"2024-04-24T09:04:06.142349+0000","last_active":"2024-04-24T09:04:24.696856+0000","last_peered":"2024-04-24T09:04:24.696856+0000","last_clean":"2024-04-24T09:04:24.696856+0000","last_became_active":"2024-04-24T09:04:06.141911+0000","last_became_peered":"2024-04-24T09:04:06.141911+0000","last_unstale":"2024-04-24T09:04:24.696856+0000","last_undegraded":"2024-04-24T09:04:24.696856+0000","last_fullsized":"2024-04-24T09:04:24.696856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:45:30.414686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'5","reported_seq":126,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:59.886677+0000","last_change":"2024-04-24T09:04:24.537349+0000","last_active":"2024-04-24T09:05:59.886677+0000","last_peered":"2024-04-24T09:05:59.886677+0000","last_clean":"2024-04-24T09:05:59.886677+0000","last_became_active":"2024-04-24T09:04:24.536738+0000","last_became_peered":"2024-04-24T09:04:24.536738+0000","last_unstale":"2024-04-24T09:05:59.886677+0000","last_undegraded":"2024-04-24T09:05:59.886677+0000","last_fullsized":"2024-04-24T09:05:59.886677+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:33.481380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.351533+0000","last_change":"2024-04-24T09:04:02.111173+0000","last_active":"2024-04-24T09:05:53.351533+0000","last_peered":"2024-04-24T09:05:53.351533+0000","last_clean":"2024-04-24T09:05:53.351533+0000","last_became_active":"2024-04-24T09:04:02.110681+0000","last_became_peered":"2024-04-24T09:04:02.110681+0000","last_unstale":"2024-04-24T09:05:53.351533+0000","last_undegraded":"2024-04-24T09:05:53.351533+0000","last_fullsized":"2024-04-24T09:05:53.351533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:29.939116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696716+0000","last_change":"2024-04-24T09:04:06.137752+0000","last_active":"2024-04-24T09:04:24.696716+0000","last_peered":"2024-04-24T09:04:24.696716+0000","last_clean":"2024-04-24T09:04:24.696716+0000","last_became_active":"2024-04-24T09:04:06.137492+0000","last_became_peered":"2024-04-24T09:04:06.137492+0000","last_unstale":"2024-04-24T09:04:24.696716+0000","last_undegraded":"2024-04-24T09:04:24.696716+0000","last_fullsized":"2024-04-24T09:04:24.696716+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:44:00.653864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.628109+0000","last_change":"2024-04-24T09:04:01.544009+0000","last_active":"2024-04-24T09:04:33.628109+0000","last_peered":"2024-04-24T09:04:33.628109+0000","last_clean":"2024-04-24T09:04:33.628109+0000","last_became_active":"2024-04-24T09:04:01.543651+0000","last_became_peered":"2024-04-24T09:04:01.543651+0000","last_unstale":"2024-04-24T09:04:33.628109+0000","last_undegraded":"2024-04-24T09:04:33.628109+0000","last_fullsized":"2024-04-24T09:04:33.628109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:10:21.969072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.373547+0000","last_change":"2024-04-24T09:04:02.115387+0000","last_active":"2024-04-24T09:05:53.373547+0000","last_peered":"2024-04-24T09:05:53.373547+0000","last_clean":"2024-04-24T09:05:53.373547+0000","last_became_active":"2024-04-24T09:04:02.115086+0000","last_became_peered":"2024-04-24T09:04:02.115086+0000","last_unstale":"2024-04-24T09:05:53.373547+0000","last_undegraded":"2024-04-24T09:05:53.373547+0000","last_fullsized":"2024-04-24T09:05:53.373547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:40:57.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532516+0000","last_change":"2024-04-24T09:04:06.146189+0000","last_active":"2024-04-24T09:04:24.532516+0000","last_peered":"2024-04-24T09:04:24.532516+0000","last_clean":"2024-04-24T09:04:24.532516+0000","last_became_active":"2024-04-24T09:04:06.145556+0000","last_became_peered":"2024-04-24T09:04:06.145556+0000","last_unstale":"2024-04-24T09:04:24.532516+0000","last_undegraded":"2024-04-24T09:04:24.532516+0000","last_fullsized":"2024-04-24T09:04:24.532516+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:41.371948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.628679+0000","last_change":"2024-04-24T09:04:01.542185+0000","last_active":"2024-04-24T09:04:33.628679+0000","last_peered":"2024-04-24T09:04:33.628679+0000","last_clean":"2024-04-24T09:04:33.628679+0000","last_became_active":"2024-04-24T09:04:01.541240+0000","last_became_peered":"2024-04-24T09:04:01.541240+0000","last_unstale":"2024-04-24T09:04:33.628679+0000","last_undegraded":"2024-04-24T09:04:33.628679+0000","last_fullsized":"2024-04-24T09:04:33.628679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:15.103276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.372576+0000","last_change":"2024-04-24T09:04:02.104054+0000","last_active":"2024-04-24T09:05:53.372576+0000","last_peered":"2024-04-24T09:05:53.372576+0000","last_clean":"2024-04-24T09:05:53.372576+0000","last_became_active":"2024-04-24T09:04:02.103673+0000","last_became_peered":"2024-04-24T09:04:02.103673+0000","last_unstale":"2024-04-24T09:05:53.372576+0000","last_undegraded":"2024-04-24T09:05:53.372576+0000","last_fullsized":"2024-04-24T09:05:53.372576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:31:58.889989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696675+0000","last_change":"2024-04-24T09:04:06.134034+0000","last_active":"2024-04-24T09:04:24.696675+0000","last_peered":"2024-04-24T09:04:24.696675+0000","last_clean":"2024-04-24T09:04:24.696675+0000","last_became_active":"2024-04-24T09:04:06.133596+0000","last_became_peered":"2024-04-24T09:04:06.133596+0000","last_unstale":"2024-04-24T09:04:24.696675+0000","last_undegraded":"2024-04-24T09:04:24.696675+0000","last_fullsized":"2024-04-24T09:04:24.696675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:17:09.572974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.528733+0000","last_change":"2024-04-24T09:04:01.543312+0000","last_active":"2024-04-24T09:04:24.528733+0000","last_peered":"2024-04-24T09:04:24.528733+0000","last_clean":"2024-04-24T09:04:24.528733+0000","last_became_active":"2024-04-24T09:04:01.542914+0000","last_became_peered":"2024-04-24T09:04:01.542914+0000","last_unstale":"2024-04-24T09:04:24.528733+0000","last_undegraded":"2024-04-24T09:04:24.528733+0000","last_fullsized":"2024-04-24T09:04:24.528733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:50.695108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.367677+0000","last_change":"2024-04-24T09:04:02.114445+0000","last_active":"2024-04-24T09:05:53.367677+0000","last_peered":"2024-04-24T09:05:53.367677+0000","last_clean":"2024-04-24T09:05:53.367677+0000","last_became_active":"2024-04-24T09:04:02.114075+0000","last_became_peered":"2024-04-24T09:04:02.114075+0000","last_unstale":"2024-04-24T09:05:53.367677+0000","last_undegraded":"2024-04-24T09:05:53.367677+0000","last_fullsized":"2024-04-24T09:05:53.367677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:35:06.933677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.525318+0000","last_change":"2024-04-24T09:04:06.142368+0000","last_active":"2024-04-24T09:04:24.525318+0000","last_peered":"2024-04-24T09:04:24.525318+0000","last_clean":"2024-04-24T09:04:24.525318+0000","last_became_active":"2024-04-24T09:04:06.141872+0000","last_became_peered":"2024-04-24T09:04:06.141872+0000","last_unstale":"2024-04-24T09:04:24.525318+0000","last_undegraded":"2024-04-24T09:04:24.525318+0000","last_fullsized":"2024-04-24T09:04:24.525318+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:13:43.733662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":119,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:06:01.891062+0000","last_change":"2024-04-24T09:04:01.556002+0000","last_active":"2024-04-24T09:06:01.891062+0000","last_peered":"2024-04-24T09:06:01.891062+0000","last_clean":"2024-04-24T09:06:01.891062+0000","last_became_active":"2024-04-24T09:04:01.555643+0000","last_became_peered":"2024-04-24T09:04:01.555643+0000","last_unstale":"2024-04-24T09:06:01.891062+0000","last_undegraded":"2024-04-24T09:06:01.891062+0000","last_fullsized":"2024-04-24T09:06:01.891062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:30:31.171184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":89,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.377508+0000","last_change":"2024-04-24T09:04:02.104252+0000","last_active":"2024-04-24T09:05:53.377508+0000","last_peered":"2024-04-24T09:05:53.377508+0000","last_clean":"2024-04-24T09:05:53.377508+0000","last_became_active":"2024-04-24T09:04:02.099264+0000","last_became_peered":"2024-04-24T09:04:02.099264+0000","last_unstale":"2024-04-24T09:05:53.377508+0000","last_undegraded":"2024-04-24T09:05:53.377508+0000","last_fullsized":"2024-04-24T09:05:53.377508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:28.469814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.516701+0000","last_change":"2024-04-24T09:04:06.132407+0000","last_active":"2024-04-24T09:04:24.516701+0000","last_peered":"2024-04-24T09:04:24.516701+0000","last_clean":"2024-04-24T09:04:24.516701+0000","last_became_active":"2024-04-24T09:04:06.131997+0000","last_became_peered":"2024-04-24T09:04:06.131997+0000","last_unstale":"2024-04-24T09:04:24.516701+0000","last_undegraded":"2024-04-24T09:04:24.516701+0000","last_fullsized":"2024-04-24T09:04:24.516701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:08:07.369326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.613853+0000","last_change":"2024-04-24T09:04:01.561039+0000","last_active":"2024-04-24T09:04:32.613853+0000","last_peered":"2024-04-24T09:04:32.613853+0000","last_clean":"2024-04-24T09:04:32.613853+0000","last_became_active":"2024-04-24T09:04:01.560543+0000","last_became_peered":"2024-04-24T09:04:01.560543+0000","last_unstale":"2024-04-24T09:04:32.613853+0000","last_undegraded":"2024-04-24T09:04:32.613853+0000","last_fullsized":"2024-04-24T09:04:32.613853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:27.706982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.370657+0000","last_change":"2024-04-24T09:04:02.110850+0000","last_active":"2024-04-24T09:05:53.370657+0000","last_peered":"2024-04-24T09:05:53.370657+0000","last_clean":"2024-04-24T09:05:53.370657+0000","last_became_active":"2024-04-24T09:04:02.110512+0000","last_became_peered":"2024-04-24T09:04:02.110512+0000","last_unstale":"2024-04-24T09:05:53.370657+0000","last_undegraded":"2024-04-24T09:05:53.370657+0000","last_fullsized":"2024-04-24T09:05:53.370657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:19:16.937202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.521039+0000","last_change":"2024-04-24T09:04:06.135200+0000","last_active":"2024-04-24T09:04:24.521039+0000","last_peered":"2024-04-24T09:04:24.521039+0000","last_clean":"2024-04-24T09:04:24.521039+0000","last_became_active":"2024-04-24T09:04:06.134680+0000","last_became_peered":"2024-04-24T09:04:06.134680+0000","last_unstale":"2024-04-24T09:04:24.521039+0000","last_undegraded":"2024-04-24T09:04:24.521039+0000","last_fullsized":"2024-04-24T09:04:24.521039+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:48:57.697740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.612218+0000","last_change":"2024-04-24T09:04:01.555069+0000","last_active":"2024-04-24T09:04:32.612218+0000","last_peered":"2024-04-24T09:04:32.612218+0000","last_clean":"2024-04-24T09:04:32.612218+0000","last_became_active":"2024-04-24T09:04:01.554737+0000","last_became_peered":"2024-04-24T09:04:01.554737+0000","last_unstale":"2024-04-24T09:04:32.612218+0000","last_undegraded":"2024-04-24T09:04:32.612218+0000","last_fullsized":"2024-04-24T09:04:32.612218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:23:00.272697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.379647+0000","last_change":"2024-04-24T09:04:02.112126+0000","last_active":"2024-04-24T09:05:53.379647+0000","last_peered":"2024-04-24T09:05:53.379647+0000","last_clean":"2024-04-24T09:05:53.379647+0000","last_became_active":"2024-04-24T09:04:02.107995+0000","last_became_peered":"2024-04-24T09:04:02.107995+0000","last_unstale":"2024-04-24T09:05:53.379647+0000","last_undegraded":"2024-04-24T09:05:53.379647+0000","last_fullsized":"2024-04-24T09:05:53.379647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:13.189829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.542302+0000","last_change":"2024-04-24T09:04:24.542302+0000","last_active":"2024-04-24T09:04:24.542302+0000","last_peered":"2024-04-24T09:04:24.542302+0000","last_clean":"2024-04-24T09:04:24.542302+0000","last_became_active":"2024-04-24T09:04:24.539848+0000","last_became_peered":"2024-04-24T09:04:24.539848+0000","last_unstale":"2024-04-24T09:04:24.542302+0000","last_undegraded":"2024-04-24T09:04:24.542302+0000","last_fullsized":"2024-04-24T09:04:24.542302+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:01:52.055510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"92'41","reported_seq":257,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:06:02.894566+0000","last_change":"2024-04-24T09:04:01.548610+0000","last_active":"2024-04-24T09:06:02.894566+0000","last_peered":"2024-04-24T09:06:02.894566+0000","last_clean":"2024-04-24T09:06:02.894566+0000","last_became_active":"2024-04-24T09:04:01.548145+0000","last_became_peered":"2024-04-24T09:04:01.548145+0000","last_unstale":"2024-04-24T09:06:02.894566+0000","last_undegraded":"2024-04-24T09:06:02.894566+0000","last_fullsized":"2024-04-24T09:06:02.894566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:58:48.425187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66287,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":189,"num_read_kb":160,"num_write":42,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.361962+0000","last_change":"2024-04-24T09:04:02.109965+0000","last_active":"2024-04-24T09:05:53.361962+0000","last_peered":"2024-04-24T09:05:53.361962+0000","last_clean":"2024-04-24T09:05:53.361962+0000","last_became_active":"2024-04-24T09:04:02.108143+0000","last_became_peered":"2024-04-24T09:04:02.108143+0000","last_unstale":"2024-04-24T09:05:53.361962+0000","last_undegraded":"2024-04-24T09:05:53.361962+0000","last_fullsized":"2024-04-24T09:05:53.361962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:01:21.083039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.534893+0000","last_change":"2024-04-24T09:04:06.145369+0000","last_active":"2024-04-24T09:04:24.534893+0000","last_peered":"2024-04-24T09:04:24.534893+0000","last_clean":"2024-04-24T09:04:24.534893+0000","last_became_active":"2024-04-24T09:04:06.144884+0000","last_became_peered":"2024-04-24T09:04:06.144884+0000","last_unstale":"2024-04-24T09:04:24.534893+0000","last_undegraded":"2024-04-24T09:04:24.534893+0000","last_fullsized":"2024-04-24T09:04:24.534893+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:29:22.780846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.629531+0000","last_change":"2024-04-24T09:04:01.548195+0000","last_active":"2024-04-24T09:04:33.629531+0000","last_peered":"2024-04-24T09:04:33.629531+0000","last_clean":"2024-04-24T09:04:33.629531+0000","last_became_active":"2024-04-24T09:04:01.544969+0000","last_became_peered":"2024-04-24T09:04:01.544969+0000","last_unstale":"2024-04-24T09:04:33.629531+0000","last_undegraded":"2024-04-24T09:04:33.629531+0000","last_fullsized":"2024-04-24T09:04:33.629531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:32:55.294629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.367195+0000","last_change":"2024-04-24T09:04:02.103785+0000","last_active":"2024-04-24T09:05:53.367195+0000","last_peered":"2024-04-24T09:05:53.367195+0000","last_clean":"2024-04-24T09:05:53.367195+0000","last_became_active":"2024-04-24T09:04:02.103463+0000","last_became_peered":"2024-04-24T09:04:02.103463+0000","last_unstale":"2024-04-24T09:05:53.367195+0000","last_undegraded":"2024-04-24T09:05:53.367195+0000","last_fullsized":"2024-04-24T09:05:53.367195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:56.385382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527499+0000","last_change":"2024-04-24T09:04:06.149955+0000","last_active":"2024-04-24T09:04:24.527499+0000","last_peered":"2024-04-24T09:04:24.527499+0000","last_clean":"2024-04-24T09:04:24.527499+0000","last_became_active":"2024-04-24T09:04:06.149445+0000","last_became_peered":"2024-04-24T09:04:06.149445+0000","last_unstale":"2024-04-24T09:04:24.527499+0000","last_undegraded":"2024-04-24T09:04:24.527499+0000","last_fullsized":"2024-04-24T09:04:24.527499+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:30:42.527007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.615210+0000","last_change":"2024-04-24T09:04:01.548740+0000","last_active":"2024-04-24T09:04:32.615210+0000","last_peered":"2024-04-24T09:04:32.615210+0000","last_clean":"2024-04-24T09:04:32.615210+0000","last_became_active":"2024-04-24T09:04:01.546060+0000","last_became_peered":"2024-04-24T09:04:01.546060+0000","last_unstale":"2024-04-24T09:04:32.615210+0000","last_undegraded":"2024-04-24T09:04:32.615210+0000","last_fullsized":"2024-04-24T09:04:32.615210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:09:16.301150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.367883+0000","last_change":"2024-04-24T09:04:24.538871+0000","last_active":"2024-04-24T09:05:53.367883+0000","last_peered":"2024-04-24T09:05:53.367883+0000","last_clean":"2024-04-24T09:05:53.367883+0000","last_became_active":"2024-04-24T09:04:24.537039+0000","last_became_peered":"2024-04-24T09:04:24.537039+0000","last_unstale":"2024-04-24T09:05:53.367883+0000","last_undegraded":"2024-04-24T09:05:53.367883+0000","last_fullsized":"2024-04-24T09:05:53.367883+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:18.520407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.518409+0000","last_change":"2024-04-24T09:04:06.136973+0000","last_active":"2024-04-24T09:04:24.518409+0000","last_peered":"2024-04-24T09:04:24.518409+0000","last_clean":"2024-04-24T09:04:24.518409+0000","last_became_active":"2024-04-24T09:04:06.136405+0000","last_became_peered":"2024-04-24T09:04:06.136405+0000","last_unstale":"2024-04-24T09:04:24.518409+0000","last_undegraded":"2024-04-24T09:04:24.518409+0000","last_fullsized":"2024-04-24T09:04:24.518409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:07.653162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":123,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:59.894114+0000","last_change":"2024-04-24T09:04:01.546853+0000","last_active":"2024-04-24T09:05:59.894114+0000","last_peered":"2024-04-24T09:05:59.894114+0000","last_clean":"2024-04-24T09:05:59.894114+0000","last_became_active":"2024-04-24T09:04:01.546441+0000","last_became_peered":"2024-04-24T09:04:01.546441+0000","last_unstale":"2024-04-24T09:05:59.894114+0000","last_undegraded":"2024-04-24T09:05:59.894114+0000","last_fullsized":"2024-04-24T09:05:59.894114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:23:56.333410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.361044+0000","last_change":"2024-04-24T09:04:02.101158+0000","last_active":"2024-04-24T09:05:53.361044+0000","last_peered":"2024-04-24T09:05:53.361044+0000","last_clean":"2024-04-24T09:05:53.361044+0000","last_became_active":"2024-04-24T09:04:02.100832+0000","last_became_peered":"2024-04-24T09:04:02.100832+0000","last_unstale":"2024-04-24T09:05:53.361044+0000","last_undegraded":"2024-04-24T09:05:53.361044+0000","last_fullsized":"2024-04-24T09:05:53.361044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:05:34.080520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533094+0000","last_change":"2024-04-24T09:04:06.149176+0000","last_active":"2024-04-24T09:04:24.533094+0000","last_peered":"2024-04-24T09:04:24.533094+0000","last_clean":"2024-04-24T09:04:24.533094+0000","last_became_active":"2024-04-24T09:04:06.145539+0000","last_became_peered":"2024-04-24T09:04:06.145539+0000","last_unstale":"2024-04-24T09:04:24.533094+0000","last_undegraded":"2024-04-24T09:04:24.533094+0000","last_fullsized":"2024-04-24T09:04:24.533094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:38.719787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.632543+0000","last_change":"2024-04-24T09:04:01.555330+0000","last_active":"2024-04-24T09:04:33.632543+0000","last_peered":"2024-04-24T09:04:33.632543+0000","last_clean":"2024-04-24T09:04:33.632543+0000","last_became_active":"2024-04-24T09:04:01.552618+0000","last_became_peered":"2024-04-24T09:04:01.552618+0000","last_unstale":"2024-04-24T09:04:33.632543+0000","last_undegraded":"2024-04-24T09:04:33.632543+0000","last_fullsized":"2024-04-24T09:04:33.632543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:30.956657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.374740+0000","last_change":"2024-04-24T09:04:02.116301+0000","last_active":"2024-04-24T09:05:53.374740+0000","last_peered":"2024-04-24T09:05:53.374740+0000","last_clean":"2024-04-24T09:05:53.374740+0000","last_became_active":"2024-04-24T09:04:02.116001+0000","last_became_peered":"2024-04-24T09:04:02.116001+0000","last_unstale":"2024-04-24T09:05:53.374740+0000","last_undegraded":"2024-04-24T09:05:53.374740+0000","last_fullsized":"2024-04-24T09:05:53.374740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:16:18.995067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.529071+0000","last_change":"2024-04-24T09:04:06.135814+0000","last_active":"2024-04-24T09:04:24.529071+0000","last_peered":"2024-04-24T09:04:24.529071+0000","last_clean":"2024-04-24T09:04:24.529071+0000","last_became_active":"2024-04-24T09:04:06.135054+0000","last_became_peered":"2024-04-24T09:04:06.135054+0000","last_unstale":"2024-04-24T09:04:24.529071+0000","last_undegraded":"2024-04-24T09:04:24.529071+0000","last_fullsized":"2024-04-24T09:04:24.529071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:51:31.150589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:34.618324+0000","last_change":"2024-04-24T09:04:01.553144+0000","last_active":"2024-04-24T09:04:34.618324+0000","last_peered":"2024-04-24T09:04:34.618324+0000","last_clean":"2024-04-24T09:04:34.618324+0000","last_became_active":"2024-04-24T09:04:01.550599+0000","last_became_peered":"2024-04-24T09:04:01.550599+0000","last_unstale":"2024-04-24T09:04:34.618324+0000","last_undegraded":"2024-04-24T09:04:34.618324+0000","last_fullsized":"2024-04-24T09:04:34.618324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:41.950209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.363246+0000","last_change":"2024-04-24T09:04:02.117064+0000","last_active":"2024-04-24T09:05:53.363246+0000","last_peered":"2024-04-24T09:05:53.363246+0000","last_clean":"2024-04-24T09:05:53.363246+0000","last_became_active":"2024-04-24T09:04:02.114528+0000","last_became_peered":"2024-04-24T09:04:02.114528+0000","last_unstale":"2024-04-24T09:05:53.363246+0000","last_undegraded":"2024-04-24T09:05:53.363246+0000","last_fullsized":"2024-04-24T09:05:53.363246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:24:53.662571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.701058+0000","last_change":"2024-04-24T09:04:24.701058+0000","last_active":"2024-04-24T09:04:24.701058+0000","last_peered":"2024-04-24T09:04:24.701058+0000","last_clean":"2024-04-24T09:04:24.701058+0000","last_became_active":"2024-04-24T09:04:24.699766+0000","last_became_peered":"2024-04-24T09:04:24.699766+0000","last_unstale":"2024-04-24T09:04:24.701058+0000","last_undegraded":"2024-04-24T09:04:24.701058+0000","last_fullsized":"2024-04-24T09:04:24.701058+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:57:48.491073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.701816+0000","last_change":"2024-04-24T09:04:24.701816+0000","last_active":"2024-04-24T09:04:24.701816+0000","last_peered":"2024-04-24T09:04:24.701816+0000","last_clean":"2024-04-24T09:04:24.701816+0000","last_became_active":"2024-04-24T09:04:24.700530+0000","last_became_peered":"2024-04-24T09:04:24.700530+0000","last_unstale":"2024-04-24T09:04:24.701816+0000","last_undegraded":"2024-04-24T09:04:24.701816+0000","last_fullsized":"2024-04-24T09:04:24.701816+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:24.614043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.367605+0000","last_change":"2024-04-24T09:04:02.104602+0000","last_active":"2024-04-24T09:05:53.367605+0000","last_peered":"2024-04-24T09:05:53.367605+0000","last_clean":"2024-04-24T09:05:53.367605+0000","last_became_active":"2024-04-24T09:04:02.104188+0000","last_became_peered":"2024-04-24T09:04:02.104188+0000","last_unstale":"2024-04-24T09:05:53.367605+0000","last_undegraded":"2024-04-24T09:05:53.367605+0000","last_fullsized":"2024-04-24T09:05:53.367605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:00:50.286952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531886+0000","last_change":"2024-04-24T09:04:06.134971+0000","last_active":"2024-04-24T09:04:24.531886+0000","last_peered":"2024-04-24T09:04:24.531886+0000","last_clean":"2024-04-24T09:04:24.531886+0000","last_became_active":"2024-04-24T09:04:06.134551+0000","last_became_peered":"2024-04-24T09:04:06.134551+0000","last_unstale":"2024-04-24T09:04:24.531886+0000","last_undegraded":"2024-04-24T09:04:24.531886+0000","last_fullsized":"2024-04-24T09:04:24.531886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:40:41.766657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532337+0000","last_change":"2024-04-24T09:04:01.551117+0000","last_active":"2024-04-24T09:04:24.532337+0000","last_peered":"2024-04-24T09:04:24.532337+0000","last_clean":"2024-04-24T09:04:24.532337+0000","last_became_active":"2024-04-24T09:04:01.550646+0000","last_became_peered":"2024-04-24T09:04:01.550646+0000","last_unstale":"2024-04-24T09:04:24.532337+0000","last_undegraded":"2024-04-24T09:04:24.532337+0000","last_fullsized":"2024-04-24T09:04:24.532337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:29:03.821300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.366569+0000","last_change":"2024-04-24T09:04:02.107111+0000","last_active":"2024-04-24T09:05:53.366569+0000","last_peered":"2024-04-24T09:05:53.366569+0000","last_clean":"2024-04-24T09:05:53.366569+0000","last_became_active":"2024-04-24T09:04:02.106185+0000","last_became_peered":"2024-04-24T09:04:02.106185+0000","last_unstale":"2024-04-24T09:05:53.366569+0000","last_undegraded":"2024-04-24T09:05:53.366569+0000","last_fullsized":"2024-04-24T09:05:53.366569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:19:03.111407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532094+0000","last_change":"2024-04-24T09:04:06.143984+0000","last_active":"2024-04-24T09:04:24.532094+0000","last_peered":"2024-04-24T09:04:24.532094+0000","last_clean":"2024-04-24T09:04:24.532094+0000","last_became_active":"2024-04-24T09:04:06.143503+0000","last_became_peered":"2024-04-24T09:04:06.143503+0000","last_unstale":"2024-04-24T09:04:24.532094+0000","last_undegraded":"2024-04-24T09:04:24.532094+0000","last_fullsized":"2024-04-24T09:04:24.532094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:35:19.317771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.352379+0000","last_change":"2024-04-24T09:04:02.111560+0000","last_active":"2024-04-24T09:05:53.352379+0000","last_peered":"2024-04-24T09:05:53.352379+0000","last_clean":"2024-04-24T09:05:53.352379+0000","last_became_active":"2024-04-24T09:04:02.109885+0000","last_became_peered":"2024-04-24T09:04:02.109885+0000","last_unstale":"2024-04-24T09:05:53.352379+0000","last_undegraded":"2024-04-24T09:05:53.352379+0000","last_fullsized":"2024-04-24T09:05:53.352379+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:35.837621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":119,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:59.892818+0000","last_change":"2024-04-24T09:04:01.542530+0000","last_active":"2024-04-24T09:05:59.892818+0000","last_peered":"2024-04-24T09:05:59.892818+0000","last_clean":"2024-04-24T09:05:59.892818+0000","last_became_active":"2024-04-24T09:04:01.541702+0000","last_became_peered":"2024-04-24T09:04:01.541702+0000","last_unstale":"2024-04-24T09:05:59.892818+0000","last_undegraded":"2024-04-24T09:05:59.892818+0000","last_fullsized":"2024-04-24T09:05:59.892818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:28:29.420020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.532686+0000","last_change":"2024-04-24T09:04:06.143004+0000","last_active":"2024-04-24T09:04:24.532686+0000","last_peered":"2024-04-24T09:04:24.532686+0000","last_clean":"2024-04-24T09:04:24.532686+0000","last_became_active":"2024-04-24T09:04:06.142444+0000","last_became_peered":"2024-04-24T09:04:06.142444+0000","last_unstale":"2024-04-24T09:04:24.532686+0000","last_undegraded":"2024-04-24T09:04:24.532686+0000","last_fullsized":"2024-04-24T09:04:24.532686+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:13.910612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.375464+0000","last_change":"2024-04-24T09:04:24.701330+0000","last_active":"2024-04-24T09:05:53.375464+0000","last_peered":"2024-04-24T09:05:53.375464+0000","last_clean":"2024-04-24T09:05:53.375464+0000","last_became_active":"2024-04-24T09:04:24.699988+0000","last_became_peered":"2024-04-24T09:04:24.699988+0000","last_unstale":"2024-04-24T09:05:53.375464+0000","last_undegraded":"2024-04-24T09:05:53.375464+0000","last_fullsized":"2024-04-24T09:05:53.375464+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:23:24.610671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:34.629794+0000","last_change":"2024-04-24T09:04:01.541206+0000","last_active":"2024-04-24T09:04:34.629794+0000","last_peered":"2024-04-24T09:04:34.629794+0000","last_clean":"2024-04-24T09:04:34.629794+0000","last_became_active":"2024-04-24T09:04:01.539888+0000","last_became_peered":"2024-04-24T09:04:01.539888+0000","last_unstale":"2024-04-24T09:04:34.629794+0000","last_undegraded":"2024-04-24T09:04:34.629794+0000","last_fullsized":"2024-04-24T09:04:34.629794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:01.550209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.531165+0000","last_change":"2024-04-24T09:02:22.629840+0000","last_active":"2024-04-24T09:04:24.531165+0000","last_peered":"2024-04-24T09:04:24.531165+0000","last_clean":"2024-04-24T09:04:24.531165+0000","last_became_active":"2024-04-24T09:02:22.628700+0000","last_became_peered":"2024-04-24T09:02:22.628700+0000","last_unstale":"2024-04-24T09:04:24.531165+0000","last_undegraded":"2024-04-24T09:04:24.531165+0000","last_fullsized":"2024-04-24T09:04:24.531165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:57:38.349295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:57:38.349295+0000","last_clean_scrub_stamp":"2024-04-24T08:57:38.349295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:54:11.409492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533333+0000","last_change":"2024-04-24T09:04:06.149548+0000","last_active":"2024-04-24T09:04:24.533333+0000","last_peered":"2024-04-24T09:04:24.533333+0000","last_clean":"2024-04-24T09:04:24.533333+0000","last_became_active":"2024-04-24T09:04:06.149214+0000","last_became_peered":"2024-04-24T09:04:06.149214+0000","last_unstale":"2024-04-24T09:04:24.533333+0000","last_undegraded":"2024-04-24T09:04:24.533333+0000","last_fullsized":"2024-04-24T09:04:24.533333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:03.801242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.352320+0000","last_change":"2024-04-24T09:04:02.110242+0000","last_active":"2024-04-24T09:05:53.352320+0000","last_peered":"2024-04-24T09:05:53.352320+0000","last_clean":"2024-04-24T09:05:53.352320+0000","last_became_active":"2024-04-24T09:04:02.109879+0000","last_became_peered":"2024-04-24T09:04:02.109879+0000","last_unstale":"2024-04-24T09:05:53.352320+0000","last_undegraded":"2024-04-24T09:05:53.352320+0000","last_fullsized":"2024-04-24T09:05:53.352320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:16.353861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.628623+0000","last_change":"2024-04-24T09:04:01.550989+0000","last_active":"2024-04-24T09:04:33.628623+0000","last_peered":"2024-04-24T09:04:33.628623+0000","last_clean":"2024-04-24T09:04:33.628623+0000","last_became_active":"2024-04-24T09:04:01.550664+0000","last_became_peered":"2024-04-24T09:04:01.550664+0000","last_unstale":"2024-04-24T09:04:33.628623+0000","last_undegraded":"2024-04-24T09:04:33.628623+0000","last_fullsized":"2024-04-24T09:04:33.628623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:33:39.521951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.538207+0000","last_change":"2024-04-24T09:04:24.538207+0000","last_active":"2024-04-24T09:04:24.538207+0000","last_peered":"2024-04-24T09:04:24.538207+0000","last_clean":"2024-04-24T09:04:24.538207+0000","last_became_active":"2024-04-24T09:04:24.536894+0000","last_became_peered":"2024-04-24T09:04:24.536894+0000","last_unstale":"2024-04-24T09:04:24.538207+0000","last_undegraded":"2024-04-24T09:04:24.538207+0000","last_fullsized":"2024-04-24T09:04:24.538207+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:32:52.860371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.608599+0000","last_change":"2024-04-24T09:04:01.539415+0000","last_active":"2024-04-24T09:04:32.608599+0000","last_peered":"2024-04-24T09:04:32.608599+0000","last_clean":"2024-04-24T09:04:32.608599+0000","last_became_active":"2024-04-24T09:04:01.539020+0000","last_became_peered":"2024-04-24T09:04:01.539020+0000","last_unstale":"2024-04-24T09:04:32.608599+0000","last_undegraded":"2024-04-24T09:04:32.608599+0000","last_fullsized":"2024-04-24T09:04:32.608599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:12:48.728029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.353539+0000","last_change":"2024-04-24T09:04:02.097560+0000","last_active":"2024-04-24T09:05:53.353539+0000","last_peered":"2024-04-24T09:05:53.353539+0000","last_clean":"2024-04-24T09:05:53.353539+0000","last_became_active":"2024-04-24T09:04:02.095612+0000","last_became_peered":"2024-04-24T09:04:02.095612+0000","last_unstale":"2024-04-24T09:05:53.353539+0000","last_undegraded":"2024-04-24T09:05:53.353539+0000","last_fullsized":"2024-04-24T09:05:53.353539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:55:47.747294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.524769+0000","last_change":"2024-04-24T09:04:06.137822+0000","last_active":"2024-04-24T09:04:24.524769+0000","last_peered":"2024-04-24T09:04:24.524769+0000","last_clean":"2024-04-24T09:04:24.524769+0000","last_became_active":"2024-04-24T09:04:06.137357+0000","last_became_peered":"2024-04-24T09:04:06.137357+0000","last_unstale":"2024-04-24T09:04:24.524769+0000","last_undegraded":"2024-04-24T09:04:24.524769+0000","last_fullsized":"2024-04-24T09:04:24.524769+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:55:13.119557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'10","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:43.555942+0000","last_change":"2024-04-24T09:04:24.544395+0000","last_active":"2024-04-24T09:04:43.555942+0000","last_peered":"2024-04-24T09:04:43.555942+0000","last_clean":"2024-04-24T09:04:43.555942+0000","last_became_active":"2024-04-24T09:04:24.535870+0000","last_became_peered":"2024-04-24T09:04:24.535870+0000","last_unstale":"2024-04-24T09:04:43.555942+0000","last_undegraded":"2024-04-24T09:04:43.555942+0000","last_fullsized":"2024-04-24T09:04:43.555942+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:29:21.271387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.374330+0000","last_change":"2024-04-24T09:04:02.106107+0000","last_active":"2024-04-24T09:05:53.374330+0000","last_peered":"2024-04-24T09:05:53.374330+0000","last_clean":"2024-04-24T09:05:53.374330+0000","last_became_active":"2024-04-24T09:04:02.105734+0000","last_became_peered":"2024-04-24T09:04:02.105734+0000","last_unstale":"2024-04-24T09:05:53.374330+0000","last_undegraded":"2024-04-24T09:05:53.374330+0000","last_fullsized":"2024-04-24T09:05:53.374330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:15.372520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.695729+0000","last_change":"2024-04-24T09:04:06.139040+0000","last_active":"2024-04-24T09:04:24.695729+0000","last_peered":"2024-04-24T09:04:24.695729+0000","last_clean":"2024-04-24T09:04:24.695729+0000","last_became_active":"2024-04-24T09:04:06.138532+0000","last_became_peered":"2024-04-24T09:04:06.138532+0000","last_unstale":"2024-04-24T09:04:24.695729+0000","last_undegraded":"2024-04-24T09:04:24.695729+0000","last_fullsized":"2024-04-24T09:04:24.695729+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:28.686052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.612766+0000","last_change":"2024-04-24T09:04:01.546859+0000","last_active":"2024-04-24T09:04:32.612766+0000","last_peered":"2024-04-24T09:04:32.612766+0000","last_clean":"2024-04-24T09:04:32.612766+0000","last_became_active":"2024-04-24T09:04:01.546463+0000","last_became_peered":"2024-04-24T09:04:01.546463+0000","last_unstale":"2024-04-24T09:04:32.612766+0000","last_undegraded":"2024-04-24T09:04:32.612766+0000","last_fullsized":"2024-04-24T09:04:32.612766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:31:12.985687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.370925+0000","last_change":"2024-04-24T09:04:02.112487+0000","last_active":"2024-04-24T09:05:53.370925+0000","last_peered":"2024-04-24T09:05:53.370925+0000","last_clean":"2024-04-24T09:05:53.370925+0000","last_became_active":"2024-04-24T09:04:02.112078+0000","last_became_peered":"2024-04-24T09:04:02.112078+0000","last_unstale":"2024-04-24T09:05:53.370925+0000","last_undegraded":"2024-04-24T09:05:53.370925+0000","last_fullsized":"2024-04-24T09:05:53.370925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:03:47.352110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.539120+0000","last_change":"2024-04-24T09:04:24.539120+0000","last_active":"2024-04-24T09:04:24.539120+0000","last_peered":"2024-04-24T09:04:24.539120+0000","last_clean":"2024-04-24T09:04:24.539120+0000","last_became_active":"2024-04-24T09:04:24.537048+0000","last_became_peered":"2024-04-24T09:04:24.537048+0000","last_unstale":"2024-04-24T09:04:24.539120+0000","last_undegraded":"2024-04-24T09:04:24.539120+0000","last_fullsized":"2024-04-24T09:04:24.539120+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:29:53.484131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.611836+0000","last_change":"2024-04-24T09:04:01.545444+0000","last_active":"2024-04-24T09:04:32.611836+0000","last_peered":"2024-04-24T09:04:32.611836+0000","last_clean":"2024-04-24T09:04:32.611836+0000","last_became_active":"2024-04-24T09:04:01.543816+0000","last_became_peered":"2024-04-24T09:04:01.543816+0000","last_unstale":"2024-04-24T09:04:32.611836+0000","last_undegraded":"2024-04-24T09:04:32.611836+0000","last_fullsized":"2024-04-24T09:04:32.611836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:02:36.221797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.306910+0000","last_change":"2024-04-24T09:04:02.109085+0000","last_active":"2024-04-24T09:05:53.306910+0000","last_peered":"2024-04-24T09:05:53.306910+0000","last_clean":"2024-04-24T09:05:53.306910+0000","last_became_active":"2024-04-24T09:04:02.108642+0000","last_became_peered":"2024-04-24T09:04:02.108642+0000","last_unstale":"2024-04-24T09:05:53.306910+0000","last_undegraded":"2024-04-24T09:05:53.306910+0000","last_fullsized":"2024-04-24T09:05:53.306910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:27:51.751369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.518972+0000","last_change":"2024-04-24T09:04:06.144940+0000","last_active":"2024-04-24T09:04:24.518972+0000","last_peered":"2024-04-24T09:04:24.518972+0000","last_clean":"2024-04-24T09:04:24.518972+0000","last_became_active":"2024-04-24T09:04:06.144553+0000","last_became_peered":"2024-04-24T09:04:06.144553+0000","last_unstale":"2024-04-24T09:04:24.518972+0000","last_undegraded":"2024-04-24T09:04:24.518972+0000","last_fullsized":"2024-04-24T09:04:24.518972+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:17.455644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.610185+0000","last_change":"2024-04-24T09:04:01.538939+0000","last_active":"2024-04-24T09:04:32.610185+0000","last_peered":"2024-04-24T09:04:32.610185+0000","last_clean":"2024-04-24T09:04:32.610185+0000","last_became_active":"2024-04-24T09:04:01.538541+0000","last_became_peered":"2024-04-24T09:04:01.538541+0000","last_unstale":"2024-04-24T09:04:32.610185+0000","last_undegraded":"2024-04-24T09:04:32.610185+0000","last_fullsized":"2024-04-24T09:04:32.610185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:08.185901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.359806+0000","last_change":"2024-04-24T09:04:02.103844+0000","last_active":"2024-04-24T09:05:53.359806+0000","last_peered":"2024-04-24T09:05:53.359806+0000","last_clean":"2024-04-24T09:05:53.359806+0000","last_became_active":"2024-04-24T09:04:02.103416+0000","last_became_peered":"2024-04-24T09:04:02.103416+0000","last_unstale":"2024-04-24T09:05:53.359806+0000","last_undegraded":"2024-04-24T09:05:53.359806+0000","last_fullsized":"2024-04-24T09:05:53.359806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:57:47.236384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.520182+0000","last_change":"2024-04-24T09:04:06.134838+0000","last_active":"2024-04-24T09:04:24.520182+0000","last_peered":"2024-04-24T09:04:24.520182+0000","last_clean":"2024-04-24T09:04:24.520182+0000","last_became_active":"2024-04-24T09:04:06.134343+0000","last_became_peered":"2024-04-24T09:04:06.134343+0000","last_unstale":"2024-04-24T09:04:24.520182+0000","last_undegraded":"2024-04-24T09:04:24.520182+0000","last_fullsized":"2024-04-24T09:04:24.520182+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:56:55.219012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696139+0000","last_change":"2024-04-24T09:04:01.557194+0000","last_active":"2024-04-24T09:04:24.696139+0000","last_peered":"2024-04-24T09:04:24.696139+0000","last_clean":"2024-04-24T09:04:24.696139+0000","last_became_active":"2024-04-24T09:04:01.556441+0000","last_became_peered":"2024-04-24T09:04:01.556441+0000","last_unstale":"2024-04-24T09:04:24.696139+0000","last_undegraded":"2024-04-24T09:04:24.696139+0000","last_fullsized":"2024-04-24T09:04:24.696139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:45:50.399096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.372082+0000","last_change":"2024-04-24T09:04:02.108076+0000","last_active":"2024-04-24T09:05:53.372082+0000","last_peered":"2024-04-24T09:05:53.372082+0000","last_clean":"2024-04-24T09:05:53.372082+0000","last_became_active":"2024-04-24T09:04:02.106829+0000","last_became_peered":"2024-04-24T09:04:02.106829+0000","last_unstale":"2024-04-24T09:05:53.372082+0000","last_undegraded":"2024-04-24T09:05:53.372082+0000","last_fullsized":"2024-04-24T09:05:53.372082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:40:23.075828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.521106+0000","last_change":"2024-04-24T09:04:06.142269+0000","last_active":"2024-04-24T09:04:24.521106+0000","last_peered":"2024-04-24T09:04:24.521106+0000","last_clean":"2024-04-24T09:04:24.521106+0000","last_became_active":"2024-04-24T09:04:06.141725+0000","last_became_peered":"2024-04-24T09:04:06.141725+0000","last_unstale":"2024-04-24T09:04:24.521106+0000","last_undegraded":"2024-04-24T09:04:24.521106+0000","last_fullsized":"2024-04-24T09:04:24.521106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:28:17.934148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:32.615156+0000","last_change":"2024-04-24T09:04:01.553134+0000","last_active":"2024-04-24T09:04:32.615156+0000","last_peered":"2024-04-24T09:04:32.615156+0000","last_clean":"2024-04-24T09:04:32.615156+0000","last_became_active":"2024-04-24T09:04:01.552099+0000","last_became_peered":"2024-04-24T09:04:01.552099+0000","last_unstale":"2024-04-24T09:04:32.615156+0000","last_undegraded":"2024-04-24T09:04:32.615156+0000","last_fullsized":"2024-04-24T09:04:32.615156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:42:36.231405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.379961+0000","last_change":"2024-04-24T09:04:02.116641+0000","last_active":"2024-04-24T09:05:53.379961+0000","last_peered":"2024-04-24T09:05:53.379961+0000","last_clean":"2024-04-24T09:05:53.379961+0000","last_became_active":"2024-04-24T09:04:02.115374+0000","last_became_peered":"2024-04-24T09:04:02.115374+0000","last_unstale":"2024-04-24T09:05:53.379961+0000","last_undegraded":"2024-04-24T09:05:53.379961+0000","last_fullsized":"2024-04-24T09:05:53.379961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:34.450990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.696698+0000","last_change":"2024-04-24T09:04:06.149968+0000","last_active":"2024-04-24T09:04:24.696698+0000","last_peered":"2024-04-24T09:04:24.696698+0000","last_clean":"2024-04-24T09:04:24.696698+0000","last_became_active":"2024-04-24T09:04:06.149425+0000","last_became_peered":"2024-04-24T09:04:06.149425+0000","last_unstale":"2024-04-24T09:04:24.696698+0000","last_undegraded":"2024-04-24T09:04:24.696698+0000","last_fullsized":"2024-04-24T09:04:24.696698+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:23:53.869004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:33.633039+0000","last_change":"2024-04-24T09:04:01.553822+0000","last_active":"2024-04-24T09:04:33.633039+0000","last_peered":"2024-04-24T09:04:33.633039+0000","last_clean":"2024-04-24T09:04:33.633039+0000","last_became_active":"2024-04-24T09:04:01.551468+0000","last_became_peered":"2024-04-24T09:04:01.551468+0000","last_unstale":"2024-04-24T09:04:33.633039+0000","last_undegraded":"2024-04-24T09:04:33.633039+0000","last_fullsized":"2024-04-24T09:04:33.633039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:56.367584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.378945+0000","last_change":"2024-04-24T09:04:02.115472+0000","last_active":"2024-04-24T09:05:53.378945+0000","last_peered":"2024-04-24T09:05:53.378945+0000","last_clean":"2024-04-24T09:05:53.378945+0000","last_became_active":"2024-04-24T09:04:02.114666+0000","last_became_peered":"2024-04-24T09:04:02.114666+0000","last_unstale":"2024-04-24T09:05:53.378945+0000","last_undegraded":"2024-04-24T09:05:53.378945+0000","last_fullsized":"2024-04-24T09:05:53.378945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:28:39.145108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.697174+0000","last_change":"2024-04-24T09:04:06.138786+0000","last_active":"2024-04-24T09:04:24.697174+0000","last_peered":"2024-04-24T09:04:24.697174+0000","last_clean":"2024-04-24T09:04:24.697174+0000","last_became_active":"2024-04-24T09:04:06.138366+0000","last_became_peered":"2024-04-24T09:04:06.138366+0000","last_unstale":"2024-04-24T09:04:24.697174+0000","last_undegraded":"2024-04-24T09:04:24.697174+0000","last_fullsized":"2024-04-24T09:04:24.697174+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:03:16.729928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533806+0000","last_change":"2024-04-24T09:04:01.547733+0000","last_active":"2024-04-24T09:04:24.533806+0000","last_peered":"2024-04-24T09:04:24.533806+0000","last_clean":"2024-04-24T09:04:24.533806+0000","last_became_active":"2024-04-24T09:04:01.544839+0000","last_became_peered":"2024-04-24T09:04:01.544839+0000","last_unstale":"2024-04-24T09:04:24.533806+0000","last_undegraded":"2024-04-24T09:04:24.533806+0000","last_fullsized":"2024-04-24T09:04:24.533806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:45:03.847092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.373123+0000","last_change":"2024-04-24T09:04:02.108383+0000","last_active":"2024-04-24T09:05:53.373123+0000","last_peered":"2024-04-24T09:05:53.373123+0000","last_clean":"2024-04-24T09:05:53.373123+0000","last_became_active":"2024-04-24T09:04:02.107883+0000","last_became_peered":"2024-04-24T09:04:02.107883+0000","last_unstale":"2024-04-24T09:05:53.373123+0000","last_undegraded":"2024-04-24T09:05:53.373123+0000","last_fullsized":"2024-04-24T09:05:53.373123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:24:44.923708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.527772+0000","last_change":"2024-04-24T09:04:06.140868+0000","last_active":"2024-04-24T09:04:24.527772+0000","last_peered":"2024-04-24T09:04:24.527772+0000","last_clean":"2024-04-24T09:04:24.527772+0000","last_became_active":"2024-04-24T09:04:06.139174+0000","last_became_peered":"2024-04-24T09:04:06.139174+0000","last_unstale":"2024-04-24T09:04:24.527772+0000","last_undegraded":"2024-04-24T09:04:24.527772+0000","last_fullsized":"2024-04-24T09:04:24.527772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:17:17.334276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:04:24.533054+0000","last_change":"2024-04-24T09:04:01.534653+0000","last_active":"2024-04-24T09:04:24.533054+0000","last_peered":"2024-04-24T09:04:24.533054+0000","last_clean":"2024-04-24T09:04:24.533054+0000","last_became_active":"2024-04-24T09:04:01.534312+0000","last_became_peered":"2024-04-24T09:04:01.534312+0000","last_unstale":"2024-04-24T09:04:24.533054+0000","last_undegraded":"2024-04-24T09:04:24.533054+0000","last_fullsized":"2024-04-24T09:04:24.533054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:53:34.301007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.365862+0000","last_change":"2024-04-24T09:04:02.102056+0000","last_active":"2024-04-24T09:05:53.365862+0000","last_peered":"2024-04-24T09:05:53.365862+0000","last_clean":"2024-04-24T09:05:53.365862+0000","last_became_active":"2024-04-24T09:04:02.101641+0000","last_became_peered":"2024-04-24T09:04:02.101641+0000","last_unstale":"2024-04-24T09:05:53.365862+0000","last_undegraded":"2024-04-24T09:05:53.365862+0000","last_fullsized":"2024-04-24T09:05:53.365862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:41.654024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.362819+0000","last_change":"2024-04-24T09:04:02.100327+0000","last_active":"2024-04-24T09:05:53.362819+0000","last_peered":"2024-04-24T09:05:53.362819+0000","last_clean":"2024-04-24T09:05:53.362819+0000","last_became_active":"2024-04-24T09:04:02.100028+0000","last_became_peered":"2024-04-24T09:04:02.100028+0000","last_unstale":"2024-04-24T09:05:53.362819+0000","last_undegraded":"2024-04-24T09:05:53.362819+0000","last_fullsized":"2024-04-24T09:05:53.362819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:15:18.728045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.355093+0000","last_change":"2024-04-24T09:04:02.101996+0000","last_active":"2024-04-24T09:05:53.355093+0000","last_peered":"2024-04-24T09:05:53.355093+0000","last_clean":"2024-04-24T09:05:53.355093+0000","last_became_active":"2024-04-24T09:04:02.101581+0000","last_became_peered":"2024-04-24T09:04:02.101581+0000","last_unstale":"2024-04-24T09:05:53.355093+0000","last_undegraded":"2024-04-24T09:05:53.355093+0000","last_fullsized":"2024-04-24T09:05:53.355093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:55.487781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.362367+0000","last_change":"2024-04-24T09:04:02.107288+0000","last_active":"2024-04-24T09:05:53.362367+0000","last_peered":"2024-04-24T09:05:53.362367+0000","last_clean":"2024-04-24T09:05:53.362367+0000","last_became_active":"2024-04-24T09:04:02.106946+0000","last_became_peered":"2024-04-24T09:04:02.106946+0000","last_unstale":"2024-04-24T09:05:53.362367+0000","last_undegraded":"2024-04-24T09:05:53.362367+0000","last_fullsized":"2024-04-24T09:05:53.362367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:38:56.821285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.366560+0000","last_change":"2024-04-24T09:04:02.116829+0000","last_active":"2024-04-24T09:05:53.366560+0000","last_peered":"2024-04-24T09:05:53.366560+0000","last_clean":"2024-04-24T09:05:53.366560+0000","last_became_active":"2024-04-24T09:04:02.116520+0000","last_became_peered":"2024-04-24T09:04:02.116520+0000","last_unstale":"2024-04-24T09:05:53.366560+0000","last_undegraded":"2024-04-24T09:05:53.366560+0000","last_fullsized":"2024-04-24T09:05:53.366560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:26.873826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.369610+0000","last_change":"2024-04-24T09:04:24.700595+0000","last_active":"2024-04-24T09:05:53.369610+0000","last_peered":"2024-04-24T09:05:53.369610+0000","last_clean":"2024-04-24T09:05:53.369610+0000","last_became_active":"2024-04-24T09:04:24.699395+0000","last_became_peered":"2024-04-24T09:04:24.699395+0000","last_unstale":"2024-04-24T09:05:53.369610+0000","last_undegraded":"2024-04-24T09:05:53.369610+0000","last_fullsized":"2024-04-24T09:05:53.369610+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:28.756302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.362263+0000","last_change":"2024-04-24T09:04:02.114087+0000","last_active":"2024-04-24T09:05:53.362263+0000","last_peered":"2024-04-24T09:05:53.362263+0000","last_clean":"2024-04-24T09:05:53.362263+0000","last_became_active":"2024-04-24T09:04:02.112681+0000","last_became_peered":"2024-04-24T09:04:02.112681+0000","last_unstale":"2024-04-24T09:05:53.362263+0000","last_undegraded":"2024-04-24T09:05:53.362263+0000","last_fullsized":"2024-04-24T09:05:53.362263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:50:01.762598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.379473+0000","last_change":"2024-04-24T09:04:02.102322+0000","last_active":"2024-04-24T09:05:53.379473+0000","last_peered":"2024-04-24T09:05:53.379473+0000","last_clean":"2024-04-24T09:05:53.379473+0000","last_became_active":"2024-04-24T09:04:02.101963+0000","last_became_peered":"2024-04-24T09:04:02.101963+0000","last_unstale":"2024-04-24T09:05:53.379473+0000","last_undegraded":"2024-04-24T09:05:53.379473+0000","last_fullsized":"2024-04-24T09:05:53.379473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:16.140829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.356404+0000","last_change":"2024-04-24T09:04:02.110570+0000","last_active":"2024-04-24T09:05:53.356404+0000","last_peered":"2024-04-24T09:05:53.356404+0000","last_clean":"2024-04-24T09:05:53.356404+0000","last_became_active":"2024-04-24T09:04:02.110119+0000","last_became_peered":"2024-04-24T09:04:02.110119+0000","last_unstale":"2024-04-24T09:05:53.356404+0000","last_undegraded":"2024-04-24T09:05:53.356404+0000","last_fullsized":"2024-04-24T09:05:53.356404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:34.436430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.373511+0000","last_change":"2024-04-24T09:04:02.108292+0000","last_active":"2024-04-24T09:05:53.373511+0000","last_peered":"2024-04-24T09:05:53.373511+0000","last_clean":"2024-04-24T09:05:53.373511+0000","last_became_active":"2024-04-24T09:04:02.107481+0000","last_became_peered":"2024-04-24T09:04:02.107481+0000","last_unstale":"2024-04-24T09:05:53.373511+0000","last_undegraded":"2024-04-24T09:05:53.373511+0000","last_fullsized":"2024-04-24T09:05:53.373511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:49.755293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.371257+0000","last_change":"2024-04-24T09:04:02.102228+0000","last_active":"2024-04-24T09:05:53.371257+0000","last_peered":"2024-04-24T09:05:53.371257+0000","last_clean":"2024-04-24T09:05:53.371257+0000","last_became_active":"2024-04-24T09:04:02.101863+0000","last_became_peered":"2024-04-24T09:04:02.101863+0000","last_unstale":"2024-04-24T09:05:53.371257+0000","last_undegraded":"2024-04-24T09:05:53.371257+0000","last_fullsized":"2024-04-24T09:05:53.371257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:02:47.308116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.379324+0000","last_change":"2024-04-24T09:04:02.106749+0000","last_active":"2024-04-24T09:05:53.379324+0000","last_peered":"2024-04-24T09:05:53.379324+0000","last_clean":"2024-04-24T09:05:53.379324+0000","last_became_active":"2024-04-24T09:04:02.106282+0000","last_became_peered":"2024-04-24T09:04:02.106282+0000","last_unstale":"2024-04-24T09:05:53.379324+0000","last_undegraded":"2024-04-24T09:05:53.379324+0000","last_fullsized":"2024-04-24T09:05:53.379324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:35:59.765230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.369805+0000","last_change":"2024-04-24T09:04:02.097825+0000","last_active":"2024-04-24T09:05:53.369805+0000","last_peered":"2024-04-24T09:05:53.369805+0000","last_clean":"2024-04-24T09:05:53.369805+0000","last_became_active":"2024-04-24T09:04:02.097488+0000","last_became_peered":"2024-04-24T09:04:02.097488+0000","last_unstale":"2024-04-24T09:05:53.369805+0000","last_undegraded":"2024-04-24T09:05:53.369805+0000","last_fullsized":"2024-04-24T09:05:53.369805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:05:59.962558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T09:05:53.357499+0000","last_change":"2024-04-24T09:04:02.113819+0000","last_active":"2024-04-24T09:05:53.357499+0000","last_peered":"2024-04-24T09:05:53.357499+0000","last_clean":"2024-04-24T09:05:53.357499+0000","last_became_active":"2024-04-24T09:04:02.113463+0000","last_became_peered":"2024-04-24T09:04:02.113463+0000","last_unstale":"2024-04-24T09:05:53.357499+0000","last_undegraded":"2024-04-24T09:05:53.357499+0000","last_fullsized":"2024-04-24T09:05:53.357499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:47:05.109583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70429,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":559,"num_read_kb":356,"num_write":175,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":211287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351017,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":507486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612656,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32868,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712284,"statfs":{"total":95995035648,"available":95961378816,"internally_reserved":0,"allocated":1732608,"data_stored":1161573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808888,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":511818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070526,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":507642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332166,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32784,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712368,"statfs":{"total":95995035648,"available":95961464832,"internally_reserved":0,"allocated":1646592,"data_stored":1103692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495693,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":502898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":569095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738485,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":504286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":569823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182498,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":503092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920858,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":508702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757332,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32792,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712360,"statfs":{"total":95995035648,"available":95961456640,"internally_reserved":0,"allocated":1654784,"data_stored":1098748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T09:06:07.022 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-24T09:06:07.023 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T09:06:07.023 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-24T09:06:07.023 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-24T09:06:07.514 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:06:07.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:07 smithi081 ceph-mon[30961]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:06:07.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:07 smithi069 ceph-mon[28170]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:06:07.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:07 smithi130 ceph-mon[31553]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:06:08.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:08 smithi081 ceph-mon[30961]: pgmap v434: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-24T09:06:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:08 smithi069 ceph-mon[28170]: pgmap v434: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-24T09:06:08.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:08 smithi130 ceph-mon[31553]: pgmap v434: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-24T09:06:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.935+0000 7fbf82400700 1 -- 172.21.15.69:0/2153397036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf7c1022e0 msgr2=0x7fbf7c10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.935+0000 7fbf82400700 1 --2- 172.21.15.69:0/2153397036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf7c1022e0 0x7fbf7c10e7f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400b0a0 tx=0x7fbf7400b3b0 comp rx=0 tx=0).stop 2024-04-24T09:06:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.936+0000 7fbf82400700 1 -- 172.21.15.69:0/2153397036 shutdown_connections 2024-04-24T09:06:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.936+0000 7fbf82400700 1 --2- 172.21.15.69:0/2153397036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf7c1022e0 0x7fbf7c10e7f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.936+0000 7fbf82400700 1 --2- 172.21.15.69:0/2153397036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf7c107920 0x7fbf7c107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:09.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.936+0000 7fbf82400700 1 --2- 172.21.15.69:0/2153397036 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf7c101920 0x7fbf7c101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:09.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.936+0000 7fbf82400700 1 -- 172.21.15.69:0/2153397036 >> 172.21.15.69:0/2153397036 conn(0x7fbf7c0fd410 msgr2=0x7fbf7c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:09.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.936+0000 7fbf82400700 1 -- 172.21.15.69:0/2153397036 shutdown_connections 2024-04-24T09:06:09.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.936+0000 7fbf82400700 1 -- 172.21.15.69:0/2153397036 wait complete. 2024-04-24T09:06:09.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.937+0000 7fbf82400700 1 Processor -- start 2024-04-24T09:06:09.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.937+0000 7fbf82400700 1 -- start start 2024-04-24T09:06:09.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.937+0000 7fbf82400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf7c101920 0x7fbf7c110c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.937+0000 7fbf82400700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf7c1022e0 0x7fbf7c111140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.937+0000 7fbf82400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf7c107920 0x7fbf7c111680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.937+0000 7fbf82400700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c112d40 con 0x7fbf7c107920 2024-04-24T09:06:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.937+0000 7fbf82400700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c06b6a0 con 0x7fbf7c1022e0 2024-04-24T09:06:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.937+0000 7fbf82400700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c112ec0 con 0x7fbf7c101920 2024-04-24T09:06:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.938+0000 7fbf81a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf7c107920 0x7fbf7c111680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.938+0000 7fbf81a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf7c107920 0x7fbf7c111680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35350/0 (socket says 172.21.15.69:35350) 2024-04-24T09:06:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.938+0000 7fbf81a00700 1 -- 172.21.15.69:0/1996655026 learned_addr learned my addr 172.21.15.69:0/1996655026 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.938+0000 7fbf81000700 1 --2- 172.21.15.69:0/1996655026 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf7c101920 0x7fbf7c110c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.938+0000 7fbf7be00700 1 --2- 172.21.15.69:0/1996655026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf7c1022e0 0x7fbf7c111140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:09.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.938+0000 7fbf81a00700 1 -- 172.21.15.69:0/1996655026 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf7c101920 msgr2=0x7fbf7c110c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.938+0000 7fbf81a00700 1 --2- 172.21.15.69:0/1996655026 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf7c101920 0x7fbf7c110c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.938+0000 7fbf81a00700 1 -- 172.21.15.69:0/1996655026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf7c1022e0 msgr2=0x7fbf7c111140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.938+0000 7fbf81a00700 1 --2- 172.21.15.69:0/1996655026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf7c1022e0 0x7fbf7c111140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.938+0000 7fbf81a00700 1 -- 172.21.15.69:0/1996655026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7c1a7600 con 0x7fbf7c107920 2024-04-24T09:06:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.939+0000 7fbf81a00700 1 --2- 172.21.15.69:0/1996655026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf7c107920 0x7fbf7c111680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009fd0 tx=0x7fbf74004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.939+0000 7fbf79600700 1 -- 172.21.15.69:0/1996655026 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74021070 con 0x7fbf7c107920 2024-04-24T09:06:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.939+0000 7fbf82400700 1 -- 172.21.15.69:0/1996655026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7c1a7890 con 0x7fbf7c107920 2024-04-24T09:06:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.939+0000 7fbf79600700 1 -- 172.21.15.69:0/1996655026 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf74007c80 con 0x7fbf7c107920 2024-04-24T09:06:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.939+0000 7fbf82400700 1 -- 172.21.15.69:0/1996655026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf7c1a7cf0 con 0x7fbf7c107920 2024-04-24T09:06:09.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.939+0000 7fbf79600700 1 -- 172.21.15.69:0/1996655026 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7402b060 con 0x7fbf7c107920 2024-04-24T09:06:09.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.940+0000 7fbf82400700 1 -- 172.21.15.69:0/1996655026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf7c0697e0 con 0x7fbf7c107920 2024-04-24T09:06:09.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.943+0000 7fbf79600700 1 -- 172.21.15.69:0/1996655026 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbf74024070 con 0x7fbf7c107920 2024-04-24T09:06:09.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.944+0000 7fbf79600700 1 --2- 172.21.15.69:0/1996655026 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbf640a3c40 0x7fbf640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:09.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.944+0000 7fbf79600700 1 -- 172.21.15.69:0/1996655026 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fbf740d18b0 con 0x7fbf7c107920 2024-04-24T09:06:09.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.945+0000 7fbf81000700 1 --2- 172.21.15.69:0/1996655026 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbf640a3c40 0x7fbf640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:09.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.945+0000 7fbf81000700 1 --2- 172.21.15.69:0/1996655026 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbf640a3c40 0x7fbf640a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbf70005950 tx=0x7fbf700058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:09.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:09.949+0000 7fbf79600700 1 -- 172.21.15.69:0/1996655026 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf7409df60 con 0x7fbf7c107920 2024-04-24T09:06:10.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:10.235+0000 7fbf82400700 1 -- 172.21.15.69:0/1996655026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbf7c02dfe0 con 0x7fbf7c107920 2024-04-24T09:06:10.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:10.235+0000 7fbf79600700 1 -- 172.21.15.69:0/1996655026 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fbf7409d860 con 0x7fbf7c107920 2024-04-24T09:06:10.235 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:10.235 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T09:06:10.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:10.238+0000 7fbf82400700 1 -- 172.21.15.69:0/1996655026 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbf640a3c40 msgr2=0x7fbf640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:10.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:10.238+0000 7fbf82400700 1 --2- 172.21.15.69:0/1996655026 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbf640a3c40 0x7fbf640a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbf70005950 tx=0x7fbf700058e0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:10.238+0000 7fbf82400700 1 -- 172.21.15.69:0/1996655026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf7c107920 msgr2=0x7fbf7c111680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:10.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:10.238+0000 7fbf82400700 1 --2- 172.21.15.69:0/1996655026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf7c107920 0x7fbf7c111680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009fd0 tx=0x7fbf74004e70 comp rx=0 tx=0).stop 2024-04-24T09:06:10.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:10.238+0000 7fbf82400700 1 -- 172.21.15.69:0/1996655026 shutdown_connections 2024-04-24T09:06:10.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:10.238+0000 7fbf82400700 1 --2- 172.21.15.69:0/1996655026 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fbf640a3c40 0x7fbf640a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:10.238+0000 7fbf82400700 1 --2- 172.21.15.69:0/1996655026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf7c107920 0x7fbf7c111680 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:10.238+0000 7fbf82400700 1 --2- 172.21.15.69:0/1996655026 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbf7c101920 0x7fbf7c110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:10.238+0000 7fbf82400700 1 --2- 172.21.15.69:0/1996655026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf7c1022e0 0x7fbf7c111140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:10.238+0000 7fbf82400700 1 -- 172.21.15.69:0/1996655026 >> 172.21.15.69:0/1996655026 conn(0x7fbf7c0fd410 msgr2=0x7fbf7c0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:10.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:10.239+0000 7fbf82400700 1 -- 172.21.15.69:0/1996655026 shutdown_connections 2024-04-24T09:06:10.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:10.239+0000 7fbf82400700 1 -- 172.21.15.69:0/1996655026 wait complete. 2024-04-24T09:06:10.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:10 smithi081 ceph-mon[30961]: pgmap v435: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-24T09:06:10.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:10 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1996655026' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T09:06:10.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:10 smithi069 ceph-mon[28170]: pgmap v435: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-24T09:06:10.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:10 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1996655026' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T09:06:10.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:10 smithi130 ceph-mon[31553]: pgmap v435: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-24T09:06:10.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:10 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1996655026' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T09:06:11.166 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-24T09:06:11.167 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:06:11.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.381+0000 7ffbe7e00640 1 -- 172.21.15.69:0/4224056389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbe8102960 msgr2=0x7ffbe8104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:11.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.381+0000 7ffbe7e00640 1 --2- 172.21.15.69:0/4224056389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbe8102960 0x7ffbe8104d50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009ca0 tx=0x7ffbd802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.381+0000 7ffbe7e00640 1 -- 172.21.15.69:0/4224056389 shutdown_connections 2024-04-24T09:06:11.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.381+0000 7ffbe7e00640 1 --2- 172.21.15.69:0/4224056389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbe8107bc0 0x7ffbe8109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.381+0000 7ffbe7e00640 1 --2- 172.21.15.69:0/4224056389 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffbe8105290 0x7ffbe8107680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.381+0000 7ffbe7e00640 1 --2- 172.21.15.69:0/4224056389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbe8102960 0x7ffbe8104d50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.381+0000 7ffbe7e00640 1 -- 172.21.15.69:0/4224056389 >> 172.21.15.69:0/4224056389 conn(0x7ffbe80fc2d0 msgr2=0x7ffbe80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:11.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.381+0000 7ffbe7e00640 1 -- 172.21.15.69:0/4224056389 shutdown_connections 2024-04-24T09:06:11.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.381+0000 7ffbe7e00640 1 -- 172.21.15.69:0/4224056389 wait complete. 2024-04-24T09:06:11.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.382+0000 7ffbe7e00640 1 Processor -- start 2024-04-24T09:06:11.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.382+0000 7ffbe7e00640 1 -- start start 2024-04-24T09:06:11.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.382+0000 7ffbe7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbe8102960 0x7ffbe819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:11.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.382+0000 7ffbe7e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbe8105290 0x7ffbe819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:11.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.382+0000 7ffbe7e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffbe8107bc0 0x7ffbe81a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:11.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.382+0000 7ffbe7e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe810d9f0 con 0x7ffbe8102960 2024-04-24T09:06:11.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.382+0000 7ffbe7e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe810d870 con 0x7ffbe8105290 2024-04-24T09:06:11.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.382+0000 7ffbe7e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe810db70 con 0x7ffbe8107bc0 2024-04-24T09:06:11.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.383+0000 7ffbe6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbe8102960 0x7ffbe819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:11.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.383+0000 7ffbe6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbe8102960 0x7ffbe819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35360/0 (socket says 172.21.15.69:35360) 2024-04-24T09:06:11.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.383+0000 7ffbe6a00640 1 -- 172.21.15.69:0/1627604324 learned_addr learned my addr 172.21.15.69:0/1627604324 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:11.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.383+0000 7ffbe7400640 1 --2- 172.21.15.69:0/1627604324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffbe8107bc0 0x7ffbe81a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:11.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.383+0000 7ffbe6a00640 1 -- 172.21.15.69:0/1627604324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffbe8107bc0 msgr2=0x7ffbe81a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:11.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.383+0000 7ffbe6a00640 1 --2- 172.21.15.69:0/1627604324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffbe8107bc0 0x7ffbe81a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.383+0000 7ffbe6a00640 1 -- 172.21.15.69:0/1627604324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbe8105290 msgr2=0x7ffbe819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:11.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.383+0000 7ffbde000640 1 --2- 172.21.15.69:0/1627604324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbe8105290 0x7ffbe819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:11.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.383+0000 7ffbe6a00640 1 --2- 172.21.15.69:0/1627604324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbe8105290 0x7ffbe819a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.383+0000 7ffbe6a00640 1 -- 172.21.15.69:0/1627604324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe81a1ea0 con 0x7ffbe8102960 2024-04-24T09:06:11.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.383+0000 7ffbe6a00640 1 --2- 172.21.15.69:0/1627604324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbe8102960 0x7ffbe819a280 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ffbd802aeb0 tx=0x7ffbd8033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:11.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.384+0000 7ffbeca00640 1 -- 172.21.15.69:0/1627604324 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8004040 con 0x7ffbe8102960 2024-04-24T09:06:11.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.384+0000 7ffbeca00640 1 -- 172.21.15.69:0/1627604324 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbd8004690 con 0x7ffbe8102960 2024-04-24T09:06:11.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.384+0000 7ffbe7e00640 1 -- 172.21.15.69:0/1627604324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe81a2130 con 0x7ffbe8102960 2024-04-24T09:06:11.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.384+0000 7ffbeca00640 1 -- 172.21.15.69:0/1627604324 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd803b410 con 0x7ffbe8102960 2024-04-24T09:06:11.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.384+0000 7ffbe7e00640 1 -- 172.21.15.69:0/1627604324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe81a26d0 con 0x7ffbe8102960 2024-04-24T09:06:11.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.385+0000 7ffbe7e00640 1 -- 172.21.15.69:0/1627604324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe81096b0 con 0x7ffbe8102960 2024-04-24T09:06:11.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.387+0000 7ffbeca00640 1 -- 172.21.15.69:0/1627604324 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ffbd80041e0 con 0x7ffbe8102960 2024-04-24T09:06:11.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.389+0000 7ffbeca00640 1 --2- 172.21.15.69:0/1627604324 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ffbcc0a3c30 0x7ffbcc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:11.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.389+0000 7ffbeca00640 1 -- 172.21.15.69:0/1627604324 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ffbd80f1f40 con 0x7ffbe8102960 2024-04-24T09:06:11.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.390+0000 7ffbde000640 1 --2- 172.21.15.69:0/1627604324 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ffbcc0a3c30 0x7ffbcc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:11.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.390+0000 7ffbde000640 1 --2- 172.21.15.69:0/1627604324 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ffbcc0a3c30 0x7ffbcc0a60f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffbe819b790 tx=0x7ffbd0005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:11.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.394+0000 7ffbeca00640 1 -- 172.21.15.69:0/1627604324 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffbd80be560 con 0x7ffbe8102960 2024-04-24T09:06:11.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.599+0000 7ffbe7e00640 1 -- 172.21.15.69:0/1627604324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbe8062c00 con 0x7ffbe8102960 2024-04-24T09:06:11.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.602+0000 7ffbeca00640 1 -- 172.21.15.69:0/1627604324 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24564 (secure 0 0 0) 0x7ffbd80bde60 con 0x7ffbe8102960 2024-04-24T09:06:11.602 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:11.602 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":92,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:24.497245+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:06:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.605+0000 7ffbe7e00640 1 -- 172.21.15.69:0/1627604324 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ffbcc0a3c30 msgr2=0x7ffbcc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.605+0000 7ffbe7e00640 1 --2- 172.21.15.69:0/1627604324 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ffbcc0a3c30 0x7ffbcc0a60f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffbe819b790 tx=0x7ffbd0005c60 comp rx=0 tx=0).stop 2024-04-24T09:06:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.605+0000 7ffbe7e00640 1 -- 172.21.15.69:0/1627604324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbe8102960 msgr2=0x7ffbe819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.605+0000 7ffbe7e00640 1 --2- 172.21.15.69:0/1627604324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbe8102960 0x7ffbe819a280 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ffbd802aeb0 tx=0x7ffbd8033000 comp rx=0 tx=0).stop 2024-04-24T09:06:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.605+0000 7ffbe7e00640 1 -- 172.21.15.69:0/1627604324 shutdown_connections 2024-04-24T09:06:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.605+0000 7ffbe7e00640 1 --2- 172.21.15.69:0/1627604324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffbe8107bc0 0x7ffbe81a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.605+0000 7ffbe7e00640 1 --2- 172.21.15.69:0/1627604324 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ffbcc0a3c30 0x7ffbcc0a60f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.605+0000 7ffbe7e00640 1 --2- 172.21.15.69:0/1627604324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbe8105290 0x7ffbe819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.605+0000 7ffbe7e00640 1 --2- 172.21.15.69:0/1627604324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbe8102960 0x7ffbe819a280 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.605+0000 7ffbe7e00640 1 -- 172.21.15.69:0/1627604324 >> 172.21.15.69:0/1627604324 conn(0x7ffbe80fc2d0 msgr2=0x7ffbe8101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:11.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.605+0000 7ffbe7e00640 1 -- 172.21.15.69:0/1627604324 shutdown_connections 2024-04-24T09:06:11.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.605+0000 7ffbe7e00640 1 -- 172.21.15.69:0/1627604324 wait complete. 2024-04-24T09:06:11.625 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:57:37.377599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:04:00.299827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:04:00.958498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T09:04:04.105314+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T09:06:11.626 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:06:11.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.835+0000 7f05e3200640 1 -- 172.21.15.69:0/3695628892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05dc10a0f0 msgr2=0x7f05dc110980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:11.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.835+0000 7f05e3200640 1 --2- 172.21.15.69:0/3695628892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05dc10a0f0 0x7f05dc110980 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009ca0 tx=0x7f05d802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.835+0000 7f05e3200640 1 -- 172.21.15.69:0/3695628892 shutdown_connections 2024-04-24T09:06:11.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.835+0000 7f05e3200640 1 --2- 172.21.15.69:0/3695628892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05dc10a0f0 0x7f05dc110980 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.835+0000 7f05e3200640 1 --2- 172.21.15.69:0/3695628892 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05dc109730 0x7f05dc109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.835+0000 7f05e3200640 1 --2- 172.21.15.69:0/3695628892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05dc108540 0x7f05dc108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.835+0000 7f05e3200640 1 -- 172.21.15.69:0/3695628892 >> 172.21.15.69:0/3695628892 conn(0x7f05dc0774d0 msgr2=0x7f05dc079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:11.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.835+0000 7f05e3200640 1 -- 172.21.15.69:0/3695628892 shutdown_connections 2024-04-24T09:06:11.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.835+0000 7f05e3200640 1 -- 172.21.15.69:0/3695628892 wait complete. 2024-04-24T09:06:11.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.837+0000 7f05e3200640 1 Processor -- start 2024-04-24T09:06:11.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.837+0000 7f05e3200640 1 -- start start 2024-04-24T09:06:11.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.837+0000 7f05e3200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05dc108540 0x7f05dc19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:11.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.837+0000 7f05e3200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05dc109730 0x7f05dc19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:11.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.837+0000 7f05e3200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05dc10a0f0 0x7f05dc1a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:11.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.837+0000 7f05e3200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc114080 con 0x7f05dc10a0f0 2024-04-24T09:06:11.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.837+0000 7f05e3200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc113f00 con 0x7f05dc109730 2024-04-24T09:06:11.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.837+0000 7f05e3200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc114200 con 0x7f05dc108540 2024-04-24T09:06:11.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.838+0000 7f05e2800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05dc10a0f0 0x7f05dc1a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:11.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.838+0000 7f05e2800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05dc10a0f0 0x7f05dc1a5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35388/0 (socket says 172.21.15.69:35388) 2024-04-24T09:06:11.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.838+0000 7f05e2800640 1 -- 172.21.15.69:0/732882729 learned_addr learned my addr 172.21.15.69:0/732882729 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:11.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.838+0000 7f05e1e00640 1 --2- 172.21.15.69:0/732882729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05dc108540 0x7f05dc19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:11.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.838+0000 7f05e1400640 1 --2- 172.21.15.69:0/732882729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05dc109730 0x7f05dc19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:11.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.838+0000 7f05e2800640 1 -- 172.21.15.69:0/732882729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05dc108540 msgr2=0x7f05dc19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:11.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.838+0000 7f05e2800640 1 --2- 172.21.15.69:0/732882729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05dc108540 0x7f05dc19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.838+0000 7f05e2800640 1 -- 172.21.15.69:0/732882729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05dc109730 msgr2=0x7f05dc19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:11.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.838+0000 7f05e2800640 1 --2- 172.21.15.69:0/732882729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05dc109730 0x7f05dc19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.838+0000 7f05e2800640 1 -- 172.21.15.69:0/732882729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05dc1a62f0 con 0x7f05dc10a0f0 2024-04-24T09:06:11.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.838+0000 7f05e1e00640 1 --2- 172.21.15.69:0/732882729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05dc108540 0x7f05dc19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:11.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.838+0000 7f05e2800640 1 --2- 172.21.15.69:0/732882729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05dc10a0f0 0x7f05dc1a5c80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f05d8031970 tx=0x7f05d8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:11.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.839+0000 7f05d6a00640 1 -- 172.21.15.69:0/732882729 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d8003d90 con 0x7f05dc10a0f0 2024-04-24T09:06:11.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.839+0000 7f05e3200640 1 -- 172.21.15.69:0/732882729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05dc1a6580 con 0x7f05dc10a0f0 2024-04-24T09:06:11.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.839+0000 7f05d6a00640 1 -- 172.21.15.69:0/732882729 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05d80044f0 con 0x7f05dc10a0f0 2024-04-24T09:06:11.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.839+0000 7f05e3200640 1 -- 172.21.15.69:0/732882729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05dc1a6b20 con 0x7f05dc10a0f0 2024-04-24T09:06:11.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.839+0000 7f05d6a00640 1 -- 172.21.15.69:0/732882729 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d8030b70 con 0x7f05dc10a0f0 2024-04-24T09:06:11.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.840+0000 7f05e3200640 1 -- 172.21.15.69:0/732882729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05dc10f8e0 con 0x7f05dc10a0f0 2024-04-24T09:06:11.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.842+0000 7f05d6a00640 1 -- 172.21.15.69:0/732882729 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f05d8030d10 con 0x7f05dc10a0f0 2024-04-24T09:06:11.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.843+0000 7f05d6a00640 1 --2- 172.21.15.69:0/732882729 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f05c40a3bb0 0x7f05c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:11.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.844+0000 7f05d6a00640 1 -- 172.21.15.69:0/732882729 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f05d80f1f50 con 0x7f05dc10a0f0 2024-04-24T09:06:11.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.844+0000 7f05e1e00640 1 --2- 172.21.15.69:0/732882729 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f05c40a3bb0 0x7f05c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:11.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.844+0000 7f05e1e00640 1 --2- 172.21.15.69:0/732882729 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f05c40a3bb0 0x7f05c40a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f05dc109590 tx=0x7f05d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:11.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:11.846+0000 7f05d6a00640 1 -- 172.21.15.69:0/732882729 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05d80be690 con 0x7f05dc10a0f0 2024-04-24T09:06:12.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.047+0000 7f05e3200640 1 -- 172.21.15.69:0/732882729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f05dc02d620 con 0x7f05dc10a0f0 2024-04-24T09:06:12.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.048+0000 7f05d6a00640 1 -- 172.21.15.69:0/732882729 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f05d80bdf90 con 0x7f05dc10a0f0 2024-04-24T09:06:12.051 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-24T09:06:12.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.051+0000 7f05e3200640 1 -- 172.21.15.69:0/732882729 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f05c40a3bb0 msgr2=0x7f05c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:12.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.051+0000 7f05e3200640 1 --2- 172.21.15.69:0/732882729 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f05c40a3bb0 0x7f05c40a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f05dc109590 tx=0x7f05d0005f90 comp rx=0 tx=0).stop 2024-04-24T09:06:12.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.051+0000 7f05e3200640 1 -- 172.21.15.69:0/732882729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05dc10a0f0 msgr2=0x7f05dc1a5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:12.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.051+0000 7f05e3200640 1 --2- 172.21.15.69:0/732882729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05dc10a0f0 0x7f05dc1a5c80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f05d8031970 tx=0x7f05d8003ba0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.051+0000 7f05e3200640 1 -- 172.21.15.69:0/732882729 shutdown_connections 2024-04-24T09:06:12.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.051+0000 7f05e3200640 1 --2- 172.21.15.69:0/732882729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05dc10a0f0 0x7f05dc1a5c80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.051+0000 7f05e3200640 1 --2- 172.21.15.69:0/732882729 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f05c40a3bb0 0x7f05c40a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.051+0000 7f05e3200640 1 --2- 172.21.15.69:0/732882729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05dc109730 0x7f05dc19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.051+0000 7f05e3200640 1 --2- 172.21.15.69:0/732882729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05dc108540 0x7f05dc19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.051+0000 7f05e3200640 1 -- 172.21.15.69:0/732882729 >> 172.21.15.69:0/732882729 conn(0x7f05dc0774d0 msgr2=0x7f05dc10cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:12.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.051+0000 7f05e3200640 1 -- 172.21.15.69:0/732882729 shutdown_connections 2024-04-24T09:06:12.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.051+0000 7f05e3200640 1 -- 172.21.15.69:0/732882729 wait complete. 2024-04-24T09:06:12.070 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:06:12.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.267+0000 7f0dd6a00640 1 -- 172.21.15.69:0/4066008240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0105e10 msgr2=0x7f0dd010c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:12.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.267+0000 7f0dd6a00640 1 --2- 172.21.15.69:0/4066008240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0105e10 0x7f0dd010c6a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009ca0 tx=0x7f0dc802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.267+0000 7f0dd6a00640 1 -- 172.21.15.69:0/4066008240 shutdown_connections 2024-04-24T09:06:12.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.267+0000 7f0dd6a00640 1 --2- 172.21.15.69:0/4066008240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0105e10 0x7f0dd010c6a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.267+0000 7f0dd6a00640 1 --2- 172.21.15.69:0/4066008240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd0105450 0x7f0dd01058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.267+0000 7f0dd6a00640 1 --2- 172.21.15.69:0/4066008240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd0104260 0x7f0dd0104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.267+0000 7f0dd6a00640 1 -- 172.21.15.69:0/4066008240 >> 172.21.15.69:0/4066008240 conn(0x7f0dd00ff7b0 msgr2=0x7f0dd0101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:12.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.267+0000 7f0dd6a00640 1 -- 172.21.15.69:0/4066008240 shutdown_connections 2024-04-24T09:06:12.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.267+0000 7f0dd6a00640 1 -- 172.21.15.69:0/4066008240 wait complete. 2024-04-24T09:06:12.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.268+0000 7f0dd6a00640 1 Processor -- start 2024-04-24T09:06:12.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.268+0000 7f0dd6a00640 1 -- start start 2024-04-24T09:06:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.268+0000 7f0dd6a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd0104260 0x7f0dd019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.268+0000 7f0dd6a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd0105450 0x7f0dd019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.268+0000 7f0dd6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0105e10 0x7f0dd01a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.268+0000 7f0dd6a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd010fdf0 con 0x7f0dd0105e10 2024-04-24T09:06:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.268+0000 7f0dd6a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd010fc70 con 0x7f0dd0104260 2024-04-24T09:06:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.268+0000 7f0dd6a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd010ff70 con 0x7f0dd0105450 2024-04-24T09:06:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.269+0000 7f0dd4a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0105e10 0x7f0dd01a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.269+0000 7f0dd4a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0105e10 0x7f0dd01a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35396/0 (socket says 172.21.15.69:35396) 2024-04-24T09:06:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.269+0000 7f0dd4a00640 1 -- 172.21.15.69:0/1978265044 learned_addr learned my addr 172.21.15.69:0/1978265044 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.269+0000 7f0dcf400640 1 --2- 172.21.15.69:0/1978265044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd0105450 0x7f0dd019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.269+0000 7f0dcfe00640 1 --2- 172.21.15.69:0/1978265044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd0104260 0x7f0dd019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.269+0000 7f0dd4a00640 1 -- 172.21.15.69:0/1978265044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd0105450 msgr2=0x7f0dd019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.269+0000 7f0dd4a00640 1 --2- 172.21.15.69:0/1978265044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd0105450 0x7f0dd019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.269+0000 7f0dd4a00640 1 -- 172.21.15.69:0/1978265044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd0104260 msgr2=0x7f0dd019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:12.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.269+0000 7f0dd4a00640 1 --2- 172.21.15.69:0/1978265044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd0104260 0x7f0dd019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.269+0000 7f0dd4a00640 1 -- 172.21.15.69:0/1978265044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd01a2040 con 0x7f0dd0105e10 2024-04-24T09:06:12.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.269+0000 7f0dcfe00640 1 --2- 172.21.15.69:0/1978265044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd0104260 0x7f0dd019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:12.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.269+0000 7f0dd4a00640 1 --2- 172.21.15.69:0/1978265044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0105e10 0x7f0dd01a19d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8031040 tx=0x7f0dc8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:12.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.270+0000 7f0dccc00640 1 -- 172.21.15.69:0/1978265044 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8032ec0 con 0x7f0dd0105e10 2024-04-24T09:06:12.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.270+0000 7f0dccc00640 1 -- 172.21.15.69:0/1978265044 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dc80044f0 con 0x7f0dd0105e10 2024-04-24T09:06:12.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.270+0000 7f0dccc00640 1 -- 172.21.15.69:0/1978265044 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8030d10 con 0x7f0dd0105e10 2024-04-24T09:06:12.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.270+0000 7f0dd6a00640 1 -- 172.21.15.69:0/1978265044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd01a22d0 con 0x7f0dd0105e10 2024-04-24T09:06:12.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.270+0000 7f0dd6a00640 1 -- 172.21.15.69:0/1978265044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd01a2820 con 0x7f0dd0105e10 2024-04-24T09:06:12.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.271+0000 7f0dd6a00640 1 -- 172.21.15.69:0/1978265044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd010b580 con 0x7f0dd0105e10 2024-04-24T09:06:12.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.273+0000 7f0dccc00640 1 -- 172.21.15.69:0/1978265044 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0dc8004040 con 0x7f0dd0105e10 2024-04-24T09:06:12.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.275+0000 7f0dccc00640 1 --2- 172.21.15.69:0/1978265044 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0db40a3d50 0x7f0db40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:12.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.275+0000 7f0dccc00640 1 -- 172.21.15.69:0/1978265044 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0dc80f1ec0 con 0x7f0dd0105e10 2024-04-24T09:06:12.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.275+0000 7f0dcfe00640 1 --2- 172.21.15.69:0/1978265044 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0db40a3d50 0x7f0db40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:12.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.276+0000 7f0dcfe00640 1 --2- 172.21.15.69:0/1978265044 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0db40a3d50 0x7f0db40a6210 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0dd01052b0 tx=0x7f0dc0005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:12.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.279+0000 7f0dccc00640 1 -- 172.21.15.69:0/1978265044 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0dc80be4e0 con 0x7f0dd0105e10 2024-04-24T09:06:12.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.482+0000 7f0dd6a00640 1 -- 172.21.15.69:0/1978265044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0dd0062c00 con 0x7f0dd0105e10 2024-04-24T09:06:12.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.483+0000 7f0dccc00640 1 -- 172.21.15.69:0/1978265044 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f0dc8039e10 con 0x7f0dd0105e10 2024-04-24T09:06:12.483 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:06:12.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.485+0000 7f0dd6a00640 1 -- 172.21.15.69:0/1978265044 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0db40a3d50 msgr2=0x7f0db40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:12.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.486+0000 7f0dd6a00640 1 --2- 172.21.15.69:0/1978265044 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0db40a3d50 0x7f0db40a6210 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0dd01052b0 tx=0x7f0dc0005e70 comp rx=0 tx=0).stop 2024-04-24T09:06:12.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.486+0000 7f0dd6a00640 1 -- 172.21.15.69:0/1978265044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0105e10 msgr2=0x7f0dd01a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:12.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.486+0000 7f0dd6a00640 1 --2- 172.21.15.69:0/1978265044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0105e10 0x7f0dd01a19d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8031040 tx=0x7f0dc8003ba0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.486+0000 7f0dd6a00640 1 -- 172.21.15.69:0/1978265044 shutdown_connections 2024-04-24T09:06:12.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.486+0000 7f0dd6a00640 1 --2- 172.21.15.69:0/1978265044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0105e10 0x7f0dd01a19d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.486+0000 7f0dd6a00640 1 --2- 172.21.15.69:0/1978265044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd0105450 0x7f0dd019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.486+0000 7f0dd6a00640 1 --2- 172.21.15.69:0/1978265044 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0db40a3d50 0x7f0db40a6210 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.486+0000 7f0dd6a00640 1 --2- 172.21.15.69:0/1978265044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd0104260 0x7f0dd019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.486+0000 7f0dd6a00640 1 -- 172.21.15.69:0/1978265044 >> 172.21.15.69:0/1978265044 conn(0x7f0dd00ff7b0 msgr2=0x7f0dd0108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:12.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.486+0000 7f0dd6a00640 1 -- 172.21.15.69:0/1978265044 shutdown_connections 2024-04-24T09:06:12.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.486+0000 7f0dd6a00640 1 -- 172.21.15.69:0/1978265044 wait complete. 2024-04-24T09:06:12.505 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:06:12.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.701+0000 7fc7f4600640 1 -- 172.21.15.69:0/4109766642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ec104280 msgr2=0x7fc7ec1046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:12.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.701+0000 7fc7f4600640 1 --2- 172.21.15.69:0/4109766642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ec104280 0x7fc7ec1046a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009ca0 tx=0x7fc7e402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.701+0000 7fc7f4600640 1 -- 172.21.15.69:0/4109766642 shutdown_connections 2024-04-24T09:06:12.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.701+0000 7fc7f4600640 1 --2- 172.21.15.69:0/4109766642 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7ec105e30 0x7fc7ec10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.701+0000 7fc7f4600640 1 --2- 172.21.15.69:0/4109766642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7ec105470 0x7fc7ec1058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.701+0000 7fc7f4600640 1 --2- 172.21.15.69:0/4109766642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ec104280 0x7fc7ec1046a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.701+0000 7fc7f4600640 1 -- 172.21.15.69:0/4109766642 >> 172.21.15.69:0/4109766642 conn(0x7fc7ec0ff7f0 msgr2=0x7fc7ec101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:12.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.701+0000 7fc7f4600640 1 -- 172.21.15.69:0/4109766642 shutdown_connections 2024-04-24T09:06:12.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.701+0000 7fc7f4600640 1 -- 172.21.15.69:0/4109766642 wait complete. 2024-04-24T09:06:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.702+0000 7fc7f4600640 1 Processor -- start 2024-04-24T09:06:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.702+0000 7fc7f4600640 1 -- start start 2024-04-24T09:06:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f4600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7ec104280 0x7fc7ec19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f4600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ec105470 0x7fc7ec19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f4600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7ec105e30 0x7fc7ec1a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f4600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec10fdc0 con 0x7fc7ec105470 2024-04-24T09:06:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f4600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec10fc40 con 0x7fc7ec105e30 2024-04-24T09:06:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f4600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec10ff40 con 0x7fc7ec104280 2024-04-24T09:06:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f1200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ec105470 0x7fc7ec19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f1200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ec105470 0x7fc7ec19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35426/0 (socket says 172.21.15.69:35426) 2024-04-24T09:06:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f1200640 1 -- 172.21.15.69:0/3404029202 learned_addr learned my addr 172.21.15.69:0/3404029202 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f1c00640 1 --2- 172.21.15.69:0/3404029202 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7ec104280 0x7fc7ec19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f2600640 1 --2- 172.21.15.69:0/3404029202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7ec105e30 0x7fc7ec1a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f1200640 1 -- 172.21.15.69:0/3404029202 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7ec104280 msgr2=0x7fc7ec19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f1200640 1 --2- 172.21.15.69:0/3404029202 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7ec104280 0x7fc7ec19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f1200640 1 -- 172.21.15.69:0/3404029202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7ec105e30 msgr2=0x7fc7ec1a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f1200640 1 --2- 172.21.15.69:0/3404029202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7ec105e30 0x7fc7ec1a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f1200640 1 -- 172.21.15.69:0/3404029202 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ec1a20e0 con 0x7fc7ec105470 2024-04-24T09:06:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f2600640 1 --2- 172.21.15.69:0/3404029202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7ec105e30 0x7fc7ec1a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.703+0000 7fc7f1c00640 1 --2- 172.21.15.69:0/3404029202 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7ec104280 0x7fc7ec19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.704+0000 7fc7f1200640 1 --2- 172.21.15.69:0/3404029202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ec105470 0x7fc7ec19aa00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800d950 tx=0x7fc7d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.704+0000 7fc7ea000640 1 -- 172.21.15.69:0/3404029202 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8017030 con 0x7fc7ec105470 2024-04-24T09:06:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.704+0000 7fc7f4600640 1 -- 172.21.15.69:0/3404029202 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ec1a23d0 con 0x7fc7ec105470 2024-04-24T09:06:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.704+0000 7fc7ea000640 1 -- 172.21.15.69:0/3404029202 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7d8004020 con 0x7fc7ec105470 2024-04-24T09:06:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.704+0000 7fc7ea000640 1 -- 172.21.15.69:0/3404029202 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80047a0 con 0x7fc7ec105470 2024-04-24T09:06:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.704+0000 7fc7f4600640 1 -- 172.21.15.69:0/3404029202 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ec1a2970 con 0x7fc7ec105470 2024-04-24T09:06:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.705+0000 7fc7f4600640 1 -- 172.21.15.69:0/3404029202 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ec10b670 con 0x7fc7ec105470 2024-04-24T09:06:12.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.708+0000 7fc7ea000640 1 -- 172.21.15.69:0/3404029202 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc7d80041d0 con 0x7fc7ec105470 2024-04-24T09:06:12.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.709+0000 7fc7ea000640 1 --2- 172.21.15.69:0/3404029202 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc7d00a3c30 0x7fc7d00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:12.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.709+0000 7fc7ea000640 1 -- 172.21.15.69:0/3404029202 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc7d80d0330 con 0x7fc7ec105470 2024-04-24T09:06:12.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.710+0000 7fc7f1c00640 1 --2- 172.21.15.69:0/3404029202 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc7d00a3c30 0x7fc7d00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:12.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.710+0000 7fc7f1c00640 1 --2- 172.21.15.69:0/3404029202 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc7d00a3c30 0x7fc7d00a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4006010 tx=0x7fc7e4009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:12.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.714+0000 7fc7ea000640 1 -- 172.21.15.69:0/3404029202 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7d809c950 con 0x7fc7ec105470 2024-04-24T09:06:12.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:12 smithi081 ceph-mon[30961]: pgmap v436: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:06:12.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:12 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1627604324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:12.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:12 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/732882729' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:06:12.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:12 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1978265044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:06:12.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:12 smithi069 ceph-mon[28170]: pgmap v436: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:06:12.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:12 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1627604324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:12.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:12 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/732882729' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:06:12.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:12 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1978265044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:06:12.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.914+0000 7fc7f4600640 1 -- 172.21.15.69:0/3404029202 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc7ec062c30 con 0x7fc7ec105470 2024-04-24T09:06:12.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.915+0000 7fc7ea000640 1 -- 172.21.15.69:0/3404029202 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7fc7d809c250 con 0x7fc7ec105470 2024-04-24T09:06:12.915 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-24T09:06:12.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.918+0000 7fc7f4600640 1 -- 172.21.15.69:0/3404029202 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc7d00a3c30 msgr2=0x7fc7d00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:12.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.918+0000 7fc7f4600640 1 --2- 172.21.15.69:0/3404029202 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc7d00a3c30 0x7fc7d00a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4006010 tx=0x7fc7e4009360 comp rx=0 tx=0).stop 2024-04-24T09:06:12.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.918+0000 7fc7f4600640 1 -- 172.21.15.69:0/3404029202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ec105470 msgr2=0x7fc7ec19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:12.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.918+0000 7fc7f4600640 1 --2- 172.21.15.69:0/3404029202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ec105470 0x7fc7ec19aa00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800d950 tx=0x7fc7d800de10 comp rx=0 tx=0).stop 2024-04-24T09:06:12.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.918+0000 7fc7f4600640 1 -- 172.21.15.69:0/3404029202 shutdown_connections 2024-04-24T09:06:12.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.918+0000 7fc7f4600640 1 --2- 172.21.15.69:0/3404029202 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc7d00a3c30 0x7fc7d00a60f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.918+0000 7fc7f4600640 1 --2- 172.21.15.69:0/3404029202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7ec105e30 0x7fc7ec1a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.918+0000 7fc7f4600640 1 --2- 172.21.15.69:0/3404029202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ec105470 0x7fc7ec19aa00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.918+0000 7fc7f4600640 1 --2- 172.21.15.69:0/3404029202 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7ec104280 0x7fc7ec19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:12.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.918+0000 7fc7f4600640 1 -- 172.21.15.69:0/3404029202 >> 172.21.15.69:0/3404029202 conn(0x7fc7ec0ff7f0 msgr2=0x7fc7ec108840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:12.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.918+0000 7fc7f4600640 1 -- 172.21.15.69:0/3404029202 shutdown_connections 2024-04-24T09:06:12.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:12.918+0000 7fc7f4600640 1 -- 172.21.15.69:0/3404029202 wait complete. 2024-04-24T09:06:12.937 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T09:06:12.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:12 smithi130 ceph-mon[31553]: pgmap v436: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:06:12.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:12 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1627604324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:12.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:12 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/732882729' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:06:12.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:12 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1978265044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:06:13.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.142+0000 7fa40b000640 1 -- 172.21.15.69:0/3581336288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4040785a0 msgr2=0x7fa404078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:13.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.142+0000 7fa40b000640 1 --2- 172.21.15.69:0/3581336288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4040785a0 0x7fa404078e40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009ca0 tx=0x7fa3fc02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.142+0000 7fa40b000640 1 -- 172.21.15.69:0/3581336288 shutdown_connections 2024-04-24T09:06:13.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.143+0000 7fa40b000640 1 --2- 172.21.15.69:0/3581336288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4040785a0 0x7fa404078e40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.143+0000 7fa40b000640 1 --2- 172.21.15.69:0/3581336288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa404077c00 0x7fa404078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.143+0000 7fa40b000640 1 --2- 172.21.15.69:0/3581336288 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4040769c0 0x7fa404076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.143+0000 7fa40b000640 1 -- 172.21.15.69:0/3581336288 >> 172.21.15.69:0/3581336288 conn(0x7fa4040ff6b0 msgr2=0x7fa404101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:13.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.143+0000 7fa40b000640 1 -- 172.21.15.69:0/3581336288 shutdown_connections 2024-04-24T09:06:13.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.143+0000 7fa40b000640 1 -- 172.21.15.69:0/3581336288 wait complete. 2024-04-24T09:06:13.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.144+0000 7fa40b000640 1 Processor -- start 2024-04-24T09:06:13.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.144+0000 7fa40b000640 1 -- start start 2024-04-24T09:06:13.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.144+0000 7fa40b000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4040769c0 0x7fa40419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:13.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.144+0000 7fa40b000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa404077c00 0x7fa40419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:13.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.144+0000 7fa40b000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4040785a0 0x7fa4041a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:13.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.144+0000 7fa40b000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa4041140c0 con 0x7fa4040769c0 2024-04-24T09:06:13.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.144+0000 7fa40b000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa404113f40 con 0x7fa404077c00 2024-04-24T09:06:13.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.144+0000 7fa40b000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa404114240 con 0x7fa4040785a0 2024-04-24T09:06:13.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.144+0000 7fa409c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4040769c0 0x7fa40419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:13.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.144+0000 7fa409c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4040769c0 0x7fa40419e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39880/0 (socket says 172.21.15.69:39880) 2024-04-24T09:06:13.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.144+0000 7fa409c00640 1 -- 172.21.15.69:0/1108506031 learned_addr learned my addr 172.21.15.69:0/1108506031 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:13.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.145+0000 7fa40a600640 1 --2- 172.21.15.69:0/1108506031 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4040785a0 0x7fa4041a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:13.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.145+0000 7fa409200640 1 --2- 172.21.15.69:0/1108506031 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa404077c00 0x7fa40419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:13.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.145+0000 7fa409c00640 1 -- 172.21.15.69:0/1108506031 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4040785a0 msgr2=0x7fa4041a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:13.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.145+0000 7fa409c00640 1 --2- 172.21.15.69:0/1108506031 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4040785a0 0x7fa4041a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.145+0000 7fa409c00640 1 -- 172.21.15.69:0/1108506031 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa404077c00 msgr2=0x7fa40419ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:13.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.145+0000 7fa409c00640 1 --2- 172.21.15.69:0/1108506031 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa404077c00 0x7fa40419ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.145+0000 7fa409c00640 1 -- 172.21.15.69:0/1108506031 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4041a62d0 con 0x7fa4040769c0 2024-04-24T09:06:13.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.145+0000 7fa409200640 1 --2- 172.21.15.69:0/1108506031 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa404077c00 0x7fa40419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:13.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.145+0000 7fa409c00640 1 --2- 172.21.15.69:0/1108506031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4040769c0 0x7fa40419e6b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800d770 tx=0x7fa3f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:13.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.145+0000 7fa402000640 1 -- 172.21.15.69:0/1108506031 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8017070 con 0x7fa4040769c0 2024-04-24T09:06:13.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.146+0000 7fa40b000640 1 -- 172.21.15.69:0/1108506031 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4041a65c0 con 0x7fa4040769c0 2024-04-24T09:06:13.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.146+0000 7fa402000640 1 -- 172.21.15.69:0/1108506031 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3f80054d0 con 0x7fa4040769c0 2024-04-24T09:06:13.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.146+0000 7fa402000640 1 -- 172.21.15.69:0/1108506031 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8020060 con 0x7fa4040769c0 2024-04-24T09:06:13.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.146+0000 7fa40b000640 1 -- 172.21.15.69:0/1108506031 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4041a6ac0 con 0x7fa4040769c0 2024-04-24T09:06:13.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.147+0000 7fa40b000640 1 -- 172.21.15.69:0/1108506031 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa404110bf0 con 0x7fa4040769c0 2024-04-24T09:06:13.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.149+0000 7fa402000640 1 -- 172.21.15.69:0/1108506031 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa3f8004c90 con 0x7fa4040769c0 2024-04-24T09:06:13.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.151+0000 7fa402000640 1 --2- 172.21.15.69:0/1108506031 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3ec0a3bb0 0x7fa3ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:13.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.151+0000 7fa402000640 1 -- 172.21.15.69:0/1108506031 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa3f80d0310 con 0x7fa4040769c0 2024-04-24T09:06:13.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.151+0000 7fa409200640 1 --2- 172.21.15.69:0/1108506031 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3ec0a3bb0 0x7fa3ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:13.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.152+0000 7fa409200640 1 --2- 172.21.15.69:0/1108506031 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3ec0a3bb0 0x7fa3ec0a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa40419fbc0 tx=0x7fa3f0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:13.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.155+0000 7fa402000640 1 -- 172.21.15.69:0/1108506031 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3f809c930 con 0x7fa4040769c0 2024-04-24T09:06:13.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.361+0000 7fa40b000640 1 -- 172.21.15.69:0/1108506031 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa404062c00 con 0x7fa4040769c0 2024-04-24T09:06:13.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.362+0000 7fa402000640 1 -- 172.21.15.69:0/1108506031 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7fa3f809c230 con 0x7fa4040769c0 2024-04-24T09:06:13.362 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:06:13.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.365+0000 7fa40b000640 1 -- 172.21.15.69:0/1108506031 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3ec0a3bb0 msgr2=0x7fa3ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:13.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.365+0000 7fa40b000640 1 --2- 172.21.15.69:0/1108506031 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3ec0a3bb0 0x7fa3ec0a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa40419fbc0 tx=0x7fa3f0005e50 comp rx=0 tx=0).stop 2024-04-24T09:06:13.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.365+0000 7fa40b000640 1 -- 172.21.15.69:0/1108506031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4040769c0 msgr2=0x7fa40419e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:13.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.365+0000 7fa40b000640 1 --2- 172.21.15.69:0/1108506031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4040769c0 0x7fa40419e6b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800d770 tx=0x7fa3f800dc30 comp rx=0 tx=0).stop 2024-04-24T09:06:13.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.365+0000 7fa40b000640 1 -- 172.21.15.69:0/1108506031 shutdown_connections 2024-04-24T09:06:13.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.365+0000 7fa40b000640 1 --2- 172.21.15.69:0/1108506031 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4040785a0 0x7fa4041a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.365+0000 7fa40b000640 1 --2- 172.21.15.69:0/1108506031 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa3ec0a3bb0 0x7fa3ec0a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.365+0000 7fa40b000640 1 --2- 172.21.15.69:0/1108506031 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa404077c00 0x7fa40419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.365+0000 7fa40b000640 1 --2- 172.21.15.69:0/1108506031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4040769c0 0x7fa40419e6b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.365+0000 7fa40b000640 1 -- 172.21.15.69:0/1108506031 >> 172.21.15.69:0/1108506031 conn(0x7fa4040ff6b0 msgr2=0x7fa404100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:13.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.365+0000 7fa40b000640 1 -- 172.21.15.69:0/1108506031 shutdown_connections 2024-04-24T09:06:13.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.365+0000 7fa40b000640 1 -- 172.21.15.69:0/1108506031 wait complete. 2024-04-24T09:06:13.384 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:06:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.588+0000 7fa6db200640 1 -- 172.21.15.69:0/1645255375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6dc102fc0 msgr2=0x7fa6dc103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.588+0000 7fa6db200640 1 --2- 172.21.15.69:0/1645255375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6dc102fc0 0x7fa6dc103440 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009900 tx=0x7fa6c402a410 comp rx=0 tx=0).stop 2024-04-24T09:06:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.588+0000 7fa6db200640 1 -- 172.21.15.69:0/1645255375 shutdown_connections 2024-04-24T09:06:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.588+0000 7fa6db200640 1 --2- 172.21.15.69:0/1645255375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6dc103980 0x7fa6dc10a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.588+0000 7fa6db200640 1 --2- 172.21.15.69:0/1645255375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6dc102fc0 0x7fa6dc103440 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.588+0000 7fa6db200640 1 --2- 172.21.15.69:0/1645255375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6dc101e70 0x7fa6dc102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.588+0000 7fa6db200640 1 -- 172.21.15.69:0/1645255375 >> 172.21.15.69:0/1645255375 conn(0x7fa6dc0fd3e0 msgr2=0x7fa6dc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.588+0000 7fa6db200640 1 -- 172.21.15.69:0/1645255375 shutdown_connections 2024-04-24T09:06:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.588+0000 7fa6db200640 1 -- 172.21.15.69:0/1645255375 wait complete. 2024-04-24T09:06:13.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.590+0000 7fa6db200640 1 Processor -- start 2024-04-24T09:06:13.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.590+0000 7fa6db200640 1 -- start start 2024-04-24T09:06:13.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.590+0000 7fa6db200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6dc101e70 0x7fa6dc19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:13.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.590+0000 7fa6db200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6dc102fc0 0x7fa6dc19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:13.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.590+0000 7fa6db200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6dc103980 0x7fa6dc1a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:13.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.590+0000 7fa6db200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc10da10 con 0x7fa6dc103980 2024-04-24T09:06:13.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.590+0000 7fa6db200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc10d890 con 0x7fa6dc102fc0 2024-04-24T09:06:13.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.590+0000 7fa6db200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc10db90 con 0x7fa6dc101e70 2024-04-24T09:06:13.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.591+0000 7fa6da800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6dc103980 0x7fa6dc1a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:13.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.591+0000 7fa6da800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6dc103980 0x7fa6dc1a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39910/0 (socket says 172.21.15.69:39910) 2024-04-24T09:06:13.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.591+0000 7fa6da800640 1 -- 172.21.15.69:0/303891930 learned_addr learned my addr 172.21.15.69:0/303891930 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:13.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.591+0000 7fa6d9e00640 1 --2- 172.21.15.69:0/303891930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6dc101e70 0x7fa6dc19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:13.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.591+0000 7fa6d3e00640 1 --2- 172.21.15.69:0/303891930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6dc102fc0 0x7fa6dc19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:13.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.591+0000 7fa6da800640 1 -- 172.21.15.69:0/303891930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6dc101e70 msgr2=0x7fa6dc19a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:13.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.591+0000 7fa6da800640 1 --2- 172.21.15.69:0/303891930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6dc101e70 0x7fa6dc19a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.591+0000 7fa6da800640 1 -- 172.21.15.69:0/303891930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6dc102fc0 msgr2=0x7fa6dc19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:13.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.591+0000 7fa6da800640 1 --2- 172.21.15.69:0/303891930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6dc102fc0 0x7fa6dc19a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.591+0000 7fa6da800640 1 -- 172.21.15.69:0/303891930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc1a1f80 con 0x7fa6dc103980 2024-04-24T09:06:13.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.591+0000 7fa6da800640 1 --2- 172.21.15.69:0/303891930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6dc103980 0x7fa6dc1a1910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400eaa0 tx=0x7fa6d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:13.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.592+0000 7fa6d2a00640 1 -- 172.21.15.69:0/303891930 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d400cdb0 con 0x7fa6dc103980 2024-04-24T09:06:13.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.592+0000 7fa6d2a00640 1 -- 172.21.15.69:0/303891930 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6d4005020 con 0x7fa6dc103980 2024-04-24T09:06:13.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.592+0000 7fa6d2a00640 1 -- 172.21.15.69:0/303891930 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d40116d0 con 0x7fa6dc103980 2024-04-24T09:06:13.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.592+0000 7fa6db200640 1 -- 172.21.15.69:0/303891930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6dc1a2270 con 0x7fa6dc103980 2024-04-24T09:06:13.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.592+0000 7fa6db200640 1 -- 172.21.15.69:0/303891930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6dc1a2790 con 0x7fa6dc103980 2024-04-24T09:06:13.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.593+0000 7fa6db200640 1 -- 172.21.15.69:0/303891930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6dc1a2a50 con 0x7fa6dc103980 2024-04-24T09:06:13.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.595+0000 7fa6d2a00640 1 -- 172.21.15.69:0/303891930 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa6d40051d0 con 0x7fa6dc103980 2024-04-24T09:06:13.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.597+0000 7fa6d2a00640 1 --2- 172.21.15.69:0/303891930 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa6c00a3c30 0x7fa6c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:13.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.598+0000 7fa6d2a00640 1 -- 172.21.15.69:0/303891930 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa6d40d09f0 con 0x7fa6dc103980 2024-04-24T09:06:13.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.598+0000 7fa6d9e00640 1 --2- 172.21.15.69:0/303891930 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa6c00a3c30 0x7fa6c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:13.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.598+0000 7fa6d9e00640 1 --2- 172.21.15.69:0/303891930 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa6c00a3c30 0x7fa6c00a60f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc005fd0 tx=0x7fa6cc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:13.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.602+0000 7fa6d2a00640 1 -- 172.21.15.69:0/303891930 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6d4016030 con 0x7fa6dc103980 2024-04-24T09:06:13.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:13 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3404029202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:06:13.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:13 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1108506031' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:06:13.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:13 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3404029202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:06:13.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:13 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1108506031' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:06:13.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.848+0000 7fa6db200640 1 -- 172.21.15.69:0/303891930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6dc1a2c20 con 0x7fa6dc103980 2024-04-24T09:06:13.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.849+0000 7fa6d2a00640 1 -- 172.21.15.69:0/303891930 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11873 (secure 0 0 0) 0x7fa6d409d010 con 0x7fa6dc103980 2024-04-24T09:06:13.849 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:13.850 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:04:01.991559+0000","modified":"2024-04-24T09:04:34.617681+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34532,"mds_1":34496,"mds_2":34460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34496":{"gid":34496,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34532":{"gid":34532,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34532,"qdb_cluster":[34460,34496,34532]},"id":1}]} 2024-04-24T09:06:13.850 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2024-04-24T09:06:13.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.852+0000 7fa6db200640 1 -- 172.21.15.69:0/303891930 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa6c00a3c30 msgr2=0x7fa6c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:13.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.852+0000 7fa6db200640 1 --2- 172.21.15.69:0/303891930 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa6c00a3c30 0x7fa6c00a60f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc005fd0 tx=0x7fa6cc005850 comp rx=0 tx=0).stop 2024-04-24T09:06:13.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.852+0000 7fa6db200640 1 -- 172.21.15.69:0/303891930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6dc103980 msgr2=0x7fa6dc1a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:13.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.852+0000 7fa6db200640 1 --2- 172.21.15.69:0/303891930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6dc103980 0x7fa6dc1a1910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400eaa0 tx=0x7fa6d400ef60 comp rx=0 tx=0).stop 2024-04-24T09:06:13.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.852+0000 7fa6db200640 1 -- 172.21.15.69:0/303891930 shutdown_connections 2024-04-24T09:06:13.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.852+0000 7fa6db200640 1 --2- 172.21.15.69:0/303891930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6dc103980 0x7fa6dc1a1910 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.852+0000 7fa6db200640 1 --2- 172.21.15.69:0/303891930 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa6c00a3c30 0x7fa6c00a60f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.852+0000 7fa6db200640 1 --2- 172.21.15.69:0/303891930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6dc102fc0 0x7fa6dc19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.852+0000 7fa6db200640 1 --2- 172.21.15.69:0/303891930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6dc101e70 0x7fa6dc19a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:13.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.852+0000 7fa6db200640 1 -- 172.21.15.69:0/303891930 >> 172.21.15.69:0/303891930 conn(0x7fa6dc0fd3e0 msgr2=0x7fa6dc074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:13.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.852+0000 7fa6db200640 1 -- 172.21.15.69:0/303891930 shutdown_connections 2024-04-24T09:06:13.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:13.852+0000 7fa6db200640 1 -- 172.21.15.69:0/303891930 wait complete. 2024-04-24T09:06:13.871 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:06:13.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:13 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3404029202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:06:13.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:13 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1108506031' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:06:14.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.075+0000 7f56fde00640 1 -- 172.21.15.69:0/4210872088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56f810a030 msgr2=0x7f56f81108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:14.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.075+0000 7f56fde00640 1 --2- 172.21.15.69:0/4210872088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56f810a030 0x7f56f81108c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009900 tx=0x7f56f002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.076+0000 7f56fde00640 1 -- 172.21.15.69:0/4210872088 shutdown_connections 2024-04-24T09:06:14.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.076+0000 7f56fde00640 1 --2- 172.21.15.69:0/4210872088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56f810a030 0x7f56f81108c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.076+0000 7f56fde00640 1 --2- 172.21.15.69:0/4210872088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56f8109670 0x7f56f8109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.076+0000 7f56fde00640 1 --2- 172.21.15.69:0/4210872088 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56f8108480 0x7f56f81088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.076+0000 7f56fde00640 1 -- 172.21.15.69:0/4210872088 >> 172.21.15.69:0/4210872088 conn(0x7f56f8077440 msgr2=0x7f56f8079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:14.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.076+0000 7f56fde00640 1 -- 172.21.15.69:0/4210872088 shutdown_connections 2024-04-24T09:06:14.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.076+0000 7f56fde00640 1 -- 172.21.15.69:0/4210872088 wait complete. 2024-04-24T09:06:14.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.077+0000 7f56fde00640 1 Processor -- start 2024-04-24T09:06:14.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.077+0000 7f56fde00640 1 -- start start 2024-04-24T09:06:14.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.077+0000 7f56fde00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56f8108480 0x7f56f819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:14.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.077+0000 7f56fde00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56f8109670 0x7f56f819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:14.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.077+0000 7f56fde00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56f810a030 0x7f56f81a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:14.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.077+0000 7f56fde00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f56f8114010 con 0x7f56f8108480 2024-04-24T09:06:14.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.077+0000 7f56fde00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f56f8113e90 con 0x7f56f8109670 2024-04-24T09:06:14.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.077+0000 7f56fde00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f56f8114190 con 0x7f56f810a030 2024-04-24T09:06:14.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.077+0000 7f56fca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56f8108480 0x7f56f819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:14.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.077+0000 7f56fca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56f8108480 0x7f56f819e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39930/0 (socket says 172.21.15.69:39930) 2024-04-24T09:06:14.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.077+0000 7f56fca00640 1 -- 172.21.15.69:0/1764095571 learned_addr learned my addr 172.21.15.69:0/1764095571 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:14.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.078+0000 7f56fd400640 1 --2- 172.21.15.69:0/1764095571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56f810a030 0x7f56f81a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:14.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.078+0000 7f56f7e00640 1 --2- 172.21.15.69:0/1764095571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56f8109670 0x7f56f819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:14.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.078+0000 7f56fca00640 1 -- 172.21.15.69:0/1764095571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56f810a030 msgr2=0x7f56f81a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:14.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.078+0000 7f56fca00640 1 --2- 172.21.15.69:0/1764095571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56f810a030 0x7f56f81a5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.078+0000 7f56fca00640 1 -- 172.21.15.69:0/1764095571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56f8109670 msgr2=0x7f56f819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:14.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.078+0000 7f56fca00640 1 --2- 172.21.15.69:0/1764095571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56f8109670 0x7f56f819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.078+0000 7f56fca00640 1 -- 172.21.15.69:0/1764095571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f81a6260 con 0x7f56f8108480 2024-04-24T09:06:14.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.078+0000 7f56f7e00640 1 --2- 172.21.15.69:0/1764095571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56f8109670 0x7f56f819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:14.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.078+0000 7f56fca00640 1 --2- 172.21.15.69:0/1764095571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56f8108480 0x7f56f819e640 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f56e800c970 tx=0x7f56e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:14.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.078+0000 7f56f5600640 1 -- 172.21.15.69:0/1764095571 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8004220 con 0x7f56f8108480 2024-04-24T09:06:14.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.078+0000 7f56f5600640 1 -- 172.21.15.69:0/1764095571 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56e80043c0 con 0x7f56f8108480 2024-04-24T09:06:14.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.078+0000 7f56f5600640 1 -- 172.21.15.69:0/1764095571 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8004a00 con 0x7f56f8108480 2024-04-24T09:06:14.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.078+0000 7f56fde00640 1 -- 172.21.15.69:0/1764095571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f81a6550 con 0x7f56f8108480 2024-04-24T09:06:14.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.079+0000 7f56fde00640 1 -- 172.21.15.69:0/1764095571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f81a6a50 con 0x7f56f8108480 2024-04-24T09:06:14.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.080+0000 7f56fde00640 1 -- 172.21.15.69:0/1764095571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f810f7a0 con 0x7f56f8108480 2024-04-24T09:06:14.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.082+0000 7f56f5600640 1 -- 172.21.15.69:0/1764095571 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f56e80079b0 con 0x7f56f8108480 2024-04-24T09:06:14.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.083+0000 7f56f5600640 1 --2- 172.21.15.69:0/1764095571 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56dc0a3bb0 0x7f56dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:14.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.083+0000 7f56f5600640 1 -- 172.21.15.69:0/1764095571 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f56e80d05d0 con 0x7f56f8108480 2024-04-24T09:06:14.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.083+0000 7f56f7e00640 1 --2- 172.21.15.69:0/1764095571 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56dc0a3bb0 0x7f56dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:14.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.084+0000 7f56f7e00640 1 --2- 172.21.15.69:0/1764095571 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56dc0a3bb0 0x7f56dc0a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f56f819fb50 tx=0x7f56e00082f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:14.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.087+0000 7f56f5600640 1 -- 172.21.15.69:0/1764095571 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56e8014030 con 0x7f56f8108480 2024-04-24T09:06:14.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.280+0000 7f56fde00640 1 -- 172.21.15.69:0/1764095571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56f8062c30 con 0x7f56f8108480 2024-04-24T09:06:14.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.282+0000 7f56f5600640 1 -- 172.21.15.69:0/1764095571 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24564 (secure 0 0 0) 0x7f56e809cbf0 con 0x7f56f8108480 2024-04-24T09:06:14.283 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:14.283 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":92,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:24.497245+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:06:14.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.285+0000 7f56fde00640 1 -- 172.21.15.69:0/1764095571 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56dc0a3bb0 msgr2=0x7f56dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:14.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.285+0000 7f56fde00640 1 --2- 172.21.15.69:0/1764095571 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56dc0a3bb0 0x7f56dc0a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f56f819fb50 tx=0x7f56e00082f0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.285+0000 7f56fde00640 1 -- 172.21.15.69:0/1764095571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56f8108480 msgr2=0x7f56f819e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:14.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.285+0000 7f56fde00640 1 --2- 172.21.15.69:0/1764095571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56f8108480 0x7f56f819e640 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f56e800c970 tx=0x7f56e800ce30 comp rx=0 tx=0).stop 2024-04-24T09:06:14.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.285+0000 7f56fde00640 1 -- 172.21.15.69:0/1764095571 shutdown_connections 2024-04-24T09:06:14.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.285+0000 7f56fde00640 1 --2- 172.21.15.69:0/1764095571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56f810a030 0x7f56f81a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.285+0000 7f56fde00640 1 --2- 172.21.15.69:0/1764095571 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f56dc0a3bb0 0x7f56dc0a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.285+0000 7f56fde00640 1 --2- 172.21.15.69:0/1764095571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56f8109670 0x7f56f819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.285+0000 7f56fde00640 1 --2- 172.21.15.69:0/1764095571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56f8108480 0x7f56f819e640 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.285+0000 7f56fde00640 1 -- 172.21.15.69:0/1764095571 >> 172.21.15.69:0/1764095571 conn(0x7f56f8077440 msgr2=0x7f56f810c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:14.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.285+0000 7f56fde00640 1 -- 172.21.15.69:0/1764095571 shutdown_connections 2024-04-24T09:06:14.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.285+0000 7f56fde00640 1 -- 172.21.15.69:0/1764095571 wait complete. 2024-04-24T09:06:14.304 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:57:37.377599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:04:00.299827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:04:00.958498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T09:04:04.105314+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T09:06:14.305 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:06:14.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.505+0000 7f3868a00640 1 -- 172.21.15.69:0/4071048802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3864108660 msgr2=0x7f3864108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.505+0000 7f3868a00640 1 --2- 172.21.15.69:0/4071048802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3864108660 0x7f3864108a80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3854009900 tx=0x7f385402a410 comp rx=0 tx=0).stop 2024-04-24T09:06:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.506+0000 7f3868a00640 1 -- 172.21.15.69:0/4071048802 shutdown_connections 2024-04-24T09:06:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.506+0000 7f3868a00640 1 --2- 172.21.15.69:0/4071048802 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f386410a210 0x7f3864110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.506+0000 7f3868a00640 1 --2- 172.21.15.69:0/4071048802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3864109850 0x7f3864109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.506+0000 7f3868a00640 1 --2- 172.21.15.69:0/4071048802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3864108660 0x7f3864108a80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.506+0000 7f3868a00640 1 -- 172.21.15.69:0/4071048802 >> 172.21.15.69:0/4071048802 conn(0x7f3864077600 msgr2=0x7f3864079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.506+0000 7f3868a00640 1 -- 172.21.15.69:0/4071048802 shutdown_connections 2024-04-24T09:06:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.506+0000 7f3868a00640 1 -- 172.21.15.69:0/4071048802 wait complete. 2024-04-24T09:06:14.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.507+0000 7f3868a00640 1 Processor -- start 2024-04-24T09:06:14.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.507+0000 7f3868a00640 1 -- start start 2024-04-24T09:06:14.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.507+0000 7f3868a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3864108660 0x7f386419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:14.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.507+0000 7f3868a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3864109850 0x7f386419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:14.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.507+0000 7f3868a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f386410a210 0x7f38641a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:14.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.507+0000 7f3868a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f38641141e0 con 0x7f3864108660 2024-04-24T09:06:14.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.507+0000 7f3868a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3864114060 con 0x7f3864109850 2024-04-24T09:06:14.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.507+0000 7f3868a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3864114360 con 0x7f386410a210 2024-04-24T09:06:14.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.507+0000 7f3861e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3864108660 0x7f386419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:14.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.507+0000 7f3861e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3864108660 0x7f386419e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39946/0 (socket says 172.21.15.69:39946) 2024-04-24T09:06:14.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.507+0000 7f3861e00640 1 -- 172.21.15.69:0/2472228714 learned_addr learned my addr 172.21.15.69:0/2472228714 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:14.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.508+0000 7f3862800640 1 --2- 172.21.15.69:0/2472228714 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f386410a210 0x7f38641a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:14.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.508+0000 7f3861400640 1 --2- 172.21.15.69:0/2472228714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3864109850 0x7f386419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:14.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.508+0000 7f3861e00640 1 -- 172.21.15.69:0/2472228714 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f386410a210 msgr2=0x7f38641a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:14.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.508+0000 7f3861e00640 1 --2- 172.21.15.69:0/2472228714 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f386410a210 0x7f38641a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.508+0000 7f3861e00640 1 -- 172.21.15.69:0/2472228714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3864109850 msgr2=0x7f386419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:14.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.508+0000 7f3861e00640 1 --2- 172.21.15.69:0/2472228714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3864109850 0x7f386419ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.508+0000 7f3861e00640 1 -- 172.21.15.69:0/2472228714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38641a6500 con 0x7f3864108660 2024-04-24T09:06:14.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.508+0000 7f3862800640 1 --2- 172.21.15.69:0/2472228714 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f386410a210 0x7f38641a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:14.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.508+0000 7f3861e00640 1 --2- 172.21.15.69:0/2472228714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3864108660 0x7f386419e8e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f38540314e0 tx=0x7f3854003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:14.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.508+0000 7f385aa00640 1 -- 172.21.15.69:0/2472228714 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385403f070 con 0x7f3864108660 2024-04-24T09:06:14.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.508+0000 7f385aa00640 1 -- 172.21.15.69:0/2472228714 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3854031e40 con 0x7f3864108660 2024-04-24T09:06:14.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.508+0000 7f3868a00640 1 -- 172.21.15.69:0/2472228714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38641a6790 con 0x7f3864108660 2024-04-24T09:06:14.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.508+0000 7f385aa00640 1 -- 172.21.15.69:0/2472228714 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854035030 con 0x7f3864108660 2024-04-24T09:06:14.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.509+0000 7f3868a00640 1 -- 172.21.15.69:0/2472228714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38641a6c80 con 0x7f3864108660 2024-04-24T09:06:14.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.510+0000 7f3868a00640 1 -- 172.21.15.69:0/2472228714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f386410fa50 con 0x7f3864108660 2024-04-24T09:06:14.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.512+0000 7f385aa00640 1 -- 172.21.15.69:0/2472228714 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3854042070 con 0x7f3864108660 2024-04-24T09:06:14.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.514+0000 7f385aa00640 1 --2- 172.21.15.69:0/2472228714 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f38480a3bb0 0x7f38480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:14.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.514+0000 7f385aa00640 1 -- 172.21.15.69:0/2472228714 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f38540f1110 con 0x7f3864108660 2024-04-24T09:06:14.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.514+0000 7f3861400640 1 --2- 172.21.15.69:0/2472228714 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f38480a3bb0 0x7f38480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:14.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.515+0000 7f3861400640 1 --2- 172.21.15.69:0/2472228714 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f38480a3bb0 0x7f38480a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3850005fd0 tx=0x7f3850005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:14.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.518+0000 7f385aa00640 1 -- 172.21.15.69:0/2472228714 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38540bd790 con 0x7f3864108660 2024-04-24T09:06:14.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:14 smithi069 ceph-mon[28170]: pgmap v437: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:06:14.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:14 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/303891930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:14.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:14 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1764095571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.708+0000 7f3868a00640 1 -- 172.21.15.69:0/2472228714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3864062c00 con 0x7f3864108660 2024-04-24T09:06:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.709+0000 7f385aa00640 1 -- 172.21.15.69:0/2472228714 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f38540bd090 con 0x7f3864108660 2024-04-24T09:06:14.708 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-24T09:06:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.711+0000 7f3868a00640 1 -- 172.21.15.69:0/2472228714 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f38480a3bb0 msgr2=0x7f38480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.711+0000 7f3868a00640 1 --2- 172.21.15.69:0/2472228714 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f38480a3bb0 0x7f38480a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3850005fd0 tx=0x7f3850005850 comp rx=0 tx=0).stop 2024-04-24T09:06:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.711+0000 7f3868a00640 1 -- 172.21.15.69:0/2472228714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3864108660 msgr2=0x7f386419e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.711+0000 7f3868a00640 1 --2- 172.21.15.69:0/2472228714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3864108660 0x7f386419e8e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f38540314e0 tx=0x7f3854003c80 comp rx=0 tx=0).stop 2024-04-24T09:06:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.711+0000 7f3868a00640 1 -- 172.21.15.69:0/2472228714 shutdown_connections 2024-04-24T09:06:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.711+0000 7f3868a00640 1 --2- 172.21.15.69:0/2472228714 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f386410a210 0x7f38641a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.711+0000 7f3868a00640 1 --2- 172.21.15.69:0/2472228714 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f38480a3bb0 0x7f38480a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.711+0000 7f3868a00640 1 --2- 172.21.15.69:0/2472228714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3864109850 0x7f386419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.711+0000 7f3868a00640 1 --2- 172.21.15.69:0/2472228714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3864108660 0x7f386419e8e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.711+0000 7f3868a00640 1 -- 172.21.15.69:0/2472228714 >> 172.21.15.69:0/2472228714 conn(0x7f3864077600 msgr2=0x7f386410cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:14.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.711+0000 7f3868a00640 1 -- 172.21.15.69:0/2472228714 shutdown_connections 2024-04-24T09:06:14.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.711+0000 7f3868a00640 1 -- 172.21.15.69:0/2472228714 wait complete. 2024-04-24T09:06:14.730 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:06:14.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:14 smithi081 ceph-mon[30961]: pgmap v437: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:06:14.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:14 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/303891930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:14.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:14 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1764095571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:14.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.935+0000 7fdaeba00640 1 -- 172.21.15.69:0/3485862472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdae4105470 msgr2=0x7fdae41058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:14.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.935+0000 7fdaeba00640 1 --2- 172.21.15.69:0/3485862472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdae4105470 0x7fdae41058f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009900 tx=0x7fdacc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.935+0000 7fdaeba00640 1 -- 172.21.15.69:0/3485862472 shutdown_connections 2024-04-24T09:06:14.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.935+0000 7fdaeba00640 1 --2- 172.21.15.69:0/3485862472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdae4105e30 0x7fdae410c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.935+0000 7fdaeba00640 1 --2- 172.21.15.69:0/3485862472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdae4105470 0x7fdae41058f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.935+0000 7fdaeba00640 1 --2- 172.21.15.69:0/3485862472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdae4104280 0x7fdae41046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.935+0000 7fdaeba00640 1 -- 172.21.15.69:0/3485862472 >> 172.21.15.69:0/3485862472 conn(0x7fdae40ff7f0 msgr2=0x7fdae4101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:14.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.935+0000 7fdaeba00640 1 -- 172.21.15.69:0/3485862472 shutdown_connections 2024-04-24T09:06:14.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.935+0000 7fdaeba00640 1 -- 172.21.15.69:0/3485862472 wait complete. 2024-04-24T09:06:14.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.936+0000 7fdaeba00640 1 Processor -- start 2024-04-24T09:06:14.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.936+0000 7fdaeba00640 1 -- start start 2024-04-24T09:06:14.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.937+0000 7fdaeba00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdae4104280 0x7fdae419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:14.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.937+0000 7fdaeba00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdae4105470 0x7fdae419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:14.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.937+0000 7fdaeba00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdae4105e30 0x7fdae41a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:14.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.937+0000 7fdaeba00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdae410fe50 con 0x7fdae4105e30 2024-04-24T09:06:14.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.937+0000 7fdaeba00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdae410fcd0 con 0x7fdae4105470 2024-04-24T09:06:14.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.937+0000 7fdaeba00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdae410ffd0 con 0x7fdae4104280 2024-04-24T09:06:14.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.937+0000 7fdae9a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdae4105e30 0x7fdae41a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:14.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.937+0000 7fdae9a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdae4105e30 0x7fdae41a1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39958/0 (socket says 172.21.15.69:39958) 2024-04-24T09:06:14.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.937+0000 7fdae9a00640 1 -- 172.21.15.69:0/4281862794 learned_addr learned my addr 172.21.15.69:0/4281862794 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:14.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.937+0000 7fdae9000640 1 --2- 172.21.15.69:0/4281862794 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdae4104280 0x7fdae419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:14.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.937+0000 7fdae3e00640 1 --2- 172.21.15.69:0/4281862794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdae4105470 0x7fdae419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:14.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.938+0000 7fdae9a00640 1 -- 172.21.15.69:0/4281862794 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdae4104280 msgr2=0x7fdae419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:14.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.938+0000 7fdae9a00640 1 --2- 172.21.15.69:0/4281862794 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdae4104280 0x7fdae419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.938+0000 7fdae9a00640 1 -- 172.21.15.69:0/4281862794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdae4105470 msgr2=0x7fdae419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:14.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.938+0000 7fdae9a00640 1 --2- 172.21.15.69:0/4281862794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdae4105470 0x7fdae419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:14.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.938+0000 7fdae9a00640 1 -- 172.21.15.69:0/4281862794 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae41a2110 con 0x7fdae4105e30 2024-04-24T09:06:14.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.938+0000 7fdae9a00640 1 --2- 172.21.15.69:0/4281862794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdae4105e30 0x7fdae41a1aa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00b970 tx=0x7fdadc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:14.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.938+0000 7fdae1600640 1 -- 172.21.15.69:0/4281862794 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc019070 con 0x7fdae4105e30 2024-04-24T09:06:14.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.938+0000 7fdae1600640 1 -- 172.21.15.69:0/4281862794 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdadc014440 con 0x7fdae4105e30 2024-04-24T09:06:14.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.938+0000 7fdaeba00640 1 -- 172.21.15.69:0/4281862794 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae41a2400 con 0x7fdae4105e30 2024-04-24T09:06:14.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.938+0000 7fdae1600640 1 -- 172.21.15.69:0/4281862794 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0075d0 con 0x7fdae4105e30 2024-04-24T09:06:14.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.939+0000 7fdaeba00640 1 -- 172.21.15.69:0/4281862794 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae41a29a0 con 0x7fdae4105e30 2024-04-24T09:06:14.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.940+0000 7fdaeba00640 1 -- 172.21.15.69:0/4281862794 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae410b620 con 0x7fdae4105e30 2024-04-24T09:06:14.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.942+0000 7fdae1600640 1 -- 172.21.15.69:0/4281862794 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdadc01c070 con 0x7fdae4105e30 2024-04-24T09:06:14.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.943+0000 7fdae1600640 1 --2- 172.21.15.69:0/4281862794 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdac80a3bb0 0x7fdac80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:14.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.943+0000 7fdae1600640 1 -- 172.21.15.69:0/4281862794 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdadc0cfc80 con 0x7fdae4105e30 2024-04-24T09:06:14.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.944+0000 7fdae9000640 1 --2- 172.21.15.69:0/4281862794 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdac80a3bb0 0x7fdac80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:14.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.944+0000 7fdae9000640 1 --2- 172.21.15.69:0/4281862794 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdac80a3bb0 0x7fdac80a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdae41052d0 tx=0x7fdad400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:14.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:14.948+0000 7fdae1600640 1 -- 172.21.15.69:0/4281862794 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdadc09bcb0 con 0x7fdae4105e30 2024-04-24T09:06:14.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:14 smithi130 ceph-mon[31553]: pgmap v437: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:06:14.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:14 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/303891930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:14.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:14 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1764095571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:15.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.150+0000 7fdaeba00640 1 -- 172.21.15.69:0/4281862794 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdae4062c00 con 0x7fdae4105e30 2024-04-24T09:06:15.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.151+0000 7fdae1600640 1 -- 172.21.15.69:0/4281862794 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fdadc09ffb0 con 0x7fdae4105e30 2024-04-24T09:06:15.151 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:06:15.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.153+0000 7fdaeba00640 1 -- 172.21.15.69:0/4281862794 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdac80a3bb0 msgr2=0x7fdac80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:15.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.153+0000 7fdaeba00640 1 --2- 172.21.15.69:0/4281862794 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdac80a3bb0 0x7fdac80a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdae41052d0 tx=0x7fdad400a2d0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.153+0000 7fdaeba00640 1 -- 172.21.15.69:0/4281862794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdae4105e30 msgr2=0x7fdae41a1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:15.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.153+0000 7fdaeba00640 1 --2- 172.21.15.69:0/4281862794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdae4105e30 0x7fdae41a1aa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00b970 tx=0x7fdadc00be30 comp rx=0 tx=0).stop 2024-04-24T09:06:15.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.153+0000 7fdaeba00640 1 -- 172.21.15.69:0/4281862794 shutdown_connections 2024-04-24T09:06:15.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.153+0000 7fdaeba00640 1 --2- 172.21.15.69:0/4281862794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdae4105e30 0x7fdae41a1aa0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.153+0000 7fdaeba00640 1 --2- 172.21.15.69:0/4281862794 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fdac80a3bb0 0x7fdac80a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.153+0000 7fdaeba00640 1 --2- 172.21.15.69:0/4281862794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdae4105470 0x7fdae419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.153+0000 7fdaeba00640 1 --2- 172.21.15.69:0/4281862794 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdae4104280 0x7fdae419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.153+0000 7fdaeba00640 1 -- 172.21.15.69:0/4281862794 >> 172.21.15.69:0/4281862794 conn(0x7fdae40ff7f0 msgr2=0x7fdae41087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:15.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.153+0000 7fdaeba00640 1 -- 172.21.15.69:0/4281862794 shutdown_connections 2024-04-24T09:06:15.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.154+0000 7fdaeba00640 1 -- 172.21.15.69:0/4281862794 wait complete. 2024-04-24T09:06:15.171 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:06:15.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.380+0000 7f0f5e400640 1 -- 172.21.15.69:0/2978797412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f5810a0f0 msgr2=0x7f0f58110980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:15.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.380+0000 7f0f5e400640 1 --2- 172.21.15.69:0/2978797412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f5810a0f0 0x7f0f58110980 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009ca0 tx=0x7f0f5002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.381+0000 7f0f5e400640 1 -- 172.21.15.69:0/2978797412 shutdown_connections 2024-04-24T09:06:15.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.381+0000 7f0f5e400640 1 --2- 172.21.15.69:0/2978797412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f5810a0f0 0x7f0f58110980 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.381+0000 7f0f5e400640 1 --2- 172.21.15.69:0/2978797412 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f58109730 0x7f0f58109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.381+0000 7f0f5e400640 1 --2- 172.21.15.69:0/2978797412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f58108540 0x7f0f58108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.381+0000 7f0f5e400640 1 -- 172.21.15.69:0/2978797412 >> 172.21.15.69:0/2978797412 conn(0x7f0f580774d0 msgr2=0x7f0f58079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:15.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.381+0000 7f0f5e400640 1 -- 172.21.15.69:0/2978797412 shutdown_connections 2024-04-24T09:06:15.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.381+0000 7f0f5e400640 1 -- 172.21.15.69:0/2978797412 wait complete. 2024-04-24T09:06:15.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.382+0000 7f0f5e400640 1 Processor -- start 2024-04-24T09:06:15.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.382+0000 7f0f5e400640 1 -- start start 2024-04-24T09:06:15.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.382+0000 7f0f5e400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f58108540 0x7f0f5819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:15.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.382+0000 7f0f5e400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f58109730 0x7f0f5819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:15.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.382+0000 7f0f5e400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f5810a0f0 0x7f0f581a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:15.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.382+0000 7f0f5e400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0f58114080 con 0x7f0f5810a0f0 2024-04-24T09:06:15.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.382+0000 7f0f5e400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0f58113f00 con 0x7f0f58109730 2024-04-24T09:06:15.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.382+0000 7f0f5e400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0f58114200 con 0x7f0f58108540 2024-04-24T09:06:15.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.382+0000 7f0f5da00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f5810a0f0 0x7f0f581a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:15.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.382+0000 7f0f5da00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f5810a0f0 0x7f0f581a5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39990/0 (socket says 172.21.15.69:39990) 2024-04-24T09:06:15.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.382+0000 7f0f5da00640 1 -- 172.21.15.69:0/3633409006 learned_addr learned my addr 172.21.15.69:0/3633409006 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:15.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.383+0000 7f0f5d000640 1 --2- 172.21.15.69:0/3633409006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f58108540 0x7f0f5819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:15.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.383+0000 7f0f57e00640 1 --2- 172.21.15.69:0/3633409006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f58109730 0x7f0f5819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:15.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.383+0000 7f0f5da00640 1 -- 172.21.15.69:0/3633409006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f58108540 msgr2=0x7f0f5819e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:15.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.383+0000 7f0f5da00640 1 --2- 172.21.15.69:0/3633409006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f58108540 0x7f0f5819e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.383+0000 7f0f5da00640 1 -- 172.21.15.69:0/3633409006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f58109730 msgr2=0x7f0f5819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:15.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.383+0000 7f0f5da00640 1 --2- 172.21.15.69:0/3633409006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f58109730 0x7f0f5819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.383+0000 7f0f5da00640 1 -- 172.21.15.69:0/3633409006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f581a62f0 con 0x7f0f5810a0f0 2024-04-24T09:06:15.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.383+0000 7f0f5da00640 1 --2- 172.21.15.69:0/3633409006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f5810a0f0 0x7f0f581a5c80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009dc0 tx=0x7f0f50003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:15.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.383+0000 7f0f55600640 1 -- 172.21.15.69:0/3633409006 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50032ec0 con 0x7f0f5810a0f0 2024-04-24T09:06:15.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.384+0000 7f0f5e400640 1 -- 172.21.15.69:0/3633409006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f581a6580 con 0x7f0f5810a0f0 2024-04-24T09:06:15.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.384+0000 7f0f55600640 1 -- 172.21.15.69:0/3633409006 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f500044f0 con 0x7f0f5810a0f0 2024-04-24T09:06:15.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.384+0000 7f0f55600640 1 -- 172.21.15.69:0/3633409006 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50030d40 con 0x7f0f5810a0f0 2024-04-24T09:06:15.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.384+0000 7f0f5e400640 1 -- 172.21.15.69:0/3633409006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f581a6b20 con 0x7f0f5810a0f0 2024-04-24T09:06:15.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.385+0000 7f0f5e400640 1 -- 172.21.15.69:0/3633409006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f5810f8e0 con 0x7f0f5810a0f0 2024-04-24T09:06:15.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.387+0000 7f0f55600640 1 -- 172.21.15.69:0/3633409006 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0f50004040 con 0x7f0f5810a0f0 2024-04-24T09:06:15.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.388+0000 7f0f55600640 1 --2- 172.21.15.69:0/3633409006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0f400a3d50 0x7f0f400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:15.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.388+0000 7f0f55600640 1 -- 172.21.15.69:0/3633409006 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0f500f1ea0 con 0x7f0f5810a0f0 2024-04-24T09:06:15.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.388+0000 7f0f5d000640 1 --2- 172.21.15.69:0/3633409006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0f400a3d50 0x7f0f400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:15.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.389+0000 7f0f5d000640 1 --2- 172.21.15.69:0/3633409006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0f400a3d50 0x7f0f400a6210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c005fd0 tx=0x7f0f4c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:15.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.392+0000 7f0f55600640 1 -- 172.21.15.69:0/3633409006 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f500be4c0 con 0x7f0f5810a0f0 2024-04-24T09:06:15.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.596+0000 7f0f5e400640 1 -- 172.21.15.69:0/3633409006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0f58062c00 con 0x7f0f5810a0f0 2024-04-24T09:06:15.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.597+0000 7f0f55600640 1 -- 172.21.15.69:0/3633409006 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f0f500bddc0 con 0x7f0f5810a0f0 2024-04-24T09:06:15.597 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-24T09:06:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.600+0000 7f0f5e400640 1 -- 172.21.15.69:0/3633409006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0f400a3d50 msgr2=0x7f0f400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.600+0000 7f0f5e400640 1 --2- 172.21.15.69:0/3633409006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0f400a3d50 0x7f0f400a6210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c005fd0 tx=0x7f0f4c005850 comp rx=0 tx=0).stop 2024-04-24T09:06:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.600+0000 7f0f5e400640 1 -- 172.21.15.69:0/3633409006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f5810a0f0 msgr2=0x7f0f581a5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.600+0000 7f0f5e400640 1 --2- 172.21.15.69:0/3633409006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f5810a0f0 0x7f0f581a5c80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009dc0 tx=0x7f0f50003ba0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.600+0000 7f0f5e400640 1 -- 172.21.15.69:0/3633409006 shutdown_connections 2024-04-24T09:06:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.600+0000 7f0f5e400640 1 --2- 172.21.15.69:0/3633409006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f5810a0f0 0x7f0f581a5c80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.600+0000 7f0f5e400640 1 --2- 172.21.15.69:0/3633409006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f0f400a3d50 0x7f0f400a6210 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.600+0000 7f0f5e400640 1 --2- 172.21.15.69:0/3633409006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f58109730 0x7f0f5819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.600+0000 7f0f5e400640 1 --2- 172.21.15.69:0/3633409006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f58108540 0x7f0f5819e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.600+0000 7f0f5e400640 1 -- 172.21.15.69:0/3633409006 >> 172.21.15.69:0/3633409006 conn(0x7f0f580774d0 msgr2=0x7f0f5810cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.600+0000 7f0f5e400640 1 -- 172.21.15.69:0/3633409006 shutdown_connections 2024-04-24T09:06:15.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.600+0000 7f0f5e400640 1 -- 172.21.15.69:0/3633409006 wait complete. 2024-04-24T09:06:15.619 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T09:06:15.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:15 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2472228714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:06:15.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:15 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/4281862794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:06:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.826+0000 7f3d38c00640 1 -- 172.21.15.69:0/2501703915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d34108540 msgr2=0x7f3d34108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.827+0000 7f3d38c00640 1 --2- 172.21.15.69:0/2501703915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d34108540 0x7f3d34108960 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009900 tx=0x7f3d2402a410 comp rx=0 tx=0).stop 2024-04-24T09:06:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.827+0000 7f3d38c00640 1 -- 172.21.15.69:0/2501703915 shutdown_connections 2024-04-24T09:06:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.827+0000 7f3d38c00640 1 --2- 172.21.15.69:0/2501703915 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d3410a0f0 0x7f3d34110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.827+0000 7f3d38c00640 1 --2- 172.21.15.69:0/2501703915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3d34109730 0x7f3d34109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.827+0000 7f3d38c00640 1 --2- 172.21.15.69:0/2501703915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d34108540 0x7f3d34108960 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.827+0000 7f3d38c00640 1 -- 172.21.15.69:0/2501703915 >> 172.21.15.69:0/2501703915 conn(0x7f3d340774d0 msgr2=0x7f3d34079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.827+0000 7f3d38c00640 1 -- 172.21.15.69:0/2501703915 shutdown_connections 2024-04-24T09:06:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.827+0000 7f3d38c00640 1 -- 172.21.15.69:0/2501703915 wait complete. 2024-04-24T09:06:15.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.828+0000 7f3d38c00640 1 Processor -- start 2024-04-24T09:06:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.828+0000 7f3d38c00640 1 -- start start 2024-04-24T09:06:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.828+0000 7f3d38c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d34108540 0x7f3d3419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.828+0000 7f3d38c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d34109730 0x7f3d3419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.828+0000 7f3d38c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3d3410a0f0 0x7f3d341a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.828+0000 7f3d38c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341140d0 con 0x7f3d34109730 2024-04-24T09:06:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.828+0000 7f3d38c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3d34113f50 con 0x7f3d3410a0f0 2024-04-24T09:06:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.828+0000 7f3d38c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3d34114250 con 0x7f3d34108540 2024-04-24T09:06:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.829+0000 7f3d2be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d34109730 0x7f3d3419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.829+0000 7f3d2be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d34109730 0x7f3d3419ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40006/0 (socket says 172.21.15.69:40006) 2024-04-24T09:06:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.829+0000 7f3d2be00640 1 -- 172.21.15.69:0/1344860881 learned_addr learned my addr 172.21.15.69:0/1344860881 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.829+0000 7f3d33400640 1 --2- 172.21.15.69:0/1344860881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d34108540 0x7f3d3419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.829+0000 7f3d33e00640 1 --2- 172.21.15.69:0/1344860881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3d3410a0f0 0x7f3d341a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.829+0000 7f3d2be00640 1 -- 172.21.15.69:0/1344860881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d34108540 msgr2=0x7f3d3419e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.829+0000 7f3d2be00640 1 --2- 172.21.15.69:0/1344860881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d34108540 0x7f3d3419e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.829+0000 7f3d2be00640 1 -- 172.21.15.69:0/1344860881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3d3410a0f0 msgr2=0x7f3d341a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.829+0000 7f3d2be00640 1 --2- 172.21.15.69:0/1344860881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3d3410a0f0 0x7f3d341a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.829+0000 7f3d2be00640 1 -- 172.21.15.69:0/1344860881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d341a6370 con 0x7f3d34109730 2024-04-24T09:06:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.829+0000 7f3d33e00640 1 --2- 172.21.15.69:0/1344860881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3d3410a0f0 0x7f3d341a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.829+0000 7f3d2be00640 1 --2- 172.21.15.69:0/1344860881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d34109730 0x7f3d3419ec90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00e9f0 tx=0x7f3d1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.830+0000 7f3d30c00640 1 -- 172.21.15.69:0/1344860881 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c00cd80 con 0x7f3d34109730 2024-04-24T09:06:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.830+0000 7f3d30c00640 1 -- 172.21.15.69:0/1344860881 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d1c0108e0 con 0x7f3d34109730 2024-04-24T09:06:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.830+0000 7f3d38c00640 1 -- 172.21.15.69:0/1344860881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d341a6660 con 0x7f3d34109730 2024-04-24T09:06:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.830+0000 7f3d30c00640 1 -- 172.21.15.69:0/1344860881 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c018920 con 0x7f3d34109730 2024-04-24T09:06:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.830+0000 7f3d38c00640 1 -- 172.21.15.69:0/1344860881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d341a6c00 con 0x7f3d34109730 2024-04-24T09:06:15.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.831+0000 7f3d38c00640 1 -- 172.21.15.69:0/1344860881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d3410f8e0 con 0x7f3d34109730 2024-04-24T09:06:15.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.833+0000 7f3d30c00640 1 -- 172.21.15.69:0/1344860881 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3d1c010a90 con 0x7f3d34109730 2024-04-24T09:06:15.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.834+0000 7f3d30c00640 1 --2- 172.21.15.69:0/1344860881 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3d180a3c30 0x7f3d180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:15.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.835+0000 7f3d30c00640 1 -- 172.21.15.69:0/1344860881 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3d1c0d0fe0 con 0x7f3d34109730 2024-04-24T09:06:15.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.835+0000 7f3d33400640 1 --2- 172.21.15.69:0/1344860881 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3d180a3c30 0x7f3d180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:15.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.835+0000 7f3d33400640 1 --2- 172.21.15.69:0/1344860881 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3d180a3c30 0x7f3d180a60f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3d24006010 tx=0x7f3d240314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:15.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:15.837+0000 7f3d30c00640 1 -- 172.21.15.69:0/1344860881 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d1c09d600 con 0x7f3d34109730 2024-04-24T09:06:15.840 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:15 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2472228714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:06:15.840 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:15 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/4281862794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:06:15.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:15 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2472228714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:06:15.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:15 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/4281862794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:06:16.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.037+0000 7f3d38c00640 1 -- 172.21.15.69:0/1344860881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3d3402d870 con 0x7f3d34109730 2024-04-24T09:06:16.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.038+0000 7f3d30c00640 1 -- 172.21.15.69:0/1344860881 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f3d1c09cf00 con 0x7f3d34109730 2024-04-24T09:06:16.037 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:06:16.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.040+0000 7f3d38c00640 1 -- 172.21.15.69:0/1344860881 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3d180a3c30 msgr2=0x7f3d180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:16.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.040+0000 7f3d38c00640 1 --2- 172.21.15.69:0/1344860881 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3d180a3c30 0x7f3d180a60f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3d24006010 tx=0x7f3d240314a0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.040+0000 7f3d38c00640 1 -- 172.21.15.69:0/1344860881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d34109730 msgr2=0x7f3d3419ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:16.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.040+0000 7f3d38c00640 1 --2- 172.21.15.69:0/1344860881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d34109730 0x7f3d3419ec90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00e9f0 tx=0x7f3d1c00eeb0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.040+0000 7f3d38c00640 1 -- 172.21.15.69:0/1344860881 shutdown_connections 2024-04-24T09:06:16.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.040+0000 7f3d38c00640 1 --2- 172.21.15.69:0/1344860881 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3d180a3c30 0x7f3d180a60f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.040+0000 7f3d38c00640 1 --2- 172.21.15.69:0/1344860881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3d3410a0f0 0x7f3d341a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.040+0000 7f3d38c00640 1 --2- 172.21.15.69:0/1344860881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d34109730 0x7f3d3419ec90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.040+0000 7f3d38c00640 1 --2- 172.21.15.69:0/1344860881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d34108540 0x7f3d3419e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.040+0000 7f3d38c00640 1 -- 172.21.15.69:0/1344860881 >> 172.21.15.69:0/1344860881 conn(0x7f3d340774d0 msgr2=0x7f3d3410cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:16.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.040+0000 7f3d38c00640 1 -- 172.21.15.69:0/1344860881 shutdown_connections 2024-04-24T09:06:16.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.041+0000 7f3d38c00640 1 -- 172.21.15.69:0/1344860881 wait complete. 2024-04-24T09:06:16.060 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:06:16.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.252+0000 7f3a90e00640 1 -- 172.21.15.69:0/472976281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a8c105c30 msgr2=0x7f3a8c10c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.252+0000 7f3a90e00640 1 --2- 172.21.15.69:0/472976281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a8c105c30 0x7f3a8c10c4c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009ca0 tx=0x7f3a8402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.252+0000 7f3a90e00640 1 -- 172.21.15.69:0/472976281 shutdown_connections 2024-04-24T09:06:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.252+0000 7f3a90e00640 1 --2- 172.21.15.69:0/472976281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a8c105c30 0x7f3a8c10c4c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.252+0000 7f3a90e00640 1 --2- 172.21.15.69:0/472976281 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a8c105270 0x7f3a8c1056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.252+0000 7f3a90e00640 1 --2- 172.21.15.69:0/472976281 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a8c104080 0x7f3a8c1044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.252+0000 7f3a90e00640 1 -- 172.21.15.69:0/472976281 >> 172.21.15.69:0/472976281 conn(0x7f3a8c0ff610 msgr2=0x7f3a8c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.252+0000 7f3a90e00640 1 -- 172.21.15.69:0/472976281 shutdown_connections 2024-04-24T09:06:16.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.253+0000 7f3a90e00640 1 -- 172.21.15.69:0/472976281 wait complete. 2024-04-24T09:06:16.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.253+0000 7f3a90e00640 1 Processor -- start 2024-04-24T09:06:16.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.254+0000 7f3a90e00640 1 -- start start 2024-04-24T09:06:16.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.254+0000 7f3a90e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a8c104080 0x7f3a8c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:16.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.254+0000 7f3a90e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a8c105270 0x7f3a8c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:16.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.254+0000 7f3a90e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a8c105c30 0x7f3a8c1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:16.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.254+0000 7f3a90e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c10fc10 con 0x7f3a8c105270 2024-04-24T09:06:16.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.254+0000 7f3a90e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c10fa90 con 0x7f3a8c105c30 2024-04-24T09:06:16.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.254+0000 7f3a90e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c10fd90 con 0x7f3a8c104080 2024-04-24T09:06:16.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.254+0000 7f3a83e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a8c105270 0x7f3a8c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:16.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.254+0000 7f3a83e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a8c105270 0x7f3a8c19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40022/0 (socket says 172.21.15.69:40022) 2024-04-24T09:06:16.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.254+0000 7f3a83e00640 1 -- 172.21.15.69:0/3617227750 learned_addr learned my addr 172.21.15.69:0/3617227750 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:16.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.255+0000 7f3a8be00640 1 --2- 172.21.15.69:0/3617227750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a8c105c30 0x7f3a8c1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:16.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.255+0000 7f3a8b400640 1 --2- 172.21.15.69:0/3617227750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a8c104080 0x7f3a8c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:16.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.255+0000 7f3a83e00640 1 -- 172.21.15.69:0/3617227750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a8c104080 msgr2=0x7f3a8c19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:16.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.255+0000 7f3a83e00640 1 --2- 172.21.15.69:0/3617227750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a8c104080 0x7f3a8c19a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.255+0000 7f3a83e00640 1 -- 172.21.15.69:0/3617227750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a8c105c30 msgr2=0x7f3a8c1a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:16.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.255+0000 7f3a83e00640 1 --2- 172.21.15.69:0/3617227750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a8c105c30 0x7f3a8c1a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.255+0000 7f3a83e00640 1 -- 172.21.15.69:0/3617227750 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c1a1e60 con 0x7f3a8c105270 2024-04-24T09:06:16.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.255+0000 7f3a8be00640 1 --2- 172.21.15.69:0/3617227750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a8c105c30 0x7f3a8c1a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:16.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.255+0000 7f3a83e00640 1 --2- 172.21.15.69:0/3617227750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a8c105270 0x7f3a8c19a780 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400d770 tx=0x7f3a7400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:16.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.255+0000 7f3a88c00640 1 -- 172.21.15.69:0/3617227750 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74017070 con 0x7f3a8c105270 2024-04-24T09:06:16.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.255+0000 7f3a88c00640 1 -- 172.21.15.69:0/3617227750 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a740054d0 con 0x7f3a8c105270 2024-04-24T09:06:16.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.255+0000 7f3a90e00640 1 -- 172.21.15.69:0/3617227750 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8c1a2150 con 0x7f3a8c105270 2024-04-24T09:06:16.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.255+0000 7f3a88c00640 1 -- 172.21.15.69:0/3617227750 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74004cf0 con 0x7f3a8c105270 2024-04-24T09:06:16.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.255+0000 7f3a90e00640 1 -- 172.21.15.69:0/3617227750 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8c1a2650 con 0x7f3a8c105270 2024-04-24T09:06:16.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.257+0000 7f3a90e00640 1 -- 172.21.15.69:0/3617227750 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a8c10b3a0 con 0x7f3a8c105270 2024-04-24T09:06:16.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.258+0000 7f3a88c00640 1 -- 172.21.15.69:0/3617227750 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3a74004e90 con 0x7f3a8c105270 2024-04-24T09:06:16.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.260+0000 7f3a88c00640 1 --2- 172.21.15.69:0/3617227750 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a700a3bb0 0x7f3a700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:16.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.260+0000 7f3a88c00640 1 -- 172.21.15.69:0/3617227750 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3a740d02e0 con 0x7f3a8c105270 2024-04-24T09:06:16.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.260+0000 7f3a8b400640 1 --2- 172.21.15.69:0/3617227750 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a700a3bb0 0x7f3a700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:16.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.261+0000 7f3a8b400640 1 --2- 172.21.15.69:0/3617227750 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a700a3bb0 0x7f3a700a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005fd0 tx=0x7f3a7c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:16.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.265+0000 7f3a88c00640 1 -- 172.21.15.69:0/3617227750 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a7409c900 con 0x7f3a8c105270 2024-04-24T09:06:16.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.506+0000 7f3a90e00640 1 -- 172.21.15.69:0/3617227750 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a8c062c30 con 0x7f3a8c105270 2024-04-24T09:06:16.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.508+0000 7f3a88c00640 1 -- 172.21.15.69:0/3617227750 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11873 (secure 0 0 0) 0x7f3a7409c200 con 0x7f3a8c105270 2024-04-24T09:06:16.508 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:16.508 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:04:01.991559+0000","modified":"2024-04-24T09:04:34.617681+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34532,"mds_1":34496,"mds_2":34460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34496":{"gid":34496,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34532":{"gid":34532,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34532,"qdb_cluster":[34460,34496,34532]},"id":1}]} 2024-04-24T09:06:16.509 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2024-04-24T09:06:16.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.511+0000 7f3a90e00640 1 -- 172.21.15.69:0/3617227750 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a700a3bb0 msgr2=0x7f3a700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:16.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.511+0000 7f3a90e00640 1 --2- 172.21.15.69:0/3617227750 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a700a3bb0 0x7f3a700a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005fd0 tx=0x7f3a7c005850 comp rx=0 tx=0).stop 2024-04-24T09:06:16.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.511+0000 7f3a90e00640 1 -- 172.21.15.69:0/3617227750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a8c105270 msgr2=0x7f3a8c19a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:16.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.511+0000 7f3a90e00640 1 --2- 172.21.15.69:0/3617227750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a8c105270 0x7f3a8c19a780 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400d770 tx=0x7f3a7400dc30 comp rx=0 tx=0).stop 2024-04-24T09:06:16.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.511+0000 7f3a90e00640 1 -- 172.21.15.69:0/3617227750 shutdown_connections 2024-04-24T09:06:16.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.511+0000 7f3a90e00640 1 --2- 172.21.15.69:0/3617227750 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f3a700a3bb0 0x7f3a700a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.511+0000 7f3a90e00640 1 --2- 172.21.15.69:0/3617227750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a8c105c30 0x7f3a8c1a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.511+0000 7f3a90e00640 1 --2- 172.21.15.69:0/3617227750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a8c105270 0x7f3a8c19a780 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.511+0000 7f3a90e00640 1 --2- 172.21.15.69:0/3617227750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3a8c104080 0x7f3a8c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.511+0000 7f3a90e00640 1 -- 172.21.15.69:0/3617227750 >> 172.21.15.69:0/3617227750 conn(0x7f3a8c0ff610 msgr2=0x7f3a8c108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:16.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.511+0000 7f3a90e00640 1 -- 172.21.15.69:0/3617227750 shutdown_connections 2024-04-24T09:06:16.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.511+0000 7f3a90e00640 1 -- 172.21.15.69:0/3617227750 wait complete. 2024-04-24T09:06:16.531 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:06:16.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.726+0000 7f40a1400640 1 -- 172.21.15.69:0/3431843783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f409c10a210 msgr2=0x7f409c110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:16.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.726+0000 7f40a1400640 1 --2- 172.21.15.69:0/3431843783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f409c10a210 0x7f409c110aa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4094009ca0 tx=0x7f409402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.726+0000 7f40a1400640 1 -- 172.21.15.69:0/3431843783 shutdown_connections 2024-04-24T09:06:16.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.726+0000 7f40a1400640 1 --2- 172.21.15.69:0/3431843783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f409c10a210 0x7f409c110aa0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.726+0000 7f40a1400640 1 --2- 172.21.15.69:0/3431843783 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f409c109850 0x7f409c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.726+0000 7f40a1400640 1 --2- 172.21.15.69:0/3431843783 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f409c108660 0x7f409c108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.726+0000 7f40a1400640 1 -- 172.21.15.69:0/3431843783 >> 172.21.15.69:0/3431843783 conn(0x7f409c077620 msgr2=0x7f409c079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:16.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.726+0000 7f40a1400640 1 -- 172.21.15.69:0/3431843783 shutdown_connections 2024-04-24T09:06:16.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.726+0000 7f40a1400640 1 -- 172.21.15.69:0/3431843783 wait complete. 2024-04-24T09:06:16.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.726+0000 7f40a1400640 1 Processor -- start 2024-04-24T09:06:16.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.727+0000 7f40a1400640 1 -- start start 2024-04-24T09:06:16.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.727+0000 7f40a1400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f409c108660 0x7f409c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:16.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.727+0000 7f40a1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f409c109850 0x7f409c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:16.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.727+0000 7f40a1400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f409c10a210 0x7f409c1a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:16.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.727+0000 7f40a1400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1141a0 con 0x7f409c109850 2024-04-24T09:06:16.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.727+0000 7f40a1400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f409c114020 con 0x7f409c108660 2024-04-24T09:06:16.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.727+0000 7f40a1400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f409c114320 con 0x7f409c10a210 2024-04-24T09:06:16.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.727+0000 7f4091e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f409c109850 0x7f409c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:16.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.727+0000 7f4091e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f409c109850 0x7f409c19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40040/0 (socket says 172.21.15.69:40040) 2024-04-24T09:06:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.727+0000 7f4091e00640 1 -- 172.21.15.69:0/2287237924 learned_addr learned my addr 172.21.15.69:0/2287237924 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.728+0000 7f409a800640 1 --2- 172.21.15.69:0/2287237924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f409c108660 0x7f409c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.728+0000 7f409b200640 1 --2- 172.21.15.69:0/2287237924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f409c10a210 0x7f409c1a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.728+0000 7f4091e00640 1 -- 172.21.15.69:0/2287237924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f409c10a210 msgr2=0x7f409c1a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.728+0000 7f4091e00640 1 --2- 172.21.15.69:0/2287237924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f409c10a210 0x7f409c1a5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.728+0000 7f4091e00640 1 -- 172.21.15.69:0/2287237924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f409c108660 msgr2=0x7f409c19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.728+0000 7f4091e00640 1 --2- 172.21.15.69:0/2287237924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f409c108660 0x7f409c19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.728+0000 7f4091e00640 1 -- 172.21.15.69:0/2287237924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c1a6470 con 0x7f409c109850 2024-04-24T09:06:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.728+0000 7f409a800640 1 --2- 172.21.15.69:0/2287237924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f409c108660 0x7f409c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.728+0000 7f4091e00640 1 --2- 172.21.15.69:0/2287237924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f409c109850 0x7f409c19ed90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f408400d770 tx=0x7f408400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.728+0000 7f4093e00640 1 -- 172.21.15.69:0/2287237924 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084017070 con 0x7f409c109850 2024-04-24T09:06:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.728+0000 7f4093e00640 1 -- 172.21.15.69:0/2287237924 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40840054d0 con 0x7f409c109850 2024-04-24T09:06:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.728+0000 7f40a1400640 1 -- 172.21.15.69:0/2287237924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409c1a6760 con 0x7f409c109850 2024-04-24T09:06:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.728+0000 7f4093e00640 1 -- 172.21.15.69:0/2287237924 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084004cf0 con 0x7f409c109850 2024-04-24T09:06:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.728+0000 7f40a1400640 1 -- 172.21.15.69:0/2287237924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409c1a6cb0 con 0x7f409c109850 2024-04-24T09:06:16.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.730+0000 7f40a1400640 1 -- 172.21.15.69:0/2287237924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f409c10f980 con 0x7f409c109850 2024-04-24T09:06:16.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.732+0000 7f4093e00640 1 -- 172.21.15.69:0/2287237924 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4084004e90 con 0x7f409c109850 2024-04-24T09:06:16.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.733+0000 7f4093e00640 1 --2- 172.21.15.69:0/2287237924 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f40800a3c30 0x7f40800a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:16.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.733+0000 7f4093e00640 1 -- 172.21.15.69:0/2287237924 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f40840d0200 con 0x7f409c109850 2024-04-24T09:06:16.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.734+0000 7f409a800640 1 --2- 172.21.15.69:0/2287237924 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f40800a3c30 0x7f40800a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:16.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.734+0000 7f409a800640 1 --2- 172.21.15.69:0/2287237924 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f40800a3c30 0x7f40800a60f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f409c1096b0 tx=0x7f408c005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:16.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.737+0000 7f4093e00640 1 -- 172.21.15.69:0/2287237924 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f408409c880 con 0x7f409c109850 2024-04-24T09:06:16.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:16 smithi081 ceph-mon[30961]: pgmap v438: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:16.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:16 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3633409006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:06:16.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:16 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1344860881' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:06:16.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:16 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3617227750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:16.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:16 smithi069 ceph-mon[28170]: pgmap v438: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:16.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:16 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3633409006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:06:16.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:16 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1344860881' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:06:16.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:16 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3617227750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:16.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.943+0000 7f40a1400640 1 -- 172.21.15.69:0/2287237924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f409c062c00 con 0x7f409c109850 2024-04-24T09:06:16.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.946+0000 7f4093e00640 1 -- 172.21.15.69:0/2287237924 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24564 (secure 0 0 0) 0x7f408409c180 con 0x7f409c109850 2024-04-24T09:06:16.945 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:16.945 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":92,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:24.497245+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:06:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.948+0000 7f40a1400640 1 -- 172.21.15.69:0/2287237924 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f40800a3c30 msgr2=0x7f40800a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.948+0000 7f40a1400640 1 --2- 172.21.15.69:0/2287237924 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f40800a3c30 0x7f40800a60f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f409c1096b0 tx=0x7f408c005e70 comp rx=0 tx=0).stop 2024-04-24T09:06:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.948+0000 7f40a1400640 1 -- 172.21.15.69:0/2287237924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f409c109850 msgr2=0x7f409c19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.948+0000 7f40a1400640 1 --2- 172.21.15.69:0/2287237924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f409c109850 0x7f409c19ed90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f408400d770 tx=0x7f408400dc30 comp rx=0 tx=0).stop 2024-04-24T09:06:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.949+0000 7f40a1400640 1 -- 172.21.15.69:0/2287237924 shutdown_connections 2024-04-24T09:06:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.949+0000 7f40a1400640 1 --2- 172.21.15.69:0/2287237924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f409c10a210 0x7f409c1a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.949+0000 7f40a1400640 1 --2- 172.21.15.69:0/2287237924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f409c109850 0x7f409c19ed90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.949+0000 7f40a1400640 1 --2- 172.21.15.69:0/2287237924 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f40800a3c30 0x7f40800a60f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.949+0000 7f40a1400640 1 --2- 172.21.15.69:0/2287237924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f409c108660 0x7f409c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.949+0000 7f40a1400640 1 -- 172.21.15.69:0/2287237924 >> 172.21.15.69:0/2287237924 conn(0x7f409c077620 msgr2=0x7f409c10cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.949+0000 7f40a1400640 1 -- 172.21.15.69:0/2287237924 shutdown_connections 2024-04-24T09:06:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:16.949+0000 7f40a1400640 1 -- 172.21.15.69:0/2287237924 wait complete. 2024-04-24T09:06:16.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:16 smithi130 ceph-mon[31553]: pgmap v438: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:16.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:16 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3633409006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:06:16.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:16 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1344860881' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:06:16.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:16 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3617227750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:16.969 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:06:17.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.179+0000 7fa182800640 1 -- 172.21.15.69:0/2515168062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa17c10a210 msgr2=0x7fa17c110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.179+0000 7fa182800640 1 --2- 172.21.15.69:0/2515168062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa17c10a210 0x7fa17c110aa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa174009ca0 tx=0x7fa17402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.179+0000 7fa182800640 1 -- 172.21.15.69:0/2515168062 shutdown_connections 2024-04-24T09:06:17.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.179+0000 7fa182800640 1 --2- 172.21.15.69:0/2515168062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa17c10a210 0x7fa17c110aa0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.179+0000 7fa182800640 1 --2- 172.21.15.69:0/2515168062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa17c109850 0x7fa17c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.179+0000 7fa182800640 1 --2- 172.21.15.69:0/2515168062 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa17c108660 0x7fa17c108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.179+0000 7fa182800640 1 -- 172.21.15.69:0/2515168062 >> 172.21.15.69:0/2515168062 conn(0x7fa17c077620 msgr2=0x7fa17c079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:17.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.179+0000 7fa182800640 1 -- 172.21.15.69:0/2515168062 shutdown_connections 2024-04-24T09:06:17.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.179+0000 7fa182800640 1 -- 172.21.15.69:0/2515168062 wait complete. 2024-04-24T09:06:17.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.180+0000 7fa182800640 1 Processor -- start 2024-04-24T09:06:17.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.180+0000 7fa182800640 1 -- start start 2024-04-24T09:06:17.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.180+0000 7fa182800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa17c108660 0x7fa17c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.180+0000 7fa182800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa17c109850 0x7fa17c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.180+0000 7fa182800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa17c10a210 0x7fa17c1a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.180+0000 7fa182800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c114190 con 0x7fa17c10a210 2024-04-24T09:06:17.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.180+0000 7fa182800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c114010 con 0x7fa17c108660 2024-04-24T09:06:17.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.180+0000 7fa182800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c114310 con 0x7fa17c109850 2024-04-24T09:06:17.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.181+0000 7fa17be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa17c10a210 0x7fa17c1a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.181+0000 7fa17be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa17c10a210 0x7fa17c1a5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40058/0 (socket says 172.21.15.69:40058) 2024-04-24T09:06:17.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.181+0000 7fa17be00640 1 -- 172.21.15.69:0/2853008003 learned_addr learned my addr 172.21.15.69:0/2853008003 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:17.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.181+0000 7fa17aa00640 1 --2- 172.21.15.69:0/2853008003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa17c109850 0x7fa17c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.181+0000 7fa17b400640 1 --2- 172.21.15.69:0/2853008003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa17c108660 0x7fa17c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.181+0000 7fa17be00640 1 -- 172.21.15.69:0/2853008003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa17c109850 msgr2=0x7fa17c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.181+0000 7fa17be00640 1 --2- 172.21.15.69:0/2853008003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa17c109850 0x7fa17c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.181+0000 7fa17be00640 1 -- 172.21.15.69:0/2853008003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa17c108660 msgr2=0x7fa17c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.181+0000 7fa17be00640 1 --2- 172.21.15.69:0/2853008003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa17c108660 0x7fa17c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.181+0000 7fa17be00640 1 -- 172.21.15.69:0/2853008003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17c1a6480 con 0x7fa17c10a210 2024-04-24T09:06:17.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.181+0000 7fa17be00640 1 --2- 172.21.15.69:0/2853008003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa17c10a210 0x7fa17c1a5e10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa174009dc0 tx=0x7fa174003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:17.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.182+0000 7fa173e00640 1 -- 172.21.15.69:0/2853008003 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174032ec0 con 0x7fa17c10a210 2024-04-24T09:06:17.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.182+0000 7fa173e00640 1 -- 172.21.15.69:0/2853008003 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1740044f0 con 0x7fa17c10a210 2024-04-24T09:06:17.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.182+0000 7fa182800640 1 -- 172.21.15.69:0/2853008003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17c1a6710 con 0x7fa17c10a210 2024-04-24T09:06:17.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.182+0000 7fa173e00640 1 -- 172.21.15.69:0/2853008003 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174030d40 con 0x7fa17c10a210 2024-04-24T09:06:17.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.182+0000 7fa182800640 1 -- 172.21.15.69:0/2853008003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa17c1a6c60 con 0x7fa17c10a210 2024-04-24T09:06:17.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.183+0000 7fa182800640 1 -- 172.21.15.69:0/2853008003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa17c10f9d0 con 0x7fa17c10a210 2024-04-24T09:06:17.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.185+0000 7fa173e00640 1 -- 172.21.15.69:0/2853008003 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa174004040 con 0x7fa17c10a210 2024-04-24T09:06:17.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.187+0000 7fa173e00640 1 --2- 172.21.15.69:0/2853008003 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa1600a3d50 0x7fa1600a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.188+0000 7fa173e00640 1 -- 172.21.15.69:0/2853008003 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa1740f1e20 con 0x7fa17c10a210 2024-04-24T09:06:17.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.188+0000 7fa17b400640 1 --2- 172.21.15.69:0/2853008003 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa1600a3d50 0x7fa1600a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.188+0000 7fa17b400640 1 --2- 172.21.15.69:0/2853008003 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa1600a3d50 0x7fa1600a6210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0058c0 tx=0x7fa16c00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:17.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.192+0000 7fa173e00640 1 -- 172.21.15.69:0/2853008003 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1740be4a0 con 0x7fa17c10a210 2024-04-24T09:06:17.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.436+0000 7fa182800640 1 -- 172.21.15.69:0/2853008003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa17c02d740 con 0x7fa17c10a210 2024-04-24T09:06:17.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.437+0000 7fa173e00640 1 -- 172.21.15.69:0/2853008003 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11873 (secure 0 0 0) 0x7fa1740bdda0 con 0x7fa17c10a210 2024-04-24T09:06:17.436 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:17.436 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:04:01.991559+0000","modified":"2024-04-24T09:04:34.617681+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34532,"mds_1":34496,"mds_2":34460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34496":{"gid":34496,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34532":{"gid":34532,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34532,"qdb_cluster":[34460,34496,34532]},"id":1}]} 2024-04-24T09:06:17.436 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2024-04-24T09:06:17.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.440+0000 7fa182800640 1 -- 172.21.15.69:0/2853008003 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa1600a3d50 msgr2=0x7fa1600a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.440+0000 7fa182800640 1 --2- 172.21.15.69:0/2853008003 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa1600a3d50 0x7fa1600a6210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0058c0 tx=0x7fa16c00a4c0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.440+0000 7fa182800640 1 -- 172.21.15.69:0/2853008003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa17c10a210 msgr2=0x7fa17c1a5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.440+0000 7fa182800640 1 --2- 172.21.15.69:0/2853008003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa17c10a210 0x7fa17c1a5e10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa174009dc0 tx=0x7fa174003ba0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.440+0000 7fa182800640 1 -- 172.21.15.69:0/2853008003 shutdown_connections 2024-04-24T09:06:17.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.440+0000 7fa182800640 1 --2- 172.21.15.69:0/2853008003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa17c10a210 0x7fa17c1a5e10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.440+0000 7fa182800640 1 --2- 172.21.15.69:0/2853008003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa17c109850 0x7fa17c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.440+0000 7fa182800640 1 --2- 172.21.15.69:0/2853008003 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa1600a3d50 0x7fa1600a6210 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.440+0000 7fa182800640 1 --2- 172.21.15.69:0/2853008003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa17c108660 0x7fa17c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.440+0000 7fa182800640 1 -- 172.21.15.69:0/2853008003 >> 172.21.15.69:0/2853008003 conn(0x7fa17c077620 msgr2=0x7fa17c10cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:17.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.440+0000 7fa182800640 1 -- 172.21.15.69:0/2853008003 shutdown_connections 2024-04-24T09:06:17.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.440+0000 7fa182800640 1 -- 172.21.15.69:0/2853008003 wait complete. 2024-04-24T09:06:17.461 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T09:04:01.991559+0000', 'modified': '2024-04-24T09:04:34.617681+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 34532, 'mds_1': 34496, 'mds_2': 34460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6837/2790982303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6836', 'nonce': 2790982303}, {'type': 'v1', 'addr': '172.21.15.130:6837', 'nonce': 2790982303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6839/4193111366', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6838', 'nonce': 4193111366}, {'type': 'v1', 'addr': '172.21.15.130:6839', 'nonce': 4193111366}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34427': {'gid': 34427, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.81:6833/3240654002', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6832', 'nonce': 3240654002}, {'type': 'v1', 'addr': '172.21.15.81:6833', 'nonce': 3240654002}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34460': {'gid': 34460, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.81:6835/3260829578', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6834', 'nonce': 3260829578}, {'type': 'v1', 'addr': '172.21.15.81:6835', 'nonce': 3260829578}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34496': {'gid': 34496, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.81:6837/1973819593', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 1973819593}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 1973819593}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34532': {'gid': 34532, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.81:6839/3185704097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6838', 'nonce': 3185704097}, {'type': 'v1', 'addr': '172.21.15.81:6839', 'nonce': 3185704097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34532, 'qdb_cluster': [34460, 34496, 34532]} 2024-04-24T09:06:17.461 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T09:06:17.462 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-24T09:06:17.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.657+0000 7f6b6fe00640 1 -- 172.21.15.69:0/1564927514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b70105330 msgr2=0x7f6b701057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.657+0000 7f6b6fe00640 1 --2- 172.21.15.69:0/1564927514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b70105330 0x7f6b701057b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009900 tx=0x7f6b5802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.657+0000 7f6b6fe00640 1 -- 172.21.15.69:0/1564927514 shutdown_connections 2024-04-24T09:06:17.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.657+0000 7f6b6fe00640 1 --2- 172.21.15.69:0/1564927514 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b70105cf0 0x7f6b7010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.657+0000 7f6b6fe00640 1 --2- 172.21.15.69:0/1564927514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b70105330 0x7f6b701057b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.657+0000 7f6b6fe00640 1 --2- 172.21.15.69:0/1564927514 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b70104140 0x7f6b70104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.657+0000 7f6b6fe00640 1 -- 172.21.15.69:0/1564927514 >> 172.21.15.69:0/1564927514 conn(0x7f6b700ff6b0 msgr2=0x7f6b70101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:17.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.657+0000 7f6b6fe00640 1 -- 172.21.15.69:0/1564927514 shutdown_connections 2024-04-24T09:06:17.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.657+0000 7f6b6fe00640 1 -- 172.21.15.69:0/1564927514 wait complete. 2024-04-24T09:06:17.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.658+0000 7f6b6fe00640 1 Processor -- start 2024-04-24T09:06:17.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.658+0000 7f6b6fe00640 1 -- start start 2024-04-24T09:06:17.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.658+0000 7f6b6fe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b70104140 0x7f6b7019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.658+0000 7f6b6fe00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b70105330 0x7f6b7019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.658+0000 7f6b6fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b70105cf0 0x7f6b701a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.658+0000 7f6b6fe00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7010fd60 con 0x7f6b70105cf0 2024-04-24T09:06:17.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.658+0000 7f6b6fe00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7010fbe0 con 0x7f6b70105330 2024-04-24T09:06:17.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.658+0000 7f6b6fe00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7010fee0 con 0x7f6b70104140 2024-04-24T09:06:17.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.658+0000 7f6b6f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b70105cf0 0x7f6b701a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.659+0000 7f6b6f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b70105cf0 0x7f6b701a1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40086/0 (socket says 172.21.15.69:40086) 2024-04-24T09:06:17.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.659+0000 7f6b6f400640 1 -- 172.21.15.69:0/1124495802 learned_addr learned my addr 172.21.15.69:0/1124495802 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:17.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.659+0000 7f6b6ea00640 1 --2- 172.21.15.69:0/1124495802 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b70104140 0x7f6b7019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.659+0000 7f6b6e000640 1 --2- 172.21.15.69:0/1124495802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b70105330 0x7f6b7019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.659+0000 7f6b6f400640 1 -- 172.21.15.69:0/1124495802 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b70104140 msgr2=0x7f6b7019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.659+0000 7f6b6f400640 1 --2- 172.21.15.69:0/1124495802 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b70104140 0x7f6b7019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.659+0000 7f6b6f400640 1 -- 172.21.15.69:0/1124495802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b70105330 msgr2=0x7f6b7019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.659+0000 7f6b6f400640 1 --2- 172.21.15.69:0/1124495802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b70105330 0x7f6b7019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.659+0000 7f6b6f400640 1 -- 172.21.15.69:0/1124495802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b701a2100 con 0x7f6b70105cf0 2024-04-24T09:06:17.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.659+0000 7f6b6f400640 1 --2- 172.21.15.69:0/1124495802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b70105cf0 0x7f6b701a1a90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800b970 tx=0x7f6b6800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:17.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.659+0000 7f6b67400640 1 -- 172.21.15.69:0/1124495802 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68019070 con 0x7f6b70105cf0 2024-04-24T09:06:17.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.660+0000 7f6b67400640 1 -- 172.21.15.69:0/1124495802 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b68014440 con 0x7f6b70105cf0 2024-04-24T09:06:17.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.660+0000 7f6b6fe00640 1 -- 172.21.15.69:0/1124495802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b701a23f0 con 0x7f6b70105cf0 2024-04-24T09:06:17.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.660+0000 7f6b67400640 1 -- 172.21.15.69:0/1124495802 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b680075d0 con 0x7f6b70105cf0 2024-04-24T09:06:17.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.660+0000 7f6b6fe00640 1 -- 172.21.15.69:0/1124495802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b701a2970 con 0x7f6b70105cf0 2024-04-24T09:06:17.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.663+0000 7f6b7837e740 1 Processor -- start 2024-04-24T09:06:17.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.663+0000 7f6b67400640 1 -- 172.21.15.69:0/1124495802 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6b6801c070 con 0x7f6b70105cf0 2024-04-24T09:06:17.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.663+0000 7f6b7837e740 1 -- start start 2024-04-24T09:06:17.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.663+0000 7f6b7837e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed95260 0x56524ed95680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.663+0000 7f6b7837e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed95c50 0x56524ed9a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.663+0000 7f6b7837e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed9adc0 0x56524ed9d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.663+0000 7f6b7837e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56524ecfab10 con 0x56524ed9adc0 2024-04-24T09:06:17.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.663+0000 7f6b7837e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56524eca12b0 con 0x56524ed95260 2024-04-24T09:06:17.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.663+0000 7f6b7837e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56524ecf6ad0 con 0x56524ed95c50 2024-04-24T09:06:17.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.664+0000 7f6b6e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed9adc0 0x56524ed9d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.664+0000 7f6b6e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed9adc0 0x56524ed9d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40092/0 (socket says 172.21.15.69:40092) 2024-04-24T09:06:17.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.664+0000 7f6b6e000640 1 -- 172.21.15.69:0/1336484664 learned_addr learned my addr 172.21.15.69:0/1336484664 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:17.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.664+0000 7f6b6ea00640 1 --2- 172.21.15.69:0/1336484664 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed95c50 0x56524ed9a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.664+0000 7f6b6e000640 1 --2- 172.21.15.69:0/1336484664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed95260 0x56524ed95680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.664+0000 7f6b6e000640 1 -- 172.21.15.69:0/1336484664 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed95c50 msgr2=0x56524ed9a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:17.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.664+0000 7f6b6e000640 1 --2- 172.21.15.69:0/1336484664 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed95c50 0x56524ed9a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.664+0000 7f6b6e000640 1 -- 172.21.15.69:0/1336484664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed95260 msgr2=0x56524ed95680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:17.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.664+0000 7f6b6e000640 1 --2- 172.21.15.69:0/1336484664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed95260 0x56524ed95680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.664+0000 7f6b6e000640 1 -- 172.21.15.69:0/1336484664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56524ea0d2d0 con 0x56524ed9adc0 2024-04-24T09:06:17.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.664+0000 7f6b67400640 1 --2- 172.21.15.69:0/1124495802 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6b540a3bb0 0x7f6b540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.665+0000 7f6b6e000640 1 --2- 172.21.15.69:0/1336484664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed9adc0 0x56524ed9d250 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6b5802a9e0 tx=0x7f6b580314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448b627ea3b280f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:17.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.665+0000 7f6b67400640 1 -- 172.21.15.69:0/1124495802 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6b680cfdf0 con 0x7f6b70105cf0 2024-04-24T09:06:17.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.665+0000 7f6b6ea00640 1 --2- 172.21.15.69:0/1124495802 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6b540a3bb0 0x7f6b540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.665+0000 7f6b5f400640 1 -- 172.21.15.69:0/1336484664 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58045070 con 0x56524ed9adc0 2024-04-24T09:06:17.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.665+0000 7f6b5f400640 1 -- 172.21.15.69:0/1336484664 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b58004030 con 0x56524ed9adc0 2024-04-24T09:06:17.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.665+0000 7f6b5f400640 1 -- 172.21.15.69:0/1336484664 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58035040 con 0x56524ed9adc0 2024-04-24T09:06:17.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.666+0000 7f6b7837e740 1 -- 172.21.15.69:0/1336484664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed9adc0 msgr2=0x56524ed9d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.666+0000 7f6b6ea00640 1 --2- 172.21.15.69:0/1124495802 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6b540a3bb0 0x7f6b540a6070 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x56524ed96050 tx=0x7f6b60008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:17.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.666+0000 7f6b7837e740 1 --2- 172.21.15.69:0/1336484664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed9adc0 0x56524ed9d250 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6b5802a9e0 tx=0x7f6b580314a0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.666+0000 7f6b7837e740 1 -- 172.21.15.69:0/1336484664 shutdown_connections 2024-04-24T09:06:17.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.666+0000 7f6b7837e740 1 --2- 172.21.15.69:0/1336484664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed9adc0 0x56524ed9d250 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.666+0000 7f6b7837e740 1 --2- 172.21.15.69:0/1336484664 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed95c50 0x56524ed9a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.666+0000 7f6b7837e740 1 --2- 172.21.15.69:0/1336484664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed95260 0x56524ed95680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.666+0000 7f6b7837e740 1 -- 172.21.15.69:0/1336484664 >> 172.21.15.69:0/1336484664 conn(0x56524ed908d0 msgr2=0x56524ed90cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:17.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.666+0000 7f6b7837e740 1 -- 172.21.15.69:0/1336484664 shutdown_connections 2024-04-24T09:06:17.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.666+0000 7f6b7837e740 1 -- 172.21.15.69:0/1336484664 wait complete. 2024-04-24T09:06:17.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.666+0000 7f6b7837e740 1 -- start start 2024-04-24T09:06:17.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.667+0000 7f6b7837e740 1 Processor -- start 2024-04-24T09:06:17.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.667+0000 7f6b7837e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed95260 0x56524ed9f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.667+0000 7f6b7837e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95c50 0x56524ed9fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.667+0000 7f6b7837e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed9adc0 0x56524eda6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.667+0000 7f6b7837e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56524ed9dd10 con 0x56524ed95c50 2024-04-24T09:06:17.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.667+0000 7f6b7837e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56524ecb4590 con 0x56524ed9adc0 2024-04-24T09:06:17.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.667+0000 7f6b7837e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56524ed9de90 con 0x56524ed95260 2024-04-24T09:06:17.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.667+0000 7f6b7837e740 10 client.0 _ll_register_callbacks cb 0x56524ec973a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:17.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.667+0000 7f6b6e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95c50 0x56524ed9fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.667+0000 7f6b6e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95c50 0x56524ed9fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40104/0 (socket says 172.21.15.69:40104) 2024-04-24T09:06:17.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.667+0000 7f6b6e000640 1 -- 172.21.15.69:0/1897569722 learned_addr learned my addr 172.21.15.69:0/1897569722 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:17.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.668+0000 7f6b6e000640 1 -- 172.21.15.69:0/1897569722 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed95260 msgr2=0x56524ed9f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:17.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.668+0000 7f6b6ea00640 1 --2- 172.21.15.69:0/1897569722 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed95260 0x56524ed9f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.668+0000 7f6b6f400640 1 --2- 172.21.15.69:0/1897569722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed9adc0 0x56524eda6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.668+0000 7f6b6e000640 1 --2- 172.21.15.69:0/1897569722 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed95260 0x56524ed9f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.668+0000 7f6b6e000640 1 -- 172.21.15.69:0/1897569722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed9adc0 msgr2=0x56524eda6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:17.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.668+0000 7f6b6e000640 1 --2- 172.21.15.69:0/1897569722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed9adc0 0x56524eda6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.668+0000 7f6b6e000640 1 -- 172.21.15.69:0/1897569722 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524ead76b0 con 0x56524ed95c50 2024-04-24T09:06:17.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.668+0000 7f6b6e000640 1 --2- 172.21.15.69:0/1897569722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95c50 0x56524ed9fc00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6b58033000 tx=0x7f6b58005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acb8a9e06c6a176 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:17.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.668+0000 7f6b53e00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:06:17.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.668+0000 7f6b53e00640 1 -- 172.21.15.69:0/1897569722 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58035070 con 0x56524ed95c50 2024-04-24T09:06:17.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.668+0000 7f6b53e00640 1 -- 172.21.15.69:0/1897569722 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6b580043e0 con 0x56524ed95c50 2024-04-24T09:06:17.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.668+0000 7f6b7837e740 1 -- 172.21.15.69:0/1897569722 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b58004030 con 0x56524ed95c50 2024-04-24T09:06:17.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.668+0000 7f6b53e00640 1 -- 172.21.15.69:0/1897569722 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6b58056050 con 0x56524ed95c50 2024-04-24T09:06:17.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.669+0000 7f6b7837e740 10 client.15225 fetch_fsmap learned FSMap version 23 2024-04-24T09:06:17.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.669+0000 7f6b7837e740 1 -- 172.21.15.69:0/1897569722 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56524eda9fb0 con 0x56524ed95c50 2024-04-24T09:06:17.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.669+0000 7f6b53e00640 1 -- 172.21.15.69:0/1897569722 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b58004030 con 0x56524ed95c50 2024-04-24T09:06:17.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.669+0000 7f6b53e00640 1 -- 172.21.15.69:0/1897569722 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f6b58051030 con 0x56524ed95c50 2024-04-24T09:06:17.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.669+0000 7f6b7837e740 10 client.15225 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T09:06:17.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.669+0000 7f6b7837e740 10 client.15225 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-24T09:06:17.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.669+0000 7f6b7837e740 20 client.15225 populate_metadata read hostname 'smithi069' 2024-04-24T09:06:17.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.669+0000 7f6b7837e740 1 --2- 172.21.15.69:0/1897569722 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x56524edac1f0 0x56524edad660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.669+0000 7f6b7837e740 4 client.15225 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:17.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.669+0000 7f6b7837e740 1 -- 172.21.15.69:0/1897569722 --> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56524ed08530 con 0x56524edac1f0 2024-04-24T09:06:17.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.670+0000 7f6b6ea00640 1 --2- 172.21.15.69:0/1897569722 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x56524edac1f0 0x56524edad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.671+0000 7f6b6ea00640 1 --2- 172.21.15.69:0/1897569722 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x56524edac1f0 0x56524edad660 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e10168cb430be49d server_cookie=f2d6ccf11cfa5be4 in_seq=0 out_seq=0 2024-04-24T09:06:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.671+0000 7f6b53e00640 10 client.15225 ms_handle_connect on v2:172.21.15.81:6834/3260829578 2024-04-24T09:06:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.673+0000 7f6b53e00640 1 -- 172.21.15.69:0/1897569722 <== mds.2 v2:172.21.15.81:6834/3260829578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56524ed08530 con 0x56524edac1f0 2024-04-24T09:06:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.673+0000 7f6b53e00640 10 client.15225 handle_command_reply: tid=0 2024-04-24T09:06:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.673+0000 7f6b7837e740 1 client.15225 shutdown 2024-04-24T09:06:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.673+0000 7f6b7837e740 1 -- 172.21.15.69:0/1897569722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95c50 msgr2=0x56524ed9fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.673+0000 7f6b7837e740 1 --2- 172.21.15.69:0/1897569722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95c50 0x56524ed9fc00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6b58033000 tx=0x7f6b58005540 comp rx=0 tx=0).stop 2024-04-24T09:06:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.673+0000 7f6b7837e740 1 -- 172.21.15.69:0/1897569722 shutdown_connections 2024-04-24T09:06:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.673+0000 7f6b7837e740 1 --2- 172.21.15.69:0/1897569722 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x56524edac1f0 0x56524edad660 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.673+0000 7f6b7837e740 1 --2- 172.21.15.69:0/1897569722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed9adc0 0x56524eda6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.674+0000 7f6b7837e740 1 --2- 172.21.15.69:0/1897569722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95c50 0x56524ed9fc00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.674+0000 7f6b7837e740 1 --2- 172.21.15.69:0/1897569722 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed95260 0x56524ed9f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.674+0000 7f6b53e00640 0 client.15225 ms_handle_reset on v2:172.21.15.81:6834/3260829578 2024-04-24T09:06:17.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.674+0000 7f6b7837e740 1 -- 172.21.15.69:0/1897569722 >> 172.21.15.69:0/1897569722 conn(0x56524ed908d0 msgr2=0x56524ed8f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:17.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.674+0000 7f6b7837e740 1 -- 172.21.15.69:0/1897569722 shutdown_connections 2024-04-24T09:06:17.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.674+0000 7f6b7837e740 1 -- 172.21.15.69:0/1897569722 wait complete. 2024-04-24T09:06:17.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.674+0000 7f6b7837e740 20 client.15225 trim_cache size 0 max 16384 2024-04-24T09:06:17.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.693+0000 7f6b7837e740 1 Processor -- start 2024-04-24T09:06:17.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.693+0000 7f6b7837e740 1 -- start start 2024-04-24T09:06:17.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.694+0000 7f6b7837e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed95260 0x56524ed9e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.694+0000 7f6b7837e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95c50 0x56524eda6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.694+0000 7f6b7837e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed9adc0 0x56524ed90cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.694+0000 7f6b7837e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56524edab9a0 con 0x56524ed95c50 2024-04-24T09:06:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.694+0000 7f6b7837e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56524edac070 con 0x56524ed95260 2024-04-24T09:06:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.694+0000 7f6b7837e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56524edac740 con 0x56524ed9adc0 2024-04-24T09:06:17.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.694+0000 7f6b6e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95c50 0x56524eda6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.694+0000 7f6b6e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95c50 0x56524eda6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40108/0 (socket says 172.21.15.69:40108) 2024-04-24T09:06:17.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.694+0000 7f6b6e000640 1 -- 172.21.15.69:0/4107134775 learned_addr learned my addr 172.21.15.69:0/4107134775 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:17.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.694+0000 7f6b6ea00640 1 --2- 172.21.15.69:0/4107134775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed95260 0x56524ed9e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.694+0000 7f6b6f400640 1 --2- 172.21.15.69:0/4107134775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed9adc0 0x56524ed90cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.694+0000 7f6b6e000640 1 -- 172.21.15.69:0/4107134775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed9adc0 msgr2=0x56524ed90cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.694+0000 7f6b6e000640 1 --2- 172.21.15.69:0/4107134775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed9adc0 0x56524ed90cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.694+0000 7f6b6e000640 1 -- 172.21.15.69:0/4107134775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed95260 msgr2=0x56524ed9e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.694+0000 7f6b6e000640 1 --2- 172.21.15.69:0/4107134775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed95260 0x56524ed9e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.694+0000 7f6b6e000640 1 -- 172.21.15.69:0/4107134775 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56524ed90700 con 0x56524ed95c50 2024-04-24T09:06:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.695+0000 7f6b6e000640 1 --2- 172.21.15.69:0/4107134775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95c50 0x56524eda6c00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6b58004ac0 tx=0x7f6b58032ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83638c65ebc74f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.695+0000 7f6b5ea00640 1 -- 172.21.15.69:0/4107134775 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58035040 con 0x56524ed95c50 2024-04-24T09:06:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.695+0000 7f6b5ea00640 1 -- 172.21.15.69:0/4107134775 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b58041030 con 0x56524ed95c50 2024-04-24T09:06:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.695+0000 7f6b5ea00640 1 -- 172.21.15.69:0/4107134775 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58045040 con 0x56524ed95c50 2024-04-24T09:06:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.696+0000 7f6b7837e740 1 -- 172.21.15.69:0/4107134775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95c50 msgr2=0x56524eda6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.696+0000 7f6b7837e740 1 --2- 172.21.15.69:0/4107134775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95c50 0x56524eda6c00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6b58004ac0 tx=0x7f6b58032ab0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.696+0000 7f6b7837e740 1 -- 172.21.15.69:0/4107134775 shutdown_connections 2024-04-24T09:06:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.696+0000 7f6b7837e740 1 --2- 172.21.15.69:0/4107134775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed9adc0 0x56524ed90cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.696+0000 7f6b7837e740 1 --2- 172.21.15.69:0/4107134775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95c50 0x56524eda6c00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.696+0000 7f6b7837e740 1 --2- 172.21.15.69:0/4107134775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed95260 0x56524ed9e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.696+0000 7f6b7837e740 1 -- 172.21.15.69:0/4107134775 >> 172.21.15.69:0/4107134775 conn(0x56524ed908d0 msgr2=0x56524ed966f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:17.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.696+0000 7f6b7837e740 1 -- 172.21.15.69:0/4107134775 shutdown_connections 2024-04-24T09:06:17.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.696+0000 7f6b7837e740 1 -- 172.21.15.69:0/4107134775 wait complete. 2024-04-24T09:06:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.696+0000 7f6b7837e740 1 -- start start 2024-04-24T09:06:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.696+0000 7f6b7837e740 1 Processor -- start 2024-04-24T09:06:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.697+0000 7f6b7837e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95260 0x56524eda2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.697+0000 7f6b7837e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed95c50 0x56524eda3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.697+0000 7f6b7837e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed9adc0 0x56524eda60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.697+0000 7f6b7837e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56524edad4e0 con 0x56524ed95260 2024-04-24T09:06:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.697+0000 7f6b7837e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56524edace10 con 0x56524ed95c50 2024-04-24T09:06:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.697+0000 7f6b7837e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56524edadbb0 con 0x56524ed9adc0 2024-04-24T09:06:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.697+0000 7f6b7837e740 10 client.0 _ll_register_callbacks cb 0x7f6b58032cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.697+0000 7f6b6ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95260 0x56524eda2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.697+0000 7f6b6ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95260 0x56524eda2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40118/0 (socket says 172.21.15.69:40118) 2024-04-24T09:06:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.697+0000 7f6b6ea00640 1 -- 172.21.15.69:0/3592875535 learned_addr learned my addr 172.21.15.69:0/3592875535 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:17.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.697+0000 7f6b6e000640 1 --2- 172.21.15.69:0/3592875535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed95c50 0x56524eda3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.697+0000 7f6b6ea00640 1 -- 172.21.15.69:0/3592875535 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed9adc0 msgr2=0x56524eda60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:17.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.697+0000 7f6b6ea00640 1 --2- 172.21.15.69:0/3592875535 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed9adc0 0x56524eda60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.697+0000 7f6b6ea00640 1 -- 172.21.15.69:0/3592875535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed95c50 msgr2=0x56524eda3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:17.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.697+0000 7f6b6ea00640 1 --2- 172.21.15.69:0/3592875535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed95c50 0x56524eda3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.697+0000 7f6b6ea00640 1 -- 172.21.15.69:0/3592875535 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524eda66b0 con 0x56524ed95260 2024-04-24T09:06:17.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.698+0000 7f6b6ea00640 1 --2- 172.21.15.69:0/3592875535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95260 0x56524eda2d90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x56524ed966c0 tx=0x7f6b600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f082cb7173edac26 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:17.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.698+0000 7f6b53e00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:06:17.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.698+0000 7f6b53e00640 1 -- 172.21.15.69:0/3592875535 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6001b040 con 0x56524ed95260 2024-04-24T09:06:17.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.698+0000 7f6b53e00640 1 -- 172.21.15.69:0/3592875535 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6b60026810 con 0x56524ed95260 2024-04-24T09:06:17.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.698+0000 7f6b7837e740 1 -- 172.21.15.69:0/3592875535 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b58041030 con 0x56524ed95260 2024-04-24T09:06:17.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.698+0000 7f6b53e00640 1 -- 172.21.15.69:0/3592875535 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6b60014460 con 0x56524ed95260 2024-04-24T09:06:17.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.698+0000 7f6b7837e740 10 client.15231 fetch_fsmap learned FSMap version 23 2024-04-24T09:06:17.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.698+0000 7f6b7837e740 1 -- 172.21.15.69:0/3592875535 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56524ee12a20 con 0x56524ed95260 2024-04-24T09:06:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.698+0000 7f6b53e00640 1 -- 172.21.15.69:0/3592875535 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b58041030 con 0x56524ed95260 2024-04-24T09:06:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.698+0000 7f6b53e00640 1 -- 172.21.15.69:0/3592875535 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f6b6002d020 con 0x56524ed95260 2024-04-24T09:06:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.698+0000 7f6b7837e740 10 client.15231 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T09:06:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.699+0000 7f6b7837e740 10 client.15231 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-24T09:06:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.699+0000 7f6b7837e740 20 client.15231 populate_metadata read hostname 'smithi069' 2024-04-24T09:06:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.699+0000 7f6b7837e740 1 --2- 172.21.15.69:0/3592875535 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x56524ee176d0 0x56524ee163e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.699+0000 7f6b7837e740 4 client.15231 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:06:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.699+0000 7f6b7837e740 1 -- 172.21.15.69:0/3592875535 --> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] -- command(tid 0: {"prefix": "status"}) -- 0x56524ed023a0 con 0x56524ee176d0 2024-04-24T09:06:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.699+0000 7f6b6e000640 1 --2- 172.21.15.69:0/3592875535 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x56524ee176d0 0x56524ee163e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.700+0000 7f6b6e000640 1 --2- 172.21.15.69:0/3592875535 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x56524ee176d0 0x56524ee163e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b9de67ac896f521 server_cookie=957c6bad61134d88 in_seq=0 out_seq=0 2024-04-24T09:06:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.700+0000 7f6b53e00640 10 client.15231 ms_handle_connect on v2:172.21.15.81:6834/3260829578 2024-04-24T09:06:17.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.701+0000 7f6b53e00640 1 -- 172.21.15.69:0/3592875535 <== mds.2 v2:172.21.15.81:6834/3260829578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56524ed023a0 con 0x56524ee176d0 2024-04-24T09:06:17.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.701+0000 7f6b53e00640 10 client.15231 handle_command_reply: tid=0 2024-04-24T09:06:17.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.701+0000 7f6b7837e740 1 client.15231 shutdown 2024-04-24T09:06:17.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.701+0000 7f6b7837e740 1 -- 172.21.15.69:0/3592875535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95260 msgr2=0x56524eda2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.701+0000 7f6b7837e740 1 --2- 172.21.15.69:0/3592875535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95260 0x56524eda2d90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x56524ed966c0 tx=0x7f6b600046c0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.701+0000 7f6b7837e740 1 -- 172.21.15.69:0/3592875535 shutdown_connections 2024-04-24T09:06:17.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.701+0000 7f6b7837e740 1 --2- 172.21.15.69:0/3592875535 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x56524ee176d0 0x56524ee163e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.701+0000 7f6b7837e740 1 --2- 172.21.15.69:0/3592875535 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56524ed9adc0 0x56524eda60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.701+0000 7f6b7837e740 1 --2- 172.21.15.69:0/3592875535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56524ed95c50 0x56524eda3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.701+0000 7f6b7837e740 1 --2- 172.21.15.69:0/3592875535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56524ed95260 0x56524eda2d90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.701+0000 7f6b53e00640 0 client.15231 ms_handle_reset on v2:172.21.15.81:6834/3260829578 2024-04-24T09:06:17.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.701+0000 7f6b7837e740 1 -- 172.21.15.69:0/3592875535 >> 172.21.15.69:0/3592875535 conn(0x56524ed908d0 msgr2=0x56524ed9e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:17.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.701+0000 7f6b7837e740 1 -- 172.21.15.69:0/3592875535 shutdown_connections 2024-04-24T09:06:17.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.701+0000 7f6b7837e740 1 -- 172.21.15.69:0/3592875535 wait complete. 2024-04-24T09:06:17.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.701+0000 7f6b7837e740 20 client.15231 trim_cache size 0 max 16384 2024-04-24T09:06:17.703 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:06:17.704 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "ef1ec7e8-0217-11ef-bc93-c7b262605968", 2024-04-24T09:06:17.704 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 2, 2024-04-24T09:06:17.704 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 34460, 2024-04-24T09:06:17.704 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-24T09:06:17.704 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-24T09:06:17.704 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-24T09:06:17.704 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 104.09282652900001, 2024-04-24T09:06:17.704 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 23, 2024-04-24T09:06:17.704 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 92, 2024-04-24T09:06:17.704 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 92, 2024-04-24T09:06:17.704 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 169.068115369 2024-04-24T09:06:17.704 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:06:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.702+0000 7f6b6fe00640 1 -- 172.21.15.69:0/1124495802 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6b540a3bb0 msgr2=0x7f6b540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.702+0000 7f6b6fe00640 1 --2- 172.21.15.69:0/1124495802 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6b540a3bb0 0x7f6b540a6070 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x56524ed96050 tx=0x7f6b60008040 comp rx=0 tx=0).stop 2024-04-24T09:06:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.702+0000 7f6b6fe00640 1 -- 172.21.15.69:0/1124495802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b70105cf0 msgr2=0x7f6b701a1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.702+0000 7f6b6fe00640 1 --2- 172.21.15.69:0/1124495802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b70105cf0 0x7f6b701a1a90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800b970 tx=0x7f6b6800be30 comp rx=0 tx=0).stop 2024-04-24T09:06:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.702+0000 7f6b6fe00640 1 -- 172.21.15.69:0/1124495802 shutdown_connections 2024-04-24T09:06:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.702+0000 7f6b6fe00640 1 --2- 172.21.15.69:0/1124495802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b70105cf0 0x7f6b701a1a90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.702+0000 7f6b6fe00640 1 --2- 172.21.15.69:0/1124495802 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6b540a3bb0 0x7f6b540a6070 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.702+0000 7f6b6fe00640 1 --2- 172.21.15.69:0/1124495802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b70105330 0x7f6b7019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.702+0000 7f6b6fe00640 1 --2- 172.21.15.69:0/1124495802 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b70104140 0x7f6b7019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.702+0000 7f6b6fe00640 1 -- 172.21.15.69:0/1124495802 >> 172.21.15.69:0/1124495802 conn(0x7f6b700ff6b0 msgr2=0x7f6b70108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.702+0000 7f6b6fe00640 1 -- 172.21.15.69:0/1124495802 shutdown_connections 2024-04-24T09:06:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.702+0000 7f6b6fe00640 1 -- 172.21.15.69:0/1124495802 wait complete. 2024-04-24T09:06:17.714 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:17 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2287237924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:17.714 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:17 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2853008003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:17.717 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-24T09:06:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:17 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2287237924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:17 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2853008003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:17.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.906+0000 7ff257800640 1 -- 172.21.15.69:0/600176237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff250109850 msgr2=0x7ff250109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.906+0000 7ff257800640 1 --2- 172.21.15.69:0/600176237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff250109850 0x7ff250109cd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009900 tx=0x7ff23c02a410 comp rx=0 tx=0).stop 2024-04-24T09:06:17.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.906+0000 7ff257800640 1 -- 172.21.15.69:0/600176237 shutdown_connections 2024-04-24T09:06:17.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.906+0000 7ff257800640 1 --2- 172.21.15.69:0/600176237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff25010a210 0x7ff250110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.906+0000 7ff257800640 1 --2- 172.21.15.69:0/600176237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff250109850 0x7ff250109cd0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.906+0000 7ff257800640 1 --2- 172.21.15.69:0/600176237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff250108660 0x7ff250108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.906+0000 7ff257800640 1 -- 172.21.15.69:0/600176237 >> 172.21.15.69:0/600176237 conn(0x7ff250077600 msgr2=0x7ff250079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:17.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.906+0000 7ff257800640 1 -- 172.21.15.69:0/600176237 shutdown_connections 2024-04-24T09:06:17.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.906+0000 7ff257800640 1 -- 172.21.15.69:0/600176237 wait complete. 2024-04-24T09:06:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.907+0000 7ff257800640 1 Processor -- start 2024-04-24T09:06:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.907+0000 7ff257800640 1 -- start start 2024-04-24T09:06:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.908+0000 7ff257800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff250108660 0x7ff25019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.908+0000 7ff257800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff250109850 0x7ff25019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.908+0000 7ff257800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff25010a210 0x7ff2501a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.908+0000 7ff257800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff250114190 con 0x7ff250108660 2024-04-24T09:06:17.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.908+0000 7ff257800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff250114010 con 0x7ff250109850 2024-04-24T09:06:17.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.908+0000 7ff257800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff250114310 con 0x7ff25010a210 2024-04-24T09:06:17.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.908+0000 7ff254e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff250108660 0x7ff25019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.908+0000 7ff254e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff250108660 0x7ff25019e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40132/0 (socket says 172.21.15.69:40132) 2024-04-24T09:06:17.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.908+0000 7ff254e00640 1 -- 172.21.15.69:0/1380725412 learned_addr learned my addr 172.21.15.69:0/1380725412 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:17.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.908+0000 7ff24fe00640 1 --2- 172.21.15.69:0/1380725412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff250109850 0x7ff25019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.908+0000 7ff254e00640 1 -- 172.21.15.69:0/1380725412 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff25010a210 msgr2=0x7ff2501a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.908+0000 7ff254e00640 1 --2- 172.21.15.69:0/1380725412 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff25010a210 0x7ff2501a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.908+0000 7ff254e00640 1 -- 172.21.15.69:0/1380725412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff250109850 msgr2=0x7ff25019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.908+0000 7ff254e00640 1 --2- 172.21.15.69:0/1380725412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff250109850 0x7ff25019ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.908+0000 7ff254e00640 1 -- 172.21.15.69:0/1380725412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2501a64e0 con 0x7ff250108660 2024-04-24T09:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.908+0000 7ff24fe00640 1 --2- 172.21.15.69:0/1380725412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff250109850 0x7ff25019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.909+0000 7ff254e00640 1 --2- 172.21.15.69:0/1380725412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff250108660 0x7ff25019e8c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff24400d770 tx=0x7ff24400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.909+0000 7ff24d600640 1 -- 172.21.15.69:0/1380725412 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244017070 con 0x7ff250108660 2024-04-24T09:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.909+0000 7ff24d600640 1 -- 172.21.15.69:0/1380725412 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff244009c30 con 0x7ff250108660 2024-04-24T09:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.909+0000 7ff257800640 1 -- 172.21.15.69:0/1380725412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2501a67d0 con 0x7ff250108660 2024-04-24T09:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.909+0000 7ff24d600640 1 -- 172.21.15.69:0/1380725412 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244003d00 con 0x7ff250108660 2024-04-24T09:06:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.909+0000 7ff257800640 1 -- 172.21.15.69:0/1380725412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2501a6d20 con 0x7ff250108660 2024-04-24T09:06:17.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.912+0000 7ff25a5f8740 1 Processor -- start 2024-04-24T09:06:17.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.912+0000 7ff24d600640 1 -- 172.21.15.69:0/1380725412 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff244003ea0 con 0x7ff250108660 2024-04-24T09:06:17.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.912+0000 7ff25a5f8740 1 -- start start 2024-04-24T09:06:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.913+0000 7ff25a5f8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37cdc090 0x562b37cdc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.913+0000 7ff25a5f8740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37cdca80 0x562b37ce1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.913+0000 7ff25a5f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37ce1bf0 0x562b37ce4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.913+0000 7ff25a5f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562b37c41b10 con 0x562b37ce1bf0 2024-04-24T09:06:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.913+0000 7ff25a5f8740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562b37be82b0 con 0x562b37cdca80 2024-04-24T09:06:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.913+0000 7ff25a5f8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562b37c3dad0 con 0x562b37cdc090 2024-04-24T09:06:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.913+0000 7ff24fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37ce1bf0 0x562b37ce4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.913+0000 7ff24fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37ce1bf0 0x562b37ce4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40140/0 (socket says 172.21.15.69:40140) 2024-04-24T09:06:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.913+0000 7ff24fe00640 1 -- 172.21.15.69:0/853872418 learned_addr learned my addr 172.21.15.69:0/853872418 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.913+0000 7ff254e00640 1 --2- 172.21.15.69:0/853872418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37cdca80 0x562b37ce1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.913+0000 7ff255800640 1 --2- 172.21.15.69:0/853872418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37cdc090 0x562b37cdc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.914+0000 7ff24fe00640 1 -- 172.21.15.69:0/853872418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37cdc090 msgr2=0x562b37cdc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:17.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.914+0000 7ff24fe00640 1 --2- 172.21.15.69:0/853872418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37cdc090 0x562b37cdc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.914+0000 7ff24fe00640 1 -- 172.21.15.69:0/853872418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37cdca80 msgr2=0x562b37ce1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:17.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.914+0000 7ff24fe00640 1 --2- 172.21.15.69:0/853872418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37cdca80 0x562b37ce1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.914+0000 7ff24fe00640 1 -- 172.21.15.69:0/853872418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b379542d0 con 0x562b37ce1bf0 2024-04-24T09:06:17.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.914+0000 7ff24d600640 1 --2- 172.21.15.69:0/1380725412 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff2380a3bb0 0x7ff2380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.914+0000 7ff24fe00640 1 --2- 172.21.15.69:0/853872418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37ce1bf0 0x562b37ce4080 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff23c006010 tx=0x7ff23c032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7707361718c529fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:17.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.914+0000 7ff24d600640 1 -- 172.21.15.69:0/1380725412 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff2440d0580 con 0x7ff250108660 2024-04-24T09:06:17.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.914+0000 7ff255800640 1 --2- 172.21.15.69:0/1380725412 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff2380a3bb0 0x7ff2380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.914+0000 7ff242000640 1 -- 172.21.15.69:0/853872418 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c03f070 con 0x562b37ce1bf0 2024-04-24T09:06:17.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.915+0000 7ff242000640 1 -- 172.21.15.69:0/853872418 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff23c031780 con 0x562b37ce1bf0 2024-04-24T09:06:17.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.915+0000 7ff242000640 1 -- 172.21.15.69:0/853872418 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c035070 con 0x562b37ce1bf0 2024-04-24T09:06:17.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.915+0000 7ff25a5f8740 1 -- 172.21.15.69:0/853872418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37ce1bf0 msgr2=0x562b37ce4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.915+0000 7ff255800640 1 --2- 172.21.15.69:0/1380725412 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff2380a3bb0 0x7ff2380a6070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff248009270 tx=0x7ff248009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:17.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.915+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/853872418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37ce1bf0 0x562b37ce4080 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff23c006010 tx=0x7ff23c032cc0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.915+0000 7ff25a5f8740 1 -- 172.21.15.69:0/853872418 shutdown_connections 2024-04-24T09:06:17.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.915+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/853872418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37ce1bf0 0x562b37ce4080 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.915+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/853872418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37cdca80 0x562b37ce1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.915+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/853872418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37cdc090 0x562b37cdc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.915+0000 7ff25a5f8740 1 -- 172.21.15.69:0/853872418 >> 172.21.15.69:0/853872418 conn(0x562b37cd7700 msgr2=0x562b37cd7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:17.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.915+0000 7ff25a5f8740 1 -- 172.21.15.69:0/853872418 shutdown_connections 2024-04-24T09:06:17.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.915+0000 7ff25a5f8740 1 -- 172.21.15.69:0/853872418 wait complete. 2024-04-24T09:06:17.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.916+0000 7ff25a5f8740 1 -- start start 2024-04-24T09:06:17.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.916+0000 7ff25a5f8740 1 Processor -- start 2024-04-24T09:06:17.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.916+0000 7ff25a5f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdc090 0x562b37ce6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.916+0000 7ff25a5f8740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37cdca80 0x562b37ce69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.916+0000 7ff25a5f8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37ce1bf0 0x562b37ced9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.916+0000 7ff25a5f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562b37ce4b40 con 0x562b37cdc090 2024-04-24T09:06:17.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.916+0000 7ff25a5f8740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562b37bfb590 con 0x562b37cdca80 2024-04-24T09:06:17.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.916+0000 7ff25a5f8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562b37ce4cc0 con 0x562b37ce1bf0 2024-04-24T09:06:17.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.917+0000 7ff25a5f8740 10 client.0 _ll_register_callbacks cb 0x562b37bde3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:17.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.917+0000 7ff254e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdc090 0x562b37ce6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.917+0000 7ff254e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdc090 0x562b37ce6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40144/0 (socket says 172.21.15.69:40144) 2024-04-24T09:06:17.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.917+0000 7ff254e00640 1 -- 172.21.15.69:0/439554567 learned_addr learned my addr 172.21.15.69:0/439554567 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:17.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.917+0000 7ff24fe00640 1 --2- 172.21.15.69:0/439554567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37cdca80 0x562b37ce69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.917+0000 7ff255800640 1 --2- 172.21.15.69:0/439554567 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37ce1bf0 0x562b37ced9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.917+0000 7ff254e00640 1 -- 172.21.15.69:0/439554567 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37ce1bf0 msgr2=0x562b37ced9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:17.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.917+0000 7ff254e00640 1 --2- 172.21.15.69:0/439554567 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37ce1bf0 0x562b37ced9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.917+0000 7ff254e00640 1 -- 172.21.15.69:0/439554567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37cdca80 msgr2=0x562b37ce69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:17.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.917+0000 7ff254e00640 1 --2- 172.21.15.69:0/439554567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37cdca80 0x562b37ce69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.917+0000 7ff254e00640 1 -- 172.21.15.69:0/439554567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b37a1e6b0 con 0x562b37cdc090 2024-04-24T09:06:17.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.917+0000 7ff24fe00640 1 --2- 172.21.15.69:0/439554567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37cdca80 0x562b37ce69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:17.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.917+0000 7ff254e00640 1 --2- 172.21.15.69:0/439554567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdc090 0x562b37ce6470 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff244098e70 tx=0x7ff244098800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6277169ee701528f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:17.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.917+0000 7ff236a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:06:17.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.918+0000 7ff236a00640 1 -- 172.21.15.69:0/439554567 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244020070 con 0x562b37cdc090 2024-04-24T09:06:17.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.918+0000 7ff236a00640 1 -- 172.21.15.69:0/439554567 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff2440cddd0 con 0x562b37cdc090 2024-04-24T09:06:17.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.918+0000 7ff25a5f8740 1 -- 172.21.15.69:0/439554567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff23c031780 con 0x562b37cdc090 2024-04-24T09:06:17.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.918+0000 7ff236a00640 1 -- 172.21.15.69:0/439554567 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff24409be00 con 0x562b37cdc090 2024-04-24T09:06:17.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.918+0000 7ff25a5f8740 10 client.15243 fetch_fsmap learned FSMap version 23 2024-04-24T09:06:17.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.918+0000 7ff25a5f8740 1 -- 172.21.15.69:0/439554567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562b37cf0e20 con 0x562b37cdc090 2024-04-24T09:06:17.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.918+0000 7ff236a00640 1 -- 172.21.15.69:0/439554567 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff23c031780 con 0x562b37cdc090 2024-04-24T09:06:17.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.918+0000 7ff236a00640 1 -- 172.21.15.69:0/439554567 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7ff244098b10 con 0x562b37cdc090 2024-04-24T09:06:17.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.918+0000 7ff25a5f8740 10 client.15243 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T09:06:17.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.918+0000 7ff25a5f8740 10 client.15243 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-24T09:06:17.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.918+0000 7ff25a5f8740 20 client.15243 populate_metadata read hostname 'smithi069' 2024-04-24T09:06:17.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.918+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/439554567 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x562b37cf34e0 0x562b37cf4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.918+0000 7ff25a5f8740 4 client.15243 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:17.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.919+0000 7ff25a5f8740 1 -- 172.21.15.69:0/439554567 --> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b37c4f530 con 0x562b37cf34e0 2024-04-24T09:06:17.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.919+0000 7ff24fe00640 1 --2- 172.21.15.69:0/439554567 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x562b37cf34e0 0x562b37cf4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.920+0000 7ff24fe00640 1 --2- 172.21.15.69:0/439554567 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x562b37cf34e0 0x562b37cf4950 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53a0b339682b9780 server_cookie=57f9c06cbbaac759 in_seq=0 out_seq=0 2024-04-24T09:06:17.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.920+0000 7ff236a00640 10 client.15243 ms_handle_connect on v2:172.21.15.81:6836/1973819593 2024-04-24T09:06:17.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.922+0000 7ff236a00640 1 -- 172.21.15.69:0/439554567 <== mds.1 v2:172.21.15.81:6836/1973819593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b37c4f530 con 0x562b37cf34e0 2024-04-24T09:06:17.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.922+0000 7ff236a00640 10 client.15243 handle_command_reply: tid=0 2024-04-24T09:06:17.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.922+0000 7ff25a5f8740 1 client.15243 shutdown 2024-04-24T09:06:17.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.923+0000 7ff25a5f8740 1 -- 172.21.15.69:0/439554567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdc090 msgr2=0x562b37ce6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.923+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/439554567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdc090 0x562b37ce6470 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff244098e70 tx=0x7ff244098800 comp rx=0 tx=0).stop 2024-04-24T09:06:17.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.923+0000 7ff25a5f8740 1 -- 172.21.15.69:0/439554567 shutdown_connections 2024-04-24T09:06:17.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.923+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/439554567 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x562b37cf34e0 0x562b37cf4950 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.923+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/439554567 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37ce1bf0 0x562b37ced9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.923+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/439554567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37cdca80 0x562b37ce69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.923+0000 7ff236a00640 0 client.15243 ms_handle_reset on v2:172.21.15.81:6836/1973819593 2024-04-24T09:06:17.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.923+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/439554567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdc090 0x562b37ce6470 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.923+0000 7ff25a5f8740 1 -- 172.21.15.69:0/439554567 >> 172.21.15.69:0/439554567 conn(0x562b37cd7700 msgr2=0x562b37cd68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:17.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.923+0000 7ff25a5f8740 1 -- 172.21.15.69:0/439554567 shutdown_connections 2024-04-24T09:06:17.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.923+0000 7ff25a5f8740 1 -- 172.21.15.69:0/439554567 wait complete. 2024-04-24T09:06:17.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.923+0000 7ff25a5f8740 20 client.15243 trim_cache size 0 max 16384 2024-04-24T09:06:17.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.943+0000 7ff25a5f8740 1 Processor -- start 2024-04-24T09:06:17.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.943+0000 7ff25a5f8740 1 -- start start 2024-04-24T09:06:17.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.943+0000 7ff25a5f8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37cdc090 0x562b37ce14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.943+0000 7ff25a5f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdca80 0x562b37ced9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.943+0000 7ff25a5f8740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37ce1bf0 0x562b37cd2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.943+0000 7ff25a5f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562b37cf2c90 con 0x562b37cdca80 2024-04-24T09:06:17.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.943+0000 7ff25a5f8740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562b37cf3360 con 0x562b37ce1bf0 2024-04-24T09:06:17.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.943+0000 7ff25a5f8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562b37cf3a30 con 0x562b37cdc090 2024-04-24T09:06:17.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.943+0000 7ff24fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdca80 0x562b37ced9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.943+0000 7ff24fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdca80 0x562b37ced9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40160/0 (socket says 172.21.15.69:40160) 2024-04-24T09:06:17.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.943+0000 7ff24fe00640 1 -- 172.21.15.69:0/997995789 learned_addr learned my addr 172.21.15.69:0/997995789 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:17.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.944+0000 7ff254e00640 1 --2- 172.21.15.69:0/997995789 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37cdc090 0x562b37ce14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.944+0000 7ff255800640 1 --2- 172.21.15.69:0/997995789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37ce1bf0 0x562b37cd2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.944+0000 7ff24fe00640 1 -- 172.21.15.69:0/997995789 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37cdc090 msgr2=0x562b37ce14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:17.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.944+0000 7ff24fe00640 1 --2- 172.21.15.69:0/997995789 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37cdc090 0x562b37ce14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.944+0000 7ff24fe00640 1 -- 172.21.15.69:0/997995789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37ce1bf0 msgr2=0x562b37cd2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:17.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.944+0000 7ff24fe00640 1 --2- 172.21.15.69:0/997995789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37ce1bf0 0x562b37cd2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.944+0000 7ff24fe00640 1 -- 172.21.15.69:0/997995789 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b37cdce60 con 0x562b37cdca80 2024-04-24T09:06:17.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.944+0000 7ff24fe00640 1 --2- 172.21.15.69:0/997995789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdca80 0x562b37ced9b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff23c005c40 tx=0x7ff23c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9be08f84f883afd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:17.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.944+0000 7ff241600640 1 -- 172.21.15.69:0/997995789 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c035040 con 0x562b37cdca80 2024-04-24T09:06:17.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.944+0000 7ff241600640 1 -- 172.21.15.69:0/997995789 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff23c052070 con 0x562b37cdca80 2024-04-24T09:06:17.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.944+0000 7ff241600640 1 -- 172.21.15.69:0/997995789 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c03f040 con 0x562b37cdca80 2024-04-24T09:06:17.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.945+0000 7ff25a5f8740 1 -- 172.21.15.69:0/997995789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdca80 msgr2=0x562b37ced9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.945+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/997995789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdca80 0x562b37ced9b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff23c005c40 tx=0x7ff23c02a410 comp rx=0 tx=0).stop 2024-04-24T09:06:17.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.945+0000 7ff25a5f8740 1 -- 172.21.15.69:0/997995789 shutdown_connections 2024-04-24T09:06:17.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.945+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/997995789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37ce1bf0 0x562b37cd2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.945+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/997995789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdca80 0x562b37ced9b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.945+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/997995789 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37cdc090 0x562b37ce14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.945+0000 7ff25a5f8740 1 -- 172.21.15.69:0/997995789 >> 172.21.15.69:0/997995789 conn(0x562b37cd7700 msgr2=0x562b37cd7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:17.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.945+0000 7ff25a5f8740 1 -- 172.21.15.69:0/997995789 shutdown_connections 2024-04-24T09:06:17.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.945+0000 7ff25a5f8740 1 -- 172.21.15.69:0/997995789 wait complete. 2024-04-24T09:06:17.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.945+0000 7ff25a5f8740 1 -- start start 2024-04-24T09:06:17.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.946+0000 7ff25a5f8740 1 Processor -- start 2024-04-24T09:06:17.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.946+0000 7ff25a5f8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37cdc090 0x562b37ce08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.946+0000 7ff25a5f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdca80 0x562b37ced980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.946+0000 7ff25a5f8740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37ce1bf0 0x562b37d57480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.946+0000 7ff25a5f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562b37cf47d0 con 0x562b37cdca80 2024-04-24T09:06:17.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.946+0000 7ff25a5f8740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562b37cf4100 con 0x562b37ce1bf0 2024-04-24T09:06:17.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.946+0000 7ff25a5f8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562b37cf4ea0 con 0x562b37cdc090 2024-04-24T09:06:17.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.946+0000 7ff25a5f8740 10 client.0 _ll_register_callbacks cb 0x7ff24400de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:17.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.946+0000 7ff24fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdca80 0x562b37ced980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.947+0000 7ff24fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdca80 0x562b37ced980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40170/0 (socket says 172.21.15.69:40170) 2024-04-24T09:06:17.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.947+0000 7ff24fe00640 1 -- 172.21.15.69:0/2401223641 learned_addr learned my addr 172.21.15.69:0/2401223641 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:17.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.947+0000 7ff254e00640 1 --2- 172.21.15.69:0/2401223641 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37cdc090 0x562b37ce08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.947+0000 7ff255800640 1 --2- 172.21.15.69:0/2401223641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37ce1bf0 0x562b37d57480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.947+0000 7ff24fe00640 1 -- 172.21.15.69:0/2401223641 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37cdc090 msgr2=0x562b37ce08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:17.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.947+0000 7ff24fe00640 1 --2- 172.21.15.69:0/2401223641 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37cdc090 0x562b37ce08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.947+0000 7ff24fe00640 1 -- 172.21.15.69:0/2401223641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37ce1bf0 msgr2=0x562b37d57480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:17.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.947+0000 7ff24fe00640 1 --2- 172.21.15.69:0/2401223641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37ce1bf0 0x562b37d57480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.947+0000 7ff24fe00640 1 -- 172.21.15.69:0/2401223641 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b37d57cb0 con 0x562b37cdca80 2024-04-24T09:06:17.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.947+0000 7ff254e00640 1 --2- 172.21.15.69:0/2401223641 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37cdc090 0x562b37ce08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:17.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.947+0000 7ff24fe00640 1 --2- 172.21.15.69:0/2401223641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdca80 0x562b37ced980 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0314e0 tx=0x7ff23c005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3045de0fb6304e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:17.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.947+0000 7ff236a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:06:17.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.948+0000 7ff236a00640 1 -- 172.21.15.69:0/2401223641 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c05b060 con 0x562b37cdca80 2024-04-24T09:06:17.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.948+0000 7ff236a00640 1 -- 172.21.15.69:0/2401223641 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff23c004230 con 0x562b37cdca80 2024-04-24T09:06:17.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.948+0000 7ff25a5f8740 1 -- 172.21.15.69:0/2401223641 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff23c052070 con 0x562b37cdca80 2024-04-24T09:06:17.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.948+0000 7ff236a00640 1 -- 172.21.15.69:0/2401223641 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff23c035070 con 0x562b37cdca80 2024-04-24T09:06:17.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.948+0000 7ff25a5f8740 10 client.15249 fetch_fsmap learned FSMap version 23 2024-04-24T09:06:17.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.948+0000 7ff25a5f8740 1 -- 172.21.15.69:0/2401223641 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562b37d58fb0 con 0x562b37cdca80 2024-04-24T09:06:17.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.948+0000 7ff236a00640 1 -- 172.21.15.69:0/2401223641 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff23c052070 con 0x562b37cdca80 2024-04-24T09:06:17.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.948+0000 7ff236a00640 1 -- 172.21.15.69:0/2401223641 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7ff23c003660 con 0x562b37cdca80 2024-04-24T09:06:17.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.948+0000 7ff25a5f8740 10 client.15249 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T09:06:17.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.948+0000 7ff25a5f8740 10 client.15249 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-24T09:06:17.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.948+0000 7ff25a5f8740 20 client.15249 populate_metadata read hostname 'smithi069' 2024-04-24T09:06:17.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.948+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/2401223641 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x562b37d5d590 0x562b37d5c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:17.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.948+0000 7ff25a5f8740 4 client.15249 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:06:17.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.948+0000 7ff25a5f8740 1 -- 172.21.15.69:0/2401223641 --> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] -- command(tid 0: {"prefix": "status"}) -- 0x562b37c493a0 con 0x562b37d5d590 2024-04-24T09:06:17.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.949+0000 7ff254e00640 1 --2- 172.21.15.69:0/2401223641 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x562b37d5d590 0x562b37d5c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.950+0000 7ff236a00640 10 client.15249 ms_handle_connect on v2:172.21.15.81:6836/1973819593 2024-04-24T09:06:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.950+0000 7ff254e00640 1 --2- 172.21.15.69:0/2401223641 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x562b37d5d590 0x562b37d5c2a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aeed17836393d722 server_cookie=11d14047ba61c0be in_seq=0 out_seq=0 2024-04-24T09:06:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.950+0000 7ff236a00640 1 -- 172.21.15.69:0/2401223641 <== mds.1 v2:172.21.15.81:6836/1973819593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562b37c493a0 con 0x562b37d5d590 2024-04-24T09:06:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.950+0000 7ff236a00640 10 client.15249 handle_command_reply: tid=0 2024-04-24T09:06:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.950+0000 7ff25a5f8740 1 client.15249 shutdown 2024-04-24T09:06:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.951+0000 7ff25a5f8740 1 -- 172.21.15.69:0/2401223641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdca80 msgr2=0x562b37ced980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.951+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/2401223641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdca80 0x562b37ced980 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0314e0 tx=0x7ff23c005520 comp rx=0 tx=0).stop 2024-04-24T09:06:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.951+0000 7ff25a5f8740 1 -- 172.21.15.69:0/2401223641 shutdown_connections 2024-04-24T09:06:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.951+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/2401223641 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x562b37d5d590 0x562b37d5c2a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.951+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/2401223641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b37ce1bf0 0x562b37d57480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.951+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/2401223641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b37cdca80 0x562b37ced980 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.951+0000 7ff25a5f8740 1 --2- 172.21.15.69:0/2401223641 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b37cdc090 0x562b37ce08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.951+0000 7ff236a00640 0 client.15249 ms_handle_reset on v2:172.21.15.81:6836/1973819593 2024-04-24T09:06:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.951+0000 7ff25a5f8740 1 -- 172.21.15.69:0/2401223641 >> 172.21.15.69:0/2401223641 conn(0x562b37cd7700 msgr2=0x562b37cdd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.951+0000 7ff25a5f8740 1 -- 172.21.15.69:0/2401223641 shutdown_connections 2024-04-24T09:06:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.951+0000 7ff25a5f8740 1 -- 172.21.15.69:0/2401223641 wait complete. 2024-04-24T09:06:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.951+0000 7ff25a5f8740 20 client.15249 trim_cache size 0 max 16384 2024-04-24T09:06:17.953 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:06:17.953 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "ef1ec7e8-0217-11ef-bc93-c7b262605968", 2024-04-24T09:06:17.954 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 1, 2024-04-24T09:06:17.954 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 34496, 2024-04-24T09:06:17.954 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-24T09:06:17.954 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-24T09:06:17.954 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-24T09:06:17.954 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 105.362753321, 2024-04-24T09:06:17.954 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 23, 2024-04-24T09:06:17.954 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 92, 2024-04-24T09:06:17.954 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 92, 2024-04-24T09:06:17.954 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 156.72762443799999 2024-04-24T09:06:17.954 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:06:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.952+0000 7ff257800640 1 -- 172.21.15.69:0/1380725412 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff2380a3bb0 msgr2=0x7ff2380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.952+0000 7ff257800640 1 --2- 172.21.15.69:0/1380725412 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff2380a3bb0 0x7ff2380a6070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff248009270 tx=0x7ff248009500 comp rx=0 tx=0).stop 2024-04-24T09:06:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.952+0000 7ff257800640 1 -- 172.21.15.69:0/1380725412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff250108660 msgr2=0x7ff25019e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.952+0000 7ff257800640 1 --2- 172.21.15.69:0/1380725412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff250108660 0x7ff25019e8c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff24400d770 tx=0x7ff24400dc30 comp rx=0 tx=0).stop 2024-04-24T09:06:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.952+0000 7ff257800640 1 -- 172.21.15.69:0/1380725412 shutdown_connections 2024-04-24T09:06:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.952+0000 7ff257800640 1 --2- 172.21.15.69:0/1380725412 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff25010a210 0x7ff2501a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.952+0000 7ff257800640 1 --2- 172.21.15.69:0/1380725412 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7ff2380a3bb0 0x7ff2380a6070 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.952+0000 7ff257800640 1 --2- 172.21.15.69:0/1380725412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff250109850 0x7ff25019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.952+0000 7ff257800640 1 --2- 172.21.15.69:0/1380725412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff250108660 0x7ff25019e8c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.952+0000 7ff257800640 1 -- 172.21.15.69:0/1380725412 >> 172.21.15.69:0/1380725412 conn(0x7ff250077600 msgr2=0x7ff25010cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:17.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.952+0000 7ff257800640 1 -- 172.21.15.69:0/1380725412 shutdown_connections 2024-04-24T09:06:17.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:17.952+0000 7ff257800640 1 -- 172.21.15.69:0/1380725412 wait complete. 2024-04-24T09:06:17.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:17 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2287237924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:17.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:17 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2853008003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:17.967 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T09:06:18.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.162+0000 7f752a200640 1 -- 172.21.15.69:0/3290840025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7524108660 msgr2=0x7f7524108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:18.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.162+0000 7f752a200640 1 --2- 172.21.15.69:0/3290840025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7524108660 0x7f7524108a80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7514009900 tx=0x7f751402a410 comp rx=0 tx=0).stop 2024-04-24T09:06:18.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.162+0000 7f752a200640 1 -- 172.21.15.69:0/3290840025 shutdown_connections 2024-04-24T09:06:18.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.162+0000 7f752a200640 1 --2- 172.21.15.69:0/3290840025 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f752410a210 0x7f7524110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.162+0000 7f752a200640 1 --2- 172.21.15.69:0/3290840025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7524109850 0x7f7524109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.162+0000 7f752a200640 1 --2- 172.21.15.69:0/3290840025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7524108660 0x7f7524108a80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.162+0000 7f752a200640 1 -- 172.21.15.69:0/3290840025 >> 172.21.15.69:0/3290840025 conn(0x7f7524077600 msgr2=0x7f7524079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:18.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.162+0000 7f752a200640 1 -- 172.21.15.69:0/3290840025 shutdown_connections 2024-04-24T09:06:18.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.162+0000 7f752a200640 1 -- 172.21.15.69:0/3290840025 wait complete. 2024-04-24T09:06:18.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.163+0000 7f752a200640 1 Processor -- start 2024-04-24T09:06:18.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.164+0000 7f752a200640 1 -- start start 2024-04-24T09:06:18.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.164+0000 7f752a200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7524108660 0x7f752419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.164+0000 7f752a200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7524109850 0x7f752419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.164+0000 7f752a200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f752410a210 0x7f75241a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.164+0000 7f752a200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7524114190 con 0x7f752410a210 2024-04-24T09:06:18.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.164+0000 7f752a200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7524114010 con 0x7f7524108660 2024-04-24T09:06:18.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.164+0000 7f752a200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7524114310 con 0x7f7524109850 2024-04-24T09:06:18.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.164+0000 7f7523e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f752410a210 0x7f75241a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:18.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.164+0000 7f7523e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f752410a210 0x7f75241a5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40188/0 (socket says 172.21.15.69:40188) 2024-04-24T09:06:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.164+0000 7f7523e00640 1 -- 172.21.15.69:0/169876909 learned_addr learned my addr 172.21.15.69:0/169876909 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.164+0000 7f7523e00640 1 -- 172.21.15.69:0/169876909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7524109850 msgr2=0x7f752419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.164+0000 7f7523400640 1 --2- 172.21.15.69:0/169876909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7524108660 0x7f752419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.164+0000 7f7523e00640 1 --2- 172.21.15.69:0/169876909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7524109850 0x7f752419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.165+0000 7f7523e00640 1 -- 172.21.15.69:0/169876909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7524108660 msgr2=0x7f752419e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.165+0000 7f7523e00640 1 --2- 172.21.15.69:0/169876909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7524108660 0x7f752419e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.165+0000 7f7523e00640 1 -- 172.21.15.69:0/169876909 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75241a64e0 con 0x7f752410a210 2024-04-24T09:06:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.165+0000 7f7523e00640 1 --2- 172.21.15.69:0/169876909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f752410a210 0x7f75241a5e70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f751c00eaa0 tx=0x7f751c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.165+0000 7f7520c00640 1 -- 172.21.15.69:0/169876909 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c00cdb0 con 0x7f752410a210 2024-04-24T09:06:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.165+0000 7f7520c00640 1 -- 172.21.15.69:0/169876909 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f751c005020 con 0x7f752410a210 2024-04-24T09:06:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.165+0000 7f752a200640 1 -- 172.21.15.69:0/169876909 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75241a67d0 con 0x7f752410a210 2024-04-24T09:06:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.165+0000 7f7520c00640 1 -- 172.21.15.69:0/169876909 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c0116d0 con 0x7f752410a210 2024-04-24T09:06:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.165+0000 7f752a200640 1 -- 172.21.15.69:0/169876909 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75241a6d20 con 0x7f752410a210 2024-04-24T09:06:18.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.168+0000 7f752d065740 1 Processor -- start 2024-04-24T09:06:18.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.169+0000 7f7520c00640 1 -- 172.21.15.69:0/169876909 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f751c0051d0 con 0x7f752410a210 2024-04-24T09:06:18.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.169+0000 7f752d065740 1 -- start start 2024-04-24T09:06:18.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.169+0000 7f752d065740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d78090 0x562861d784b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.169+0000 7f752d065740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d7d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.169+0000 7f752d065740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d7dbf0 0x562861d80080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.169+0000 7f752d065740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562861cddb10 con 0x562861d78a80 2024-04-24T09:06:18.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.169+0000 7f752d065740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562861c842b0 con 0x562861d78090 2024-04-24T09:06:18.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.169+0000 7f752d065740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562861cd9ad0 con 0x562861d7dbf0 2024-04-24T09:06:18.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.170+0000 7f7523400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d7d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:18.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.170+0000 7f7523400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d7d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40198/0 (socket says 172.21.15.69:40198) 2024-04-24T09:06:18.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.170+0000 7f7523400640 1 -- 172.21.15.69:0/1500740219 learned_addr learned my addr 172.21.15.69:0/1500740219 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:18.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.170+0000 7f751be00640 1 --2- 172.21.15.69:0/1500740219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d78090 0x562861d784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:18.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.170+0000 7f751be00640 1 --2- 172.21.15.69:0/1500740219 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d7dbf0 0x562861d80080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:18.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.170+0000 7f7523400640 1 -- 172.21.15.69:0/1500740219 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d7dbf0 msgr2=0x562861d80080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:18.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.170+0000 7f7523400640 1 --2- 172.21.15.69:0/1500740219 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d7dbf0 0x562861d80080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.170+0000 7f7523400640 1 -- 172.21.15.69:0/1500740219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d78090 msgr2=0x562861d784b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:18.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.170+0000 7f7523400640 1 --2- 172.21.15.69:0/1500740219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d78090 0x562861d784b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.170+0000 7f7523400640 1 -- 172.21.15.69:0/1500740219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628619f02d0 con 0x562861d78a80 2024-04-24T09:06:18.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.170+0000 7f751be00640 1 --2- 172.21.15.69:0/1500740219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d78090 0x562861d784b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:18.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.171+0000 7f7520c00640 1 --2- 172.21.15.69:0/169876909 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f75080a3bb0 0x7f75080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.171+0000 7f7523400640 1 --2- 172.21.15.69:0/1500740219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d7d580 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7514006010 tx=0x7f7514009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebdd8be68cd871f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:18.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.171+0000 7f7520c00640 1 -- 172.21.15.69:0/169876909 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f751c0d09a0 con 0x7f752410a210 2024-04-24T09:06:18.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.171+0000 7f7513e00640 1 -- 172.21.15.69:0/1500740219 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751403f070 con 0x562861d78a80 2024-04-24T09:06:18.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.171+0000 7f751be00640 1 --2- 172.21.15.69:0/169876909 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f75080a3bb0 0x7f75080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:18.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.171+0000 7f7513e00640 1 -- 172.21.15.69:0/1500740219 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7514032e40 con 0x562861d78a80 2024-04-24T09:06:18.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.171+0000 7f7513e00640 1 -- 172.21.15.69:0/1500740219 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514035070 con 0x562861d78a80 2024-04-24T09:06:18.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.172+0000 7f752d065740 1 -- 172.21.15.69:0/1500740219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 msgr2=0x562861d7d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:18.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.172+0000 7f752d065740 1 --2- 172.21.15.69:0/1500740219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d7d580 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7514006010 tx=0x7f7514009680 comp rx=0 tx=0).stop 2024-04-24T09:06:18.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.172+0000 7f751be00640 1 --2- 172.21.15.69:0/169876909 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f75080a3bb0 0x7f75080a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f750c0058c0 tx=0x7f750c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:18.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.172+0000 7f752d065740 1 -- 172.21.15.69:0/1500740219 shutdown_connections 2024-04-24T09:06:18.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.172+0000 7f752d065740 1 --2- 172.21.15.69:0/1500740219 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d7dbf0 0x562861d80080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.172+0000 7f752d065740 1 --2- 172.21.15.69:0/1500740219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d7d580 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.172+0000 7f752d065740 1 --2- 172.21.15.69:0/1500740219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d78090 0x562861d784b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.172+0000 7f752d065740 1 -- 172.21.15.69:0/1500740219 >> 172.21.15.69:0/1500740219 conn(0x562861d73700 msgr2=0x562861d73b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:18.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.172+0000 7f752d065740 1 -- 172.21.15.69:0/1500740219 shutdown_connections 2024-04-24T09:06:18.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.172+0000 7f752d065740 1 -- 172.21.15.69:0/1500740219 wait complete. 2024-04-24T09:06:18.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.172+0000 7f752d065740 1 -- start start 2024-04-24T09:06:18.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.172+0000 7f752d065740 1 Processor -- start 2024-04-24T09:06:18.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.173+0000 7f752d065740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d78090 0x562861d82420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.173+0000 7f752d065740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d82960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.173+0000 7f752d065740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d7dbf0 0x562861d899a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.173+0000 7f752d065740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562861d80b40 con 0x562861d78a80 2024-04-24T09:06:18.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.173+0000 7f752d065740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562861c97590 con 0x562861d7dbf0 2024-04-24T09:06:18.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.173+0000 7f752d065740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562861d80cc0 con 0x562861d78090 2024-04-24T09:06:18.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.173+0000 7f752d065740 10 client.0 _ll_register_callbacks cb 0x562861c7a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:18.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.173+0000 7f751be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d82960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:18.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.173+0000 7f751be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d82960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40208/0 (socket says 172.21.15.69:40208) 2024-04-24T09:06:18.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.173+0000 7f751be00640 1 -- 172.21.15.69:0/1901963003 learned_addr learned my addr 172.21.15.69:0/1901963003 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:18.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.173+0000 7f7523400640 1 --2- 172.21.15.69:0/1901963003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d78090 0x562861d82420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:18.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.173+0000 7f7523e00640 1 --2- 172.21.15.69:0/1901963003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d7dbf0 0x562861d899a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:18.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.174+0000 7f751be00640 1 -- 172.21.15.69:0/1901963003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d78090 msgr2=0x562861d82420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:18.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.174+0000 7f751be00640 1 --2- 172.21.15.69:0/1901963003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d78090 0x562861d82420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.174+0000 7f751be00640 1 -- 172.21.15.69:0/1901963003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d7dbf0 msgr2=0x562861d899a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:18.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.174+0000 7f751be00640 1 --2- 172.21.15.69:0/1901963003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d7dbf0 0x562861d899a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.174+0000 7f751be00640 1 -- 172.21.15.69:0/1901963003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562861aba6b0 con 0x562861d78a80 2024-04-24T09:06:18.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.174+0000 7f7523400640 1 --2- 172.21.15.69:0/1901963003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d78090 0x562861d82420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:18.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.174+0000 7f751be00640 1 --2- 172.21.15.69:0/1901963003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d82960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f750c00fbb0 tx=0x7f750c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672b92299e652e91 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:18.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.174+0000 7f7510c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:06:18.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.174+0000 7f7510c00640 1 -- 172.21.15.69:0/1901963003 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c018070 con 0x562861d78a80 2024-04-24T09:06:18.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.174+0000 7f7510c00640 1 -- 172.21.15.69:0/1901963003 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f750c01dcc0 con 0x562861d78a80 2024-04-24T09:06:18.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.174+0000 7f752d065740 1 -- 172.21.15.69:0/1901963003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562861ceb530 con 0x562861d78a80 2024-04-24T09:06:18.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.174+0000 7f7510c00640 1 -- 172.21.15.69:0/1901963003 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f750c029480 con 0x562861d78a80 2024-04-24T09:06:18.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.175+0000 7f752d065740 10 client.15261 fetch_fsmap learned FSMap version 23 2024-04-24T09:06:18.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.175+0000 7f752d065740 1 -- 172.21.15.69:0/1901963003 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562861d8cde0 con 0x562861d78a80 2024-04-24T09:06:18.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.175+0000 7f7510c00640 1 -- 172.21.15.69:0/1901963003 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562861ceb530 con 0x562861d78a80 2024-04-24T09:06:18.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.175+0000 7f7510c00640 1 -- 172.21.15.69:0/1901963003 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f750c01e6b0 con 0x562861d78a80 2024-04-24T09:06:18.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.175+0000 7f752d065740 10 client.15261 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T09:06:18.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.175+0000 7f752d065740 10 client.15261 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T09:06:18.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.175+0000 7f752d065740 20 client.15261 populate_metadata read hostname 'smithi069' 2024-04-24T09:06:18.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.175+0000 7f752d065740 1 --2- 172.21.15.69:0/1901963003 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562861d8f4e0 0x562861d90950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.175+0000 7f752d065740 4 client.15261 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:18.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.175+0000 7f752d065740 1 -- 172.21.15.69:0/1901963003 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562861ce53a0 con 0x562861d8f4e0 2024-04-24T09:06:18.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.176+0000 7f7523400640 1 --2- 172.21.15.69:0/1901963003 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562861d8f4e0 0x562861d90950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:18.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.177+0000 7f7523400640 1 --2- 172.21.15.69:0/1901963003 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562861d8f4e0 0x562861d90950 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec87a4ddc896c548 server_cookie=d59efad69a53b11d in_seq=0 out_seq=0 2024-04-24T09:06:18.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.177+0000 7f7510c00640 10 client.15261 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:06:18.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.179+0000 7f7510c00640 1 -- 172.21.15.69:0/1901963003 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562861ce53a0 con 0x562861d8f4e0 2024-04-24T09:06:18.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.179+0000 7f7510c00640 10 client.15261 handle_command_reply: tid=0 2024-04-24T09:06:18.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.179+0000 7f752d065740 1 client.15261 shutdown 2024-04-24T09:06:18.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.179+0000 7f752d065740 1 -- 172.21.15.69:0/1901963003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 msgr2=0x562861d82960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:18.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.179+0000 7f752d065740 1 --2- 172.21.15.69:0/1901963003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d82960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f750c00fbb0 tx=0x7f750c0046c0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.179+0000 7f752d065740 1 -- 172.21.15.69:0/1901963003 shutdown_connections 2024-04-24T09:06:18.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.179+0000 7f752d065740 1 --2- 172.21.15.69:0/1901963003 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562861d8f4e0 0x562861d90950 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.179+0000 7f752d065740 1 --2- 172.21.15.69:0/1901963003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d7dbf0 0x562861d899a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.179+0000 7f752d065740 1 --2- 172.21.15.69:0/1901963003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d82960 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.179+0000 7f752d065740 1 --2- 172.21.15.69:0/1901963003 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d78090 0x562861d82420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.179+0000 7f7510c00640 0 client.15261 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:06:18.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.179+0000 7f752d065740 1 -- 172.21.15.69:0/1901963003 >> 172.21.15.69:0/1901963003 conn(0x562861d73700 msgr2=0x562861d728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:18.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.179+0000 7f752d065740 1 -- 172.21.15.69:0/1901963003 shutdown_connections 2024-04-24T09:06:18.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.179+0000 7f752d065740 1 -- 172.21.15.69:0/1901963003 wait complete. 2024-04-24T09:06:18.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.179+0000 7f752d065740 20 client.15261 trim_cache size 0 max 16384 2024-04-24T09:06:18.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.199+0000 7f752d065740 1 Processor -- start 2024-04-24T09:06:18.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.199+0000 7f752d065740 1 -- start start 2024-04-24T09:06:18.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.199+0000 7f752d065740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d78090 0x562861d7d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.199+0000 7f752d065740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d89960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.199+0000 7f752d065740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d7dbf0 0x562861d6e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.199+0000 7f752d065740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562861d8ec90 con 0x562861d78a80 2024-04-24T09:06:18.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.199+0000 7f752d065740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562861d8f360 con 0x562861d7dbf0 2024-04-24T09:06:18.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.199+0000 7f752d065740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562861d8fa30 con 0x562861d78090 2024-04-24T09:06:18.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.200+0000 7f751be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d89960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:18.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.200+0000 7f751be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d89960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40214/0 (socket says 172.21.15.69:40214) 2024-04-24T09:06:18.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.200+0000 7f751be00640 1 -- 172.21.15.69:0/849567504 learned_addr learned my addr 172.21.15.69:0/849567504 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:18.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.200+0000 7f7523e00640 1 --2- 172.21.15.69:0/849567504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d7dbf0 0x562861d6e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:18.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.200+0000 7f7523400640 1 --2- 172.21.15.69:0/849567504 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d78090 0x562861d7d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:18.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.200+0000 7f751be00640 1 -- 172.21.15.69:0/849567504 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d78090 msgr2=0x562861d7d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:18.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.200+0000 7f751be00640 1 --2- 172.21.15.69:0/849567504 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d78090 0x562861d7d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.200+0000 7f751be00640 1 -- 172.21.15.69:0/849567504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d7dbf0 msgr2=0x562861d6e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:18.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.200+0000 7f751be00640 1 --2- 172.21.15.69:0/849567504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d7dbf0 0x562861d6e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.200+0000 7f751be00640 1 -- 172.21.15.69:0/849567504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562861d64e20 con 0x562861d78a80 2024-04-24T09:06:18.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.201+0000 7f751be00640 1 --2- 172.21.15.69:0/849567504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d89960 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f750c012d20 tx=0x7f750c01d800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f025c68cbfa2b77 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:18.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.201+0000 7f7513400640 1 -- 172.21.15.69:0/849567504 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c03a070 con 0x562861d78a80 2024-04-24T09:06:18.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.201+0000 7f7513400640 1 -- 172.21.15.69:0/849567504 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f750c0044f0 con 0x562861d78a80 2024-04-24T09:06:18.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.201+0000 7f7513400640 1 -- 172.21.15.69:0/849567504 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c029820 con 0x562861d78a80 2024-04-24T09:06:18.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.201+0000 7f752d065740 1 -- 172.21.15.69:0/849567504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 msgr2=0x562861d89960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:18.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.201+0000 7f752d065740 1 --2- 172.21.15.69:0/849567504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d89960 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f750c012d20 tx=0x7f750c01d800 comp rx=0 tx=0).stop 2024-04-24T09:06:18.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.201+0000 7f752d065740 1 -- 172.21.15.69:0/849567504 shutdown_connections 2024-04-24T09:06:18.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.201+0000 7f752d065740 1 --2- 172.21.15.69:0/849567504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d7dbf0 0x562861d6e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.201+0000 7f752d065740 1 --2- 172.21.15.69:0/849567504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78a80 0x562861d89960 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.201+0000 7f752d065740 1 --2- 172.21.15.69:0/849567504 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d78090 0x562861d7d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.201+0000 7f752d065740 1 -- 172.21.15.69:0/849567504 >> 172.21.15.69:0/849567504 conn(0x562861d73700 msgr2=0x562861d73ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:18.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.202+0000 7f752d065740 1 -- 172.21.15.69:0/849567504 shutdown_connections 2024-04-24T09:06:18.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.202+0000 7f752d065740 1 -- 172.21.15.69:0/849567504 wait complete. 2024-04-24T09:06:18.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.202+0000 7f752d065740 1 -- start start 2024-04-24T09:06:18.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.202+0000 7f752d065740 1 Processor -- start 2024-04-24T09:06:18.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.202+0000 7f752d065740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78090 0x562861d878b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.202+0000 7f752d065740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d78a80 0x562861df1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.202+0000 7f752d065740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d7dbf0 0x562861df39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.202+0000 7f752d065740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562861d907d0 con 0x562861d78090 2024-04-24T09:06:18.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.202+0000 7f752d065740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562861d90100 con 0x562861d7dbf0 2024-04-24T09:06:18.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.202+0000 7f752d065740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562861d90ea0 con 0x562861d78a80 2024-04-24T09:06:18.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.203+0000 7f752d065740 10 client.0 _ll_register_callbacks cb 0x7f750c00f830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:18.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.203+0000 7f7523400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78090 0x562861d878b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:18.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.203+0000 7f7523400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78090 0x562861d878b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40218/0 (socket says 172.21.15.69:40218) 2024-04-24T09:06:18.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.203+0000 7f7523400640 1 -- 172.21.15.69:0/2212545879 learned_addr learned my addr 172.21.15.69:0/2212545879 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:18.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.203+0000 7f751be00640 1 --2- 172.21.15.69:0/2212545879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d78a80 0x562861df1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:18.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.203+0000 7f7523400640 1 -- 172.21.15.69:0/2212545879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d78a80 msgr2=0x562861df1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:18.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.203+0000 7f7523400640 1 --2- 172.21.15.69:0/2212545879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d78a80 0x562861df1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.203+0000 7f7523400640 1 -- 172.21.15.69:0/2212545879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d7dbf0 msgr2=0x562861df39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:18.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.203+0000 7f7523400640 1 --2- 172.21.15.69:0/2212545879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d7dbf0 0x562861df39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.203+0000 7f7523400640 1 -- 172.21.15.69:0/2212545879 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562861df3f30 con 0x562861d78090 2024-04-24T09:06:18.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.203+0000 7f7523400640 1 --2- 172.21.15.69:0/2212545879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78090 0x562861d878b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7514009900 tx=0x7f7514005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17e760704f77cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:18.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.203+0000 7f7510c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:06:18.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.204+0000 7f7510c00640 1 -- 172.21.15.69:0/2212545879 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514050070 con 0x562861d78090 2024-04-24T09:06:18.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.204+0000 7f752d065740 1 -- 172.21.15.69:0/2212545879 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f750c0044f0 con 0x562861d78090 2024-04-24T09:06:18.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.204+0000 7f7510c00640 1 -- 172.21.15.69:0/2212545879 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7514004040 con 0x562861d78090 2024-04-24T09:06:18.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.204+0000 7f7510c00640 1 -- 172.21.15.69:0/2212545879 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f751405c050 con 0x562861d78090 2024-04-24T09:06:18.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.204+0000 7f752d065740 10 client.15267 fetch_fsmap learned FSMap version 23 2024-04-24T09:06:18.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.204+0000 7f752d065740 1 -- 172.21.15.69:0/2212545879 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562861df5230 con 0x562861d78090 2024-04-24T09:06:18.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.204+0000 7f7510c00640 1 -- 172.21.15.69:0/2212545879 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f750c0044f0 con 0x562861d78090 2024-04-24T09:06:18.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.204+0000 7f7510c00640 1 -- 172.21.15.69:0/2212545879 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f7514031950 con 0x562861d78090 2024-04-24T09:06:18.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.204+0000 7f752d065740 10 client.15267 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T09:06:18.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.204+0000 7f752d065740 10 client.15267 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T09:06:18.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.204+0000 7f752d065740 20 client.15267 populate_metadata read hostname 'smithi069' 2024-04-24T09:06:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.204+0000 7f752d065740 1 --2- 172.21.15.69:0/2212545879 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562861df9a50 0x562861df8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.204+0000 7f752d065740 4 client.15267 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:06:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.204+0000 7f752d065740 1 -- 172.21.15.69:0/2212545879 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "status"}) -- 0x562861c61860 con 0x562861df9a50 2024-04-24T09:06:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.205+0000 7f751be00640 1 --2- 172.21.15.69:0/2212545879 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562861df9a50 0x562861df8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.206+0000 7f751be00640 1 --2- 172.21.15.69:0/2212545879 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562861df9a50 0x562861df8760 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d221b561ff0bf4 server_cookie=e8a063f0d184b5c5 in_seq=0 out_seq=0 2024-04-24T09:06:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.206+0000 7f7510c00640 10 client.15267 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:06:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.206+0000 7f7510c00640 1 -- 172.21.15.69:0/2212545879 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562861c61860 con 0x562861df9a50 2024-04-24T09:06:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.206+0000 7f7510c00640 10 client.15267 handle_command_reply: tid=0 2024-04-24T09:06:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.206+0000 7f752d065740 1 client.15267 shutdown 2024-04-24T09:06:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.207+0000 7f752d065740 1 -- 172.21.15.69:0/2212545879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78090 msgr2=0x562861d878b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.207+0000 7f752d065740 1 --2- 172.21.15.69:0/2212545879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78090 0x562861d878b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7514009900 tx=0x7f7514005a20 comp rx=0 tx=0).stop 2024-04-24T09:06:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.207+0000 7f752d065740 1 -- 172.21.15.69:0/2212545879 shutdown_connections 2024-04-24T09:06:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.207+0000 7f752d065740 1 --2- 172.21.15.69:0/2212545879 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562861df9a50 0x562861df8760 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.207+0000 7f752d065740 1 --2- 172.21.15.69:0/2212545879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562861d7dbf0 0x562861df39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.207+0000 7f752d065740 1 --2- 172.21.15.69:0/2212545879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562861d78a80 0x562861df1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.207+0000 7f7510c00640 0 client.15267 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:06:18.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.207+0000 7f752d065740 1 --2- 172.21.15.69:0/2212545879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562861d78090 0x562861d878b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.207+0000 7f752d065740 1 -- 172.21.15.69:0/2212545879 >> 172.21.15.69:0/2212545879 conn(0x562861d73700 msgr2=0x562861d79e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:18.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.207+0000 7f752d065740 1 -- 172.21.15.69:0/2212545879 shutdown_connections 2024-04-24T09:06:18.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.207+0000 7f752d065740 1 -- 172.21.15.69:0/2212545879 wait complete. 2024-04-24T09:06:18.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.207+0000 7f752d065740 20 client.15267 trim_cache size 0 max 16384 2024-04-24T09:06:18.209 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:06:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "ef1ec7e8-0217-11ef-bc93-c7b262605968", 2024-04-24T09:06:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-04-24T09:06:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 34532, 2024-04-24T09:06:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-24T09:06:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-24T09:06:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-24T09:06:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 135.66898719100001, 2024-04-24T09:06:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 23, 2024-04-24T09:06:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 92, 2024-04-24T09:06:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 92, 2024-04-24T09:06:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 140.73715794200001 2024-04-24T09:06:18.210 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:06:18.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.208+0000 7f752a200640 1 -- 172.21.15.69:0/169876909 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f75080a3bb0 msgr2=0x7f75080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:18.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.208+0000 7f752a200640 1 --2- 172.21.15.69:0/169876909 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f75080a3bb0 0x7f75080a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f750c0058c0 tx=0x7f750c005e50 comp rx=0 tx=0).stop 2024-04-24T09:06:18.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.208+0000 7f752a200640 1 -- 172.21.15.69:0/169876909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f752410a210 msgr2=0x7f75241a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:18.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.208+0000 7f752a200640 1 --2- 172.21.15.69:0/169876909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f752410a210 0x7f75241a5e70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f751c00eaa0 tx=0x7f751c00ef60 comp rx=0 tx=0).stop 2024-04-24T09:06:18.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.208+0000 7f752a200640 1 -- 172.21.15.69:0/169876909 shutdown_connections 2024-04-24T09:06:18.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.208+0000 7f752a200640 1 --2- 172.21.15.69:0/169876909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f752410a210 0x7f75241a5e70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.208+0000 7f752a200640 1 --2- 172.21.15.69:0/169876909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7524109850 0x7f752419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.208+0000 7f752a200640 1 --2- 172.21.15.69:0/169876909 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f75080a3bb0 0x7f75080a6070 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.208+0000 7f752a200640 1 --2- 172.21.15.69:0/169876909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7524108660 0x7f752419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:18.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.208+0000 7f752a200640 1 -- 172.21.15.69:0/169876909 >> 172.21.15.69:0/169876909 conn(0x7f7524077600 msgr2=0x7f752410cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:18.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.208+0000 7f752a200640 1 -- 172.21.15.69:0/169876909 shutdown_connections 2024-04-24T09:06:18.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:18.208+0000 7f752a200640 1 -- 172.21.15.69:0/169876909 wait complete. 2024-04-24T09:06:18.222 INFO:tasks.cephadm:Setup complete, yielding 2024-04-24T09:06:18.222 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T09:06:18.234 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-04-24T09:06:18.234 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-24T09:06:18.649 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:06:18.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:18 smithi081 ceph-mon[30961]: pgmap v439: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:06:18.831 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:18 smithi069 ceph-mon[28170]: pgmap v439: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:06:18.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:18 smithi130 ceph-mon[31553]: pgmap v439: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:06:19.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:19 smithi069 ceph-mon[28170]: pgmap v440: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:19.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:19 smithi130 ceph-mon[31553]: pgmap v440: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:20.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:19 smithi081 ceph-mon[30961]: pgmap v440: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.190+0000 7f25dd600700 1 -- 172.21.15.69:0/12296326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25d81035e0 msgr2=0x7f25d81059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.190+0000 7f25dd600700 1 --2- 172.21.15.69:0/12296326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25d81035e0 0x7f25d81059d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009990 tx=0x7f25c8009ca0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.190+0000 7f25dd600700 1 -- 172.21.15.69:0/12296326 shutdown_connections 2024-04-24T09:06:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.190+0000 7f25dd600700 1 --2- 172.21.15.69:0/12296326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f25d8108840 0x7f25d810acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.190+0000 7f25dd600700 1 --2- 172.21.15.69:0/12296326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25d81035e0 0x7f25d81059d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.190+0000 7f25dd600700 1 --2- 172.21.15.69:0/12296326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f25d8105f10 0x7f25d8108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.190+0000 7f25dd600700 1 -- 172.21.15.69:0/12296326 >> 172.21.15.69:0/12296326 conn(0x7f25d80fcff0 msgr2=0x7f25d80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.190+0000 7f25dd600700 1 -- 172.21.15.69:0/12296326 shutdown_connections 2024-04-24T09:06:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.190+0000 7f25dd600700 1 -- 172.21.15.69:0/12296326 wait complete. 2024-04-24T09:06:21.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.191+0000 7f25dd600700 1 Processor -- start 2024-04-24T09:06:21.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.192+0000 7f25dd600700 1 -- start start 2024-04-24T09:06:21.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.192+0000 7f25dd600700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f25d81035e0 0x7f25d8198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.192+0000 7f25dd600700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f25d8105f10 0x7f25d81991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.192+0000 7f25dd600700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25d8108840 0x7f25d81a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.192+0000 7f25dd600700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f25d810e600 con 0x7f25d8108840 2024-04-24T09:06:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.192+0000 7f25dd600700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f25d810e480 con 0x7f25d81035e0 2024-04-24T09:06:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.192+0000 7f25dd600700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f25d810e780 con 0x7f25d8105f10 2024-04-24T09:06:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.192+0000 7f25d7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25d8108840 0x7f25d81a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.192+0000 7f25d7400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25d8108840 0x7f25d81a0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40246/0 (socket says 172.21.15.69:40246) 2024-04-24T09:06:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.192+0000 7f25d7400700 1 -- 172.21.15.69:0/1794036263 learned_addr learned my addr 172.21.15.69:0/1794036263 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.193+0000 7f25d6a00700 1 --2- 172.21.15.69:0/1794036263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f25d81035e0 0x7f25d8198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.193+0000 7f25ce000700 1 --2- 172.21.15.69:0/1794036263 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f25d8105f10 0x7f25d81991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.193+0000 7f25d7400700 1 -- 172.21.15.69:0/1794036263 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f25d8105f10 msgr2=0x7f25d81991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.193+0000 7f25d7400700 1 --2- 172.21.15.69:0/1794036263 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f25d8105f10 0x7f25d81991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.193+0000 7f25d7400700 1 -- 172.21.15.69:0/1794036263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f25d81035e0 msgr2=0x7f25d8198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.193+0000 7f25d7400700 1 --2- 172.21.15.69:0/1794036263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f25d81035e0 0x7f25d8198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.193+0000 7f25d7400700 1 -- 172.21.15.69:0/1794036263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d81a0930 con 0x7f25d8108840 2024-04-24T09:06:21.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.193+0000 7f25d6a00700 1 --2- 172.21.15.69:0/1794036263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f25d81035e0 0x7f25d8198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:21.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.193+0000 7f25d7400700 1 --2- 172.21.15.69:0/1794036263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25d8108840 0x7f25d81a0260 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f25d00094f0 tx=0x7f25d0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:21.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.194+0000 7f25cfe00700 1 -- 172.21.15.69:0/1794036263 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0004db0 con 0x7f25d8108840 2024-04-24T09:06:21.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.194+0000 7f25cfe00700 1 -- 172.21.15.69:0/1794036263 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25d000db60 con 0x7f25d8108840 2024-04-24T09:06:21.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.194+0000 7f25cfe00700 1 -- 172.21.15.69:0/1794036263 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0015400 con 0x7f25d8108840 2024-04-24T09:06:21.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.194+0000 7f25dd600700 1 -- 172.21.15.69:0/1794036263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d81a0c20 con 0x7f25d8108840 2024-04-24T09:06:21.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.194+0000 7f25dd600700 1 -- 172.21.15.69:0/1794036263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d81a1160 con 0x7f25d8108840 2024-04-24T09:06:21.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.195+0000 7f25dd600700 1 -- 172.21.15.69:0/1794036263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d80697e0 con 0x7f25d8108840 2024-04-24T09:06:21.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.197+0000 7f25cfe00700 1 -- 172.21.15.69:0/1794036263 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f25d0005020 con 0x7f25d8108840 2024-04-24T09:06:21.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.199+0000 7f25cfe00700 1 --2- 172.21.15.69:0/1794036263 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f25bc0a3c40 0x7f25bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.200+0000 7f25cfe00700 1 -- 172.21.15.69:0/1794036263 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f25d00d0340 con 0x7f25d8108840 2024-04-24T09:06:21.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.200+0000 7f25d6a00700 1 --2- 172.21.15.69:0/1794036263 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f25bc0a3c40 0x7f25bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.200+0000 7f25d6a00700 1 --2- 172.21.15.69:0/1794036263 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f25bc0a3c40 0x7f25bc0a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f25c8000c00 tx=0x7f25c8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:21.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.204+0000 7f25cfe00700 1 -- 172.21.15.69:0/1794036263 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25d0010070 con 0x7f25d8108840 2024-04-24T09:06:21.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.451+0000 7f25dd600700 1 -- 172.21.15.69:0/1794036263 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f25d8062f30 con 0x7f25bc0a3c40 2024-04-24T09:06:21.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.454+0000 7f25cfe00700 1 -- 172.21.15.69:0/1794036263 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f25d8062f30 con 0x7f25bc0a3c40 2024-04-24T09:06:21.453 INFO:teuthology.orchestra.run.smithi069.stdout:Backend: cephadm 2024-04-24T09:06:21.453 INFO:teuthology.orchestra.run.smithi069.stdout:Available: Yes 2024-04-24T09:06:21.454 INFO:teuthology.orchestra.run.smithi069.stdout:Paused: No 2024-04-24T09:06:21.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.457+0000 7f25dd600700 1 -- 172.21.15.69:0/1794036263 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f25bc0a3c40 msgr2=0x7f25bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.457+0000 7f25dd600700 1 --2- 172.21.15.69:0/1794036263 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f25bc0a3c40 0x7f25bc0a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f25c8000c00 tx=0x7f25c8005fd0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.457+0000 7f25dd600700 1 -- 172.21.15.69:0/1794036263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25d8108840 msgr2=0x7f25d81a0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.457+0000 7f25dd600700 1 --2- 172.21.15.69:0/1794036263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25d8108840 0x7f25d81a0260 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f25d00094f0 tx=0x7f25d0007f90 comp rx=0 tx=0).stop 2024-04-24T09:06:21.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.458+0000 7f25dd600700 1 -- 172.21.15.69:0/1794036263 shutdown_connections 2024-04-24T09:06:21.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.458+0000 7f25dd600700 1 --2- 172.21.15.69:0/1794036263 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f25bc0a3c40 0x7f25bc0a6100 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.458+0000 7f25dd600700 1 --2- 172.21.15.69:0/1794036263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25d8108840 0x7f25d81a0260 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.458+0000 7f25dd600700 1 --2- 172.21.15.69:0/1794036263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f25d81035e0 0x7f25d8198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.458+0000 7f25dd600700 1 --2- 172.21.15.69:0/1794036263 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f25d8105f10 0x7f25d81991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.458+0000 7f25dd600700 1 -- 172.21.15.69:0/1794036263 >> 172.21.15.69:0/1794036263 conn(0x7f25d80fcff0 msgr2=0x7f25d8104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:21.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.458+0000 7f25dd600700 1 -- 172.21.15.69:0/1794036263 shutdown_connections 2024-04-24T09:06:21.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:21.458+0000 7f25dd600700 1 -- 172.21.15.69:0/1794036263 wait complete. 2024-04-24T09:06:22.380 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-24T09:06:22.812 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:06:22.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:22 smithi081 ceph-mon[30961]: pgmap v441: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:06:22.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:22 smithi081 ceph-mon[30961]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:06:22.840 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:22 smithi069 ceph-mon[28170]: pgmap v441: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:06:22.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:22 smithi069 ceph-mon[28170]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:06:22.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:22 smithi130 ceph-mon[31553]: pgmap v441: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:06:22.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:22 smithi130 ceph-mon[31553]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:06:24.720 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:24 smithi069 ceph-mon[28170]: pgmap v442: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:06:24.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:24 smithi081 ceph-mon[30961]: pgmap v442: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:06:24.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:24 smithi130 ceph-mon[31553]: pgmap v442: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:06:25.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.286+0000 7f0a1e200700 1 -- 172.21.15.69:0/2173417749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a18105f90 msgr2=0x7f0a181063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.286+0000 7f0a1e200700 1 --2- 172.21.15.69:0/2173417749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a18105f90 0x7f0a181063f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0a040099c0 tx=0x7f0a04009cd0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.286+0000 7f0a1e200700 1 -- 172.21.15.69:0/2173417749 shutdown_connections 2024-04-24T09:06:25.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.286+0000 7f0a1e200700 1 --2- 172.21.15.69:0/2173417749 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a18106930 0x7f0a1810d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.286+0000 7f0a1e200700 1 --2- 172.21.15.69:0/2173417749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a18104d90 0x7f0a181051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.286+0000 7f0a1e200700 1 --2- 172.21.15.69:0/2173417749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a18105f90 0x7f0a181063f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.286+0000 7f0a1e200700 1 -- 172.21.15.69:0/2173417749 >> 172.21.15.69:0/2173417749 conn(0x7f0a18100350 msgr2=0x7f0a18102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:25.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.286+0000 7f0a1e200700 1 -- 172.21.15.69:0/2173417749 shutdown_connections 2024-04-24T09:06:25.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.286+0000 7f0a1e200700 1 -- 172.21.15.69:0/2173417749 wait complete. 2024-04-24T09:06:25.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.288+0000 7f0a1e200700 1 Processor -- start 2024-04-24T09:06:25.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.288+0000 7f0a1e200700 1 -- start start 2024-04-24T09:06:25.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.288+0000 7f0a1e200700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a18104d90 0x7f0a1819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.288+0000 7f0a1e200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a18105f90 0x7f0a1819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.288+0000 7f0a1e200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a18106930 0x7f0a181a2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.288+0000 7f0a1e200700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0a18110880 con 0x7f0a18105f90 2024-04-24T09:06:25.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.288+0000 7f0a1e200700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0a18110700 con 0x7f0a18104d90 2024-04-24T09:06:25.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.289+0000 7f0a1e200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0a18110a00 con 0x7f0a18106930 2024-04-24T09:06:25.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.289+0000 7f0a16a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a18105f90 0x7f0a1819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.289+0000 7f0a16a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a18105f90 0x7f0a1819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55002/0 (socket says 172.21.15.69:55002) 2024-04-24T09:06:25.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.289+0000 7f0a16a00700 1 -- 172.21.15.69:0/2594829413 learned_addr learned my addr 172.21.15.69:0/2594829413 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:25.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.289+0000 7f0a17400700 1 --2- 172.21.15.69:0/2594829413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a18104d90 0x7f0a1819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.289+0000 7f0a17e00700 1 --2- 172.21.15.69:0/2594829413 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a18106930 0x7f0a181a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.289+0000 7f0a16a00700 1 -- 172.21.15.69:0/2594829413 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a18106930 msgr2=0x7f0a181a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.289+0000 7f0a16a00700 1 --2- 172.21.15.69:0/2594829413 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a18106930 0x7f0a181a2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.289+0000 7f0a16a00700 1 -- 172.21.15.69:0/2594829413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a18104d90 msgr2=0x7f0a1819aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.289+0000 7f0a16a00700 1 --2- 172.21.15.69:0/2594829413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a18104d90 0x7f0a1819aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.289+0000 7f0a16a00700 1 -- 172.21.15.69:0/2594829413 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a181a2b20 con 0x7f0a18105f90 2024-04-24T09:06:25.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.289+0000 7f0a17400700 1 --2- 172.21.15.69:0/2594829413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a18104d90 0x7f0a1819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:25.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.290+0000 7f0a16a00700 1 --2- 172.21.15.69:0/2594829413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a18105f90 0x7f0a1819b400 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400b580 tx=0x7f0a04009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:25.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.290+0000 7f0a0fe00700 1 -- 172.21.15.69:0/2594829413 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a04027070 con 0x7f0a18105f90 2024-04-24T09:06:25.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.290+0000 7f0a1e200700 1 -- 172.21.15.69:0/2594829413 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a181a2db0 con 0x7f0a18105f90 2024-04-24T09:06:25.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.290+0000 7f0a0fe00700 1 -- 172.21.15.69:0/2594829413 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a040040d0 con 0x7f0a18105f90 2024-04-24T09:06:25.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.290+0000 7f0a0fe00700 1 -- 172.21.15.69:0/2594829413 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a04022e10 con 0x7f0a18105f90 2024-04-24T09:06:25.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.290+0000 7f0a1e200700 1 -- 172.21.15.69:0/2594829413 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a181a3200 con 0x7f0a18105f90 2024-04-24T09:06:25.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.291+0000 7f0a1e200700 1 -- 172.21.15.69:0/2594829413 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a180697e0 con 0x7f0a18105f90 2024-04-24T09:06:25.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.293+0000 7f0a0fe00700 1 -- 172.21.15.69:0/2594829413 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0a0402a070 con 0x7f0a18105f90 2024-04-24T09:06:25.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.296+0000 7f0a0fe00700 1 --2- 172.21.15.69:0/2594829413 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f09fc0a3cc0 0x7f09fc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.296+0000 7f0a0fe00700 1 -- 172.21.15.69:0/2594829413 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0a040d1530 con 0x7f0a18105f90 2024-04-24T09:06:25.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.296+0000 7f0a17400700 1 --2- 172.21.15.69:0/2594829413 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f09fc0a3cc0 0x7f09fc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.297+0000 7f0a17400700 1 --2- 172.21.15.69:0/2594829413 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f09fc0a3cc0 0x7f09fc0a6180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0a08005950 tx=0x7f0a080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:25.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.300+0000 7f0a0fe00700 1 -- 172.21.15.69:0/2594829413 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0a04013070 con 0x7f0a18105f90 2024-04-24T09:06:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.536+0000 7f0a1e200700 1 -- 172.21.15.69:0/2594829413 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0a18062f30 con 0x7f09fc0a3cc0 2024-04-24T09:06:25.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.581+0000 7f0a0fe00700 1 -- 172.21.15.69:0/2594829413 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f0a18062f30 con 0x7f09fc0a3cc0 2024-04-24T09:06:25.581 INFO:teuthology.orchestra.run.smithi069.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T09:06:25.581 INFO:teuthology.orchestra.run.smithi069.stdout:mds.a smithi069 running (2m) 28s ago 2m 12.4M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 3eefa30160f8 2024-04-24T09:06:25.582 INFO:teuthology.orchestra.run.smithi069.stdout:mds.b smithi081 running (2m) 75s ago 2m 24.6M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 3c6bdbb22db8 2024-04-24T09:06:25.582 INFO:teuthology.orchestra.run.smithi069.stdout:mds.c smithi130 running (2m) 76s ago 2m 14.4M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 2a62d12c03c3 2024-04-24T09:06:25.582 INFO:teuthology.orchestra.run.smithi069.stdout:mds.d smithi069 running (3m) 28s ago 3m 12.5M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 4e35c7e74122 2024-04-24T09:06:25.582 INFO:teuthology.orchestra.run.smithi069.stdout:mds.e smithi081 running (2m) 75s ago 2m 15.9M - 19.0.0-3116-g1f81fb39 b98aea5aa06f ad16eaa7ced9 2024-04-24T09:06:25.582 INFO:teuthology.orchestra.run.smithi069.stdout:mds.f smithi130 running (2m) 76s ago 2m 12.4M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 47c7d8f55162 2024-04-24T09:06:25.582 INFO:teuthology.orchestra.run.smithi069.stdout:mds.g smithi069 running (2m) 28s ago 2m 12.3M - 19.0.0-3116-g1f81fb39 b98aea5aa06f bf470faf116c 2024-04-24T09:06:25.582 INFO:teuthology.orchestra.run.smithi069.stdout:mds.h smithi081 running (2m) 75s ago 2m 15.7M - 19.0.0-3116-g1f81fb39 b98aea5aa06f e2dbdb3a3aad 2024-04-24T09:06:25.582 INFO:teuthology.orchestra.run.smithi069.stdout:mds.i smithi130 running (2m) 76s ago 2m 14.4M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 4a37e99aaa24 2024-04-24T09:06:25.582 INFO:teuthology.orchestra.run.smithi069.stdout:mds.j smithi069 running (3m) 28s ago 3m 12.7M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 85b5983e6d72 2024-04-24T09:06:25.582 INFO:teuthology.orchestra.run.smithi069.stdout:mds.k smithi081 running (3m) 75s ago 3m 15.8M - 19.0.0-3116-g1f81fb39 b98aea5aa06f ec97d78eb5b7 2024-04-24T09:06:25.582 INFO:teuthology.orchestra.run.smithi069.stdout:mds.l smithi130 running (3m) 76s ago 3m 12.5M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 0ea7a1311dc2 2024-04-24T09:06:25.582 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.x smithi069 *:9283,8765 running (13m) 28s ago 12m 515M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 6e4c09281b32 2024-04-24T09:06:25.582 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.y smithi081 *:8443,8765 running (10m) 75s ago 10m 452M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 86e9cccf0aaa 2024-04-24T09:06:25.583 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.z smithi130 *:8443,8765 running (10m) 76s ago 10m 452M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 9a364bef97f3 2024-04-24T09:06:25.583 INFO:teuthology.orchestra.run.smithi069.stdout:mon.a smithi069 running (13m) 28s ago 13m 51.6M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 9d225ee88aa4 2024-04-24T09:06:25.583 INFO:teuthology.orchestra.run.smithi069.stdout:mon.b smithi081 running (10m) 75s ago 10m 39.3M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 500bbac1214c 2024-04-24T09:06:25.583 INFO:teuthology.orchestra.run.smithi069.stdout:mon.c smithi130 running (10m) 76s ago 10m 40.7M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f dc34ddfa7159 2024-04-24T09:06:25.583 INFO:teuthology.orchestra.run.smithi069.stdout:osd.0 smithi069 running (10m) 28s ago 10m 45.4M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f e67b884e8a6d 2024-04-24T09:06:25.583 INFO:teuthology.orchestra.run.smithi069.stdout:osd.1 smithi081 running (9m) 75s ago 9m 46.4M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 189486e7cdeb 2024-04-24T09:06:25.583 INFO:teuthology.orchestra.run.smithi069.stdout:osd.2 smithi130 running (8m) 76s ago 8m 46.6M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 5ce7313f55d6 2024-04-24T09:06:25.583 INFO:teuthology.orchestra.run.smithi069.stdout:osd.3 smithi069 running (8m) 28s ago 8m 52.2M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f b87ab8c4e038 2024-04-24T09:06:25.583 INFO:teuthology.orchestra.run.smithi069.stdout:osd.4 smithi081 running (7m) 75s ago 7m 42.2M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 6a42c299965c 2024-04-24T09:06:25.583 INFO:teuthology.orchestra.run.smithi069.stdout:osd.5 smithi130 running (7m) 76s ago 7m 51.9M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 52b36dcfbb86 2024-04-24T09:06:25.583 INFO:teuthology.orchestra.run.smithi069.stdout:osd.6 smithi069 running (6m) 28s ago 6m 42.2M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 3eff8f82b658 2024-04-24T09:06:25.583 INFO:teuthology.orchestra.run.smithi069.stdout:osd.7 smithi081 running (6m) 75s ago 6m 51.4M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8397aef235ea 2024-04-24T09:06:25.583 INFO:teuthology.orchestra.run.smithi069.stdout:osd.8 smithi130 running (5m) 76s ago 5m 42.4M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 3837203adaa9 2024-04-24T09:06:25.583 INFO:teuthology.orchestra.run.smithi069.stdout:osd.9 smithi069 running (4m) 28s ago 4m 42.4M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 08a6b188a4d4 2024-04-24T09:06:25.583 INFO:teuthology.orchestra.run.smithi069.stdout:osd.10 smithi081 running (4m) 75s ago 4m 53.3M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 79a5e3de6fa4 2024-04-24T09:06:25.584 INFO:teuthology.orchestra.run.smithi069.stdout:osd.11 smithi130 running (3m) 76s ago 3m 40.9M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f e681b68ca305 2024-04-24T09:06:25.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.584+0000 7f0a1e200700 1 -- 172.21.15.69:0/2594829413 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f09fc0a3cc0 msgr2=0x7f09fc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.584+0000 7f0a1e200700 1 --2- 172.21.15.69:0/2594829413 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f09fc0a3cc0 0x7f09fc0a6180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0a08005950 tx=0x7f0a080058e0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.584+0000 7f0a1e200700 1 -- 172.21.15.69:0/2594829413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a18105f90 msgr2=0x7f0a1819b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.584+0000 7f0a1e200700 1 --2- 172.21.15.69:0/2594829413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a18105f90 0x7f0a1819b400 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400b580 tx=0x7f0a04009f80 comp rx=0 tx=0).stop 2024-04-24T09:06:25.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.584+0000 7f0a1e200700 1 -- 172.21.15.69:0/2594829413 shutdown_connections 2024-04-24T09:06:25.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.584+0000 7f0a1e200700 1 --2- 172.21.15.69:0/2594829413 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f09fc0a3cc0 0x7f09fc0a6180 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.584+0000 7f0a1e200700 1 --2- 172.21.15.69:0/2594829413 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0a18106930 0x7f0a181a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.584+0000 7f0a1e200700 1 --2- 172.21.15.69:0/2594829413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a18104d90 0x7f0a1819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.584+0000 7f0a1e200700 1 --2- 172.21.15.69:0/2594829413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a18105f90 0x7f0a1819b400 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.584+0000 7f0a1e200700 1 -- 172.21.15.69:0/2594829413 >> 172.21.15.69:0/2594829413 conn(0x7f0a18100350 msgr2=0x7f0a18109290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:25.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.585+0000 7f0a1e200700 1 -- 172.21.15.69:0/2594829413 shutdown_connections 2024-04-24T09:06:25.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:25.585+0000 7f0a1e200700 1 -- 172.21.15.69:0/2594829413 wait complete. 2024-04-24T09:06:26.510 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-24T09:06:26.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:26 smithi081 ceph-mon[30961]: pgmap v443: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:26.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:26 smithi081 ceph-mon[30961]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:06:26.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:26 smithi069 ceph-mon[28170]: pgmap v443: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:26.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:26 smithi069 ceph-mon[28170]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:06:26.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:26 smithi130 ceph-mon[31553]: pgmap v443: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:26.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:26 smithi130 ceph-mon[31553]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:06:26.965 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:06:28.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:28 smithi069 ceph-mon[28170]: pgmap v444: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:06:28.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:28 smithi081 ceph-mon[30961]: pgmap v444: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:06:28.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:28 smithi130 ceph-mon[31553]: pgmap v444: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:06:29.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.317+0000 7f85c7e00700 1 -- 172.21.15.69:0/3641596473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c010f140 msgr2=0x7f85c0111530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:29.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.317+0000 7f85c7e00700 1 --2- 172.21.15.69:0/3641596473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c010f140 0x7f85c0111530 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009990 tx=0x7f85b8009ca0 comp rx=0 tx=0).stop 2024-04-24T09:06:29.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.318+0000 7f85c7e00700 1 -- 172.21.15.69:0/3641596473 shutdown_connections 2024-04-24T09:06:29.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.318+0000 7f85c7e00700 1 --2- 172.21.15.69:0/3641596473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c010f140 0x7f85c0111530 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:29.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.318+0000 7f85c7e00700 1 --2- 172.21.15.69:0/3641596473 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85c0076b20 0x7f85c0074f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:29.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.318+0000 7f85c7e00700 1 --2- 172.21.15.69:0/3641596473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85c0075550 0x7f85c00759d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:29.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.318+0000 7f85c7e00700 1 -- 172.21.15.69:0/3641596473 >> 172.21.15.69:0/3641596473 conn(0x7f85c00fe0c0 msgr2=0x7f85c0100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:29.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.318+0000 7f85c7e00700 1 -- 172.21.15.69:0/3641596473 shutdown_connections 2024-04-24T09:06:29.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.318+0000 7f85c7e00700 1 -- 172.21.15.69:0/3641596473 wait complete. 2024-04-24T09:06:29.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.319+0000 7f85c7e00700 1 Processor -- start 2024-04-24T09:06:29.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.319+0000 7f85c7e00700 1 -- start start 2024-04-24T09:06:29.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.320+0000 7f85c7e00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85c0075550 0x7f85c019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:29.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.320+0000 7f85c7e00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85c0076b20 0x7f85c019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:29.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.320+0000 7f85c7e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c010f140 0x7f85c01a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:29.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.320+0000 7f85c7e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f85c0114c70 con 0x7f85c010f140 2024-04-24T09:06:29.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.320+0000 7f85c7e00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f85c0114af0 con 0x7f85c0076b20 2024-04-24T09:06:29.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.320+0000 7f85c7e00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f85c0114df0 con 0x7f85c0075550 2024-04-24T09:06:29.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.320+0000 7f85c5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c010f140 0x7f85c01a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:29.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.320+0000 7f85c5e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c010f140 0x7f85c01a6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55024/0 (socket says 172.21.15.69:55024) 2024-04-24T09:06:29.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.320+0000 7f85c5e00700 1 -- 172.21.15.69:0/4286859182 learned_addr learned my addr 172.21.15.69:0/4286859182 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:29.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.320+0000 7f85c5400700 1 --2- 172.21.15.69:0/4286859182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85c0075550 0x7f85c019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:29.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.320+0000 7f85c4a00700 1 --2- 172.21.15.69:0/4286859182 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85c0076b20 0x7f85c019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:29.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.321+0000 7f85c5e00700 1 -- 172.21.15.69:0/4286859182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85c0075550 msgr2=0x7f85c019f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:29.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.321+0000 7f85c5e00700 1 --2- 172.21.15.69:0/4286859182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85c0075550 0x7f85c019f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:29.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.321+0000 7f85c5e00700 1 -- 172.21.15.69:0/4286859182 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85c0076b20 msgr2=0x7f85c019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:29.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.321+0000 7f85c5e00700 1 --2- 172.21.15.69:0/4286859182 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85c0076b20 0x7f85c019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:29.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.321+0000 7f85c5e00700 1 -- 172.21.15.69:0/4286859182 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c01a6f10 con 0x7f85c010f140 2024-04-24T09:06:29.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.321+0000 7f85c5e00700 1 --2- 172.21.15.69:0/4286859182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c010f140 0x7f85c01a6870 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f85b8005910 tx=0x7f85b8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:29.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.321+0000 7f85be000700 1 -- 172.21.15.69:0/4286859182 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8027070 con 0x7f85c010f140 2024-04-24T09:06:29.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.321+0000 7f85be000700 1 -- 172.21.15.69:0/4286859182 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85b80041d0 con 0x7f85c010f140 2024-04-24T09:06:29.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.321+0000 7f85c7e00700 1 -- 172.21.15.69:0/4286859182 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c01a71a0 con 0x7f85c010f140 2024-04-24T09:06:29.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.321+0000 7f85be000700 1 -- 172.21.15.69:0/4286859182 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b800f460 con 0x7f85c010f140 2024-04-24T09:06:29.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.321+0000 7f85c7e00700 1 -- 172.21.15.69:0/4286859182 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c01a7650 con 0x7f85c010f140 2024-04-24T09:06:29.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.322+0000 7f85c7e00700 1 -- 172.21.15.69:0/4286859182 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85c00697e0 con 0x7f85c010f140 2024-04-24T09:06:29.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.325+0000 7f85be000700 1 -- 172.21.15.69:0/4286859182 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f85b802a070 con 0x7f85c010f140 2024-04-24T09:06:29.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.326+0000 7f85be000700 1 --2- 172.21.15.69:0/4286859182 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f85a80a3cc0 0x7f85a80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:29.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.326+0000 7f85be000700 1 -- 172.21.15.69:0/4286859182 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f85b80d1d80 con 0x7f85c010f140 2024-04-24T09:06:29.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.326+0000 7f85c5400700 1 --2- 172.21.15.69:0/4286859182 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f85a80a3cc0 0x7f85a80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:29.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.327+0000 7f85c5400700 1 --2- 172.21.15.69:0/4286859182 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f85a80a3cc0 0x7f85a80a6180 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f85c0076f00 tx=0x7f85b4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:29.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.329+0000 7f85be000700 1 -- 172.21.15.69:0/4286859182 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f85b8031020 con 0x7f85c010f140 2024-04-24T09:06:29.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.566+0000 7f85c7e00700 1 -- 172.21.15.69:0/4286859182 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f85c0062f30 con 0x7f85a80a3cc0 2024-04-24T09:06:29.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.574+0000 7f85be000700 1 -- 172.21.15.69:0/4286859182 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f85c0062f30 con 0x7f85a80a3cc0 2024-04-24T09:06:29.574 INFO:teuthology.orchestra.run.smithi069.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-24T09:06:29.574 INFO:teuthology.orchestra.run.smithi069.stdout:mds.all 12/12 80s ago 3m smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l;count:12 2024-04-24T09:06:29.574 INFO:teuthology.orchestra.run.smithi069.stdout:mgr 3/3 80s ago 10m smithi069=x;smithi081=y;smithi130=z;count:3 2024-04-24T09:06:29.574 INFO:teuthology.orchestra.run.smithi069.stdout:mon 3/3 80s ago 11m smithi069:172.21.15.69=a;smithi081:172.21.15.81=b;smithi130:172.21.15.130=c;count:3 2024-04-24T09:06:29.574 INFO:teuthology.orchestra.run.smithi069.stdout:osd 12 80s ago - 2024-04-24T09:06:29.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.577+0000 7f85c7e00700 1 -- 172.21.15.69:0/4286859182 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f85a80a3cc0 msgr2=0x7f85a80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:29.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.577+0000 7f85c7e00700 1 --2- 172.21.15.69:0/4286859182 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f85a80a3cc0 0x7f85a80a6180 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f85c0076f00 tx=0x7f85b4009040 comp rx=0 tx=0).stop 2024-04-24T09:06:29.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.577+0000 7f85c7e00700 1 -- 172.21.15.69:0/4286859182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c010f140 msgr2=0x7f85c01a6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:29.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.577+0000 7f85c7e00700 1 --2- 172.21.15.69:0/4286859182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c010f140 0x7f85c01a6870 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f85b8005910 tx=0x7f85b8005ef0 comp rx=0 tx=0).stop 2024-04-24T09:06:29.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.577+0000 7f85c7e00700 1 -- 172.21.15.69:0/4286859182 shutdown_connections 2024-04-24T09:06:29.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.577+0000 7f85c7e00700 1 --2- 172.21.15.69:0/4286859182 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f85a80a3cc0 0x7f85a80a6180 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:29.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.577+0000 7f85c7e00700 1 --2- 172.21.15.69:0/4286859182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c010f140 0x7f85c01a6870 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:29.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.577+0000 7f85c7e00700 1 --2- 172.21.15.69:0/4286859182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85c0075550 0x7f85c019f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:29.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.577+0000 7f85c7e00700 1 --2- 172.21.15.69:0/4286859182 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85c0076b20 0x7f85c019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:29.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.577+0000 7f85c7e00700 1 -- 172.21.15.69:0/4286859182 >> 172.21.15.69:0/4286859182 conn(0x7f85c00fe0c0 msgr2=0x7f85c01024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:29.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.577+0000 7f85c7e00700 1 -- 172.21.15.69:0/4286859182 shutdown_connections 2024-04-24T09:06:29.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:29.578+0000 7f85c7e00700 1 -- 172.21.15.69:0/4286859182 wait complete. 2024-04-24T09:06:30.511 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-24T09:06:30.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:30 smithi081 ceph-mon[30961]: pgmap v445: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:30.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:30 smithi069 ceph-mon[28170]: pgmap v445: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:30.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:30 smithi130 ceph-mon[31553]: pgmap v445: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:30.958 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:06:31.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:31 smithi081 ceph-mon[30961]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:06:31.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:31 smithi069 ceph-mon[28170]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:06:31.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:31 smithi130 ceph-mon[31553]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:06:32.721 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:32 smithi069 ceph-mon[28170]: pgmap v446: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:06:32.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:32 smithi081 ceph-mon[30961]: pgmap v446: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:06:32.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:32 smithi130 ceph-mon[31553]: pgmap v446: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:06:33.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.278+0000 7f1383800700 1 -- 172.21.15.69:0/3418218406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f137c105f90 msgr2=0x7f137c1063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:33.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.278+0000 7f1383800700 1 --2- 172.21.15.69:0/3418218406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f137c105f90 0x7f137c1063f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f13680099c0 tx=0x7f1368009cd0 comp rx=0 tx=0).stop 2024-04-24T09:06:33.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.278+0000 7f1383800700 1 -- 172.21.15.69:0/3418218406 shutdown_connections 2024-04-24T09:06:33.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.278+0000 7f1383800700 1 --2- 172.21.15.69:0/3418218406 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f137c106930 0x7f137c10d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:33.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.278+0000 7f1383800700 1 --2- 172.21.15.69:0/3418218406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f137c104d90 0x7f137c1051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:33.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.278+0000 7f1383800700 1 --2- 172.21.15.69:0/3418218406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f137c105f90 0x7f137c1063f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:33.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.278+0000 7f1383800700 1 -- 172.21.15.69:0/3418218406 >> 172.21.15.69:0/3418218406 conn(0x7f137c100350 msgr2=0x7f137c102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:33.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.279+0000 7f1383800700 1 -- 172.21.15.69:0/3418218406 shutdown_connections 2024-04-24T09:06:33.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.279+0000 7f1383800700 1 -- 172.21.15.69:0/3418218406 wait complete. 2024-04-24T09:06:33.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.280+0000 7f1383800700 1 Processor -- start 2024-04-24T09:06:33.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.280+0000 7f1383800700 1 -- start start 2024-04-24T09:06:33.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.281+0000 7f1383800700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f137c104d90 0x7f137c19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:33.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.281+0000 7f1383800700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f137c105f90 0x7f137c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:33.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.281+0000 7f1383800700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f137c106930 0x7f137c1a2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:33.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.281+0000 7f1383800700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f137c110880 con 0x7f137c105f90 2024-04-24T09:06:33.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.281+0000 7f1383800700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f137c110700 con 0x7f137c104d90 2024-04-24T09:06:33.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.281+0000 7f1383800700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f137c110a00 con 0x7f137c106930 2024-04-24T09:06:33.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.281+0000 7f137be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f137c105f90 0x7f137c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:33.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.281+0000 7f137be00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f137c105f90 0x7f137c19b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46942/0 (socket says 172.21.15.69:46942) 2024-04-24T09:06:33.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.281+0000 7f137be00700 1 -- 172.21.15.69:0/1475244720 learned_addr learned my addr 172.21.15.69:0/1475244720 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:33.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.281+0000 7f1380e00700 1 --2- 172.21.15.69:0/1475244720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f137c104d90 0x7f137c19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:33.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.282+0000 7f1381800700 1 --2- 172.21.15.69:0/1475244720 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f137c106930 0x7f137c1a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:33.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.282+0000 7f137be00700 1 -- 172.21.15.69:0/1475244720 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f137c106930 msgr2=0x7f137c1a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:33.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.282+0000 7f137be00700 1 --2- 172.21.15.69:0/1475244720 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f137c106930 0x7f137c1a2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:33.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.282+0000 7f137be00700 1 -- 172.21.15.69:0/1475244720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f137c104d90 msgr2=0x7f137c19aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:33.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.282+0000 7f137be00700 1 --2- 172.21.15.69:0/1475244720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f137c104d90 0x7f137c19aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:33.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.282+0000 7f137be00700 1 -- 172.21.15.69:0/1475244720 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137c1a2b20 con 0x7f137c105f90 2024-04-24T09:06:33.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.282+0000 7f1380e00700 1 --2- 172.21.15.69:0/1475244720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f137c104d90 0x7f137c19aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:33.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.282+0000 7f137be00700 1 --2- 172.21.15.69:0/1475244720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f137c105f90 0x7f137c19b400 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f136800b580 tx=0x7f1368009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:33.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.282+0000 7f1379600700 1 -- 172.21.15.69:0/1475244720 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1368027070 con 0x7f137c105f90 2024-04-24T09:06:33.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.283+0000 7f1379600700 1 -- 172.21.15.69:0/1475244720 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13680040d0 con 0x7f137c105f90 2024-04-24T09:06:33.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.283+0000 7f1383800700 1 -- 172.21.15.69:0/1475244720 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137c1a2db0 con 0x7f137c105f90 2024-04-24T09:06:33.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.283+0000 7f1379600700 1 -- 172.21.15.69:0/1475244720 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1368022e10 con 0x7f137c105f90 2024-04-24T09:06:33.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.283+0000 7f1383800700 1 -- 172.21.15.69:0/1475244720 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f137c1a3200 con 0x7f137c105f90 2024-04-24T09:06:33.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.284+0000 7f1383800700 1 -- 172.21.15.69:0/1475244720 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f137c0697e0 con 0x7f137c105f90 2024-04-24T09:06:33.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.286+0000 7f1379600700 1 -- 172.21.15.69:0/1475244720 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f136802a070 con 0x7f137c105f90 2024-04-24T09:06:33.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.288+0000 7f1379600700 1 --2- 172.21.15.69:0/1475244720 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13640a3cc0 0x7f13640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:33.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.288+0000 7f1379600700 1 -- 172.21.15.69:0/1475244720 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f13680d1590 con 0x7f137c105f90 2024-04-24T09:06:33.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.289+0000 7f1380e00700 1 --2- 172.21.15.69:0/1475244720 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13640a3cc0 0x7f13640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:33.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.290+0000 7f1380e00700 1 --2- 172.21.15.69:0/1475244720 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13640a3cc0 0x7f13640a6180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1370005950 tx=0x7f13700058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:33.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.293+0000 7f1379600700 1 -- 172.21.15.69:0/1475244720 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1368013070 con 0x7f137c105f90 2024-04-24T09:06:33.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.531+0000 7f1383800700 1 -- 172.21.15.69:0/1475244720 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f137c062f30 con 0x7f13640a3cc0 2024-04-24T09:06:33.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.535+0000 7f1379600700 1 -- 172.21.15.69:0/1475244720 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f137c062f30 con 0x7f13640a3cc0 2024-04-24T09:06:33.535 INFO:teuthology.orchestra.run.smithi069.stdout:HOST ADDR LABELS STATUS 2024-04-24T09:06:33.535 INFO:teuthology.orchestra.run.smithi069.stdout:smithi069 172.21.15.69 2024-04-24T09:06:33.535 INFO:teuthology.orchestra.run.smithi069.stdout:smithi081 172.21.15.81 2024-04-24T09:06:33.535 INFO:teuthology.orchestra.run.smithi069.stdout:smithi130 172.21.15.130 2024-04-24T09:06:33.535 INFO:teuthology.orchestra.run.smithi069.stdout:3 hosts in cluster 2024-04-24T09:06:33.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.538+0000 7f1383800700 1 -- 172.21.15.69:0/1475244720 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13640a3cc0 msgr2=0x7f13640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:33.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.538+0000 7f1383800700 1 --2- 172.21.15.69:0/1475244720 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13640a3cc0 0x7f13640a6180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1370005950 tx=0x7f13700058e0 comp rx=0 tx=0).stop 2024-04-24T09:06:33.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.538+0000 7f1383800700 1 -- 172.21.15.69:0/1475244720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f137c105f90 msgr2=0x7f137c19b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:33.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.538+0000 7f1383800700 1 --2- 172.21.15.69:0/1475244720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f137c105f90 0x7f137c19b400 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f136800b580 tx=0x7f1368009f80 comp rx=0 tx=0).stop 2024-04-24T09:06:33.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.539+0000 7f1383800700 1 -- 172.21.15.69:0/1475244720 shutdown_connections 2024-04-24T09:06:33.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.539+0000 7f1383800700 1 --2- 172.21.15.69:0/1475244720 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13640a3cc0 0x7f13640a6180 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:33.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.539+0000 7f1383800700 1 --2- 172.21.15.69:0/1475244720 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f137c106930 0x7f137c1a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:33.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.539+0000 7f1383800700 1 --2- 172.21.15.69:0/1475244720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f137c104d90 0x7f137c19aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:33.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.539+0000 7f1383800700 1 --2- 172.21.15.69:0/1475244720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f137c105f90 0x7f137c19b400 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:33.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.539+0000 7f1383800700 1 -- 172.21.15.69:0/1475244720 >> 172.21.15.69:0/1475244720 conn(0x7f137c100350 msgr2=0x7f137c109290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:33.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.539+0000 7f1383800700 1 -- 172.21.15.69:0/1475244720 shutdown_connections 2024-04-24T09:06:33.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:33.539+0000 7f1383800700 1 -- 172.21.15.69:0/1475244720 wait complete. 2024-04-24T09:06:34.630 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-24T09:06:34.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:34 smithi081 ceph-mon[30961]: pgmap v447: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:06:34.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:34 smithi081 ceph-mon[30961]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:06:34.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:34 smithi069 ceph-mon[28170]: pgmap v447: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:06:34.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:34 smithi069 ceph-mon[28170]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:06:34.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:34 smithi130 ceph-mon[31553]: pgmap v447: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:06:34.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:34 smithi130 ceph-mon[31553]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:06:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:06:36.802 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:36 smithi069 ceph-mon[28170]: pgmap v448: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:36.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:36 smithi081 ceph-mon[30961]: pgmap v448: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:36.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:36 smithi130 ceph-mon[31553]: pgmap v448: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:37.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.364+0000 7f23c0a00700 1 -- 172.21.15.69:0/4253628974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc109190 msgr2=0x7f23bc1095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:37.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.364+0000 7f23c0a00700 1 --2- 172.21.15.69:0/4253628974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc109190 0x7f23bc1095b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0099c0 tx=0x7f23ac009cd0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.364+0000 7f23c0a00700 1 -- 172.21.15.69:0/4253628974 shutdown_connections 2024-04-24T09:06:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.364+0000 7f23c0a00700 1 --2- 172.21.15.69:0/4253628974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23bc10ad30 0x7f23bc1115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.364+0000 7f23c0a00700 1 --2- 172.21.15.69:0/4253628974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc109190 0x7f23bc1095b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.364+0000 7f23c0a00700 1 --2- 172.21.15.69:0/4253628974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23bc10a390 0x7f23bc10a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.364+0000 7f23c0a00700 1 -- 172.21.15.69:0/4253628974 >> 172.21.15.69:0/4253628974 conn(0x7f23bc078160 msgr2=0x7f23bc07a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.364+0000 7f23c0a00700 1 -- 172.21.15.69:0/4253628974 shutdown_connections 2024-04-24T09:06:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.364+0000 7f23c0a00700 1 -- 172.21.15.69:0/4253628974 wait complete. 2024-04-24T09:06:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.365+0000 7f23c0a00700 1 Processor -- start 2024-04-24T09:06:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.365+0000 7f23c0a00700 1 -- start start 2024-04-24T09:06:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.366+0000 7f23c0a00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23bc109190 0x7f23bc19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.366+0000 7f23c0a00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc10a390 0x7f23bc19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.366+0000 7f23c0a00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23bc10ad30 0x7f23bc1a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.366+0000 7f23c0a00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc114ce0 con 0x7f23bc10a390 2024-04-24T09:06:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.366+0000 7f23c0a00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc114b60 con 0x7f23bc109190 2024-04-24T09:06:37.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.366+0000 7f23c0a00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc114e60 con 0x7f23bc10ad30 2024-04-24T09:06:37.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.366+0000 7f23b9400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc10a390 0x7f23bc19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.366+0000 7f23b9400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc10a390 0x7f23bc19f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46954/0 (socket says 172.21.15.69:46954) 2024-04-24T09:06:37.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.366+0000 7f23b9400700 1 -- 172.21.15.69:0/280424110 learned_addr learned my addr 172.21.15.69:0/280424110 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:37.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.366+0000 7f23b9e00700 1 --2- 172.21.15.69:0/280424110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23bc109190 0x7f23bc19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.366+0000 7f23ba800700 1 --2- 172.21.15.69:0/280424110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23bc10ad30 0x7f23bc1a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.367+0000 7f23b9400700 1 -- 172.21.15.69:0/280424110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23bc10ad30 msgr2=0x7f23bc1a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:37.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.367+0000 7f23b9400700 1 --2- 172.21.15.69:0/280424110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23bc10ad30 0x7f23bc1a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.367+0000 7f23b9400700 1 -- 172.21.15.69:0/280424110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23bc109190 msgr2=0x7f23bc19f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:37.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.367+0000 7f23b9400700 1 --2- 172.21.15.69:0/280424110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23bc109190 0x7f23bc19f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.367+0000 7f23b9400700 1 -- 172.21.15.69:0/280424110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc1a6fc0 con 0x7f23bc10a390 2024-04-24T09:06:37.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.367+0000 7f23ba800700 1 --2- 172.21.15.69:0/280424110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23bc10ad30 0x7f23bc1a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:37.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.367+0000 7f23b9400700 1 --2- 172.21.15.69:0/280424110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc10a390 0x7f23bc19f8f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f23a800c950 tx=0x7f23a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:37.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.368+0000 7f23b2a00700 1 -- 172.21.15.69:0/280424110 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8007ac0 con 0x7f23bc10a390 2024-04-24T09:06:37.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.368+0000 7f23b2a00700 1 -- 172.21.15.69:0/280424110 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23a8004ce0 con 0x7f23bc10a390 2024-04-24T09:06:37.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.368+0000 7f23c0a00700 1 -- 172.21.15.69:0/280424110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23bc1a72b0 con 0x7f23bc10a390 2024-04-24T09:06:37.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.368+0000 7f23b2a00700 1 -- 172.21.15.69:0/280424110 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a80055e0 con 0x7f23bc10a390 2024-04-24T09:06:37.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.368+0000 7f23c0a00700 1 -- 172.21.15.69:0/280424110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23bc1a7870 con 0x7f23bc10a390 2024-04-24T09:06:37.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.369+0000 7f23c0a00700 1 -- 172.21.15.69:0/280424110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23bc0697e0 con 0x7f23bc10a390 2024-04-24T09:06:37.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.371+0000 7f23b2a00700 1 -- 172.21.15.69:0/280424110 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f23a8004750 con 0x7f23bc10a390 2024-04-24T09:06:37.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.372+0000 7f23b2a00700 1 --2- 172.21.15.69:0/280424110 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f23a00a3c40 0x7f23a00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.372+0000 7f23b2a00700 1 -- 172.21.15.69:0/280424110 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f23a80d0310 con 0x7f23bc10a390 2024-04-24T09:06:37.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.373+0000 7f23b9e00700 1 --2- 172.21.15.69:0/280424110 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f23a00a3c40 0x7f23a00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.374+0000 7f23b9e00700 1 --2- 172.21.15.69:0/280424110 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f23a00a3c40 0x7f23a00a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f23ac000c00 tx=0x7f23ac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:37.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.377+0000 7f23b2a00700 1 -- 172.21.15.69:0/280424110 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23a8014030 con 0x7f23bc10a390 2024-04-24T09:06:37.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.617+0000 7f23c0a00700 1 -- 172.21.15.69:0/280424110 --> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f23bc062f30 con 0x7f23a00a3c40 2024-04-24T09:06:37.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.621+0000 7f23b2a00700 1 -- 172.21.15.69:0/280424110 <== mgr.14152 v2:172.21.15.69:6800/1098677190 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f23bc062f30 con 0x7f23a00a3c40 2024-04-24T09:06:37.621 INFO:teuthology.orchestra.run.smithi069.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-24T09:06:37.621 INFO:teuthology.orchestra.run.smithi069.stdout:smithi069 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y0400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-24T09:06:37.621 INFO:teuthology.orchestra.run.smithi069.stdout:smithi081 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400N7400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-24T09:06:37.621 INFO:teuthology.orchestra.run.smithi069.stdout:smithi130 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500Q1400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-24T09:06:37.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.625+0000 7f23c0a00700 1 -- 172.21.15.69:0/280424110 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f23a00a3c40 msgr2=0x7f23a00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:37.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.625+0000 7f23c0a00700 1 --2- 172.21.15.69:0/280424110 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f23a00a3c40 0x7f23a00a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f23ac000c00 tx=0x7f23ac005fb0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.625+0000 7f23c0a00700 1 -- 172.21.15.69:0/280424110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc10a390 msgr2=0x7f23bc19f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:37.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.625+0000 7f23c0a00700 1 --2- 172.21.15.69:0/280424110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc10a390 0x7f23bc19f8f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f23a800c950 tx=0x7f23a800cc60 comp rx=0 tx=0).stop 2024-04-24T09:06:37.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.625+0000 7f23c0a00700 1 -- 172.21.15.69:0/280424110 shutdown_connections 2024-04-24T09:06:37.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.625+0000 7f23c0a00700 1 --2- 172.21.15.69:0/280424110 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f23a00a3c40 0x7f23a00a6100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.625+0000 7f23c0a00700 1 --2- 172.21.15.69:0/280424110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23bc10ad30 0x7f23bc1a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.625+0000 7f23c0a00700 1 --2- 172.21.15.69:0/280424110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23bc109190 0x7f23bc19f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.625+0000 7f23c0a00700 1 --2- 172.21.15.69:0/280424110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23bc10a390 0x7f23bc19f8f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.625+0000 7f23c0a00700 1 -- 172.21.15.69:0/280424110 >> 172.21.15.69:0/280424110 conn(0x7f23bc078160 msgr2=0x7f23bc10d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:37.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.625+0000 7f23c0a00700 1 -- 172.21.15.69:0/280424110 shutdown_connections 2024-04-24T09:06:37.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:37.625+0000 7f23c0a00700 1 -- 172.21.15.69:0/280424110 wait complete. 2024-04-24T09:06:38.610 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T09:06:38.619 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-04-24T09:06:38.619 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T09:06:38.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:38 smithi081 ceph-mon[30961]: pgmap v449: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:06:38.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:38 smithi069 ceph-mon[28170]: pgmap v449: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:06:38.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:38 smithi130 ceph-mon[31553]: pgmap v449: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:06:39.062 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:06:39.770 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:39 smithi069 ceph-mon[28170]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:06:39.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:39 smithi081 ceph-mon[30961]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:06:39.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:39 smithi130 ceph-mon[31553]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:06:40.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:40 smithi081 ceph-mon[30961]: pgmap v450: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:40.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:40 smithi069 ceph-mon[28170]: pgmap v450: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:40.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:40 smithi130 ceph-mon[31553]: pgmap v450: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:41.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.496+0000 7fc29cc00700 1 -- 172.21.15.69:0/3035606429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103d60 msgr2=0x7fc2981041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:41.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.496+0000 7fc29cc00700 1 --2- 172.21.15.69:0/3035606429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103d60 0x7fc2981041c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc2840099c0 tx=0x7fc284009cd0 comp rx=0 tx=0).stop 2024-04-24T09:06:41.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.496+0000 7fc29cc00700 1 -- 172.21.15.69:0/3035606429 shutdown_connections 2024-04-24T09:06:41.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.496+0000 7fc29cc00700 1 --2- 172.21.15.69:0/3035606429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc298104700 0x7fc29810afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:41.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.496+0000 7fc29cc00700 1 --2- 172.21.15.69:0/3035606429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc298102b60 0x7fc298102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:41.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.496+0000 7fc29cc00700 1 --2- 172.21.15.69:0/3035606429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103d60 0x7fc2981041c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:41.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.496+0000 7fc29cc00700 1 -- 172.21.15.69:0/3035606429 >> 172.21.15.69:0/3035606429 conn(0x7fc2980fe0c0 msgr2=0x7fc298100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:41.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.496+0000 7fc29cc00700 1 -- 172.21.15.69:0/3035606429 shutdown_connections 2024-04-24T09:06:41.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.496+0000 7fc29cc00700 1 -- 172.21.15.69:0/3035606429 wait complete. 2024-04-24T09:06:41.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.497+0000 7fc29cc00700 1 Processor -- start 2024-04-24T09:06:41.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.497+0000 7fc29cc00700 1 -- start start 2024-04-24T09:06:41.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.498+0000 7fc29cc00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc298102b60 0x7fc298198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:41.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.498+0000 7fc29cc00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103d60 0x7fc298199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:41.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.498+0000 7fc29cc00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc298104700 0x7fc2981a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:41.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.498+0000 7fc29cc00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc29810e5d0 con 0x7fc298103d60 2024-04-24T09:06:41.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.498+0000 7fc29cc00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc29810e450 con 0x7fc298104700 2024-04-24T09:06:41.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.498+0000 7fc29cc00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc29810e750 con 0x7fc298102b60 2024-04-24T09:06:41.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.498+0000 7fc295400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103d60 0x7fc298199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:41.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.498+0000 7fc295400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103d60 0x7fc298199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46970/0 (socket says 172.21.15.69:46970) 2024-04-24T09:06:41.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.498+0000 7fc295400700 1 -- 172.21.15.69:0/3907213294 learned_addr learned my addr 172.21.15.69:0/3907213294 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:41.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.498+0000 7fc295e00700 1 --2- 172.21.15.69:0/3907213294 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc298102b60 0x7fc298198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:41.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.499+0000 7fc296800700 1 --2- 172.21.15.69:0/3907213294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc298104700 0x7fc2981a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:41.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.499+0000 7fc295400700 1 -- 172.21.15.69:0/3907213294 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc298102b60 msgr2=0x7fc298198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:41.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.499+0000 7fc295400700 1 --2- 172.21.15.69:0/3907213294 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc298102b60 0x7fc298198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:41.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.499+0000 7fc295400700 1 -- 172.21.15.69:0/3907213294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc298104700 msgr2=0x7fc2981a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:41.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.499+0000 7fc295400700 1 --2- 172.21.15.69:0/3907213294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc298104700 0x7fc2981a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:41.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.499+0000 7fc295400700 1 -- 172.21.15.69:0/3907213294 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2981a0840 con 0x7fc298103d60 2024-04-24T09:06:41.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.499+0000 7fc295e00700 1 --2- 172.21.15.69:0/3907213294 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc298102b60 0x7fc298198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:41.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.499+0000 7fc295400700 1 --2- 172.21.15.69:0/3907213294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103d60 0x7fc298199120 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc284000c00 tx=0x7fc2840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:41.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.499+0000 7fc28ea00700 1 -- 172.21.15.69:0/3907213294 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc284027070 con 0x7fc298103d60 2024-04-24T09:06:41.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.499+0000 7fc29cc00700 1 -- 172.21.15.69:0/3907213294 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2981a0ad0 con 0x7fc298103d60 2024-04-24T09:06:41.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.499+0000 7fc28ea00700 1 -- 172.21.15.69:0/3907213294 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc284022470 con 0x7fc298103d60 2024-04-24T09:06:41.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.499+0000 7fc28ea00700 1 -- 172.21.15.69:0/3907213294 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28400f6c0 con 0x7fc298103d60 2024-04-24T09:06:41.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.500+0000 7fc29cc00700 1 -- 172.21.15.69:0/3907213294 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2981a0f80 con 0x7fc298103d60 2024-04-24T09:06:41.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.501+0000 7fc29cc00700 1 -- 172.21.15.69:0/3907213294 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2980697e0 con 0x7fc298103d60 2024-04-24T09:06:41.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.503+0000 7fc28ea00700 1 -- 172.21.15.69:0/3907213294 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc28402a070 con 0x7fc298103d60 2024-04-24T09:06:41.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.504+0000 7fc28ea00700 1 --2- 172.21.15.69:0/3907213294 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc27c0a3cc0 0x7fc27c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:41.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.504+0000 7fc28ea00700 1 -- 172.21.15.69:0/3907213294 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc2840d17b0 con 0x7fc298103d60 2024-04-24T09:06:41.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.504+0000 7fc295e00700 1 --2- 172.21.15.69:0/3907213294 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc27c0a3cc0 0x7fc27c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:41.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.505+0000 7fc295e00700 1 --2- 172.21.15.69:0/3907213294 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc27c0a3cc0 0x7fc27c0a6180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc298103bc0 tx=0x7fc288009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:41.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.509+0000 7fc28ea00700 1 -- 172.21.15.69:0/3907213294 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc28405eff0 con 0x7fc298103d60 2024-04-24T09:06:41.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:41 smithi069 ceph-mon[28170]: pgmap v451: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:06:41.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.797+0000 7fc29cc00700 1 -- 172.21.15.69:0/3907213294 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fc298063700 con 0x7fc298103d60 2024-04-24T09:06:41.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.799+0000 7fc28ea00700 1 -- 172.21.15.69:0/3907213294 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) ==== 76+0+3052 (secure 0 0 0) 0x7fc284013070 con 0x7fc298103d60 2024-04-24T09:06:41.801 INFO:teuthology.orchestra.run.smithi069.stdout:e23 2024-04-24T09:06:41.801 INFO:teuthology.orchestra.run.smithi069.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T09:06:41.801 INFO:teuthology.orchestra.run.smithi069.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T09:06:41.801 INFO:teuthology.orchestra.run.smithi069.stdout:legacy client fscid: 1 2024-04-24T09:06:41.801 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:41.801 INFO:teuthology.orchestra.run.smithi069.stdout:Filesystem 'cephfs' (1) 2024-04-24T09:06:41.801 INFO:teuthology.orchestra.run.smithi069.stdout:fs_name cephfs 2024-04-24T09:06:41.801 INFO:teuthology.orchestra.run.smithi069.stdout:epoch 23 2024-04-24T09:06:41.802 INFO:teuthology.orchestra.run.smithi069.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T09:06:41.802 INFO:teuthology.orchestra.run.smithi069.stdout:created 2024-04-24T09:04:01.991559+0000 2024-04-24T09:06:41.802 INFO:teuthology.orchestra.run.smithi069.stdout:modified 2024-04-24T09:04:34.617681+0000 2024-04-24T09:06:41.802 INFO:teuthology.orchestra.run.smithi069.stdout:tableserver 0 2024-04-24T09:06:41.802 INFO:teuthology.orchestra.run.smithi069.stdout:root 0 2024-04-24T09:06:41.802 INFO:teuthology.orchestra.run.smithi069.stdout:session_timeout 300 2024-04-24T09:06:41.802 INFO:teuthology.orchestra.run.smithi069.stdout:session_autoclose 300 2024-04-24T09:06:41.802 INFO:teuthology.orchestra.run.smithi069.stdout:max_file_size 1099511627776 2024-04-24T09:06:41.802 INFO:teuthology.orchestra.run.smithi069.stdout:max_xattr_size 65536 2024-04-24T09:06:41.802 INFO:teuthology.orchestra.run.smithi069.stdout:required_client_features {} 2024-04-24T09:06:41.802 INFO:teuthology.orchestra.run.smithi069.stdout:last_failure 0 2024-04-24T09:06:41.802 INFO:teuthology.orchestra.run.smithi069.stdout:last_failure_osd_epoch 0 2024-04-24T09:06:41.802 INFO:teuthology.orchestra.run.smithi069.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T09:06:41.802 INFO:teuthology.orchestra.run.smithi069.stdout:max_mds 3 2024-04-24T09:06:41.803 INFO:teuthology.orchestra.run.smithi069.stdout:in 0,1,2 2024-04-24T09:06:41.803 INFO:teuthology.orchestra.run.smithi069.stdout:up {0=34532,1=34496,2=34460} 2024-04-24T09:06:41.803 INFO:teuthology.orchestra.run.smithi069.stdout:failed 2024-04-24T09:06:41.803 INFO:teuthology.orchestra.run.smithi069.stdout:damaged 2024-04-24T09:06:41.803 INFO:teuthology.orchestra.run.smithi069.stdout:stopped 2024-04-24T09:06:41.803 INFO:teuthology.orchestra.run.smithi069.stdout:data_pools [3,4] 2024-04-24T09:06:41.803 INFO:teuthology.orchestra.run.smithi069.stdout:metadata_pool 2 2024-04-24T09:06:41.803 INFO:teuthology.orchestra.run.smithi069.stdout:inline_data disabled 2024-04-24T09:06:41.803 INFO:teuthology.orchestra.run.smithi069.stdout:balancer 2024-04-24T09:06:41.803 INFO:teuthology.orchestra.run.smithi069.stdout:bal_rank_mask -1 2024-04-24T09:06:41.803 INFO:teuthology.orchestra.run.smithi069.stdout:standby_count_wanted 0 2024-04-24T09:06:41.803 INFO:teuthology.orchestra.run.smithi069.stdout:qdb_cluster leader: 34532 members: 34460,34496,34532 2024-04-24T09:06:41.803 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.b{0:34532} state up:active seq 3 addr [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:06:41.803 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.k{0:34427} state up:standby-replay seq 1 addr [v2:172.21.15.81:6832/3240654002,v1:172.21.15.81:6833/3240654002] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:06:41.804 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.h{1:34496} state up:active seq 14 addr [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:06:41.804 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.i{1:24487} state up:standby-replay seq 1 addr [v2:172.21.15.130:6838/4193111366,v1:172.21.15.130:6839/4193111366] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:06:41.804 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.e{2:34460} state up:active seq 18 addr [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:06:41.804 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.c{2:24469} state up:standby-replay seq 1 addr [v2:172.21.15.130:6836/2790982303,v1:172.21.15.130:6837/2790982303] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:06:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:41.804 INFO:teuthology.orchestra.run.smithi069.stdout:Standby daemons: 2024-04-24T09:06:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:41.804 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.j{-1:14526} state up:standby seq 1 addr [v2:172.21.15.69:6834/106106849,v1:172.21.15.69:6835/106106849] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:06:41.804 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.d{-1:14544} state up:standby seq 1 addr [v2:172.21.15.69:6836/2217103498,v1:172.21.15.69:6837/2217103498] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:06:41.804 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.g{-1:14622} state up:standby seq 1 addr [v2:172.21.15.69:6838/3637851934,v1:172.21.15.69:6839/3637851934] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:06:41.804 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.a{-1:14640} state up:standby seq 1 addr [v2:172.21.15.69:6840/1747080062,v1:172.21.15.69:6841/1747080062] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:06:41.804 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.l{-1:24403} state up:standby seq 1 addr [v2:172.21.15.130:6832/1510374637,v1:172.21.15.130:6833/1510374637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:06:41.804 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.f{-1:24454} state up:standby seq 1 addr [v2:172.21.15.130:6834/1889126439,v1:172.21.15.130:6835/1889126439] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:06:41.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.802+0000 7fc29cc00700 1 -- 172.21.15.69:0/3907213294 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc27c0a3cc0 msgr2=0x7fc27c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:41.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.802+0000 7fc29cc00700 1 --2- 172.21.15.69:0/3907213294 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc27c0a3cc0 0x7fc27c0a6180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc298103bc0 tx=0x7fc288009040 comp rx=0 tx=0).stop 2024-04-24T09:06:41.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.802+0000 7fc29cc00700 1 -- 172.21.15.69:0/3907213294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103d60 msgr2=0x7fc298199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:41.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.802+0000 7fc29cc00700 1 --2- 172.21.15.69:0/3907213294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103d60 0x7fc298199120 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc284000c00 tx=0x7fc2840047b0 comp rx=0 tx=0).stop 2024-04-24T09:06:41.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.802+0000 7fc29cc00700 1 -- 172.21.15.69:0/3907213294 shutdown_connections 2024-04-24T09:06:41.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.802+0000 7fc29cc00700 1 --2- 172.21.15.69:0/3907213294 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc27c0a3cc0 0x7fc27c0a6180 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:41.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.802+0000 7fc29cc00700 1 --2- 172.21.15.69:0/3907213294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc298104700 0x7fc2981a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:41.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.802+0000 7fc29cc00700 1 --2- 172.21.15.69:0/3907213294 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc298102b60 0x7fc298198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:41.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.802+0000 7fc29cc00700 1 --2- 172.21.15.69:0/3907213294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103d60 0x7fc298199120 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:41.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.802+0000 7fc29cc00700 1 -- 172.21.15.69:0/3907213294 >> 172.21.15.69:0/3907213294 conn(0x7fc2980fe0c0 msgr2=0x7fc298107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:41.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.802+0000 7fc29cc00700 1 -- 172.21.15.69:0/3907213294 shutdown_connections 2024-04-24T09:06:41.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:41.802+0000 7fc29cc00700 1 -- 172.21.15.69:0/3907213294 wait complete. 2024-04-24T09:06:41.806 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2024-04-24T09:06:41.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:41 smithi130 ceph-mon[31553]: pgmap v451: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:06:42.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:41 smithi081 ceph-mon[30961]: pgmap v451: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:06:42.787 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-24T09:06:42.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:42 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3907213294' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T09:06:42.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:42 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3907213294' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T09:06:43.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:42 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3907213294' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T09:06:43.249 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:06:43.779 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:43 smithi069 ceph-mon[28170]: pgmap v452: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:06:43.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:43 smithi130 ceph-mon[31553]: pgmap v452: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:06:44.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:43 smithi081 ceph-mon[30961]: pgmap v452: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:06:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.773+0000 7f2697c00700 1 -- 172.21.15.69:0/430220689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2690106930 msgr2=0x7f269010d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.773+0000 7f2697c00700 1 --2- 172.21.15.69:0/430220689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2690106930 0x7f269010d1e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f268800b0d0 tx=0x7f268800b3e0 comp rx=0 tx=0).stop 2024-04-24T09:06:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.773+0000 7f2697c00700 1 -- 172.21.15.69:0/430220689 shutdown_connections 2024-04-24T09:06:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.773+0000 7f2697c00700 1 --2- 172.21.15.69:0/430220689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2690106930 0x7f269010d1e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.773+0000 7f2697c00700 1 --2- 172.21.15.69:0/430220689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2690104d90 0x7f26901051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.773+0000 7f2697c00700 1 --2- 172.21.15.69:0/430220689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2690105f90 0x7f26901063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.773+0000 7f2697c00700 1 -- 172.21.15.69:0/430220689 >> 172.21.15.69:0/430220689 conn(0x7f2690100350 msgr2=0x7f2690102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.773+0000 7f2697c00700 1 -- 172.21.15.69:0/430220689 shutdown_connections 2024-04-24T09:06:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.773+0000 7f2697c00700 1 -- 172.21.15.69:0/430220689 wait complete. 2024-04-24T09:06:45.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.774+0000 7f2697c00700 1 Processor -- start 2024-04-24T09:06:45.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.774+0000 7f2697c00700 1 -- start start 2024-04-24T09:06:45.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.775+0000 7f2697c00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2690104d90 0x7f269019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:45.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.775+0000 7f2697c00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2690105f90 0x7f269019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:45.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.775+0000 7f2697c00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2690106930 0x7f26901a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:45.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.775+0000 7f2697c00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2690110880 con 0x7f2690104d90 2024-04-24T09:06:45.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.775+0000 7f2697c00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2690110700 con 0x7f2690105f90 2024-04-24T09:06:45.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.775+0000 7f2697c00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2690110a00 con 0x7f2690106930 2024-04-24T09:06:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.775+0000 7f2695200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2690104d90 0x7f269019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.775+0000 7f2695200700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2690104d90 0x7f269019ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60818/0 (socket says 172.21.15.69:60818) 2024-04-24T09:06:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.775+0000 7f2695200700 1 -- 172.21.15.69:0/1130363006 learned_addr learned my addr 172.21.15.69:0/1130363006 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.775+0000 7f268fe00700 1 --2- 172.21.15.69:0/1130363006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2690105f90 0x7f269019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.776+0000 7f2695c00700 1 --2- 172.21.15.69:0/1130363006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2690106930 0x7f26901a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.776+0000 7f2695200700 1 -- 172.21.15.69:0/1130363006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2690106930 msgr2=0x7f26901a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.776+0000 7f2695200700 1 --2- 172.21.15.69:0/1130363006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2690106930 0x7f26901a2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.776+0000 7f2695200700 1 -- 172.21.15.69:0/1130363006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2690105f90 msgr2=0x7f269019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.776+0000 7f2695200700 1 --2- 172.21.15.69:0/1130363006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2690105f90 0x7f269019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.776+0000 7f2695200700 1 -- 172.21.15.69:0/1130363006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26901a2a00 con 0x7f2690104d90 2024-04-24T09:06:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.776+0000 7f2695c00700 1 --2- 172.21.15.69:0/1130363006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2690106930 0x7f26901a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.776+0000 7f2695200700 1 --2- 172.21.15.69:0/1130363006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2690104d90 0x7f269019ada0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f268400c9c0 tx=0x7f268400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.777+0000 7f268d600700 1 -- 172.21.15.69:0/1130363006 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684007ac0 con 0x7f2690104d90 2024-04-24T09:06:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.777+0000 7f268d600700 1 -- 172.21.15.69:0/1130363006 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2684007c60 con 0x7f2690104d90 2024-04-24T09:06:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.777+0000 7f2697c00700 1 -- 172.21.15.69:0/1130363006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26901a2cf0 con 0x7f2690104d90 2024-04-24T09:06:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.777+0000 7f268d600700 1 -- 172.21.15.69:0/1130363006 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684018660 con 0x7f2690104d90 2024-04-24T09:06:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.777+0000 7f2697c00700 1 -- 172.21.15.69:0/1130363006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26901a3180 con 0x7f2690104d90 2024-04-24T09:06:45.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.778+0000 7f2697c00700 1 -- 172.21.15.69:0/1130363006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f269002dfe0 con 0x7f2690104d90 2024-04-24T09:06:45.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.780+0000 7f268d600700 1 -- 172.21.15.69:0/1130363006 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f268400f450 con 0x7f2690104d90 2024-04-24T09:06:45.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.781+0000 7f268d600700 1 --2- 172.21.15.69:0/1130363006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f26780a3c40 0x7f26780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:45.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.781+0000 7f268d600700 1 -- 172.21.15.69:0/1130363006 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f26840d0840 con 0x7f2690104d90 2024-04-24T09:06:45.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.782+0000 7f268fe00700 1 --2- 172.21.15.69:0/1130363006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f26780a3c40 0x7f26780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.782+0000 7f268fe00700 1 --2- 172.21.15.69:0/1130363006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f26780a3c40 0x7f26780a6100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f267c005950 tx=0x7f267c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:45.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:45.786+0000 7f268d600700 1 -- 172.21.15.69:0/1130363006 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2684014030 con 0x7f2690104d90 2024-04-24T09:06:46.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:46.014+0000 7f2697c00700 1 -- 172.21.15.69:0/1130363006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f2690063800 con 0x7f2690104d90 2024-04-24T09:06:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:46.015+0000 7f268d600700 1 -- 172.21.15.69:0/1130363006 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) ==== 56+0+5591 (secure 0 0 0) 0x7f268409c810 con 0x7f2690104d90 2024-04-24T09:06:46.016 INFO:teuthology.orchestra.run.smithi069.stdout:epoch 92 2024-04-24T09:06:46.016 INFO:teuthology.orchestra.run.smithi069.stdout:fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T09:06:46.016 INFO:teuthology.orchestra.run.smithi069.stdout:created 2024-04-24T08:53:15.725881+0000 2024-04-24T09:06:46.016 INFO:teuthology.orchestra.run.smithi069.stdout:modified 2024-04-24T09:04:24.497245+0000 2024-04-24T09:06:46.016 INFO:teuthology.orchestra.run.smithi069.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-24T09:06:46.016 INFO:teuthology.orchestra.run.smithi069.stdout:crush_version 27 2024-04-24T09:06:46.017 INFO:teuthology.orchestra.run.smithi069.stdout:full_ratio 0.95 2024-04-24T09:06:46.017 INFO:teuthology.orchestra.run.smithi069.stdout:backfillfull_ratio 0.9 2024-04-24T09:06:46.017 INFO:teuthology.orchestra.run.smithi069.stdout:nearfull_ratio 0.85 2024-04-24T09:06:46.017 INFO:teuthology.orchestra.run.smithi069.stdout:require_min_compat_client luminous 2024-04-24T09:06:46.017 INFO:teuthology.orchestra.run.smithi069.stdout:min_compat_client luminous 2024-04-24T09:06:46.017 INFO:teuthology.orchestra.run.smithi069.stdout:require_osd_release squid 2024-04-24T09:06:46.017 INFO:teuthology.orchestra.run.smithi069.stdout:stretch_mode_enabled false 2024-04-24T09:06:46.017 INFO:teuthology.orchestra.run.smithi069.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-24T09:06:46.017 INFO:teuthology.orchestra.run.smithi069.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-24T09:06:46.017 INFO:teuthology.orchestra.run.smithi069.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-24T09:06:46.017 INFO:teuthology.orchestra.run.smithi069.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-24T09:06:46.017 INFO:teuthology.orchestra.run.smithi069.stdout:max_osd 12 2024-04-24T09:06:46.017 INFO:teuthology.orchestra.run.smithi069.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6802/340713595,v1:172.21.15.69:6803/340713595] [v2:172.21.15.69:6804/340713595,v1:172.21.15.69:6805/340713595] exists,up 2c221fcb-ab5f-4f37-b5b4-40366081fbb9 2024-04-24T09:06:46.017 INFO:teuthology.orchestra.run.smithi069.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6800/3515249330,v1:172.21.15.81:6801/3515249330] [v2:172.21.15.81:6802/3515249330,v1:172.21.15.81:6803/3515249330] exists,up dae4a019-875a-48ff-96d8-c19efa5987ff 2024-04-24T09:06:46.018 INFO:teuthology.orchestra.run.smithi069.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6800/3109352638,v1:172.21.15.130:6801/3109352638] [v2:172.21.15.130:6802/3109352638,v1:172.21.15.130:6803/3109352638] exists,up 62f0fd9a-64fe-4680-9107-ae8a97490dec 2024-04-24T09:06:46.018 INFO:teuthology.orchestra.run.smithi069.stdout:osd.3 up in weight 1 up_from 25 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6810/3549681529,v1:172.21.15.69:6811/3549681529] [v2:172.21.15.69:6812/3549681529,v1:172.21.15.69:6813/3549681529] exists,up 09c5d15c-129e-45cf-9891-5b7a7f4a450f 2024-04-24T09:06:46.018 INFO:teuthology.orchestra.run.smithi069.stdout:osd.4 up in weight 1 up_from 33 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6808/575869990,v1:172.21.15.81:6809/575869990] [v2:172.21.15.81:6810/575869990,v1:172.21.15.81:6811/575869990] exists,up ae0c645d-b56e-4348-88c5-bf4f6738e0a1 2024-04-24T09:06:46.018 INFO:teuthology.orchestra.run.smithi069.stdout:osd.5 up in weight 1 up_from 38 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6808/1208873184,v1:172.21.15.130:6809/1208873184] [v2:172.21.15.130:6810/1208873184,v1:172.21.15.130:6811/1208873184] exists,up dd082142-bbe7-40f2-85c3-94cf0cf96466 2024-04-24T09:06:46.018 INFO:teuthology.orchestra.run.smithi069.stdout:osd.6 up in weight 1 up_from 46 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6818/2606282387,v1:172.21.15.69:6819/2606282387] [v2:172.21.15.69:6820/2606282387,v1:172.21.15.69:6821/2606282387] exists,up 58e89c83-2d93-42a3-9fa4-e6b08810c776 2024-04-24T09:06:46.018 INFO:teuthology.orchestra.run.smithi069.stdout:osd.7 up in weight 1 up_from 51 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6816/4105801153,v1:172.21.15.81:6817/4105801153] [v2:172.21.15.81:6818/4105801153,v1:172.21.15.81:6819/4105801153] exists,up f1daca7a-630f-4a56-bd3d-713fc3f88ede 2024-04-24T09:06:46.018 INFO:teuthology.orchestra.run.smithi069.stdout:osd.8 up in weight 1 up_from 59 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6816/3037039143,v1:172.21.15.130:6817/3037039143] [v2:172.21.15.130:6818/3037039143,v1:172.21.15.130:6819/3037039143] exists,up 151820a5-68cc-4d46-af26-eff7f37e3186 2024-04-24T09:06:46.018 INFO:teuthology.orchestra.run.smithi069.stdout:osd.9 up in weight 1 up_from 67 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6826/4110734159,v1:172.21.15.69:6827/4110734159] [v2:172.21.15.69:6828/4110734159,v1:172.21.15.69:6829/4110734159] exists,up 9155e0e5-52f4-48d3-b024-cf346de31211 2024-04-24T09:06:46.018 INFO:teuthology.orchestra.run.smithi069.stdout:osd.10 up in weight 1 up_from 73 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6824/3563037549,v1:172.21.15.81:6825/3563037549] [v2:172.21.15.81:6826/3563037549,v1:172.21.15.81:6827/3563037549] exists,up a2f8028d-7966-4ceb-9e12-a728d89665ee 2024-04-24T09:06:46.018 INFO:teuthology.orchestra.run.smithi069.stdout:osd.11 up in weight 1 up_from 81 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6824/1208587980,v1:172.21.15.130:6825/1208587980] [v2:172.21.15.130:6826/1208587980,v1:172.21.15.130:6827/1208587980] exists,up ebe21375-9751-4541-9dbf-0d5e0c4bbd5b 2024-04-24T09:06:46.018 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.4 [2,11] 2024-04-24T09:06:46.018 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.6 [8,11] 2024-04-24T09:06:46.018 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.1b [2,4] 2024-04-24T09:06:46.018 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.2 [3,9] 2024-04-24T09:06:46.019 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.13 [4,9] 2024-04-24T09:06:46.019 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.24 [4,9] 2024-04-24T09:06:46.019 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 4.2 [9,1] 2024-04-24T09:06:46.019 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 4.3 [5,8] 2024-04-24T09:06:46.019 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 4.e [9,11] 2024-04-24T09:06:46.019 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 4.12 [9,10] 2024-04-24T09:06:46.019 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/3826117580 expires 2024-04-25T08:54:23.196411+0000 2024-04-24T09:06:46.019 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/1006368686 expires 2024-04-25T08:54:23.196411+0000 2024-04-24T09:06:46.019 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6800/3470783122 expires 2024-04-25T08:54:23.196411+0000 2024-04-24T09:06:46.019 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/2102503077 expires 2024-04-25T08:53:48.948375+0000 2024-04-24T09:06:46.019 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6801/3470783122 expires 2024-04-25T08:54:23.196411+0000 2024-04-24T09:06:46.019 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/1041454348 expires 2024-04-25T08:53:48.948375+0000 2024-04-24T09:06:46.019 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6800/3911821813 expires 2024-04-25T08:53:48.948375+0000 2024-04-24T09:06:46.019 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6801/3911821813 expires 2024-04-25T08:53:48.948375+0000 2024-04-24T09:06:46.019 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/4072155336 expires 2024-04-25T08:53:48.948375+0000 2024-04-24T09:06:46.020 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/1278076203 expires 2024-04-25T08:54:23.196411+0000 2024-04-24T09:06:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:46.018+0000 7f2697c00700 1 -- 172.21.15.69:0/1130363006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f26780a3c40 msgr2=0x7f26780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:46.018+0000 7f2697c00700 1 --2- 172.21.15.69:0/1130363006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f26780a3c40 0x7f26780a6100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f267c005950 tx=0x7f267c00a300 comp rx=0 tx=0).stop 2024-04-24T09:06:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:46.019+0000 7f2697c00700 1 -- 172.21.15.69:0/1130363006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2690104d90 msgr2=0x7f269019ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:46.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:46.019+0000 7f2697c00700 1 --2- 172.21.15.69:0/1130363006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2690104d90 0x7f269019ada0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f268400c9c0 tx=0x7f268400ccd0 comp rx=0 tx=0).stop 2024-04-24T09:06:46.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:46.019+0000 7f2697c00700 1 -- 172.21.15.69:0/1130363006 shutdown_connections 2024-04-24T09:06:46.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:46.019+0000 7f2697c00700 1 --2- 172.21.15.69:0/1130363006 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f26780a3c40 0x7f26780a6100 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:46.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:46.019+0000 7f2697c00700 1 --2- 172.21.15.69:0/1130363006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2690106930 0x7f26901a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:46.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:46.019+0000 7f2697c00700 1 --2- 172.21.15.69:0/1130363006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2690104d90 0x7f269019ada0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:46.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:46.019+0000 7f2697c00700 1 --2- 172.21.15.69:0/1130363006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2690105f90 0x7f269019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:46.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:46.019+0000 7f2697c00700 1 -- 172.21.15.69:0/1130363006 >> 172.21.15.69:0/1130363006 conn(0x7f2690100350 msgr2=0x7f2690109230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:46.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:46.019+0000 7f2697c00700 1 -- 172.21.15.69:0/1130363006 shutdown_connections 2024-04-24T09:06:46.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:46.019+0000 7f2697c00700 1 -- 172.21.15.69:0/1130363006 wait complete. 2024-04-24T09:06:46.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:46 smithi081 ceph-mon[30961]: pgmap v453: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:46.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:46 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1130363006' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T09:06:46.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:46 smithi069 ceph-mon[28170]: pgmap v453: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:46.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:46 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1130363006' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T09:06:46.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:46 smithi130 ceph-mon[31553]: pgmap v453: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:46.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:46 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1130363006' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T09:06:46.989 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-24T09:06:47.003 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:06:47.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.216+0000 7fb6f0800640 1 -- 172.21.15.69:0/1648368199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8108660 msgr2=0x7fb6e8108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:47.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.216+0000 7fb6f0800640 1 --2- 172.21.15.69:0/1648368199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8108660 0x7fb6e8108a80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009900 tx=0x7fb6dc02a410 comp rx=0 tx=0).stop 2024-04-24T09:06:47.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.216+0000 7fb6f0800640 1 -- 172.21.15.69:0/1648368199 shutdown_connections 2024-04-24T09:06:47.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.216+0000 7fb6f0800640 1 --2- 172.21.15.69:0/1648368199 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6e810a210 0x7fb6e8110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.216+0000 7fb6f0800640 1 --2- 172.21.15.69:0/1648368199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6e8109850 0x7fb6e8109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.216+0000 7fb6f0800640 1 --2- 172.21.15.69:0/1648368199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8108660 0x7fb6e8108a80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.216+0000 7fb6f0800640 1 -- 172.21.15.69:0/1648368199 >> 172.21.15.69:0/1648368199 conn(0x7fb6e8077620 msgr2=0x7fb6e8079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:47.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.216+0000 7fb6f0800640 1 -- 172.21.15.69:0/1648368199 shutdown_connections 2024-04-24T09:06:47.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.217+0000 7fb6f0800640 1 -- 172.21.15.69:0/1648368199 wait complete. 2024-04-24T09:06:47.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.218+0000 7fb6f0800640 1 Processor -- start 2024-04-24T09:06:47.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.218+0000 7fb6f0800640 1 -- start start 2024-04-24T09:06:47.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.218+0000 7fb6f0800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6e8108660 0x7fb6e819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:47.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.218+0000 7fb6f0800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8109850 0x7fb6e819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:47.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.218+0000 7fb6f0800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6e810a210 0x7fb6e81a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:47.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.218+0000 7fb6f0800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8114190 con 0x7fb6e8109850 2024-04-24T09:06:47.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.218+0000 7fb6f0800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8114010 con 0x7fb6e8108660 2024-04-24T09:06:47.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.218+0000 7fb6f0800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8114310 con 0x7fb6e810a210 2024-04-24T09:06:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.219+0000 7fb6ed400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8109850 0x7fb6e819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.219+0000 7fb6ed400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8109850 0x7fb6e819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60838/0 (socket says 172.21.15.69:60838) 2024-04-24T09:06:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.219+0000 7fb6ed400640 1 -- 172.21.15.69:0/1123063383 learned_addr learned my addr 172.21.15.69:0/1123063383 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.219+0000 7fb6ede00640 1 --2- 172.21.15.69:0/1123063383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6e8108660 0x7fb6e819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.219+0000 7fb6ee800640 1 --2- 172.21.15.69:0/1123063383 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6e810a210 0x7fb6e81a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.219+0000 7fb6ed400640 1 -- 172.21.15.69:0/1123063383 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6e810a210 msgr2=0x7fb6e81a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.219+0000 7fb6ed400640 1 --2- 172.21.15.69:0/1123063383 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6e810a210 0x7fb6e81a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.219+0000 7fb6ed400640 1 -- 172.21.15.69:0/1123063383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6e8108660 msgr2=0x7fb6e819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.219+0000 7fb6ed400640 1 --2- 172.21.15.69:0/1123063383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6e8108660 0x7fb6e819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.219+0000 7fb6ed400640 1 -- 172.21.15.69:0/1123063383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e81a64f0 con 0x7fb6e8109850 2024-04-24T09:06:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.219+0000 7fb6ede00640 1 --2- 172.21.15.69:0/1123063383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6e8108660 0x7fb6e819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.220+0000 7fb6ed400640 1 --2- 172.21.15.69:0/1123063383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8109850 0x7fb6e819ee10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800d950 tx=0x7fb6d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:47.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.220+0000 7fb6e2a00640 1 -- 172.21.15.69:0/1123063383 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8017070 con 0x7fb6e8109850 2024-04-24T09:06:47.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.220+0000 7fb6e2a00640 1 -- 172.21.15.69:0/1123063383 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6d8009c30 con 0x7fb6e8109850 2024-04-24T09:06:47.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.220+0000 7fb6f0800640 1 -- 172.21.15.69:0/1123063383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e81a67e0 con 0x7fb6e8109850 2024-04-24T09:06:47.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.220+0000 7fb6e2a00640 1 -- 172.21.15.69:0/1123063383 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d800f620 con 0x7fb6e8109850 2024-04-24T09:06:47.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.220+0000 7fb6f0800640 1 -- 172.21.15.69:0/1123063383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e81a6d30 con 0x7fb6e8109850 2024-04-24T09:06:47.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.222+0000 7fb6f0800640 1 -- 172.21.15.69:0/1123063383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e810faa0 con 0x7fb6e8109850 2024-04-24T09:06:47.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.224+0000 7fb6e2a00640 1 -- 172.21.15.69:0/1123063383 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb6d80109a0 con 0x7fb6e8109850 2024-04-24T09:06:47.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.225+0000 7fb6e2a00640 1 --2- 172.21.15.69:0/1123063383 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb6d00a3bb0 0x7fb6d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:47.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.226+0000 7fb6e2a00640 1 -- 172.21.15.69:0/1123063383 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb6d80d04a0 con 0x7fb6e8109850 2024-04-24T09:06:47.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.226+0000 7fb6ede00640 1 --2- 172.21.15.69:0/1123063383 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb6d00a3bb0 0x7fb6d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:47.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.227+0000 7fb6ede00640 1 --2- 172.21.15.69:0/1123063383 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb6d00a3bb0 0x7fb6d00a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc000c00 tx=0x7fb6dc032c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:47.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.230+0000 7fb6e2a00640 1 -- 172.21.15.69:0/1123063383 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6d8009de0 con 0x7fb6e8109850 2024-04-24T09:06:47.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.434+0000 7fb6f0800640 1 -- 172.21.15.69:0/1123063383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6e8062c00 con 0x7fb6e8109850 2024-04-24T09:06:47.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.439+0000 7fb6e2a00640 1 -- 172.21.15.69:0/1123063383 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24564 (secure 0 0 0) 0x7fb6d809cac0 con 0x7fb6e8109850 2024-04-24T09:06:47.439 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:47.439 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":92,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:24.497245+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:06:47.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.441+0000 7fb6f0800640 1 -- 172.21.15.69:0/1123063383 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb6d00a3bb0 msgr2=0x7fb6d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:47.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.442+0000 7fb6f0800640 1 --2- 172.21.15.69:0/1123063383 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb6d00a3bb0 0x7fb6d00a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc000c00 tx=0x7fb6dc032c40 comp rx=0 tx=0).stop 2024-04-24T09:06:47.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.442+0000 7fb6f0800640 1 -- 172.21.15.69:0/1123063383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8109850 msgr2=0x7fb6e819ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:47.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.442+0000 7fb6f0800640 1 --2- 172.21.15.69:0/1123063383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8109850 0x7fb6e819ee10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800d950 tx=0x7fb6d800de10 comp rx=0 tx=0).stop 2024-04-24T09:06:47.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.442+0000 7fb6f0800640 1 -- 172.21.15.69:0/1123063383 shutdown_connections 2024-04-24T09:06:47.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.442+0000 7fb6f0800640 1 --2- 172.21.15.69:0/1123063383 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6e810a210 0x7fb6e81a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.442+0000 7fb6f0800640 1 --2- 172.21.15.69:0/1123063383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6e8109850 0x7fb6e819ee10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.442+0000 7fb6f0800640 1 --2- 172.21.15.69:0/1123063383 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb6d00a3bb0 0x7fb6d00a6070 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.442+0000 7fb6f0800640 1 --2- 172.21.15.69:0/1123063383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6e8108660 0x7fb6e819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.442+0000 7fb6f0800640 1 -- 172.21.15.69:0/1123063383 >> 172.21.15.69:0/1123063383 conn(0x7fb6e8077620 msgr2=0x7fb6e810cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:47.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.442+0000 7fb6f0800640 1 -- 172.21.15.69:0/1123063383 shutdown_connections 2024-04-24T09:06:47.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.442+0000 7fb6f0800640 1 -- 172.21.15.69:0/1123063383 wait complete. 2024-04-24T09:06:47.462 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:57:37.377599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:04:00.299827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:04:00.958498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T09:04:04.105314+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T09:06:47.463 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:06:47.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.668+0000 7f136e600640 1 -- 172.21.15.69:0/3008680901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1368105cf0 msgr2=0x7f136810c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:47.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.668+0000 7f136e600640 1 --2- 172.21.15.69:0/3008680901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1368105cf0 0x7f136810c580 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1360009ca0 tx=0x7f136002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.668+0000 7f136e600640 1 -- 172.21.15.69:0/3008680901 shutdown_connections 2024-04-24T09:06:47.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.668+0000 7f136e600640 1 --2- 172.21.15.69:0/3008680901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1368105cf0 0x7f136810c580 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.668+0000 7f136e600640 1 --2- 172.21.15.69:0/3008680901 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1368105330 0x7f13681057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.668+0000 7f136e600640 1 --2- 172.21.15.69:0/3008680901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1368104140 0x7f1368104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.668+0000 7f136e600640 1 -- 172.21.15.69:0/3008680901 >> 172.21.15.69:0/3008680901 conn(0x7f13680ff6b0 msgr2=0x7f1368101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:47.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.668+0000 7f136e600640 1 -- 172.21.15.69:0/3008680901 shutdown_connections 2024-04-24T09:06:47.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.668+0000 7f136e600640 1 -- 172.21.15.69:0/3008680901 wait complete. 2024-04-24T09:06:47.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.669+0000 7f136e600640 1 Processor -- start 2024-04-24T09:06:47.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.669+0000 7f136e600640 1 -- start start 2024-04-24T09:06:47.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.670+0000 7f136e600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1368104140 0x7f136819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:47.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.670+0000 7f136e600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1368105330 0x7f136819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:47.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.670+0000 7f136e600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1368105cf0 0x7f13681a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:47.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.670+0000 7f136e600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f136810fc80 con 0x7f1368105cf0 2024-04-24T09:06:47.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.670+0000 7f136e600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f136810fb00 con 0x7f1368105330 2024-04-24T09:06:47.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.670+0000 7f136e600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f136810fe00 con 0x7f1368104140 2024-04-24T09:06:47.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.670+0000 7f136dc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1368105cf0 0x7f13681a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:47.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.670+0000 7f136dc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1368105cf0 0x7f13681a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60848/0 (socket says 172.21.15.69:60848) 2024-04-24T09:06:47.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.670+0000 7f136dc00640 1 -- 172.21.15.69:0/3616196637 learned_addr learned my addr 172.21.15.69:0/3616196637 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:47.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.670+0000 7f135fe00640 1 --2- 172.21.15.69:0/3616196637 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1368105330 0x7f136819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:47.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.670+0000 7f136d200640 1 --2- 172.21.15.69:0/3616196637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1368104140 0x7f136819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.670+0000 7f136dc00640 1 -- 172.21.15.69:0/3616196637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1368104140 msgr2=0x7f136819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.670+0000 7f136dc00640 1 --2- 172.21.15.69:0/3616196637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1368104140 0x7f136819a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.670+0000 7f136dc00640 1 -- 172.21.15.69:0/3616196637 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1368105330 msgr2=0x7f136819a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.671+0000 7f136dc00640 1 --2- 172.21.15.69:0/3616196637 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1368105330 0x7f136819a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.671+0000 7f136dc00640 1 -- 172.21.15.69:0/3616196637 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13681a1ef0 con 0x7f1368105cf0 2024-04-24T09:06:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.671+0000 7f135fe00640 1 --2- 172.21.15.69:0/3616196637 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1368105330 0x7f136819a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.671+0000 7f136dc00640 1 --2- 172.21.15.69:0/3616196637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1368105cf0 0x7f13681a1880 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1360009dc0 tx=0x7f1360003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.671+0000 7f1366000640 1 -- 172.21.15.69:0/3616196637 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360032ec0 con 0x7f1368105cf0 2024-04-24T09:06:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.671+0000 7f1366000640 1 -- 172.21.15.69:0/3616196637 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13600044f0 con 0x7f1368105cf0 2024-04-24T09:06:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.671+0000 7f136e600640 1 -- 172.21.15.69:0/3616196637 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13681a2180 con 0x7f1368105cf0 2024-04-24T09:06:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.671+0000 7f1366000640 1 -- 172.21.15.69:0/3616196637 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360030d40 con 0x7f1368105cf0 2024-04-24T09:06:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.671+0000 7f136e600640 1 -- 172.21.15.69:0/3616196637 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13681a26d0 con 0x7f1368105cf0 2024-04-24T09:06:47.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.672+0000 7f136e600640 1 -- 172.21.15.69:0/3616196637 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f136810b460 con 0x7f1368105cf0 2024-04-24T09:06:47.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.674+0000 7f1366000640 1 -- 172.21.15.69:0/3616196637 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1360004040 con 0x7f1368105cf0 2024-04-24T09:06:47.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.676+0000 7f1366000640 1 --2- 172.21.15.69:0/3616196637 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13480a3d50 0x7f13480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:47.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.676+0000 7f1366000640 1 -- 172.21.15.69:0/3616196637 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f13600f1e70 con 0x7f1368105cf0 2024-04-24T09:06:47.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.676+0000 7f136d200640 1 --2- 172.21.15.69:0/3616196637 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13480a3d50 0x7f13480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:47.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.677+0000 7f136d200640 1 --2- 172.21.15.69:0/3616196637 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13480a3d50 0x7f13480a6210 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1350005fd0 tx=0x7f1350005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:47.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.680+0000 7f1366000640 1 -- 172.21.15.69:0/3616196637 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f13600be490 con 0x7f1368105cf0 2024-04-24T09:06:47.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:47 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1123063383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:47.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:47 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1123063383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.883+0000 7f136e600640 1 -- 172.21.15.69:0/3616196637 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1368062c00 con 0x7f1368105cf0 2024-04-24T09:06:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.884+0000 7f1366000640 1 -- 172.21.15.69:0/3616196637 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f13600bdd90 con 0x7f1368105cf0 2024-04-24T09:06:47.884 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-24T09:06:47.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.887+0000 7f136e600640 1 -- 172.21.15.69:0/3616196637 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13480a3d50 msgr2=0x7f13480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:47.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.887+0000 7f136e600640 1 --2- 172.21.15.69:0/3616196637 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13480a3d50 0x7f13480a6210 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1350005fd0 tx=0x7f1350005850 comp rx=0 tx=0).stop 2024-04-24T09:06:47.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.887+0000 7f136e600640 1 -- 172.21.15.69:0/3616196637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1368105cf0 msgr2=0x7f13681a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:47.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.887+0000 7f136e600640 1 --2- 172.21.15.69:0/3616196637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1368105cf0 0x7f13681a1880 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1360009dc0 tx=0x7f1360003ba0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.887+0000 7f136e600640 1 -- 172.21.15.69:0/3616196637 shutdown_connections 2024-04-24T09:06:47.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.887+0000 7f136e600640 1 --2- 172.21.15.69:0/3616196637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1368105cf0 0x7f13681a1880 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.887+0000 7f136e600640 1 --2- 172.21.15.69:0/3616196637 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f13480a3d50 0x7f13480a6210 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.887+0000 7f136e600640 1 --2- 172.21.15.69:0/3616196637 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1368105330 0x7f136819a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.887+0000 7f136e600640 1 --2- 172.21.15.69:0/3616196637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1368104140 0x7f136819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:47.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.887+0000 7f136e600640 1 -- 172.21.15.69:0/3616196637 >> 172.21.15.69:0/3616196637 conn(0x7f13680ff6b0 msgr2=0x7f1368108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:47.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.888+0000 7f136e600640 1 -- 172.21.15.69:0/3616196637 shutdown_connections 2024-04-24T09:06:47.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:47.888+0000 7f136e600640 1 -- 172.21.15.69:0/3616196637 wait complete. 2024-04-24T09:06:47.907 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:06:47.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:47 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1123063383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:48.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.110+0000 7f4df9000640 1 -- 172.21.15.69:0/3662301153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df4104260 msgr2=0x7f4df4104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:48.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.110+0000 7f4df9000640 1 --2- 172.21.15.69:0/3662301153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df4104260 0x7f4df4104680 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009900 tx=0x7f4de402a410 comp rx=0 tx=0).stop 2024-04-24T09:06:48.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.111+0000 7f4df9000640 1 -- 172.21.15.69:0/3662301153 shutdown_connections 2024-04-24T09:06:48.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.111+0000 7f4df9000640 1 --2- 172.21.15.69:0/3662301153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4df4105e10 0x7f4df410c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.111+0000 7f4df9000640 1 --2- 172.21.15.69:0/3662301153 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4df4105450 0x7f4df41058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.111+0000 7f4df9000640 1 --2- 172.21.15.69:0/3662301153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df4104260 0x7f4df4104680 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.111+0000 7f4df9000640 1 -- 172.21.15.69:0/3662301153 >> 172.21.15.69:0/3662301153 conn(0x7f4df40ff7f0 msgr2=0x7f4df4101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:48.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.111+0000 7f4df9000640 1 -- 172.21.15.69:0/3662301153 shutdown_connections 2024-04-24T09:06:48.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.111+0000 7f4df9000640 1 -- 172.21.15.69:0/3662301153 wait complete. 2024-04-24T09:06:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.112+0000 7f4df9000640 1 Processor -- start 2024-04-24T09:06:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.112+0000 7f4df9000640 1 -- start start 2024-04-24T09:06:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.112+0000 7f4df9000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4df4104260 0x7f4df419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.112+0000 7f4df9000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4df4105450 0x7f4df419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.112+0000 7f4df9000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df4105e10 0x7f4df41a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.112+0000 7f4df9000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4df410fdf0 con 0x7f4df4105e10 2024-04-24T09:06:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.112+0000 7f4df9000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4df410fc70 con 0x7f4df4104260 2024-04-24T09:06:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.112+0000 7f4df9000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4df410ff70 con 0x7f4df4105450 2024-04-24T09:06:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.113+0000 7f4df2800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df4105e10 0x7f4df41a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.113+0000 7f4df2800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df4105e10 0x7f4df41a1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60864/0 (socket says 172.21.15.69:60864) 2024-04-24T09:06:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.113+0000 7f4df2800640 1 -- 172.21.15.69:0/807771981 learned_addr learned my addr 172.21.15.69:0/807771981 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.113+0000 7f4df1400640 1 --2- 172.21.15.69:0/807771981 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4df4105450 0x7f4df419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.113+0000 7f4df1e00640 1 --2- 172.21.15.69:0/807771981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4df4104260 0x7f4df419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.113+0000 7f4df2800640 1 -- 172.21.15.69:0/807771981 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4df4105450 msgr2=0x7f4df419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.113+0000 7f4df2800640 1 --2- 172.21.15.69:0/807771981 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4df4105450 0x7f4df419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.113+0000 7f4df2800640 1 -- 172.21.15.69:0/807771981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4df4104260 msgr2=0x7f4df419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.113+0000 7f4df2800640 1 --2- 172.21.15.69:0/807771981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4df4104260 0x7f4df419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.113+0000 7f4df2800640 1 -- 172.21.15.69:0/807771981 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df41a20f0 con 0x7f4df4105e10 2024-04-24T09:06:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.113+0000 7f4df2800640 1 --2- 172.21.15.69:0/807771981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df4105e10 0x7f4df41a1a80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00eaa0 tx=0x7f4dec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.114+0000 7f4deaa00640 1 -- 172.21.15.69:0/807771981 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00cdb0 con 0x7f4df4105e10 2024-04-24T09:06:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.114+0000 7f4deaa00640 1 -- 172.21.15.69:0/807771981 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dec005020 con 0x7f4df4105e10 2024-04-24T09:06:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.114+0000 7f4df9000640 1 -- 172.21.15.69:0/807771981 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df41a23e0 con 0x7f4df4105e10 2024-04-24T09:06:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.114+0000 7f4deaa00640 1 -- 172.21.15.69:0/807771981 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec0116d0 con 0x7f4df4105e10 2024-04-24T09:06:48.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.114+0000 7f4df9000640 1 -- 172.21.15.69:0/807771981 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df41a2930 con 0x7f4df4105e10 2024-04-24T09:06:48.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.115+0000 7f4df9000640 1 -- 172.21.15.69:0/807771981 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df410b6a0 con 0x7f4df4105e10 2024-04-24T09:06:48.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.117+0000 7f4deaa00640 1 -- 172.21.15.69:0/807771981 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4dec0051d0 con 0x7f4df4105e10 2024-04-24T09:06:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.118+0000 7f4deaa00640 1 --2- 172.21.15.69:0/807771981 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4dd80a3bb0 0x7f4dd80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.119+0000 7f4deaa00640 1 -- 172.21.15.69:0/807771981 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f4dec0d0920 con 0x7f4df4105e10 2024-04-24T09:06:48.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.119+0000 7f4df1e00640 1 --2- 172.21.15.69:0/807771981 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4dd80a3bb0 0x7f4dd80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:48.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.119+0000 7f4df1e00640 1 --2- 172.21.15.69:0/807771981 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4dd80a3bb0 0x7f4dd80a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4de4006010 tx=0x7f4de40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:48.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.123+0000 7f4deaa00640 1 -- 172.21.15.69:0/807771981 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4dec016030 con 0x7f4df4105e10 2024-04-24T09:06:48.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.326+0000 7f4df9000640 1 -- 172.21.15.69:0/807771981 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4df4062c00 con 0x7f4df4105e10 2024-04-24T09:06:48.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.327+0000 7f4deaa00640 1 -- 172.21.15.69:0/807771981 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f4dec09cfa0 con 0x7f4df4105e10 2024-04-24T09:06:48.327 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:06:48.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.330+0000 7f4df9000640 1 -- 172.21.15.69:0/807771981 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4dd80a3bb0 msgr2=0x7f4dd80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:48.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.330+0000 7f4df9000640 1 --2- 172.21.15.69:0/807771981 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4dd80a3bb0 0x7f4dd80a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4de4006010 tx=0x7f4de40314a0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.330+0000 7f4df9000640 1 -- 172.21.15.69:0/807771981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df4105e10 msgr2=0x7f4df41a1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:48.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.330+0000 7f4df9000640 1 --2- 172.21.15.69:0/807771981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df4105e10 0x7f4df41a1a80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00eaa0 tx=0x7f4dec00ef60 comp rx=0 tx=0).stop 2024-04-24T09:06:48.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.330+0000 7f4df9000640 1 -- 172.21.15.69:0/807771981 shutdown_connections 2024-04-24T09:06:48.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.330+0000 7f4df9000640 1 --2- 172.21.15.69:0/807771981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df4105e10 0x7f4df41a1a80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.330+0000 7f4df9000640 1 --2- 172.21.15.69:0/807771981 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4df4105450 0x7f4df419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.330+0000 7f4df9000640 1 --2- 172.21.15.69:0/807771981 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f4dd80a3bb0 0x7f4dd80a6070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.330+0000 7f4df9000640 1 --2- 172.21.15.69:0/807771981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4df4104260 0x7f4df419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.330+0000 7f4df9000640 1 -- 172.21.15.69:0/807771981 >> 172.21.15.69:0/807771981 conn(0x7f4df40ff7f0 msgr2=0x7f4df4108870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:48.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.330+0000 7f4df9000640 1 -- 172.21.15.69:0/807771981 shutdown_connections 2024-04-24T09:06:48.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.330+0000 7f4df9000640 1 -- 172.21.15.69:0/807771981 wait complete. 2024-04-24T09:06:48.349 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:06:48.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.548+0000 7fb713000640 1 -- 172.21.15.69:0/2386407422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb70c105470 msgr2=0x7fb70c1058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:48.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.548+0000 7fb713000640 1 --2- 172.21.15.69:0/2386407422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb70c105470 0x7fb70c1058f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009900 tx=0x7fb6f802a410 comp rx=0 tx=0).stop 2024-04-24T09:06:48.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.548+0000 7fb713000640 1 -- 172.21.15.69:0/2386407422 shutdown_connections 2024-04-24T09:06:48.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.548+0000 7fb713000640 1 --2- 172.21.15.69:0/2386407422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb70c105e30 0x7fb70c10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.548+0000 7fb713000640 1 --2- 172.21.15.69:0/2386407422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb70c105470 0x7fb70c1058f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.548+0000 7fb713000640 1 --2- 172.21.15.69:0/2386407422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb70c104280 0x7fb70c1046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.548+0000 7fb713000640 1 -- 172.21.15.69:0/2386407422 >> 172.21.15.69:0/2386407422 conn(0x7fb70c0ff7f0 msgr2=0x7fb70c101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:48.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.548+0000 7fb713000640 1 -- 172.21.15.69:0/2386407422 shutdown_connections 2024-04-24T09:06:48.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.548+0000 7fb713000640 1 -- 172.21.15.69:0/2386407422 wait complete. 2024-04-24T09:06:48.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.549+0000 7fb713000640 1 Processor -- start 2024-04-24T09:06:48.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.549+0000 7fb713000640 1 -- start start 2024-04-24T09:06:48.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.549+0000 7fb713000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb70c104280 0x7fb70c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:48.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.550+0000 7fb713000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb70c105470 0x7fb70c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:48.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.550+0000 7fb713000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb70c105e30 0x7fb70c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:48.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.550+0000 7fb713000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c10fe10 con 0x7fb70c105470 2024-04-24T09:06:48.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.550+0000 7fb713000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c10fc90 con 0x7fb70c104280 2024-04-24T09:06:48.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.550+0000 7fb713000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c10ff90 con 0x7fb70c105e30 2024-04-24T09:06:48.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.550+0000 7fb70b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb70c105470 0x7fb70c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:48.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.550+0000 7fb70b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb70c105470 0x7fb70c19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60876/0 (socket says 172.21.15.69:60876) 2024-04-24T09:06:48.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.550+0000 7fb70b400640 1 -- 172.21.15.69:0/3028927160 learned_addr learned my addr 172.21.15.69:0/3028927160 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:48.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.550+0000 7fb70be00640 1 --2- 172.21.15.69:0/3028927160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb70c104280 0x7fb70c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:48.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.550+0000 7fb711000640 1 --2- 172.21.15.69:0/3028927160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb70c105e30 0x7fb70c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:48.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.550+0000 7fb70b400640 1 -- 172.21.15.69:0/3028927160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb70c105e30 msgr2=0x7fb70c1a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:48.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.550+0000 7fb70b400640 1 --2- 172.21.15.69:0/3028927160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb70c105e30 0x7fb70c1a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.550+0000 7fb70b400640 1 -- 172.21.15.69:0/3028927160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb70c104280 msgr2=0x7fb70c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:48.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.550+0000 7fb70b400640 1 --2- 172.21.15.69:0/3028927160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb70c104280 0x7fb70c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.550+0000 7fb70b400640 1 -- 172.21.15.69:0/3028927160 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c1a20c0 con 0x7fb70c105470 2024-04-24T09:06:48.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.551+0000 7fb70b400640 1 --2- 172.21.15.69:0/3028927160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb70c105470 0x7fb70c19a9e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80314e0 tx=0x7fb6f8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:48.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.551+0000 7fb708c00640 1 -- 172.21.15.69:0/3028927160 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f803f070 con 0x7fb70c105470 2024-04-24T09:06:48.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.551+0000 7fb708c00640 1 -- 172.21.15.69:0/3028927160 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6f8031e40 con 0x7fb70c105470 2024-04-24T09:06:48.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.551+0000 7fb713000640 1 -- 172.21.15.69:0/3028927160 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70c1a2350 con 0x7fb70c105470 2024-04-24T09:06:48.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.551+0000 7fb708c00640 1 -- 172.21.15.69:0/3028927160 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8035030 con 0x7fb70c105470 2024-04-24T09:06:48.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.551+0000 7fb713000640 1 -- 172.21.15.69:0/3028927160 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70c1a2890 con 0x7fb70c105470 2024-04-24T09:06:48.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.552+0000 7fb713000640 1 -- 172.21.15.69:0/3028927160 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70c10b620 con 0x7fb70c105470 2024-04-24T09:06:48.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.554+0000 7fb708c00640 1 -- 172.21.15.69:0/3028927160 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb6f8042070 con 0x7fb70c105470 2024-04-24T09:06:48.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.556+0000 7fb708c00640 1 --2- 172.21.15.69:0/3028927160 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb6f00a3c30 0x7fb6f00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:48.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.556+0000 7fb708c00640 1 -- 172.21.15.69:0/3028927160 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb6f80f1140 con 0x7fb70c105470 2024-04-24T09:06:48.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.556+0000 7fb70be00640 1 --2- 172.21.15.69:0/3028927160 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb6f00a3c30 0x7fb6f00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:48.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.557+0000 7fb70be00640 1 --2- 172.21.15.69:0/3028927160 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb6f00a3c30 0x7fb6f00a60f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0058c0 tx=0x7fb6fc00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:48.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.560+0000 7fb708c00640 1 -- 172.21.15.69:0/3028927160 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6f80bd760 con 0x7fb70c105470 2024-04-24T09:06:48.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.766+0000 7fb713000640 1 -- 172.21.15.69:0/3028927160 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb70c062c00 con 0x7fb70c105470 2024-04-24T09:06:48.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.767+0000 7fb708c00640 1 -- 172.21.15.69:0/3028927160 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7fb6f80bd060 con 0x7fb70c105470 2024-04-24T09:06:48.767 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-24T09:06:48.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.770+0000 7fb713000640 1 -- 172.21.15.69:0/3028927160 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb6f00a3c30 msgr2=0x7fb6f00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:48.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.770+0000 7fb713000640 1 --2- 172.21.15.69:0/3028927160 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb6f00a3c30 0x7fb6f00a60f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0058c0 tx=0x7fb6fc00a4c0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.770+0000 7fb713000640 1 -- 172.21.15.69:0/3028927160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb70c105470 msgr2=0x7fb70c19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:48.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.770+0000 7fb713000640 1 --2- 172.21.15.69:0/3028927160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb70c105470 0x7fb70c19a9e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80314e0 tx=0x7fb6f8003c80 comp rx=0 tx=0).stop 2024-04-24T09:06:48.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.770+0000 7fb713000640 1 -- 172.21.15.69:0/3028927160 shutdown_connections 2024-04-24T09:06:48.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.770+0000 7fb713000640 1 --2- 172.21.15.69:0/3028927160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb70c105e30 0x7fb70c1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.770+0000 7fb713000640 1 --2- 172.21.15.69:0/3028927160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb70c105470 0x7fb70c19a9e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.770+0000 7fb713000640 1 --2- 172.21.15.69:0/3028927160 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb6f00a3c30 0x7fb6f00a60f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.770+0000 7fb713000640 1 --2- 172.21.15.69:0/3028927160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb70c104280 0x7fb70c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.770+0000 7fb713000640 1 -- 172.21.15.69:0/3028927160 >> 172.21.15.69:0/3028927160 conn(0x7fb70c0ff7f0 msgr2=0x7fb70c1087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:48.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.770+0000 7fb713000640 1 -- 172.21.15.69:0/3028927160 shutdown_connections 2024-04-24T09:06:48.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.770+0000 7fb713000640 1 -- 172.21.15.69:0/3028927160 wait complete. 2024-04-24T09:06:48.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:48 smithi069 ceph-mon[28170]: pgmap v454: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:06:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:48 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3616196637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:06:48.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:48 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/807771981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:06:48.790 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T09:06:48.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:48 smithi081 ceph-mon[30961]: pgmap v454: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:06:48.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:48 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3616196637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:06:48.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:48 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/807771981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:06:48.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:48 smithi130 ceph-mon[31553]: pgmap v454: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-24T09:06:48.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:48 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3616196637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:06:48.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:48 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/807771981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:06:48.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.982+0000 7f4953e00640 1 -- 172.21.15.69:0/3116402767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4954109730 msgr2=0x7f4954109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:48.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.982+0000 7f4953e00640 1 --2- 172.21.15.69:0/3116402767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4954109730 0x7f4954109bb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f493c009900 tx=0x7f493c02a410 comp rx=0 tx=0).stop 2024-04-24T09:06:48.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.982+0000 7f4953e00640 1 -- 172.21.15.69:0/3116402767 shutdown_connections 2024-04-24T09:06:48.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.982+0000 7f4953e00640 1 --2- 172.21.15.69:0/3116402767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f495410a0f0 0x7f4954110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.982+0000 7f4953e00640 1 --2- 172.21.15.69:0/3116402767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4954109730 0x7f4954109bb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.982+0000 7f4953e00640 1 --2- 172.21.15.69:0/3116402767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4954108540 0x7f4954108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.982+0000 7f4953e00640 1 -- 172.21.15.69:0/3116402767 >> 172.21.15.69:0/3116402767 conn(0x7f49540774d0 msgr2=0x7f4954079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:48.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.982+0000 7f4953e00640 1 -- 172.21.15.69:0/3116402767 shutdown_connections 2024-04-24T09:06:48.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.982+0000 7f4953e00640 1 -- 172.21.15.69:0/3116402767 wait complete. 2024-04-24T09:06:48.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.983+0000 7f4953e00640 1 Processor -- start 2024-04-24T09:06:48.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.983+0000 7f4953e00640 1 -- start start 2024-04-24T09:06:48.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.983+0000 7f4953e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4954108540 0x7f495419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:48.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.983+0000 7f4953e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4954109730 0x7f495419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:48.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.983+0000 7f4953e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f495410a0f0 0x7f49541a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:48.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.983+0000 7f4953e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4954114130 con 0x7f4954109730 2024-04-24T09:06:48.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.983+0000 7f4953e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4954113fb0 con 0x7f4954108540 2024-04-24T09:06:48.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.983+0000 7f4953e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f49541142b0 con 0x7f495410a0f0 2024-04-24T09:06:48.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.984+0000 7f494be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4954109730 0x7f495419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:48.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.984+0000 7f494be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4954109730 0x7f495419ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60894/0 (socket says 172.21.15.69:60894) 2024-04-24T09:06:48.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.984+0000 7f494be00640 1 -- 172.21.15.69:0/1104769175 learned_addr learned my addr 172.21.15.69:0/1104769175 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:48.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.984+0000 7f4953400640 1 --2- 172.21.15.69:0/1104769175 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f495410a0f0 0x7f49541a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:48.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.984+0000 7f4952a00640 1 --2- 172.21.15.69:0/1104769175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4954108540 0x7f495419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:48.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.984+0000 7f494be00640 1 -- 172.21.15.69:0/1104769175 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f495410a0f0 msgr2=0x7f49541a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:48.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.984+0000 7f494be00640 1 --2- 172.21.15.69:0/1104769175 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f495410a0f0 0x7f49541a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.984+0000 7f494be00640 1 -- 172.21.15.69:0/1104769175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4954108540 msgr2=0x7f495419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:48.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.984+0000 7f494be00640 1 --2- 172.21.15.69:0/1104769175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4954108540 0x7f495419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:48.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.984+0000 7f494be00640 1 -- 172.21.15.69:0/1104769175 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49541a6420 con 0x7f4954109730 2024-04-24T09:06:48.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.984+0000 7f4953400640 1 --2- 172.21.15.69:0/1104769175 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f495410a0f0 0x7f49541a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:48.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.984+0000 7f4952a00640 1 --2- 172.21.15.69:0/1104769175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4954108540 0x7f495419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:48.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.984+0000 7f494be00640 1 --2- 172.21.15.69:0/1104769175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4954109730 0x7f495419ed40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f493c0314e0 tx=0x7f493c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:48.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.985+0000 7f494b400640 1 -- 172.21.15.69:0/1104769175 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c03f070 con 0x7f4954109730 2024-04-24T09:06:48.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.985+0000 7f494b400640 1 -- 172.21.15.69:0/1104769175 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f493c031e40 con 0x7f4954109730 2024-04-24T09:06:48.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.985+0000 7f494b400640 1 -- 172.21.15.69:0/1104769175 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c035030 con 0x7f4954109730 2024-04-24T09:06:48.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.985+0000 7f4953e00640 1 -- 172.21.15.69:0/1104769175 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49541a66b0 con 0x7f4954109730 2024-04-24T09:06:48.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.985+0000 7f4953e00640 1 -- 172.21.15.69:0/1104769175 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49541a6c20 con 0x7f4954109730 2024-04-24T09:06:48.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.986+0000 7f4953e00640 1 -- 172.21.15.69:0/1104769175 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495410f900 con 0x7f4954109730 2024-04-24T09:06:48.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.988+0000 7f494b400640 1 -- 172.21.15.69:0/1104769175 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f493c042070 con 0x7f4954109730 2024-04-24T09:06:48.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.989+0000 7f494b400640 1 --2- 172.21.15.69:0/1104769175 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f49380a3bb0 0x7f49380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:48.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.989+0000 7f494b400640 1 -- 172.21.15.69:0/1104769175 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f493c0f12b0 con 0x7f4954109730 2024-04-24T09:06:48.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.989+0000 7f4952a00640 1 --2- 172.21.15.69:0/1104769175 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f49380a3bb0 0x7f49380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:48.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.990+0000 7f4952a00640 1 --2- 172.21.15.69:0/1104769175 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f49380a3bb0 0x7f49380a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4944006fd0 tx=0x7f4944008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:48.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:48.993+0000 7f494b400640 1 -- 172.21.15.69:0/1104769175 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f493c0bd8d0 con 0x7f4954109730 2024-04-24T09:06:49.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.179+0000 7f4953e00640 1 -- 172.21.15.69:0/1104769175 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4954062c00 con 0x7f4954109730 2024-04-24T09:06:49.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.180+0000 7f494b400640 1 -- 172.21.15.69:0/1104769175 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f493c0bd1d0 con 0x7f4954109730 2024-04-24T09:06:49.180 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:06:49.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.182+0000 7f4953e00640 1 -- 172.21.15.69:0/1104769175 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f49380a3bb0 msgr2=0x7f49380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:49.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.182+0000 7f4953e00640 1 --2- 172.21.15.69:0/1104769175 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f49380a3bb0 0x7f49380a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4944006fd0 tx=0x7f4944008040 comp rx=0 tx=0).stop 2024-04-24T09:06:49.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.182+0000 7f4953e00640 1 -- 172.21.15.69:0/1104769175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4954109730 msgr2=0x7f495419ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:49.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.182+0000 7f4953e00640 1 --2- 172.21.15.69:0/1104769175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4954109730 0x7f495419ed40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f493c0314e0 tx=0x7f493c003d50 comp rx=0 tx=0).stop 2024-04-24T09:06:49.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.183+0000 7f4953e00640 1 -- 172.21.15.69:0/1104769175 shutdown_connections 2024-04-24T09:06:49.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.183+0000 7f4953e00640 1 --2- 172.21.15.69:0/1104769175 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f495410a0f0 0x7f49541a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.183+0000 7f4953e00640 1 --2- 172.21.15.69:0/1104769175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4954109730 0x7f495419ed40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.183+0000 7f4953e00640 1 --2- 172.21.15.69:0/1104769175 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f49380a3bb0 0x7f49380a6070 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.183+0000 7f4953e00640 1 --2- 172.21.15.69:0/1104769175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4954108540 0x7f495419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.183+0000 7f4953e00640 1 -- 172.21.15.69:0/1104769175 >> 172.21.15.69:0/1104769175 conn(0x7f49540774d0 msgr2=0x7f495410cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:49.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.183+0000 7f4953e00640 1 -- 172.21.15.69:0/1104769175 shutdown_connections 2024-04-24T09:06:49.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.183+0000 7f4953e00640 1 -- 172.21.15.69:0/1104769175 wait complete. 2024-04-24T09:06:49.201 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:06:49.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.403+0000 7fab26800640 1 -- 172.21.15.69:0/3850297972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab201010d0 msgr2=0x7fab20101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:49.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.403+0000 7fab26800640 1 --2- 172.21.15.69:0/3850297972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab201010d0 0x7fab20101530 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fab14009ca0 tx=0x7fab1402a950 comp rx=0 tx=0).stop 2024-04-24T09:06:49.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.403+0000 7fab26800640 1 -- 172.21.15.69:0/3850297972 shutdown_connections 2024-04-24T09:06:49.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.403+0000 7fab26800640 1 --2- 172.21.15.69:0/3850297972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab20101a70 0x7fab20101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.403+0000 7fab26800640 1 --2- 172.21.15.69:0/3850297972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab201010d0 0x7fab20101530 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.403+0000 7fab26800640 1 --2- 172.21.15.69:0/3850297972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab20100770 0x7fab20100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.403+0000 7fab26800640 1 -- 172.21.15.69:0/3850297972 >> 172.21.15.69:0/3850297972 conn(0x7fab200fc2d0 msgr2=0x7fab200fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:49.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.403+0000 7fab26800640 1 -- 172.21.15.69:0/3850297972 shutdown_connections 2024-04-24T09:06:49.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.403+0000 7fab26800640 1 -- 172.21.15.69:0/3850297972 wait complete. 2024-04-24T09:06:49.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.404+0000 7fab26800640 1 Processor -- start 2024-04-24T09:06:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.404+0000 7fab26800640 1 -- start start 2024-04-24T09:06:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.404+0000 7fab26800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab20100770 0x7fab2019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.404+0000 7fab26800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab201010d0 0x7fab2019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.404+0000 7fab26800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab20101a70 0x7fab201a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.405+0000 7fab26800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fab2010fbc0 con 0x7fab20100770 2024-04-24T09:06:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.405+0000 7fab26800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fab2010fa40 con 0x7fab201010d0 2024-04-24T09:06:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.405+0000 7fab26800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fab2010fd40 con 0x7fab20101a70 2024-04-24T09:06:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.405+0000 7fab25400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab20100770 0x7fab2019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.405+0000 7fab25400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab20100770 0x7fab2019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60906/0 (socket says 172.21.15.69:60906) 2024-04-24T09:06:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.405+0000 7fab25400640 1 -- 172.21.15.69:0/3559500826 learned_addr learned my addr 172.21.15.69:0/3559500826 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.405+0000 7fab25e00640 1 --2- 172.21.15.69:0/3559500826 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab20101a70 0x7fab201a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:49.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.405+0000 7fab24a00640 1 --2- 172.21.15.69:0/3559500826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab201010d0 0x7fab2019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:49.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.405+0000 7fab25400640 1 -- 172.21.15.69:0/3559500826 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab20101a70 msgr2=0x7fab201a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:49.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.405+0000 7fab25400640 1 --2- 172.21.15.69:0/3559500826 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab20101a70 0x7fab201a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.405+0000 7fab25400640 1 -- 172.21.15.69:0/3559500826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab201010d0 msgr2=0x7fab2019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:49.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.405+0000 7fab25400640 1 --2- 172.21.15.69:0/3559500826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab201010d0 0x7fab2019a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.405+0000 7fab25400640 1 -- 172.21.15.69:0/3559500826 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab201a1eb0 con 0x7fab20100770 2024-04-24T09:06:49.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.406+0000 7fab25400640 1 --2- 172.21.15.69:0/3559500826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab20100770 0x7fab2019a290 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fab1000c890 tx=0x7fab1000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:49.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.406+0000 7fab1e000640 1 -- 172.21.15.69:0/3559500826 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10004220 con 0x7fab20100770 2024-04-24T09:06:49.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.406+0000 7fab1e000640 1 -- 172.21.15.69:0/3559500826 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab100043c0 con 0x7fab20100770 2024-04-24T09:06:49.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.406+0000 7fab26800640 1 -- 172.21.15.69:0/3559500826 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab201a21a0 con 0x7fab20100770 2024-04-24T09:06:49.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.406+0000 7fab1e000640 1 -- 172.21.15.69:0/3559500826 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10004a00 con 0x7fab20100770 2024-04-24T09:06:49.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.406+0000 7fab26800640 1 -- 172.21.15.69:0/3559500826 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab201a26f0 con 0x7fab20100770 2024-04-24T09:06:49.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.407+0000 7fab26800640 1 -- 172.21.15.69:0/3559500826 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab2010b4c0 con 0x7fab20100770 2024-04-24T09:06:49.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.409+0000 7fab1e000640 1 -- 172.21.15.69:0/3559500826 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fab10004ba0 con 0x7fab20100770 2024-04-24T09:06:49.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.410+0000 7fab1e000640 1 --2- 172.21.15.69:0/3559500826 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fab080a3bb0 0x7fab080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:49.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.411+0000 7fab1e000640 1 -- 172.21.15.69:0/3559500826 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fab100d0500 con 0x7fab20100770 2024-04-24T09:06:49.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.411+0000 7fab24a00640 1 --2- 172.21.15.69:0/3559500826 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fab080a3bb0 0x7fab080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:49.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.412+0000 7fab24a00640 1 --2- 172.21.15.69:0/3559500826 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fab080a3bb0 0x7fab080a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fab14006010 tx=0x7fab140093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:49.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.414+0000 7fab1e000640 1 -- 172.21.15.69:0/3559500826 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab10014030 con 0x7fab20100770 2024-04-24T09:06:49.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.648+0000 7fab26800640 1 -- 172.21.15.69:0/3559500826 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab2002d620 con 0x7fab20100770 2024-04-24T09:06:49.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.649+0000 7fab1e000640 1 -- 172.21.15.69:0/3559500826 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11873 (secure 0 0 0) 0x7fab1009cb20 con 0x7fab20100770 2024-04-24T09:06:49.649 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:49.649 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:04:01.991559+0000","modified":"2024-04-24T09:04:34.617681+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34532,"mds_1":34496,"mds_2":34460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34496":{"gid":34496,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34532":{"gid":34532,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34532,"qdb_cluster":[34460,34496,34532]},"id":1}]} 2024-04-24T09:06:49.649 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2024-04-24T09:06:49.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.652+0000 7fab26800640 1 -- 172.21.15.69:0/3559500826 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fab080a3bb0 msgr2=0x7fab080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:49.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.652+0000 7fab26800640 1 --2- 172.21.15.69:0/3559500826 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fab080a3bb0 0x7fab080a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fab14006010 tx=0x7fab140093f0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.652+0000 7fab26800640 1 -- 172.21.15.69:0/3559500826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab20100770 msgr2=0x7fab2019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:49.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.652+0000 7fab26800640 1 --2- 172.21.15.69:0/3559500826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab20100770 0x7fab2019a290 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fab1000c890 tx=0x7fab1000cd50 comp rx=0 tx=0).stop 2024-04-24T09:06:49.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.653+0000 7fab26800640 1 -- 172.21.15.69:0/3559500826 shutdown_connections 2024-04-24T09:06:49.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.653+0000 7fab26800640 1 --2- 172.21.15.69:0/3559500826 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab20101a70 0x7fab201a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.653+0000 7fab26800640 1 --2- 172.21.15.69:0/3559500826 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fab080a3bb0 0x7fab080a6070 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.653+0000 7fab26800640 1 --2- 172.21.15.69:0/3559500826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab201010d0 0x7fab2019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.653+0000 7fab26800640 1 --2- 172.21.15.69:0/3559500826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab20100770 0x7fab2019a290 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.653+0000 7fab26800640 1 -- 172.21.15.69:0/3559500826 >> 172.21.15.69:0/3559500826 conn(0x7fab200fc2d0 msgr2=0x7fab200ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:49.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.653+0000 7fab26800640 1 -- 172.21.15.69:0/3559500826 shutdown_connections 2024-04-24T09:06:49.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.653+0000 7fab26800640 1 -- 172.21.15.69:0/3559500826 wait complete. 2024-04-24T09:06:49.670 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:06:49.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:49 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3028927160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:06:49.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:49 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1104769175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:06:49.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:49 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3028927160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:06:49.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:49 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1104769175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:06:49.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.861+0000 7f73d3e00640 1 -- 172.21.15.69:0/3266802392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4102fc0 msgr2=0x7f73d4103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:49.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.861+0000 7f73d3e00640 1 --2- 172.21.15.69:0/3266802392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4102fc0 0x7f73d4103440 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f73bc009ca0 tx=0x7f73bc02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.861+0000 7f73d3e00640 1 -- 172.21.15.69:0/3266802392 shutdown_connections 2024-04-24T09:06:49.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.861+0000 7f73d3e00640 1 --2- 172.21.15.69:0/3266802392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d4103980 0x7f73d410a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.861+0000 7f73d3e00640 1 --2- 172.21.15.69:0/3266802392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4102fc0 0x7f73d4103440 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.861+0000 7f73d3e00640 1 --2- 172.21.15.69:0/3266802392 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73d4101e70 0x7f73d4102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.861+0000 7f73d3e00640 1 -- 172.21.15.69:0/3266802392 >> 172.21.15.69:0/3266802392 conn(0x7f73d40fd3e0 msgr2=0x7f73d40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:49.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.861+0000 7f73d3e00640 1 -- 172.21.15.69:0/3266802392 shutdown_connections 2024-04-24T09:06:49.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.861+0000 7f73d3e00640 1 -- 172.21.15.69:0/3266802392 wait complete. 2024-04-24T09:06:49.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.862+0000 7f73d3e00640 1 Processor -- start 2024-04-24T09:06:49.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.862+0000 7f73d3e00640 1 -- start start 2024-04-24T09:06:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.862+0000 7f73d3e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73d4101e70 0x7f73d419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.862+0000 7f73d3e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d4102fc0 0x7f73d419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.862+0000 7f73d3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4103980 0x7f73d41a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.862+0000 7f73d3e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73d410d9c0 con 0x7f73d4103980 2024-04-24T09:06:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.862+0000 7f73d3e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f73d410d840 con 0x7f73d4102fc0 2024-04-24T09:06:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.862+0000 7f73d3e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f73d410db40 con 0x7f73d4101e70 2024-04-24T09:06:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.862+0000 7f73d3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4103980 0x7f73d41a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.862+0000 7f73d3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4103980 0x7f73d41a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60936/0 (socket says 172.21.15.69:60936) 2024-04-24T09:06:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.862+0000 7f73d3400640 1 -- 172.21.15.69:0/1477145063 learned_addr learned my addr 172.21.15.69:0/1477145063 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.863+0000 7f73d2a00640 1 --2- 172.21.15.69:0/1477145063 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73d4101e70 0x7f73d419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.863+0000 7f73d3400640 1 -- 172.21.15.69:0/1477145063 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73d4101e70 msgr2=0x7f73d419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.863+0000 7f73d3400640 1 --2- 172.21.15.69:0/1477145063 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73d4101e70 0x7f73d419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.863+0000 7f73d3400640 1 -- 172.21.15.69:0/1477145063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d4102fc0 msgr2=0x7f73d419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:49.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.863+0000 7f73d2000640 1 --2- 172.21.15.69:0/1477145063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d4102fc0 0x7f73d419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.863+0000 7f73d3400640 1 --2- 172.21.15.69:0/1477145063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d4102fc0 0x7f73d419a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.863+0000 7f73d3400640 1 -- 172.21.15.69:0/1477145063 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d41a1f20 con 0x7f73d4103980 2024-04-24T09:06:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.863+0000 7f73d3400640 1 --2- 172.21.15.69:0/1477145063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4103980 0x7f73d41a18b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00d750 tx=0x7f73cc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.863+0000 7f73cb400640 1 -- 172.21.15.69:0/1477145063 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc017070 con 0x7f73d4103980 2024-04-24T09:06:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.863+0000 7f73cb400640 1 -- 172.21.15.69:0/1477145063 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73cc0054d0 con 0x7f73d4103980 2024-04-24T09:06:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.863+0000 7f73cb400640 1 -- 172.21.15.69:0/1477145063 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc004c60 con 0x7f73d4103980 2024-04-24T09:06:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.863+0000 7f73d3e00640 1 -- 172.21.15.69:0/1477145063 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d41a2210 con 0x7f73d4103980 2024-04-24T09:06:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.863+0000 7f73d3e00640 1 -- 172.21.15.69:0/1477145063 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d41a2710 con 0x7f73d4103980 2024-04-24T09:06:49.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.864+0000 7f73d3e00640 1 -- 172.21.15.69:0/1477145063 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d41a2970 con 0x7f73d4103980 2024-04-24T09:06:49.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.866+0000 7f73cb400640 1 -- 172.21.15.69:0/1477145063 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f73cc004e00 con 0x7f73d4103980 2024-04-24T09:06:49.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.869+0000 7f73cb400640 1 --2- 172.21.15.69:0/1477145063 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f73b80a3bb0 0x7f73b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:49.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.869+0000 7f73cb400640 1 -- 172.21.15.69:0/1477145063 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f73cc0d0260 con 0x7f73d4103980 2024-04-24T09:06:49.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.869+0000 7f73d2a00640 1 --2- 172.21.15.69:0/1477145063 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f73b80a3bb0 0x7f73b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:49.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.869+0000 7f73d2a00640 1 --2- 172.21.15.69:0/1477145063 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f73b80a3bb0 0x7f73b80a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f73c4005fd0 tx=0x7f73c4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:49.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:49.873+0000 7f73cb400640 1 -- 172.21.15.69:0/1477145063 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73cc09c880 con 0x7f73d4103980 2024-04-24T09:06:49.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:49 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3028927160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:06:49.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:49 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1104769175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:06:50.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.060+0000 7f73d3e00640 1 -- 172.21.15.69:0/1477145063 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73d41a2d00 con 0x7f73d4103980 2024-04-24T09:06:50.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.064+0000 7f73cb400640 1 -- 172.21.15.69:0/1477145063 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24564 (secure 0 0 0) 0x7f73cc09c180 con 0x7f73d4103980 2024-04-24T09:06:50.065 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:50.065 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":92,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:24.497245+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:06:50.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.067+0000 7f73d3e00640 1 -- 172.21.15.69:0/1477145063 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f73b80a3bb0 msgr2=0x7f73b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:50.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.067+0000 7f73d3e00640 1 --2- 172.21.15.69:0/1477145063 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f73b80a3bb0 0x7f73b80a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f73c4005fd0 tx=0x7f73c4005850 comp rx=0 tx=0).stop 2024-04-24T09:06:50.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.067+0000 7f73d3e00640 1 -- 172.21.15.69:0/1477145063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4103980 msgr2=0x7f73d41a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:50.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.067+0000 7f73d3e00640 1 --2- 172.21.15.69:0/1477145063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4103980 0x7f73d41a18b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00d750 tx=0x7f73cc00dc10 comp rx=0 tx=0).stop 2024-04-24T09:06:50.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.067+0000 7f73d3e00640 1 -- 172.21.15.69:0/1477145063 shutdown_connections 2024-04-24T09:06:50.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.067+0000 7f73d3e00640 1 --2- 172.21.15.69:0/1477145063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4103980 0x7f73d41a18b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.067+0000 7f73d3e00640 1 --2- 172.21.15.69:0/1477145063 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f73b80a3bb0 0x7f73b80a6070 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.067+0000 7f73d3e00640 1 --2- 172.21.15.69:0/1477145063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d4102fc0 0x7f73d419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.067+0000 7f73d3e00640 1 --2- 172.21.15.69:0/1477145063 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73d4101e70 0x7f73d419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.067+0000 7f73d3e00640 1 -- 172.21.15.69:0/1477145063 >> 172.21.15.69:0/1477145063 conn(0x7f73d40fd3e0 msgr2=0x7f73d4108820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:50.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.067+0000 7f73d3e00640 1 -- 172.21.15.69:0/1477145063 shutdown_connections 2024-04-24T09:06:50.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.067+0000 7f73d3e00640 1 -- 172.21.15.69:0/1477145063 wait complete. 2024-04-24T09:06:50.086 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:57:37.377599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:04:00.299827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:04:00.958498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T09:04:04.105314+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T09:06:50.087 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:06:50.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.269+0000 7fab76c00640 1 -- 172.21.15.69:0/1466648712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab70104140 msgr2=0x7fab70104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:50.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.269+0000 7fab76c00640 1 --2- 172.21.15.69:0/1466648712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab70104140 0x7fab70104560 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fab64009900 tx=0x7fab6402a410 comp rx=0 tx=0).stop 2024-04-24T09:06:50.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.269+0000 7fab76c00640 1 -- 172.21.15.69:0/1466648712 shutdown_connections 2024-04-24T09:06:50.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.269+0000 7fab76c00640 1 --2- 172.21.15.69:0/1466648712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab70105cf0 0x7fab7010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.269+0000 7fab76c00640 1 --2- 172.21.15.69:0/1466648712 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab70105330 0x7fab701057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.269+0000 7fab76c00640 1 --2- 172.21.15.69:0/1466648712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab70104140 0x7fab70104560 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.269+0000 7fab76c00640 1 -- 172.21.15.69:0/1466648712 >> 172.21.15.69:0/1466648712 conn(0x7fab700ff6b0 msgr2=0x7fab70101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:50.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.269+0000 7fab76c00640 1 -- 172.21.15.69:0/1466648712 shutdown_connections 2024-04-24T09:06:50.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.269+0000 7fab76c00640 1 -- 172.21.15.69:0/1466648712 wait complete. 2024-04-24T09:06:50.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.270+0000 7fab76c00640 1 Processor -- start 2024-04-24T09:06:50.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.270+0000 7fab76c00640 1 -- start start 2024-04-24T09:06:50.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.271+0000 7fab76c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab70104140 0x7fab7019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.271+0000 7fab76c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab70105330 0x7fab7019a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.271+0000 7fab76c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab70105cf0 0x7fab701a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.271+0000 7fab76c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fab7010fc80 con 0x7fab70104140 2024-04-24T09:06:50.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.271+0000 7fab76c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fab7010fb00 con 0x7fab70105330 2024-04-24T09:06:50.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.271+0000 7fab76c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fab7010fe00 con 0x7fab70105cf0 2024-04-24T09:06:50.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.271+0000 7fab75800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab70104140 0x7fab7019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:50.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.271+0000 7fab75800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab70104140 0x7fab7019a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60956/0 (socket says 172.21.15.69:60956) 2024-04-24T09:06:50.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.271+0000 7fab75800640 1 -- 172.21.15.69:0/2670633572 learned_addr learned my addr 172.21.15.69:0/2670633572 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:50.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.271+0000 7fab74e00640 1 --2- 172.21.15.69:0/2670633572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab70105330 0x7fab7019a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:50.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.272+0000 7fab75800640 1 -- 172.21.15.69:0/2670633572 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab70105cf0 msgr2=0x7fab701a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:50.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.272+0000 7fab75800640 1 --2- 172.21.15.69:0/2670633572 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab70105cf0 0x7fab701a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.272+0000 7fab75800640 1 -- 172.21.15.69:0/2670633572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab70105330 msgr2=0x7fab7019a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:50.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.272+0000 7fab75800640 1 --2- 172.21.15.69:0/2670633572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab70105330 0x7fab7019a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.272+0000 7fab75800640 1 -- 172.21.15.69:0/2670633572 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab701a1f60 con 0x7fab70104140 2024-04-24T09:06:50.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.272+0000 7fab74e00640 1 --2- 172.21.15.69:0/2670633572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab70105330 0x7fab7019a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:50.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.272+0000 7fab75800640 1 --2- 172.21.15.69:0/2670633572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab70104140 0x7fab7019a340 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fab640314e0 tx=0x7fab640042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:50.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.272+0000 7fab6e000640 1 -- 172.21.15.69:0/2670633572 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6403f030 con 0x7fab70104140 2024-04-24T09:06:50.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.272+0000 7fab6e000640 1 -- 172.21.15.69:0/2670633572 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab64003cb0 con 0x7fab70104140 2024-04-24T09:06:50.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.272+0000 7fab6e000640 1 -- 172.21.15.69:0/2670633572 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64035070 con 0x7fab70104140 2024-04-24T09:06:50.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.272+0000 7fab76c00640 1 -- 172.21.15.69:0/2670633572 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab701a21f0 con 0x7fab70104140 2024-04-24T09:06:50.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.272+0000 7fab76c00640 1 -- 172.21.15.69:0/2670633572 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab701a2690 con 0x7fab70104140 2024-04-24T09:06:50.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.273+0000 7fab76c00640 1 -- 172.21.15.69:0/2670633572 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab7010b4e0 con 0x7fab70104140 2024-04-24T09:06:50.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.277+0000 7fab6e000640 1 -- 172.21.15.69:0/2670633572 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fab64042030 con 0x7fab70104140 2024-04-24T09:06:50.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.279+0000 7fab6e000640 1 --2- 172.21.15.69:0/2670633572 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fab580a3bb0 0x7fab580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.279+0000 7fab6e000640 1 -- 172.21.15.69:0/2670633572 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fab640f12d0 con 0x7fab70104140 2024-04-24T09:06:50.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.279+0000 7fab74e00640 1 --2- 172.21.15.69:0/2670633572 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fab580a3bb0 0x7fab580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:50.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.280+0000 7fab74e00640 1 --2- 172.21.15.69:0/2670633572 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fab580a3bb0 0x7fab580a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fab7019b850 tx=0x7fab5c005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:50.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.282+0000 7fab6e000640 1 -- 172.21.15.69:0/2670633572 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab640bd8f0 con 0x7fab70104140 2024-04-24T09:06:50.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.482+0000 7fab76c00640 1 -- 172.21.15.69:0/2670633572 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fab70062c00 con 0x7fab70104140 2024-04-24T09:06:50.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.483+0000 7fab6e000640 1 -- 172.21.15.69:0/2670633572 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fab640bd1f0 con 0x7fab70104140 2024-04-24T09:06:50.483 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-24T09:06:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.486+0000 7fab76c00640 1 -- 172.21.15.69:0/2670633572 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fab580a3bb0 msgr2=0x7fab580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.486+0000 7fab76c00640 1 --2- 172.21.15.69:0/2670633572 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fab580a3bb0 0x7fab580a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fab7019b850 tx=0x7fab5c005d30 comp rx=0 tx=0).stop 2024-04-24T09:06:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.486+0000 7fab76c00640 1 -- 172.21.15.69:0/2670633572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab70104140 msgr2=0x7fab7019a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.486+0000 7fab76c00640 1 --2- 172.21.15.69:0/2670633572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab70104140 0x7fab7019a340 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fab640314e0 tx=0x7fab640042e0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.486+0000 7fab76c00640 1 -- 172.21.15.69:0/2670633572 shutdown_connections 2024-04-24T09:06:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.486+0000 7fab76c00640 1 --2- 172.21.15.69:0/2670633572 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab70105cf0 0x7fab701a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.486+0000 7fab76c00640 1 --2- 172.21.15.69:0/2670633572 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fab580a3bb0 0x7fab580a6070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.486+0000 7fab76c00640 1 --2- 172.21.15.69:0/2670633572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab70105330 0x7fab7019a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.486+0000 7fab76c00640 1 --2- 172.21.15.69:0/2670633572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab70104140 0x7fab7019a340 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.486+0000 7fab76c00640 1 -- 172.21.15.69:0/2670633572 >> 172.21.15.69:0/2670633572 conn(0x7fab700ff6b0 msgr2=0x7fab701086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.486+0000 7fab76c00640 1 -- 172.21.15.69:0/2670633572 shutdown_connections 2024-04-24T09:06:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.486+0000 7fab76c00640 1 -- 172.21.15.69:0/2670633572 wait complete. 2024-04-24T09:06:50.505 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:06:50.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.709+0000 7f6dcca00640 1 -- 172.21.15.69:0/523857565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc8105e30 msgr2=0x7f6dc810c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:50.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.709+0000 7f6dcca00640 1 --2- 172.21.15.69:0/523857565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc8105e30 0x7f6dc810c6c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009ca0 tx=0x7f6dc002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.710+0000 7f6dcca00640 1 -- 172.21.15.69:0/523857565 shutdown_connections 2024-04-24T09:06:50.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.710+0000 7f6dcca00640 1 --2- 172.21.15.69:0/523857565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc8105e30 0x7f6dc810c6c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.710+0000 7f6dcca00640 1 --2- 172.21.15.69:0/523857565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dc8105470 0x7f6dc81058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.710+0000 7f6dcca00640 1 --2- 172.21.15.69:0/523857565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6dc8104280 0x7f6dc81046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.710+0000 7f6dcca00640 1 -- 172.21.15.69:0/523857565 >> 172.21.15.69:0/523857565 conn(0x7f6dc80ff7f0 msgr2=0x7f6dc8101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:50.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.710+0000 7f6dcca00640 1 -- 172.21.15.69:0/523857565 shutdown_connections 2024-04-24T09:06:50.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.710+0000 7f6dcca00640 1 -- 172.21.15.69:0/523857565 wait complete. 2024-04-24T09:06:50.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.711+0000 7f6dcca00640 1 Processor -- start 2024-04-24T09:06:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.711+0000 7f6dcca00640 1 -- start start 2024-04-24T09:06:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.711+0000 7f6dcca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc8104280 0x7f6dc819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.711+0000 7f6dcca00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dc8105470 0x7f6dc819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.711+0000 7f6dcca00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6dc8105e30 0x7f6dc81a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.711+0000 7f6dcca00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc810fe10 con 0x7f6dc8104280 2024-04-24T09:06:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.711+0000 7f6dcca00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc810fc90 con 0x7f6dc8105470 2024-04-24T09:06:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.711+0000 7f6dcca00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc810ff90 con 0x7f6dc8105e30 2024-04-24T09:06:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.711+0000 7f6dc5e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc8104280 0x7f6dc819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.711+0000 7f6dc5e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc8104280 0x7f6dc819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60970/0 (socket says 172.21.15.69:60970) 2024-04-24T09:06:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.711+0000 7f6dc5e00640 1 -- 172.21.15.69:0/780097861 learned_addr learned my addr 172.21.15.69:0/780097861 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.712+0000 7f6dc5400640 1 --2- 172.21.15.69:0/780097861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dc8105470 0x7f6dc819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.712+0000 7f6dc6800640 1 --2- 172.21.15.69:0/780097861 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6dc8105e30 0x7f6dc81a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.712+0000 7f6dc5e00640 1 -- 172.21.15.69:0/780097861 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6dc8105e30 msgr2=0x7f6dc81a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.712+0000 7f6dc5e00640 1 --2- 172.21.15.69:0/780097861 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6dc8105e30 0x7f6dc81a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.712+0000 7f6dc5e00640 1 -- 172.21.15.69:0/780097861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dc8105470 msgr2=0x7f6dc819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.712+0000 7f6dc5e00640 1 --2- 172.21.15.69:0/780097861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dc8105470 0x7f6dc819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.712+0000 7f6dc5e00640 1 -- 172.21.15.69:0/780097861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc81a1ff0 con 0x7f6dc8104280 2024-04-24T09:06:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.712+0000 7f6dc6800640 1 --2- 172.21.15.69:0/780097861 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6dc8105e30 0x7f6dc81a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.712+0000 7f6dc5400640 1 --2- 172.21.15.69:0/780097861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dc8105470 0x7f6dc819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:50.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.712+0000 7f6dc5e00640 1 --2- 172.21.15.69:0/780097861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc8104280 0x7f6dc819a3d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6db800c970 tx=0x7f6db800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:50.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.712+0000 7f6dbea00640 1 -- 172.21.15.69:0/780097861 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8004220 con 0x7f6dc8104280 2024-04-24T09:06:50.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.713+0000 7f6dbea00640 1 -- 172.21.15.69:0/780097861 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6db80043c0 con 0x7f6dc8104280 2024-04-24T09:06:50.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.713+0000 7f6dbea00640 1 -- 172.21.15.69:0/780097861 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8004a00 con 0x7f6dc8104280 2024-04-24T09:06:50.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.713+0000 7f6dcca00640 1 -- 172.21.15.69:0/780097861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc81a22e0 con 0x7f6dc8104280 2024-04-24T09:06:50.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.713+0000 7f6dcca00640 1 -- 172.21.15.69:0/780097861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc81a2830 con 0x7f6dc8104280 2024-04-24T09:06:50.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.714+0000 7f6dcca00640 1 -- 172.21.15.69:0/780097861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc810b4d0 con 0x7f6dc8104280 2024-04-24T09:06:50.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.716+0000 7f6dbea00640 1 -- 172.21.15.69:0/780097861 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6db80079b0 con 0x7f6dc8104280 2024-04-24T09:06:50.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.718+0000 7f6dbea00640 1 --2- 172.21.15.69:0/780097861 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6dac0a3c30 0x7f6dac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.718+0000 7f6dbea00640 1 -- 172.21.15.69:0/780097861 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6db80d0580 con 0x7f6dc8104280 2024-04-24T09:06:50.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.718+0000 7f6dc5400640 1 --2- 172.21.15.69:0/780097861 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6dac0a3c30 0x7f6dac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:50.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.719+0000 7f6dc5400640 1 --2- 172.21.15.69:0/780097861 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6dac0a3c30 0x7f6dac0a60f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6dc819b8e0 tx=0x7f6db4005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:50.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.722+0000 7f6dbea00640 1 -- 172.21.15.69:0/780097861 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6db8014030 con 0x7f6dc8104280 2024-04-24T09:06:50.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:50 smithi081 ceph-mon[30961]: pgmap v455: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:50.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:50 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3559500826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:50.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:50 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1477145063' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:50.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:50 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2670633572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:06:50.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:50 smithi069 ceph-mon[28170]: pgmap v455: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:50.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:50 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3559500826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:50.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:50 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1477145063' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:50.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:50 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2670633572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:06:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.911+0000 7f6dcca00640 1 -- 172.21.15.69:0/780097861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6dc8062c30 con 0x7f6dc8104280 2024-04-24T09:06:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.912+0000 7f6dbea00640 1 -- 172.21.15.69:0/780097861 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f6db809cba0 con 0x7f6dc8104280 2024-04-24T09:06:50.912 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:06:50.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.915+0000 7f6dcca00640 1 -- 172.21.15.69:0/780097861 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6dac0a3c30 msgr2=0x7f6dac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:50.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.915+0000 7f6dcca00640 1 --2- 172.21.15.69:0/780097861 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6dac0a3c30 0x7f6dac0a60f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6dc819b8e0 tx=0x7f6db4005d90 comp rx=0 tx=0).stop 2024-04-24T09:06:50.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.915+0000 7f6dcca00640 1 -- 172.21.15.69:0/780097861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc8104280 msgr2=0x7f6dc819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:50.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.915+0000 7f6dcca00640 1 --2- 172.21.15.69:0/780097861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc8104280 0x7f6dc819a3d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6db800c970 tx=0x7f6db800ce30 comp rx=0 tx=0).stop 2024-04-24T09:06:50.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.915+0000 7f6dcca00640 1 -- 172.21.15.69:0/780097861 shutdown_connections 2024-04-24T09:06:50.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.915+0000 7f6dcca00640 1 --2- 172.21.15.69:0/780097861 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6dc8105e30 0x7f6dc81a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.915+0000 7f6dcca00640 1 --2- 172.21.15.69:0/780097861 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f6dac0a3c30 0x7f6dac0a60f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.915+0000 7f6dcca00640 1 --2- 172.21.15.69:0/780097861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dc8105470 0x7f6dc819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.915+0000 7f6dcca00640 1 --2- 172.21.15.69:0/780097861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6dc8104280 0x7f6dc819a3d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.915+0000 7f6dcca00640 1 -- 172.21.15.69:0/780097861 >> 172.21.15.69:0/780097861 conn(0x7f6dc80ff7f0 msgr2=0x7f6dc81086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:50.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.916+0000 7f6dcca00640 1 -- 172.21.15.69:0/780097861 shutdown_connections 2024-04-24T09:06:50.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:50.916+0000 7f6dcca00640 1 -- 172.21.15.69:0/780097861 wait complete. 2024-04-24T09:06:50.934 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:06:50.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:50 smithi130 ceph-mon[31553]: pgmap v455: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T09:06:50.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:50 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3559500826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:50.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:50 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1477145063' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:50.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:50 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2670633572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:06:51.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.136+0000 7f9bf4200640 1 -- 172.21.15.69:0/734603373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bec104260 msgr2=0x7f9bec104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:51.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.136+0000 7f9bf4200640 1 --2- 172.21.15.69:0/734603373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bec104260 0x7f9bec104680 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009900 tx=0x7f9bdc02a410 comp rx=0 tx=0).stop 2024-04-24T09:06:51.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.137+0000 7f9bf4200640 1 -- 172.21.15.69:0/734603373 shutdown_connections 2024-04-24T09:06:51.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.137+0000 7f9bf4200640 1 --2- 172.21.15.69:0/734603373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bec105e10 0x7f9bec10c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.137+0000 7f9bf4200640 1 --2- 172.21.15.69:0/734603373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9bec105450 0x7f9bec1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.137+0000 7f9bf4200640 1 --2- 172.21.15.69:0/734603373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bec104260 0x7f9bec104680 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.137+0000 7f9bf4200640 1 -- 172.21.15.69:0/734603373 >> 172.21.15.69:0/734603373 conn(0x7f9bec0ff7f0 msgr2=0x7f9bec101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:51.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.137+0000 7f9bf4200640 1 -- 172.21.15.69:0/734603373 shutdown_connections 2024-04-24T09:06:51.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.137+0000 7f9bf4200640 1 -- 172.21.15.69:0/734603373 wait complete. 2024-04-24T09:06:51.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.138+0000 7f9bf4200640 1 Processor -- start 2024-04-24T09:06:51.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.138+0000 7f9bf4200640 1 -- start start 2024-04-24T09:06:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.138+0000 7f9bf4200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9bec104260 0x7f9bec19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.138+0000 7f9bf4200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bec105450 0x7f9bec19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.138+0000 7f9bf4200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bec105e10 0x7f9bec1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.138+0000 7f9bf4200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec10fe00 con 0x7f9bec105e10 2024-04-24T09:06:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.138+0000 7f9bf4200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec10fc80 con 0x7f9bec105450 2024-04-24T09:06:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.138+0000 7f9bf4200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec10ff80 con 0x7f9bec104260 2024-04-24T09:06:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.138+0000 7f9bf2200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bec105e10 0x7f9bec1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.139+0000 7f9bf2200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bec105e10 0x7f9bec1a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60996/0 (socket says 172.21.15.69:60996) 2024-04-24T09:06:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.139+0000 7f9bf2200640 1 -- 172.21.15.69:0/3405088411 learned_addr learned my addr 172.21.15.69:0/3405088411 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.139+0000 7f9bf1800640 1 --2- 172.21.15.69:0/3405088411 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9bec104260 0x7f9bec19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.139+0000 7f9bf0e00640 1 --2- 172.21.15.69:0/3405088411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bec105450 0x7f9bec19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.139+0000 7f9bf2200640 1 -- 172.21.15.69:0/3405088411 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9bec104260 msgr2=0x7f9bec19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.139+0000 7f9bf2200640 1 --2- 172.21.15.69:0/3405088411 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9bec104260 0x7f9bec19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.139+0000 7f9bf2200640 1 -- 172.21.15.69:0/3405088411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bec105450 msgr2=0x7f9bec19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:51.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.139+0000 7f9bf2200640 1 --2- 172.21.15.69:0/3405088411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bec105450 0x7f9bec19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.139+0000 7f9bf2200640 1 -- 172.21.15.69:0/3405088411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bec1a2160 con 0x7f9bec105e10 2024-04-24T09:06:51.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.139+0000 7f9bf2200640 1 --2- 172.21.15.69:0/3405088411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bec105e10 0x7f9bec1a1af0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9be400b970 tx=0x7f9be400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:51.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.140+0000 7f9bea000640 1 -- 172.21.15.69:0/3405088411 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4019070 con 0x7f9bec105e10 2024-04-24T09:06:51.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.140+0000 7f9bea000640 1 -- 172.21.15.69:0/3405088411 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9be4014440 con 0x7f9bec105e10 2024-04-24T09:06:51.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.140+0000 7f9bf4200640 1 -- 172.21.15.69:0/3405088411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bec1a2450 con 0x7f9bec105e10 2024-04-24T09:06:51.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.140+0000 7f9bea000640 1 -- 172.21.15.69:0/3405088411 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be40075d0 con 0x7f9bec105e10 2024-04-24T09:06:51.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.140+0000 7f9bf4200640 1 -- 172.21.15.69:0/3405088411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bec1a29d0 con 0x7f9bec105e10 2024-04-24T09:06:51.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.141+0000 7f9bf4200640 1 -- 172.21.15.69:0/3405088411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bec10b600 con 0x7f9bec105e10 2024-04-24T09:06:51.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.143+0000 7f9bea000640 1 -- 172.21.15.69:0/3405088411 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9be401c070 con 0x7f9bec105e10 2024-04-24T09:06:51.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.144+0000 7f9bea000640 1 --2- 172.21.15.69:0/3405088411 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9bd40a3bb0 0x7f9bd40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:51.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.145+0000 7f9bea000640 1 -- 172.21.15.69:0/3405088411 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9be40cfdf0 con 0x7f9bec105e10 2024-04-24T09:06:51.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.145+0000 7f9bf1800640 1 --2- 172.21.15.69:0/3405088411 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9bd40a3bb0 0x7f9bd40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.145+0000 7f9bf1800640 1 --2- 172.21.15.69:0/3405088411 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9bd40a3bb0 0x7f9bd40a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc000c00 tx=0x7f9bdc0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:51.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.149+0000 7f9bea000640 1 -- 172.21.15.69:0/3405088411 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9be409c410 con 0x7f9bec105e10 2024-04-24T09:06:51.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.347+0000 7f9bf4200640 1 -- 172.21.15.69:0/3405088411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9bec062c00 con 0x7f9bec105e10 2024-04-24T09:06:51.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.348+0000 7f9bea000640 1 -- 172.21.15.69:0/3405088411 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f9be409bd10 con 0x7f9bec105e10 2024-04-24T09:06:51.348 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-24T09:06:51.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.350+0000 7f9bf4200640 1 -- 172.21.15.69:0/3405088411 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9bd40a3bb0 msgr2=0x7f9bd40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:51.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.350+0000 7f9bf4200640 1 --2- 172.21.15.69:0/3405088411 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9bd40a3bb0 0x7f9bd40a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc000c00 tx=0x7f9bdc0314a0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.350+0000 7f9bf4200640 1 -- 172.21.15.69:0/3405088411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bec105e10 msgr2=0x7f9bec1a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:51.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.350+0000 7f9bf4200640 1 --2- 172.21.15.69:0/3405088411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bec105e10 0x7f9bec1a1af0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9be400b970 tx=0x7f9be400be30 comp rx=0 tx=0).stop 2024-04-24T09:06:51.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.351+0000 7f9bf4200640 1 -- 172.21.15.69:0/3405088411 shutdown_connections 2024-04-24T09:06:51.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.351+0000 7f9bf4200640 1 --2- 172.21.15.69:0/3405088411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9bec105e10 0x7f9bec1a1af0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.351+0000 7f9bf4200640 1 --2- 172.21.15.69:0/3405088411 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f9bd40a3bb0 0x7f9bd40a6070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.351+0000 7f9bf4200640 1 --2- 172.21.15.69:0/3405088411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bec105450 0x7f9bec19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.351+0000 7f9bf4200640 1 --2- 172.21.15.69:0/3405088411 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9bec104260 0x7f9bec19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.351+0000 7f9bf4200640 1 -- 172.21.15.69:0/3405088411 >> 172.21.15.69:0/3405088411 conn(0x7f9bec0ff7f0 msgr2=0x7f9bec1087d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:51.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.351+0000 7f9bf4200640 1 -- 172.21.15.69:0/3405088411 shutdown_connections 2024-04-24T09:06:51.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.351+0000 7f9bf4200640 1 -- 172.21.15.69:0/3405088411 wait complete. 2024-04-24T09:06:51.370 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T09:06:51.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.578+0000 7fa617400640 1 -- 172.21.15.69:0/4284036767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa610105450 msgr2=0x7fa6101058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:51.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.578+0000 7fa617400640 1 --2- 172.21.15.69:0/4284036767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa610105450 0x7fa6101058d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009900 tx=0x7fa5fc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.578+0000 7fa617400640 1 -- 172.21.15.69:0/4284036767 shutdown_connections 2024-04-24T09:06:51.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.578+0000 7fa617400640 1 --2- 172.21.15.69:0/4284036767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa610105e10 0x7fa61010c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.578+0000 7fa617400640 1 --2- 172.21.15.69:0/4284036767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa610105450 0x7fa6101058d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.578+0000 7fa617400640 1 --2- 172.21.15.69:0/4284036767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa610104260 0x7fa610104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.578+0000 7fa617400640 1 -- 172.21.15.69:0/4284036767 >> 172.21.15.69:0/4284036767 conn(0x7fa6100ff7f0 msgr2=0x7fa610101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:51.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.578+0000 7fa617400640 1 -- 172.21.15.69:0/4284036767 shutdown_connections 2024-04-24T09:06:51.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.578+0000 7fa617400640 1 -- 172.21.15.69:0/4284036767 wait complete. 2024-04-24T09:06:51.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.579+0000 7fa617400640 1 Processor -- start 2024-04-24T09:06:51.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.579+0000 7fa617400640 1 -- start start 2024-04-24T09:06:51.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.580+0000 7fa617400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa610104260 0x7fa61019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:51.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.580+0000 7fa617400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa610105450 0x7fa61019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:51.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.580+0000 7fa617400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa610105e10 0x7fa6101a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:51.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.580+0000 7fa617400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa61010fda0 con 0x7fa610104260 2024-04-24T09:06:51.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.580+0000 7fa617400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa61010fc20 con 0x7fa610105e10 2024-04-24T09:06:51.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.580+0000 7fa617400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa61010ff20 con 0x7fa610105450 2024-04-24T09:06:51.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.580+0000 7fa614a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa610104260 0x7fa61019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.580+0000 7fa614a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa610104260 0x7fa61019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32786/0 (socket says 172.21.15.69:32786) 2024-04-24T09:06:51.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.580+0000 7fa614a00640 1 -- 172.21.15.69:0/3295782530 learned_addr learned my addr 172.21.15.69:0/3295782530 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:51.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.580+0000 7fa614a00640 1 -- 172.21.15.69:0/3295782530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa610105450 msgr2=0x7fa61019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:51.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.580+0000 7fa614a00640 1 --2- 172.21.15.69:0/3295782530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa610105450 0x7fa61019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.580+0000 7fa614a00640 1 -- 172.21.15.69:0/3295782530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa610105e10 msgr2=0x7fa6101a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:51.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.580+0000 7fa615400640 1 --2- 172.21.15.69:0/3295782530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa610105e10 0x7fa6101a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.580+0000 7fa614a00640 1 --2- 172.21.15.69:0/3295782530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa610105e10 0x7fa6101a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.580+0000 7fa614a00640 1 -- 172.21.15.69:0/3295782530 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6101a2050 con 0x7fa610104260 2024-04-24T09:06:51.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.581+0000 7fa614a00640 1 --2- 172.21.15.69:0/3295782530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa610104260 0x7fa61019a430 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa60400d770 tx=0x7fa60400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:51.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.581+0000 7fa60d600640 1 -- 172.21.15.69:0/3295782530 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604017070 con 0x7fa610104260 2024-04-24T09:06:51.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.581+0000 7fa60d600640 1 -- 172.21.15.69:0/3295782530 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa604009c30 con 0x7fa610104260 2024-04-24T09:06:51.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.581+0000 7fa617400640 1 -- 172.21.15.69:0/3295782530 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6101a2340 con 0x7fa610104260 2024-04-24T09:06:51.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.581+0000 7fa60d600640 1 -- 172.21.15.69:0/3295782530 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604003d00 con 0x7fa610104260 2024-04-24T09:06:51.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.581+0000 7fa617400640 1 -- 172.21.15.69:0/3295782530 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6101a2890 con 0x7fa610104260 2024-04-24T09:06:51.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.582+0000 7fa617400640 1 -- 172.21.15.69:0/3295782530 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa61010b5a0 con 0x7fa610104260 2024-04-24T09:06:51.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.584+0000 7fa60d600640 1 -- 172.21.15.69:0/3295782530 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa604003ea0 con 0x7fa610104260 2024-04-24T09:06:51.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.586+0000 7fa60d600640 1 --2- 172.21.15.69:0/3295782530 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa5f80a3bb0 0x7fa5f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:51.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.586+0000 7fa60d600640 1 -- 172.21.15.69:0/3295782530 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa6040d0580 con 0x7fa610104260 2024-04-24T09:06:51.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.586+0000 7fa60fe00640 1 --2- 172.21.15.69:0/3295782530 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa5f80a3bb0 0x7fa5f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.587+0000 7fa60fe00640 1 --2- 172.21.15.69:0/3295782530 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa5f80a3bb0 0x7fa5f80a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc02a910 tx=0x7fa5fc0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:51.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.590+0000 7fa60d600640 1 -- 172.21.15.69:0/3295782530 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa60409cba0 con 0x7fa610104260 2024-04-24T09:06:51.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.794+0000 7fa617400640 1 -- 172.21.15.69:0/3295782530 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa610062c30 con 0x7fa610104260 2024-04-24T09:06:51.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.795+0000 7fa60d600640 1 -- 172.21.15.69:0/3295782530 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7fa604009de0 con 0x7fa610104260 2024-04-24T09:06:51.795 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:06:51.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.797+0000 7fa617400640 1 -- 172.21.15.69:0/3295782530 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa5f80a3bb0 msgr2=0x7fa5f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:51.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.797+0000 7fa617400640 1 --2- 172.21.15.69:0/3295782530 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa5f80a3bb0 0x7fa5f80a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc02a910 tx=0x7fa5fc0314a0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.797+0000 7fa617400640 1 -- 172.21.15.69:0/3295782530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa610104260 msgr2=0x7fa61019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.797+0000 7fa617400640 1 --2- 172.21.15.69:0/3295782530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa610104260 0x7fa61019a430 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa60400d770 tx=0x7fa60400dc30 comp rx=0 tx=0).stop 2024-04-24T09:06:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.797+0000 7fa617400640 1 -- 172.21.15.69:0/3295782530 shutdown_connections 2024-04-24T09:06:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.797+0000 7fa617400640 1 --2- 172.21.15.69:0/3295782530 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa5f80a3bb0 0x7fa5f80a6070 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.797+0000 7fa617400640 1 --2- 172.21.15.69:0/3295782530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa610105e10 0x7fa6101a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.797+0000 7fa617400640 1 --2- 172.21.15.69:0/3295782530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa610105450 0x7fa61019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.797+0000 7fa617400640 1 --2- 172.21.15.69:0/3295782530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa610104260 0x7fa61019a430 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.798+0000 7fa617400640 1 -- 172.21.15.69:0/3295782530 >> 172.21.15.69:0/3295782530 conn(0x7fa6100ff7f0 msgr2=0x7fa610108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.798+0000 7fa617400640 1 -- 172.21.15.69:0/3295782530 shutdown_connections 2024-04-24T09:06:51.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:51.798+0000 7fa617400640 1 -- 172.21.15.69:0/3295782530 wait complete. 2024-04-24T09:06:51.814 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:51 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/780097861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:06:51.814 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:51 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3405088411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:06:51.816 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:06:51.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:51 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/780097861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:06:51.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:51 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3405088411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:06:51.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:51 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/780097861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:06:51.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:51 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3405088411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:06:52.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.017+0000 7fa6eb200640 1 -- 172.21.15.69:0/255785549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ec102fc0 msgr2=0x7fa6ec103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.017+0000 7fa6eb200640 1 --2- 172.21.15.69:0/255785549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ec102fc0 0x7fa6ec103440 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009900 tx=0x7fa6d802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.017+0000 7fa6eb200640 1 -- 172.21.15.69:0/255785549 shutdown_connections 2024-04-24T09:06:52.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.017+0000 7fa6eb200640 1 --2- 172.21.15.69:0/255785549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6ec103980 0x7fa6ec10a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.017+0000 7fa6eb200640 1 --2- 172.21.15.69:0/255785549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ec102fc0 0x7fa6ec103440 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.017+0000 7fa6eb200640 1 --2- 172.21.15.69:0/255785549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6ec101e70 0x7fa6ec102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.017+0000 7fa6eb200640 1 -- 172.21.15.69:0/255785549 >> 172.21.15.69:0/255785549 conn(0x7fa6ec0fd3e0 msgr2=0x7fa6ec0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:52.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.017+0000 7fa6eb200640 1 -- 172.21.15.69:0/255785549 shutdown_connections 2024-04-24T09:06:52.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.017+0000 7fa6eb200640 1 -- 172.21.15.69:0/255785549 wait complete. 2024-04-24T09:06:52.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.018+0000 7fa6eb200640 1 Processor -- start 2024-04-24T09:06:52.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.018+0000 7fa6eb200640 1 -- start start 2024-04-24T09:06:52.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.018+0000 7fa6eb200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6ec101e70 0x7fa6ec19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.018+0000 7fa6eb200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6ec102fc0 0x7fa6ec19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.018+0000 7fa6eb200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ec103980 0x7fa6ec1a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.018+0000 7fa6eb200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec10d9a0 con 0x7fa6ec103980 2024-04-24T09:06:52.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.018+0000 7fa6eb200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec10d820 con 0x7fa6ec101e70 2024-04-24T09:06:52.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.018+0000 7fa6eb200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec10db20 con 0x7fa6ec102fc0 2024-04-24T09:06:52.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.019+0000 7fa6ea800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ec103980 0x7fa6ec1a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.019+0000 7fa6ea800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ec103980 0x7fa6ec1a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32794/0 (socket says 172.21.15.69:32794) 2024-04-24T09:06:52.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.019+0000 7fa6ea800640 1 -- 172.21.15.69:0/3931824862 learned_addr learned my addr 172.21.15.69:0/3931824862 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:52.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.019+0000 7fa6e9e00640 1 --2- 172.21.15.69:0/3931824862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6ec101e70 0x7fa6ec19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.019+0000 7fa6e9400640 1 --2- 172.21.15.69:0/3931824862 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6ec102fc0 0x7fa6ec19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.019+0000 7fa6ea800640 1 -- 172.21.15.69:0/3931824862 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6ec102fc0 msgr2=0x7fa6ec19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.019+0000 7fa6ea800640 1 --2- 172.21.15.69:0/3931824862 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6ec102fc0 0x7fa6ec19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.019+0000 7fa6ea800640 1 -- 172.21.15.69:0/3931824862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6ec101e70 msgr2=0x7fa6ec19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.019+0000 7fa6ea800640 1 --2- 172.21.15.69:0/3931824862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6ec101e70 0x7fa6ec19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.019+0000 7fa6ea800640 1 -- 172.21.15.69:0/3931824862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec1a1ea0 con 0x7fa6ec103980 2024-04-24T09:06:52.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.020+0000 7fa6ea800640 1 --2- 172.21.15.69:0/3931824862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ec103980 0x7fa6ec1a1830 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400cce0 tx=0x7fa6e400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:52.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.020+0000 7fa6e2a00640 1 -- 172.21.15.69:0/3931824862 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400f450 con 0x7fa6ec103980 2024-04-24T09:06:52.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.020+0000 7fa6e2a00640 1 -- 172.21.15.69:0/3931824862 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6e400faa0 con 0x7fa6ec103980 2024-04-24T09:06:52.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.020+0000 7fa6e2a00640 1 -- 172.21.15.69:0/3931824862 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4018b70 con 0x7fa6ec103980 2024-04-24T09:06:52.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.020+0000 7fa6eb200640 1 -- 172.21.15.69:0/3931824862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ec1a2190 con 0x7fa6ec103980 2024-04-24T09:06:52.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.020+0000 7fa6eb200640 1 -- 172.21.15.69:0/3931824862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ec1a26e0 con 0x7fa6ec103980 2024-04-24T09:06:52.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.021+0000 7fa6eb200640 1 -- 172.21.15.69:0/3931824862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ec1a29a0 con 0x7fa6ec103980 2024-04-24T09:06:52.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.023+0000 7fa6e2a00640 1 -- 172.21.15.69:0/3931824862 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa6e4018d10 con 0x7fa6ec103980 2024-04-24T09:06:52.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.025+0000 7fa6e2a00640 1 --2- 172.21.15.69:0/3931824862 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa6d00a3c30 0x7fa6d00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.025+0000 7fa6e2a00640 1 -- 172.21.15.69:0/3931824862 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa6e40d0c20 con 0x7fa6ec103980 2024-04-24T09:06:52.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.025+0000 7fa6e9e00640 1 --2- 172.21.15.69:0/3931824862 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa6d00a3c30 0x7fa6d00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.026+0000 7fa6e9e00640 1 --2- 172.21.15.69:0/3931824862 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa6d00a3c30 0x7fa6d00a60f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc005fd0 tx=0x7fa6dc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:52.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.029+0000 7fa6e2a00640 1 -- 172.21.15.69:0/3931824862 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6e4014030 con 0x7fa6ec103980 2024-04-24T09:06:52.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.276+0000 7fa6eb200640 1 -- 172.21.15.69:0/3931824862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6ec1a2b70 con 0x7fa6ec103980 2024-04-24T09:06:52.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.278+0000 7fa6e2a00640 1 -- 172.21.15.69:0/3931824862 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11873 (secure 0 0 0) 0x7fa6e409d240 con 0x7fa6ec103980 2024-04-24T09:06:52.278 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:52.278 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:04:01.991559+0000","modified":"2024-04-24T09:04:34.617681+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34532,"mds_1":34496,"mds_2":34460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34496":{"gid":34496,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34532":{"gid":34532,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34532,"qdb_cluster":[34460,34496,34532]},"id":1}]} 2024-04-24T09:06:52.278 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2024-04-24T09:06:52.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.280+0000 7fa6eb200640 1 -- 172.21.15.69:0/3931824862 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa6d00a3c30 msgr2=0x7fa6d00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.280+0000 7fa6eb200640 1 --2- 172.21.15.69:0/3931824862 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa6d00a3c30 0x7fa6d00a60f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc005fd0 tx=0x7fa6dc005850 comp rx=0 tx=0).stop 2024-04-24T09:06:52.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.280+0000 7fa6eb200640 1 -- 172.21.15.69:0/3931824862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ec103980 msgr2=0x7fa6ec1a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.280+0000 7fa6eb200640 1 --2- 172.21.15.69:0/3931824862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ec103980 0x7fa6ec1a1830 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400cce0 tx=0x7fa6e400a620 comp rx=0 tx=0).stop 2024-04-24T09:06:52.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.281+0000 7fa6eb200640 1 -- 172.21.15.69:0/3931824862 shutdown_connections 2024-04-24T09:06:52.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.281+0000 7fa6eb200640 1 --2- 172.21.15.69:0/3931824862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ec103980 0x7fa6ec1a1830 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.281+0000 7fa6eb200640 1 --2- 172.21.15.69:0/3931824862 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6ec102fc0 0x7fa6ec19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.281+0000 7fa6eb200640 1 --2- 172.21.15.69:0/3931824862 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fa6d00a3c30 0x7fa6d00a60f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.281+0000 7fa6eb200640 1 --2- 172.21.15.69:0/3931824862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6ec101e70 0x7fa6ec19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.281+0000 7fa6eb200640 1 -- 172.21.15.69:0/3931824862 >> 172.21.15.69:0/3931824862 conn(0x7fa6ec0fd3e0 msgr2=0x7fa6ec1087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:52.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.281+0000 7fa6eb200640 1 -- 172.21.15.69:0/3931824862 shutdown_connections 2024-04-24T09:06:52.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.281+0000 7fa6eb200640 1 -- 172.21.15.69:0/3931824862 wait complete. 2024-04-24T09:06:52.299 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:06:52.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.481+0000 7fedb9800640 1 -- 172.21.15.69:0/1892541411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb4105450 msgr2=0x7fedb41058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.481+0000 7fedb9800640 1 --2- 172.21.15.69:0/1892541411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb4105450 0x7fedb41058d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009900 tx=0x7fed9c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.481+0000 7fedb9800640 1 -- 172.21.15.69:0/1892541411 shutdown_connections 2024-04-24T09:06:52.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.481+0000 7fedb9800640 1 --2- 172.21.15.69:0/1892541411 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedb4105e10 0x7fedb410c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.481+0000 7fedb9800640 1 --2- 172.21.15.69:0/1892541411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb4105450 0x7fedb41058d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.481+0000 7fedb9800640 1 --2- 172.21.15.69:0/1892541411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedb4104260 0x7fedb4104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.481+0000 7fedb9800640 1 -- 172.21.15.69:0/1892541411 >> 172.21.15.69:0/1892541411 conn(0x7fedb40ff7f0 msgr2=0x7fedb4101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:52.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.481+0000 7fedb9800640 1 -- 172.21.15.69:0/1892541411 shutdown_connections 2024-04-24T09:06:52.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.481+0000 7fedb9800640 1 -- 172.21.15.69:0/1892541411 wait complete. 2024-04-24T09:06:52.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.482+0000 7fedb9800640 1 Processor -- start 2024-04-24T09:06:52.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.482+0000 7fedb9800640 1 -- start start 2024-04-24T09:06:52.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb9800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedb4104260 0x7fedb419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb9800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb4105450 0x7fedb419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb9800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedb4105e10 0x7fedb41a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb9800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fedb410fdf0 con 0x7fedb4105450 2024-04-24T09:06:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb9800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fedb410fc70 con 0x7fedb4104260 2024-04-24T09:06:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb9800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fedb410ff70 con 0x7fedb4105e10 2024-04-24T09:06:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb2000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb4105450 0x7fedb419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb2000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb4105450 0x7fedb419a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32810/0 (socket says 172.21.15.69:32810) 2024-04-24T09:06:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb2000640 1 -- 172.21.15.69:0/1821219023 learned_addr learned my addr 172.21.15.69:0/1821219023 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb2a00640 1 --2- 172.21.15.69:0/1821219023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedb4104260 0x7fedb419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb3400640 1 --2- 172.21.15.69:0/1821219023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedb4105e10 0x7fedb41a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb2000640 1 -- 172.21.15.69:0/1821219023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedb4105e10 msgr2=0x7fedb41a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb2000640 1 --2- 172.21.15.69:0/1821219023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedb4105e10 0x7fedb41a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb2000640 1 -- 172.21.15.69:0/1821219023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedb4104260 msgr2=0x7fedb419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb2000640 1 --2- 172.21.15.69:0/1821219023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedb4104260 0x7fedb419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb2000640 1 -- 172.21.15.69:0/1821219023 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb41a2030 con 0x7fedb4105450 2024-04-24T09:06:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.483+0000 7fedb2a00640 1 --2- 172.21.15.69:0/1821219023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedb4104260 0x7fedb419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:52.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.484+0000 7fedb2000640 1 --2- 172.21.15.69:0/1821219023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb4105450 0x7fedb419a950 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fed9c02a910 tx=0x7fed9c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:52.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.484+0000 7fedab400640 1 -- 172.21.15.69:0/1821219023 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c03f070 con 0x7fedb4105450 2024-04-24T09:06:52.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.484+0000 7fedb9800640 1 -- 172.21.15.69:0/1821219023 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb41a22c0 con 0x7fedb4105450 2024-04-24T09:06:52.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.484+0000 7fedab400640 1 -- 172.21.15.69:0/1821219023 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed9c031e40 con 0x7fedb4105450 2024-04-24T09:06:52.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.484+0000 7fedab400640 1 -- 172.21.15.69:0/1821219023 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c035030 con 0x7fedb4105450 2024-04-24T09:06:52.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.484+0000 7fedb9800640 1 -- 172.21.15.69:0/1821219023 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb41a27b0 con 0x7fedb4105450 2024-04-24T09:06:52.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.485+0000 7fedb9800640 1 -- 172.21.15.69:0/1821219023 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb410b520 con 0x7fedb4105450 2024-04-24T09:06:52.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.487+0000 7fedab400640 1 -- 172.21.15.69:0/1821219023 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fed9c042070 con 0x7fedb4105450 2024-04-24T09:06:52.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.489+0000 7fedab400640 1 --2- 172.21.15.69:0/1821219023 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fed980a3bb0 0x7fed980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.489+0000 7fedab400640 1 -- 172.21.15.69:0/1821219023 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fed9c0f1790 con 0x7fedb4105450 2024-04-24T09:06:52.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.489+0000 7fedb2a00640 1 --2- 172.21.15.69:0/1821219023 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fed980a3bb0 0x7fed980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.489+0000 7fedb2a00640 1 --2- 172.21.15.69:0/1821219023 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fed980a3bb0 0x7fed980a6070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fedb41052b0 tx=0x7feda4005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:52.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.495+0000 7fedab400640 1 -- 172.21.15.69:0/1821219023 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fed9c0bddb0 con 0x7fedb4105450 2024-04-24T09:06:52.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.698+0000 7fedb9800640 1 -- 172.21.15.69:0/1821219023 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedb402d870 con 0x7fedb4105450 2024-04-24T09:06:52.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.701+0000 7fedab400640 1 -- 172.21.15.69:0/1821219023 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24564 (secure 0 0 0) 0x7fed9c0a80c0 con 0x7fedb4105450 2024-04-24T09:06:52.701 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:06:52.701 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":92,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:24.497245+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:06:52.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.703+0000 7fedb9800640 1 -- 172.21.15.69:0/1821219023 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fed980a3bb0 msgr2=0x7fed980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.703+0000 7fedb9800640 1 --2- 172.21.15.69:0/1821219023 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fed980a3bb0 0x7fed980a6070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fedb41052b0 tx=0x7feda4005db0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.703+0000 7fedb9800640 1 -- 172.21.15.69:0/1821219023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb4105450 msgr2=0x7fedb419a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.703+0000 7fedb9800640 1 --2- 172.21.15.69:0/1821219023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb4105450 0x7fedb419a950 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fed9c02a910 tx=0x7fed9c003cd0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.703+0000 7fedb9800640 1 -- 172.21.15.69:0/1821219023 shutdown_connections 2024-04-24T09:06:52.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.704+0000 7fedb9800640 1 --2- 172.21.15.69:0/1821219023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedb4105e10 0x7fedb41a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.704+0000 7fedb9800640 1 --2- 172.21.15.69:0/1821219023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb4105450 0x7fedb419a950 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.704+0000 7fedb9800640 1 --2- 172.21.15.69:0/1821219023 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fed980a3bb0 0x7fed980a6070 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.704+0000 7fedb9800640 1 --2- 172.21.15.69:0/1821219023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedb4104260 0x7fedb419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.704+0000 7fedb9800640 1 -- 172.21.15.69:0/1821219023 >> 172.21.15.69:0/1821219023 conn(0x7fedb40ff7f0 msgr2=0x7fedb41086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:52.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.704+0000 7fedb9800640 1 -- 172.21.15.69:0/1821219023 shutdown_connections 2024-04-24T09:06:52.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.704+0000 7fedb9800640 1 -- 172.21.15.69:0/1821219023 wait complete. 2024-04-24T09:06:52.721 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:52 smithi069 ceph-mon[28170]: pgmap v456: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:06:52.721 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:52 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3295782530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:06:52.721 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:52 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3931824862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:52.725 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T09:04:01.991559+0000', 'modified': '2024-04-24T09:04:34.617681+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 34532, 'mds_1': 34496, 'mds_2': 34460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6837/2790982303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6836', 'nonce': 2790982303}, {'type': 'v1', 'addr': '172.21.15.130:6837', 'nonce': 2790982303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6839/4193111366', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6838', 'nonce': 4193111366}, {'type': 'v1', 'addr': '172.21.15.130:6839', 'nonce': 4193111366}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34427': {'gid': 34427, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.81:6833/3240654002', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6832', 'nonce': 3240654002}, {'type': 'v1', 'addr': '172.21.15.81:6833', 'nonce': 3240654002}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34460': {'gid': 34460, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.81:6835/3260829578', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6834', 'nonce': 3260829578}, {'type': 'v1', 'addr': '172.21.15.81:6835', 'nonce': 3260829578}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34496': {'gid': 34496, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.81:6837/1973819593', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 1973819593}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 1973819593}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34532': {'gid': 34532, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.81:6839/3185704097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6838', 'nonce': 3185704097}, {'type': 'v1', 'addr': '172.21.15.81:6839', 'nonce': 3185704097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34532, 'qdb_cluster': [34460, 34496, 34532]} 2024-04-24T09:06:52.725 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T09:06:52.725 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-24T09:06:52.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:52 smithi081 ceph-mon[30961]: pgmap v456: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:06:52.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:52 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3295782530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:06:52.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:52 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3931824862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:52.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.906+0000 7f1159400640 1 -- 172.21.15.69:0/1648805595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154105e10 msgr2=0x7f115410c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.906+0000 7f1159400640 1 --2- 172.21.15.69:0/1648805595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154105e10 0x7f115410c6a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f114c009ca0 tx=0x7f114c02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.906+0000 7f1159400640 1 -- 172.21.15.69:0/1648805595 shutdown_connections 2024-04-24T09:06:52.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.906+0000 7f1159400640 1 --2- 172.21.15.69:0/1648805595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154105e10 0x7f115410c6a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.906+0000 7f1159400640 1 --2- 172.21.15.69:0/1648805595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1154105450 0x7f11541058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.906+0000 7f1159400640 1 --2- 172.21.15.69:0/1648805595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1154104260 0x7f1154104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.906+0000 7f1159400640 1 -- 172.21.15.69:0/1648805595 >> 172.21.15.69:0/1648805595 conn(0x7f11540ff7b0 msgr2=0x7f1154101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:52.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.906+0000 7f1159400640 1 -- 172.21.15.69:0/1648805595 shutdown_connections 2024-04-24T09:06:52.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.907+0000 7f1159400640 1 -- 172.21.15.69:0/1648805595 wait complete. 2024-04-24T09:06:52.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.907+0000 7f1159400640 1 Processor -- start 2024-04-24T09:06:52.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.908+0000 7f1159400640 1 -- start start 2024-04-24T09:06:52.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.908+0000 7f1159400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1154104260 0x7f115419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.908+0000 7f1159400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154105450 0x7f115419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.908+0000 7f1159400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1154105e10 0x7f11541a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.908+0000 7f1159400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f115410fdf0 con 0x7f1154105450 2024-04-24T09:06:52.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.908+0000 7f1159400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f115410fc70 con 0x7f1154105e10 2024-04-24T09:06:52.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.908+0000 7f1159400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f115410ff70 con 0x7f1154104260 2024-04-24T09:06:52.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.908+0000 7f114be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154105450 0x7f115419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.908+0000 7f114be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154105450 0x7f115419a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57530/0 (socket says 172.21.15.69:57530) 2024-04-24T09:06:52.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.908+0000 7f114be00640 1 -- 172.21.15.69:0/672842141 learned_addr learned my addr 172.21.15.69:0/672842141 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:52.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.908+0000 7f1152800640 1 --2- 172.21.15.69:0/672842141 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1154104260 0x7f115419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.909+0000 7f1153200640 1 --2- 172.21.15.69:0/672842141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1154105e10 0x7f11541a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.909+0000 7f114be00640 1 -- 172.21.15.69:0/672842141 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1154104260 msgr2=0x7f115419a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.909+0000 7f114be00640 1 --2- 172.21.15.69:0/672842141 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1154104260 0x7f115419a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.909+0000 7f114be00640 1 -- 172.21.15.69:0/672842141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1154105e10 msgr2=0x7f11541a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.909+0000 7f114be00640 1 --2- 172.21.15.69:0/672842141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1154105e10 0x7f11541a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.909+0000 7f114be00640 1 -- 172.21.15.69:0/672842141 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11541a1fa0 con 0x7f1154105450 2024-04-24T09:06:52.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.909+0000 7f114be00640 1 --2- 172.21.15.69:0/672842141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154105450 0x7f115419a8c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f113c00c920 tx=0x7f113c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:52.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.909+0000 7f114b400640 1 -- 172.21.15.69:0/672842141 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c007c70 con 0x7f1154105450 2024-04-24T09:06:52.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.909+0000 7f114b400640 1 -- 172.21.15.69:0/672842141 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f113c007e10 con 0x7f1154105450 2024-04-24T09:06:52.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.909+0000 7f114b400640 1 -- 172.21.15.69:0/672842141 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c018820 con 0x7f1154105450 2024-04-24T09:06:52.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.909+0000 7f1159400640 1 -- 172.21.15.69:0/672842141 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11541a2290 con 0x7f1154105450 2024-04-24T09:06:52.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.909+0000 7f1159400640 1 -- 172.21.15.69:0/672842141 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11541a2790 con 0x7f1154105450 2024-04-24T09:06:52.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.912+0000 7f115c18e740 1 Processor -- start 2024-04-24T09:06:52.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.912+0000 7f114b400640 1 -- 172.21.15.69:0/672842141 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f113c0189c0 con 0x7f1154105450 2024-04-24T09:06:52.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.913+0000 7f115c18e740 1 -- start start 2024-04-24T09:06:52.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.913+0000 7f115c18e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867ab090 0x55f3867ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.913+0000 7f115c18e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867aba80 0x55f3867b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.913+0000 7f115c18e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867b0bf0 0x55f3867b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.913+0000 7f115c18e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f386710b10 con 0x55f3867aba80 2024-04-24T09:06:52.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.913+0000 7f115c18e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f3866b72b0 con 0x55f3867b0bf0 2024-04-24T09:06:52.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.913+0000 7f115c18e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f38670cad0 con 0x55f3867ab090 2024-04-24T09:06:52.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.913+0000 7f1152800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867aba80 0x55f3867b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.913+0000 7f1152800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867aba80 0x55f3867b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57538/0 (socket says 172.21.15.69:57538) 2024-04-24T09:06:52.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.913+0000 7f1152800640 1 -- 172.21.15.69:0/3349109339 learned_addr learned my addr 172.21.15.69:0/3349109339 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:52.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.914+0000 7f1153200640 1 --2- 172.21.15.69:0/3349109339 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867ab090 0x55f3867ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.914+0000 7f114be00640 1 --2- 172.21.15.69:0/3349109339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867b0bf0 0x55f3867b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.914+0000 7f1152800640 1 -- 172.21.15.69:0/3349109339 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867ab090 msgr2=0x55f3867ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:52.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.914+0000 7f1152800640 1 --2- 172.21.15.69:0/3349109339 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867ab090 0x55f3867ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.914+0000 7f1152800640 1 -- 172.21.15.69:0/3349109339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867b0bf0 msgr2=0x55f3867b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:52.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.914+0000 7f1152800640 1 --2- 172.21.15.69:0/3349109339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867b0bf0 0x55f3867b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.914+0000 7f1152800640 1 -- 172.21.15.69:0/3349109339 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3864232d0 con 0x55f3867aba80 2024-04-24T09:06:52.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.914+0000 7f114be00640 1 --2- 172.21.15.69:0/3349109339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867b0bf0 0x55f3867b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:52.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.914+0000 7f1152800640 1 --2- 172.21.15.69:0/3349109339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867aba80 0x55f3867b0580 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f114400bc80 tx=0x7f114400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3e27c94c55667c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:52.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.914+0000 7f114b400640 1 --2- 172.21.15.69:0/672842141 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f11380a3bb0 0x7f11380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.915+0000 7f114b400640 1 -- 172.21.15.69:0/672842141 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f113c0d09a0 con 0x7f1154105450 2024-04-24T09:06:52.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.915+0000 7f1153200640 1 --2- 172.21.15.69:0/672842141 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f11380a3bb0 0x7f11380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.915+0000 7f1143400640 1 -- 172.21.15.69:0/3349109339 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144015070 con 0x55f3867aba80 2024-04-24T09:06:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.915+0000 7f1143400640 1 -- 172.21.15.69:0/3349109339 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11440118f0 con 0x55f3867aba80 2024-04-24T09:06:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.915+0000 7f1143400640 1 -- 172.21.15.69:0/3349109339 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144010830 con 0x55f3867aba80 2024-04-24T09:06:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.916+0000 7f115c18e740 1 -- 172.21.15.69:0/3349109339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867aba80 msgr2=0x55f3867b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.916+0000 7f1153200640 1 --2- 172.21.15.69:0/672842141 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f11380a3bb0 0x7f11380a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55f3867ac2e0 tx=0x7f114c031a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.916+0000 7f115c18e740 1 --2- 172.21.15.69:0/3349109339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867aba80 0x55f3867b0580 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f114400bc80 tx=0x7f114400f730 comp rx=0 tx=0).stop 2024-04-24T09:06:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.916+0000 7f115c18e740 1 -- 172.21.15.69:0/3349109339 shutdown_connections 2024-04-24T09:06:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.916+0000 7f115c18e740 1 --2- 172.21.15.69:0/3349109339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867b0bf0 0x55f3867b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.916+0000 7f115c18e740 1 --2- 172.21.15.69:0/3349109339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867aba80 0x55f3867b0580 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.916+0000 7f115c18e740 1 --2- 172.21.15.69:0/3349109339 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867ab090 0x55f3867ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.916+0000 7f115c18e740 1 -- 172.21.15.69:0/3349109339 >> 172.21.15.69:0/3349109339 conn(0x55f3867a6700 msgr2=0x55f3867a6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.916+0000 7f115c18e740 1 -- 172.21.15.69:0/3349109339 shutdown_connections 2024-04-24T09:06:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.916+0000 7f115c18e740 1 -- 172.21.15.69:0/3349109339 wait complete. 2024-04-24T09:06:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.916+0000 7f115c18e740 1 -- start start 2024-04-24T09:06:52.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.917+0000 7f115c18e740 1 Processor -- start 2024-04-24T09:06:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.917+0000 7f115c18e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867ab090 0x55f3867b5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.917+0000 7f115c18e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867aba80 0x55f3867b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.917+0000 7f115c18e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867b0bf0 0x55f3867bca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.917+0000 7f115c18e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f3867b3ba0 con 0x55f3867ab090 2024-04-24T09:06:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.917+0000 7f115c18e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f3866ca590 con 0x55f3867b0bf0 2024-04-24T09:06:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.917+0000 7f115c18e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f3867b3d20 con 0x55f3867aba80 2024-04-24T09:06:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.917+0000 7f115c18e740 10 client.0 _ll_register_callbacks cb 0x55f3866ad3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.917+0000 7f1152800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867ab090 0x55f3867b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.917+0000 7f1152800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867ab090 0x55f3867b5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57544/0 (socket says 172.21.15.69:57544) 2024-04-24T09:06:52.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.917+0000 7f1152800640 1 -- 172.21.15.69:0/3543947418 learned_addr learned my addr 172.21.15.69:0/3543947418 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:52.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.918+0000 7f114be00640 1 --2- 172.21.15.69:0/3543947418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867aba80 0x55f3867b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.918+0000 7f1153200640 1 --2- 172.21.15.69:0/3543947418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867b0bf0 0x55f3867bca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.918+0000 7f1152800640 1 -- 172.21.15.69:0/3543947418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867aba80 msgr2=0x55f3867b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:52.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.918+0000 7f1152800640 1 --2- 172.21.15.69:0/3543947418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867aba80 0x55f3867b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.918+0000 7f1152800640 1 -- 172.21.15.69:0/3543947418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867b0bf0 msgr2=0x55f3867bca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:52.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.918+0000 7f1152800640 1 --2- 172.21.15.69:0/3543947418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867b0bf0 0x55f3867bca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.918+0000 7f1152800640 1 -- 172.21.15.69:0/3543947418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3864ed6b0 con 0x55f3867ab090 2024-04-24T09:06:52.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.918+0000 7f1152800640 1 --2- 172.21.15.69:0/3543947418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867ab090 0x55f3867b5510 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f114400b800 tx=0x7f114400bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8df2ac08365978e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:52.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.918+0000 7f1137e00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:06:52.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.918+0000 7f1137e00640 1 -- 172.21.15.69:0/3543947418 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144028060 con 0x55f3867ab090 2024-04-24T09:06:52.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.918+0000 7f1137e00640 1 -- 172.21.15.69:0/3543947418 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1144011aa0 con 0x55f3867ab090 2024-04-24T09:06:52.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.918+0000 7f115c18e740 1 -- 172.21.15.69:0/3543947418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11440118f0 con 0x55f3867ab090 2024-04-24T09:06:52.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.919+0000 7f1137e00640 1 -- 172.21.15.69:0/3543947418 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f114402f050 con 0x55f3867ab090 2024-04-24T09:06:52.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.919+0000 7f115c18e740 10 client.15399 fetch_fsmap learned FSMap version 23 2024-04-24T09:06:52.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.919+0000 7f115c18e740 1 -- 172.21.15.69:0/3543947418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f3867bfd70 con 0x55f3867ab090 2024-04-24T09:06:52.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.919+0000 7f1137e00640 1 -- 172.21.15.69:0/3543947418 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11440118f0 con 0x55f3867ab090 2024-04-24T09:06:52.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.919+0000 7f1137e00640 1 -- 172.21.15.69:0/3543947418 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f1144004680 con 0x55f3867ab090 2024-04-24T09:06:52.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.919+0000 7f115c18e740 10 client.15399 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T09:06:52.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.919+0000 7f115c18e740 10 client.15399 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-24T09:06:52.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.919+0000 7f115c18e740 20 client.15399 populate_metadata read hostname 'smithi069' 2024-04-24T09:06:52.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.920+0000 7f115c18e740 1 --2- 172.21.15.69:0/3543947418 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55f3867c2050 0x55f3867c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.920+0000 7f115c18e740 4 client.15399 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:52.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.920+0000 7f115c18e740 1 -- 172.21.15.69:0/3543947418 --> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f38671e530 con 0x55f3867c2050 2024-04-24T09:06:52.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.920+0000 7f114be00640 1 --2- 172.21.15.69:0/3543947418 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55f3867c2050 0x55f3867c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.921+0000 7f114be00640 1 --2- 172.21.15.69:0/3543947418 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55f3867c2050 0x55f3867c34c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4032ca228c1f5dec server_cookie=5b4d078a4f2171c9 in_seq=0 out_seq=0 2024-04-24T09:06:52.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.921+0000 7f1137e00640 10 client.15399 ms_handle_connect on v2:172.21.15.81:6834/3260829578 2024-04-24T09:06:52.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.923+0000 7f1137e00640 1 -- 172.21.15.69:0/3543947418 <== mds.2 v2:172.21.15.81:6834/3260829578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f38671e530 con 0x55f3867c2050 2024-04-24T09:06:52.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.923+0000 7f1137e00640 10 client.15399 handle_command_reply: tid=0 2024-04-24T09:06:52.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.923+0000 7f115c18e740 1 client.15399 shutdown 2024-04-24T09:06:52.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.923+0000 7f115c18e740 1 -- 172.21.15.69:0/3543947418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867ab090 msgr2=0x55f3867b5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.923+0000 7f115c18e740 1 --2- 172.21.15.69:0/3543947418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867ab090 0x55f3867b5510 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f114400b800 tx=0x7f114400bfa0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.923+0000 7f115c18e740 1 -- 172.21.15.69:0/3543947418 shutdown_connections 2024-04-24T09:06:52.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.923+0000 7f115c18e740 1 --2- 172.21.15.69:0/3543947418 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55f3867c2050 0x55f3867c34c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.923+0000 7f115c18e740 1 --2- 172.21.15.69:0/3543947418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867b0bf0 0x55f3867bca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.923+0000 7f115c18e740 1 --2- 172.21.15.69:0/3543947418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867aba80 0x55f3867b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.923+0000 7f115c18e740 1 --2- 172.21.15.69:0/3543947418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867ab090 0x55f3867b5510 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.923+0000 7f115c18e740 1 -- 172.21.15.69:0/3543947418 >> 172.21.15.69:0/3543947418 conn(0x55f3867a6700 msgr2=0x55f3867a58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:52.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.923+0000 7f1137e00640 0 client.15399 ms_handle_reset on v2:172.21.15.81:6834/3260829578 2024-04-24T09:06:52.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.924+0000 7f115c18e740 1 -- 172.21.15.69:0/3543947418 shutdown_connections 2024-04-24T09:06:52.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.924+0000 7f115c18e740 1 -- 172.21.15.69:0/3543947418 wait complete. 2024-04-24T09:06:52.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.924+0000 7f115c18e740 20 client.15399 trim_cache size 0 max 16384 2024-04-24T09:06:52.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.942+0000 7f115c18e740 1 Processor -- start 2024-04-24T09:06:52.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.942+0000 7f115c18e740 1 -- start start 2024-04-24T09:06:52.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.943+0000 7f115c18e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867ab090 0x55f3867c1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.943+0000 7f115c18e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867aba80 0x55f3867b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.943+0000 7f115c18e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867b0bf0 0x55f3867a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.943+0000 7f115c18e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f3867c1800 con 0x55f3867ab090 2024-04-24T09:06:52.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.943+0000 7f115c18e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f3867c1ed0 con 0x55f3867b0bf0 2024-04-24T09:06:52.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.943+0000 7f115c18e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f3867c25a0 con 0x55f3867aba80 2024-04-24T09:06:52.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.943+0000 7f1152800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867ab090 0x55f3867c1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.943+0000 7f1152800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867ab090 0x55f3867c1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57556/0 (socket says 172.21.15.69:57556) 2024-04-24T09:06:52.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.943+0000 7f1152800640 1 -- 172.21.15.69:0/2906254336 learned_addr learned my addr 172.21.15.69:0/2906254336 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:52.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.943+0000 7f1153200640 1 --2- 172.21.15.69:0/2906254336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867b0bf0 0x55f3867a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.943+0000 7f114be00640 1 --2- 172.21.15.69:0/2906254336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867aba80 0x55f3867b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.944+0000 7f1152800640 1 -- 172.21.15.69:0/2906254336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867aba80 msgr2=0x55f3867b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:52.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.944+0000 7f1152800640 1 --2- 172.21.15.69:0/2906254336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867aba80 0x55f3867b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.944+0000 7f1152800640 1 -- 172.21.15.69:0/2906254336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867b0bf0 msgr2=0x55f3867a6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:52.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.944+0000 7f1152800640 1 --2- 172.21.15.69:0/2906254336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867b0bf0 0x55f3867a6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.944+0000 7f1152800640 1 -- 172.21.15.69:0/2906254336 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3867a73d0 con 0x55f3867ab090 2024-04-24T09:06:52.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.944+0000 7f1152800640 1 --2- 172.21.15.69:0/2906254336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867ab090 0x55f3867c1170 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1144004ad0 tx=0x7f114400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c599af88c345410 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:52.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.944+0000 7f1142a00640 1 -- 172.21.15.69:0/2906254336 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144028070 con 0x55f3867ab090 2024-04-24T09:06:52.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.944+0000 7f1142a00640 1 -- 172.21.15.69:0/2906254336 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1144004cc0 con 0x55f3867ab090 2024-04-24T09:06:52.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.944+0000 7f1142a00640 1 -- 172.21.15.69:0/2906254336 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144015070 con 0x55f3867ab090 2024-04-24T09:06:52.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.945+0000 7f115c18e740 1 -- 172.21.15.69:0/2906254336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867ab090 msgr2=0x55f3867c1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.945+0000 7f115c18e740 1 --2- 172.21.15.69:0/2906254336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867ab090 0x55f3867c1170 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1144004ad0 tx=0x7f114400f730 comp rx=0 tx=0).stop 2024-04-24T09:06:52.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.945+0000 7f115c18e740 1 -- 172.21.15.69:0/2906254336 shutdown_connections 2024-04-24T09:06:52.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.945+0000 7f115c18e740 1 --2- 172.21.15.69:0/2906254336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867b0bf0 0x55f3867a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.945+0000 7f115c18e740 1 --2- 172.21.15.69:0/2906254336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867aba80 0x55f3867b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.945+0000 7f115c18e740 1 --2- 172.21.15.69:0/2906254336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867ab090 0x55f3867c1170 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.945+0000 7f115c18e740 1 -- 172.21.15.69:0/2906254336 >> 172.21.15.69:0/2906254336 conn(0x55f3867a6700 msgr2=0x55f3867b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:52.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.945+0000 7f115c18e740 1 -- 172.21.15.69:0/2906254336 shutdown_connections 2024-04-24T09:06:52.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.945+0000 7f115c18e740 1 -- 172.21.15.69:0/2906254336 wait complete. 2024-04-24T09:06:52.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.945+0000 7f115c18e740 1 -- start start 2024-04-24T09:06:52.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.946+0000 7f115c18e740 1 Processor -- start 2024-04-24T09:06:52.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.946+0000 7f115c18e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867ab090 0x55f3867ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.946+0000 7f115c18e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867aba80 0x55f3867ae580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.946+0000 7f115c18e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867b0bf0 0x55f3867aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.946+0000 7f115c18e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f3867c3340 con 0x55f3867b0bf0 2024-04-24T09:06:52.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.946+0000 7f115c18e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f3867c2c70 con 0x55f3867aba80 2024-04-24T09:06:52.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.946+0000 7f115c18e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f3867c3a10 con 0x55f3867ab090 2024-04-24T09:06:52.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.946+0000 7f115c18e740 10 client.0 _ll_register_callbacks cb 0x7f114400b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.946+0000 7f1153200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867b0bf0 0x55f3867aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.946+0000 7f1153200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867b0bf0 0x55f3867aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57560/0 (socket says 172.21.15.69:57560) 2024-04-24T09:06:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.946+0000 7f1153200640 1 -- 172.21.15.69:0/2959101759 learned_addr learned my addr 172.21.15.69:0/2959101759 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.947+0000 7f1152800640 1 --2- 172.21.15.69:0/2959101759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867ab090 0x55f3867ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.947+0000 7f114be00640 1 --2- 172.21.15.69:0/2959101759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867aba80 0x55f3867ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.947+0000 7f1153200640 1 -- 172.21.15.69:0/2959101759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867ab090 msgr2=0x55f3867ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.947+0000 7f1153200640 1 --2- 172.21.15.69:0/2959101759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867ab090 0x55f3867ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.947+0000 7f1153200640 1 -- 172.21.15.69:0/2959101759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867aba80 msgr2=0x55f3867ae580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.947+0000 7f1153200640 1 --2- 172.21.15.69:0/2959101759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867aba80 0x55f3867ae580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.947+0000 7f1153200640 1 -- 172.21.15.69:0/2959101759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3867baa40 con 0x55f3867b0bf0 2024-04-24T09:06:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.947+0000 7f1153200640 1 --2- 172.21.15.69:0/2959101759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867b0bf0 0x55f3867aeac0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f114c02aeb0 tx=0x7f114c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a68c784409c47d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.947+0000 7f1137e00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:06:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.947+0000 7f1137e00640 1 -- 172.21.15.69:0/2959101759 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c056060 con 0x55f3867b0bf0 2024-04-24T09:06:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.948+0000 7f1137e00640 1 -- 172.21.15.69:0/2959101759 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f114c004450 con 0x55f3867b0bf0 2024-04-24T09:06:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.948+0000 7f115c18e740 1 -- 172.21.15.69:0/2959101759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1144004cc0 con 0x55f3867b0bf0 2024-04-24T09:06:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.948+0000 7f1137e00640 1 -- 172.21.15.69:0/2959101759 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f114c004690 con 0x55f3867b0bf0 2024-04-24T09:06:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.948+0000 7f115c18e740 10 client.15405 fetch_fsmap learned FSMap version 23 2024-04-24T09:06:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.948+0000 7f115c18e740 1 -- 172.21.15.69:0/2959101759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f3867bbd40 con 0x55f3867b0bf0 2024-04-24T09:06:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.948+0000 7f1137e00640 1 -- 172.21.15.69:0/2959101759 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1144004cc0 con 0x55f3867b0bf0 2024-04-24T09:06:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.948+0000 7f1137e00640 1 -- 172.21.15.69:0/2959101759 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f114c050070 con 0x55f3867b0bf0 2024-04-24T09:06:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.948+0000 7f115c18e740 10 client.15405 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T09:06:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.948+0000 7f115c18e740 10 client.15405 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-24T09:06:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.948+0000 7f115c18e740 20 client.15405 populate_metadata read hostname 'smithi069' 2024-04-24T09:06:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.948+0000 7f115c18e740 1 --2- 172.21.15.69:0/2959101759 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55f38682c400 0x55f38682b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.948+0000 7f115c18e740 4 client.15405 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:06:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.948+0000 7f115c18e740 1 -- 172.21.15.69:0/2959101759 --> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] -- command(tid 0: {"prefix": "status"}) -- 0x55f3867183a0 con 0x55f38682c400 2024-04-24T09:06:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.949+0000 7f1152800640 1 --2- 172.21.15.69:0/2959101759 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55f38682c400 0x55f38682b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:52.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.950+0000 7f1137e00640 10 client.15405 ms_handle_connect on v2:172.21.15.81:6834/3260829578 2024-04-24T09:06:52.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.950+0000 7f1152800640 1 --2- 172.21.15.69:0/2959101759 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55f38682c400 0x55f38682b110 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b97e8baa600f6a3f server_cookie=3fca152bd9406773 in_seq=0 out_seq=0 2024-04-24T09:06:52.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.950+0000 7f1137e00640 1 -- 172.21.15.69:0/2959101759 <== mds.2 v2:172.21.15.81:6834/3260829578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f3867183a0 con 0x55f38682c400 2024-04-24T09:06:52.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.950+0000 7f1137e00640 10 client.15405 handle_command_reply: tid=0 2024-04-24T09:06:52.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.950+0000 7f115c18e740 1 client.15405 shutdown 2024-04-24T09:06:52.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.951+0000 7f115c18e740 1 -- 172.21.15.69:0/2959101759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867b0bf0 msgr2=0x55f3867aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.951+0000 7f115c18e740 1 --2- 172.21.15.69:0/2959101759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867b0bf0 0x55f3867aeac0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f114c02aeb0 tx=0x7f114c0095c0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.951+0000 7f115c18e740 1 -- 172.21.15.69:0/2959101759 shutdown_connections 2024-04-24T09:06:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.951+0000 7f115c18e740 1 --2- 172.21.15.69:0/2959101759 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55f38682c400 0x55f38682b110 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.951+0000 7f115c18e740 1 --2- 172.21.15.69:0/2959101759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f3867b0bf0 0x55f3867aeac0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.951+0000 7f1137e00640 0 client.15405 ms_handle_reset on v2:172.21.15.81:6834/3260829578 2024-04-24T09:06:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.951+0000 7f115c18e740 1 --2- 172.21.15.69:0/2959101759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3867aba80 0x55f3867ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.951+0000 7f115c18e740 1 --2- 172.21.15.69:0/2959101759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f3867ab090 0x55f3867ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.951+0000 7f115c18e740 1 -- 172.21.15.69:0/2959101759 >> 172.21.15.69:0/2959101759 conn(0x55f3867a6700 msgr2=0x55f3867b0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.951+0000 7f115c18e740 1 -- 172.21.15.69:0/2959101759 shutdown_connections 2024-04-24T09:06:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.951+0000 7f115c18e740 1 -- 172.21.15.69:0/2959101759 wait complete. 2024-04-24T09:06:52.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.951+0000 7f115c18e740 20 client.15405 trim_cache size 0 max 16384 2024-04-24T09:06:52.955 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:06:52.955 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "ef1ec7e8-0217-11ef-bc93-c7b262605968", 2024-04-24T09:06:52.955 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 2, 2024-04-24T09:06:52.955 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 34460, 2024-04-24T09:06:52.955 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-24T09:06:52.955 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-24T09:06:52.955 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-24T09:06:52.955 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 139.34363494999999, 2024-04-24T09:06:52.955 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 23, 2024-04-24T09:06:52.955 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 92, 2024-04-24T09:06:52.956 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 92, 2024-04-24T09:06:52.956 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 204.31891442599999 2024-04-24T09:06:52.956 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:06:52.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.952+0000 7f1159400640 1 -- 172.21.15.69:0/672842141 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f11380a3bb0 msgr2=0x7f11380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.952+0000 7f1159400640 1 --2- 172.21.15.69:0/672842141 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f11380a3bb0 0x7f11380a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55f3867ac2e0 tx=0x7f114c031a40 comp rx=0 tx=0).stop 2024-04-24T09:06:52.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.952+0000 7f1159400640 1 -- 172.21.15.69:0/672842141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154105450 msgr2=0x7f115419a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:52.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.952+0000 7f1159400640 1 --2- 172.21.15.69:0/672842141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154105450 0x7f115419a8c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f113c00c920 tx=0x7f113c00cde0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.952+0000 7f1159400640 1 -- 172.21.15.69:0/672842141 shutdown_connections 2024-04-24T09:06:52.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.952+0000 7f1159400640 1 --2- 172.21.15.69:0/672842141 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f11380a3bb0 0x7f11380a6070 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.952+0000 7f1159400640 1 --2- 172.21.15.69:0/672842141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1154105e10 0x7f11541a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.952+0000 7f1159400640 1 --2- 172.21.15.69:0/672842141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1154105450 0x7f115419a8c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.952+0000 7f1159400640 1 --2- 172.21.15.69:0/672842141 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1154104260 0x7f115419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:52.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.952+0000 7f1159400640 1 -- 172.21.15.69:0/672842141 >> 172.21.15.69:0/672842141 conn(0x7f11540ff7b0 msgr2=0x7f11541086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:52.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.952+0000 7f1159400640 1 -- 172.21.15.69:0/672842141 shutdown_connections 2024-04-24T09:06:52.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:52.952+0000 7f1159400640 1 -- 172.21.15.69:0/672842141 wait complete. 2024-04-24T09:06:52.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:52 smithi130 ceph-mon[31553]: pgmap v456: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T09:06:52.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:52 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3295782530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:06:52.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:52 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3931824862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:52.968 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-24T09:06:53.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.153+0000 7f2420e00640 1 -- 172.21.15.69:0/4234330032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f241c101f90 msgr2=0x7f241c1023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.153+0000 7f2420e00640 1 --2- 172.21.15.69:0/4234330032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f241c101f90 0x7f241c1023b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f240c009900 tx=0x7f240c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.153+0000 7f2420e00640 1 -- 172.21.15.69:0/4234330032 shutdown_connections 2024-04-24T09:06:53.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.153+0000 7f2420e00640 1 --2- 172.21.15.69:0/4234330032 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f241c103b40 0x7f241c10a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.153+0000 7f2420e00640 1 --2- 172.21.15.69:0/4234330032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f241c103180 0x7f241c103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.153+0000 7f2420e00640 1 --2- 172.21.15.69:0/4234330032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f241c101f90 0x7f241c1023b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.153+0000 7f2420e00640 1 -- 172.21.15.69:0/4234330032 >> 172.21.15.69:0/4234330032 conn(0x7f241c0fd500 msgr2=0x7f241c0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:53.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.153+0000 7f2420e00640 1 -- 172.21.15.69:0/4234330032 shutdown_connections 2024-04-24T09:06:53.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.153+0000 7f2420e00640 1 -- 172.21.15.69:0/4234330032 wait complete. 2024-04-24T09:06:53.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.154+0000 7f2420e00640 1 Processor -- start 2024-04-24T09:06:53.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.154+0000 7f2420e00640 1 -- start start 2024-04-24T09:06:53.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.155+0000 7f2420e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f241c101f90 0x7f241c19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.155+0000 7f2420e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f241c103180 0x7f241c19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.155+0000 7f2420e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f241c103b40 0x7f241c1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.155+0000 7f2420e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f241c10da70 con 0x7f241c103180 2024-04-24T09:06:53.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.155+0000 7f2420e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f241c10d8f0 con 0x7f241c103b40 2024-04-24T09:06:53.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.155+0000 7f2420e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f241c10dbf0 con 0x7f241c101f90 2024-04-24T09:06:53.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.155+0000 7f241aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f241c103180 0x7f241c19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.155+0000 7f241aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f241c103180 0x7f241c19a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57572/0 (socket says 172.21.15.69:57572) 2024-04-24T09:06:53.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.155+0000 7f241aa00640 1 -- 172.21.15.69:0/1761426445 learned_addr learned my addr 172.21.15.69:0/1761426445 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:53.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.155+0000 7f241aa00640 1 -- 172.21.15.69:0/1761426445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f241c101f90 msgr2=0x7f241c19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.155+0000 7f241aa00640 1 --2- 172.21.15.69:0/1761426445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f241c101f90 0x7f241c19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.155+0000 7f241aa00640 1 -- 172.21.15.69:0/1761426445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f241c103b40 msgr2=0x7f241c1a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.155+0000 7f241aa00640 1 --2- 172.21.15.69:0/1761426445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f241c103b40 0x7f241c1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.155+0000 7f241aa00640 1 -- 172.21.15.69:0/1761426445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c1a1f50 con 0x7f241c103180 2024-04-24T09:06:53.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.156+0000 7f241aa00640 1 --2- 172.21.15.69:0/1761426445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f241c103180 0x7f241c19a8c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f240800d950 tx=0x7f240800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:53.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.156+0000 7f2413e00640 1 -- 172.21.15.69:0/1761426445 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408017070 con 0x7f241c103180 2024-04-24T09:06:53.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.156+0000 7f2413e00640 1 -- 172.21.15.69:0/1761426445 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2408009c30 con 0x7f241c103180 2024-04-24T09:06:53.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.156+0000 7f2420e00640 1 -- 172.21.15.69:0/1761426445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241c1a2240 con 0x7f241c103180 2024-04-24T09:06:53.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.156+0000 7f2413e00640 1 -- 172.21.15.69:0/1761426445 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240800f620 con 0x7f241c103180 2024-04-24T09:06:53.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.156+0000 7f2420e00640 1 -- 172.21.15.69:0/1761426445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241c1a2790 con 0x7f241c103180 2024-04-24T09:06:53.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.159+0000 7f2425260740 1 Processor -- start 2024-04-24T09:06:53.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.159+0000 7f2425260740 1 -- start start 2024-04-24T09:06:53.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.159+0000 7f2425260740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6a9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.159+0000 7f2425260740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6a9c50 0x55836e6ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.159+0000 7f2425260740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6aedc0 0x55836e6b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.159+0000 7f2425260740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55836e60eb10 con 0x55836e6a9260 2024-04-24T09:06:53.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.159+0000 7f2425260740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55836e5b52b0 con 0x55836e6aedc0 2024-04-24T09:06:53.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.159+0000 7f2425260740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55836e60aad0 con 0x55836e6a9c50 2024-04-24T09:06:53.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.159+0000 7f2413e00640 1 -- 172.21.15.69:0/1761426445 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f24080109a0 con 0x7f241c103180 2024-04-24T09:06:53.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.160+0000 7f241be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.160+0000 7f241be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6a9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57586/0 (socket says 172.21.15.69:57586) 2024-04-24T09:06:53.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.160+0000 7f241be00640 1 -- 172.21.15.69:0/226017927 learned_addr learned my addr 172.21.15.69:0/226017927 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:53.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.160+0000 7f241b400640 1 --2- 172.21.15.69:0/226017927 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6a9c50 0x55836e6ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.160+0000 7f241aa00640 1 --2- 172.21.15.69:0/226017927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6aedc0 0x55836e6b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.160+0000 7f241be00640 1 -- 172.21.15.69:0/226017927 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6a9c50 msgr2=0x55836e6ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:53.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.160+0000 7f241be00640 1 --2- 172.21.15.69:0/226017927 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6a9c50 0x55836e6ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.160+0000 7f241be00640 1 -- 172.21.15.69:0/226017927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6aedc0 msgr2=0x55836e6b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:53.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.160+0000 7f241be00640 1 --2- 172.21.15.69:0/226017927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6aedc0 0x55836e6b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.160+0000 7f241be00640 1 -- 172.21.15.69:0/226017927 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55836e3212d0 con 0x55836e6a9260 2024-04-24T09:06:53.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.160+0000 7f241be00640 1 --2- 172.21.15.69:0/226017927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6a9680 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2414007cf0 tx=0x7f241400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c059bbe58794c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:53.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.161+0000 7f2410c00640 1 -- 172.21.15.69:0/226017927 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414014070 con 0x55836e6a9260 2024-04-24T09:06:53.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.161+0000 7f2410c00640 1 -- 172.21.15.69:0/226017927 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f241400fb00 con 0x55836e6a9260 2024-04-24T09:06:53.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.161+0000 7f2410c00640 1 -- 172.21.15.69:0/226017927 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241400eb20 con 0x55836e6a9260 2024-04-24T09:06:53.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.161+0000 7f2413e00640 1 --2- 172.21.15.69:0/1761426445 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f24000a3bb0 0x7f24000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.162+0000 7f2425260740 1 -- 172.21.15.69:0/226017927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 msgr2=0x55836e6a9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.162+0000 7f2425260740 1 --2- 172.21.15.69:0/226017927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6a9680 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2414007cf0 tx=0x7f241400c6a0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.162+0000 7f2413e00640 1 -- 172.21.15.69:0/1761426445 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f24080d04a0 con 0x7f241c103180 2024-04-24T09:06:53.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.162+0000 7f241b400640 1 --2- 172.21.15.69:0/1761426445 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f24000a3bb0 0x7f24000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.162+0000 7f2425260740 1 -- 172.21.15.69:0/226017927 shutdown_connections 2024-04-24T09:06:53.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.162+0000 7f2425260740 1 --2- 172.21.15.69:0/226017927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6aedc0 0x55836e6b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.162+0000 7f2425260740 1 --2- 172.21.15.69:0/226017927 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6a9c50 0x55836e6ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.162+0000 7f2425260740 1 --2- 172.21.15.69:0/226017927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6a9680 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.162+0000 7f2425260740 1 -- 172.21.15.69:0/226017927 >> 172.21.15.69:0/226017927 conn(0x55836e6a48d0 msgr2=0x55836e6a4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:53.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.162+0000 7f2425260740 1 -- 172.21.15.69:0/226017927 shutdown_connections 2024-04-24T09:06:53.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.162+0000 7f2425260740 1 -- 172.21.15.69:0/226017927 wait complete. 2024-04-24T09:06:53.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.162+0000 7f241b400640 1 --2- 172.21.15.69:0/1761426445 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f24000a3bb0 0x7f24000a6070 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55836e6aa6c0 tx=0x7f240c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.163+0000 7f2425260740 1 -- start start 2024-04-24T09:06:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.163+0000 7f2425260740 1 Processor -- start 2024-04-24T09:06:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.163+0000 7f2425260740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.163+0000 7f2425260740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6a9c50 0x55836e6b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.164+0000 7f2425260740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6aedc0 0x55836e6bac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.164+0000 7f2425260740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55836e6b1d70 con 0x55836e6a9260 2024-04-24T09:06:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.164+0000 7f2425260740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55836e5c8590 con 0x55836e6a9c50 2024-04-24T09:06:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.164+0000 7f2425260740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55836e6b1ef0 con 0x55836e6aedc0 2024-04-24T09:06:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.164+0000 7f2425260740 10 client.0 _ll_register_callbacks cb 0x55836e5ab3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.164+0000 7f241b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.164+0000 7f241b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57596/0 (socket says 172.21.15.69:57596) 2024-04-24T09:06:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.164+0000 7f241b400640 1 -- 172.21.15.69:0/1615424398 learned_addr learned my addr 172.21.15.69:0/1615424398 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.164+0000 7f241b400640 1 -- 172.21.15.69:0/1615424398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6aedc0 msgr2=0x55836e6bac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.164+0000 7f241b400640 1 --2- 172.21.15.69:0/1615424398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6aedc0 0x55836e6bac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.164+0000 7f241b400640 1 -- 172.21.15.69:0/1615424398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6a9c50 msgr2=0x55836e6b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.164+0000 7f241b400640 1 --2- 172.21.15.69:0/1615424398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6a9c50 0x55836e6b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.164+0000 7f241b400640 1 -- 172.21.15.69:0/1615424398 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55836e3eb6b0 con 0x55836e6a9260 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.164+0000 7f241b400640 1 --2- 172.21.15.69:0/1615424398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6b36d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f241c102fe0 tx=0x7f240c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b7b87bb2869a21 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.164+0000 7f2405600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.165+0000 7f2405600640 1 -- 172.21.15.69:0/1615424398 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c042070 con 0x55836e6a9260 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.165+0000 7f2405600640 1 -- 172.21.15.69:0/1615424398 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f240c031cd0 con 0x55836e6a9260 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.165+0000 7f2425260740 1 -- 172.21.15.69:0/1615424398 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f241400fb00 con 0x55836e6a9260 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.165+0000 7f2405600640 1 -- 172.21.15.69:0/1615424398 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f240c057050 con 0x55836e6a9260 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.165+0000 7f2425260740 10 client.15417 fetch_fsmap learned FSMap version 23 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.165+0000 7f2425260740 1 -- 172.21.15.69:0/1615424398 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55836e6be030 con 0x55836e6a9260 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.165+0000 7f2405600640 1 -- 172.21.15.69:0/1615424398 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f241400fb00 con 0x55836e6a9260 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.165+0000 7f2405600640 1 -- 172.21.15.69:0/1615424398 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f240c03a040 con 0x55836e6a9260 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.166+0000 7f2425260740 10 client.15417 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.166+0000 7f2425260740 10 client.15417 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.166+0000 7f2425260740 20 client.15417 populate_metadata read hostname 'smithi069' 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.166+0000 7f2425260740 1 --2- 172.21.15.69:0/1615424398 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x55836e6c0800 0x55836e6c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.166+0000 7f2425260740 4 client.15417 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:53.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.166+0000 7f2425260740 1 -- 172.21.15.69:0/1615424398 --> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55836e61c530 con 0x55836e6c0800 2024-04-24T09:06:53.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.166+0000 7f241aa00640 1 --2- 172.21.15.69:0/1615424398 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x55836e6c0800 0x55836e6c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.167+0000 7f241aa00640 1 --2- 172.21.15.69:0/1615424398 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x55836e6c0800 0x55836e6c1c70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d863826978944bf server_cookie=ba950e17b54f2784 in_seq=0 out_seq=0 2024-04-24T09:06:53.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.167+0000 7f2405600640 10 client.15417 ms_handle_connect on v2:172.21.15.81:6836/1973819593 2024-04-24T09:06:53.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.169+0000 7f2405600640 1 -- 172.21.15.69:0/1615424398 <== mds.1 v2:172.21.15.81:6836/1973819593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55836e61c530 con 0x55836e6c0800 2024-04-24T09:06:53.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.169+0000 7f2405600640 10 client.15417 handle_command_reply: tid=0 2024-04-24T09:06:53.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.169+0000 7f2425260740 1 client.15417 shutdown 2024-04-24T09:06:53.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.169+0000 7f2425260740 1 -- 172.21.15.69:0/1615424398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 msgr2=0x55836e6b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.169+0000 7f2425260740 1 --2- 172.21.15.69:0/1615424398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6b36d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f241c102fe0 tx=0x7f240c005fd0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.169+0000 7f2425260740 1 -- 172.21.15.69:0/1615424398 shutdown_connections 2024-04-24T09:06:53.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.169+0000 7f2425260740 1 --2- 172.21.15.69:0/1615424398 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x55836e6c0800 0x55836e6c1c70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.169+0000 7f2425260740 1 --2- 172.21.15.69:0/1615424398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6aedc0 0x55836e6bac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.169+0000 7f2425260740 1 --2- 172.21.15.69:0/1615424398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6a9c50 0x55836e6b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.169+0000 7f2425260740 1 --2- 172.21.15.69:0/1615424398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6b36d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.169+0000 7f2405600640 0 client.15417 ms_handle_reset on v2:172.21.15.81:6836/1973819593 2024-04-24T09:06:53.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.169+0000 7f2425260740 1 -- 172.21.15.69:0/1615424398 >> 172.21.15.69:0/1615424398 conn(0x55836e6a48d0 msgr2=0x55836e6a3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:53.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.170+0000 7f2425260740 1 -- 172.21.15.69:0/1615424398 shutdown_connections 2024-04-24T09:06:53.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.170+0000 7f2425260740 1 -- 172.21.15.69:0/1615424398 wait complete. 2024-04-24T09:06:53.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.170+0000 7f2425260740 20 client.15417 trim_cache size 0 max 16384 2024-04-24T09:06:53.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.189+0000 7f2425260740 1 Processor -- start 2024-04-24T09:06:53.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.189+0000 7f2425260740 1 -- start start 2024-04-24T09:06:53.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.189+0000 7f2425260740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.189+0000 7f2425260740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6a9c50 0x55836e6ab630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.189+0000 7f2425260740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6aedc0 0x55836e6bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.189+0000 7f2425260740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55836e6bffb0 con 0x55836e6a9260 2024-04-24T09:06:53.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.189+0000 7f2425260740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55836e6c0680 con 0x55836e6a9c50 2024-04-24T09:06:53.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.189+0000 7f2425260740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55836e6c0d50 con 0x55836e6aedc0 2024-04-24T09:06:53.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.190+0000 7f241b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.190+0000 7f241b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6ab0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57602/0 (socket says 172.21.15.69:57602) 2024-04-24T09:06:53.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.190+0000 7f241b400640 1 -- 172.21.15.69:0/903766446 learned_addr learned my addr 172.21.15.69:0/903766446 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:53.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.190+0000 7f241be00640 1 --2- 172.21.15.69:0/903766446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6aedc0 0x55836e6bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.190+0000 7f241aa00640 1 --2- 172.21.15.69:0/903766446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6a9c50 0x55836e6ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.190+0000 7f241b400640 1 -- 172.21.15.69:0/903766446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6aedc0 msgr2=0x55836e6bac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:53.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.190+0000 7f241b400640 1 --2- 172.21.15.69:0/903766446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6aedc0 0x55836e6bac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.190+0000 7f241b400640 1 -- 172.21.15.69:0/903766446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6a9c50 msgr2=0x55836e6ab630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:53.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.190+0000 7f241b400640 1 --2- 172.21.15.69:0/903766446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6a9c50 0x55836e6ab630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.190+0000 7f241b400640 1 -- 172.21.15.69:0/903766446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55836e6a4700 con 0x55836e6a9260 2024-04-24T09:06:53.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.190+0000 7f241be00640 1 --2- 172.21.15.69:0/903766446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6aedc0 0x55836e6bac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:53.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.190+0000 7f241b400640 1 --2- 172.21.15.69:0/903766446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6ab0f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f240c005ec0 tx=0x7f240c003fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5cfbf29bd0fdb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:53.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.191+0000 7f2407e00640 1 -- 172.21.15.69:0/903766446 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c042070 con 0x55836e6a9260 2024-04-24T09:06:53.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.191+0000 7f2407e00640 1 -- 172.21.15.69:0/903766446 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f240c04e070 con 0x55836e6a9260 2024-04-24T09:06:53.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.191+0000 7f2407e00640 1 -- 172.21.15.69:0/903766446 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c035040 con 0x55836e6a9260 2024-04-24T09:06:53.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.191+0000 7f2425260740 1 -- 172.21.15.69:0/903766446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 msgr2=0x55836e6ab0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.191+0000 7f2425260740 1 --2- 172.21.15.69:0/903766446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6ab0f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f240c005ec0 tx=0x7f240c003fa0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.191+0000 7f2425260740 1 -- 172.21.15.69:0/903766446 shutdown_connections 2024-04-24T09:06:53.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.191+0000 7f2425260740 1 --2- 172.21.15.69:0/903766446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6aedc0 0x55836e6bac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.191+0000 7f2425260740 1 --2- 172.21.15.69:0/903766446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6a9c50 0x55836e6ab630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.191+0000 7f2425260740 1 --2- 172.21.15.69:0/903766446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6ab0f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.191+0000 7f2425260740 1 -- 172.21.15.69:0/903766446 >> 172.21.15.69:0/903766446 conn(0x55836e6a48d0 msgr2=0x55836e6bf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:53.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.191+0000 7f2425260740 1 -- 172.21.15.69:0/903766446 shutdown_connections 2024-04-24T09:06:53.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.191+0000 7f2425260740 1 -- 172.21.15.69:0/903766446 wait complete. 2024-04-24T09:06:53.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.192+0000 7f2425260740 1 -- start start 2024-04-24T09:06:53.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.192+0000 7f2425260740 1 Processor -- start 2024-04-24T09:06:53.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.192+0000 7f2425260740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.192+0000 7f2425260740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6a9c50 0x55836e722790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.192+0000 7f2425260740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6aedc0 0x55836e724ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.192+0000 7f2425260740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55836e6c1af0 con 0x55836e6a9260 2024-04-24T09:06:53.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.192+0000 7f2425260740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55836e6c1420 con 0x55836e6aedc0 2024-04-24T09:06:53.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.192+0000 7f2425260740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55836e6c21c0 con 0x55836e6a9c50 2024-04-24T09:06:53.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.192+0000 7f2425260740 10 client.0 _ll_register_callbacks cb 0x7f240c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:53.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.193+0000 7f241b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.193+0000 7f241b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6b8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57618/0 (socket says 172.21.15.69:57618) 2024-04-24T09:06:53.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.193+0000 7f241b400640 1 -- 172.21.15.69:0/2903557100 learned_addr learned my addr 172.21.15.69:0/2903557100 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:53.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.193+0000 7f241aa00640 1 --2- 172.21.15.69:0/2903557100 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6a9c50 0x55836e722790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.193+0000 7f241be00640 1 --2- 172.21.15.69:0/2903557100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6aedc0 0x55836e724ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.193+0000 7f241b400640 1 -- 172.21.15.69:0/2903557100 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6a9c50 msgr2=0x55836e722790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:53.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.193+0000 7f241b400640 1 --2- 172.21.15.69:0/2903557100 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6a9c50 0x55836e722790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.193+0000 7f241b400640 1 -- 172.21.15.69:0/2903557100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6aedc0 msgr2=0x55836e724ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:53.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.193+0000 7f241b400640 1 --2- 172.21.15.69:0/2903557100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6aedc0 0x55836e724ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.193+0000 7f241b400640 1 -- 172.21.15.69:0/2903557100 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55836e725510 con 0x55836e6a9260 2024-04-24T09:06:53.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.193+0000 7f241be00640 1 --2- 172.21.15.69:0/2903557100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6aedc0 0x55836e724ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:53.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.193+0000 7f241b400640 1 --2- 172.21.15.69:0/2903557100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6b8c00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f240c005730 tx=0x7f240c03cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcadb7cc6f479557 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:53.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.193+0000 7f2405600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:06:53.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.194+0000 7f2405600640 1 -- 172.21.15.69:0/2903557100 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c042070 con 0x55836e6a9260 2024-04-24T09:06:53.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.194+0000 7f2425260740 1 -- 172.21.15.69:0/2903557100 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f240c04e070 con 0x55836e6a9260 2024-04-24T09:06:53.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.194+0000 7f2405600640 1 -- 172.21.15.69:0/2903557100 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f240c031d60 con 0x55836e6a9260 2024-04-24T09:06:53.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.194+0000 7f2405600640 1 -- 172.21.15.69:0/2903557100 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f240c071050 con 0x55836e6a9260 2024-04-24T09:06:53.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.194+0000 7f2425260740 10 client.15423 fetch_fsmap learned FSMap version 23 2024-04-24T09:06:53.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.194+0000 7f2425260740 1 -- 172.21.15.69:0/2903557100 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55836e726780 con 0x55836e6a9260 2024-04-24T09:06:53.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.194+0000 7f2405600640 1 -- 172.21.15.69:0/2903557100 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f240c04e070 con 0x55836e6a9260 2024-04-24T09:06:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.194+0000 7f2405600640 1 -- 172.21.15.69:0/2903557100 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f240c03a040 con 0x55836e6a9260 2024-04-24T09:06:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.194+0000 7f2425260740 10 client.15423 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T09:06:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.194+0000 7f2425260740 10 client.15423 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-24T09:06:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.194+0000 7f2425260740 20 client.15423 populate_metadata read hostname 'smithi069' 2024-04-24T09:06:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.195+0000 7f2425260740 1 --2- 172.21.15.69:0/2903557100 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x55836e72b000 0x55836e729d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.195+0000 7f2425260740 4 client.15423 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:06:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.195+0000 7f2425260740 1 -- 172.21.15.69:0/2903557100 --> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] -- command(tid 0: {"prefix": "status"}) -- 0x55836e6163a0 con 0x55836e72b000 2024-04-24T09:06:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.195+0000 7f241aa00640 1 --2- 172.21.15.69:0/2903557100 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x55836e72b000 0x55836e729d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.196+0000 7f241aa00640 1 --2- 172.21.15.69:0/2903557100 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x55836e72b000 0x55836e729d10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f40838ffbf8f4674 server_cookie=70d34d5d3dacf181 in_seq=0 out_seq=0 2024-04-24T09:06:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.196+0000 7f2405600640 10 client.15423 ms_handle_connect on v2:172.21.15.81:6836/1973819593 2024-04-24T09:06:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.197+0000 7f2405600640 1 -- 172.21.15.69:0/2903557100 <== mds.1 v2:172.21.15.81:6836/1973819593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55836e6163a0 con 0x55836e72b000 2024-04-24T09:06:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.197+0000 7f2405600640 10 client.15423 handle_command_reply: tid=0 2024-04-24T09:06:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.197+0000 7f2425260740 1 client.15423 shutdown 2024-04-24T09:06:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.197+0000 7f2425260740 1 -- 172.21.15.69:0/2903557100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 msgr2=0x55836e6b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.197+0000 7f2425260740 1 --2- 172.21.15.69:0/2903557100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6b8c00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f240c005730 tx=0x7f240c03cf30 comp rx=0 tx=0).stop 2024-04-24T09:06:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.197+0000 7f2425260740 1 -- 172.21.15.69:0/2903557100 shutdown_connections 2024-04-24T09:06:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.197+0000 7f2425260740 1 --2- 172.21.15.69:0/2903557100 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x55836e72b000 0x55836e729d10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.197+0000 7f2425260740 1 --2- 172.21.15.69:0/2903557100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55836e6aedc0 0x55836e724ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.197+0000 7f2425260740 1 --2- 172.21.15.69:0/2903557100 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55836e6a9c50 0x55836e722790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.197+0000 7f2425260740 1 --2- 172.21.15.69:0/2903557100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55836e6a9260 0x55836e6b8c00 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.197+0000 7f2405600640 0 client.15423 ms_handle_reset on v2:172.21.15.81:6836/1973819593 2024-04-24T09:06:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.197+0000 7f2425260740 1 -- 172.21.15.69:0/2903557100 >> 172.21.15.69:0/2903557100 conn(0x55836e6a48d0 msgr2=0x55836e6ab0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.197+0000 7f2425260740 1 -- 172.21.15.69:0/2903557100 shutdown_connections 2024-04-24T09:06:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.197+0000 7f2425260740 1 -- 172.21.15.69:0/2903557100 wait complete. 2024-04-24T09:06:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.197+0000 7f2425260740 20 client.15423 trim_cache size 0 max 16384 2024-04-24T09:06:53.200 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:06:53.201 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "ef1ec7e8-0217-11ef-bc93-c7b262605968", 2024-04-24T09:06:53.201 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 1, 2024-04-24T09:06:53.201 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 34496, 2024-04-24T09:06:53.201 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-24T09:06:53.201 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-24T09:06:53.201 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-24T09:06:53.201 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 140.60999412499999, 2024-04-24T09:06:53.201 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 23, 2024-04-24T09:06:53.201 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 92, 2024-04-24T09:06:53.201 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 92, 2024-04-24T09:06:53.201 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 191.974865765 2024-04-24T09:06:53.201 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:06:53.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.198+0000 7f2420e00640 1 -- 172.21.15.69:0/1761426445 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f24000a3bb0 msgr2=0x7f24000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.198+0000 7f2420e00640 1 --2- 172.21.15.69:0/1761426445 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f24000a3bb0 0x7f24000a6070 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55836e6aa6c0 tx=0x7f240c0314a0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.198+0000 7f2420e00640 1 -- 172.21.15.69:0/1761426445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f241c103180 msgr2=0x7f241c19a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.198+0000 7f2420e00640 1 --2- 172.21.15.69:0/1761426445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f241c103180 0x7f241c19a8c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f240800d950 tx=0x7f240800de10 comp rx=0 tx=0).stop 2024-04-24T09:06:53.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.198+0000 7f2420e00640 1 -- 172.21.15.69:0/1761426445 shutdown_connections 2024-04-24T09:06:53.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.198+0000 7f2420e00640 1 --2- 172.21.15.69:0/1761426445 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f24000a3bb0 0x7f24000a6070 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.198+0000 7f2420e00640 1 --2- 172.21.15.69:0/1761426445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f241c103b40 0x7f241c1a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.198+0000 7f2420e00640 1 --2- 172.21.15.69:0/1761426445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f241c103180 0x7f241c19a8c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.198+0000 7f2420e00640 1 --2- 172.21.15.69:0/1761426445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f241c101f90 0x7f241c19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.198+0000 7f2420e00640 1 -- 172.21.15.69:0/1761426445 >> 172.21.15.69:0/1761426445 conn(0x7f241c0fd500 msgr2=0x7f241c074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:53.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.198+0000 7f2420e00640 1 -- 172.21.15.69:0/1761426445 shutdown_connections 2024-04-24T09:06:53.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.198+0000 7f2420e00640 1 -- 172.21.15.69:0/1761426445 wait complete. 2024-04-24T09:06:53.214 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T09:06:53.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.393+0000 7f97ed400640 1 -- 172.21.15.69:0/1222368790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8100780 msgr2=0x7f97e8100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.393+0000 7f97ed400640 1 --2- 172.21.15.69:0/1222368790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8100780 0x7f97e8100ba0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009900 tx=0x7f97d802a400 comp rx=0 tx=0).stop 2024-04-24T09:06:53.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.393+0000 7f97ed400640 1 -- 172.21.15.69:0/1222368790 shutdown_connections 2024-04-24T09:06:53.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.393+0000 7f97ed400640 1 --2- 172.21.15.69:0/1222368790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e8101a80 0x7f97e8101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.393+0000 7f97ed400640 1 --2- 172.21.15.69:0/1222368790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97e81010e0 0x7f97e8101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.393+0000 7f97ed400640 1 --2- 172.21.15.69:0/1222368790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8100780 0x7f97e8100ba0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.393+0000 7f97ed400640 1 -- 172.21.15.69:0/1222368790 >> 172.21.15.69:0/1222368790 conn(0x7f97e80fc3a0 msgr2=0x7f97e80fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:53.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.393+0000 7f97ed400640 1 -- 172.21.15.69:0/1222368790 shutdown_connections 2024-04-24T09:06:53.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.393+0000 7f97ed400640 1 -- 172.21.15.69:0/1222368790 wait complete. 2024-04-24T09:06:53.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.395+0000 7f97ed400640 1 Processor -- start 2024-04-24T09:06:53.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.395+0000 7f97ed400640 1 -- start start 2024-04-24T09:06:53.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.395+0000 7f97ed400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97e8100780 0x7f97e819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.395+0000 7f97ed400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e81010e0 0x7f97e819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.395+0000 7f97ed400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8101a80 0x7f97e81a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.395+0000 7f97ed400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f97e810fcb0 con 0x7f97e8101a80 2024-04-24T09:06:53.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.395+0000 7f97ed400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f97e810fb30 con 0x7f97e81010e0 2024-04-24T09:06:53.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.395+0000 7f97ed400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f97e810fe30 con 0x7f97e8100780 2024-04-24T09:06:53.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.396+0000 7f97eca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8101a80 0x7f97e81a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.396+0000 7f97eca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8101a80 0x7f97e81a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57642/0 (socket says 172.21.15.69:57642) 2024-04-24T09:06:53.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.396+0000 7f97eca00640 1 -- 172.21.15.69:0/1783760911 learned_addr learned my addr 172.21.15.69:0/1783760911 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:53.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.396+0000 7f97e7e00640 1 --2- 172.21.15.69:0/1783760911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97e8100780 0x7f97e819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.396+0000 7f97e7400640 1 --2- 172.21.15.69:0/1783760911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e81010e0 0x7f97e819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.396+0000 7f97eca00640 1 -- 172.21.15.69:0/1783760911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97e8100780 msgr2=0x7f97e819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.396+0000 7f97eca00640 1 --2- 172.21.15.69:0/1783760911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97e8100780 0x7f97e819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.396+0000 7f97eca00640 1 -- 172.21.15.69:0/1783760911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e81010e0 msgr2=0x7f97e819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.396+0000 7f97eca00640 1 --2- 172.21.15.69:0/1783760911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e81010e0 0x7f97e819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.396+0000 7f97eca00640 1 -- 172.21.15.69:0/1783760911 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e81a1e50 con 0x7f97e8101a80 2024-04-24T09:06:53.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.396+0000 7f97e7400640 1 --2- 172.21.15.69:0/1783760911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e81010e0 0x7f97e819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:53.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.396+0000 7f97eca00640 1 --2- 172.21.15.69:0/1783760911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8101a80 0x7f97e81a17e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f97e000cce0 tx=0x7f97e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:53.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.397+0000 7f97e4c00640 1 -- 172.21.15.69:0/1783760911 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e000f450 con 0x7f97e8101a80 2024-04-24T09:06:53.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.397+0000 7f97e4c00640 1 -- 172.21.15.69:0/1783760911 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97e000faa0 con 0x7f97e8101a80 2024-04-24T09:06:53.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.397+0000 7f97e4c00640 1 -- 172.21.15.69:0/1783760911 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0018b70 con 0x7f97e8101a80 2024-04-24T09:06:53.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.397+0000 7f97ed400640 1 -- 172.21.15.69:0/1783760911 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e81a2140 con 0x7f97e8101a80 2024-04-24T09:06:53.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.397+0000 7f97ed400640 1 -- 172.21.15.69:0/1783760911 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e81a2690 con 0x7f97e8101a80 2024-04-24T09:06:53.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.401+0000 7f97f1820740 1 Processor -- start 2024-04-24T09:06:53.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.401+0000 7f97e4c00640 1 -- 172.21.15.69:0/1783760911 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f97e0018d10 con 0x7f97e8101a80 2024-04-24T09:06:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.401+0000 7f97f1820740 1 -- start start 2024-04-24T09:06:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97f1820740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01260 0x5577e3b01680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97f1820740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01c50 0x5577e3b06750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97f1820740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 0x5577e3b09250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97f1820740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5577e3a66b10 con 0x5577e3b01c50 2024-04-24T09:06:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97f1820740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5577e3a0d2b0 con 0x5577e3b01260 2024-04-24T09:06:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97f1820740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5577e3a62ad0 con 0x5577e3b06dc0 2024-04-24T09:06:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97e7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01c50 0x5577e3b06750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97e7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01c50 0x5577e3b06750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57658/0 (socket says 172.21.15.69:57658) 2024-04-24T09:06:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97e7e00640 1 -- 172.21.15.69:0/3732683902 learned_addr learned my addr 172.21.15.69:0/3732683902 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97e7400640 1 --2- 172.21.15.69:0/3732683902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01260 0x5577e3b01680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97e7400640 1 --2- 172.21.15.69:0/3732683902 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 0x5577e3b09250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97e7e00640 1 -- 172.21.15.69:0/3732683902 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 msgr2=0x5577e3b09250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97e7e00640 1 --2- 172.21.15.69:0/3732683902 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 0x5577e3b09250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97e7e00640 1 -- 172.21.15.69:0/3732683902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01260 msgr2=0x5577e3b01680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97e7e00640 1 --2- 172.21.15.69:0/3732683902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01260 0x5577e3b01680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97e7e00640 1 -- 172.21.15.69:0/3732683902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e37792d0 con 0x5577e3b01c50 2024-04-24T09:06:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.402+0000 7f97e7400640 1 --2- 172.21.15.69:0/3732683902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01260 0x5577e3b01680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.403+0000 7f97e7e00640 1 --2- 172.21.15.69:0/3732683902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01c50 0x5577e3b06750 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f97d802a9f0 tx=0x7f97d8004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08aa129b88e38e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.403+0000 7f97dd200640 1 -- 172.21.15.69:0/3732683902 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8035030 con 0x5577e3b01c50 2024-04-24T09:06:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.403+0000 7f97dd200640 1 -- 172.21.15.69:0/3732683902 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d8003ca0 con 0x5577e3b01c50 2024-04-24T09:06:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.403+0000 7f97dd200640 1 -- 172.21.15.69:0/3732683902 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8031cd0 con 0x5577e3b01c50 2024-04-24T09:06:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.403+0000 7f97e4c00640 1 --2- 172.21.15.69:0/1783760911 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f97cc0a3c30 0x7f97cc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.404+0000 7f97f1820740 1 -- 172.21.15.69:0/3732683902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01c50 msgr2=0x5577e3b06750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.404+0000 7f97e4c00640 1 -- 172.21.15.69:0/1783760911 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f97e00d0c20 con 0x7f97e8101a80 2024-04-24T09:06:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.404+0000 7f97f1820740 1 --2- 172.21.15.69:0/3732683902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01c50 0x5577e3b06750 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f97d802a9f0 tx=0x7f97d8004390 comp rx=0 tx=0).stop 2024-04-24T09:06:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.404+0000 7f97e7400640 1 --2- 172.21.15.69:0/1783760911 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f97cc0a3c30 0x7f97cc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.404+0000 7f97f1820740 1 -- 172.21.15.69:0/3732683902 shutdown_connections 2024-04-24T09:06:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.404+0000 7f97f1820740 1 --2- 172.21.15.69:0/3732683902 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 0x5577e3b09250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.404+0000 7f97f1820740 1 --2- 172.21.15.69:0/3732683902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01c50 0x5577e3b06750 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.404+0000 7f97f1820740 1 --2- 172.21.15.69:0/3732683902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01260 0x5577e3b01680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.404+0000 7f97f1820740 1 -- 172.21.15.69:0/3732683902 >> 172.21.15.69:0/3732683902 conn(0x5577e3afc8d0 msgr2=0x5577e3afccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.404+0000 7f97f1820740 1 -- 172.21.15.69:0/3732683902 shutdown_connections 2024-04-24T09:06:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.404+0000 7f97f1820740 1 -- 172.21.15.69:0/3732683902 wait complete. 2024-04-24T09:06:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.404+0000 7f97e7400640 1 --2- 172.21.15.69:0/1783760911 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f97cc0a3c30 0x7f97cc0a60f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5577e3b024b0 tx=0x7f97d4005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.405+0000 7f97f1820740 1 -- start start 2024-04-24T09:06:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.405+0000 7f97f1820740 1 Processor -- start 2024-04-24T09:06:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.405+0000 7f97f1820740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01260 0x5577e3b0b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.405+0000 7f97f1820740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01c50 0x5577e3b0bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.405+0000 7f97f1820740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 0x5577e3b12ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.405+0000 7f97f1820740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5577e3b09d10 con 0x5577e3b01260 2024-04-24T09:06:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.405+0000 7f97f1820740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5577e3a20590 con 0x5577e3b01c50 2024-04-24T09:06:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.405+0000 7f97f1820740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5577e3b09e90 con 0x5577e3b06dc0 2024-04-24T09:06:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.405+0000 7f97f1820740 10 client.0 _ll_register_callbacks cb 0x5577e3a033a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.406+0000 7f97e7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01260 0x5577e3b0b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.406+0000 7f97e7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01260 0x5577e3b0b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57670/0 (socket says 172.21.15.69:57670) 2024-04-24T09:06:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.406+0000 7f97e7e00640 1 -- 172.21.15.69:0/190791035 learned_addr learned my addr 172.21.15.69:0/190791035 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.406+0000 7f97eca00640 1 --2- 172.21.15.69:0/190791035 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 0x5577e3b12ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.406+0000 7f97e7e00640 1 -- 172.21.15.69:0/190791035 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 msgr2=0x5577e3b12ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.406+0000 7f97e7e00640 1 --2- 172.21.15.69:0/190791035 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 0x5577e3b12ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.406+0000 7f97e7e00640 1 -- 172.21.15.69:0/190791035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01c50 msgr2=0x5577e3b0bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.406+0000 7f97e7e00640 1 --2- 172.21.15.69:0/190791035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01c50 0x5577e3b0bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.406+0000 7f97e7e00640 1 -- 172.21.15.69:0/190791035 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e38436b0 con 0x5577e3b01260 2024-04-24T09:06:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.406+0000 7f97e7e00640 1 --2- 172.21.15.69:0/190791035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01260 0x5577e3b0b620 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f97d802a920 tx=0x7f97d80317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4615b3655d3b569 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.406+0000 7f97d1600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:06:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.406+0000 7f97d1600640 1 -- 172.21.15.69:0/190791035 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8035040 con 0x5577e3b01260 2024-04-24T09:06:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.407+0000 7f97d1600640 1 -- 172.21.15.69:0/190791035 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f97d80035f0 con 0x5577e3b01260 2024-04-24T09:06:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.407+0000 7f97f1820740 1 -- 172.21.15.69:0/190791035 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d8003ca0 con 0x5577e3b01260 2024-04-24T09:06:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.407+0000 7f97d1600640 1 -- 172.21.15.69:0/190791035 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f97d8031e60 con 0x5577e3b01260 2024-04-24T09:06:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.407+0000 7f97f1820740 10 client.15435 fetch_fsmap learned FSMap version 23 2024-04-24T09:06:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.407+0000 7f97f1820740 1 -- 172.21.15.69:0/190791035 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5577e3b15e80 con 0x5577e3b01260 2024-04-24T09:06:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.407+0000 7f97d1600640 1 -- 172.21.15.69:0/190791035 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d8003ca0 con 0x5577e3b01260 2024-04-24T09:06:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.407+0000 7f97d1600640 1 -- 172.21.15.69:0/190791035 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f97d804e400 con 0x5577e3b01260 2024-04-24T09:06:53.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.407+0000 7f97f1820740 10 client.15435 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T09:06:53.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.407+0000 7f97f1820740 10 client.15435 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T09:06:53.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.407+0000 7f97f1820740 20 client.15435 populate_metadata read hostname 'smithi069' 2024-04-24T09:06:53.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.407+0000 7f97f1820740 1 --2- 172.21.15.69:0/190791035 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5577e3b180f0 0x5577e3b19560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.407+0000 7f97f1820740 4 client.15435 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:53.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.407+0000 7f97f1820740 1 -- 172.21.15.69:0/190791035 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577e3a74530 con 0x5577e3b180f0 2024-04-24T09:06:53.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.408+0000 7f97e7400640 1 --2- 172.21.15.69:0/190791035 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5577e3b180f0 0x5577e3b19560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.409+0000 7f97e7400640 1 --2- 172.21.15.69:0/190791035 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5577e3b180f0 0x5577e3b19560 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbec5b6ec51f090a server_cookie=97be1deba646d16a in_seq=0 out_seq=0 2024-04-24T09:06:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.409+0000 7f97d1600640 10 client.15435 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:06:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.411+0000 7f97d1600640 1 -- 172.21.15.69:0/190791035 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5577e3a74530 con 0x5577e3b180f0 2024-04-24T09:06:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.411+0000 7f97d1600640 10 client.15435 handle_command_reply: tid=0 2024-04-24T09:06:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.411+0000 7f97f1820740 1 client.15435 shutdown 2024-04-24T09:06:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.411+0000 7f97f1820740 1 -- 172.21.15.69:0/190791035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01260 msgr2=0x5577e3b0b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.411+0000 7f97f1820740 1 --2- 172.21.15.69:0/190791035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01260 0x5577e3b0b620 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f97d802a920 tx=0x7f97d80317b0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.411+0000 7f97f1820740 1 -- 172.21.15.69:0/190791035 shutdown_connections 2024-04-24T09:06:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.411+0000 7f97f1820740 1 --2- 172.21.15.69:0/190791035 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5577e3b180f0 0x5577e3b19560 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.411+0000 7f97f1820740 1 --2- 172.21.15.69:0/190791035 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 0x5577e3b12ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.411+0000 7f97f1820740 1 --2- 172.21.15.69:0/190791035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01c50 0x5577e3b0bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.411+0000 7f97f1820740 1 --2- 172.21.15.69:0/190791035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01260 0x5577e3b0b620 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.411+0000 7f97f1820740 1 -- 172.21.15.69:0/190791035 >> 172.21.15.69:0/190791035 conn(0x5577e3afc8d0 msgr2=0x5577e3afb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.411+0000 7f97d1600640 0 client.15435 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:06:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.411+0000 7f97f1820740 1 -- 172.21.15.69:0/190791035 shutdown_connections 2024-04-24T09:06:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.411+0000 7f97f1820740 1 -- 172.21.15.69:0/190791035 wait complete. 2024-04-24T09:06:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.411+0000 7f97f1820740 20 client.15435 trim_cache size 0 max 16384 2024-04-24T09:06:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.431+0000 7f97f1820740 1 Processor -- start 2024-04-24T09:06:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.431+0000 7f97f1820740 1 -- start start 2024-04-24T09:06:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.431+0000 7f97f1820740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01260 0x5577e3b09c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.431+0000 7f97f1820740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01c50 0x5577e3afccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.431+0000 7f97f1820740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 0x5577e3b024e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.431+0000 7f97f1820740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5577e3b178a0 con 0x5577e3b01c50 2024-04-24T09:06:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.431+0000 7f97f1820740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5577e3b17f70 con 0x5577e3b01260 2024-04-24T09:06:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.431+0000 7f97f1820740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5577e3b18640 con 0x5577e3b06dc0 2024-04-24T09:06:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.431+0000 7f97e7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01c50 0x5577e3afccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.431+0000 7f97e7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01c50 0x5577e3afccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57674/0 (socket says 172.21.15.69:57674) 2024-04-24T09:06:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.431+0000 7f97e7400640 1 -- 172.21.15.69:0/2377675110 learned_addr learned my addr 172.21.15.69:0/2377675110 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.431+0000 7f97e7400640 1 -- 172.21.15.69:0/2377675110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 msgr2=0x5577e3b024e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.431+0000 7f97e7400640 1 --2- 172.21.15.69:0/2377675110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 0x5577e3b024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.431+0000 7f97e7400640 1 -- 172.21.15.69:0/2377675110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01260 msgr2=0x5577e3b09c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.431+0000 7f97e7400640 1 --2- 172.21.15.69:0/2377675110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01260 0x5577e3b09c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.432+0000 7f97e7400640 1 -- 172.21.15.69:0/2377675110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e3b02030 con 0x5577e3b01c50 2024-04-24T09:06:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.432+0000 7f97e7400640 1 --2- 172.21.15.69:0/2377675110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01c50 0x5577e3afccb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f97d4000c00 tx=0x7f97d4012070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785f241e7afdd008 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:53.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.432+0000 7f97d3e00640 1 -- 172.21.15.69:0/2377675110 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d4010b30 con 0x5577e3b01c50 2024-04-24T09:06:53.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.432+0000 7f97d3e00640 1 -- 172.21.15.69:0/2377675110 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d40054d0 con 0x5577e3b01c50 2024-04-24T09:06:53.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.432+0000 7f97d3e00640 1 -- 172.21.15.69:0/2377675110 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d4004d10 con 0x5577e3b01c50 2024-04-24T09:06:53.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.433+0000 7f97f1820740 1 -- 172.21.15.69:0/2377675110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01c50 msgr2=0x5577e3afccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.433+0000 7f97f1820740 1 --2- 172.21.15.69:0/2377675110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01c50 0x5577e3afccb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f97d4000c00 tx=0x7f97d4012070 comp rx=0 tx=0).stop 2024-04-24T09:06:53.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.433+0000 7f97f1820740 1 -- 172.21.15.69:0/2377675110 shutdown_connections 2024-04-24T09:06:53.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.433+0000 7f97f1820740 1 --2- 172.21.15.69:0/2377675110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 0x5577e3b024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.433+0000 7f97f1820740 1 --2- 172.21.15.69:0/2377675110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01c50 0x5577e3afccb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.433+0000 7f97f1820740 1 --2- 172.21.15.69:0/2377675110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01260 0x5577e3b09c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.433+0000 7f97f1820740 1 -- 172.21.15.69:0/2377675110 >> 172.21.15.69:0/2377675110 conn(0x5577e3afc8d0 msgr2=0x5577e3b06680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:53.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.433+0000 7f97f1820740 1 -- 172.21.15.69:0/2377675110 shutdown_connections 2024-04-24T09:06:53.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.433+0000 7f97f1820740 1 -- 172.21.15.69:0/2377675110 wait complete. 2024-04-24T09:06:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.433+0000 7f97f1820740 1 -- start start 2024-04-24T09:06:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.434+0000 7f97f1820740 1 Processor -- start 2024-04-24T09:06:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.434+0000 7f97f1820740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01260 0x5577e3b10ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.434+0000 7f97f1820740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01c50 0x5577e3b10ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.434+0000 7f97f1820740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 0x5577e3b11530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.434+0000 7f97f1820740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5577e3b193e0 con 0x5577e3b01260 2024-04-24T09:06:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.434+0000 7f97f1820740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5577e3b18d10 con 0x5577e3b01c50 2024-04-24T09:06:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.434+0000 7f97f1820740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5577e3b19ab0 con 0x5577e3b06dc0 2024-04-24T09:06:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.434+0000 7f97f1820740 10 client.0 _ll_register_callbacks cb 0x7f97d8005d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.434+0000 7f97e7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01260 0x5577e3b10ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.434+0000 7f97e7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01260 0x5577e3b10ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57686/0 (socket says 172.21.15.69:57686) 2024-04-24T09:06:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.434+0000 7f97e7e00640 1 -- 172.21.15.69:0/2743246759 learned_addr learned my addr 172.21.15.69:0/2743246759 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.435+0000 7f97e7400640 1 --2- 172.21.15.69:0/2743246759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01c50 0x5577e3b10ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.435+0000 7f97e7e00640 1 -- 172.21.15.69:0/2743246759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 msgr2=0x5577e3b11530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:53.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.435+0000 7f97e7e00640 1 --2- 172.21.15.69:0/2743246759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 0x5577e3b11530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.435+0000 7f97e7e00640 1 -- 172.21.15.69:0/2743246759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01c50 msgr2=0x5577e3b10ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:53.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.435+0000 7f97e7e00640 1 --2- 172.21.15.69:0/2743246759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01c50 0x5577e3b10ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.435+0000 7f97e7e00640 1 -- 172.21.15.69:0/2743246759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e3b11ba0 con 0x5577e3b01260 2024-04-24T09:06:53.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.435+0000 7f97e7400640 1 --2- 172.21.15.69:0/2743246759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01c50 0x5577e3b10ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:53.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.435+0000 7f97e7e00640 1 --2- 172.21.15.69:0/2743246759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01260 0x5577e3b10ab0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f97d804e9c0 tx=0x7f97d8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3318c2ca70d6a5bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:53.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.435+0000 7f97d1600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:06:53.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.435+0000 7f97d1600640 1 -- 172.21.15.69:0/2743246759 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8035070 con 0x5577e3b01260 2024-04-24T09:06:53.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.435+0000 7f97d1600640 1 -- 172.21.15.69:0/2743246759 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f97d80035f0 con 0x5577e3b01260 2024-04-24T09:06:53.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.435+0000 7f97f1820740 1 -- 172.21.15.69:0/2743246759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d40054d0 con 0x5577e3b01260 2024-04-24T09:06:53.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.436+0000 7f97d1600640 1 -- 172.21.15.69:0/2743246759 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f97d804ec50 con 0x5577e3b01260 2024-04-24T09:06:53.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.436+0000 7f97f1820740 10 client.15441 fetch_fsmap learned FSMap version 23 2024-04-24T09:06:53.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.436+0000 7f97f1820740 1 -- 172.21.15.69:0/2743246759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5577e3b7e170 con 0x5577e3b01260 2024-04-24T09:06:53.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.436+0000 7f97d1600640 1 -- 172.21.15.69:0/2743246759 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d40054d0 con 0x5577e3b01260 2024-04-24T09:06:53.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.436+0000 7f97d1600640 1 -- 172.21.15.69:0/2743246759 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f97d803c7b0 con 0x5577e3b01260 2024-04-24T09:06:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.436+0000 7f97f1820740 10 client.15441 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T09:06:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.436+0000 7f97f1820740 10 client.15441 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T09:06:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.436+0000 7f97f1820740 20 client.15441 populate_metadata read hostname 'smithi069' 2024-04-24T09:06:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.436+0000 7f97f1820740 1 --2- 172.21.15.69:0/2743246759 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5577e3b82bb0 0x5577e3b818c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.436+0000 7f97f1820740 4 client.15441 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:06:53.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.436+0000 7f97f1820740 1 -- 172.21.15.69:0/2743246759 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "status"}) -- 0x5577e3a6e3a0 con 0x5577e3b82bb0 2024-04-24T09:06:53.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.437+0000 7f97e7400640 1 --2- 172.21.15.69:0/2743246759 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5577e3b82bb0 0x5577e3b818c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.438+0000 7f97d1600640 10 client.15441 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:06:53.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.438+0000 7f97e7400640 1 --2- 172.21.15.69:0/2743246759 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5577e3b82bb0 0x5577e3b818c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c376e19dd5d42903 server_cookie=1fa0d8597334b877 in_seq=0 out_seq=0 2024-04-24T09:06:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.438+0000 7f97d1600640 1 -- 172.21.15.69:0/2743246759 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5577e3a6e3a0 con 0x5577e3b82bb0 2024-04-24T09:06:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.438+0000 7f97d1600640 10 client.15441 handle_command_reply: tid=0 2024-04-24T09:06:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.438+0000 7f97f1820740 1 client.15441 shutdown 2024-04-24T09:06:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.438+0000 7f97f1820740 1 -- 172.21.15.69:0/2743246759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01260 msgr2=0x5577e3b10ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.438+0000 7f97f1820740 1 --2- 172.21.15.69:0/2743246759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01260 0x5577e3b10ab0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f97d804e9c0 tx=0x7f97d8001e50 comp rx=0 tx=0).stop 2024-04-24T09:06:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.438+0000 7f97f1820740 1 -- 172.21.15.69:0/2743246759 shutdown_connections 2024-04-24T09:06:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.438+0000 7f97f1820740 1 --2- 172.21.15.69:0/2743246759 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5577e3b82bb0 0x5577e3b818c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.438+0000 7f97f1820740 1 --2- 172.21.15.69:0/2743246759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5577e3b06dc0 0x5577e3b11530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.438+0000 7f97f1820740 1 --2- 172.21.15.69:0/2743246759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577e3b01c50 0x5577e3b10ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.438+0000 7f97f1820740 1 --2- 172.21.15.69:0/2743246759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577e3b01260 0x5577e3b10ab0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.438+0000 7f97d1600640 0 client.15441 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:06:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.438+0000 7f97f1820740 1 -- 172.21.15.69:0/2743246759 >> 172.21.15.69:0/2743246759 conn(0x5577e3afc8d0 msgr2=0x5577e3afccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:53.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.438+0000 7f97f1820740 1 -- 172.21.15.69:0/2743246759 shutdown_connections 2024-04-24T09:06:53.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.438+0000 7f97f1820740 1 -- 172.21.15.69:0/2743246759 wait complete. 2024-04-24T09:06:53.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.439+0000 7f97f1820740 20 client.15441 trim_cache size 0 max 16384 2024-04-24T09:06:53.442 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:06:53.442 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "ef1ec7e8-0217-11ef-bc93-c7b262605968", 2024-04-24T09:06:53.442 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-04-24T09:06:53.442 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 34532, 2024-04-24T09:06:53.442 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-24T09:06:53.442 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-24T09:06:53.442 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-24T09:06:53.442 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 170.901630633, 2024-04-24T09:06:53.442 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 23, 2024-04-24T09:06:53.443 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 92, 2024-04-24T09:06:53.443 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 92, 2024-04-24T09:06:53.443 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 175.96979970999999 2024-04-24T09:06:53.443 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:06:53.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.439+0000 7f97ed400640 1 -- 172.21.15.69:0/1783760911 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f97cc0a3c30 msgr2=0x7f97cc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.439+0000 7f97ed400640 1 --2- 172.21.15.69:0/1783760911 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f97cc0a3c30 0x7f97cc0a60f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5577e3b024b0 tx=0x7f97d4005ce0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.439+0000 7f97ed400640 1 -- 172.21.15.69:0/1783760911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8101a80 msgr2=0x7f97e81a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.439+0000 7f97ed400640 1 --2- 172.21.15.69:0/1783760911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8101a80 0x7f97e81a17e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f97e000cce0 tx=0x7f97e000a620 comp rx=0 tx=0).stop 2024-04-24T09:06:53.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.439+0000 7f97ed400640 1 -- 172.21.15.69:0/1783760911 shutdown_connections 2024-04-24T09:06:53.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.439+0000 7f97ed400640 1 --2- 172.21.15.69:0/1783760911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97e8101a80 0x7f97e81a17e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.439+0000 7f97ed400640 1 --2- 172.21.15.69:0/1783760911 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f97cc0a3c30 0x7f97cc0a60f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.439+0000 7f97ed400640 1 --2- 172.21.15.69:0/1783760911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e81010e0 0x7f97e819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.439+0000 7f97ed400640 1 --2- 172.21.15.69:0/1783760911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97e8100780 0x7f97e819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.439+0000 7f97ed400640 1 -- 172.21.15.69:0/1783760911 >> 172.21.15.69:0/1783760911 conn(0x7f97e80fc3a0 msgr2=0x7f97e80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:53.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.439+0000 7f97ed400640 1 -- 172.21.15.69:0/1783760911 shutdown_connections 2024-04-24T09:06:53.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.439+0000 7f97ed400640 1 -- 172.21.15.69:0/1783760911 wait complete. 2024-04-24T09:06:53.455 INFO:teuthology.run_tasks:Running task sequential... 2024-04-24T09:06:53.464 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-24T09:06:53.464 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-24T09:06:53.465 INFO:teuthology.task.print:Enabling mgr modules 2024-04-24T09:06:53.465 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-24T09:06:53.466 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T09:06:53.466 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-04-24T09:06:53.466 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-24T09:06:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.639+0000 7f48dfa00640 1 -- 172.21.15.69:0/996178124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d80749e0 msgr2=0x7f48d8074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.639+0000 7f48dfa00640 1 --2- 172.21.15.69:0/996178124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d80749e0 0x7f48d8074e40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009900 tx=0x7f48c402a480 comp rx=0 tx=0).stop 2024-04-24T09:06:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.639+0000 7f48dfa00640 1 -- 172.21.15.69:0/996178124 shutdown_connections 2024-04-24T09:06:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.639+0000 7f48dfa00640 1 --2- 172.21.15.69:0/996178124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8109f10 0x7f48d810c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.639+0000 7f48dfa00640 1 --2- 172.21.15.69:0/996178124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d80749e0 0x7f48d8074e40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.639+0000 7f48dfa00640 1 --2- 172.21.15.69:0/996178124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48d8075fc0 0x7f48d8074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.639+0000 7f48dfa00640 1 -- 172.21.15.69:0/996178124 >> 172.21.15.69:0/996178124 conn(0x7f48d80fd5f0 msgr2=0x7f48d80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.639+0000 7f48dfa00640 1 -- 172.21.15.69:0/996178124 shutdown_connections 2024-04-24T09:06:53.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.639+0000 7f48dfa00640 1 -- 172.21.15.69:0/996178124 wait complete. 2024-04-24T09:06:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.640+0000 7f48dfa00640 1 Processor -- start 2024-04-24T09:06:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.640+0000 7f48dfa00640 1 -- start start 2024-04-24T09:06:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.641+0000 7f48dfa00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d80749e0 0x7f48d819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.641+0000 7f48dfa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48d8075fc0 0x7f48d819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.641+0000 7f48dfa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d8109f10 0x7f48d81a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.641+0000 7f48dfa00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f48d810fe00 con 0x7f48d8109f10 2024-04-24T09:06:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.641+0000 7f48dfa00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f48d810fc80 con 0x7f48d80749e0 2024-04-24T09:06:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.641+0000 7f48dfa00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f48d810ff80 con 0x7f48d8075fc0 2024-04-24T09:06:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.641+0000 7f48dda00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d8109f10 0x7f48d81a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.641+0000 7f48dda00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d8109f10 0x7f48d81a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57708/0 (socket says 172.21.15.69:57708) 2024-04-24T09:06:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.641+0000 7f48dda00640 1 -- 172.21.15.69:0/1750267540 learned_addr learned my addr 172.21.15.69:0/1750267540 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.641+0000 7f48dd000640 1 --2- 172.21.15.69:0/1750267540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d80749e0 0x7f48d819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.641+0000 7f48d7e00640 1 --2- 172.21.15.69:0/1750267540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48d8075fc0 0x7f48d819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.641+0000 7f48dda00640 1 -- 172.21.15.69:0/1750267540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48d8075fc0 msgr2=0x7f48d819a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.641+0000 7f48dda00640 1 --2- 172.21.15.69:0/1750267540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48d8075fc0 0x7f48d819a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.641+0000 7f48dda00640 1 -- 172.21.15.69:0/1750267540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d80749e0 msgr2=0x7f48d819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.641+0000 7f48dda00640 1 --2- 172.21.15.69:0/1750267540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d80749e0 0x7f48d819a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.642+0000 7f48dda00640 1 -- 172.21.15.69:0/1750267540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d81a1f40 con 0x7f48d8109f10 2024-04-24T09:06:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.642+0000 7f48dd000640 1 --2- 172.21.15.69:0/1750267540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d80749e0 0x7f48d819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.642+0000 7f48dda00640 1 --2- 172.21.15.69:0/1750267540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d8109f10 0x7f48d81a18d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f48d000cce0 tx=0x7f48d000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.642+0000 7f48d5600640 1 -- 172.21.15.69:0/1750267540 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d000f450 con 0x7f48d8109f10 2024-04-24T09:06:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.642+0000 7f48d5600640 1 -- 172.21.15.69:0/1750267540 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48d000faa0 con 0x7f48d8109f10 2024-04-24T09:06:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.642+0000 7f48dfa00640 1 -- 172.21.15.69:0/1750267540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d81a2230 con 0x7f48d8109f10 2024-04-24T09:06:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.642+0000 7f48d5600640 1 -- 172.21.15.69:0/1750267540 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d0018b70 con 0x7f48d8109f10 2024-04-24T09:06:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.642+0000 7f48dfa00640 1 -- 172.21.15.69:0/1750267540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d81a2700 con 0x7f48d8109f10 2024-04-24T09:06:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.643+0000 7f48dfa00640 1 -- 172.21.15.69:0/1750267540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d8108cd0 con 0x7f48d8109f10 2024-04-24T09:06:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.647+0000 7f48d5600640 1 -- 172.21.15.69:0/1750267540 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f48d0018d10 con 0x7f48d8109f10 2024-04-24T09:06:53.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.648+0000 7f48d5600640 1 --2- 172.21.15.69:0/1750267540 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f48c00a3c50 0x7f48c00a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:53.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.648+0000 7f48d5600640 1 -- 172.21.15.69:0/1750267540 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f48d00d0c30 con 0x7f48d8109f10 2024-04-24T09:06:53.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.649+0000 7f48dd000640 1 --2- 172.21.15.69:0/1750267540 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f48c00a3c50 0x7f48c00a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.649+0000 7f48dd000640 1 --2- 172.21.15.69:0/1750267540 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f48c00a3c50 0x7f48c00a6110 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0058c0 tx=0x7f48cc005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:53.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.653+0000 7f48d5600640 1 -- 172.21.15.69:0/1750267540 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48d0014030 con 0x7f48d8109f10 2024-04-24T09:06:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:53 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1821219023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:53.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:53 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1821219023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:53.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:53.918+0000 7f48dfa00640 1 -- 172.21.15.69:0/1750267540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f48d8062c00 con 0x7f48d8109f10 2024-04-24T09:06:53.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:53 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1821219023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:06:54.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.565+0000 7f48d5600640 1 -- 172.21.15.69:0/1750267540 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f48d009d2b0 con 0x7f48d8109f10 2024-04-24T09:06:54.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.569+0000 7f48dfa00640 1 -- 172.21.15.69:0/1750267540 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f48c00a3c50 msgr2=0x7f48c00a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:54.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.569+0000 7f48dfa00640 1 --2- 172.21.15.69:0/1750267540 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f48c00a3c50 0x7f48c00a6110 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0058c0 tx=0x7f48cc005ec0 comp rx=0 tx=0).stop 2024-04-24T09:06:54.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.569+0000 7f48dfa00640 1 -- 172.21.15.69:0/1750267540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d8109f10 msgr2=0x7f48d81a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.569+0000 7f48dfa00640 1 --2- 172.21.15.69:0/1750267540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d8109f10 0x7f48d81a18d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f48d000cce0 tx=0x7f48d000a620 comp rx=0 tx=0).stop 2024-04-24T09:06:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.569+0000 7f48dfa00640 1 -- 172.21.15.69:0/1750267540 shutdown_connections 2024-04-24T09:06:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.569+0000 7f48dfa00640 1 --2- 172.21.15.69:0/1750267540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d8109f10 0x7f48d81a18d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.569+0000 7f48dfa00640 1 --2- 172.21.15.69:0/1750267540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48d8075fc0 0x7f48d819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.569+0000 7f48dfa00640 1 --2- 172.21.15.69:0/1750267540 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f48c00a3c50 0x7f48c00a6110 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.569+0000 7f48dfa00640 1 --2- 172.21.15.69:0/1750267540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d80749e0 0x7f48d819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:54.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.569+0000 7f48dfa00640 1 -- 172.21.15.69:0/1750267540 >> 172.21.15.69:0/1750267540 conn(0x7f48d80fd5f0 msgr2=0x7f48d8105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:54.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.569+0000 7f48dfa00640 1 -- 172.21.15.69:0/1750267540 shutdown_connections 2024-04-24T09:06:54.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.569+0000 7f48dfa00640 1 -- 172.21.15.69:0/1750267540 wait complete. 2024-04-24T09:06:54.598 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-24T09:06:54.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.777+0000 7f3577200640 1 -- 172.21.15.69:0/2910000480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3578102900 msgr2=0x7f3578104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:54.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.777+0000 7f3577200640 1 --2- 172.21.15.69:0/2910000480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3578102900 0x7f3578104cf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3568009900 tx=0x7f356802a410 comp rx=0 tx=0).stop 2024-04-24T09:06:54.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.777+0000 7f3577200640 1 -- 172.21.15.69:0/2910000480 shutdown_connections 2024-04-24T09:06:54.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.777+0000 7f3577200640 1 --2- 172.21.15.69:0/2910000480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3578107b60 0x7f3578109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:54.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.777+0000 7f3577200640 1 --2- 172.21.15.69:0/2910000480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3578105230 0x7f3578107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:54.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.777+0000 7f3577200640 1 --2- 172.21.15.69:0/2910000480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3578102900 0x7f3578104cf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:54.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.777+0000 7f3577200640 1 -- 172.21.15.69:0/2910000480 >> 172.21.15.69:0/2910000480 conn(0x7f35780fc2f0 msgr2=0x7f35780fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:54.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.778+0000 7f3577200640 1 -- 172.21.15.69:0/2910000480 shutdown_connections 2024-04-24T09:06:54.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.778+0000 7f3577200640 1 -- 172.21.15.69:0/2910000480 wait complete. 2024-04-24T09:06:54.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.779+0000 7f3577200640 1 Processor -- start 2024-04-24T09:06:54.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.779+0000 7f3577200640 1 -- start start 2024-04-24T09:06:54.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.779+0000 7f3577200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3578102900 0x7f3578198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.779+0000 7f3577200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3578105230 0x7f3578198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.779+0000 7f3577200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3578107b60 0x7f357819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.779+0000 7f3577200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f357810da30 con 0x7f3578107b60 2024-04-24T09:06:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.779+0000 7f3577200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f357810d8b0 con 0x7f3578102900 2024-04-24T09:06:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.779+0000 7f3577200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f357810dbb0 con 0x7f3578105230 2024-04-24T09:06:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.779+0000 7f3576800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3578107b60 0x7f357819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.779+0000 7f3576800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3578107b60 0x7f357819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57744/0 (socket says 172.21.15.69:57744) 2024-04-24T09:06:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.779+0000 7f3576800640 1 -- 172.21.15.69:0/3449240628 learned_addr learned my addr 172.21.15.69:0/3449240628 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.779+0000 7f3575e00640 1 --2- 172.21.15.69:0/3449240628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3578102900 0x7f3578198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.780+0000 7f356fe00640 1 --2- 172.21.15.69:0/3449240628 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3578105230 0x7f3578198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.780+0000 7f3576800640 1 -- 172.21.15.69:0/3449240628 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3578105230 msgr2=0x7f3578198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.780+0000 7f3576800640 1 --2- 172.21.15.69:0/3449240628 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3578105230 0x7f3578198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.780+0000 7f3576800640 1 -- 172.21.15.69:0/3449240628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3578102900 msgr2=0x7f3578198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.780+0000 7f3576800640 1 --2- 172.21.15.69:0/3449240628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3578102900 0x7f3578198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:54.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.780+0000 7f3576800640 1 -- 172.21.15.69:0/3449240628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357819fd20 con 0x7f3578107b60 2024-04-24T09:06:54.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.780+0000 7f3576800640 1 --2- 172.21.15.69:0/3449240628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3578107b60 0x7f357819f6b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f357000eb10 tx=0x7f357000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:54.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.780+0000 7f356ea00640 1 -- 172.21.15.69:0/3449240628 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570004b20 con 0x7f3578107b60 2024-04-24T09:06:54.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.781+0000 7f356ea00640 1 -- 172.21.15.69:0/3449240628 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3570004cc0 con 0x7f3578107b60 2024-04-24T09:06:54.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.781+0000 7f356ea00640 1 -- 172.21.15.69:0/3449240628 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570010690 con 0x7f3578107b60 2024-04-24T09:06:54.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.781+0000 7f3577200640 1 -- 172.21.15.69:0/3449240628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f357819ffb0 con 0x7f3578107b60 2024-04-24T09:06:54.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.781+0000 7f3577200640 1 -- 172.21.15.69:0/3449240628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35781a04d0 con 0x7f3578107b60 2024-04-24T09:06:54.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.782+0000 7f3577200640 1 -- 172.21.15.69:0/3449240628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35780fdde0 con 0x7f3578107b60 2024-04-24T09:06:54.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.784+0000 7f356ea00640 1 -- 172.21.15.69:0/3449240628 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f3570005020 con 0x7f3578107b60 2024-04-24T09:06:54.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.786+0000 7f356ea00640 1 --2- 172.21.15.69:0/3449240628 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f355c0a3c70 0x7f355c0a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:54.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.787+0000 7f3575e00640 1 -- 172.21.15.69:0/3449240628 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f355c0a3c70 msgr2=0x7f355c0a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1098677190 2024-04-24T09:06:54.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.787+0000 7f356ea00640 1 -- 172.21.15.69:0/3449240628 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f35700d0d10 con 0x7f3578107b60 2024-04-24T09:06:54.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.787+0000 7f3575e00640 1 --2- 172.21.15.69:0/3449240628 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f355c0a3c70 0x7f355c0a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T09:06:54.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.791+0000 7f356ea00640 1 -- 172.21.15.69:0/3449240628 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35700d6020 con 0x7f3578107b60 2024-04-24T09:06:54.884 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:54 smithi130 ceph-mon[31553]: pgmap v457: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:06:54.884 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:54 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1750267540' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T09:06:54.884 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:06:54 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: ignoring --setuser ceph since I am not root 2024-04-24T09:06:54.885 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:06:54 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: ignoring --setgroup ceph since I am not root 2024-04-24T09:06:54.885 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:06:54 smithi130 ceph-mgr[33571]: -- 172.21.15.130:0/3818213792 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dd32b825a0 con 0x55dd32b57000 2024-04-24T09:06:54.885 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:06:54 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:06:54.882+0000 7f8e81ba5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T09:06:54.885 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:54 smithi081 ceph-mon[30961]: pgmap v457: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:06:54.885 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:54 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1750267540' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T09:06:54.885 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:06:54 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: ignoring --setuser ceph since I am not root 2024-04-24T09:06:54.885 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:06:54 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: ignoring --setgroup ceph since I am not root 2024-04-24T09:06:54.886 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:06:54 smithi081 ceph-mgr[32341]: -- 172.21.15.81:0/2250977882 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563476a6a5a0 con 0x563476a3ec00 2024-04-24T09:06:54.895 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:54 smithi069 ceph-mon[28170]: pgmap v457: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T09:06:54.895 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:54 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1750267540' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T09:06:54.896 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:06:54 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: ignoring --setuser ceph since I am not root 2024-04-24T09:06:54.896 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:06:54 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: ignoring --setgroup ceph since I am not root 2024-04-24T09:06:54.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.988+0000 7f3575e00640 1 -- 172.21.15.69:0/3449240628 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f355c0a3c70 msgr2=0x7f355c0a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1098677190 2024-04-24T09:06:54.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.988+0000 7f3575e00640 1 --2- 172.21.15.69:0/3449240628 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f355c0a3c70 0x7f355c0a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T09:06:54.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:54.995+0000 7f3577200640 1 -- 172.21.15.69:0/3449240628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f357802d620 con 0x7f3578107b60 2024-04-24T09:06:55.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.001+0000 7f356ea00640 1 -- 172.21.15.69:0/3449240628 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v27)=0 v27) ==== 138+0+0 (secure 0 0 0) 0x7f357009d330 con 0x7f3578107b60 2024-04-24T09:06:55.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.004+0000 7f3577200640 1 -- 172.21.15.69:0/3449240628 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f355c0a3c70 msgr2=0x7f355c0a6130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:06:55.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.004+0000 7f3577200640 1 --2- 172.21.15.69:0/3449240628 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f355c0a3c70 0x7f355c0a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.004+0000 7f3577200640 1 -- 172.21.15.69:0/3449240628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3578107b60 msgr2=0x7f357819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:55.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.004+0000 7f3577200640 1 --2- 172.21.15.69:0/3449240628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3578107b60 0x7f357819f6b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f357000eb10 tx=0x7f357000c2d0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.005+0000 7f3577200640 1 -- 172.21.15.69:0/3449240628 shutdown_connections 2024-04-24T09:06:55.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.005+0000 7f3577200640 1 --2- 172.21.15.69:0/3449240628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3578107b60 0x7f357819f6b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.005+0000 7f3577200640 1 --2- 172.21.15.69:0/3449240628 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3578105230 0x7f3578198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.005+0000 7f3577200640 1 --2- 172.21.15.69:0/3449240628 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f355c0a3c70 0x7f355c0a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.005+0000 7f3577200640 1 --2- 172.21.15.69:0/3449240628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3578102900 0x7f3578198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.005+0000 7f3577200640 1 -- 172.21.15.69:0/3449240628 >> 172.21.15.69:0/3449240628 conn(0x7f35780fc2f0 msgr2=0x7f35781034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:55.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.005+0000 7f3577200640 1 -- 172.21.15.69:0/3449240628 shutdown_connections 2024-04-24T09:06:55.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.005+0000 7f3577200640 1 -- 172.21.15.69:0/3449240628 wait complete. 2024-04-24T09:06:55.023 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-24T09:06:55.137 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:06:54 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:06:54.885+0000 7f6174459200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T09:06:55.149 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:06:54 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:06:54.895+0000 7fe467ea5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T09:06:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.211+0000 7fb952000640 1 -- 172.21.15.69:0/2097007370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb94c1039a0 msgr2=0x7fb94c10a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.211+0000 7fb952000640 1 --2- 172.21.15.69:0/2097007370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb94c1039a0 0x7fb94c10a2c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb944009ca0 tx=0x7fb94402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.211+0000 7fb952000640 1 -- 172.21.15.69:0/2097007370 shutdown_connections 2024-04-24T09:06:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.211+0000 7fb952000640 1 --2- 172.21.15.69:0/2097007370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb94c1039a0 0x7fb94c10a2c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.211+0000 7fb952000640 1 --2- 172.21.15.69:0/2097007370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb94c102fe0 0x7fb94c103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.211+0000 7fb952000640 1 --2- 172.21.15.69:0/2097007370 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb94c101e90 0x7fb94c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.211+0000 7fb952000640 1 -- 172.21.15.69:0/2097007370 >> 172.21.15.69:0/2097007370 conn(0x7fb94c0fd400 msgr2=0x7fb94c0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.211+0000 7fb952000640 1 -- 172.21.15.69:0/2097007370 shutdown_connections 2024-04-24T09:06:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.211+0000 7fb952000640 1 -- 172.21.15.69:0/2097007370 wait complete. 2024-04-24T09:06:55.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.212+0000 7fb952000640 1 Processor -- start 2024-04-24T09:06:55.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.212+0000 7fb952000640 1 -- start start 2024-04-24T09:06:55.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.212+0000 7fb952000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb94c101e90 0x7fb94c075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:55.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.212+0000 7fb952000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb94c102fe0 0x7fb94c0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:55.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.212+0000 7fb952000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb94c1039a0 0x7fb94c074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:55.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.212+0000 7fb952000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c10d9f0 con 0x7fb94c102fe0 2024-04-24T09:06:55.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.212+0000 7fb952000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c10d870 con 0x7fb94c101e90 2024-04-24T09:06:55.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.212+0000 7fb952000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c10db70 con 0x7fb94c1039a0 2024-04-24T09:06:55.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.213+0000 7fb94be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb94c102fe0 0x7fb94c0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:55.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.213+0000 7fb94be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb94c102fe0 0x7fb94c0743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57770/0 (socket says 172.21.15.69:57770) 2024-04-24T09:06:55.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.213+0000 7fb94be00640 1 -- 172.21.15.69:0/1976924759 learned_addr learned my addr 172.21.15.69:0/1976924759 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:55.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.213+0000 7fb950c00640 1 --2- 172.21.15.69:0/1976924759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb94c101e90 0x7fb94c075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:55.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.213+0000 7fb951600640 1 --2- 172.21.15.69:0/1976924759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb94c1039a0 0x7fb94c074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:55.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.213+0000 7fb94be00640 1 -- 172.21.15.69:0/1976924759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb94c1039a0 msgr2=0x7fb94c074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:55.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.213+0000 7fb94be00640 1 --2- 172.21.15.69:0/1976924759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb94c1039a0 0x7fb94c074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.213+0000 7fb94be00640 1 -- 172.21.15.69:0/1976924759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb94c101e90 msgr2=0x7fb94c075db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:55.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.213+0000 7fb94be00640 1 --2- 172.21.15.69:0/1976924759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb94c101e90 0x7fb94c075db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.213+0000 7fb94be00640 1 -- 172.21.15.69:0/1976924759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94c0751c0 con 0x7fb94c102fe0 2024-04-24T09:06:55.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.213+0000 7fb94be00640 1 --2- 172.21.15.69:0/1976924759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb94c102fe0 0x7fb94c0743f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb93800d770 tx=0x7fb93800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:55.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.214+0000 7fb949600640 1 -- 172.21.15.69:0/1976924759 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938017070 con 0x7fb94c102fe0 2024-04-24T09:06:55.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.214+0000 7fb949600640 1 -- 172.21.15.69:0/1976924759 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb938009c30 con 0x7fb94c102fe0 2024-04-24T09:06:55.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.214+0000 7fb952000640 1 -- 172.21.15.69:0/1976924759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94c1a2180 con 0x7fb94c102fe0 2024-04-24T09:06:55.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.214+0000 7fb949600640 1 -- 172.21.15.69:0/1976924759 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938003d00 con 0x7fb94c102fe0 2024-04-24T09:06:55.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.214+0000 7fb952000640 1 -- 172.21.15.69:0/1976924759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94c1a26d0 con 0x7fb94c102fe0 2024-04-24T09:06:55.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.215+0000 7fb952000640 1 -- 172.21.15.69:0/1976924759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb94c1091a0 con 0x7fb94c102fe0 2024-04-24T09:06:55.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.218+0000 7fb949600640 1 -- 172.21.15.69:0/1976924759 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fb938003ea0 con 0x7fb94c102fe0 2024-04-24T09:06:55.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.219+0000 7fb949600640 1 --2- 172.21.15.69:0/1976924759 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb9340a3bf0 0x7fb9340a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:55.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.219+0000 7fb949600640 1 -- 172.21.15.69:0/1976924759 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb9380d05d0 con 0x7fb94c102fe0 2024-04-24T09:06:55.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.219+0000 7fb950c00640 1 -- 172.21.15.69:0/1976924759 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb9340a3bf0 msgr2=0x7fb9340a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1098677190 2024-04-24T09:06:55.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.219+0000 7fb950c00640 1 --2- 172.21.15.69:0/1976924759 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb9340a3bf0 0x7fb9340a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T09:06:55.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.223+0000 7fb949600640 1 -- 172.21.15.69:0/1976924759 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb93809cbf0 con 0x7fb94c102fe0 2024-04-24T09:06:55.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.420+0000 7fb950c00640 1 -- 172.21.15.69:0/1976924759 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb9340a3bf0 msgr2=0x7fb9340a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1098677190 2024-04-24T09:06:55.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.420+0000 7fb950c00640 1 --2- 172.21.15.69:0/1976924759 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb9340a3bf0 0x7fb9340a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T09:06:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.427+0000 7fb952000640 1 -- 172.21.15.69:0/1976924759 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fb94c062c00 con 0x7fb94c102fe0 2024-04-24T09:06:55.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.433+0000 7fb949600640 1 -- 172.21.15.69:0/1976924759 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v28)=0 v28) ==== 133+0+0 (secure 0 0 0) 0x7fb938009de0 con 0x7fb94c102fe0 2024-04-24T09:06:55.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.435+0000 7fb952000640 1 -- 172.21.15.69:0/1976924759 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb9340a3bf0 msgr2=0x7fb9340a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:06:55.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.435+0000 7fb952000640 1 --2- 172.21.15.69:0/1976924759 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb9340a3bf0 0x7fb9340a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.435+0000 7fb952000640 1 -- 172.21.15.69:0/1976924759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb94c102fe0 msgr2=0x7fb94c0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:55.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.435+0000 7fb952000640 1 --2- 172.21.15.69:0/1976924759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb94c102fe0 0x7fb94c0743f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb93800d770 tx=0x7fb93800dc30 comp rx=0 tx=0).stop 2024-04-24T09:06:55.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.435+0000 7fb952000640 1 -- 172.21.15.69:0/1976924759 shutdown_connections 2024-04-24T09:06:55.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.435+0000 7fb952000640 1 --2- 172.21.15.69:0/1976924759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb94c1039a0 0x7fb94c074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.435+0000 7fb952000640 1 --2- 172.21.15.69:0/1976924759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb94c102fe0 0x7fb94c0743f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.435+0000 7fb952000640 1 --2- 172.21.15.69:0/1976924759 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fb9340a3bf0 0x7fb9340a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.435+0000 7fb952000640 1 --2- 172.21.15.69:0/1976924759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb94c101e90 0x7fb94c075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:55.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.435+0000 7fb952000640 1 -- 172.21.15.69:0/1976924759 >> 172.21.15.69:0/1976924759 conn(0x7fb94c0fd400 msgr2=0x7fb94c106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:55.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.436+0000 7fb952000640 1 -- 172.21.15.69:0/1976924759 shutdown_connections 2024-04-24T09:06:55.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:55.436+0000 7fb952000640 1 -- 172.21.15.69:0/1976924759 wait complete. 2024-04-24T09:06:55.451 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:06:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:06:55.148+0000 7fe467ea5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T09:06:55.451 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:06:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:06:55.286+0000 7fe467ea5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T09:06:55.453 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:06:55 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:06:55.142+0000 7f8e81ba5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T09:06:55.453 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:06:55 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:06:55.283+0000 7f8e81ba5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T09:06:55.454 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-24T09:06:55.465 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-24T09:06:55.465 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-24T09:06:55.465 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-04-24T09:06:55.465 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T09:06:55.465 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T09:06:55.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:06:55.465 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-04-24T09:06:55.516 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-24T09:06:55.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:06:55.518 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T09:06:55.567 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:06:55 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:06:55.137+0000 7f6174459200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T09:06:55.567 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:06:55 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:06:55.268+0000 7f6174459200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T09:06:55.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:06:55.588 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-04-24T09:06:55.692 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-24T09:06:55.693 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T09:06:55.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:55 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1750267540' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-24T09:06:55.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:55 smithi069 ceph-mon[28170]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-24T09:06:55.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:55 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3449240628' entity='client.admin' 2024-04-24T09:06:55.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:55 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1976924759' entity='client.admin' 2024-04-24T09:06:55.954 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:06:55 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:06:55.676+0000 7f8e81ba5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T09:06:55.954 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:06:55 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:06:55.788+0000 7f8e81ba5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T09:06:55.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:55 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1750267540' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-24T09:06:55.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:55 smithi130 ceph-mon[31553]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-24T09:06:55.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:55 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3449240628' entity='client.admin' 2024-04-24T09:06:55.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:55 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1976924759' entity='client.admin' 2024-04-24T09:06:56.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:55 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1750267540' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-24T09:06:56.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:55 smithi081 ceph-mon[30961]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-24T09:06:56.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:55 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3449240628' entity='client.admin' 2024-04-24T09:06:56.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:55 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1976924759' entity='client.admin' 2024-04-24T09:06:56.067 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:06:55 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:06:55.658+0000 7f6174459200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T09:06:56.067 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:06:55 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:06:55.769+0000 7f6174459200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T09:06:56.090 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:06:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:06:55.707+0000 7fe467ea5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T09:06:56.091 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:06:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:06:55.825+0000 7fe467ea5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T09:06:56.104 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:06:57.567 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:06:57 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:06:57.283+0000 7f6174459200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T09:06:57.567 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:06:57 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:06:57.505+0000 7f6174459200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T09:06:57.660 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:06:57 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:06:57.398+0000 7fe467ea5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T09:06:57.661 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:06:57 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:06:57.633+0000 7fe467ea5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T09:06:57.703 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:06:57 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:06:57.313+0000 7f8e81ba5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T09:06:57.703 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:06:57 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:06:57.536+0000 7f8e81ba5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T09:06:58.042 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:06:57 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:06:57.863+0000 7fe467ea5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T09:06:58.067 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:06:57 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:06:57.720+0000 7f6174459200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T09:06:58.142 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:06:57 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:06:57.754+0000 7f8e81ba5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T09:06:58.313 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:06:58 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:06:58.268+0000 7fe467ea5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T09:06:58.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.354+0000 7efc16e00700 1 -- 172.21.15.69:0/3280562869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc1006ba30 msgr2=0x7efc1010f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.354+0000 7efc16e00700 1 --2- 172.21.15.69:0/3280562869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc1006ba30 0x7efc1010f9a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efc0800b0a0 tx=0x7efc0800b3b0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.355+0000 7efc16e00700 1 -- 172.21.15.69:0/3280562869 shutdown_connections 2024-04-24T09:06:58.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.355+0000 7efc16e00700 1 --2- 172.21.15.69:0/3280562869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc1006ba30 0x7efc1010f9a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.355+0000 7efc16e00700 1 --2- 172.21.15.69:0/3280562869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc10108b60 0x7efc10108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.355+0000 7efc16e00700 1 --2- 172.21.15.69:0/3280562869 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc1006b090 0x7efc1006b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.355+0000 7efc16e00700 1 -- 172.21.15.69:0/3280562869 >> 172.21.15.69:0/3280562869 conn(0x7efc10078960 msgr2=0x7efc10078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.355+0000 7efc16e00700 1 -- 172.21.15.69:0/3280562869 shutdown_connections 2024-04-24T09:06:58.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.355+0000 7efc16e00700 1 -- 172.21.15.69:0/3280562869 wait complete. 2024-04-24T09:06:58.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.356+0000 7efc16e00700 1 Processor -- start 2024-04-24T09:06:58.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.356+0000 7efc16e00700 1 -- start start 2024-04-24T09:06:58.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.356+0000 7efc16e00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc1006b090 0x7efc10109a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.356+0000 7efc16e00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc1006ba30 0x7efc10109f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.356+0000 7efc16e00700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc10108b60 0x7efc1010a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.356+0000 7efc16e00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efc10112d10 con 0x7efc1006ba30 2024-04-24T09:06:58.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.356+0000 7efc16e00700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efc10112b90 con 0x7efc10108b60 2024-04-24T09:06:58.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.356+0000 7efc16e00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efc10112e90 con 0x7efc1006b090 2024-04-24T09:06:58.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.357+0000 7efc0f400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc1006ba30 0x7efc10109f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.357+0000 7efc0f400700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc1006ba30 0x7efc10109f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57784/0 (socket says 172.21.15.69:57784) 2024-04-24T09:06:58.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.357+0000 7efc0f400700 1 -- 172.21.15.69:0/1217548080 learned_addr learned my addr 172.21.15.69:0/1217548080 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:06:58.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.357+0000 7efc0fe00700 1 --2- 172.21.15.69:0/1217548080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc1006b090 0x7efc10109a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.357+0000 7efc14e00700 1 --2- 172.21.15.69:0/1217548080 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc10108b60 0x7efc1010a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.357+0000 7efc0f400700 1 -- 172.21.15.69:0/1217548080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc1006b090 msgr2=0x7efc10109a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.357+0000 7efc0f400700 1 --2- 172.21.15.69:0/1217548080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc1006b090 0x7efc10109a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.357+0000 7efc0f400700 1 -- 172.21.15.69:0/1217548080 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc10108b60 msgr2=0x7efc1010a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.357+0000 7efc0f400700 1 --2- 172.21.15.69:0/1217548080 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc10108b60 0x7efc1010a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.357+0000 7efc0f400700 1 -- 172.21.15.69:0/1217548080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc101aba10 con 0x7efc1006ba30 2024-04-24T09:06:58.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.357+0000 7efc14e00700 1 --2- 172.21.15.69:0/1217548080 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc10108b60 0x7efc1010a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:58.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.358+0000 7efc0f400700 1 --2- 172.21.15.69:0/1217548080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc1006ba30 0x7efc10109f60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00b530 tx=0x7efbfc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.358+0000 7efc0cc00700 1 -- 172.21.15.69:0/1217548080 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc015020 con 0x7efc1006ba30 2024-04-24T09:06:58.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.358+0000 7efc0cc00700 1 -- 172.21.15.69:0/1217548080 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efbfc011ce0 con 0x7efc1006ba30 2024-04-24T09:06:58.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.358+0000 7efc16e00700 1 -- 172.21.15.69:0/1217548080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc101abd00 con 0x7efc1006ba30 2024-04-24T09:06:58.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.358+0000 7efc0cc00700 1 -- 172.21.15.69:0/1217548080 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc00f2e0 con 0x7efc1006ba30 2024-04-24T09:06:58.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.358+0000 7efc16e00700 1 -- 172.21.15.69:0/1217548080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc101ac1e0 con 0x7efc1006ba30 2024-04-24T09:06:58.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.359+0000 7efc16e00700 1 -- 172.21.15.69:0/1217548080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc101041d0 con 0x7efc1006ba30 2024-04-24T09:06:58.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.362+0000 7efc0cc00700 1 -- 172.21.15.69:0/1217548080 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7efbfc018020 con 0x7efc1006ba30 2024-04-24T09:06:58.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.363+0000 7efc0cc00700 1 --2- 172.21.15.69:0/1217548080 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efbf40a3c90 0x7efbf40a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.363+0000 7efc0cc00700 1 -- 172.21.15.69:0/1217548080 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7efbfc0cffd0 con 0x7efc1006ba30 2024-04-24T09:06:58.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.363+0000 7efc0fe00700 1 -- 172.21.15.69:0/1217548080 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efbf40a3c90 msgr2=0x7efbf40a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1098677190 2024-04-24T09:06:58.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.363+0000 7efc0fe00700 1 --2- 172.21.15.69:0/1217548080 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efbf40a3c90 0x7efbf40a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T09:06:58.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.374+0000 7efc0cc00700 1 -- 172.21.15.69:0/1217548080 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efbfc01f020 con 0x7efc1006ba30 2024-04-24T09:06:58.453 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:06:58 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:06:58.142+0000 7f8e81ba5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T09:06:58.453 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:06:58 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:06:58.265+0000 7f8e81ba5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T09:06:58.473 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:06:58 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:06:58.098+0000 7f6174459200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T09:06:58.473 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:06:58 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:06:58.224+0000 7f6174459200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T09:06:58.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.564+0000 7efc0fe00700 1 -- 172.21.15.69:0/1217548080 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efbf40a3c90 msgr2=0x7efbf40a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1098677190 2024-04-24T09:06:58.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.564+0000 7efc0fe00700 1 --2- 172.21.15.69:0/1217548080 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efbf40a3c90 0x7efbf40a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T09:06:58.565 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:06:58 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:06:58.403+0000 7fe467ea5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T09:06:58.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.650+0000 7efc16e00700 1 -- 172.21.15.69:0/1217548080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efc1002dfe0 con 0x7efc1006ba30 2024-04-24T09:06:58.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.650+0000 7efc0cc00700 1 -- 172.21.15.69:0/1217548080 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+88 (secure 0 0 0) 0x7efbfc010380 con 0x7efc1006ba30 2024-04-24T09:06:58.651 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T09:06:58.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.653+0000 7efc16e00700 1 -- 172.21.15.69:0/1217548080 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efbf40a3c90 msgr2=0x7efbf40a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:06:58.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.653+0000 7efc16e00700 1 --2- 172.21.15.69:0/1217548080 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efbf40a3c90 0x7efbf40a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.653+0000 7efc16e00700 1 -- 172.21.15.69:0/1217548080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc1006ba30 msgr2=0x7efc10109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.653+0000 7efc16e00700 1 --2- 172.21.15.69:0/1217548080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc1006ba30 0x7efc10109f60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00b530 tx=0x7efbfc00b840 comp rx=0 tx=0).stop 2024-04-24T09:06:58.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.653+0000 7efc16e00700 1 -- 172.21.15.69:0/1217548080 shutdown_connections 2024-04-24T09:06:58.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.654+0000 7efc16e00700 1 --2- 172.21.15.69:0/1217548080 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7efbf40a3c90 0x7efbf40a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.654+0000 7efc16e00700 1 --2- 172.21.15.69:0/1217548080 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc10108b60 0x7efc1010a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.654+0000 7efc16e00700 1 --2- 172.21.15.69:0/1217548080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc1006b090 0x7efc10109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.654+0000 7efc16e00700 1 --2- 172.21.15.69:0/1217548080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc1006ba30 0x7efc10109f60 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.654+0000 7efc16e00700 1 -- 172.21.15.69:0/1217548080 >> 172.21.15.69:0/1217548080 conn(0x7efc10078960 msgr2=0x7efc1010dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.654+0000 7efc16e00700 1 -- 172.21.15.69:0/1217548080 shutdown_connections 2024-04-24T09:06:58.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:58.654+0000 7efc16e00700 1 -- 172.21.15.69:0/1217548080 wait complete. 2024-04-24T09:06:58.726 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:06:58 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1217548080' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T09:06:58.726 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:06:58 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:06:58.474+0000 7f6174459200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T09:06:58.726 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:06:58 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:06:58.726+0000 7f6174459200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T09:06:58.772 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:06:58 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:06:58.518+0000 7f8e81ba5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T09:06:58.773 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:06:58 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1217548080' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T09:06:58.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:06:58 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1217548080' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T09:06:58.841 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:06:58 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:06:58.664+0000 7fe467ea5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T09:06:59.203 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:06:58 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:06:58.772+0000 7f8e81ba5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T09:06:59.341 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:06:58 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:06:58.928+0000 7fe467ea5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T09:06:59.540 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T09:06:59.540 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T09:06:59.540 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi069.front.sepia.ceph.com 2024-04-24T09:06:59.540 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T09:06:59.540 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T09:06:59.540 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T09:06:59.540 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T09:06:59.540 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T09:06:59.540 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T09:06:59.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:06:59.541 DEBUG:teuthology.orchestra.run.smithi069:> ip addr 2024-04-24T09:06:59.634 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T09:06:59.634 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T09:06:59.634 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T09:06:59.634 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-24T09:06:59.634 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2024-04-24T09:06:59.634 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-24T09:06:59.634 INFO:teuthology.orchestra.run.smithi069.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T09:06:59.634 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c2 brd ff:ff:ff:ff:ff:ff 2024-04-24T09:06:59.634 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f0 2024-04-24T09:06:59.634 INFO:teuthology.orchestra.run.smithi069.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T09:06:59.635 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c3 brd ff:ff:ff:ff:ff:ff 2024-04-24T09:06:59.635 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f1 2024-04-24T09:06:59.635 INFO:teuthology.orchestra.run.smithi069.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T09:06:59.635 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:36 brd ff:ff:ff:ff:ff:ff 2024-04-24T09:06:59.635 INFO:teuthology.orchestra.run.smithi069.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T09:06:59.635 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:37 brd ff:ff:ff:ff:ff:ff 2024-04-24T09:06:59.635 INFO:teuthology.orchestra.run.smithi069.stdout: inet 172.21.15.69/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-24T09:06:59.635 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft 41804sec preferred_lft 41804sec 2024-04-24T09:06:59.635 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::ec4:7aff:fe8f:d737/64 scope link 2024-04-24T09:06:59.635 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-24T09:06:59.635 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T09:06:59.636 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T09:06:59.636 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-24T09:06:59.636 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add name ceph-brx type bridge 2024-04-24T09:06:59.636 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr flush dev ceph-brx 2024-04-24T09:06:59.636 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set ceph-brx up 2024-04-24T09:06:59.636 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T09:06:59.636 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-24T09:06:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T09:06:59.756 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:06:59 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:06:59.726+0000 7fe467ea5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T09:06:59.817 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:06:59 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:06:59.498+0000 7f6174459200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T09:06:59.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:06:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:06:59.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:06:59.858 DEBUG:teuthology.orchestra.run.smithi069:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T09:06:59.919 INFO:teuthology.orchestra.run.smithi069.stdout:1 2024-04-24T09:06:59.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:06:59.922 DEBUG:teuthology.orchestra.run.smithi069:> route 2024-04-24T09:06:59.953 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:06:59 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:06:59.571+0000 7f8e81ba5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T09:06:59.962 INFO:teuthology.orchestra.run.smithi069.stdout:Kernel IP routing table 2024-04-24T09:06:59.962 INFO:teuthology.orchestra.run.smithi069.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T09:06:59.962 INFO:teuthology.orchestra.run.smithi069.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T09:06:59.962 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T09:06:59.962 INFO:teuthology.orchestra.run.smithi069.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T09:06:59.963 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T09:06:59.963 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-24T09:06:59.963 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T09:06:59.963 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T09:06:59.963 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T09:06:59.963 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-24T09:07:00.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T09:07:00.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:07:00.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:07:00.196 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-04-24T09:07:00.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:07:00.234 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list-id 2024-04-24T09:07:00.317 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:07:00 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:07:00.043+0000 7f6174459200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T09:07:00.317 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:07:00 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:07:00.168+0000 7f6174459200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T09:07:00.324 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T09:07:00.324 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-24T09:07:00.325 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T09:07:00.325 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T09:07:00.325 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-24T09:07:00.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T09:07:00.443 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:00 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:00.313+0000 7fe467ea5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T09:07:00.453 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:07:00 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:07:00.129+0000 7f8e81ba5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T09:07:00.453 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:07:00 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:07:00.256+0000 7f8e81ba5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T09:07:00.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:07:00.489 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T09:07:00.489 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T09:07:00.489 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-24T09:07:00.489 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T09:07:00.489 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T09:07:00.490 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T09:07:00.490 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T09:07:00.490 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T09:07:00.490 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-24T09:07:00.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T09:07:00.701 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:00 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:00.449+0000 7fe467ea5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T09:07:00.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:07:00.739 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T09:07:00.739 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-24T09:07:00.739 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set brx.0 up 2024-04-24T09:07:00.739 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T09:07:00.739 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-24T09:07:00.817 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:07:00 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:07:00.430+0000 7f6174459200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T09:07:00.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T09:07:00.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:07:00.926 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-04-24T09:07:00.926 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T09:07:00.926 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T09:07:00.953 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:07:00 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:07:00.522+0000 7f8e81ba5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T09:07:00.969 INFO:teuthology.orchestra.run.smithi069.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T09:07:00.969 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T09:07:00.969 DEBUG:teuthology.orchestra.run.smithi069:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T09:07:01.060 DEBUG:teuthology.orchestra.run.smithi069:> sudo modprobe fuse 2024-04-24T09:07:01.090 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:00 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:00.734+0000 7fe467ea5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T09:07:01.134 DEBUG:teuthology.orchestra.run.smithi069:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T09:07:01.227 INFO:teuthology.orchestra.run.smithi069.stdout:/proc 2024-04-24T09:07:01.228 INFO:teuthology.orchestra.run.smithi069.stdout:/sys 2024-04-24T09:07:01.228 INFO:teuthology.orchestra.run.smithi069.stdout:/dev 2024-04-24T09:07:01.228 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/security 2024-04-24T09:07:01.228 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/shm 2024-04-24T09:07:01.228 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/pts 2024-04-24T09:07:01.228 INFO:teuthology.orchestra.run.smithi069.stdout:/run 2024-04-24T09:07:01.228 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/cgroup 2024-04-24T09:07:01.228 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/pstore 2024-04-24T09:07:01.228 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/bpf 2024-04-24T09:07:01.228 INFO:teuthology.orchestra.run.smithi069.stdout:/ 2024-04-24T09:07:01.228 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/selinux 2024-04-24T09:07:01.228 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T09:07:01.228 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/hugepages 2024-04-24T09:07:01.229 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/mqueue 2024-04-24T09:07:01.229 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/debug 2024-04-24T09:07:01.229 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/tracing 2024-04-24T09:07:01.229 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/fuse/connections 2024-04-24T09:07:01.229 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/config 2024-04-24T09:07:01.229 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T09:07:01.229 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T09:07:01.229 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T09:07:01.229 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T09:07:01.229 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/1000 2024-04-24T09:07:01.229 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/ceph 2024-04-24T09:07:01.230 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay 2024-04-24T09:07:01.230 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/4d06f77eaa5259c6bb17e8da1a8bed637ae880067f1021e3b3253ba509cd4732/merged 2024-04-24T09:07:01.230 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/7aa7cdb5c56355e95e408e531d393412286fa016cd83957227deaa009667270f/merged 2024-04-24T09:07:01.230 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/0 2024-04-24T09:07:01.230 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T09:07:01.230 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/6aecbd89fecaf0c822196acf6515275ab566f7ad98a825bfa2eb0f2477ed3ef0/merged 2024-04-24T09:07:01.230 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/fc760ee0fbe4ee78eac7c0091cf89c81988700e9a93cc7388a4946ebb780b034/merged 2024-04-24T09:07:01.230 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/443d842c61993dee7c7e6375365c522c0ec18fb322755dc7810647248857eb11/merged 2024-04-24T09:07:01.230 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/3e7dd254f74c7bf87fc37ff571c3fde92575b5d15c6ab6ef290263c26678ac0b/merged 2024-04-24T09:07:01.230 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/8197b9d0a14eb515e2c5c719db4fcdb2277f32478d09b381aad8fe9825d176ec/merged 2024-04-24T09:07:01.230 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/a111db2d886c68cdb65f91da6610327ea1f86955133a2f1d38e556a7977d08a4/merged 2024-04-24T09:07:01.231 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/fca4d0b958c67da344f2869839ad24e35e54dd2c9c941594a4fdbb065d485b47/merged 2024-04-24T09:07:01.231 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/5b8c61dc253e7f4c20a19c8e8aac7cb535181777cd68295075df6b087a11c3d1/merged 2024-04-24T09:07:01.231 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns 2024-04-24T09:07:01.231 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T09:07:01.231 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T09:07:01.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:07:01.232 DEBUG:teuthology.orchestra.run.smithi069:> ls /sys/fs/fuse/connections 2024-04-24T09:07:01.273 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T09:07:01.274 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 --client_fs=cephfs) 2024-04-24T09:07:01.320 DEBUG:teuthology.orchestra.run.smithi069:> sudo modprobe fuse 2024-04-24T09:07:01.393 DEBUG:teuthology.orchestra.run.smithi069:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T09:07:01.432 INFO:teuthology.orchestra.run.smithi069.stdout:/proc 2024-04-24T09:07:01.432 INFO:teuthology.orchestra.run.smithi069.stdout:/sys 2024-04-24T09:07:01.432 INFO:teuthology.orchestra.run.smithi069.stdout:/dev 2024-04-24T09:07:01.432 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/security 2024-04-24T09:07:01.432 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/shm 2024-04-24T09:07:01.432 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/pts 2024-04-24T09:07:01.433 INFO:teuthology.orchestra.run.smithi069.stdout:/run 2024-04-24T09:07:01.433 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/cgroup 2024-04-24T09:07:01.433 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/pstore 2024-04-24T09:07:01.433 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/bpf 2024-04-24T09:07:01.433 INFO:teuthology.orchestra.run.smithi069.stdout:/ 2024-04-24T09:07:01.433 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/selinux 2024-04-24T09:07:01.433 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T09:07:01.433 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/hugepages 2024-04-24T09:07:01.433 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/mqueue 2024-04-24T09:07:01.433 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/debug 2024-04-24T09:07:01.433 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/tracing 2024-04-24T09:07:01.433 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/fuse/connections 2024-04-24T09:07:01.433 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/config 2024-04-24T09:07:01.433 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T09:07:01.434 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T09:07:01.434 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T09:07:01.434 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T09:07:01.434 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/1000 2024-04-24T09:07:01.434 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/ceph 2024-04-24T09:07:01.434 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay 2024-04-24T09:07:01.434 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/4d06f77eaa5259c6bb17e8da1a8bed637ae880067f1021e3b3253ba509cd4732/merged 2024-04-24T09:07:01.434 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/7aa7cdb5c56355e95e408e531d393412286fa016cd83957227deaa009667270f/merged 2024-04-24T09:07:01.434 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/0 2024-04-24T09:07:01.434 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T09:07:01.434 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/6aecbd89fecaf0c822196acf6515275ab566f7ad98a825bfa2eb0f2477ed3ef0/merged 2024-04-24T09:07:01.434 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/fc760ee0fbe4ee78eac7c0091cf89c81988700e9a93cc7388a4946ebb780b034/merged 2024-04-24T09:07:01.435 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/443d842c61993dee7c7e6375365c522c0ec18fb322755dc7810647248857eb11/merged 2024-04-24T09:07:01.435 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/3e7dd254f74c7bf87fc37ff571c3fde92575b5d15c6ab6ef290263c26678ac0b/merged 2024-04-24T09:07:01.435 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/8197b9d0a14eb515e2c5c719db4fcdb2277f32478d09b381aad8fe9825d176ec/merged 2024-04-24T09:07:01.435 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/a111db2d886c68cdb65f91da6610327ea1f86955133a2f1d38e556a7977d08a4/merged 2024-04-24T09:07:01.435 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/fca4d0b958c67da344f2869839ad24e35e54dd2c9c941594a4fdbb065d485b47/merged 2024-04-24T09:07:01.435 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/5b8c61dc253e7f4c20a19c8e8aac7cb535181777cd68295075df6b087a11c3d1/merged 2024-04-24T09:07:01.435 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns 2024-04-24T09:07:01.435 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T09:07:01.435 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T09:07:01.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:07:01.436 DEBUG:teuthology.orchestra.run.smithi069:> ls /sys/fs/fuse/connections 2024-04-24T09:07:01.453 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:07:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:07:01.172+0000 7f8e81ba5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T09:07:01.453 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:07:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:07:01.297+0000 7f8e81ba5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T09:07:01.481 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi069.stderr:2024-04-24T09:07:01.481+0000 7f0e8460e480 -1 init, newargv = 0x559c21378800 newargc=15 2024-04-24T09:07:01.481 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi069.stderr:ceph-fuse[79744]: starting ceph client 2024-04-24T09:07:01.513 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi069.stderr:ceph-fuse[79744]: starting fuse 2024-04-24T09:07:01.516 INFO:teuthology.orchestra.run.smithi069.stdout:114 2024-04-24T09:07:01.516 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-04-24T09:07:01.516 DEBUG:teuthology.orchestra.run.smithi069:> sudo stdin-killer -- python3 -c ' 2024-04-24T09:07:01.516 DEBUG:teuthology.orchestra.run.smithi069:> import glob 2024-04-24T09:07:01.516 DEBUG:teuthology.orchestra.run.smithi069:> import re 2024-04-24T09:07:01.517 DEBUG:teuthology.orchestra.run.smithi069:> import os 2024-04-24T09:07:01.517 DEBUG:teuthology.orchestra.run.smithi069:> import subprocess 2024-04-24T09:07:01.517 DEBUG:teuthology.orchestra.run.smithi069:> 2024-04-24T09:07:01.517 DEBUG:teuthology.orchestra.run.smithi069:> def _find_admin_socket(client_name): 2024-04-24T09:07:01.517 DEBUG:teuthology.orchestra.run.smithi069:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T09:07:01.517 DEBUG:teuthology.orchestra.run.smithi069:> files = glob.glob(asok_path) 2024-04-24T09:07:01.517 DEBUG:teuthology.orchestra.run.smithi069:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T09:07:01.517 DEBUG:teuthology.orchestra.run.smithi069:> 2024-04-24T09:07:01.517 DEBUG:teuthology.orchestra.run.smithi069:> # Given a non-glob path, it better be there 2024-04-24T09:07:01.517 DEBUG:teuthology.orchestra.run.smithi069:> if "*" not in asok_path: 2024-04-24T09:07:01.517 DEBUG:teuthology.orchestra.run.smithi069:> assert(len(files) == 1) 2024-04-24T09:07:01.517 DEBUG:teuthology.orchestra.run.smithi069:> return files[0] 2024-04-24T09:07:01.517 DEBUG:teuthology.orchestra.run.smithi069:> 2024-04-24T09:07:01.517 DEBUG:teuthology.orchestra.run.smithi069:> for f in files: 2024-04-24T09:07:01.517 DEBUG:teuthology.orchestra.run.smithi069:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T09:07:01.517 DEBUG:teuthology.orchestra.run.smithi069:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T09:07:01.518 DEBUG:teuthology.orchestra.run.smithi069:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T09:07:01.518 DEBUG:teuthology.orchestra.run.smithi069:> contents = proc_f.read() 2024-04-24T09:07:01.518 DEBUG:teuthology.orchestra.run.smithi069:> if mountpoint in contents: 2024-04-24T09:07:01.518 DEBUG:teuthology.orchestra.run.smithi069:> return f 2024-04-24T09:07:01.518 DEBUG:teuthology.orchestra.run.smithi069:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T09:07:01.518 DEBUG:teuthology.orchestra.run.smithi069:> 2024-04-24T09:07:01.518 DEBUG:teuthology.orchestra.run.smithi069:> print(_find_admin_socket("client.0")) 2024-04-24T09:07:01.518 DEBUG:teuthology.orchestra.run.smithi069:> ' 2024-04-24T09:07:01.519 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:07:01 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:07:01.071+0000 7f6174459200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T09:07:01.519 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:07:01 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:07:01.195+0000 7f6174459200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T09:07:01.519 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:07:01 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:07:01.519+0000 7f6174459200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T09:07:01.696 INFO:teuthology.orchestra.run.smithi069.stdout:/var/run/ceph/ceph-client.0.79744.asok 2024-04-24T09:07:01.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:07:01.710 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:01 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:01.416+0000 7fe467ea5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T09:07:01.710 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:01 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:01.550+0000 7fe467ea5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T09:07:01.713 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.79744.asok 2024-04-24T09:07:01.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:07:01.713 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.79744.asok status 2024-04-24T09:07:01.817 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:07:01 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:07:01.696+0000 7f6174459200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T09:07:01.915 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:07:01.915 INFO:teuthology.orchestra.run.smithi069.stdout: "metadata": { 2024-04-24T09:07:01.915 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_sha1": "1f81fb397ae98da7563d451a78c61574c8f4e6e0", 2024-04-24T09:07:01.915 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)", 2024-04-24T09:07:01.915 INFO:teuthology.orchestra.run.smithi069.stdout: "entity_id": "0", 2024-04-24T09:07:01.915 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": "smithi069", 2024-04-24T09:07:01.915 INFO:teuthology.orchestra.run.smithi069.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T09:07:01.915 INFO:teuthology.orchestra.run.smithi069.stdout: "pid": "79744", 2024-04-24T09:07:01.915 INFO:teuthology.orchestra.run.smithi069.stdout: "root": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68" 2024-04-24T09:07:01.915 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:07:01.915 INFO:teuthology.orchestra.run.smithi069.stdout: "dentry_count": 0, 2024-04-24T09:07:01.915 INFO:teuthology.orchestra.run.smithi069.stdout: "dentry_pinned_count": 0, 2024-04-24T09:07:01.916 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 15489, 2024-04-24T09:07:01.916 INFO:teuthology.orchestra.run.smithi069.stdout: "inst": { 2024-04-24T09:07:01.916 INFO:teuthology.orchestra.run.smithi069.stdout: "name": { 2024-04-24T09:07:01.916 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "client", 2024-04-24T09:07:01.916 INFO:teuthology.orchestra.run.smithi069.stdout: "num": 15489 2024-04-24T09:07:01.916 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:07:01.916 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": { 2024-04-24T09:07:01.916 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-04-24T09:07:01.916 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "192.168.0.1:0", 2024-04-24T09:07:01.916 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 1464562581 2024-04-24T09:07:01.916 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:07:01.916 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:07:01.916 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": { 2024-04-24T09:07:01.916 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-04-24T09:07:01.916 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "192.168.0.1:0", 2024-04-24T09:07:01.917 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 1464562581 2024-04-24T09:07:01.917 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:07:01.917 INFO:teuthology.orchestra.run.smithi069.stdout: "inst_str": "client.15489 192.168.0.1:0/1464562581", 2024-04-24T09:07:01.917 INFO:teuthology.orchestra.run.smithi069.stdout: "addr_str": "192.168.0.1:0/1464562581", 2024-04-24T09:07:01.917 INFO:teuthology.orchestra.run.smithi069.stdout: "inode_count": 5, 2024-04-24T09:07:01.917 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_epoch": 23, 2024-04-24T09:07:01.917 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_epoch": 92, 2024-04-24T09:07:01.917 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_epoch_barrier": 0, 2024-04-24T09:07:01.917 INFO:teuthology.orchestra.run.smithi069.stdout: "blocklisted": false, 2024-04-24T09:07:01.917 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs" 2024-04-24T09:07:01.917 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:07:01.928 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:07:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:07:01.620+0000 7f8e81ba5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T09:07:01.928 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:07:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:07:01.800+0000 7f8e81ba5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T09:07:01.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:07:01.929 DEBUG:teuthology.orchestra.run.smithi069:> stat --file-system '--printf=%T 2024-04-24T09:07:01.929 DEBUG:teuthology.orchestra.run.smithi069:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T09:07:01.976 INFO:teuthology.orchestra.run.smithi069.stdout:fuseblk 2024-04-24T09:07:01.976 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T09:07:01.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:07:01.976 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T09:07:02.076 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:01 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:01.890+0000 7fe467ea5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T09:07:02.076 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:02 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:02.076+0000 7fe467ea5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T09:07:02.095 INFO:teuthology.run_tasks:Running task exec... 2024-04-24T09:07:02.105 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T09:07:02.105 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-04-24T09:07:02.106 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-24T09:07:02.204 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:07:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:07:01.928+0000 7f8e81ba5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T09:07:02.204 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:07:02 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:07:02.078+0000 7f8e81ba5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T09:07:02.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.289+0000 7f8649000640 1 -- 172.21.15.69:0/1086231440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86441039a0 msgr2=0x7f864410a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:02.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.289+0000 7f8649000640 1 --2- 172.21.15.69:0/1086231440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86441039a0 0x7f864410a2c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f863c009ca0 tx=0x7f863c02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:07:02.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.289+0000 7f8649000640 1 -- 172.21.15.69:0/1086231440 shutdown_connections 2024-04-24T09:07:02.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.289+0000 7f8649000640 1 --2- 172.21.15.69:0/1086231440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86441039a0 0x7f864410a2c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:02.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.289+0000 7f8649000640 1 --2- 172.21.15.69:0/1086231440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8644102fe0 0x7f8644103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:02.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.289+0000 7f8649000640 1 --2- 172.21.15.69:0/1086231440 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8644101e90 0x7f86441022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:02.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.289+0000 7f8649000640 1 -- 172.21.15.69:0/1086231440 >> 172.21.15.69:0/1086231440 conn(0x7f86440fd400 msgr2=0x7f86440ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:02.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.289+0000 7f8649000640 1 -- 172.21.15.69:0/1086231440 shutdown_connections 2024-04-24T09:07:02.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.289+0000 7f8649000640 1 -- 172.21.15.69:0/1086231440 wait complete. 2024-04-24T09:07:02.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.290+0000 7f8649000640 1 Processor -- start 2024-04-24T09:07:02.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.291+0000 7f8649000640 1 -- start start 2024-04-24T09:07:02.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.291+0000 7f8649000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8644101e90 0x7f8644198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:02.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.291+0000 7f8649000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8644102fe0 0x7f8644198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:02.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.291+0000 7f8649000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f86441039a0 0x7f864419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:02.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.291+0000 7f8649000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f864410da40 con 0x7f8644102fe0 2024-04-24T09:07:02.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.291+0000 7f8649000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f864410d8c0 con 0x7f8644101e90 2024-04-24T09:07:02.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.291+0000 7f8649000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f864410dbc0 con 0x7f86441039a0 2024-04-24T09:07:02.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.291+0000 7f8642a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8644102fe0 0x7f8644198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:02.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.291+0000 7f8642a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8644102fe0 0x7f8644198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57790/0 (socket says 172.21.15.69:57790) 2024-04-24T09:07:02.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.291+0000 7f8642a00640 1 -- 172.21.15.69:0/1333802950 learned_addr learned my addr 172.21.15.69:0/1333802950 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:02.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.291+0000 7f8643400640 1 --2- 172.21.15.69:0/1333802950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8644101e90 0x7f8644198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:02.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.291+0000 7f8643e00640 1 --2- 172.21.15.69:0/1333802950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f86441039a0 0x7f864419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:02.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.292+0000 7f8642a00640 1 -- 172.21.15.69:0/1333802950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f86441039a0 msgr2=0x7f864419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:02.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.292+0000 7f8642a00640 1 --2- 172.21.15.69:0/1333802950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f86441039a0 0x7f864419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:02.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.292+0000 7f8642a00640 1 -- 172.21.15.69:0/1333802950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8644101e90 msgr2=0x7f8644198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:02.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.292+0000 7f8642a00640 1 --2- 172.21.15.69:0/1333802950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8644101e90 0x7f8644198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:02.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.292+0000 7f8642a00640 1 -- 172.21.15.69:0/1333802950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864419fc70 con 0x7f8644102fe0 2024-04-24T09:07:02.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.292+0000 7f8642a00640 1 --2- 172.21.15.69:0/1333802950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8644102fe0 0x7f8644198590 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f863000d770 tx=0x7f863000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:02.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.292+0000 7f863be00640 1 -- 172.21.15.69:0/1333802950 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630017070 con 0x7f8644102fe0 2024-04-24T09:07:02.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.292+0000 7f863be00640 1 -- 172.21.15.69:0/1333802950 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8630009c30 con 0x7f8644102fe0 2024-04-24T09:07:02.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.292+0000 7f863be00640 1 -- 172.21.15.69:0/1333802950 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630003d00 con 0x7f8644102fe0 2024-04-24T09:07:02.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.292+0000 7f8649000640 1 -- 172.21.15.69:0/1333802950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864419ff60 con 0x7f8644102fe0 2024-04-24T09:07:02.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.292+0000 7f8649000640 1 -- 172.21.15.69:0/1333802950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86441a0420 con 0x7f8644102fe0 2024-04-24T09:07:02.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.294+0000 7f8649000640 1 -- 172.21.15.69:0/1333802950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8644109140 con 0x7f8644102fe0 2024-04-24T09:07:02.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.296+0000 7f863be00640 1 -- 172.21.15.69:0/1333802950 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f8630003ea0 con 0x7f8644102fe0 2024-04-24T09:07:02.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.297+0000 7f863be00640 1 --2- 172.21.15.69:0/1333802950 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f86280a3bf0 0x7f86280a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:02.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.297+0000 7f863be00640 1 -- 172.21.15.69:0/1333802950 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f86300d0570 con 0x7f8644102fe0 2024-04-24T09:07:02.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.297+0000 7f8643400640 1 -- 172.21.15.69:0/1333802950 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f86280a3bf0 msgr2=0x7f86280a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1098677190 2024-04-24T09:07:02.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.298+0000 7f8643400640 1 --2- 172.21.15.69:0/1333802950 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f86280a3bf0 0x7f86280a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T09:07:02.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.301+0000 7f863be00640 1 -- 172.21.15.69:0/1333802950 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f863009cbf0 con 0x7f8644102fe0 2024-04-24T09:07:02.317 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:07:01 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:07:01.822+0000 7f6174459200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T09:07:02.317 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:07:01 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:07:01.984+0000 7f6174459200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T09:07:02.341 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:02 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:02.214+0000 7fe467ea5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T09:07:02.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.499+0000 7f8643400640 1 -- 172.21.15.69:0/1333802950 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f86280a3bf0 msgr2=0x7f86280a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1098677190 2024-04-24T09:07:02.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.499+0000 7f8643400640 1 --2- 172.21.15.69:0/1333802950 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f86280a3bf0 0x7f86280a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T09:07:02.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.537+0000 7f8649000640 1 -- 172.21.15.69:0/1333802950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f8644062ca0 con 0x7f8644102fe0 2024-04-24T09:07:02.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.591+0000 7f863be00640 1 -- 172.21.15.69:0/1333802950 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v24) ==== 117+0+0 (secure 0 0 0) 0x7f8630009de0 con 0x7f8644102fe0 2024-04-24T09:07:02.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.594+0000 7f8649000640 1 -- 172.21.15.69:0/1333802950 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f86280a3bf0 msgr2=0x7f86280a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:07:02.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.594+0000 7f8649000640 1 --2- 172.21.15.69:0/1333802950 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f86280a3bf0 0x7f86280a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:02.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.594+0000 7f8649000640 1 -- 172.21.15.69:0/1333802950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8644102fe0 msgr2=0x7f8644198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:02.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.594+0000 7f8649000640 1 --2- 172.21.15.69:0/1333802950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8644102fe0 0x7f8644198590 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f863000d770 tx=0x7f863000dc30 comp rx=0 tx=0).stop 2024-04-24T09:07:02.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.594+0000 7f8649000640 1 -- 172.21.15.69:0/1333802950 shutdown_connections 2024-04-24T09:07:02.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.594+0000 7f8649000640 1 --2- 172.21.15.69:0/1333802950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f86441039a0 0x7f864419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:02.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.594+0000 7f8649000640 1 --2- 172.21.15.69:0/1333802950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8644102fe0 0x7f8644198590 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:02.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.594+0000 7f8649000640 1 --2- 172.21.15.69:0/1333802950 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f86280a3bf0 0x7f86280a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:02.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.594+0000 7f8649000640 1 --2- 172.21.15.69:0/1333802950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8644101e90 0x7f8644198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:02.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.594+0000 7f8649000640 1 -- 172.21.15.69:0/1333802950 >> 172.21.15.69:0/1333802950 conn(0x7f86440fd400 msgr2=0x7f8644106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:02.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.594+0000 7f8649000640 1 -- 172.21.15.69:0/1333802950 shutdown_connections 2024-04-24T09:07:02.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.594+0000 7f8649000640 1 -- 172.21.15.69:0/1333802950 wait complete. 2024-04-24T09:07:02.613 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-24T09:07:02.625 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-24T09:07:02.676 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:07:02.743 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:02 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:02.371+0000 7fe467ea5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T09:07:02.744 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:02 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1333802950' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-24T09:07:02.817 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:07:02 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:07:02.363+0000 7f6174459200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T09:07:02.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:02 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1333802950' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-24T09:07:02.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.870+0000 7f7b3a400640 1 -- 172.21.15.69:0/1925973724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b34102ff0 msgr2=0x7f7b34103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:02.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.870+0000 7f7b3a400640 1 --2- 172.21.15.69:0/1925973724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b34102ff0 0x7f7b34103470 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009900 tx=0x7f7b2002a410 comp rx=0 tx=0).stop 2024-04-24T09:07:02.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.870+0000 7f7b3a400640 1 -- 172.21.15.69:0/1925973724 shutdown_connections 2024-04-24T09:07:02.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.870+0000 7f7b3a400640 1 --2- 172.21.15.69:0/1925973724 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b341039b0 0x7f7b3410a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:02.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.870+0000 7f7b3a400640 1 --2- 172.21.15.69:0/1925973724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b34102ff0 0x7f7b34103470 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:02.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.871+0000 7f7b3a400640 1 --2- 172.21.15.69:0/1925973724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b34101ea0 0x7f7b341022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:02.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.871+0000 7f7b3a400640 1 -- 172.21.15.69:0/1925973724 >> 172.21.15.69:0/1925973724 conn(0x7f7b340fd410 msgr2=0x7f7b340ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:02.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.871+0000 7f7b3a400640 1 -- 172.21.15.69:0/1925973724 shutdown_connections 2024-04-24T09:07:02.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.871+0000 7f7b3a400640 1 -- 172.21.15.69:0/1925973724 wait complete. 2024-04-24T09:07:02.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.871+0000 7f7b3a400640 1 Processor -- start 2024-04-24T09:07:02.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.872+0000 7f7b3a400640 1 -- start start 2024-04-24T09:07:02.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.872+0000 7f7b3a400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b34101ea0 0x7f7b341980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:02.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.872+0000 7f7b3a400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b34102ff0 0x7f7b34198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:02.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.872+0000 7f7b3a400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b341039b0 0x7f7b3419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:02.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.872+0000 7f7b3a400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3410da40 con 0x7f7b34102ff0 2024-04-24T09:07:02.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.872+0000 7f7b3a400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3410d8c0 con 0x7f7b341039b0 2024-04-24T09:07:02.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.872+0000 7f7b3a400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3410dbc0 con 0x7f7b34101ea0 2024-04-24T09:07:02.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.872+0000 7f7b33e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b34102ff0 0x7f7b34198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:02.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.872+0000 7f7b33e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b34102ff0 0x7f7b34198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45702/0 (socket says 172.21.15.69:45702) 2024-04-24T09:07:02.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.872+0000 7f7b33e00640 1 -- 172.21.15.69:0/1516412888 learned_addr learned my addr 172.21.15.69:0/1516412888 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:02.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.872+0000 7f7b39a00640 1 --2- 172.21.15.69:0/1516412888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b341039b0 0x7f7b3419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:02.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.873+0000 7f7b39000640 1 --2- 172.21.15.69:0/1516412888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b34101ea0 0x7f7b341980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:02.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.873+0000 7f7b33e00640 1 -- 172.21.15.69:0/1516412888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b34101ea0 msgr2=0x7f7b341980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:02.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.873+0000 7f7b33e00640 1 --2- 172.21.15.69:0/1516412888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b34101ea0 0x7f7b341980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:02.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.873+0000 7f7b33e00640 1 -- 172.21.15.69:0/1516412888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b341039b0 msgr2=0x7f7b3419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:02.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.873+0000 7f7b33e00640 1 --2- 172.21.15.69:0/1516412888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b341039b0 0x7f7b3419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:02.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.873+0000 7f7b33e00640 1 -- 172.21.15.69:0/1516412888 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3419fd00 con 0x7f7b34102ff0 2024-04-24T09:07:02.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.873+0000 7f7b33e00640 1 --2- 172.21.15.69:0/1516412888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b34102ff0 0x7f7b34198620 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7b200314e0 tx=0x7f7b20003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:02.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.873+0000 7f7b31600640 1 -- 172.21.15.69:0/1516412888 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2003f070 con 0x7f7b34102ff0 2024-04-24T09:07:02.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.873+0000 7f7b31600640 1 -- 172.21.15.69:0/1516412888 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b20031c40 con 0x7f7b34102ff0 2024-04-24T09:07:02.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.873+0000 7f7b3a400640 1 -- 172.21.15.69:0/1516412888 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3419ff90 con 0x7f7b34102ff0 2024-04-24T09:07:02.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.874+0000 7f7b31600640 1 -- 172.21.15.69:0/1516412888 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20035030 con 0x7f7b34102ff0 2024-04-24T09:07:02.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.874+0000 7f7b3a400640 1 -- 172.21.15.69:0/1516412888 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b341a0390 con 0x7f7b34102ff0 2024-04-24T09:07:02.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.875+0000 7f7b3a400640 1 -- 172.21.15.69:0/1516412888 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b341091b0 con 0x7f7b34102ff0 2024-04-24T09:07:02.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.877+0000 7f7b31600640 1 -- 172.21.15.69:0/1516412888 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f7b20042070 con 0x7f7b34102ff0 2024-04-24T09:07:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.878+0000 7f7b31600640 1 --2- 172.21.15.69:0/1516412888 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7b1c0a3c00 0x7f7b1c0a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.878+0000 7f7b31600640 1 -- 172.21.15.69:0/1516412888 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7b200f0fd0 con 0x7f7b34102ff0 2024-04-24T09:07:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.878+0000 7f7b39000640 1 -- 172.21.15.69:0/1516412888 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7b1c0a3c00 msgr2=0x7f7b1c0a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1098677190 2024-04-24T09:07:02.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.878+0000 7f7b39000640 1 --2- 172.21.15.69:0/1516412888 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7b1c0a3c00 0x7f7b1c0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T09:07:02.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:02.882+0000 7f7b31600640 1 -- 172.21.15.69:0/1516412888 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b200bd650 con 0x7f7b34102ff0 2024-04-24T09:07:02.953 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:07:02 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:07:02.457+0000 7f8e81ba5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T09:07:02.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:02 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1333802950' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-24T09:07:03.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.079+0000 7f7b39000640 1 -- 172.21.15.69:0/1516412888 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7b1c0a3c00 msgr2=0x7f7b1c0a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1098677190 2024-04-24T09:07:03.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.079+0000 7f7b39000640 1 --2- 172.21.15.69:0/1516412888 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7b1c0a3c00 0x7f7b1c0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T09:07:03.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.084+0000 7f7b3a400640 1 -- 172.21.15.69:0/1516412888 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b34062d40 con 0x7f7b34102ff0 2024-04-24T09:07:03.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.086+0000 7f7b31600640 1 -- 172.21.15.69:0/1516412888 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24564 (secure 0 0 0) 0x7f7b200bcf50 con 0x7f7b34102ff0 2024-04-24T09:07:03.085 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:07:03.085 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":92,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:04:24.497245+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:07:03.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.089+0000 7f7b3a400640 1 -- 172.21.15.69:0/1516412888 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7b1c0a3c00 msgr2=0x7f7b1c0a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:07:03.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.089+0000 7f7b3a400640 1 --2- 172.21.15.69:0/1516412888 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7b1c0a3c00 0x7f7b1c0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.089+0000 7f7b3a400640 1 -- 172.21.15.69:0/1516412888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b34102ff0 msgr2=0x7f7b34198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:03.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.089+0000 7f7b3a400640 1 --2- 172.21.15.69:0/1516412888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b34102ff0 0x7f7b34198620 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7b200314e0 tx=0x7f7b20003c10 comp rx=0 tx=0).stop 2024-04-24T09:07:03.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.089+0000 7f7b3a400640 1 -- 172.21.15.69:0/1516412888 shutdown_connections 2024-04-24T09:07:03.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.089+0000 7f7b3a400640 1 --2- 172.21.15.69:0/1516412888 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f7b1c0a3c00 0x7f7b1c0a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.089+0000 7f7b3a400640 1 --2- 172.21.15.69:0/1516412888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b341039b0 0x7f7b3419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.089+0000 7f7b3a400640 1 --2- 172.21.15.69:0/1516412888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b34102ff0 0x7f7b34198620 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.089+0000 7f7b3a400640 1 --2- 172.21.15.69:0/1516412888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b34101ea0 0x7f7b341980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.089+0000 7f7b3a400640 1 -- 172.21.15.69:0/1516412888 >> 172.21.15.69:0/1516412888 conn(0x7f7b340fd410 msgr2=0x7f7b34106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:03.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.089+0000 7f7b3a400640 1 -- 172.21.15.69:0/1516412888 shutdown_connections 2024-04-24T09:07:03.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.089+0000 7f7b3a400640 1 -- 172.21.15.69:0/1516412888 wait complete. 2024-04-24T09:07:03.090 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:02 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:02.755+0000 7fe467ea5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T09:07:03.106 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:57:37.377599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:04:00.299827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:04:00.958498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T09:04:04.105314+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T09:07:03.107 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:07:03.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.294+0000 7f946ac00640 1 -- 172.21.15.69:0/1190789686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94641052c0 msgr2=0x7f94641076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:03.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.294+0000 7f946ac00640 1 --2- 172.21.15.69:0/1190789686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94641052c0 0x7f94641076b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9450009900 tx=0x7f945002a410 comp rx=0 tx=0).stop 2024-04-24T09:07:03.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.294+0000 7f946ac00640 1 -- 172.21.15.69:0/1190789686 shutdown_connections 2024-04-24T09:07:03.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.294+0000 7f946ac00640 1 --2- 172.21.15.69:0/1190789686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9464107bf0 0x7f946410a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.294+0000 7f946ac00640 1 --2- 172.21.15.69:0/1190789686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94641052c0 0x7f94641076b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.294+0000 7f946ac00640 1 --2- 172.21.15.69:0/1190789686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9464102990 0x7f9464104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.294+0000 7f946ac00640 1 -- 172.21.15.69:0/1190789686 >> 172.21.15.69:0/1190789686 conn(0x7f94640fc3a0 msgr2=0x7f94640fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:03.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.294+0000 7f946ac00640 1 -- 172.21.15.69:0/1190789686 shutdown_connections 2024-04-24T09:07:03.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.294+0000 7f946ac00640 1 -- 172.21.15.69:0/1190789686 wait complete. 2024-04-24T09:07:03.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.295+0000 7f946ac00640 1 Processor -- start 2024-04-24T09:07:03.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.295+0000 7f946ac00640 1 -- start start 2024-04-24T09:07:03.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.295+0000 7f946ac00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9464102990 0x7f946419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:03.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.295+0000 7f946ac00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94641052c0 0x7f946419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:03.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.296+0000 7f946ac00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464107bf0 0x7f94641a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:03.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.296+0000 7f946ac00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f946410da70 con 0x7f9464107bf0 2024-04-24T09:07:03.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.296+0000 7f946ac00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f946410d8f0 con 0x7f94641052c0 2024-04-24T09:07:03.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.296+0000 7f946ac00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f946410dbf0 con 0x7f9464102990 2024-04-24T09:07:03.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.296+0000 7f946a200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464107bf0 0x7f94641a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:03.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.296+0000 7f946a200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464107bf0 0x7f94641a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45714/0 (socket says 172.21.15.69:45714) 2024-04-24T09:07:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.296+0000 7f946a200640 1 -- 172.21.15.69:0/1719059002 learned_addr learned my addr 172.21.15.69:0/1719059002 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.296+0000 7f9468e00640 1 --2- 172.21.15.69:0/1719059002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94641052c0 0x7f946419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.296+0000 7f9469800640 1 --2- 172.21.15.69:0/1719059002 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9464102990 0x7f946419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.296+0000 7f946a200640 1 -- 172.21.15.69:0/1719059002 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9464102990 msgr2=0x7f946419a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.296+0000 7f946a200640 1 --2- 172.21.15.69:0/1719059002 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9464102990 0x7f946419a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.296+0000 7f946a200640 1 -- 172.21.15.69:0/1719059002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94641052c0 msgr2=0x7f946419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.296+0000 7f946a200640 1 --2- 172.21.15.69:0/1719059002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94641052c0 0x7f946419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.296+0000 7f946a200640 1 -- 172.21.15.69:0/1719059002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94641a1f60 con 0x7f9464107bf0 2024-04-24T09:07:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.297+0000 7f946a200640 1 --2- 172.21.15.69:0/1719059002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464107bf0 0x7f94641a18f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f945c00eb10 tx=0x7f945c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.297+0000 7f9462000640 1 -- 172.21.15.69:0/1719059002 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c004b20 con 0x7f9464107bf0 2024-04-24T09:07:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.297+0000 7f9462000640 1 -- 172.21.15.69:0/1719059002 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f945c004cc0 con 0x7f9464107bf0 2024-04-24T09:07:03.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.297+0000 7f9462000640 1 -- 172.21.15.69:0/1719059002 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c010690 con 0x7f9464107bf0 2024-04-24T09:07:03.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.297+0000 7f946ac00640 1 -- 172.21.15.69:0/1719059002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94641a2250 con 0x7f9464107bf0 2024-04-24T09:07:03.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.297+0000 7f946ac00640 1 -- 172.21.15.69:0/1719059002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94641a2720 con 0x7f9464107bf0 2024-04-24T09:07:03.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.298+0000 7f946ac00640 1 -- 172.21.15.69:0/1719059002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9464109740 con 0x7f9464107bf0 2024-04-24T09:07:03.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.300+0000 7f9462000640 1 -- 172.21.15.69:0/1719059002 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f945c005020 con 0x7f9464107bf0 2024-04-24T09:07:03.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.301+0000 7f9462000640 1 --2- 172.21.15.69:0/1719059002 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f944c0a3c00 0x7f944c0a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:03.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.301+0000 7f9462000640 1 -- 172.21.15.69:0/1719059002 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f945c0d0de0 con 0x7f9464107bf0 2024-04-24T09:07:03.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.301+0000 7f9469800640 1 -- 172.21.15.69:0/1719059002 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f944c0a3c00 msgr2=0x7f944c0a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1098677190 2024-04-24T09:07:03.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.302+0000 7f9469800640 1 --2- 172.21.15.69:0/1719059002 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f944c0a3c00 0x7f944c0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T09:07:03.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.305+0000 7f9462000640 1 -- 172.21.15.69:0/1719059002 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f945c09d460 con 0x7f9464107bf0 2024-04-24T09:07:03.424 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:07:03.137+0000 7f6174459200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T09:07:03.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.503+0000 7f9469800640 1 -- 172.21.15.69:0/1719059002 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f944c0a3c00 msgr2=0x7f944c0a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1098677190 2024-04-24T09:07:03.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.503+0000 7f9469800640 1 --2- 172.21.15.69:0/1719059002 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f944c0a3c00 0x7f944c0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T09:07:03.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.505+0000 7f946ac00640 1 -- 172.21.15.69:0/1719059002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9464062c00 con 0x7f9464107bf0 2024-04-24T09:07:03.504 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:07:03.219+0000 7f8e81ba5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T09:07:03.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.506+0000 7f9462000640 1 -- 172.21.15.69:0/1719059002 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f945c09cd60 con 0x7f9464107bf0 2024-04-24T09:07:03.505 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-24T09:07:03.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.509+0000 7f946ac00640 1 -- 172.21.15.69:0/1719059002 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f944c0a3c00 msgr2=0x7f944c0a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:07:03.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.509+0000 7f946ac00640 1 --2- 172.21.15.69:0/1719059002 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f944c0a3c00 0x7f944c0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.509+0000 7f946ac00640 1 -- 172.21.15.69:0/1719059002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464107bf0 msgr2=0x7f94641a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:03.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.509+0000 7f946ac00640 1 --2- 172.21.15.69:0/1719059002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464107bf0 0x7f94641a18f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f945c00eb10 tx=0x7f945c00c2d0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.509+0000 7f946ac00640 1 -- 172.21.15.69:0/1719059002 shutdown_connections 2024-04-24T09:07:03.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.509+0000 7f946ac00640 1 --2- 172.21.15.69:0/1719059002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464107bf0 0x7f94641a18f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.509+0000 7f946ac00640 1 --2- 172.21.15.69:0/1719059002 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7f944c0a3c00 0x7f944c0a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.509+0000 7f946ac00640 1 --2- 172.21.15.69:0/1719059002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94641052c0 0x7f946419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.509+0000 7f946ac00640 1 --2- 172.21.15.69:0/1719059002 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9464102990 0x7f946419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.509+0000 7f946ac00640 1 -- 172.21.15.69:0/1719059002 >> 172.21.15.69:0/1719059002 conn(0x7f94640fc3a0 msgr2=0x7f9464102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:03.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.509+0000 7f946ac00640 1 -- 172.21.15.69:0/1719059002 shutdown_connections 2024-04-24T09:07:03.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.509+0000 7f946ac00640 1 -- 172.21.15.69:0/1719059002 wait complete. 2024-04-24T09:07:03.525 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:07:03.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.723+0000 7fc1fec00640 1 -- 172.21.15.69:0/154422677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1f80fffb0 msgr2=0x7fc1f8100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:03.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.723+0000 7fc1fec00640 1 --2- 172.21.15.69:0/154422677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1f80fffb0 0x7fc1f8100430 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009900 tx=0x7fc1e402a410 comp rx=0 tx=0).stop 2024-04-24T09:07:03.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.723+0000 7fc1fec00640 1 -- 172.21.15.69:0/154422677 shutdown_connections 2024-04-24T09:07:03.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.723+0000 7fc1fec00640 1 --2- 172.21.15.69:0/154422677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1f8100970 0x7fc1f8100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.723+0000 7fc1fec00640 1 --2- 172.21.15.69:0/154422677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1f80fffb0 0x7fc1f8100430 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.723+0000 7fc1fec00640 1 --2- 172.21.15.69:0/154422677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1f8102430 0x7fc1f8102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.723+0000 7fc1fec00640 1 -- 172.21.15.69:0/154422677 >> 172.21.15.69:0/154422677 conn(0x7fc1f80fbb30 msgr2=0x7fc1f80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:03.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.723+0000 7fc1fec00640 1 -- 172.21.15.69:0/154422677 shutdown_connections 2024-04-24T09:07:03.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.723+0000 7fc1fec00640 1 -- 172.21.15.69:0/154422677 wait complete. 2024-04-24T09:07:03.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.724+0000 7fc1fec00640 1 Processor -- start 2024-04-24T09:07:03.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.724+0000 7fc1fec00640 1 -- start start 2024-04-24T09:07:03.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.724+0000 7fc1fec00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1f80fffb0 0x7fc1f819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.724+0000 7fc1fec00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1f8100970 0x7fc1f819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.724+0000 7fc1fec00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1f8102430 0x7fc1f81a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.724+0000 7fc1fec00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f810fc70 con 0x7fc1f8102430 2024-04-24T09:07:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.724+0000 7fc1fec00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f810faf0 con 0x7fc1f8100970 2024-04-24T09:07:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.724+0000 7fc1fec00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f810fdf0 con 0x7fc1f80fffb0 2024-04-24T09:07:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.725+0000 7fc1fe200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1f8102430 0x7fc1f81a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.725+0000 7fc1fe200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1f8102430 0x7fc1f81a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45746/0 (socket says 172.21.15.69:45746) 2024-04-24T09:07:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.725+0000 7fc1fe200640 1 -- 172.21.15.69:0/589966845 learned_addr learned my addr 172.21.15.69:0/589966845 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.725+0000 7fc1fd800640 1 --2- 172.21.15.69:0/589966845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1f80fffb0 0x7fc1f819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.725+0000 7fc1fce00640 1 --2- 172.21.15.69:0/589966845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1f8100970 0x7fc1f819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.725+0000 7fc1fe200640 1 -- 172.21.15.69:0/589966845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1f80fffb0 msgr2=0x7fc1f819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.725+0000 7fc1fe200640 1 --2- 172.21.15.69:0/589966845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1f80fffb0 0x7fc1f819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.725+0000 7fc1fe200640 1 -- 172.21.15.69:0/589966845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1f8100970 msgr2=0x7fc1f819a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:03.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.725+0000 7fc1fe200640 1 --2- 172.21.15.69:0/589966845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1f8100970 0x7fc1f819a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.725+0000 7fc1fe200640 1 -- 172.21.15.69:0/589966845 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f81a1f50 con 0x7fc1f8102430 2024-04-24T09:07:03.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.725+0000 7fc1fe200640 1 --2- 172.21.15.69:0/589966845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1f8102430 0x7fc1f81a18e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000eb10 tx=0x7fc1f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:03.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.726+0000 7fc1f6000640 1 -- 172.21.15.69:0/589966845 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0004b20 con 0x7fc1f8102430 2024-04-24T09:07:03.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.726+0000 7fc1f6000640 1 -- 172.21.15.69:0/589966845 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1f0004cc0 con 0x7fc1f8102430 2024-04-24T09:07:03.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.726+0000 7fc1f6000640 1 -- 172.21.15.69:0/589966845 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0010690 con 0x7fc1f8102430 2024-04-24T09:07:03.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.726+0000 7fc1fec00640 1 -- 172.21.15.69:0/589966845 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f81a2240 con 0x7fc1f8102430 2024-04-24T09:07:03.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.726+0000 7fc1fec00640 1 -- 172.21.15.69:0/589966845 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f81a2740 con 0x7fc1f8102430 2024-04-24T09:07:03.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.727+0000 7fc1fec00640 1 -- 172.21.15.69:0/589966845 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f810bfe0 con 0x7fc1f8102430 2024-04-24T09:07:03.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.729+0000 7fc1f6000640 1 -- 172.21.15.69:0/589966845 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7fc1f0005020 con 0x7fc1f8102430 2024-04-24T09:07:03.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.730+0000 7fc1f6000640 1 --2- 172.21.15.69:0/589966845 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc1e00a3c00 0x7fc1e00a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:03.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.730+0000 7fc1f6000640 1 -- 172.21.15.69:0/589966845 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc1f00d0fc0 con 0x7fc1f8102430 2024-04-24T09:07:03.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.731+0000 7fc1fd800640 1 -- 172.21.15.69:0/589966845 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc1e00a3c00 msgr2=0x7fc1e00a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1098677190 2024-04-24T09:07:03.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.731+0000 7fc1fd800640 1 --2- 172.21.15.69:0/589966845 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc1e00a3c00 0x7fc1e00a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T09:07:03.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.735+0000 7fc1f6000640 1 -- 172.21.15.69:0/589966845 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc1f009d640 con 0x7fc1f8102430 2024-04-24T09:07:03.817 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y[32337]: 2024-04-24T09:07:03.425+0000 7f6174459200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T09:07:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1333802950' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-24T09:07:03.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-mon[30961]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:07:03.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1516412888' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:07:03.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-mon[30961]: Standby manager daemon y restarted 2024-04-24T09:07:03.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-mon[30961]: Standby manager daemon y started 2024-04-24T09:07:03.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-mon[30961]: from='mgr.? 172.21.15.81:0/486298241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T09:07:03.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-mon[30961]: from='mgr.? 172.21.15.81:0/486298241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T09:07:03.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-mon[30961]: from='mgr.? 172.21.15.81:0/486298241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T09:07:03.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-mon[30961]: from='mgr.? 172.21.15.81:0/486298241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T09:07:03.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1719059002' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:07:03.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-mon[30961]: Standby manager daemon z restarted 2024-04-24T09:07:03.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-mon[30961]: Standby manager daemon z started 2024-04-24T09:07:03.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-mon[30961]: from='mgr.? 172.21.15.130:0/3202452813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T09:07:03.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-mon[30961]: from='mgr.? 172.21.15.130:0/3202452813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T09:07:03.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-mon[30961]: from='mgr.? 172.21.15.130:0/3202452813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T09:07:03.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:03 smithi081 ceph-mon[30961]: from='mgr.? 172.21.15.130:0/3202452813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T09:07:03.840 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:03.555+0000 7fe467ea5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T09:07:03.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1333802950' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-24T09:07:03.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-mon[28170]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:07:03.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1516412888' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:07:03.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-mon[28170]: Standby manager daemon y restarted 2024-04-24T09:07:03.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-mon[28170]: Standby manager daemon y started 2024-04-24T09:07:03.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-mon[28170]: from='mgr.? 172.21.15.81:0/486298241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T09:07:03.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-mon[28170]: from='mgr.? 172.21.15.81:0/486298241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T09:07:03.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-mon[28170]: from='mgr.? 172.21.15.81:0/486298241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T09:07:03.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-mon[28170]: from='mgr.? 172.21.15.81:0/486298241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T09:07:03.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1719059002' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:07:03.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-mon[28170]: Standby manager daemon z restarted 2024-04-24T09:07:03.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-mon[28170]: Standby manager daemon z started 2024-04-24T09:07:03.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-mon[28170]: from='mgr.? 172.21.15.130:0/3202452813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T09:07:03.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-mon[28170]: from='mgr.? 172.21.15.130:0/3202452813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T09:07:03.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-mon[28170]: from='mgr.? 172.21.15.130:0/3202452813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T09:07:03.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-mon[28170]: from='mgr.? 172.21.15.130:0/3202452813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T09:07:03.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.884+0000 7fc1f6000640 1 -- 172.21.15.69:0/589966845 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7fc1f00995b0 con 0x7fc1f8102430 2024-04-24T09:07:03.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.886+0000 7fc1f6000640 1 -- 172.21.15.69:0/589966845 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc1e00a3c00 msgr2=0x7fc1e00a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:07:03.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.886+0000 7fc1f6000640 1 --2- 172.21.15.69:0/589966845 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc1e00a3c00 0x7fc1e00a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.951+0000 7fc1fec00640 1 -- 172.21.15.69:0/589966845 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc1f802d840 con 0x7fc1f8102430 2024-04-24T09:07:03.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.951+0000 7fc1f6000640 1 -- 172.21.15.69:0/589966845 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fc1f00618c0 con 0x7fc1f8102430 2024-04-24T09:07:03.950 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:07:03.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.954+0000 7fc1fec00640 1 -- 172.21.15.69:0/589966845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1f8102430 msgr2=0x7fc1f81a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:03.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.954+0000 7fc1fec00640 1 --2- 172.21.15.69:0/589966845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1f8102430 0x7fc1f81a18e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000eb10 tx=0x7fc1f000c2d0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.954+0000 7fc1fec00640 1 -- 172.21.15.69:0/589966845 shutdown_connections 2024-04-24T09:07:03.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.954+0000 7fc1fec00640 1 --2- 172.21.15.69:0/589966845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1f8102430 0x7fc1f81a18e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.954+0000 7fc1fec00640 1 --2- 172.21.15.69:0/589966845 >> [v2:172.21.15.69:6800/1098677190,v1:172.21.15.69:6801/1098677190] conn(0x7fc1e00a3c00 0x7fc1e00a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.954+0000 7fc1fec00640 1 --2- 172.21.15.69:0/589966845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1f8100970 0x7fc1f819a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.954+0000 7fc1fec00640 1 --2- 172.21.15.69:0/589966845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1f80fffb0 0x7fc1f819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:03.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.954+0000 7fc1fec00640 1 -- 172.21.15.69:0/589966845 >> 172.21.15.69:0/589966845 conn(0x7fc1f80fbb30 msgr2=0x7fc1f80fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:03.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.954+0000 7fc1fec00640 1 -- 172.21.15.69:0/589966845 shutdown_connections 2024-04-24T09:07:03.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:03.954+0000 7fc1fec00640 1 -- 172.21.15.69:0/589966845 wait complete. 2024-04-24T09:07:03.953 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z[33567]: 2024-04-24T09:07:03.504+0000 7f8e81ba5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T09:07:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1333802950' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-24T09:07:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-mon[31553]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:07:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1516412888' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:07:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-mon[31553]: Standby manager daemon y restarted 2024-04-24T09:07:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-mon[31553]: Standby manager daemon y started 2024-04-24T09:07:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-mon[31553]: from='mgr.? 172.21.15.81:0/486298241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T09:07:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-mon[31553]: from='mgr.? 172.21.15.81:0/486298241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T09:07:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-mon[31553]: from='mgr.? 172.21.15.81:0/486298241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T09:07:03.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-mon[31553]: from='mgr.? 172.21.15.81:0/486298241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T09:07:03.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1719059002' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:07:03.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-mon[31553]: Standby manager daemon z restarted 2024-04-24T09:07:03.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-mon[31553]: Standby manager daemon z started 2024-04-24T09:07:03.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-mon[31553]: from='mgr.? 172.21.15.130:0/3202452813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T09:07:03.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-mon[31553]: from='mgr.? 172.21.15.130:0/3202452813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T09:07:03.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-mon[31553]: from='mgr.? 172.21.15.130:0/3202452813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T09:07:03.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:03 smithi130 ceph-mon[31553]: from='mgr.? 172.21.15.130:0/3202452813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T09:07:03.970 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:07:04.160 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:03 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:03.861+0000 7fe467ea5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T09:07:04.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.176+0000 7fd61a600640 1 -- 172.21.15.69:0/1245502245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6140785a0 msgr2=0x7fd614078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.176+0000 7fd61a600640 1 --2- 172.21.15.69:0/1245502245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6140785a0 0x7fd614078e40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009c30 tx=0x7fd60c02a720 comp rx=0 tx=0).stop 2024-04-24T09:07:04.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.177+0000 7fd61a600640 1 -- 172.21.15.69:0/1245502245 shutdown_connections 2024-04-24T09:07:04.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.177+0000 7fd61a600640 1 --2- 172.21.15.69:0/1245502245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6140785a0 0x7fd614078e40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.177+0000 7fd61a600640 1 --2- 172.21.15.69:0/1245502245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd614077c00 0x7fd614078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.177+0000 7fd61a600640 1 --2- 172.21.15.69:0/1245502245 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6140769c0 0x7fd614076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.177+0000 7fd61a600640 1 -- 172.21.15.69:0/1245502245 >> 172.21.15.69:0/1245502245 conn(0x7fd6140ff7f0 msgr2=0x7fd614101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:04.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.177+0000 7fd61a600640 1 -- 172.21.15.69:0/1245502245 shutdown_connections 2024-04-24T09:07:04.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.177+0000 7fd61a600640 1 -- 172.21.15.69:0/1245502245 wait complete. 2024-04-24T09:07:04.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.178+0000 7fd61a600640 1 Processor -- start 2024-04-24T09:07:04.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.178+0000 7fd61a600640 1 -- start start 2024-04-24T09:07:04.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.179+0000 7fd61a600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6140769c0 0x7fd61419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.179+0000 7fd61a600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd614077c00 0x7fd61419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.179+0000 7fd61a600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6140785a0 0x7fd6141a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.179+0000 7fd61a600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd614114190 con 0x7fd614077c00 2024-04-24T09:07:04.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.179+0000 7fd61a600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd614114010 con 0x7fd6140769c0 2024-04-24T09:07:04.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.179+0000 7fd61a600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd614114310 con 0x7fd6140785a0 2024-04-24T09:07:04.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.179+0000 7fd613400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6140769c0 0x7fd61419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.179+0000 7fd612a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd614077c00 0x7fd61419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.179+0000 7fd613400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6140769c0 0x7fd61419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.69:40512/0 (socket says 172.21.15.69:40512) 2024-04-24T09:07:04.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.179+0000 7fd612a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd614077c00 0x7fd61419ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45836/0 (socket says 172.21.15.69:45836) 2024-04-24T09:07:04.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.179+0000 7fd613400640 1 -- 172.21.15.69:0/2898749886 learned_addr learned my addr 172.21.15.69:0/2898749886 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:04.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.180+0000 7fd613e00640 1 --2- 172.21.15.69:0/2898749886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6140785a0 0x7fd6141a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.180+0000 7fd613400640 1 -- 172.21.15.69:0/2898749886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6140785a0 msgr2=0x7fd6141a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.180+0000 7fd613400640 1 --2- 172.21.15.69:0/2898749886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6140785a0 0x7fd6141a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.180+0000 7fd613400640 1 -- 172.21.15.69:0/2898749886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd614077c00 msgr2=0x7fd61419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.180+0000 7fd613400640 1 --2- 172.21.15.69:0/2898749886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd614077c00 0x7fd61419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.180+0000 7fd613400640 1 -- 172.21.15.69:0/2898749886 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6141a6470 con 0x7fd6140769c0 2024-04-24T09:07:04.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.181+0000 7fd613400640 1 --2- 172.21.15.69:0/2898749886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6140769c0 0x7fd61419e850 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd60400d770 tx=0x7fd60400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:04.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.181+0000 7fd60be00640 1 -- 172.21.15.69:0/2898749886 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604017070 con 0x7fd6140769c0 2024-04-24T09:07:04.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.181+0000 7fd61a600640 1 -- 172.21.15.69:0/2898749886 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6141a6760 con 0x7fd6140769c0 2024-04-24T09:07:04.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.181+0000 7fd61a600640 1 -- 172.21.15.69:0/2898749886 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6141a6cb0 con 0x7fd6140769c0 2024-04-24T09:07:04.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.182+0000 7fd60be00640 1 -- 172.21.15.69:0/2898749886 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6040054d0 con 0x7fd6140769c0 2024-04-24T09:07:04.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.182+0000 7fd60be00640 1 -- 172.21.15.69:0/2898749886 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604004cf0 con 0x7fd6140769c0 2024-04-24T09:07:04.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.183+0000 7fd61a600640 1 -- 172.21.15.69:0/2898749886 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f40052c0 con 0x7fd6140769c0 2024-04-24T09:07:04.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.188+0000 7fd60be00640 1 -- 172.21.15.69:0/2898749886 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7fd604009c30 con 0x7fd6140769c0 2024-04-24T09:07:04.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.190+0000 7fd60be00640 1 -- 172.21.15.69:0/2898749886 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd6040d0080 con 0x7fd6140769c0 2024-04-24T09:07:04.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.200+0000 7fd60be00640 1 -- 172.21.15.69:0/2898749886 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd60409ca00 con 0x7fd6140769c0 2024-04-24T09:07:04.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.414+0000 7fd61a600640 1 -- 172.21.15.69:0/2898749886 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd5f4005880 con 0x7fd6140769c0 2024-04-24T09:07:04.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.415+0000 7fd60be00640 1 -- 172.21.15.69:0/2898749886 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fd60409c300 con 0x7fd6140769c0 2024-04-24T09:07:04.414 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-24T09:07:04.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.418+0000 7fd61a600640 1 -- 172.21.15.69:0/2898749886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6140769c0 msgr2=0x7fd61419e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.418+0000 7fd61a600640 1 --2- 172.21.15.69:0/2898749886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6140769c0 0x7fd61419e850 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd60400d770 tx=0x7fd60400dc30 comp rx=0 tx=0).stop 2024-04-24T09:07:04.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.418+0000 7fd61a600640 1 -- 172.21.15.69:0/2898749886 shutdown_connections 2024-04-24T09:07:04.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.418+0000 7fd61a600640 1 --2- 172.21.15.69:0/2898749886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6140785a0 0x7fd6141a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.418+0000 7fd61a600640 1 --2- 172.21.15.69:0/2898749886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd614077c00 0x7fd61419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.418+0000 7fd61a600640 1 --2- 172.21.15.69:0/2898749886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6140769c0 0x7fd61419e850 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.418+0000 7fd61a600640 1 -- 172.21.15.69:0/2898749886 >> 172.21.15.69:0/2898749886 conn(0x7fd6140ff7f0 msgr2=0x7fd6141012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:04.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.418+0000 7fd61a600640 1 -- 172.21.15.69:0/2898749886 shutdown_connections 2024-04-24T09:07:04.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.418+0000 7fd61a600640 1 -- 172.21.15.69:0/2898749886 wait complete. 2024-04-24T09:07:04.437 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T09:07:04.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.624+0000 7fbd7bc00640 1 -- 172.21.15.69:0/82794547 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd74100920 msgr2=0x7fbd74100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.624+0000 7fbd7bc00640 1 --2- 172.21.15.69:0/82794547 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd74100920 0x7fbd74100d40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009900 tx=0x7fbd6802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.624+0000 7fbd7bc00640 1 -- 172.21.15.69:0/82794547 shutdown_connections 2024-04-24T09:07:04.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.624+0000 7fbd7bc00640 1 --2- 172.21.15.69:0/82794547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd74101c20 0x7fbd741020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.624+0000 7fbd7bc00640 1 --2- 172.21.15.69:0/82794547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd74101280 0x7fbd741016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.624+0000 7fbd7bc00640 1 --2- 172.21.15.69:0/82794547 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd74100920 0x7fbd74100d40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.624+0000 7fbd7bc00640 1 -- 172.21.15.69:0/82794547 >> 172.21.15.69:0/82794547 conn(0x7fbd740fc4c0 msgr2=0x7fbd740fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:04.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.624+0000 7fbd7bc00640 1 -- 172.21.15.69:0/82794547 shutdown_connections 2024-04-24T09:07:04.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.624+0000 7fbd7bc00640 1 -- 172.21.15.69:0/82794547 wait complete. 2024-04-24T09:07:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.625+0000 7fbd7bc00640 1 Processor -- start 2024-04-24T09:07:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.625+0000 7fbd7bc00640 1 -- start start 2024-04-24T09:07:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd7bc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd74100920 0x7fbd7419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd7bc00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd74101280 0x7fbd7419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd7bc00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd74101c20 0x7fbd741a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd7bc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7410fdf0 con 0x7fbd74100920 2024-04-24T09:07:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd7bc00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7410fc70 con 0x7fbd74101280 2024-04-24T09:07:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd7bc00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7410ff70 con 0x7fbd74101c20 2024-04-24T09:07:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd79200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd74100920 0x7fbd7419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd79200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd74100920 0x7fbd7419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45928/0 (socket says 172.21.15.69:45928) 2024-04-24T09:07:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd79200640 1 -- 172.21.15.69:0/1730600384 learned_addr learned my addr 172.21.15.69:0/1730600384 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd79c00640 1 --2- 172.21.15.69:0/1730600384 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd74101c20 0x7fbd741a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd73e00640 1 --2- 172.21.15.69:0/1730600384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd74101280 0x7fbd7419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd79200640 1 -- 172.21.15.69:0/1730600384 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd74101c20 msgr2=0x7fbd741a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd79200640 1 --2- 172.21.15.69:0/1730600384 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd74101c20 0x7fbd741a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd79200640 1 -- 172.21.15.69:0/1730600384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd74101280 msgr2=0x7fbd7419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd79200640 1 --2- 172.21.15.69:0/1730600384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd74101280 0x7fbd7419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd79200640 1 -- 172.21.15.69:0/1730600384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd741a2020 con 0x7fbd74100920 2024-04-24T09:07:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.626+0000 7fbd79c00640 1 --2- 172.21.15.69:0/1730600384 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd74101c20 0x7fbd741a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.627+0000 7fbd79200640 1 --2- 172.21.15.69:0/1730600384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd74100920 0x7fbd7419a400 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbd680314e0 tx=0x7fbd68003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.627+0000 7fbd71600640 1 -- 172.21.15.69:0/1730600384 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6803f070 con 0x7fbd74100920 2024-04-24T09:07:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.627+0000 7fbd7bc00640 1 -- 172.21.15.69:0/1730600384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd741a22b0 con 0x7fbd74100920 2024-04-24T09:07:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.627+0000 7fbd71600640 1 -- 172.21.15.69:0/1730600384 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd68031e40 con 0x7fbd74100920 2024-04-24T09:07:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.627+0000 7fbd71600640 1 -- 172.21.15.69:0/1730600384 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68035030 con 0x7fbd74100920 2024-04-24T09:07:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.627+0000 7fbd7bc00640 1 -- 172.21.15.69:0/1730600384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd741a27f0 con 0x7fbd74100920 2024-04-24T09:07:04.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.628+0000 7fbd7bc00640 1 -- 172.21.15.69:0/1730600384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd7410b860 con 0x7fbd74100920 2024-04-24T09:07:04.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.630+0000 7fbd71600640 1 -- 172.21.15.69:0/1730600384 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7fbd68042070 con 0x7fbd74100920 2024-04-24T09:07:04.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.632+0000 7fbd71600640 1 -- 172.21.15.69:0/1730600384 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fbd680f0c60 con 0x7fbd74100920 2024-04-24T09:07:04.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.640+0000 7fbd71600640 1 -- 172.21.15.69:0/1730600384 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd680bd580 con 0x7fbd74100920 2024-04-24T09:07:04.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.836+0000 7fbd7bc00640 1 -- 172.21.15.69:0/1730600384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbd74062c00 con 0x7fbd74100920 2024-04-24T09:07:04.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.837+0000 7fbd71600640 1 -- 172.21.15.69:0/1730600384 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fbd680bce80 con 0x7fbd74100920 2024-04-24T09:07:04.837 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:07:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.840+0000 7fbd7bc00640 1 -- 172.21.15.69:0/1730600384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd74100920 msgr2=0x7fbd7419a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.840+0000 7fbd7bc00640 1 --2- 172.21.15.69:0/1730600384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd74100920 0x7fbd7419a400 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbd680314e0 tx=0x7fbd68003c80 comp rx=0 tx=0).stop 2024-04-24T09:07:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.840+0000 7fbd7bc00640 1 -- 172.21.15.69:0/1730600384 shutdown_connections 2024-04-24T09:07:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.840+0000 7fbd7bc00640 1 --2- 172.21.15.69:0/1730600384 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd74101c20 0x7fbd741a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.840+0000 7fbd7bc00640 1 --2- 172.21.15.69:0/1730600384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd74101280 0x7fbd7419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.840+0000 7fbd7bc00640 1 --2- 172.21.15.69:0/1730600384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd74100920 0x7fbd7419a400 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.840+0000 7fbd7bc00640 1 -- 172.21.15.69:0/1730600384 >> 172.21.15.69:0/1730600384 conn(0x7fbd740fc4c0 msgr2=0x7fbd740fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:04.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.840+0000 7fbd7bc00640 1 -- 172.21.15.69:0/1730600384 shutdown_connections 2024-04-24T09:07:04.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:04.841+0000 7fbd7bc00640 1 -- 172.21.15.69:0/1730600384 wait complete. 2024-04-24T09:07:04.841 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:04.428+0000 7fe3b8800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:07:04.841 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:04.428+0000 7fe3b8800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:07:04.841 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:04.428+0000 7fe3b8800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:07:04.841 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:04.428+0000 7fe3b8800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:07:04.842 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:04.428+0000 7fe3b8800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:07:04.842 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:04.428+0000 7fe3bd800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:07:04.842 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:04.428+0000 7fe3bd800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:07:04.842 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:04.428+0000 7fe3bd800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:07:04.842 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:04.428+0000 7fe3bd800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:07:04.842 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:07:04.428+0000 7fe3bd800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:07:04.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-04-24T09:07:04.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: Active manager daemon x restarted 2024-04-24T09:07:04.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: Activating manager daemon x 2024-04-24T09:07:04.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T09:07:04.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: mgrmap e21: x(active, starting, since 0.0192152s), standbys: z, y 2024-04-24T09:07:04.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:07:04.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:07:04.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:07:04.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T09:07:04.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T09:07:04.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T09:07:04.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T09:07:04.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:07:04.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T09:07:04.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T09:07:04.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T09:07:04.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T09:07:04.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T09:07:04.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T09:07:04.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T09:07:04.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T09:07:04.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T09:07:04.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T09:07:04.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:07:04.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:07:04.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:07:04.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:07:04.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:07:04.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T09:07:04.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:07:04.845 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:07:04.845 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:07:04.845 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:07:04.845 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:07:04.845 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:07:04.845 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T09:07:04.845 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T09:07:04.845 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T09:07:04.845 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: Manager daemon x is now available 2024-04-24T09:07:04.845 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/589966845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:07:04.845 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:07:04.845 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T09:07:04.845 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T09:07:04.846 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:04 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2898749886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:07:04.860 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:07:04.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-04-24T09:07:04.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: Active manager daemon x restarted 2024-04-24T09:07:04.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: Activating manager daemon x 2024-04-24T09:07:04.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T09:07:04.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: mgrmap e21: x(active, starting, since 0.0192152s), standbys: z, y 2024-04-24T09:07:04.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:07:04.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:07:04.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:07:04.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T09:07:04.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T09:07:04.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T09:07:04.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T09:07:04.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:07:04.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T09:07:04.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T09:07:04.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T09:07:04.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T09:07:04.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T09:07:04.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T09:07:04.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T09:07:04.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T09:07:04.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T09:07:04.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T09:07:04.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:07:04.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:07:04.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:07:04.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:07:04.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:07:04.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T09:07:04.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:07:04.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:07:04.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:07:04.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:07:04.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:07:04.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:07:04.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T09:07:04.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T09:07:04.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T09:07:04.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: Manager daemon x is now available 2024-04-24T09:07:04.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/589966845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:07:04.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:07:04.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T09:07:04.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T09:07:04.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:04 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2898749886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:07:05.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.050+0000 7fe1f6800640 1 -- 172.21.15.69:0/1707276506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f0105230 msgr2=0x7fe1f0107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:05.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.050+0000 7fe1f6800640 1 --2- 172.21.15.69:0/1707276506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f0105230 0x7fe1f0107620 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009900 tx=0x7fe1dc02a410 comp rx=0 tx=0).stop 2024-04-24T09:07:05.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.051+0000 7fe1f6800640 1 -- 172.21.15.69:0/1707276506 shutdown_connections 2024-04-24T09:07:05.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.051+0000 7fe1f6800640 1 --2- 172.21.15.69:0/1707276506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1f0107b60 0x7fe1f0109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.051+0000 7fe1f6800640 1 --2- 172.21.15.69:0/1707276506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f0105230 0x7fe1f0107620 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.051+0000 7fe1f6800640 1 --2- 172.21.15.69:0/1707276506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1f0102900 0x7fe1f0104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.051+0000 7fe1f6800640 1 -- 172.21.15.69:0/1707276506 >> 172.21.15.69:0/1707276506 conn(0x7fe1f00fc2d0 msgr2=0x7fe1f00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:05.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.051+0000 7fe1f6800640 1 -- 172.21.15.69:0/1707276506 shutdown_connections 2024-04-24T09:07:05.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.051+0000 7fe1f6800640 1 -- 172.21.15.69:0/1707276506 wait complete. 2024-04-24T09:07:05.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.052+0000 7fe1f6800640 1 Processor -- start 2024-04-24T09:07:05.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.052+0000 7fe1f6800640 1 -- start start 2024-04-24T09:07:05.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.053+0000 7fe1f6800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1f0102900 0x7fe1f019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:05.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.053+0000 7fe1f6800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f0105230 0x7fe1f019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:05.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.053+0000 7fe1f6800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1f0107b60 0x7fe1f01a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:05.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.053+0000 7fe1f6800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f010da10 con 0x7fe1f0105230 2024-04-24T09:07:05.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.053+0000 7fe1f6800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f010d890 con 0x7fe1f0107b60 2024-04-24T09:07:05.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.053+0000 7fe1f6800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f010db90 con 0x7fe1f0102900 2024-04-24T09:07:05.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.053+0000 7fe1f4a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f0105230 0x7fe1f019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:05.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.053+0000 7fe1f4a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f0105230 0x7fe1f019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45938/0 (socket says 172.21.15.69:45938) 2024-04-24T09:07:05.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.053+0000 7fe1f4a00640 1 -- 172.21.15.69:0/1318799716 learned_addr learned my addr 172.21.15.69:0/1318799716 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:05.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.053+0000 7fe1f5e00640 1 --2- 172.21.15.69:0/1318799716 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1f0107b60 0x7fe1f01a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:05.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.053+0000 7fe1f5400640 1 --2- 172.21.15.69:0/1318799716 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1f0102900 0x7fe1f019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:05.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.053+0000 7fe1f4a00640 1 -- 172.21.15.69:0/1318799716 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1f0102900 msgr2=0x7fe1f019a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.053+0000 7fe1f4a00640 1 --2- 172.21.15.69:0/1318799716 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1f0102900 0x7fe1f019a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.053+0000 7fe1f4a00640 1 -- 172.21.15.69:0/1318799716 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1f0107b60 msgr2=0x7fe1f01a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.053+0000 7fe1f4a00640 1 --2- 172.21.15.69:0/1318799716 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1f0107b60 0x7fe1f01a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.053+0000 7fe1f4a00640 1 -- 172.21.15.69:0/1318799716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f01a1ee0 con 0x7fe1f0105230 2024-04-24T09:07:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.054+0000 7fe1f4a00640 1 --2- 172.21.15.69:0/1318799716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f0105230 0x7fe1f019a800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0314e0 tx=0x7fe1dc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.054+0000 7fe1ee000640 1 -- 172.21.15.69:0/1318799716 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc03f070 con 0x7fe1f0105230 2024-04-24T09:07:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.054+0000 7fe1ee000640 1 -- 172.21.15.69:0/1318799716 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1dc031c40 con 0x7fe1f0105230 2024-04-24T09:07:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.054+0000 7fe1f6800640 1 -- 172.21.15.69:0/1318799716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f01a2170 con 0x7fe1f0105230 2024-04-24T09:07:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.054+0000 7fe1ee000640 1 -- 172.21.15.69:0/1318799716 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc035030 con 0x7fe1f0105230 2024-04-24T09:07:05.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.054+0000 7fe1f6800640 1 -- 172.21.15.69:0/1318799716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f01a2660 con 0x7fe1f0105230 2024-04-24T09:07:05.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.056+0000 7fe1f6800640 1 -- 172.21.15.69:0/1318799716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f01096b0 con 0x7fe1f0105230 2024-04-24T09:07:05.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.057+0000 7fe1ee000640 1 -- 172.21.15.69:0/1318799716 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fe1dc042070 con 0x7fe1f0105230 2024-04-24T09:07:05.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.059+0000 7fe1ee000640 1 --2- 172.21.15.69:0/1318799716 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe1d80a39b0 0x7fe1d80a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:05.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.060+0000 7fe1ee000640 1 -- 172.21.15.69:0/1318799716 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe1dc0f0d30 con 0x7fe1f0105230 2024-04-24T09:07:05.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.060+0000 7fe1f5400640 1 --2- 172.21.15.69:0/1318799716 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe1d80a39b0 0x7fe1d80a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:05.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.061+0000 7fe1f5400640 1 --2- 172.21.15.69:0/1318799716 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe1d80a39b0 0x7fe1d80a5e70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4005fd0 tx=0x7fe1e4005850 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:05.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.064+0000 7fe1ee000640 1 -- 172.21.15.69:0/1318799716 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe1dc0bd560 con 0x7fe1f0105230 2024-04-24T09:07:05.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-04-24T09:07:05.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: Active manager daemon x restarted 2024-04-24T09:07:05.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: Activating manager daemon x 2024-04-24T09:07:05.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T09:07:05.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: mgrmap e21: x(active, starting, since 0.0192152s), standbys: z, y 2024-04-24T09:07:05.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:07:05.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:07:05.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:07:05.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T09:07:05.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T09:07:05.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T09:07:05.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T09:07:05.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:07:05.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T09:07:05.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T09:07:05.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T09:07:05.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T09:07:05.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T09:07:05.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T09:07:05.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T09:07:05.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T09:07:05.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T09:07:05.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T09:07:05.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:07:05.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:07:05.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:07:05.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:07:05.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:07:05.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T09:07:05.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:07:05.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:07:05.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:07:05.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:07:05.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:07:05.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:07:05.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T09:07:05.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T09:07:05.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T09:07:05.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: Manager daemon x is now available 2024-04-24T09:07:05.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/589966845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:07:05.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:07:05.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T09:07:05.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T09:07:05.070 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:04 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2898749886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:07:05.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.281+0000 7fe1f6800640 1 -- 172.21.15.69:0/1318799716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1f0062c00 con 0x7fe1f0105230 2024-04-24T09:07:05.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.284+0000 7fe1ee000640 1 -- 172.21.15.69:0/1318799716 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24941 (secure 0 0 0) 0x7fe1dc0bce60 con 0x7fe1f0105230 2024-04-24T09:07:05.284 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:07:05.284 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":93,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:07:03.864053+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/660205566":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/1098677190":"2024-04-25T09:07:03.863940+0000","172.21.15.69:0/3145713060":"2024-04-25T09:07:03.863940+0000","172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/3008138297":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/2901345957":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/1098677190":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:07:05.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.286+0000 7fe1f6800640 1 -- 172.21.15.69:0/1318799716 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe1d80a39b0 msgr2=0x7fe1d80a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:05.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.286+0000 7fe1f6800640 1 --2- 172.21.15.69:0/1318799716 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe1d80a39b0 0x7fe1d80a5e70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4005fd0 tx=0x7fe1e4005850 comp rx=0 tx=0).stop 2024-04-24T09:07:05.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.286+0000 7fe1f6800640 1 -- 172.21.15.69:0/1318799716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f0105230 msgr2=0x7fe1f019a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:05.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.286+0000 7fe1f6800640 1 --2- 172.21.15.69:0/1318799716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f0105230 0x7fe1f019a800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0314e0 tx=0x7fe1dc003c10 comp rx=0 tx=0).stop 2024-04-24T09:07:05.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.286+0000 7fe1f6800640 1 -- 172.21.15.69:0/1318799716 shutdown_connections 2024-04-24T09:07:05.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.286+0000 7fe1f6800640 1 --2- 172.21.15.69:0/1318799716 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe1d80a39b0 0x7fe1d80a5e70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.286+0000 7fe1f6800640 1 --2- 172.21.15.69:0/1318799716 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1f0107b60 0x7fe1f01a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.286+0000 7fe1f6800640 1 --2- 172.21.15.69:0/1318799716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1f0105230 0x7fe1f019a800 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.286+0000 7fe1f6800640 1 --2- 172.21.15.69:0/1318799716 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1f0102900 0x7fe1f019a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.286+0000 7fe1f6800640 1 -- 172.21.15.69:0/1318799716 >> 172.21.15.69:0/1318799716 conn(0x7fe1f00fc2d0 msgr2=0x7fe1f0102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:05.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.286+0000 7fe1f6800640 1 -- 172.21.15.69:0/1318799716 shutdown_connections 2024-04-24T09:07:05.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.286+0000 7fe1f6800640 1 -- 172.21.15.69:0/1318799716 wait complete. 2024-04-24T09:07:05.304 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:57:37.377599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:04:00.299827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:04:00.958498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T09:04:04.105314+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T09:07:05.304 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:07:05.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.479+0000 7f4808c00640 1 -- 172.21.15.69:0/1051027772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4804103b40 msgr2=0x7f480410a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:05.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.479+0000 7f4808c00640 1 --2- 172.21.15.69:0/1051027772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4804103b40 0x7f480410a3d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009ca0 tx=0x7f47fc02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.479+0000 7f4808c00640 1 -- 172.21.15.69:0/1051027772 shutdown_connections 2024-04-24T09:07:05.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.479+0000 7f4808c00640 1 --2- 172.21.15.69:0/1051027772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4804103b40 0x7f480410a3d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.479+0000 7f4808c00640 1 --2- 172.21.15.69:0/1051027772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4804103180 0x7f4804103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.479+0000 7f4808c00640 1 --2- 172.21.15.69:0/1051027772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4804101f90 0x7f48041023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.479+0000 7f4808c00640 1 -- 172.21.15.69:0/1051027772 >> 172.21.15.69:0/1051027772 conn(0x7f48040fd500 msgr2=0x7f48040ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:05.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.479+0000 7f4808c00640 1 -- 172.21.15.69:0/1051027772 shutdown_connections 2024-04-24T09:07:05.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.479+0000 7f4808c00640 1 -- 172.21.15.69:0/1051027772 wait complete. 2024-04-24T09:07:05.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.480+0000 7f4808c00640 1 Processor -- start 2024-04-24T09:07:05.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.480+0000 7f4808c00640 1 -- start start 2024-04-24T09:07:05.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.480+0000 7f4808c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4804101f90 0x7f480419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:05.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.480+0000 7f4808c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4804103180 0x7f480419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:05.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.480+0000 7f4808c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4804103b40 0x7f48041a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:05.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.480+0000 7f4808c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f480410da70 con 0x7f4804103b40 2024-04-24T09:07:05.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.480+0000 7f4808c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f480410d8f0 con 0x7f4804101f90 2024-04-24T09:07:05.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.480+0000 7f4808c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f480410dbf0 con 0x7f4804103180 2024-04-24T09:07:05.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.480+0000 7f4803e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4804103b40 0x7f48041a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:05.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.480+0000 7f4803e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4804103b40 0x7f48041a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45954/0 (socket says 172.21.15.69:45954) 2024-04-24T09:07:05.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.480+0000 7f4803e00640 1 -- 172.21.15.69:0/3788925650 learned_addr learned my addr 172.21.15.69:0/3788925650 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:05.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.481+0000 7f47fbe00640 1 --2- 172.21.15.69:0/3788925650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4804103180 0x7f480419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:05.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.481+0000 7f4803400640 1 --2- 172.21.15.69:0/3788925650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4804101f90 0x7f480419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:05.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.481+0000 7f4803e00640 1 -- 172.21.15.69:0/3788925650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4804103180 msgr2=0x7f480419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:05.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.481+0000 7f4803e00640 1 --2- 172.21.15.69:0/3788925650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4804103180 0x7f480419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.481+0000 7f4803e00640 1 -- 172.21.15.69:0/3788925650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4804101f90 msgr2=0x7f480419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:05.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.481+0000 7f4803e00640 1 --2- 172.21.15.69:0/3788925650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4804101f90 0x7f480419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.481+0000 7f4803e00640 1 -- 172.21.15.69:0/3788925650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48041a1f50 con 0x7f4804103b40 2024-04-24T09:07:05.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.481+0000 7f4803400640 1 --2- 172.21.15.69:0/3788925650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4804101f90 0x7f480419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:05.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.481+0000 7f4803e00640 1 --2- 172.21.15.69:0/3788925650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4804103b40 0x7f48041a18e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f47fc031040 tx=0x7f47fc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:05.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.481+0000 7f4800c00640 1 -- 172.21.15.69:0/3788925650 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc032ec0 con 0x7f4804103b40 2024-04-24T09:07:05.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.481+0000 7f4800c00640 1 -- 172.21.15.69:0/3788925650 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47fc0044f0 con 0x7f4804103b40 2024-04-24T09:07:05.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.481+0000 7f4800c00640 1 -- 172.21.15.69:0/3788925650 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc030d40 con 0x7f4804103b40 2024-04-24T09:07:05.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.482+0000 7f4808c00640 1 -- 172.21.15.69:0/3788925650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48041a21e0 con 0x7f4804103b40 2024-04-24T09:07:05.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.482+0000 7f4808c00640 1 -- 172.21.15.69:0/3788925650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48041a26e0 con 0x7f4804103b40 2024-04-24T09:07:05.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.483+0000 7f4808c00640 1 -- 172.21.15.69:0/3788925650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48041a2940 con 0x7f4804103b40 2024-04-24T09:07:05.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.485+0000 7f4800c00640 1 -- 172.21.15.69:0/3788925650 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f47fc0046a0 con 0x7f4804103b40 2024-04-24T09:07:05.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.487+0000 7f4800c00640 1 --2- 172.21.15.69:0/3788925650 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f47e80a3bd0 0x7f47e80a6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:05.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.487+0000 7f4800c00640 1 -- 172.21.15.69:0/3788925650 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f47fc0f1bd0 con 0x7f4804103b40 2024-04-24T09:07:05.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.487+0000 7f4803400640 1 --2- 172.21.15.69:0/3788925650 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f47e80a3bd0 0x7f47e80a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:05.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.488+0000 7f4803400640 1 --2- 172.21.15.69:0/3788925650 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f47e80a3bd0 0x7f47e80a6090 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f47f4005d00 tx=0x7f47f4005c90 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:05.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.491+0000 7f4800c00640 1 -- 172.21.15.69:0/3788925650 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47fc0be440 con 0x7f4804103b40 2024-04-24T09:07:05.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.691+0000 7f4808c00640 1 -- 172.21.15.69:0/3788925650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f48041a2cd0 con 0x7f4804103b40 2024-04-24T09:07:05.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.692+0000 7f4800c00640 1 -- 172.21.15.69:0/3788925650 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f47fc0bdd40 con 0x7f4804103b40 2024-04-24T09:07:05.692 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-24T09:07:05.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.695+0000 7f4808c00640 1 -- 172.21.15.69:0/3788925650 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f47e80a3bd0 msgr2=0x7f47e80a6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:05.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.695+0000 7f4808c00640 1 --2- 172.21.15.69:0/3788925650 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f47e80a3bd0 0x7f47e80a6090 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f47f4005d00 tx=0x7f47f4005c90 comp rx=0 tx=0).stop 2024-04-24T09:07:05.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.695+0000 7f4808c00640 1 -- 172.21.15.69:0/3788925650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4804103b40 msgr2=0x7f48041a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:05.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.695+0000 7f4808c00640 1 --2- 172.21.15.69:0/3788925650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4804103b40 0x7f48041a18e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f47fc031040 tx=0x7f47fc003ba0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.695+0000 7f4808c00640 1 -- 172.21.15.69:0/3788925650 shutdown_connections 2024-04-24T09:07:05.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.695+0000 7f4808c00640 1 --2- 172.21.15.69:0/3788925650 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f47e80a3bd0 0x7f47e80a6090 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.695+0000 7f4808c00640 1 --2- 172.21.15.69:0/3788925650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4804103b40 0x7f48041a18e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.695+0000 7f4808c00640 1 --2- 172.21.15.69:0/3788925650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4804103180 0x7f480419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.695+0000 7f4808c00640 1 --2- 172.21.15.69:0/3788925650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4804101f90 0x7f480419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.695+0000 7f4808c00640 1 -- 172.21.15.69:0/3788925650 >> 172.21.15.69:0/3788925650 conn(0x7f48040fd500 msgr2=0x7f4804074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:05.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.695+0000 7f4808c00640 1 -- 172.21.15.69:0/3788925650 shutdown_connections 2024-04-24T09:07:05.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.695+0000 7f4808c00640 1 -- 172.21.15.69:0/3788925650 wait complete. 2024-04-24T09:07:05.713 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:07:05.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.905+0000 7f667d400640 1 -- 172.21.15.69:0/3400911873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6678107c80 msgr2=0x7f667810a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:05.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.905+0000 7f667d400640 1 --2- 172.21.15.69:0/3400911873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6678107c80 0x7f667810a070 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6670009be0 tx=0x7f667002a6c0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.905+0000 7f667d400640 1 -- 172.21.15.69:0/3400911873 shutdown_connections 2024-04-24T09:07:05.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.905+0000 7f667d400640 1 --2- 172.21.15.69:0/3400911873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6678107c80 0x7f667810a070 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.905+0000 7f667d400640 1 --2- 172.21.15.69:0/3400911873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6678105350 0x7f6678107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.905+0000 7f667d400640 1 --2- 172.21.15.69:0/3400911873 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6678102a20 0x7f6678104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.905+0000 7f667d400640 1 -- 172.21.15.69:0/3400911873 >> 172.21.15.69:0/3400911873 conn(0x7f66780fc3b0 msgr2=0x7f66780fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:05.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.905+0000 7f667d400640 1 -- 172.21.15.69:0/3400911873 shutdown_connections 2024-04-24T09:07:05.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.905+0000 7f667d400640 1 -- 172.21.15.69:0/3400911873 wait complete. 2024-04-24T09:07:05.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.906+0000 7f667d400640 1 Processor -- start 2024-04-24T09:07:05.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.906+0000 7f667d400640 1 -- start start 2024-04-24T09:07:05.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.906+0000 7f667d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6678102a20 0x7f6678198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:05.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.906+0000 7f667d400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6678105350 0x7f6678198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:05.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.906+0000 7f667d400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6678107c80 0x7f667819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:05.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.906+0000 7f667d400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f667810da90 con 0x7f6678102a20 2024-04-24T09:07:05.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.906+0000 7f667d400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f667810d910 con 0x7f6678105350 2024-04-24T09:07:05.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.906+0000 7f667d400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f667810dc10 con 0x7f6678107c80 2024-04-24T09:07:05.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.906+0000 7f6677e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6678102a20 0x7f6678198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:05.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.906+0000 7f6677e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6678102a20 0x7f6678198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45970/0 (socket says 172.21.15.69:45970) 2024-04-24T09:07:05.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.906+0000 7f6677e00640 1 -- 172.21.15.69:0/4244725658 learned_addr learned my addr 172.21.15.69:0/4244725658 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:05.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.907+0000 7f6677400640 1 --2- 172.21.15.69:0/4244725658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6678105350 0x7f6678198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:05.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.907+0000 7f667ca00640 1 --2- 172.21.15.69:0/4244725658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6678107c80 0x7f667819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:05.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.907+0000 7f6677e00640 1 -- 172.21.15.69:0/4244725658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6678107c80 msgr2=0x7f667819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:05.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.907+0000 7f6677e00640 1 --2- 172.21.15.69:0/4244725658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6678107c80 0x7f667819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.907+0000 7f6677e00640 1 -- 172.21.15.69:0/4244725658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6678105350 msgr2=0x7f6678198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:05.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.907+0000 7f6677e00640 1 --2- 172.21.15.69:0/4244725658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6678105350 0x7f6678198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:05.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.907+0000 7f6677e00640 1 -- 172.21.15.69:0/4244725658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667819fd30 con 0x7f6678102a20 2024-04-24T09:07:05.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.907+0000 7f6677400640 1 --2- 172.21.15.69:0/4244725658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6678105350 0x7f6678198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:05.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.907+0000 7f6677e00640 1 --2- 172.21.15.69:0/4244725658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6678102a20 0x7f6678198110 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f666800e9f0 tx=0x7f666800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:05.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.907+0000 7f6674c00640 1 -- 172.21.15.69:0/4244725658 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666800cd80 con 0x7f6678102a20 2024-04-24T09:07:05.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.907+0000 7f6674c00640 1 -- 172.21.15.69:0/4244725658 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66680054d0 con 0x7f6678102a20 2024-04-24T09:07:05.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.907+0000 7f667d400640 1 -- 172.21.15.69:0/4244725658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667819ffc0 con 0x7f6678102a20 2024-04-24T09:07:05.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.907+0000 7f6674c00640 1 -- 172.21.15.69:0/4244725658 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668010490 con 0x7f6678102a20 2024-04-24T09:07:05.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.908+0000 7f667d400640 1 -- 172.21.15.69:0/4244725658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66781a0430 con 0x7f6678102a20 2024-04-24T09:07:05.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.909+0000 7f667d400640 1 -- 172.21.15.69:0/4244725658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66780fdea0 con 0x7f6678102a20 2024-04-24T09:07:05.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.910+0000 7f6674c00640 1 -- 172.21.15.69:0/4244725658 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f6668005020 con 0x7f6678102a20 2024-04-24T09:07:05.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.912+0000 7f6674c00640 1 --2- 172.21.15.69:0/4244725658 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f665c0a39b0 0x7f665c0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:05.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.912+0000 7f6674c00640 1 -- 172.21.15.69:0/4244725658 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f66680d0950 con 0x7f6678102a20 2024-04-24T09:07:05.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.912+0000 7f6677400640 1 --2- 172.21.15.69:0/4244725658 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f665c0a39b0 0x7f665c0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:05.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.913+0000 7f6677400640 1 --2- 172.21.15.69:0/4244725658 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f665c0a39b0 0x7f665c0a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6678199620 tx=0x7f6664009260 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:05.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:05.916+0000 7f6674c00640 1 -- 172.21.15.69:0/4244725658 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f666809d180 con 0x7f6678102a20 2024-04-24T09:07:05.948 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:05 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1730600384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:07:05.948 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:05 smithi081 ceph-mon[30961]: mgrmap e22: x(active, since 1.03801s), standbys: z, y 2024-04-24T09:07:05.948 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:05 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:05.948 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:05 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1318799716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:07:05.948 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:05 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:05.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:05 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1730600384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:07:05.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:05 smithi130 ceph-mon[31553]: mgrmap e22: x(active, since 1.03801s), standbys: z, y 2024-04-24T09:07:05.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:05 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:05.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:05 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1318799716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:07:05.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:05 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:06.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:05 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1730600384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:07:06.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:05 smithi069 ceph-mon[28170]: mgrmap e22: x(active, since 1.03801s), standbys: z, y 2024-04-24T09:07:06.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:05 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:06.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:05 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1318799716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:07:06.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:05 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:06.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.125+0000 7f667d400640 1 -- 172.21.15.69:0/4244725658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6678062c00 con 0x7f6678102a20 2024-04-24T09:07:06.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.125+0000 7f6674c00640 1 -- 172.21.15.69:0/4244725658 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f666809ca80 con 0x7f6678102a20 2024-04-24T09:07:06.125 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:07:06.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.128+0000 7f667d400640 1 -- 172.21.15.69:0/4244725658 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f665c0a39b0 msgr2=0x7f665c0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:06.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.128+0000 7f667d400640 1 --2- 172.21.15.69:0/4244725658 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f665c0a39b0 0x7f665c0a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6678199620 tx=0x7f6664009260 comp rx=0 tx=0).stop 2024-04-24T09:07:06.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.128+0000 7f667d400640 1 -- 172.21.15.69:0/4244725658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6678102a20 msgr2=0x7f6678198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:06.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.128+0000 7f667d400640 1 --2- 172.21.15.69:0/4244725658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6678102a20 0x7f6678198110 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f666800e9f0 tx=0x7f666800edf0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.128+0000 7f667d400640 1 -- 172.21.15.69:0/4244725658 shutdown_connections 2024-04-24T09:07:06.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.128+0000 7f667d400640 1 --2- 172.21.15.69:0/4244725658 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f665c0a39b0 0x7f665c0a5e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.128+0000 7f667d400640 1 --2- 172.21.15.69:0/4244725658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6678107c80 0x7f667819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.128+0000 7f667d400640 1 --2- 172.21.15.69:0/4244725658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6678105350 0x7f6678198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.128+0000 7f667d400640 1 --2- 172.21.15.69:0/4244725658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6678102a20 0x7f6678198110 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.128+0000 7f667d400640 1 -- 172.21.15.69:0/4244725658 >> 172.21.15.69:0/4244725658 conn(0x7f66780fc3b0 msgr2=0x7f6678103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:06.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.128+0000 7f667d400640 1 -- 172.21.15.69:0/4244725658 shutdown_connections 2024-04-24T09:07:06.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.128+0000 7f667d400640 1 -- 172.21.15.69:0/4244725658 wait complete. 2024-04-24T09:07:06.145 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:07:06.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.326+0000 7fad69c00640 1 -- 172.21.15.69:0/4226298694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad64102ac0 msgr2=0x7fad64104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:06.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.326+0000 7fad69c00640 1 --2- 172.21.15.69:0/4226298694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad64102ac0 0x7fad64104eb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fad54009900 tx=0x7fad5402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.327+0000 7fad69c00640 1 -- 172.21.15.69:0/4226298694 shutdown_connections 2024-04-24T09:07:06.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.327+0000 7fad69c00640 1 --2- 172.21.15.69:0/4226298694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad64107d20 0x7fad6410a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.327+0000 7fad69c00640 1 --2- 172.21.15.69:0/4226298694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad641053f0 0x7fad641077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.327+0000 7fad69c00640 1 --2- 172.21.15.69:0/4226298694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad64102ac0 0x7fad64104eb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.327+0000 7fad69c00640 1 -- 172.21.15.69:0/4226298694 >> 172.21.15.69:0/4226298694 conn(0x7fad640fc4b0 msgr2=0x7fad640fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:06.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.327+0000 7fad69c00640 1 -- 172.21.15.69:0/4226298694 shutdown_connections 2024-04-24T09:07:06.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.327+0000 7fad69c00640 1 -- 172.21.15.69:0/4226298694 wait complete. 2024-04-24T09:07:06.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.327+0000 7fad69c00640 1 Processor -- start 2024-04-24T09:07:06.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.328+0000 7fad69c00640 1 -- start start 2024-04-24T09:07:06.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.328+0000 7fad69c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad64102ac0 0x7fad6419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:06.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.328+0000 7fad69c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad641053f0 0x7fad6419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:06.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.328+0000 7fad69c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad64107d20 0x7fad641a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:06.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.328+0000 7fad69c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fad6410dbd0 con 0x7fad64107d20 2024-04-24T09:07:06.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.328+0000 7fad69c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fad6410da50 con 0x7fad64102ac0 2024-04-24T09:07:06.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.328+0000 7fad69c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fad6410dd50 con 0x7fad641053f0 2024-04-24T09:07:06.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.328+0000 7fad63400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad64107d20 0x7fad641a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:06.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.328+0000 7fad63400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad64107d20 0x7fad641a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45992/0 (socket says 172.21.15.69:45992) 2024-04-24T09:07:06.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.328+0000 7fad63400640 1 -- 172.21.15.69:0/82116143 learned_addr learned my addr 172.21.15.69:0/82116143 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:06.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.328+0000 7fad62000640 1 --2- 172.21.15.69:0/82116143 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad641053f0 0x7fad6419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.328+0000 7fad62a00640 1 --2- 172.21.15.69:0/82116143 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad64102ac0 0x7fad6419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.329+0000 7fad63400640 1 -- 172.21.15.69:0/82116143 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad641053f0 msgr2=0x7fad6419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.329+0000 7fad63400640 1 --2- 172.21.15.69:0/82116143 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad641053f0 0x7fad6419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.329+0000 7fad63400640 1 -- 172.21.15.69:0/82116143 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad64102ac0 msgr2=0x7fad6419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.329+0000 7fad63400640 1 --2- 172.21.15.69:0/82116143 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad64102ac0 0x7fad6419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.329+0000 7fad63400640 1 -- 172.21.15.69:0/82116143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad641a20e0 con 0x7fad64107d20 2024-04-24T09:07:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.329+0000 7fad63400640 1 --2- 172.21.15.69:0/82116143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad64107d20 0x7fad641a1a70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00eb10 tx=0x7fad5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.329+0000 7fad5b400640 1 -- 172.21.15.69:0/82116143 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c004b20 con 0x7fad64107d20 2024-04-24T09:07:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.329+0000 7fad5b400640 1 -- 172.21.15.69:0/82116143 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad5c004cc0 con 0x7fad64107d20 2024-04-24T09:07:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.329+0000 7fad5b400640 1 -- 172.21.15.69:0/82116143 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c010690 con 0x7fad64107d20 2024-04-24T09:07:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.329+0000 7fad69c00640 1 -- 172.21.15.69:0/82116143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad641a23d0 con 0x7fad64107d20 2024-04-24T09:07:06.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.330+0000 7fad69c00640 1 -- 172.21.15.69:0/82116143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad641a2920 con 0x7fad64107d20 2024-04-24T09:07:06.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.331+0000 7fad69c00640 1 -- 172.21.15.69:0/82116143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad64109870 con 0x7fad64107d20 2024-04-24T09:07:06.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.333+0000 7fad5b400640 1 -- 172.21.15.69:0/82116143 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fad5c005020 con 0x7fad64107d20 2024-04-24T09:07:06.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.334+0000 7fad5b400640 1 --2- 172.21.15.69:0/82116143 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fad480a3a30 0x7fad480a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:06.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.334+0000 7fad5b400640 1 -- 172.21.15.69:0/82116143 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fad5c0d0a40 con 0x7fad64107d20 2024-04-24T09:07:06.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.334+0000 7fad62a00640 1 --2- 172.21.15.69:0/82116143 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fad480a3a30 0x7fad480a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:06.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.335+0000 7fad62a00640 1 --2- 172.21.15.69:0/82116143 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fad480a3a30 0x7fad480a5ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fad54000c00 tx=0x7fad540314a0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:06.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.339+0000 7fad5b400640 1 -- 172.21.15.69:0/82116143 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad5c09d270 con 0x7fad64107d20 2024-04-24T09:07:06.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.557+0000 7fad69c00640 1 -- 172.21.15.69:0/82116143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fad64062c00 con 0x7fad64107d20 2024-04-24T09:07:06.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.558+0000 7fad5b400640 1 -- 172.21.15.69:0/82116143 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fad5c09cb70 con 0x7fad64107d20 2024-04-24T09:07:06.558 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-24T09:07:06.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.560+0000 7fad69c00640 1 -- 172.21.15.69:0/82116143 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fad480a3a30 msgr2=0x7fad480a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:06.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.560+0000 7fad69c00640 1 --2- 172.21.15.69:0/82116143 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fad480a3a30 0x7fad480a5ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fad54000c00 tx=0x7fad540314a0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.560+0000 7fad69c00640 1 -- 172.21.15.69:0/82116143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad64107d20 msgr2=0x7fad641a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:06.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.560+0000 7fad69c00640 1 --2- 172.21.15.69:0/82116143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad64107d20 0x7fad641a1a70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00eb10 tx=0x7fad5c00c2d0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.560+0000 7fad69c00640 1 -- 172.21.15.69:0/82116143 shutdown_connections 2024-04-24T09:07:06.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.560+0000 7fad69c00640 1 --2- 172.21.15.69:0/82116143 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fad480a3a30 0x7fad480a5ef0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.560+0000 7fad69c00640 1 --2- 172.21.15.69:0/82116143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad64107d20 0x7fad641a1a70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.560+0000 7fad69c00640 1 --2- 172.21.15.69:0/82116143 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad641053f0 0x7fad6419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.560+0000 7fad69c00640 1 --2- 172.21.15.69:0/82116143 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad64102ac0 0x7fad6419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.560+0000 7fad69c00640 1 -- 172.21.15.69:0/82116143 >> 172.21.15.69:0/82116143 conn(0x7fad640fc4b0 msgr2=0x7fad64102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:06.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.561+0000 7fad69c00640 1 -- 172.21.15.69:0/82116143 shutdown_connections 2024-04-24T09:07:06.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.561+0000 7fad69c00640 1 -- 172.21.15.69:0/82116143 wait complete. 2024-04-24T09:07:06.579 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T09:07:06.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.775+0000 7f3ece400640 1 -- 172.21.15.69:0/1311694896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ec8102ac0 msgr2=0x7f3ec8104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:06.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.775+0000 7f3ece400640 1 --2- 172.21.15.69:0/1311694896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ec8102ac0 0x7f3ec8104eb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009900 tx=0x7f3eb802a410 comp rx=0 tx=0).stop 2024-04-24T09:07:06.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.775+0000 7f3ece400640 1 -- 172.21.15.69:0/1311694896 shutdown_connections 2024-04-24T09:07:06.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.775+0000 7f3ece400640 1 --2- 172.21.15.69:0/1311694896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ec8107d20 0x7f3ec810a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.775+0000 7f3ece400640 1 --2- 172.21.15.69:0/1311694896 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ec81053f0 0x7f3ec81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.775+0000 7f3ece400640 1 --2- 172.21.15.69:0/1311694896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ec8102ac0 0x7f3ec8104eb0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.775+0000 7f3ece400640 1 -- 172.21.15.69:0/1311694896 >> 172.21.15.69:0/1311694896 conn(0x7f3ec80fc4b0 msgr2=0x7f3ec80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:06.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.775+0000 7f3ece400640 1 -- 172.21.15.69:0/1311694896 shutdown_connections 2024-04-24T09:07:06.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.775+0000 7f3ece400640 1 -- 172.21.15.69:0/1311694896 wait complete. 2024-04-24T09:07:06.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.776+0000 7f3ece400640 1 Processor -- start 2024-04-24T09:07:06.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.777+0000 7f3ece400640 1 -- start start 2024-04-24T09:07:06.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.777+0000 7f3ece400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ec8102ac0 0x7f3ec8102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:06.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.777+0000 7f3ece400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ec81053f0 0x7f3ec8100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:06.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.777+0000 7f3ece400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ec8107d20 0x7f3ec8100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:06.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.777+0000 7f3ece400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec810db80 con 0x7f3ec8107d20 2024-04-24T09:07:06.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.777+0000 7f3ece400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec810da00 con 0x7f3ec81053f0 2024-04-24T09:07:06.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.777+0000 7f3ece400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec810dd00 con 0x7f3ec8102ac0 2024-04-24T09:07:06.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.777+0000 7f3ec7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ec8107d20 0x7f3ec8100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:06.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.777+0000 7f3ec7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ec8107d20 0x7f3ec8100fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46020/0 (socket says 172.21.15.69:46020) 2024-04-24T09:07:06.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.777+0000 7f3ec7e00640 1 -- 172.21.15.69:0/3679096262 learned_addr learned my addr 172.21.15.69:0/3679096262 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:06.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.777+0000 7f3ec7400640 1 --2- 172.21.15.69:0/3679096262 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ec8102ac0 0x7f3ec8102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:06.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.778+0000 7f3ec6a00640 1 --2- 172.21.15.69:0/3679096262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ec81053f0 0x7f3ec8100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:06.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.778+0000 7f3ec7e00640 1 -- 172.21.15.69:0/3679096262 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ec8102ac0 msgr2=0x7f3ec8102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:06.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.778+0000 7f3ec7e00640 1 --2- 172.21.15.69:0/3679096262 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ec8102ac0 0x7f3ec8102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.778+0000 7f3ec7e00640 1 -- 172.21.15.69:0/3679096262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ec81053f0 msgr2=0x7f3ec8100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:06.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.778+0000 7f3ec7e00640 1 --2- 172.21.15.69:0/3679096262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ec81053f0 0x7f3ec8100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:06.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.778+0000 7f3ec7e00640 1 -- 172.21.15.69:0/3679096262 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec8101840 con 0x7f3ec8107d20 2024-04-24T09:07:06.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.778+0000 7f3ec6a00640 1 --2- 172.21.15.69:0/3679096262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ec81053f0 0x7f3ec8100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:06.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.778+0000 7f3ec7e00640 1 --2- 172.21.15.69:0/3679096262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ec8107d20 0x7f3ec8100fb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000eb10 tx=0x7f3ec000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:06.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.778+0000 7f3ebfe00640 1 -- 172.21.15.69:0/3679096262 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0004b20 con 0x7f3ec8107d20 2024-04-24T09:07:06.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.779+0000 7f3ebfe00640 1 -- 172.21.15.69:0/3679096262 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ec0004cc0 con 0x7f3ec8107d20 2024-04-24T09:07:06.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.779+0000 7f3ece400640 1 -- 172.21.15.69:0/3679096262 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec81a2400 con 0x7f3ec8107d20 2024-04-24T09:07:06.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.779+0000 7f3ebfe00640 1 -- 172.21.15.69:0/3679096262 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0018630 con 0x7f3ec8107d20 2024-04-24T09:07:06.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.779+0000 7f3ece400640 1 -- 172.21.15.69:0/3679096262 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec81a2900 con 0x7f3ec8107d20 2024-04-24T09:07:06.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.780+0000 7f3ece400640 1 -- 172.21.15.69:0/3679096262 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec80fdfa0 con 0x7f3ec8107d20 2024-04-24T09:07:06.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.782+0000 7f3ebfe00640 1 -- 172.21.15.69:0/3679096262 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f3ec0005000 con 0x7f3ec8107d20 2024-04-24T09:07:06.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.783+0000 7f3ebfe00640 1 --2- 172.21.15.69:0/3679096262 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f3eac0a3d20 0x7f3eac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:06.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.783+0000 7f3ebfe00640 1 -- 172.21.15.69:0/3679096262 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f3ec00d1330 con 0x7f3ec8107d20 2024-04-24T09:07:06.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.783+0000 7f3ec7400640 1 --2- 172.21.15.69:0/3679096262 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f3eac0a3d20 0x7f3eac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:06.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.784+0000 7f3ec7400640 1 --2- 172.21.15.69:0/3679096262 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f3eac0a3d20 0x7f3eac0a61e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8006010 tx=0x7f3eb8000bc0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:06.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:06.791+0000 7f3ebfe00640 1 -- 172.21.15.69:0/3679096262 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ec009d630 con 0x7f3ec8107d20 2024-04-24T09:07:06.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:06 smithi069 ceph-mon[28170]: [24/Apr/2024:09:07:05] ENGINE Bus STARTING 2024-04-24T09:07:06.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:06 smithi069 ceph-mon[28170]: [24/Apr/2024:09:07:05] ENGINE Serving on https://172.21.15.69:7150 2024-04-24T09:07:06.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:06 smithi069 ceph-mon[28170]: [24/Apr/2024:09:07:05] ENGINE Serving on http://172.21.15.69:8765 2024-04-24T09:07:06.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:06 smithi069 ceph-mon[28170]: [24/Apr/2024:09:07:05] ENGINE Bus STARTED 2024-04-24T09:07:06.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:06 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3788925650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:07:06.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:06 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/4244725658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:07:06.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:06 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/82116143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:07:06.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:06 smithi130 ceph-mon[31553]: [24/Apr/2024:09:07:05] ENGINE Bus STARTING 2024-04-24T09:07:06.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:06 smithi130 ceph-mon[31553]: [24/Apr/2024:09:07:05] ENGINE Serving on https://172.21.15.69:7150 2024-04-24T09:07:06.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:06 smithi130 ceph-mon[31553]: [24/Apr/2024:09:07:05] ENGINE Serving on http://172.21.15.69:8765 2024-04-24T09:07:06.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:06 smithi130 ceph-mon[31553]: [24/Apr/2024:09:07:05] ENGINE Bus STARTED 2024-04-24T09:07:06.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:06 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3788925650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:07:06.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:06 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/4244725658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:07:06.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:06 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/82116143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:07:07.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.010+0000 7f3ece400640 1 -- 172.21.15.69:0/3679096262 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3ec8062c00 con 0x7f3ec8107d20 2024-04-24T09:07:07.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.011+0000 7f3ebfe00640 1 -- 172.21.15.69:0/3679096262 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f3ec009cf30 con 0x7f3ec8107d20 2024-04-24T09:07:07.011 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:07:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.013+0000 7f3ece400640 1 -- 172.21.15.69:0/3679096262 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f3eac0a3d20 msgr2=0x7f3eac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.013+0000 7f3ece400640 1 --2- 172.21.15.69:0/3679096262 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f3eac0a3d20 0x7f3eac0a61e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8006010 tx=0x7f3eb8000bc0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.013+0000 7f3ece400640 1 -- 172.21.15.69:0/3679096262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ec8107d20 msgr2=0x7f3ec8100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:07.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.013+0000 7f3ece400640 1 --2- 172.21.15.69:0/3679096262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ec8107d20 0x7f3ec8100fb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000eb10 tx=0x7f3ec000c2d0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.014+0000 7f3ece400640 1 -- 172.21.15.69:0/3679096262 shutdown_connections 2024-04-24T09:07:07.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.014+0000 7f3ece400640 1 --2- 172.21.15.69:0/3679096262 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f3eac0a3d20 0x7f3eac0a61e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.014+0000 7f3ece400640 1 --2- 172.21.15.69:0/3679096262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ec8107d20 0x7f3ec8100fb0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.014+0000 7f3ece400640 1 --2- 172.21.15.69:0/3679096262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ec81053f0 0x7f3ec8100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.014+0000 7f3ece400640 1 --2- 172.21.15.69:0/3679096262 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ec8102ac0 0x7f3ec8102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.014+0000 7f3ece400640 1 -- 172.21.15.69:0/3679096262 >> 172.21.15.69:0/3679096262 conn(0x7f3ec80fc4b0 msgr2=0x7f3ec8104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:07.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.014+0000 7f3ece400640 1 -- 172.21.15.69:0/3679096262 shutdown_connections 2024-04-24T09:07:07.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.014+0000 7f3ece400640 1 -- 172.21.15.69:0/3679096262 wait complete. 2024-04-24T09:07:07.031 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-24T09:07:07.031 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:07:07.066 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:06 smithi081 ceph-mon[30961]: [24/Apr/2024:09:07:05] ENGINE Bus STARTING 2024-04-24T09:07:07.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:06 smithi081 ceph-mon[30961]: [24/Apr/2024:09:07:05] ENGINE Serving on https://172.21.15.69:7150 2024-04-24T09:07:07.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:06 smithi081 ceph-mon[30961]: [24/Apr/2024:09:07:05] ENGINE Serving on http://172.21.15.69:8765 2024-04-24T09:07:07.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:06 smithi081 ceph-mon[30961]: [24/Apr/2024:09:07:05] ENGINE Bus STARTED 2024-04-24T09:07:07.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:06 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3788925650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:07:07.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:06 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/4244725658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:07:07.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:06 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/82116143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:07:07.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.224+0000 7f1065c00640 1 -- 172.21.15.69:0/2540077461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1060100f00 msgr2=0x7f1060101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:07.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.224+0000 7f1065c00640 1 --2- 172.21.15.69:0/2540077461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1060100f00 0x7f1060101340 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1050009900 tx=0x7f105002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.224+0000 7f1065c00640 1 -- 172.21.15.69:0/2540077461 shutdown_connections 2024-04-24T09:07:07.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.224+0000 7f1065c00640 1 --2- 172.21.15.69:0/2540077461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1060102a90 0x7f10601093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.224+0000 7f1065c00640 1 --2- 172.21.15.69:0/2540077461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10601020d0 0x7f1060102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.224+0000 7f1065c00640 1 --2- 172.21.15.69:0/2540077461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1060100f00 0x7f1060101340 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.224+0000 7f1065c00640 1 -- 172.21.15.69:0/2540077461 >> 172.21.15.69:0/2540077461 conn(0x7f10600fc4e0 msgr2=0x7f10600fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:07.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.225+0000 7f1065c00640 1 -- 172.21.15.69:0/2540077461 shutdown_connections 2024-04-24T09:07:07.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.225+0000 7f1065c00640 1 -- 172.21.15.69:0/2540077461 wait complete. 2024-04-24T09:07:07.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.226+0000 7f1065c00640 1 Processor -- start 2024-04-24T09:07:07.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.226+0000 7f1065c00640 1 -- start start 2024-04-24T09:07:07.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.226+0000 7f1065c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1060100f00 0x7f10601982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:07.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.226+0000 7f1065c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10601020d0 0x7f1060198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:07.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.226+0000 7f1065c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1060102a90 0x7f106019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:07.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.226+0000 7f1065c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f106010dc00 con 0x7f1060102a90 2024-04-24T09:07:07.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.226+0000 7f1065c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f106010da80 con 0x7f1060100f00 2024-04-24T09:07:07.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.226+0000 7f1065c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f106010dd80 con 0x7f10601020d0 2024-04-24T09:07:07.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.226+0000 7f105f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1060102a90 0x7f106019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:07.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.226+0000 7f105ea00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1060100f00 0x7f10601982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:07.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.227+0000 7f105ea00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1060100f00 0x7f10601982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.69:40700/0 (socket says 172.21.15.69:40700) 2024-04-24T09:07:07.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.227+0000 7f105f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1060102a90 0x7f106019f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46032/0 (socket says 172.21.15.69:46032) 2024-04-24T09:07:07.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.227+0000 7f105ea00640 1 -- 172.21.15.69:0/2163779627 learned_addr learned my addr 172.21.15.69:0/2163779627 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:07.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.227+0000 7f105e000640 1 --2- 172.21.15.69:0/2163779627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10601020d0 0x7f1060198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:07.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.227+0000 7f105ea00640 1 -- 172.21.15.69:0/2163779627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10601020d0 msgr2=0x7f1060198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:07.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.227+0000 7f105ea00640 1 --2- 172.21.15.69:0/2163779627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10601020d0 0x7f1060198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.227+0000 7f105ea00640 1 -- 172.21.15.69:0/2163779627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1060102a90 msgr2=0x7f106019f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:07.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.227+0000 7f105ea00640 1 --2- 172.21.15.69:0/2163779627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1060102a90 0x7f106019f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.227+0000 7f105ea00640 1 -- 172.21.15.69:0/2163779627 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106019fee0 con 0x7f1060100f00 2024-04-24T09:07:07.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.227+0000 7f105ea00640 1 --2- 172.21.15.69:0/2163779627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1060100f00 0x7f10601982c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f10500314e0 tx=0x7f1050003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:07.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.228+0000 7f1057400640 1 -- 172.21.15.69:0/2163779627 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105003f070 con 0x7f1060100f00 2024-04-24T09:07:07.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.228+0000 7f1065c00640 1 -- 172.21.15.69:0/2163779627 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10601a0170 con 0x7f1060100f00 2024-04-24T09:07:07.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.228+0000 7f1057400640 1 -- 172.21.15.69:0/2163779627 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1050031e40 con 0x7f1060100f00 2024-04-24T09:07:07.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.228+0000 7f1057400640 1 -- 172.21.15.69:0/2163779627 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050035030 con 0x7f1060100f00 2024-04-24T09:07:07.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.228+0000 7f1065c00640 1 -- 172.21.15.69:0/2163779627 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10601a0660 con 0x7f1060100f00 2024-04-24T09:07:07.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.229+0000 7f1065c00640 1 -- 172.21.15.69:0/2163779627 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1060108360 con 0x7f1060100f00 2024-04-24T09:07:07.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.231+0000 7f1057400640 1 -- 172.21.15.69:0/2163779627 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f1050042070 con 0x7f1060100f00 2024-04-24T09:07:07.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.233+0000 7f1057400640 1 --2- 172.21.15.69:0/2163779627 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f10440a3d20 0x7f10440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:07.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.233+0000 7f1057400640 1 -- 172.21.15.69:0/2163779627 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f10500f1aa0 con 0x7f1060100f00 2024-04-24T09:07:07.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.233+0000 7f105e000640 1 --2- 172.21.15.69:0/2163779627 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f10440a3d20 0x7f10440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:07.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.234+0000 7f105e000640 1 --2- 172.21.15.69:0/2163779627 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f10440a3d20 0x7f10440a61e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f104c005fd0 tx=0x7f104c005e20 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:07.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.238+0000 7f1057400640 1 -- 172.21.15.69:0/2163779627 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10500bdda0 con 0x7f1060100f00 2024-04-24T09:07:07.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.490+0000 7f1065c00640 1 -- 172.21.15.69:0/2163779627 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f106002d620 con 0x7f1060100f00 2024-04-24T09:07:07.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.491+0000 7f1057400640 1 -- 172.21.15.69:0/2163779627 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11873 (secure 0 0 0) 0x7f10500bd6a0 con 0x7f1060100f00 2024-04-24T09:07:07.492 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:07:07.492 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:04:01.991559+0000","modified":"2024-04-24T09:07:02.537848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34532,"mds_1":34496,"mds_2":34460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34496":{"gid":34496,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34532":{"gid":34532,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34532,"qdb_cluster":[34532,34460,34496]},"id":1}]} 2024-04-24T09:07:07.492 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 24 2024-04-24T09:07:07.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.494+0000 7f1065c00640 1 -- 172.21.15.69:0/2163779627 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f10440a3d20 msgr2=0x7f10440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:07.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.494+0000 7f1065c00640 1 --2- 172.21.15.69:0/2163779627 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f10440a3d20 0x7f10440a61e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f104c005fd0 tx=0x7f104c005e20 comp rx=0 tx=0).stop 2024-04-24T09:07:07.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.494+0000 7f1065c00640 1 -- 172.21.15.69:0/2163779627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1060100f00 msgr2=0x7f10601982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:07.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.494+0000 7f1065c00640 1 --2- 172.21.15.69:0/2163779627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1060100f00 0x7f10601982c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f10500314e0 tx=0x7f1050003c80 comp rx=0 tx=0).stop 2024-04-24T09:07:07.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.494+0000 7f1065c00640 1 -- 172.21.15.69:0/2163779627 shutdown_connections 2024-04-24T09:07:07.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.494+0000 7f1065c00640 1 --2- 172.21.15.69:0/2163779627 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f10440a3d20 0x7f10440a61e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.494+0000 7f1065c00640 1 --2- 172.21.15.69:0/2163779627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1060102a90 0x7f106019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.494+0000 7f1065c00640 1 --2- 172.21.15.69:0/2163779627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10601020d0 0x7f1060198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.494+0000 7f1065c00640 1 --2- 172.21.15.69:0/2163779627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1060100f00 0x7f10601982c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.494+0000 7f1065c00640 1 -- 172.21.15.69:0/2163779627 >> 172.21.15.69:0/2163779627 conn(0x7f10600fc4e0 msgr2=0x7f1060105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:07.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.494+0000 7f1065c00640 1 -- 172.21.15.69:0/2163779627 shutdown_connections 2024-04-24T09:07:07.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.494+0000 7f1065c00640 1 -- 172.21.15.69:0/2163779627 wait complete. 2024-04-24T09:07:07.513 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-24T09:07:07.514 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-24T09:07:07.514 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T09:07:07.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.705+0000 7f7bdfc00640 1 -- 172.21.15.69:0/2094924983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bd80769c0 msgr2=0x7f7bd8076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:07.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.705+0000 7f7bdfc00640 1 --2- 172.21.15.69:0/2094924983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bd80769c0 0x7f7bd8076de0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009900 tx=0x7f7bcc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.705+0000 7f7bdfc00640 1 -- 172.21.15.69:0/2094924983 shutdown_connections 2024-04-24T09:07:07.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.705+0000 7f7bdfc00640 1 --2- 172.21.15.69:0/2094924983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7bd80785a0 0x7f7bd8078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.705+0000 7f7bdfc00640 1 --2- 172.21.15.69:0/2094924983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bd8077c00 0x7f7bd8078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.705+0000 7f7bdfc00640 1 --2- 172.21.15.69:0/2094924983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bd80769c0 0x7f7bd8076de0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.705+0000 7f7bdfc00640 1 -- 172.21.15.69:0/2094924983 >> 172.21.15.69:0/2094924983 conn(0x7f7bd80ff7f0 msgr2=0x7f7bd8101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:07.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.705+0000 7f7bdfc00640 1 -- 172.21.15.69:0/2094924983 shutdown_connections 2024-04-24T09:07:07.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.705+0000 7f7bdfc00640 1 -- 172.21.15.69:0/2094924983 wait complete. 2024-04-24T09:07:07.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.707+0000 7f7bdfc00640 1 Processor -- start 2024-04-24T09:07:07.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.707+0000 7f7bdfc00640 1 -- start start 2024-04-24T09:07:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.707+0000 7f7bdfc00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bd80769c0 0x7f7bd819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.707+0000 7f7bdfc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bd8077c00 0x7f7bd819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.707+0000 7f7bdfc00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7bd80785a0 0x7f7bd81a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.707+0000 7f7bdfc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd8114200 con 0x7f7bd8077c00 2024-04-24T09:07:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.707+0000 7f7bdfc00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd8114080 con 0x7f7bd80769c0 2024-04-24T09:07:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.707+0000 7f7bdfc00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd8114380 con 0x7f7bd80785a0 2024-04-24T09:07:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.707+0000 7f7bd7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bd8077c00 0x7f7bd819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.707+0000 7f7bd7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bd8077c00 0x7f7bd819edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46056/0 (socket says 172.21.15.69:46056) 2024-04-24T09:07:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.707+0000 7f7bd7e00640 1 -- 172.21.15.69:0/3115941210 learned_addr learned my addr 172.21.15.69:0/3115941210 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.708+0000 7f7bddc00640 1 --2- 172.21.15.69:0/3115941210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7bd80785a0 0x7f7bd81a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.708+0000 7f7bdd200640 1 --2- 172.21.15.69:0/3115941210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bd80769c0 0x7f7bd819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.708+0000 7f7bd7e00640 1 -- 172.21.15.69:0/3115941210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7bd80785a0 msgr2=0x7f7bd81a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.708+0000 7f7bd7e00640 1 --2- 172.21.15.69:0/3115941210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7bd80785a0 0x7f7bd81a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.708+0000 7f7bd7e00640 1 -- 172.21.15.69:0/3115941210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bd80769c0 msgr2=0x7f7bd819e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:07.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.708+0000 7f7bd7e00640 1 --2- 172.21.15.69:0/3115941210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bd80769c0 0x7f7bd819e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.708+0000 7f7bd7e00640 1 -- 172.21.15.69:0/3115941210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd81a64a0 con 0x7f7bd8077c00 2024-04-24T09:07:07.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.708+0000 7f7bd7e00640 1 --2- 172.21.15.69:0/3115941210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bd8077c00 0x7f7bd819edc0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400e9f0 tx=0x7f7bc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:07.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.709+0000 7f7bd5600640 1 -- 172.21.15.69:0/3115941210 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc400cd80 con 0x7f7bd8077c00 2024-04-24T09:07:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.709+0000 7f7bd5600640 1 -- 172.21.15.69:0/3115941210 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bc40108e0 con 0x7f7bd8077c00 2024-04-24T09:07:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.709+0000 7f7bd5600640 1 -- 172.21.15.69:0/3115941210 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4018920 con 0x7f7bd8077c00 2024-04-24T09:07:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.709+0000 7f7bdfc00640 1 -- 172.21.15.69:0/3115941210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd81a6790 con 0x7f7bd8077c00 2024-04-24T09:07:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.709+0000 7f7bdfc00640 1 -- 172.21.15.69:0/3115941210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd81a6ce0 con 0x7f7bd8077c00 2024-04-24T09:07:07.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.710+0000 7f7bdfc00640 1 -- 172.21.15.69:0/3115941210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd8110dd0 con 0x7f7bd8077c00 2024-04-24T09:07:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.713+0000 7f7bd5600640 1 -- 172.21.15.69:0/3115941210 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f7bc4010a90 con 0x7f7bd8077c00 2024-04-24T09:07:07.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.715+0000 7f7bd5600640 1 --2- 172.21.15.69:0/3115941210 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7bc00a3d20 0x7f7bc00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:07.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.715+0000 7f7bd5600640 1 -- 172.21.15.69:0/3115941210 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7bc40d0f30 con 0x7f7bd8077c00 2024-04-24T09:07:07.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.715+0000 7f7bdd200640 1 --2- 172.21.15.69:0/3115941210 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7bc00a3d20 0x7f7bc00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:07.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.716+0000 7f7bdd200640 1 --2- 172.21.15.69:0/3115941210 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7bc00a3d20 0x7f7bc00a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc000c00 tx=0x7f7bcc0314a0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:07.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.720+0000 7f7bd5600640 1 -- 172.21.15.69:0/3115941210 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7bc409d230 con 0x7f7bd8077c00 2024-04-24T09:07:07.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:07 smithi069 ceph-mon[28170]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:07:07.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:07 smithi069 ceph-mon[28170]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-24T09:07:07.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:07 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3679096262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:07:07.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:07 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:07.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:07 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:07.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:07 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:07.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:07 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:07.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:07 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2163779627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:07:07.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.921+0000 7f7bdfc00640 1 -- 172.21.15.69:0/3115941210 --> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7bd8062440 con 0x7f7bc00a3d20 2024-04-24T09:07:07.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:07 smithi130 ceph-mon[31553]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:07:07.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:07 smithi130 ceph-mon[31553]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-24T09:07:07.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:07 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3679096262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:07:07.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:07 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:07.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:07 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:07.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:07 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:07.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:07 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:07.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:07 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2163779627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:07:07.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.956+0000 7f7bd5600640 1 -- 172.21.15.69:0/3115941210 <== mgr.15465 v2:172.21.15.69:6800/2822728880 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339473 (secure 0 0 0) 0x7f7bd8062440 con 0x7f7bc00a3d20 2024-04-24T09:07:07.961 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:07:07.963 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-24T09:07:07.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.961+0000 7f7bdfc00640 1 -- 172.21.15.69:0/3115941210 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7bc00a3d20 msgr2=0x7f7bc00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:07.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.961+0000 7f7bdfc00640 1 --2- 172.21.15.69:0/3115941210 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7bc00a3d20 0x7f7bc00a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc000c00 tx=0x7f7bcc0314a0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.961+0000 7f7bdfc00640 1 -- 172.21.15.69:0/3115941210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bd8077c00 msgr2=0x7f7bd819edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:07.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.961+0000 7f7bdfc00640 1 --2- 172.21.15.69:0/3115941210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bd8077c00 0x7f7bd819edc0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400e9f0 tx=0x7f7bc400eeb0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.961+0000 7f7bdfc00640 1 -- 172.21.15.69:0/3115941210 shutdown_connections 2024-04-24T09:07:07.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.961+0000 7f7bdfc00640 1 --2- 172.21.15.69:0/3115941210 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7bc00a3d20 0x7f7bc00a61e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.961+0000 7f7bdfc00640 1 --2- 172.21.15.69:0/3115941210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7bd80785a0 0x7f7bd81a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.961+0000 7f7bdfc00640 1 --2- 172.21.15.69:0/3115941210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bd8077c00 0x7f7bd819edc0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.961+0000 7f7bdfc00640 1 --2- 172.21.15.69:0/3115941210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bd80769c0 0x7f7bd819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:07.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.961+0000 7f7bdfc00640 1 -- 172.21.15.69:0/3115941210 >> 172.21.15.69:0/3115941210 conn(0x7f7bd80ff7f0 msgr2=0x7f7bd8100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:07.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.961+0000 7f7bdfc00640 1 -- 172.21.15.69:0/3115941210 shutdown_connections 2024-04-24T09:07:07.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:07.961+0000 7f7bdfc00640 1 -- 172.21.15.69:0/3115941210 wait complete. 2024-04-24T09:07:07.978 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-24T09:07:07.901521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193204,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1163,"num_read_kb":747,"num_write":483,"num_write_kb":4390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":877,"ondisk_log_size":877,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356972,"kb_used_data":15940,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124584852,"statfs":{"total":1151940427776,"available":1151574888448,"internally_reserved":0,"allocated":16322560,"data_stored":9719928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.983030"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031762+0000","last_change":"2024-04-24T09:04:02.107965+0000","last_active":"2024-04-24T09:07:04.031762+0000","last_peered":"2024-04-24T09:07:04.031762+0000","last_clean":"2024-04-24T09:07:04.031762+0000","last_became_active":"2024-04-24T09:04:02.106827+0000","last_became_peered":"2024-04-24T09:04:02.106827+0000","last_unstale":"2024-04-24T09:07:04.031762+0000","last_undegraded":"2024-04-24T09:07:04.031762+0000","last_fullsized":"2024-04-24T09:07:04.031762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:20:23.930599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341563+0000","last_change":"2024-04-24T09:04:02.105449+0000","last_active":"2024-04-24T09:07:04.341563+0000","last_peered":"2024-04-24T09:07:04.341563+0000","last_clean":"2024-04-24T09:07:04.341563+0000","last_became_active":"2024-04-24T09:04:02.101842+0000","last_became_peered":"2024-04-24T09:04:02.101842+0000","last_unstale":"2024-04-24T09:07:04.341563+0000","last_undegraded":"2024-04-24T09:07:04.341563+0000","last_fullsized":"2024-04-24T09:07:04.341563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:46.911768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341388+0000","last_change":"2024-04-24T09:04:02.118686+0000","last_active":"2024-04-24T09:07:04.341388+0000","last_peered":"2024-04-24T09:07:04.341388+0000","last_clean":"2024-04-24T09:07:04.341388+0000","last_became_active":"2024-04-24T09:04:02.117549+0000","last_became_peered":"2024-04-24T09:04:02.117549+0000","last_unstale":"2024-04-24T09:07:04.341388+0000","last_undegraded":"2024-04-24T09:07:04.341388+0000","last_fullsized":"2024-04-24T09:07:04.341388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:18:16.233434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086531+0000","last_change":"2024-04-24T09:04:02.120336+0000","last_active":"2024-04-24T09:07:04.086531+0000","last_peered":"2024-04-24T09:07:04.086531+0000","last_clean":"2024-04-24T09:07:04.086531+0000","last_became_active":"2024-04-24T09:04:02.116574+0000","last_became_peered":"2024-04-24T09:04:02.116574+0000","last_unstale":"2024-04-24T09:07:04.086531+0000","last_undegraded":"2024-04-24T09:07:04.086531+0000","last_fullsized":"2024-04-24T09:07:04.086531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:10:42.366531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921771+0000","last_change":"2024-04-24T09:04:02.113600+0000","last_active":"2024-04-24T09:07:03.921771+0000","last_peered":"2024-04-24T09:07:03.921771+0000","last_clean":"2024-04-24T09:07:03.921771+0000","last_became_active":"2024-04-24T09:04:02.113143+0000","last_became_peered":"2024-04-24T09:04:02.113143+0000","last_unstale":"2024-04-24T09:07:03.921771+0000","last_undegraded":"2024-04-24T09:07:03.921771+0000","last_fullsized":"2024-04-24T09:07:03.921771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:31:27.666661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.882380+0000","last_change":"2024-04-24T09:04:02.106395+0000","last_active":"2024-04-24T09:07:03.882380+0000","last_peered":"2024-04-24T09:07:03.882380+0000","last_clean":"2024-04-24T09:07:03.882380+0000","last_became_active":"2024-04-24T09:04:02.105942+0000","last_became_peered":"2024-04-24T09:04:02.105942+0000","last_unstale":"2024-04-24T09:07:03.882380+0000","last_undegraded":"2024-04-24T09:07:03.882380+0000","last_fullsized":"2024-04-24T09:07:03.882380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:20:10.655605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341117+0000","last_change":"2024-04-24T09:04:02.100279+0000","last_active":"2024-04-24T09:07:04.341117+0000","last_peered":"2024-04-24T09:07:04.341117+0000","last_clean":"2024-04-24T09:07:04.341117+0000","last_became_active":"2024-04-24T09:04:02.099745+0000","last_became_peered":"2024-04-24T09:04:02.099745+0000","last_unstale":"2024-04-24T09:07:04.341117+0000","last_undegraded":"2024-04-24T09:07:04.341117+0000","last_fullsized":"2024-04-24T09:07:04.341117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:13.150412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029846+0000","last_change":"2024-04-24T09:04:02.103294+0000","last_active":"2024-04-24T09:07:04.029846+0000","last_peered":"2024-04-24T09:07:04.029846+0000","last_clean":"2024-04-24T09:07:04.029846+0000","last_became_active":"2024-04-24T09:04:02.102114+0000","last_became_peered":"2024-04-24T09:04:02.102114+0000","last_unstale":"2024-04-24T09:07:04.029846+0000","last_undegraded":"2024-04-24T09:07:04.029846+0000","last_fullsized":"2024-04-24T09:07:04.029846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:26:26.916097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342942+0000","last_change":"2024-04-24T09:04:02.104936+0000","last_active":"2024-04-24T09:07:04.342942+0000","last_peered":"2024-04-24T09:07:04.342942+0000","last_clean":"2024-04-24T09:07:04.342942+0000","last_became_active":"2024-04-24T09:04:02.104540+0000","last_became_peered":"2024-04-24T09:04:02.104540+0000","last_unstale":"2024-04-24T09:07:04.342942+0000","last_undegraded":"2024-04-24T09:07:04.342942+0000","last_fullsized":"2024-04-24T09:07:04.342942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:36.010735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883910+0000","last_change":"2024-04-24T09:04:02.102050+0000","last_active":"2024-04-24T09:07:03.883910+0000","last_peered":"2024-04-24T09:07:03.883910+0000","last_clean":"2024-04-24T09:07:03.883910+0000","last_became_active":"2024-04-24T09:04:02.101623+0000","last_became_peered":"2024-04-24T09:04:02.101623+0000","last_unstale":"2024-04-24T09:07:03.883910+0000","last_undegraded":"2024-04-24T09:07:03.883910+0000","last_fullsized":"2024-04-24T09:07:03.883910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:13:33.606200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341928+0000","last_change":"2024-04-24T09:04:02.115804+0000","last_active":"2024-04-24T09:07:04.341928+0000","last_peered":"2024-04-24T09:07:04.341928+0000","last_clean":"2024-04-24T09:07:04.341928+0000","last_became_active":"2024-04-24T09:04:02.115093+0000","last_became_peered":"2024-04-24T09:04:02.115093+0000","last_unstale":"2024-04-24T09:07:04.341928+0000","last_undegraded":"2024-04-24T09:07:04.341928+0000","last_fullsized":"2024-04-24T09:07:04.341928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:26:56.764485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086355+0000","last_change":"2024-04-24T09:04:02.116305+0000","last_active":"2024-04-24T09:07:04.086355+0000","last_peered":"2024-04-24T09:07:04.086355+0000","last_clean":"2024-04-24T09:07:04.086355+0000","last_became_active":"2024-04-24T09:04:02.115956+0000","last_became_peered":"2024-04-24T09:04:02.115956+0000","last_unstale":"2024-04-24T09:07:04.086355+0000","last_undegraded":"2024-04-24T09:07:04.086355+0000","last_fullsized":"2024-04-24T09:07:04.086355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:38.618317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.877469+0000","last_change":"2024-04-24T09:04:02.101155+0000","last_active":"2024-04-24T09:07:03.877469+0000","last_peered":"2024-04-24T09:07:03.877469+0000","last_clean":"2024-04-24T09:07:03.877469+0000","last_became_active":"2024-04-24T09:04:02.099719+0000","last_became_peered":"2024-04-24T09:04:02.099719+0000","last_unstale":"2024-04-24T09:07:03.877469+0000","last_undegraded":"2024-04-24T09:07:03.877469+0000","last_fullsized":"2024-04-24T09:07:03.877469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:46:27.910987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.340928+0000","last_change":"2024-04-24T09:04:02.114818+0000","last_active":"2024-04-24T09:07:04.340928+0000","last_peered":"2024-04-24T09:07:04.340928+0000","last_clean":"2024-04-24T09:07:04.340928+0000","last_became_active":"2024-04-24T09:04:02.112947+0000","last_became_peered":"2024-04-24T09:04:02.112947+0000","last_unstale":"2024-04-24T09:07:04.340928+0000","last_undegraded":"2024-04-24T09:07:04.340928+0000","last_fullsized":"2024-04-24T09:07:04.340928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:16.544420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341005+0000","last_change":"2024-04-24T09:04:02.120672+0000","last_active":"2024-04-24T09:07:04.341005+0000","last_peered":"2024-04-24T09:07:04.341005+0000","last_clean":"2024-04-24T09:07:04.341005+0000","last_became_active":"2024-04-24T09:04:02.120378+0000","last_became_peered":"2024-04-24T09:04:02.120378+0000","last_unstale":"2024-04-24T09:07:04.341005+0000","last_undegraded":"2024-04-24T09:07:04.341005+0000","last_fullsized":"2024-04-24T09:07:04.341005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:17.781189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342217+0000","last_change":"2024-04-24T09:04:02.107022+0000","last_active":"2024-04-24T09:07:04.342217+0000","last_peered":"2024-04-24T09:07:04.342217+0000","last_clean":"2024-04-24T09:07:04.342217+0000","last_became_active":"2024-04-24T09:04:02.106645+0000","last_became_peered":"2024-04-24T09:04:02.106645+0000","last_unstale":"2024-04-24T09:07:04.342217+0000","last_undegraded":"2024-04-24T09:07:04.342217+0000","last_fullsized":"2024-04-24T09:07:04.342217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:47.217718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031638+0000","last_change":"2024-04-24T09:04:02.115511+0000","last_active":"2024-04-24T09:07:04.031638+0000","last_peered":"2024-04-24T09:07:04.031638+0000","last_clean":"2024-04-24T09:07:04.031638+0000","last_became_active":"2024-04-24T09:04:02.115197+0000","last_became_peered":"2024-04-24T09:04:02.115197+0000","last_unstale":"2024-04-24T09:07:04.031638+0000","last_undegraded":"2024-04-24T09:07:04.031638+0000","last_fullsized":"2024-04-24T09:07:04.031638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:18.883688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.026662+0000","last_change":"2024-04-24T09:04:02.102108+0000","last_active":"2024-04-24T09:07:04.026662+0000","last_peered":"2024-04-24T09:07:04.026662+0000","last_clean":"2024-04-24T09:07:04.026662+0000","last_became_active":"2024-04-24T09:04:02.101739+0000","last_became_peered":"2024-04-24T09:04:02.101739+0000","last_unstale":"2024-04-24T09:07:04.026662+0000","last_undegraded":"2024-04-24T09:07:04.026662+0000","last_fullsized":"2024-04-24T09:07:04.026662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:57:30.768716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343732+0000","last_change":"2024-04-24T09:04:02.108911+0000","last_active":"2024-04-24T09:07:04.343732+0000","last_peered":"2024-04-24T09:07:04.343732+0000","last_clean":"2024-04-24T09:07:04.343732+0000","last_became_active":"2024-04-24T09:04:02.105678+0000","last_became_peered":"2024-04-24T09:04:02.105678+0000","last_unstale":"2024-04-24T09:07:04.343732+0000","last_undegraded":"2024-04-24T09:07:04.343732+0000","last_fullsized":"2024-04-24T09:07:04.343732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:36:12.216271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343846+0000","last_change":"2024-04-24T09:04:06.136320+0000","last_active":"2024-04-24T09:07:04.343846+0000","last_peered":"2024-04-24T09:07:04.343846+0000","last_clean":"2024-04-24T09:07:04.343846+0000","last_became_active":"2024-04-24T09:04:06.133851+0000","last_became_peered":"2024-04-24T09:04:06.133851+0000","last_unstale":"2024-04-24T09:07:04.343846+0000","last_undegraded":"2024-04-24T09:07:04.343846+0000","last_fullsized":"2024-04-24T09:07:04.343846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:31:39.781234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.026762+0000","last_change":"2024-04-24T09:04:01.543818+0000","last_active":"2024-04-24T09:07:04.026762+0000","last_peered":"2024-04-24T09:07:04.026762+0000","last_clean":"2024-04-24T09:07:04.026762+0000","last_became_active":"2024-04-24T09:04:01.543435+0000","last_became_peered":"2024-04-24T09:04:01.543435+0000","last_unstale":"2024-04-24T09:07:04.026762+0000","last_undegraded":"2024-04-24T09:07:04.026762+0000","last_fullsized":"2024-04-24T09:07:04.026762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:08:30.176211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031206+0000","last_change":"2024-04-24T09:04:02.101646+0000","last_active":"2024-04-24T09:07:04.031206+0000","last_peered":"2024-04-24T09:07:04.031206+0000","last_clean":"2024-04-24T09:07:04.031206+0000","last_became_active":"2024-04-24T09:04:02.101284+0000","last_became_peered":"2024-04-24T09:04:02.101284+0000","last_unstale":"2024-04-24T09:07:04.031206+0000","last_undegraded":"2024-04-24T09:07:04.031206+0000","last_fullsized":"2024-04-24T09:07:04.031206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:36:32.827216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.085981+0000","last_change":"2024-04-24T09:04:06.130476+0000","last_active":"2024-04-24T09:07:04.085981+0000","last_peered":"2024-04-24T09:07:04.085981+0000","last_clean":"2024-04-24T09:07:04.085981+0000","last_became_active":"2024-04-24T09:04:06.129707+0000","last_became_peered":"2024-04-24T09:04:06.129707+0000","last_unstale":"2024-04-24T09:07:04.085981+0000","last_undegraded":"2024-04-24T09:07:04.085981+0000","last_fullsized":"2024-04-24T09:07:04.085981+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:25:36.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031243+0000","last_change":"2024-04-24T09:04:01.551708+0000","last_active":"2024-04-24T09:07:04.031243+0000","last_peered":"2024-04-24T09:07:04.031243+0000","last_clean":"2024-04-24T09:07:04.031243+0000","last_became_active":"2024-04-24T09:04:01.551089+0000","last_became_peered":"2024-04-24T09:04:01.551089+0000","last_unstale":"2024-04-24T09:07:04.031243+0000","last_undegraded":"2024-04-24T09:07:04.031243+0000","last_fullsized":"2024-04-24T09:07:04.031243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:18:22.515729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341702+0000","last_change":"2024-04-24T09:04:02.117356+0000","last_active":"2024-04-24T09:07:04.341702+0000","last_peered":"2024-04-24T09:07:04.341702+0000","last_clean":"2024-04-24T09:07:04.341702+0000","last_became_active":"2024-04-24T09:04:02.116972+0000","last_became_peered":"2024-04-24T09:04:02.116972+0000","last_unstale":"2024-04-24T09:07:04.341702+0000","last_undegraded":"2024-04-24T09:07:04.341702+0000","last_fullsized":"2024-04-24T09:07:04.341702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:46:29.349952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086191+0000","last_change":"2024-04-24T09:04:06.141538+0000","last_active":"2024-04-24T09:07:04.086191+0000","last_peered":"2024-04-24T09:07:04.086191+0000","last_clean":"2024-04-24T09:07:04.086191+0000","last_became_active":"2024-04-24T09:04:06.140975+0000","last_became_peered":"2024-04-24T09:04:06.140975+0000","last_unstale":"2024-04-24T09:07:04.086191+0000","last_undegraded":"2024-04-24T09:07:04.086191+0000","last_fullsized":"2024-04-24T09:07:04.086191+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:58:31.688828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.922715+0000","last_change":"2024-04-24T09:04:01.561039+0000","last_active":"2024-04-24T09:07:03.922715+0000","last_peered":"2024-04-24T09:07:03.922715+0000","last_clean":"2024-04-24T09:07:03.922715+0000","last_became_active":"2024-04-24T09:04:01.560594+0000","last_became_peered":"2024-04-24T09:04:01.560594+0000","last_unstale":"2024-04-24T09:07:03.922715+0000","last_undegraded":"2024-04-24T09:07:03.922715+0000","last_fullsized":"2024-04-24T09:07:03.922715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:59:19.707826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342953+0000","last_change":"2024-04-24T09:04:02.106007+0000","last_active":"2024-04-24T09:07:04.342953+0000","last_peered":"2024-04-24T09:07:04.342953+0000","last_clean":"2024-04-24T09:07:04.342953+0000","last_became_active":"2024-04-24T09:04:02.099786+0000","last_became_peered":"2024-04-24T09:04:02.099786+0000","last_unstale":"2024-04-24T09:07:04.342953+0000","last_undegraded":"2024-04-24T09:07:04.342953+0000","last_fullsized":"2024-04-24T09:07:04.342953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:56.288417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.085876+0000","last_change":"2024-04-24T09:04:06.143636+0000","last_active":"2024-04-24T09:07:04.085876+0000","last_peered":"2024-04-24T09:07:04.085876+0000","last_clean":"2024-04-24T09:07:04.085876+0000","last_became_active":"2024-04-24T09:04:06.143105+0000","last_became_peered":"2024-04-24T09:04:06.143105+0000","last_unstale":"2024-04-24T09:07:04.085876+0000","last_undegraded":"2024-04-24T09:07:04.085876+0000","last_fullsized":"2024-04-24T09:07:04.085876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:04:23.211067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.922078+0000","last_change":"2024-04-24T09:04:01.548403+0000","last_active":"2024-04-24T09:07:03.922078+0000","last_peered":"2024-04-24T09:07:03.922078+0000","last_clean":"2024-04-24T09:07:03.922078+0000","last_became_active":"2024-04-24T09:04:01.547447+0000","last_became_peered":"2024-04-24T09:04:01.547447+0000","last_unstale":"2024-04-24T09:07:03.922078+0000","last_undegraded":"2024-04-24T09:07:03.922078+0000","last_fullsized":"2024-04-24T09:07:03.922078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:14:08.160127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883567+0000","last_change":"2024-04-24T09:04:02.121410+0000","last_active":"2024-04-24T09:07:03.883567+0000","last_peered":"2024-04-24T09:07:03.883567+0000","last_clean":"2024-04-24T09:07:03.883567+0000","last_became_active":"2024-04-24T09:04:02.120821+0000","last_became_peered":"2024-04-24T09:04:02.120821+0000","last_unstale":"2024-04-24T09:07:03.883567+0000","last_undegraded":"2024-04-24T09:07:03.883567+0000","last_fullsized":"2024-04-24T09:07:03.883567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:39:05.525171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343281+0000","last_change":"2024-04-24T09:04:06.142349+0000","last_active":"2024-04-24T09:07:04.343281+0000","last_peered":"2024-04-24T09:07:04.343281+0000","last_clean":"2024-04-24T09:07:04.343281+0000","last_became_active":"2024-04-24T09:04:06.141911+0000","last_became_peered":"2024-04-24T09:04:06.141911+0000","last_unstale":"2024-04-24T09:07:04.343281+0000","last_undegraded":"2024-04-24T09:07:04.343281+0000","last_fullsized":"2024-04-24T09:07:04.343281+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:45:30.414686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":196,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.444417+0000","last_change":"2024-04-24T09:04:24.537349+0000","last_active":"2024-04-24T09:07:04.444417+0000","last_peered":"2024-04-24T09:07:04.444417+0000","last_clean":"2024-04-24T09:07:04.444417+0000","last_became_active":"2024-04-24T09:04:24.536738+0000","last_became_peered":"2024-04-24T09:04:24.536738+0000","last_unstale":"2024-04-24T09:07:04.444417+0000","last_undegraded":"2024-04-24T09:07:04.444417+0000","last_fullsized":"2024-04-24T09:07:04.444417+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:33.481380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1897,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":153,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921779+0000","last_change":"2024-04-24T09:04:02.111173+0000","last_active":"2024-04-24T09:07:03.921779+0000","last_peered":"2024-04-24T09:07:03.921779+0000","last_clean":"2024-04-24T09:07:03.921779+0000","last_became_active":"2024-04-24T09:04:02.110681+0000","last_became_peered":"2024-04-24T09:04:02.110681+0000","last_unstale":"2024-04-24T09:07:03.921779+0000","last_undegraded":"2024-04-24T09:07:03.921779+0000","last_fullsized":"2024-04-24T09:07:03.921779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:29.939116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343333+0000","last_change":"2024-04-24T09:04:06.137752+0000","last_active":"2024-04-24T09:07:04.343333+0000","last_peered":"2024-04-24T09:07:04.343333+0000","last_clean":"2024-04-24T09:07:04.343333+0000","last_became_active":"2024-04-24T09:04:06.137492+0000","last_became_peered":"2024-04-24T09:04:06.137492+0000","last_unstale":"2024-04-24T09:07:04.343333+0000","last_undegraded":"2024-04-24T09:07:04.343333+0000","last_fullsized":"2024-04-24T09:07:04.343333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:44:00.653864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.882216+0000","last_change":"2024-04-24T09:04:01.544009+0000","last_active":"2024-04-24T09:07:03.882216+0000","last_peered":"2024-04-24T09:07:03.882216+0000","last_clean":"2024-04-24T09:07:03.882216+0000","last_became_active":"2024-04-24T09:04:01.543651+0000","last_became_peered":"2024-04-24T09:04:01.543651+0000","last_unstale":"2024-04-24T09:07:03.882216+0000","last_undegraded":"2024-04-24T09:07:03.882216+0000","last_fullsized":"2024-04-24T09:07:03.882216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:10:21.969072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.881564+0000","last_change":"2024-04-24T09:04:02.115387+0000","last_active":"2024-04-24T09:07:03.881564+0000","last_peered":"2024-04-24T09:07:03.881564+0000","last_clean":"2024-04-24T09:07:03.881564+0000","last_became_active":"2024-04-24T09:04:02.115086+0000","last_became_peered":"2024-04-24T09:04:02.115086+0000","last_unstale":"2024-04-24T09:07:03.881564+0000","last_undegraded":"2024-04-24T09:07:03.881564+0000","last_fullsized":"2024-04-24T09:07:03.881564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:40:57.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031574+0000","last_change":"2024-04-24T09:04:06.146189+0000","last_active":"2024-04-24T09:07:04.031574+0000","last_peered":"2024-04-24T09:07:04.031574+0000","last_clean":"2024-04-24T09:07:04.031574+0000","last_became_active":"2024-04-24T09:04:06.145556+0000","last_became_peered":"2024-04-24T09:04:06.145556+0000","last_unstale":"2024-04-24T09:07:04.031574+0000","last_undegraded":"2024-04-24T09:07:04.031574+0000","last_fullsized":"2024-04-24T09:07:04.031574+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:41.371948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.874786+0000","last_change":"2024-04-24T09:04:01.542185+0000","last_active":"2024-04-24T09:07:03.874786+0000","last_peered":"2024-04-24T09:07:03.874786+0000","last_clean":"2024-04-24T09:07:03.874786+0000","last_became_active":"2024-04-24T09:04:01.541240+0000","last_became_peered":"2024-04-24T09:04:01.541240+0000","last_unstale":"2024-04-24T09:07:03.874786+0000","last_undegraded":"2024-04-24T09:07:03.874786+0000","last_fullsized":"2024-04-24T09:07:03.874786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:15.103276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029990+0000","last_change":"2024-04-24T09:04:02.104054+0000","last_active":"2024-04-24T09:07:04.029990+0000","last_peered":"2024-04-24T09:07:04.029990+0000","last_clean":"2024-04-24T09:07:04.029990+0000","last_became_active":"2024-04-24T09:04:02.103673+0000","last_became_peered":"2024-04-24T09:04:02.103673+0000","last_unstale":"2024-04-24T09:07:04.029990+0000","last_undegraded":"2024-04-24T09:07:04.029990+0000","last_fullsized":"2024-04-24T09:07:04.029990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:31:58.889989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341462+0000","last_change":"2024-04-24T09:04:06.134034+0000","last_active":"2024-04-24T09:07:04.341462+0000","last_peered":"2024-04-24T09:07:04.341462+0000","last_clean":"2024-04-24T09:07:04.341462+0000","last_became_active":"2024-04-24T09:04:06.133596+0000","last_became_peered":"2024-04-24T09:04:06.133596+0000","last_unstale":"2024-04-24T09:07:04.341462+0000","last_undegraded":"2024-04-24T09:07:04.341462+0000","last_fullsized":"2024-04-24T09:07:04.341462+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:17:09.572974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880523+0000","last_change":"2024-04-24T09:04:01.543312+0000","last_active":"2024-04-24T09:07:03.880523+0000","last_peered":"2024-04-24T09:07:03.880523+0000","last_clean":"2024-04-24T09:07:03.880523+0000","last_became_active":"2024-04-24T09:04:01.542914+0000","last_became_peered":"2024-04-24T09:04:01.542914+0000","last_unstale":"2024-04-24T09:07:03.880523+0000","last_undegraded":"2024-04-24T09:07:03.880523+0000","last_fullsized":"2024-04-24T09:07:03.880523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:50.695108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.025810+0000","last_change":"2024-04-24T09:04:02.114445+0000","last_active":"2024-04-24T09:07:04.025810+0000","last_peered":"2024-04-24T09:07:04.025810+0000","last_clean":"2024-04-24T09:07:04.025810+0000","last_became_active":"2024-04-24T09:04:02.114075+0000","last_became_peered":"2024-04-24T09:04:02.114075+0000","last_unstale":"2024-04-24T09:07:04.025810+0000","last_undegraded":"2024-04-24T09:07:04.025810+0000","last_fullsized":"2024-04-24T09:07:04.025810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:35:06.933677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029729+0000","last_change":"2024-04-24T09:04:06.142368+0000","last_active":"2024-04-24T09:07:04.029729+0000","last_peered":"2024-04-24T09:07:04.029729+0000","last_clean":"2024-04-24T09:07:04.029729+0000","last_became_active":"2024-04-24T09:04:06.141872+0000","last_became_peered":"2024-04-24T09:04:06.141872+0000","last_unstale":"2024-04-24T09:07:04.029729+0000","last_undegraded":"2024-04-24T09:07:04.029729+0000","last_fullsized":"2024-04-24T09:07:04.029729+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:13:43.733662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":184,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883924+0000","last_change":"2024-04-24T09:04:01.556002+0000","last_active":"2024-04-24T09:07:03.883924+0000","last_peered":"2024-04-24T09:07:03.883924+0000","last_clean":"2024-04-24T09:07:03.883924+0000","last_became_active":"2024-04-24T09:04:01.555643+0000","last_became_peered":"2024-04-24T09:04:01.555643+0000","last_unstale":"2024-04-24T09:07:03.883924+0000","last_undegraded":"2024-04-24T09:07:03.883924+0000","last_fullsized":"2024-04-24T09:07:03.883924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:30:31.171184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":150,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342926+0000","last_change":"2024-04-24T09:04:02.104252+0000","last_active":"2024-04-24T09:07:04.342926+0000","last_peered":"2024-04-24T09:07:04.342926+0000","last_clean":"2024-04-24T09:07:04.342926+0000","last_became_active":"2024-04-24T09:04:02.099264+0000","last_became_peered":"2024-04-24T09:04:02.099264+0000","last_unstale":"2024-04-24T09:07:04.342926+0000","last_undegraded":"2024-04-24T09:07:04.342926+0000","last_fullsized":"2024-04-24T09:07:04.342926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:28.469814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.025340+0000","last_change":"2024-04-24T09:04:06.132407+0000","last_active":"2024-04-24T09:07:04.025340+0000","last_peered":"2024-04-24T09:07:04.025340+0000","last_clean":"2024-04-24T09:07:04.025340+0000","last_became_active":"2024-04-24T09:04:06.131997+0000","last_became_peered":"2024-04-24T09:04:06.131997+0000","last_unstale":"2024-04-24T09:07:04.025340+0000","last_undegraded":"2024-04-24T09:07:04.025340+0000","last_fullsized":"2024-04-24T09:07:04.025340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:08:07.369326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341438+0000","last_change":"2024-04-24T09:04:01.561039+0000","last_active":"2024-04-24T09:07:04.341438+0000","last_peered":"2024-04-24T09:07:04.341438+0000","last_clean":"2024-04-24T09:07:04.341438+0000","last_became_active":"2024-04-24T09:04:01.560543+0000","last_became_peered":"2024-04-24T09:04:01.560543+0000","last_unstale":"2024-04-24T09:07:04.341438+0000","last_undegraded":"2024-04-24T09:07:04.341438+0000","last_fullsized":"2024-04-24T09:07:04.341438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:27.706982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341945+0000","last_change":"2024-04-24T09:04:02.110850+0000","last_active":"2024-04-24T09:07:04.341945+0000","last_peered":"2024-04-24T09:07:04.341945+0000","last_clean":"2024-04-24T09:07:04.341945+0000","last_became_active":"2024-04-24T09:04:02.110512+0000","last_became_peered":"2024-04-24T09:04:02.110512+0000","last_unstale":"2024-04-24T09:07:04.341945+0000","last_undegraded":"2024-04-24T09:07:04.341945+0000","last_fullsized":"2024-04-24T09:07:04.341945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:19:16.937202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086513+0000","last_change":"2024-04-24T09:04:06.135200+0000","last_active":"2024-04-24T09:07:04.086513+0000","last_peered":"2024-04-24T09:07:04.086513+0000","last_clean":"2024-04-24T09:07:04.086513+0000","last_became_active":"2024-04-24T09:04:06.134680+0000","last_became_peered":"2024-04-24T09:04:06.134680+0000","last_unstale":"2024-04-24T09:07:04.086513+0000","last_undegraded":"2024-04-24T09:07:04.086513+0000","last_fullsized":"2024-04-24T09:07:04.086513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:48:57.697740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883243+0000","last_change":"2024-04-24T09:04:01.555069+0000","last_active":"2024-04-24T09:07:03.883243+0000","last_peered":"2024-04-24T09:07:03.883243+0000","last_clean":"2024-04-24T09:07:03.883243+0000","last_became_active":"2024-04-24T09:04:01.554737+0000","last_became_peered":"2024-04-24T09:04:01.554737+0000","last_unstale":"2024-04-24T09:07:03.883243+0000","last_undegraded":"2024-04-24T09:07:03.883243+0000","last_fullsized":"2024-04-24T09:07:03.883243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:23:00.272697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342607+0000","last_change":"2024-04-24T09:04:02.112126+0000","last_active":"2024-04-24T09:07:04.342607+0000","last_peered":"2024-04-24T09:07:04.342607+0000","last_clean":"2024-04-24T09:07:04.342607+0000","last_became_active":"2024-04-24T09:04:02.107995+0000","last_became_peered":"2024-04-24T09:04:02.107995+0000","last_unstale":"2024-04-24T09:07:04.342607+0000","last_undegraded":"2024-04-24T09:07:04.342607+0000","last_fullsized":"2024-04-24T09:07:04.342607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:13.189829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.881797+0000","last_change":"2024-04-24T09:04:24.542302+0000","last_active":"2024-04-24T09:07:03.881797+0000","last_peered":"2024-04-24T09:07:03.881797+0000","last_clean":"2024-04-24T09:07:03.881797+0000","last_became_active":"2024-04-24T09:04:24.539848+0000","last_became_peered":"2024-04-24T09:04:24.539848+0000","last_unstale":"2024-04-24T09:07:03.881797+0000","last_undegraded":"2024-04-24T09:07:03.881797+0000","last_fullsized":"2024-04-24T09:07:03.881797+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:01:52.055510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"93'46","reported_seq":389,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.442501+0000","last_change":"2024-04-24T09:04:01.548610+0000","last_active":"2024-04-24T09:07:04.442501+0000","last_peered":"2024-04-24T09:07:04.442501+0000","last_clean":"2024-04-24T09:07:04.442501+0000","last_became_active":"2024-04-24T09:04:01.548145+0000","last_became_peered":"2024-04-24T09:04:01.548145+0000","last_unstale":"2024-04-24T09:07:04.442501+0000","last_undegraded":"2024-04-24T09:07:04.442501+0000","last_fullsized":"2024-04-24T09:07:04.442501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:58:48.425187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71876,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":313,"num_read_kb":227,"num_write":47,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.922226+0000","last_change":"2024-04-24T09:04:02.109965+0000","last_active":"2024-04-24T09:07:03.922226+0000","last_peered":"2024-04-24T09:07:03.922226+0000","last_clean":"2024-04-24T09:07:03.922226+0000","last_became_active":"2024-04-24T09:04:02.108143+0000","last_became_peered":"2024-04-24T09:04:02.108143+0000","last_unstale":"2024-04-24T09:07:03.922226+0000","last_undegraded":"2024-04-24T09:07:03.922226+0000","last_fullsized":"2024-04-24T09:07:03.922226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:01:21.083039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.884532+0000","last_change":"2024-04-24T09:04:06.145369+0000","last_active":"2024-04-24T09:07:03.884532+0000","last_peered":"2024-04-24T09:07:03.884532+0000","last_clean":"2024-04-24T09:07:03.884532+0000","last_became_active":"2024-04-24T09:04:06.144884+0000","last_became_peered":"2024-04-24T09:04:06.144884+0000","last_unstale":"2024-04-24T09:07:03.884532+0000","last_undegraded":"2024-04-24T09:07:03.884532+0000","last_fullsized":"2024-04-24T09:07:03.884532+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:29:22.780846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343099+0000","last_change":"2024-04-24T09:04:01.548195+0000","last_active":"2024-04-24T09:07:04.343099+0000","last_peered":"2024-04-24T09:07:04.343099+0000","last_clean":"2024-04-24T09:07:04.343099+0000","last_became_active":"2024-04-24T09:04:01.544969+0000","last_became_peered":"2024-04-24T09:04:01.544969+0000","last_unstale":"2024-04-24T09:07:04.343099+0000","last_undegraded":"2024-04-24T09:07:04.343099+0000","last_fullsized":"2024-04-24T09:07:04.343099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:32:55.294629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.875735+0000","last_change":"2024-04-24T09:04:02.103785+0000","last_active":"2024-04-24T09:07:03.875735+0000","last_peered":"2024-04-24T09:07:03.875735+0000","last_clean":"2024-04-24T09:07:03.875735+0000","last_became_active":"2024-04-24T09:04:02.103463+0000","last_became_peered":"2024-04-24T09:04:02.103463+0000","last_unstale":"2024-04-24T09:07:03.875735+0000","last_undegraded":"2024-04-24T09:07:03.875735+0000","last_fullsized":"2024-04-24T09:07:03.875735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:56.385382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031518+0000","last_change":"2024-04-24T09:04:06.149955+0000","last_active":"2024-04-24T09:07:04.031518+0000","last_peered":"2024-04-24T09:07:04.031518+0000","last_clean":"2024-04-24T09:07:04.031518+0000","last_became_active":"2024-04-24T09:04:06.149445+0000","last_became_peered":"2024-04-24T09:04:06.149445+0000","last_unstale":"2024-04-24T09:07:04.031518+0000","last_undegraded":"2024-04-24T09:07:04.031518+0000","last_fullsized":"2024-04-24T09:07:04.031518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:30:42.527007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.882015+0000","last_change":"2024-04-24T09:04:01.548740+0000","last_active":"2024-04-24T09:07:03.882015+0000","last_peered":"2024-04-24T09:07:03.882015+0000","last_clean":"2024-04-24T09:07:03.882015+0000","last_became_active":"2024-04-24T09:04:01.546060+0000","last_became_peered":"2024-04-24T09:04:01.546060+0000","last_unstale":"2024-04-24T09:07:03.882015+0000","last_undegraded":"2024-04-24T09:07:03.882015+0000","last_fullsized":"2024-04-24T09:07:03.882015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:09:16.301150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341787+0000","last_change":"2024-04-24T09:04:24.538871+0000","last_active":"2024-04-24T09:07:04.341787+0000","last_peered":"2024-04-24T09:07:04.341787+0000","last_clean":"2024-04-24T09:07:04.341787+0000","last_became_active":"2024-04-24T09:04:24.537039+0000","last_became_peered":"2024-04-24T09:04:24.537039+0000","last_unstale":"2024-04-24T09:07:04.341787+0000","last_undegraded":"2024-04-24T09:07:04.341787+0000","last_fullsized":"2024-04-24T09:07:04.341787+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:18.520407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.026577+0000","last_change":"2024-04-24T09:04:06.136973+0000","last_active":"2024-04-24T09:07:04.026577+0000","last_peered":"2024-04-24T09:07:04.026577+0000","last_clean":"2024-04-24T09:07:04.026577+0000","last_became_active":"2024-04-24T09:04:06.136405+0000","last_became_peered":"2024-04-24T09:04:06.136405+0000","last_unstale":"2024-04-24T09:07:04.026577+0000","last_undegraded":"2024-04-24T09:07:04.026577+0000","last_fullsized":"2024-04-24T09:07:04.026577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:07.653162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":191,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086263+0000","last_change":"2024-04-24T09:04:01.546853+0000","last_active":"2024-04-24T09:07:04.086263+0000","last_peered":"2024-04-24T09:07:04.086263+0000","last_clean":"2024-04-24T09:07:04.086263+0000","last_became_active":"2024-04-24T09:04:01.546441+0000","last_became_peered":"2024-04-24T09:04:01.546441+0000","last_unstale":"2024-04-24T09:07:04.086263+0000","last_undegraded":"2024-04-24T09:07:04.086263+0000","last_fullsized":"2024-04-24T09:07:04.086263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:23:56.333410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341712+0000","last_change":"2024-04-24T09:04:02.101158+0000","last_active":"2024-04-24T09:07:04.341712+0000","last_peered":"2024-04-24T09:07:04.341712+0000","last_clean":"2024-04-24T09:07:04.341712+0000","last_became_active":"2024-04-24T09:04:02.100832+0000","last_became_peered":"2024-04-24T09:04:02.100832+0000","last_unstale":"2024-04-24T09:07:04.341712+0000","last_undegraded":"2024-04-24T09:07:04.341712+0000","last_fullsized":"2024-04-24T09:07:04.341712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:05:34.080520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.878892+0000","last_change":"2024-04-24T09:04:06.149176+0000","last_active":"2024-04-24T09:07:03.878892+0000","last_peered":"2024-04-24T09:07:03.878892+0000","last_clean":"2024-04-24T09:07:03.878892+0000","last_became_active":"2024-04-24T09:04:06.145539+0000","last_became_peered":"2024-04-24T09:04:06.145539+0000","last_unstale":"2024-04-24T09:07:03.878892+0000","last_undegraded":"2024-04-24T09:07:03.878892+0000","last_fullsized":"2024-04-24T09:07:03.878892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:38.719787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921175+0000","last_change":"2024-04-24T09:04:01.555330+0000","last_active":"2024-04-24T09:07:03.921175+0000","last_peered":"2024-04-24T09:07:03.921175+0000","last_clean":"2024-04-24T09:07:03.921175+0000","last_became_active":"2024-04-24T09:04:01.552618+0000","last_became_peered":"2024-04-24T09:04:01.552618+0000","last_unstale":"2024-04-24T09:07:03.921175+0000","last_undegraded":"2024-04-24T09:07:03.921175+0000","last_fullsized":"2024-04-24T09:07:03.921175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:30.956657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880986+0000","last_change":"2024-04-24T09:04:02.116301+0000","last_active":"2024-04-24T09:07:03.880986+0000","last_peered":"2024-04-24T09:07:03.880986+0000","last_clean":"2024-04-24T09:07:03.880986+0000","last_became_active":"2024-04-24T09:04:02.116001+0000","last_became_peered":"2024-04-24T09:04:02.116001+0000","last_unstale":"2024-04-24T09:07:03.880986+0000","last_undegraded":"2024-04-24T09:07:03.880986+0000","last_fullsized":"2024-04-24T09:07:03.880986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:16:18.995067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880626+0000","last_change":"2024-04-24T09:04:06.135814+0000","last_active":"2024-04-24T09:07:03.880626+0000","last_peered":"2024-04-24T09:07:03.880626+0000","last_clean":"2024-04-24T09:07:03.880626+0000","last_became_active":"2024-04-24T09:04:06.135054+0000","last_became_peered":"2024-04-24T09:04:06.135054+0000","last_unstale":"2024-04-24T09:07:03.880626+0000","last_undegraded":"2024-04-24T09:07:03.880626+0000","last_fullsized":"2024-04-24T09:07:03.880626+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:51:31.150589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.922651+0000","last_change":"2024-04-24T09:04:01.553144+0000","last_active":"2024-04-24T09:07:03.922651+0000","last_peered":"2024-04-24T09:07:03.922651+0000","last_clean":"2024-04-24T09:07:03.922651+0000","last_became_active":"2024-04-24T09:04:01.550599+0000","last_became_peered":"2024-04-24T09:04:01.550599+0000","last_unstale":"2024-04-24T09:07:03.922651+0000","last_undegraded":"2024-04-24T09:07:03.922651+0000","last_fullsized":"2024-04-24T09:07:03.922651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:41.950209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031628+0000","last_change":"2024-04-24T09:04:02.117064+0000","last_active":"2024-04-24T09:07:04.031628+0000","last_peered":"2024-04-24T09:07:04.031628+0000","last_clean":"2024-04-24T09:07:04.031628+0000","last_became_active":"2024-04-24T09:04:02.114528+0000","last_became_peered":"2024-04-24T09:04:02.114528+0000","last_unstale":"2024-04-24T09:07:04.031628+0000","last_undegraded":"2024-04-24T09:07:04.031628+0000","last_fullsized":"2024-04-24T09:07:04.031628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:24:53.662571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921692+0000","last_change":"2024-04-24T09:04:24.701058+0000","last_active":"2024-04-24T09:07:03.921692+0000","last_peered":"2024-04-24T09:07:03.921692+0000","last_clean":"2024-04-24T09:07:03.921692+0000","last_became_active":"2024-04-24T09:04:24.699766+0000","last_became_peered":"2024-04-24T09:04:24.699766+0000","last_unstale":"2024-04-24T09:07:03.921692+0000","last_undegraded":"2024-04-24T09:07:03.921692+0000","last_fullsized":"2024-04-24T09:07:03.921692+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:57:48.491073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.026283+0000","last_change":"2024-04-24T09:04:24.701816+0000","last_active":"2024-04-24T09:07:04.026283+0000","last_peered":"2024-04-24T09:07:04.026283+0000","last_clean":"2024-04-24T09:07:04.026283+0000","last_became_active":"2024-04-24T09:04:24.700530+0000","last_became_peered":"2024-04-24T09:04:24.700530+0000","last_unstale":"2024-04-24T09:07:04.026283+0000","last_undegraded":"2024-04-24T09:07:04.026283+0000","last_fullsized":"2024-04-24T09:07:04.026283+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:24.614043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883310+0000","last_change":"2024-04-24T09:04:02.104602+0000","last_active":"2024-04-24T09:07:03.883310+0000","last_peered":"2024-04-24T09:07:03.883310+0000","last_clean":"2024-04-24T09:07:03.883310+0000","last_became_active":"2024-04-24T09:04:02.104188+0000","last_became_peered":"2024-04-24T09:04:02.104188+0000","last_unstale":"2024-04-24T09:07:03.883310+0000","last_undegraded":"2024-04-24T09:07:03.883310+0000","last_fullsized":"2024-04-24T09:07:03.883310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:00:50.286952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.876427+0000","last_change":"2024-04-24T09:04:06.134971+0000","last_active":"2024-04-24T09:07:03.876427+0000","last_peered":"2024-04-24T09:07:03.876427+0000","last_clean":"2024-04-24T09:07:03.876427+0000","last_became_active":"2024-04-24T09:04:06.134551+0000","last_became_peered":"2024-04-24T09:04:06.134551+0000","last_unstale":"2024-04-24T09:07:03.876427+0000","last_undegraded":"2024-04-24T09:07:03.876427+0000","last_fullsized":"2024-04-24T09:07:03.876427+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:40:41.766657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341353+0000","last_change":"2024-04-24T09:04:01.551117+0000","last_active":"2024-04-24T09:07:04.341353+0000","last_peered":"2024-04-24T09:07:04.341353+0000","last_clean":"2024-04-24T09:07:04.341353+0000","last_became_active":"2024-04-24T09:04:01.550646+0000","last_became_peered":"2024-04-24T09:04:01.550646+0000","last_unstale":"2024-04-24T09:07:04.341353+0000","last_undegraded":"2024-04-24T09:07:04.341353+0000","last_fullsized":"2024-04-24T09:07:04.341353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:29:03.821300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.882733+0000","last_change":"2024-04-24T09:04:02.107111+0000","last_active":"2024-04-24T09:07:03.882733+0000","last_peered":"2024-04-24T09:07:03.882733+0000","last_clean":"2024-04-24T09:07:03.882733+0000","last_became_active":"2024-04-24T09:04:02.106185+0000","last_became_peered":"2024-04-24T09:04:02.106185+0000","last_unstale":"2024-04-24T09:07:03.882733+0000","last_undegraded":"2024-04-24T09:07:03.882733+0000","last_fullsized":"2024-04-24T09:07:03.882733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:19:03.111407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.877283+0000","last_change":"2024-04-24T09:04:06.143984+0000","last_active":"2024-04-24T09:07:03.877283+0000","last_peered":"2024-04-24T09:07:03.877283+0000","last_clean":"2024-04-24T09:07:03.877283+0000","last_became_active":"2024-04-24T09:04:06.143503+0000","last_became_peered":"2024-04-24T09:04:06.143503+0000","last_unstale":"2024-04-24T09:07:03.877283+0000","last_undegraded":"2024-04-24T09:07:03.877283+0000","last_fullsized":"2024-04-24T09:07:03.877283+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:35:19.317771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880870+0000","last_change":"2024-04-24T09:04:02.111560+0000","last_active":"2024-04-24T09:07:03.880870+0000","last_peered":"2024-04-24T09:07:03.880870+0000","last_clean":"2024-04-24T09:07:03.880870+0000","last_became_active":"2024-04-24T09:04:02.109885+0000","last_became_peered":"2024-04-24T09:04:02.109885+0000","last_unstale":"2024-04-24T09:07:03.880870+0000","last_undegraded":"2024-04-24T09:07:03.880870+0000","last_fullsized":"2024-04-24T09:07:03.880870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:35.837621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":186,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.874716+0000","last_change":"2024-04-24T09:04:01.542530+0000","last_active":"2024-04-24T09:07:03.874716+0000","last_peered":"2024-04-24T09:07:03.874716+0000","last_clean":"2024-04-24T09:07:03.874716+0000","last_became_active":"2024-04-24T09:04:01.541702+0000","last_became_peered":"2024-04-24T09:04:01.541702+0000","last_unstale":"2024-04-24T09:07:03.874716+0000","last_undegraded":"2024-04-24T09:07:03.874716+0000","last_fullsized":"2024-04-24T09:07:03.874716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:28:29.420020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":150,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341380+0000","last_change":"2024-04-24T09:04:06.143004+0000","last_active":"2024-04-24T09:07:04.341380+0000","last_peered":"2024-04-24T09:07:04.341380+0000","last_clean":"2024-04-24T09:07:04.341380+0000","last_became_active":"2024-04-24T09:04:06.142444+0000","last_became_peered":"2024-04-24T09:04:06.142444+0000","last_unstale":"2024-04-24T09:07:04.341380+0000","last_undegraded":"2024-04-24T09:07:04.341380+0000","last_fullsized":"2024-04-24T09:07:04.341380+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:13.910612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.025614+0000","last_change":"2024-04-24T09:04:24.701330+0000","last_active":"2024-04-24T09:07:04.025614+0000","last_peered":"2024-04-24T09:07:04.025614+0000","last_clean":"2024-04-24T09:07:04.025614+0000","last_became_active":"2024-04-24T09:04:24.699988+0000","last_became_peered":"2024-04-24T09:04:24.699988+0000","last_unstale":"2024-04-24T09:07:04.025614+0000","last_undegraded":"2024-04-24T09:07:04.025614+0000","last_fullsized":"2024-04-24T09:07:04.025614+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:23:24.610671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086340+0000","last_change":"2024-04-24T09:04:01.541206+0000","last_active":"2024-04-24T09:07:04.086340+0000","last_peered":"2024-04-24T09:07:04.086340+0000","last_clean":"2024-04-24T09:07:04.086340+0000","last_became_active":"2024-04-24T09:04:01.539888+0000","last_became_peered":"2024-04-24T09:04:01.539888+0000","last_unstale":"2024-04-24T09:07:04.086340+0000","last_undegraded":"2024-04-24T09:07:04.086340+0000","last_fullsized":"2024-04-24T09:07:04.086340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:01.550209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341017+0000","last_change":"2024-04-24T09:02:22.629840+0000","last_active":"2024-04-24T09:07:04.341017+0000","last_peered":"2024-04-24T09:07:04.341017+0000","last_clean":"2024-04-24T09:07:04.341017+0000","last_became_active":"2024-04-24T09:02:22.628700+0000","last_became_peered":"2024-04-24T09:02:22.628700+0000","last_unstale":"2024-04-24T09:07:04.341017+0000","last_undegraded":"2024-04-24T09:07:04.341017+0000","last_fullsized":"2024-04-24T09:07:04.341017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:57:38.349295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:57:38.349295+0000","last_clean_scrub_stamp":"2024-04-24T08:57:38.349295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:54:11.409492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883073+0000","last_change":"2024-04-24T09:04:06.149548+0000","last_active":"2024-04-24T09:07:03.883073+0000","last_peered":"2024-04-24T09:07:03.883073+0000","last_clean":"2024-04-24T09:07:03.883073+0000","last_became_active":"2024-04-24T09:04:06.149214+0000","last_became_peered":"2024-04-24T09:04:06.149214+0000","last_unstale":"2024-04-24T09:07:03.883073+0000","last_undegraded":"2024-04-24T09:07:03.883073+0000","last_fullsized":"2024-04-24T09:07:03.883073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:03.801242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.874677+0000","last_change":"2024-04-24T09:04:02.110242+0000","last_active":"2024-04-24T09:07:03.874677+0000","last_peered":"2024-04-24T09:07:03.874677+0000","last_clean":"2024-04-24T09:07:03.874677+0000","last_became_active":"2024-04-24T09:04:02.109879+0000","last_became_peered":"2024-04-24T09:04:02.109879+0000","last_unstale":"2024-04-24T09:07:03.874677+0000","last_undegraded":"2024-04-24T09:07:03.874677+0000","last_fullsized":"2024-04-24T09:07:03.874677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:16.353861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341211+0000","last_change":"2024-04-24T09:04:01.550989+0000","last_active":"2024-04-24T09:07:04.341211+0000","last_peered":"2024-04-24T09:07:04.341211+0000","last_clean":"2024-04-24T09:07:04.341211+0000","last_became_active":"2024-04-24T09:04:01.550664+0000","last_became_peered":"2024-04-24T09:04:01.550664+0000","last_unstale":"2024-04-24T09:07:04.341211+0000","last_undegraded":"2024-04-24T09:07:04.341211+0000","last_fullsized":"2024-04-24T09:07:04.341211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:33:39.521951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.030916+0000","last_change":"2024-04-24T09:04:24.538207+0000","last_active":"2024-04-24T09:07:04.030916+0000","last_peered":"2024-04-24T09:07:04.030916+0000","last_clean":"2024-04-24T09:07:04.030916+0000","last_became_active":"2024-04-24T09:04:24.536894+0000","last_became_peered":"2024-04-24T09:04:24.536894+0000","last_unstale":"2024-04-24T09:07:04.030916+0000","last_undegraded":"2024-04-24T09:07:04.030916+0000","last_fullsized":"2024-04-24T09:07:04.030916+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:32:52.860371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.922335+0000","last_change":"2024-04-24T09:04:01.539415+0000","last_active":"2024-04-24T09:07:03.922335+0000","last_peered":"2024-04-24T09:07:03.922335+0000","last_clean":"2024-04-24T09:07:03.922335+0000","last_became_active":"2024-04-24T09:04:01.539020+0000","last_became_peered":"2024-04-24T09:04:01.539020+0000","last_unstale":"2024-04-24T09:07:03.922335+0000","last_undegraded":"2024-04-24T09:07:03.922335+0000","last_fullsized":"2024-04-24T09:07:03.922335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:12:48.728029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.877295+0000","last_change":"2024-04-24T09:04:02.097560+0000","last_active":"2024-04-24T09:07:03.877295+0000","last_peered":"2024-04-24T09:07:03.877295+0000","last_clean":"2024-04-24T09:07:03.877295+0000","last_became_active":"2024-04-24T09:04:02.095612+0000","last_became_peered":"2024-04-24T09:04:02.095612+0000","last_unstale":"2024-04-24T09:07:03.877295+0000","last_undegraded":"2024-04-24T09:07:03.877295+0000","last_fullsized":"2024-04-24T09:07:03.877295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:55:47.747294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029686+0000","last_change":"2024-04-24T09:04:06.137822+0000","last_active":"2024-04-24T09:07:04.029686+0000","last_peered":"2024-04-24T09:07:04.029686+0000","last_clean":"2024-04-24T09:07:04.029686+0000","last_became_active":"2024-04-24T09:04:06.137357+0000","last_became_peered":"2024-04-24T09:04:06.137357+0000","last_unstale":"2024-04-24T09:07:04.029686+0000","last_undegraded":"2024-04-24T09:07:04.029686+0000","last_fullsized":"2024-04-24T09:07:04.029686+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:55:13.119557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'46","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.275253+0000","last_change":"2024-04-24T09:04:24.544395+0000","last_active":"2024-04-24T09:07:04.275253+0000","last_peered":"2024-04-24T09:07:04.275253+0000","last_clean":"2024-04-24T09:07:04.275253+0000","last_became_active":"2024-04-24T09:04:24.535870+0000","last_became_peered":"2024-04-24T09:04:24.535870+0000","last_unstale":"2024-04-24T09:07:04.275253+0000","last_undegraded":"2024-04-24T09:07:04.275253+0000","last_fullsized":"2024-04-24T09:07:04.275253+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:29:21.271387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031426+0000","last_change":"2024-04-24T09:04:02.106107+0000","last_active":"2024-04-24T09:07:04.031426+0000","last_peered":"2024-04-24T09:07:04.031426+0000","last_clean":"2024-04-24T09:07:04.031426+0000","last_became_active":"2024-04-24T09:04:02.105734+0000","last_became_peered":"2024-04-24T09:04:02.105734+0000","last_unstale":"2024-04-24T09:07:04.031426+0000","last_undegraded":"2024-04-24T09:07:04.031426+0000","last_fullsized":"2024-04-24T09:07:04.031426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:15.372520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342631+0000","last_change":"2024-04-24T09:04:06.139040+0000","last_active":"2024-04-24T09:07:04.342631+0000","last_peered":"2024-04-24T09:07:04.342631+0000","last_clean":"2024-04-24T09:07:04.342631+0000","last_became_active":"2024-04-24T09:04:06.138532+0000","last_became_peered":"2024-04-24T09:04:06.138532+0000","last_unstale":"2024-04-24T09:07:04.342631+0000","last_undegraded":"2024-04-24T09:07:04.342631+0000","last_fullsized":"2024-04-24T09:07:04.342631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:28.686052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.087005+0000","last_change":"2024-04-24T09:04:01.546859+0000","last_active":"2024-04-24T09:07:04.087005+0000","last_peered":"2024-04-24T09:07:04.087005+0000","last_clean":"2024-04-24T09:07:04.087005+0000","last_became_active":"2024-04-24T09:04:01.546463+0000","last_became_peered":"2024-04-24T09:04:01.546463+0000","last_unstale":"2024-04-24T09:07:04.087005+0000","last_undegraded":"2024-04-24T09:07:04.087005+0000","last_fullsized":"2024-04-24T09:07:04.087005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:31:12.985687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341595+0000","last_change":"2024-04-24T09:04:02.112487+0000","last_active":"2024-04-24T09:07:04.341595+0000","last_peered":"2024-04-24T09:07:04.341595+0000","last_clean":"2024-04-24T09:07:04.341595+0000","last_became_active":"2024-04-24T09:04:02.112078+0000","last_became_peered":"2024-04-24T09:04:02.112078+0000","last_unstale":"2024-04-24T09:07:04.341595+0000","last_undegraded":"2024-04-24T09:07:04.341595+0000","last_fullsized":"2024-04-24T09:07:04.341595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:03:47.352110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086370+0000","last_change":"2024-04-24T09:04:24.539120+0000","last_active":"2024-04-24T09:07:04.086370+0000","last_peered":"2024-04-24T09:07:04.086370+0000","last_clean":"2024-04-24T09:07:04.086370+0000","last_became_active":"2024-04-24T09:04:24.537048+0000","last_became_peered":"2024-04-24T09:04:24.537048+0000","last_unstale":"2024-04-24T09:07:04.086370+0000","last_undegraded":"2024-04-24T09:07:04.086370+0000","last_fullsized":"2024-04-24T09:07:04.086370+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:29:53.484131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342056+0000","last_change":"2024-04-24T09:04:01.545444+0000","last_active":"2024-04-24T09:07:04.342056+0000","last_peered":"2024-04-24T09:07:04.342056+0000","last_clean":"2024-04-24T09:07:04.342056+0000","last_became_active":"2024-04-24T09:04:01.543816+0000","last_became_peered":"2024-04-24T09:04:01.543816+0000","last_unstale":"2024-04-24T09:07:04.342056+0000","last_undegraded":"2024-04-24T09:07:04.342056+0000","last_fullsized":"2024-04-24T09:07:04.342056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:02:36.221797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921232+0000","last_change":"2024-04-24T09:04:02.109085+0000","last_active":"2024-04-24T09:07:03.921232+0000","last_peered":"2024-04-24T09:07:03.921232+0000","last_clean":"2024-04-24T09:07:03.921232+0000","last_became_active":"2024-04-24T09:04:02.108642+0000","last_became_peered":"2024-04-24T09:04:02.108642+0000","last_unstale":"2024-04-24T09:07:03.921232+0000","last_undegraded":"2024-04-24T09:07:03.921232+0000","last_fullsized":"2024-04-24T09:07:03.921232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:27:51.751369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.026721+0000","last_change":"2024-04-24T09:04:06.144940+0000","last_active":"2024-04-24T09:07:04.026721+0000","last_peered":"2024-04-24T09:07:04.026721+0000","last_clean":"2024-04-24T09:07:04.026721+0000","last_became_active":"2024-04-24T09:04:06.144553+0000","last_became_peered":"2024-04-24T09:04:06.144553+0000","last_unstale":"2024-04-24T09:07:04.026721+0000","last_undegraded":"2024-04-24T09:07:04.026721+0000","last_fullsized":"2024-04-24T09:07:04.026721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:17.455644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029831+0000","last_change":"2024-04-24T09:04:01.538939+0000","last_active":"2024-04-24T09:07:04.029831+0000","last_peered":"2024-04-24T09:07:04.029831+0000","last_clean":"2024-04-24T09:07:04.029831+0000","last_became_active":"2024-04-24T09:04:01.538541+0000","last_became_peered":"2024-04-24T09:04:01.538541+0000","last_unstale":"2024-04-24T09:07:04.029831+0000","last_undegraded":"2024-04-24T09:07:04.029831+0000","last_fullsized":"2024-04-24T09:07:04.029831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:08.185901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.030917+0000","last_change":"2024-04-24T09:04:02.103844+0000","last_active":"2024-04-24T09:07:04.030917+0000","last_peered":"2024-04-24T09:07:04.030917+0000","last_clean":"2024-04-24T09:07:04.030917+0000","last_became_active":"2024-04-24T09:04:02.103416+0000","last_became_peered":"2024-04-24T09:04:02.103416+0000","last_unstale":"2024-04-24T09:07:04.030917+0000","last_undegraded":"2024-04-24T09:07:04.030917+0000","last_fullsized":"2024-04-24T09:07:04.030917+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:57:47.236384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086087+0000","last_change":"2024-04-24T09:04:06.134838+0000","last_active":"2024-04-24T09:07:04.086087+0000","last_peered":"2024-04-24T09:07:04.086087+0000","last_clean":"2024-04-24T09:07:04.086087+0000","last_became_active":"2024-04-24T09:04:06.134343+0000","last_became_peered":"2024-04-24T09:04:06.134343+0000","last_unstale":"2024-04-24T09:07:04.086087+0000","last_undegraded":"2024-04-24T09:07:04.086087+0000","last_fullsized":"2024-04-24T09:07:04.086087+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:56:55.219012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343211+0000","last_change":"2024-04-24T09:04:01.557194+0000","last_active":"2024-04-24T09:07:04.343211+0000","last_peered":"2024-04-24T09:07:04.343211+0000","last_clean":"2024-04-24T09:07:04.343211+0000","last_became_active":"2024-04-24T09:04:01.556441+0000","last_became_peered":"2024-04-24T09:04:01.556441+0000","last_unstale":"2024-04-24T09:07:04.343211+0000","last_undegraded":"2024-04-24T09:07:04.343211+0000","last_fullsized":"2024-04-24T09:07:04.343211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:45:50.399096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.030826+0000","last_change":"2024-04-24T09:04:02.108076+0000","last_active":"2024-04-24T09:07:04.030826+0000","last_peered":"2024-04-24T09:07:04.030826+0000","last_clean":"2024-04-24T09:07:04.030826+0000","last_became_active":"2024-04-24T09:04:02.106829+0000","last_became_peered":"2024-04-24T09:04:02.106829+0000","last_unstale":"2024-04-24T09:07:04.030826+0000","last_undegraded":"2024-04-24T09:07:04.030826+0000","last_fullsized":"2024-04-24T09:07:04.030826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:40:23.075828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086526+0000","last_change":"2024-04-24T09:04:06.142269+0000","last_active":"2024-04-24T09:07:04.086526+0000","last_peered":"2024-04-24T09:07:04.086526+0000","last_clean":"2024-04-24T09:07:04.086526+0000","last_became_active":"2024-04-24T09:04:06.141725+0000","last_became_peered":"2024-04-24T09:04:06.141725+0000","last_unstale":"2024-04-24T09:07:04.086526+0000","last_undegraded":"2024-04-24T09:07:04.086526+0000","last_fullsized":"2024-04-24T09:07:04.086526+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:28:17.934148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921558+0000","last_change":"2024-04-24T09:04:01.553134+0000","last_active":"2024-04-24T09:07:03.921558+0000","last_peered":"2024-04-24T09:07:03.921558+0000","last_clean":"2024-04-24T09:07:03.921558+0000","last_became_active":"2024-04-24T09:04:01.552099+0000","last_became_peered":"2024-04-24T09:04:01.552099+0000","last_unstale":"2024-04-24T09:07:03.921558+0000","last_undegraded":"2024-04-24T09:07:03.921558+0000","last_fullsized":"2024-04-24T09:07:03.921558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:42:36.231405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341535+0000","last_change":"2024-04-24T09:04:02.116641+0000","last_active":"2024-04-24T09:07:04.341535+0000","last_peered":"2024-04-24T09:07:04.341535+0000","last_clean":"2024-04-24T09:07:04.341535+0000","last_became_active":"2024-04-24T09:04:02.115374+0000","last_became_peered":"2024-04-24T09:04:02.115374+0000","last_unstale":"2024-04-24T09:07:04.341535+0000","last_undegraded":"2024-04-24T09:07:04.341535+0000","last_fullsized":"2024-04-24T09:07:04.341535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:34.450990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342365+0000","last_change":"2024-04-24T09:04:06.149968+0000","last_active":"2024-04-24T09:07:04.342365+0000","last_peered":"2024-04-24T09:07:04.342365+0000","last_clean":"2024-04-24T09:07:04.342365+0000","last_became_active":"2024-04-24T09:04:06.149425+0000","last_became_peered":"2024-04-24T09:04:06.149425+0000","last_unstale":"2024-04-24T09:07:04.342365+0000","last_undegraded":"2024-04-24T09:07:04.342365+0000","last_fullsized":"2024-04-24T09:07:04.342365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:23:53.869004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343669+0000","last_change":"2024-04-24T09:04:01.553822+0000","last_active":"2024-04-24T09:07:04.343669+0000","last_peered":"2024-04-24T09:07:04.343669+0000","last_clean":"2024-04-24T09:07:04.343669+0000","last_became_active":"2024-04-24T09:04:01.551468+0000","last_became_peered":"2024-04-24T09:04:01.551468+0000","last_unstale":"2024-04-24T09:07:04.343669+0000","last_undegraded":"2024-04-24T09:07:04.343669+0000","last_fullsized":"2024-04-24T09:07:04.343669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:56.367584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342035+0000","last_change":"2024-04-24T09:04:02.115472+0000","last_active":"2024-04-24T09:07:04.342035+0000","last_peered":"2024-04-24T09:07:04.342035+0000","last_clean":"2024-04-24T09:07:04.342035+0000","last_became_active":"2024-04-24T09:04:02.114666+0000","last_became_peered":"2024-04-24T09:04:02.114666+0000","last_unstale":"2024-04-24T09:07:04.342035+0000","last_undegraded":"2024-04-24T09:07:04.342035+0000","last_fullsized":"2024-04-24T09:07:04.342035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:28:39.145108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341009+0000","last_change":"2024-04-24T09:04:06.138786+0000","last_active":"2024-04-24T09:07:04.341009+0000","last_peered":"2024-04-24T09:07:04.341009+0000","last_clean":"2024-04-24T09:07:04.341009+0000","last_became_active":"2024-04-24T09:04:06.138366+0000","last_became_peered":"2024-04-24T09:04:06.138366+0000","last_unstale":"2024-04-24T09:07:04.341009+0000","last_undegraded":"2024-04-24T09:07:04.341009+0000","last_fullsized":"2024-04-24T09:07:04.341009+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:03:16.729928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"93'39","reported_seq":76,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.265142+0000","last_change":"2024-04-24T09:04:01.547733+0000","last_active":"2024-04-24T09:07:04.265142+0000","last_peered":"2024-04-24T09:07:04.265142+0000","last_clean":"2024-04-24T09:07:04.265142+0000","last_became_active":"2024-04-24T09:04:01.544839+0000","last_became_peered":"2024-04-24T09:04:01.544839+0000","last_unstale":"2024-04-24T09:07:04.265142+0000","last_undegraded":"2024-04-24T09:07:04.265142+0000","last_fullsized":"2024-04-24T09:07:04.265142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:45:03.847092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341471+0000","last_change":"2024-04-24T09:04:02.108383+0000","last_active":"2024-04-24T09:07:04.341471+0000","last_peered":"2024-04-24T09:07:04.341471+0000","last_clean":"2024-04-24T09:07:04.341471+0000","last_became_active":"2024-04-24T09:04:02.107883+0000","last_became_peered":"2024-04-24T09:04:02.107883+0000","last_unstale":"2024-04-24T09:07:04.341471+0000","last_undegraded":"2024-04-24T09:07:04.341471+0000","last_fullsized":"2024-04-24T09:07:04.341471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:24:44.923708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880074+0000","last_change":"2024-04-24T09:04:06.140868+0000","last_active":"2024-04-24T09:07:03.880074+0000","last_peered":"2024-04-24T09:07:03.880074+0000","last_clean":"2024-04-24T09:07:03.880074+0000","last_became_active":"2024-04-24T09:04:06.139174+0000","last_became_peered":"2024-04-24T09:04:06.139174+0000","last_unstale":"2024-04-24T09:07:03.880074+0000","last_undegraded":"2024-04-24T09:07:03.880074+0000","last_fullsized":"2024-04-24T09:07:03.880074+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:17:17.334276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.878570+0000","last_change":"2024-04-24T09:04:01.534653+0000","last_active":"2024-04-24T09:07:03.878570+0000","last_peered":"2024-04-24T09:07:03.878570+0000","last_clean":"2024-04-24T09:07:03.878570+0000","last_became_active":"2024-04-24T09:04:01.534312+0000","last_became_peered":"2024-04-24T09:04:01.534312+0000","last_unstale":"2024-04-24T09:07:03.878570+0000","last_undegraded":"2024-04-24T09:07:03.878570+0000","last_fullsized":"2024-04-24T09:07:03.878570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:53:34.301007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341883+0000","last_change":"2024-04-24T09:04:02.102056+0000","last_active":"2024-04-24T09:07:04.341883+0000","last_peered":"2024-04-24T09:07:04.341883+0000","last_clean":"2024-04-24T09:07:04.341883+0000","last_became_active":"2024-04-24T09:04:02.101641+0000","last_became_peered":"2024-04-24T09:04:02.101641+0000","last_unstale":"2024-04-24T09:07:04.341883+0000","last_undegraded":"2024-04-24T09:07:04.341883+0000","last_fullsized":"2024-04-24T09:07:04.341883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:41.654024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031176+0000","last_change":"2024-04-24T09:04:02.100327+0000","last_active":"2024-04-24T09:07:04.031176+0000","last_peered":"2024-04-24T09:07:04.031176+0000","last_clean":"2024-04-24T09:07:04.031176+0000","last_became_active":"2024-04-24T09:04:02.100028+0000","last_became_peered":"2024-04-24T09:04:02.100028+0000","last_unstale":"2024-04-24T09:07:04.031176+0000","last_undegraded":"2024-04-24T09:07:04.031176+0000","last_fullsized":"2024-04-24T09:07:04.031176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:15:18.728045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343576+0000","last_change":"2024-04-24T09:04:02.101996+0000","last_active":"2024-04-24T09:07:04.343576+0000","last_peered":"2024-04-24T09:07:04.343576+0000","last_clean":"2024-04-24T09:07:04.343576+0000","last_became_active":"2024-04-24T09:04:02.101581+0000","last_became_peered":"2024-04-24T09:04:02.101581+0000","last_unstale":"2024-04-24T09:07:04.343576+0000","last_undegraded":"2024-04-24T09:07:04.343576+0000","last_fullsized":"2024-04-24T09:07:04.343576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:55.487781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342748+0000","last_change":"2024-04-24T09:04:02.107288+0000","last_active":"2024-04-24T09:07:04.342748+0000","last_peered":"2024-04-24T09:07:04.342748+0000","last_clean":"2024-04-24T09:07:04.342748+0000","last_became_active":"2024-04-24T09:04:02.106946+0000","last_became_peered":"2024-04-24T09:04:02.106946+0000","last_unstale":"2024-04-24T09:07:04.342748+0000","last_undegraded":"2024-04-24T09:07:04.342748+0000","last_fullsized":"2024-04-24T09:07:04.342748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:38:56.821285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341274+0000","last_change":"2024-04-24T09:04:02.116829+0000","last_active":"2024-04-24T09:07:04.341274+0000","last_peered":"2024-04-24T09:07:04.341274+0000","last_clean":"2024-04-24T09:07:04.341274+0000","last_became_active":"2024-04-24T09:04:02.116520+0000","last_became_peered":"2024-04-24T09:04:02.116520+0000","last_unstale":"2024-04-24T09:07:04.341274+0000","last_undegraded":"2024-04-24T09:07:04.341274+0000","last_fullsized":"2024-04-24T09:07:04.341274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:26.873826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.876882+0000","last_change":"2024-04-24T09:04:24.700595+0000","last_active":"2024-04-24T09:07:03.876882+0000","last_peered":"2024-04-24T09:07:03.876882+0000","last_clean":"2024-04-24T09:07:03.876882+0000","last_became_active":"2024-04-24T09:04:24.699395+0000","last_became_peered":"2024-04-24T09:04:24.699395+0000","last_unstale":"2024-04-24T09:07:03.876882+0000","last_undegraded":"2024-04-24T09:07:03.876882+0000","last_fullsized":"2024-04-24T09:07:03.876882+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:28.756302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.032684+0000","last_change":"2024-04-24T09:04:02.114087+0000","last_active":"2024-04-24T09:07:04.032684+0000","last_peered":"2024-04-24T09:07:04.032684+0000","last_clean":"2024-04-24T09:07:04.032684+0000","last_became_active":"2024-04-24T09:04:02.112681+0000","last_became_peered":"2024-04-24T09:04:02.112681+0000","last_unstale":"2024-04-24T09:07:04.032684+0000","last_undegraded":"2024-04-24T09:07:04.032684+0000","last_fullsized":"2024-04-24T09:07:04.032684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:50:01.762598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.344340+0000","last_change":"2024-04-24T09:04:02.102322+0000","last_active":"2024-04-24T09:07:04.344340+0000","last_peered":"2024-04-24T09:07:04.344340+0000","last_clean":"2024-04-24T09:07:04.344340+0000","last_became_active":"2024-04-24T09:04:02.101963+0000","last_became_peered":"2024-04-24T09:04:02.101963+0000","last_unstale":"2024-04-24T09:07:04.344340+0000","last_undegraded":"2024-04-24T09:07:04.344340+0000","last_fullsized":"2024-04-24T09:07:04.344340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:16.140829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031553+0000","last_change":"2024-04-24T09:04:02.110570+0000","last_active":"2024-04-24T09:07:04.031553+0000","last_peered":"2024-04-24T09:07:04.031553+0000","last_clean":"2024-04-24T09:07:04.031553+0000","last_became_active":"2024-04-24T09:04:02.110119+0000","last_became_peered":"2024-04-24T09:04:02.110119+0000","last_unstale":"2024-04-24T09:07:04.031553+0000","last_undegraded":"2024-04-24T09:07:04.031553+0000","last_fullsized":"2024-04-24T09:07:04.031553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:34.436430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880582+0000","last_change":"2024-04-24T09:04:02.108292+0000","last_active":"2024-04-24T09:07:03.880582+0000","last_peered":"2024-04-24T09:07:03.880582+0000","last_clean":"2024-04-24T09:07:03.880582+0000","last_became_active":"2024-04-24T09:04:02.107481+0000","last_became_peered":"2024-04-24T09:04:02.107481+0000","last_unstale":"2024-04-24T09:07:03.880582+0000","last_undegraded":"2024-04-24T09:07:03.880582+0000","last_fullsized":"2024-04-24T09:07:03.880582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:49.755293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029964+0000","last_change":"2024-04-24T09:04:02.102228+0000","last_active":"2024-04-24T09:07:04.029964+0000","last_peered":"2024-04-24T09:07:04.029964+0000","last_clean":"2024-04-24T09:07:04.029964+0000","last_became_active":"2024-04-24T09:04:02.101863+0000","last_became_peered":"2024-04-24T09:04:02.101863+0000","last_unstale":"2024-04-24T09:07:04.029964+0000","last_undegraded":"2024-04-24T09:07:04.029964+0000","last_fullsized":"2024-04-24T09:07:04.029964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:02:47.308116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342657+0000","last_change":"2024-04-24T09:04:02.106749+0000","last_active":"2024-04-24T09:07:04.342657+0000","last_peered":"2024-04-24T09:07:04.342657+0000","last_clean":"2024-04-24T09:07:04.342657+0000","last_became_active":"2024-04-24T09:04:02.106282+0000","last_became_peered":"2024-04-24T09:04:02.106282+0000","last_unstale":"2024-04-24T09:07:04.342657+0000","last_undegraded":"2024-04-24T09:07:04.342657+0000","last_fullsized":"2024-04-24T09:07:04.342657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:35:59.765230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.877236+0000","last_change":"2024-04-24T09:04:02.097825+0000","last_active":"2024-04-24T09:07:03.877236+0000","last_peered":"2024-04-24T09:07:03.877236+0000","last_clean":"2024-04-24T09:07:03.877236+0000","last_became_active":"2024-04-24T09:04:02.097488+0000","last_became_peered":"2024-04-24T09:04:02.097488+0000","last_unstale":"2024-04-24T09:07:03.877236+0000","last_undegraded":"2024-04-24T09:07:03.877236+0000","last_fullsized":"2024-04-24T09:07:03.877236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:05:59.962558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031865+0000","last_change":"2024-04-24T09:04:02.113819+0000","last_active":"2024-04-24T09:07:04.031865+0000","last_peered":"2024-04-24T09:07:04.031865+0000","last_clean":"2024-04-24T09:07:04.031865+0000","last_became_active":"2024-04-24T09:04:02.113463+0000","last_became_peered":"2024-04-24T09:04:02.113463+0000","last_unstale":"2024-04-24T09:07:04.031865+0000","last_undegraded":"2024-04-24T09:07:04.031865+0000","last_fullsized":"2024-04-24T09:07:04.031865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:47:05.109583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602602,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1027,"num_read_kb":628,"num_write":300,"num_write_kb":1450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2039808,"data_stored":1807806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351028,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28432,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716720,"statfs":{"total":95995035648,"available":95965921280,"internally_reserved":0,"allocated":1445888,"data_stored":910510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612667,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33200,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93711952,"statfs":{"total":95995035648,"available":95961038848,"internally_reserved":0,"allocated":1888256,"data_stored":1308281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808899,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1466368,"data_stored":914842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070537,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":517450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332176,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32784,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712368,"statfs":{"total":95995035648,"available":95961464832,"internally_reserved":0,"allocated":1646592,"data_stored":1104678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495704,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":512706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":573902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":504286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":574630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182509,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1037716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920869,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":519496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757343,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32940,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712212,"statfs":{"total":95995035648,"available":95961305088,"internally_reserved":0,"allocated":1806336,"data_stored":1241431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":206079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T09:07:07.984 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T09:07:08.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:07 smithi081 ceph-mon[30961]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:07:08.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:07 smithi081 ceph-mon[30961]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-24T09:07:08.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:07 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3679096262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:07:08.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:07 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:08.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:07 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:08.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:07 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:08.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:07 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:08.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:07 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2163779627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:07:08.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.218+0000 7f4ecfe00640 1 -- 172.21.15.69:0/811886280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ed0101ea0 msgr2=0x7f4ed01022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:08.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.218+0000 7f4ecfe00640 1 --2- 172.21.15.69:0/811886280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ed0101ea0 0x7f4ed01022c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009900 tx=0x7f4ec002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.219+0000 7f4ecfe00640 1 -- 172.21.15.69:0/811886280 shutdown_connections 2024-04-24T09:07:08.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.219+0000 7f4ecfe00640 1 --2- 172.21.15.69:0/811886280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ed01039b0 0x7f4ed010a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.219+0000 7f4ecfe00640 1 --2- 172.21.15.69:0/811886280 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ed0102ff0 0x7f4ed0103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.219+0000 7f4ecfe00640 1 --2- 172.21.15.69:0/811886280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ed0101ea0 0x7f4ed01022c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.219+0000 7f4ecfe00640 1 -- 172.21.15.69:0/811886280 >> 172.21.15.69:0/811886280 conn(0x7f4ed00fd410 msgr2=0x7f4ed00ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:08.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.219+0000 7f4ecfe00640 1 -- 172.21.15.69:0/811886280 shutdown_connections 2024-04-24T09:07:08.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.219+0000 7f4ecfe00640 1 -- 172.21.15.69:0/811886280 wait complete. 2024-04-24T09:07:08.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.220+0000 7f4ecfe00640 1 Processor -- start 2024-04-24T09:07:08.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.220+0000 7f4ecfe00640 1 -- start start 2024-04-24T09:07:08.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.220+0000 7f4ecfe00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ed0101ea0 0x7f4ed0198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:08.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.220+0000 7f4ecfe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ed0102ff0 0x7f4ed0198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:08.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.220+0000 7f4ecfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ed01039b0 0x7f4ed019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:08.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.220+0000 7f4ecfe00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed010daf0 con 0x7f4ed01039b0 2024-04-24T09:07:08.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.220+0000 7f4ecfe00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed010d970 con 0x7f4ed0101ea0 2024-04-24T09:07:08.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.220+0000 7f4ecfe00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed010dc70 con 0x7f4ed0102ff0 2024-04-24T09:07:08.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.220+0000 7f4ecf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ed01039b0 0x7f4ed019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:08.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.220+0000 7f4ecf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ed01039b0 0x7f4ed019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46074/0 (socket says 172.21.15.69:46074) 2024-04-24T09:07:08.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.220+0000 7f4ecf400640 1 -- 172.21.15.69:0/826092004 learned_addr learned my addr 172.21.15.69:0/826092004 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:08.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.221+0000 7f4ecea00640 1 --2- 172.21.15.69:0/826092004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ed0101ea0 0x7f4ed0198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:08.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.221+0000 7f4ece000640 1 --2- 172.21.15.69:0/826092004 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ed0102ff0 0x7f4ed0198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:08.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.221+0000 7f4ecf400640 1 -- 172.21.15.69:0/826092004 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ed0102ff0 msgr2=0x7f4ed0198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:08.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.221+0000 7f4ecf400640 1 --2- 172.21.15.69:0/826092004 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ed0102ff0 0x7f4ed0198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.221+0000 7f4ecf400640 1 -- 172.21.15.69:0/826092004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ed0101ea0 msgr2=0x7f4ed0198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:08.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.221+0000 7f4ecf400640 1 --2- 172.21.15.69:0/826092004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ed0101ea0 0x7f4ed0198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.221+0000 7f4ecf400640 1 -- 172.21.15.69:0/826092004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed019fe60 con 0x7f4ed01039b0 2024-04-24T09:07:08.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.221+0000 7f4ece000640 1 --2- 172.21.15.69:0/826092004 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ed0102ff0 0x7f4ed0198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:08.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.221+0000 7f4ecf400640 1 --2- 172.21.15.69:0/826092004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ed01039b0 0x7f4ed019f7f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800b970 tx=0x7f4ec800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:08.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.224+0000 7f4ec7400640 1 -- 172.21.15.69:0/826092004 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8019070 con 0x7f4ed01039b0 2024-04-24T09:07:08.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.224+0000 7f4ecfe00640 1 -- 172.21.15.69:0/826092004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed01a0150 con 0x7f4ed01039b0 2024-04-24T09:07:08.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.224+0000 7f4ec7400640 1 -- 172.21.15.69:0/826092004 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ec8014440 con 0x7f4ed01039b0 2024-04-24T09:07:08.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.224+0000 7f4ec7400640 1 -- 172.21.15.69:0/826092004 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80075d0 con 0x7f4ed01039b0 2024-04-24T09:07:08.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.224+0000 7f4ecfe00640 1 -- 172.21.15.69:0/826092004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed01a0720 con 0x7f4ed01039b0 2024-04-24T09:07:08.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.226+0000 7f4ecfe00640 1 -- 172.21.15.69:0/826092004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed0109280 con 0x7f4ed01039b0 2024-04-24T09:07:08.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.228+0000 7f4ec7400640 1 -- 172.21.15.69:0/826092004 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f4ec801c070 con 0x7f4ed01039b0 2024-04-24T09:07:08.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.229+0000 7f4ec7400640 1 --2- 172.21.15.69:0/826092004 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4eb40a3d20 0x7f4eb40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:08.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.229+0000 7f4ec7400640 1 -- 172.21.15.69:0/826092004 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4ec80d0060 con 0x7f4ed01039b0 2024-04-24T09:07:08.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.229+0000 7f4ecea00640 1 --2- 172.21.15.69:0/826092004 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4eb40a3d20 0x7f4eb40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:08.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.230+0000 7f4ecea00640 1 --2- 172.21.15.69:0/826092004 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4eb40a3d20 0x7f4eb40a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0000c00 tx=0x7f4ec00314a0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:08.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.238+0000 7f4ec7400640 1 -- 172.21.15.69:0/826092004 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ec809c4d0 con 0x7f4ed01039b0 2024-04-24T09:07:08.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.476+0000 7f4ecfe00640 1 -- 172.21.15.69:0/826092004 --> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ed0062440 con 0x7f4eb40a3d20 2024-04-24T09:07:08.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.487+0000 7f4ec7400640 1 -- 172.21.15.69:0/826092004 <== mgr.15465 v2:172.21.15.69:6800/2822728880 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339473 (secure 0 0 0) 0x7f4ed0062440 con 0x7f4eb40a3d20 2024-04-24T09:07:08.492 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:07:08.493 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-24T09:07:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.492+0000 7f4ecfe00640 1 -- 172.21.15.69:0/826092004 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4eb40a3d20 msgr2=0x7f4eb40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.492+0000 7f4ecfe00640 1 --2- 172.21.15.69:0/826092004 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4eb40a3d20 0x7f4eb40a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0000c00 tx=0x7f4ec00314a0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.492+0000 7f4ecfe00640 1 -- 172.21.15.69:0/826092004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ed01039b0 msgr2=0x7f4ed019f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.492+0000 7f4ecfe00640 1 --2- 172.21.15.69:0/826092004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ed01039b0 0x7f4ed019f7f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800b970 tx=0x7f4ec800be30 comp rx=0 tx=0).stop 2024-04-24T09:07:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.492+0000 7f4ecfe00640 1 -- 172.21.15.69:0/826092004 shutdown_connections 2024-04-24T09:07:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.492+0000 7f4ecfe00640 1 --2- 172.21.15.69:0/826092004 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4eb40a3d20 0x7f4eb40a61e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.492+0000 7f4ecfe00640 1 --2- 172.21.15.69:0/826092004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ed01039b0 0x7f4ed019f7f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.492+0000 7f4ecfe00640 1 --2- 172.21.15.69:0/826092004 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ed0102ff0 0x7f4ed0198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.492+0000 7f4ecfe00640 1 --2- 172.21.15.69:0/826092004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ed0101ea0 0x7f4ed0198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.492+0000 7f4ecfe00640 1 -- 172.21.15.69:0/826092004 >> 172.21.15.69:0/826092004 conn(0x7f4ed00fd410 msgr2=0x7f4ed0106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:08.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.492+0000 7f4ecfe00640 1 -- 172.21.15.69:0/826092004 shutdown_connections 2024-04-24T09:07:08.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.492+0000 7f4ecfe00640 1 -- 172.21.15.69:0/826092004 wait complete. 2024-04-24T09:07:08.509 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-24T09:07:07.901521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193204,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1163,"num_read_kb":747,"num_write":483,"num_write_kb":4390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":877,"ondisk_log_size":877,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356972,"kb_used_data":15940,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124584852,"statfs":{"total":1151940427776,"available":1151574888448,"internally_reserved":0,"allocated":16322560,"data_stored":9719928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.983030"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031762+0000","last_change":"2024-04-24T09:04:02.107965+0000","last_active":"2024-04-24T09:07:04.031762+0000","last_peered":"2024-04-24T09:07:04.031762+0000","last_clean":"2024-04-24T09:07:04.031762+0000","last_became_active":"2024-04-24T09:04:02.106827+0000","last_became_peered":"2024-04-24T09:04:02.106827+0000","last_unstale":"2024-04-24T09:07:04.031762+0000","last_undegraded":"2024-04-24T09:07:04.031762+0000","last_fullsized":"2024-04-24T09:07:04.031762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:20:23.930599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341563+0000","last_change":"2024-04-24T09:04:02.105449+0000","last_active":"2024-04-24T09:07:04.341563+0000","last_peered":"2024-04-24T09:07:04.341563+0000","last_clean":"2024-04-24T09:07:04.341563+0000","last_became_active":"2024-04-24T09:04:02.101842+0000","last_became_peered":"2024-04-24T09:04:02.101842+0000","last_unstale":"2024-04-24T09:07:04.341563+0000","last_undegraded":"2024-04-24T09:07:04.341563+0000","last_fullsized":"2024-04-24T09:07:04.341563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:46.911768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341388+0000","last_change":"2024-04-24T09:04:02.118686+0000","last_active":"2024-04-24T09:07:04.341388+0000","last_peered":"2024-04-24T09:07:04.341388+0000","last_clean":"2024-04-24T09:07:04.341388+0000","last_became_active":"2024-04-24T09:04:02.117549+0000","last_became_peered":"2024-04-24T09:04:02.117549+0000","last_unstale":"2024-04-24T09:07:04.341388+0000","last_undegraded":"2024-04-24T09:07:04.341388+0000","last_fullsized":"2024-04-24T09:07:04.341388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:18:16.233434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086531+0000","last_change":"2024-04-24T09:04:02.120336+0000","last_active":"2024-04-24T09:07:04.086531+0000","last_peered":"2024-04-24T09:07:04.086531+0000","last_clean":"2024-04-24T09:07:04.086531+0000","last_became_active":"2024-04-24T09:04:02.116574+0000","last_became_peered":"2024-04-24T09:04:02.116574+0000","last_unstale":"2024-04-24T09:07:04.086531+0000","last_undegraded":"2024-04-24T09:07:04.086531+0000","last_fullsized":"2024-04-24T09:07:04.086531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:10:42.366531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921771+0000","last_change":"2024-04-24T09:04:02.113600+0000","last_active":"2024-04-24T09:07:03.921771+0000","last_peered":"2024-04-24T09:07:03.921771+0000","last_clean":"2024-04-24T09:07:03.921771+0000","last_became_active":"2024-04-24T09:04:02.113143+0000","last_became_peered":"2024-04-24T09:04:02.113143+0000","last_unstale":"2024-04-24T09:07:03.921771+0000","last_undegraded":"2024-04-24T09:07:03.921771+0000","last_fullsized":"2024-04-24T09:07:03.921771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:31:27.666661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.882380+0000","last_change":"2024-04-24T09:04:02.106395+0000","last_active":"2024-04-24T09:07:03.882380+0000","last_peered":"2024-04-24T09:07:03.882380+0000","last_clean":"2024-04-24T09:07:03.882380+0000","last_became_active":"2024-04-24T09:04:02.105942+0000","last_became_peered":"2024-04-24T09:04:02.105942+0000","last_unstale":"2024-04-24T09:07:03.882380+0000","last_undegraded":"2024-04-24T09:07:03.882380+0000","last_fullsized":"2024-04-24T09:07:03.882380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:20:10.655605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341117+0000","last_change":"2024-04-24T09:04:02.100279+0000","last_active":"2024-04-24T09:07:04.341117+0000","last_peered":"2024-04-24T09:07:04.341117+0000","last_clean":"2024-04-24T09:07:04.341117+0000","last_became_active":"2024-04-24T09:04:02.099745+0000","last_became_peered":"2024-04-24T09:04:02.099745+0000","last_unstale":"2024-04-24T09:07:04.341117+0000","last_undegraded":"2024-04-24T09:07:04.341117+0000","last_fullsized":"2024-04-24T09:07:04.341117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:13.150412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029846+0000","last_change":"2024-04-24T09:04:02.103294+0000","last_active":"2024-04-24T09:07:04.029846+0000","last_peered":"2024-04-24T09:07:04.029846+0000","last_clean":"2024-04-24T09:07:04.029846+0000","last_became_active":"2024-04-24T09:04:02.102114+0000","last_became_peered":"2024-04-24T09:04:02.102114+0000","last_unstale":"2024-04-24T09:07:04.029846+0000","last_undegraded":"2024-04-24T09:07:04.029846+0000","last_fullsized":"2024-04-24T09:07:04.029846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:26:26.916097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342942+0000","last_change":"2024-04-24T09:04:02.104936+0000","last_active":"2024-04-24T09:07:04.342942+0000","last_peered":"2024-04-24T09:07:04.342942+0000","last_clean":"2024-04-24T09:07:04.342942+0000","last_became_active":"2024-04-24T09:04:02.104540+0000","last_became_peered":"2024-04-24T09:04:02.104540+0000","last_unstale":"2024-04-24T09:07:04.342942+0000","last_undegraded":"2024-04-24T09:07:04.342942+0000","last_fullsized":"2024-04-24T09:07:04.342942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:36.010735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883910+0000","last_change":"2024-04-24T09:04:02.102050+0000","last_active":"2024-04-24T09:07:03.883910+0000","last_peered":"2024-04-24T09:07:03.883910+0000","last_clean":"2024-04-24T09:07:03.883910+0000","last_became_active":"2024-04-24T09:04:02.101623+0000","last_became_peered":"2024-04-24T09:04:02.101623+0000","last_unstale":"2024-04-24T09:07:03.883910+0000","last_undegraded":"2024-04-24T09:07:03.883910+0000","last_fullsized":"2024-04-24T09:07:03.883910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:13:33.606200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341928+0000","last_change":"2024-04-24T09:04:02.115804+0000","last_active":"2024-04-24T09:07:04.341928+0000","last_peered":"2024-04-24T09:07:04.341928+0000","last_clean":"2024-04-24T09:07:04.341928+0000","last_became_active":"2024-04-24T09:04:02.115093+0000","last_became_peered":"2024-04-24T09:04:02.115093+0000","last_unstale":"2024-04-24T09:07:04.341928+0000","last_undegraded":"2024-04-24T09:07:04.341928+0000","last_fullsized":"2024-04-24T09:07:04.341928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:26:56.764485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086355+0000","last_change":"2024-04-24T09:04:02.116305+0000","last_active":"2024-04-24T09:07:04.086355+0000","last_peered":"2024-04-24T09:07:04.086355+0000","last_clean":"2024-04-24T09:07:04.086355+0000","last_became_active":"2024-04-24T09:04:02.115956+0000","last_became_peered":"2024-04-24T09:04:02.115956+0000","last_unstale":"2024-04-24T09:07:04.086355+0000","last_undegraded":"2024-04-24T09:07:04.086355+0000","last_fullsized":"2024-04-24T09:07:04.086355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:38.618317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.877469+0000","last_change":"2024-04-24T09:04:02.101155+0000","last_active":"2024-04-24T09:07:03.877469+0000","last_peered":"2024-04-24T09:07:03.877469+0000","last_clean":"2024-04-24T09:07:03.877469+0000","last_became_active":"2024-04-24T09:04:02.099719+0000","last_became_peered":"2024-04-24T09:04:02.099719+0000","last_unstale":"2024-04-24T09:07:03.877469+0000","last_undegraded":"2024-04-24T09:07:03.877469+0000","last_fullsized":"2024-04-24T09:07:03.877469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:46:27.910987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.340928+0000","last_change":"2024-04-24T09:04:02.114818+0000","last_active":"2024-04-24T09:07:04.340928+0000","last_peered":"2024-04-24T09:07:04.340928+0000","last_clean":"2024-04-24T09:07:04.340928+0000","last_became_active":"2024-04-24T09:04:02.112947+0000","last_became_peered":"2024-04-24T09:04:02.112947+0000","last_unstale":"2024-04-24T09:07:04.340928+0000","last_undegraded":"2024-04-24T09:07:04.340928+0000","last_fullsized":"2024-04-24T09:07:04.340928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:16.544420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341005+0000","last_change":"2024-04-24T09:04:02.120672+0000","last_active":"2024-04-24T09:07:04.341005+0000","last_peered":"2024-04-24T09:07:04.341005+0000","last_clean":"2024-04-24T09:07:04.341005+0000","last_became_active":"2024-04-24T09:04:02.120378+0000","last_became_peered":"2024-04-24T09:04:02.120378+0000","last_unstale":"2024-04-24T09:07:04.341005+0000","last_undegraded":"2024-04-24T09:07:04.341005+0000","last_fullsized":"2024-04-24T09:07:04.341005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:17.781189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342217+0000","last_change":"2024-04-24T09:04:02.107022+0000","last_active":"2024-04-24T09:07:04.342217+0000","last_peered":"2024-04-24T09:07:04.342217+0000","last_clean":"2024-04-24T09:07:04.342217+0000","last_became_active":"2024-04-24T09:04:02.106645+0000","last_became_peered":"2024-04-24T09:04:02.106645+0000","last_unstale":"2024-04-24T09:07:04.342217+0000","last_undegraded":"2024-04-24T09:07:04.342217+0000","last_fullsized":"2024-04-24T09:07:04.342217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:47.217718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031638+0000","last_change":"2024-04-24T09:04:02.115511+0000","last_active":"2024-04-24T09:07:04.031638+0000","last_peered":"2024-04-24T09:07:04.031638+0000","last_clean":"2024-04-24T09:07:04.031638+0000","last_became_active":"2024-04-24T09:04:02.115197+0000","last_became_peered":"2024-04-24T09:04:02.115197+0000","last_unstale":"2024-04-24T09:07:04.031638+0000","last_undegraded":"2024-04-24T09:07:04.031638+0000","last_fullsized":"2024-04-24T09:07:04.031638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:18.883688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.026662+0000","last_change":"2024-04-24T09:04:02.102108+0000","last_active":"2024-04-24T09:07:04.026662+0000","last_peered":"2024-04-24T09:07:04.026662+0000","last_clean":"2024-04-24T09:07:04.026662+0000","last_became_active":"2024-04-24T09:04:02.101739+0000","last_became_peered":"2024-04-24T09:04:02.101739+0000","last_unstale":"2024-04-24T09:07:04.026662+0000","last_undegraded":"2024-04-24T09:07:04.026662+0000","last_fullsized":"2024-04-24T09:07:04.026662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:57:30.768716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343732+0000","last_change":"2024-04-24T09:04:02.108911+0000","last_active":"2024-04-24T09:07:04.343732+0000","last_peered":"2024-04-24T09:07:04.343732+0000","last_clean":"2024-04-24T09:07:04.343732+0000","last_became_active":"2024-04-24T09:04:02.105678+0000","last_became_peered":"2024-04-24T09:04:02.105678+0000","last_unstale":"2024-04-24T09:07:04.343732+0000","last_undegraded":"2024-04-24T09:07:04.343732+0000","last_fullsized":"2024-04-24T09:07:04.343732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:36:12.216271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343846+0000","last_change":"2024-04-24T09:04:06.136320+0000","last_active":"2024-04-24T09:07:04.343846+0000","last_peered":"2024-04-24T09:07:04.343846+0000","last_clean":"2024-04-24T09:07:04.343846+0000","last_became_active":"2024-04-24T09:04:06.133851+0000","last_became_peered":"2024-04-24T09:04:06.133851+0000","last_unstale":"2024-04-24T09:07:04.343846+0000","last_undegraded":"2024-04-24T09:07:04.343846+0000","last_fullsized":"2024-04-24T09:07:04.343846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:31:39.781234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.026762+0000","last_change":"2024-04-24T09:04:01.543818+0000","last_active":"2024-04-24T09:07:04.026762+0000","last_peered":"2024-04-24T09:07:04.026762+0000","last_clean":"2024-04-24T09:07:04.026762+0000","last_became_active":"2024-04-24T09:04:01.543435+0000","last_became_peered":"2024-04-24T09:04:01.543435+0000","last_unstale":"2024-04-24T09:07:04.026762+0000","last_undegraded":"2024-04-24T09:07:04.026762+0000","last_fullsized":"2024-04-24T09:07:04.026762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:08:30.176211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031206+0000","last_change":"2024-04-24T09:04:02.101646+0000","last_active":"2024-04-24T09:07:04.031206+0000","last_peered":"2024-04-24T09:07:04.031206+0000","last_clean":"2024-04-24T09:07:04.031206+0000","last_became_active":"2024-04-24T09:04:02.101284+0000","last_became_peered":"2024-04-24T09:04:02.101284+0000","last_unstale":"2024-04-24T09:07:04.031206+0000","last_undegraded":"2024-04-24T09:07:04.031206+0000","last_fullsized":"2024-04-24T09:07:04.031206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:36:32.827216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.085981+0000","last_change":"2024-04-24T09:04:06.130476+0000","last_active":"2024-04-24T09:07:04.085981+0000","last_peered":"2024-04-24T09:07:04.085981+0000","last_clean":"2024-04-24T09:07:04.085981+0000","last_became_active":"2024-04-24T09:04:06.129707+0000","last_became_peered":"2024-04-24T09:04:06.129707+0000","last_unstale":"2024-04-24T09:07:04.085981+0000","last_undegraded":"2024-04-24T09:07:04.085981+0000","last_fullsized":"2024-04-24T09:07:04.085981+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:25:36.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031243+0000","last_change":"2024-04-24T09:04:01.551708+0000","last_active":"2024-04-24T09:07:04.031243+0000","last_peered":"2024-04-24T09:07:04.031243+0000","last_clean":"2024-04-24T09:07:04.031243+0000","last_became_active":"2024-04-24T09:04:01.551089+0000","last_became_peered":"2024-04-24T09:04:01.551089+0000","last_unstale":"2024-04-24T09:07:04.031243+0000","last_undegraded":"2024-04-24T09:07:04.031243+0000","last_fullsized":"2024-04-24T09:07:04.031243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:18:22.515729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341702+0000","last_change":"2024-04-24T09:04:02.117356+0000","last_active":"2024-04-24T09:07:04.341702+0000","last_peered":"2024-04-24T09:07:04.341702+0000","last_clean":"2024-04-24T09:07:04.341702+0000","last_became_active":"2024-04-24T09:04:02.116972+0000","last_became_peered":"2024-04-24T09:04:02.116972+0000","last_unstale":"2024-04-24T09:07:04.341702+0000","last_undegraded":"2024-04-24T09:07:04.341702+0000","last_fullsized":"2024-04-24T09:07:04.341702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:46:29.349952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086191+0000","last_change":"2024-04-24T09:04:06.141538+0000","last_active":"2024-04-24T09:07:04.086191+0000","last_peered":"2024-04-24T09:07:04.086191+0000","last_clean":"2024-04-24T09:07:04.086191+0000","last_became_active":"2024-04-24T09:04:06.140975+0000","last_became_peered":"2024-04-24T09:04:06.140975+0000","last_unstale":"2024-04-24T09:07:04.086191+0000","last_undegraded":"2024-04-24T09:07:04.086191+0000","last_fullsized":"2024-04-24T09:07:04.086191+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:58:31.688828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.922715+0000","last_change":"2024-04-24T09:04:01.561039+0000","last_active":"2024-04-24T09:07:03.922715+0000","last_peered":"2024-04-24T09:07:03.922715+0000","last_clean":"2024-04-24T09:07:03.922715+0000","last_became_active":"2024-04-24T09:04:01.560594+0000","last_became_peered":"2024-04-24T09:04:01.560594+0000","last_unstale":"2024-04-24T09:07:03.922715+0000","last_undegraded":"2024-04-24T09:07:03.922715+0000","last_fullsized":"2024-04-24T09:07:03.922715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:59:19.707826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342953+0000","last_change":"2024-04-24T09:04:02.106007+0000","last_active":"2024-04-24T09:07:04.342953+0000","last_peered":"2024-04-24T09:07:04.342953+0000","last_clean":"2024-04-24T09:07:04.342953+0000","last_became_active":"2024-04-24T09:04:02.099786+0000","last_became_peered":"2024-04-24T09:04:02.099786+0000","last_unstale":"2024-04-24T09:07:04.342953+0000","last_undegraded":"2024-04-24T09:07:04.342953+0000","last_fullsized":"2024-04-24T09:07:04.342953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:56.288417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.085876+0000","last_change":"2024-04-24T09:04:06.143636+0000","last_active":"2024-04-24T09:07:04.085876+0000","last_peered":"2024-04-24T09:07:04.085876+0000","last_clean":"2024-04-24T09:07:04.085876+0000","last_became_active":"2024-04-24T09:04:06.143105+0000","last_became_peered":"2024-04-24T09:04:06.143105+0000","last_unstale":"2024-04-24T09:07:04.085876+0000","last_undegraded":"2024-04-24T09:07:04.085876+0000","last_fullsized":"2024-04-24T09:07:04.085876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:04:23.211067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.922078+0000","last_change":"2024-04-24T09:04:01.548403+0000","last_active":"2024-04-24T09:07:03.922078+0000","last_peered":"2024-04-24T09:07:03.922078+0000","last_clean":"2024-04-24T09:07:03.922078+0000","last_became_active":"2024-04-24T09:04:01.547447+0000","last_became_peered":"2024-04-24T09:04:01.547447+0000","last_unstale":"2024-04-24T09:07:03.922078+0000","last_undegraded":"2024-04-24T09:07:03.922078+0000","last_fullsized":"2024-04-24T09:07:03.922078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:14:08.160127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883567+0000","last_change":"2024-04-24T09:04:02.121410+0000","last_active":"2024-04-24T09:07:03.883567+0000","last_peered":"2024-04-24T09:07:03.883567+0000","last_clean":"2024-04-24T09:07:03.883567+0000","last_became_active":"2024-04-24T09:04:02.120821+0000","last_became_peered":"2024-04-24T09:04:02.120821+0000","last_unstale":"2024-04-24T09:07:03.883567+0000","last_undegraded":"2024-04-24T09:07:03.883567+0000","last_fullsized":"2024-04-24T09:07:03.883567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:39:05.525171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343281+0000","last_change":"2024-04-24T09:04:06.142349+0000","last_active":"2024-04-24T09:07:04.343281+0000","last_peered":"2024-04-24T09:07:04.343281+0000","last_clean":"2024-04-24T09:07:04.343281+0000","last_became_active":"2024-04-24T09:04:06.141911+0000","last_became_peered":"2024-04-24T09:04:06.141911+0000","last_unstale":"2024-04-24T09:07:04.343281+0000","last_undegraded":"2024-04-24T09:07:04.343281+0000","last_fullsized":"2024-04-24T09:07:04.343281+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:45:30.414686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":196,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.444417+0000","last_change":"2024-04-24T09:04:24.537349+0000","last_active":"2024-04-24T09:07:04.444417+0000","last_peered":"2024-04-24T09:07:04.444417+0000","last_clean":"2024-04-24T09:07:04.444417+0000","last_became_active":"2024-04-24T09:04:24.536738+0000","last_became_peered":"2024-04-24T09:04:24.536738+0000","last_unstale":"2024-04-24T09:07:04.444417+0000","last_undegraded":"2024-04-24T09:07:04.444417+0000","last_fullsized":"2024-04-24T09:07:04.444417+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:33.481380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1897,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":153,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921779+0000","last_change":"2024-04-24T09:04:02.111173+0000","last_active":"2024-04-24T09:07:03.921779+0000","last_peered":"2024-04-24T09:07:03.921779+0000","last_clean":"2024-04-24T09:07:03.921779+0000","last_became_active":"2024-04-24T09:04:02.110681+0000","last_became_peered":"2024-04-24T09:04:02.110681+0000","last_unstale":"2024-04-24T09:07:03.921779+0000","last_undegraded":"2024-04-24T09:07:03.921779+0000","last_fullsized":"2024-04-24T09:07:03.921779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:29.939116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343333+0000","last_change":"2024-04-24T09:04:06.137752+0000","last_active":"2024-04-24T09:07:04.343333+0000","last_peered":"2024-04-24T09:07:04.343333+0000","last_clean":"2024-04-24T09:07:04.343333+0000","last_became_active":"2024-04-24T09:04:06.137492+0000","last_became_peered":"2024-04-24T09:04:06.137492+0000","last_unstale":"2024-04-24T09:07:04.343333+0000","last_undegraded":"2024-04-24T09:07:04.343333+0000","last_fullsized":"2024-04-24T09:07:04.343333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:44:00.653864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.882216+0000","last_change":"2024-04-24T09:04:01.544009+0000","last_active":"2024-04-24T09:07:03.882216+0000","last_peered":"2024-04-24T09:07:03.882216+0000","last_clean":"2024-04-24T09:07:03.882216+0000","last_became_active":"2024-04-24T09:04:01.543651+0000","last_became_peered":"2024-04-24T09:04:01.543651+0000","last_unstale":"2024-04-24T09:07:03.882216+0000","last_undegraded":"2024-04-24T09:07:03.882216+0000","last_fullsized":"2024-04-24T09:07:03.882216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:10:21.969072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.881564+0000","last_change":"2024-04-24T09:04:02.115387+0000","last_active":"2024-04-24T09:07:03.881564+0000","last_peered":"2024-04-24T09:07:03.881564+0000","last_clean":"2024-04-24T09:07:03.881564+0000","last_became_active":"2024-04-24T09:04:02.115086+0000","last_became_peered":"2024-04-24T09:04:02.115086+0000","last_unstale":"2024-04-24T09:07:03.881564+0000","last_undegraded":"2024-04-24T09:07:03.881564+0000","last_fullsized":"2024-04-24T09:07:03.881564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:40:57.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031574+0000","last_change":"2024-04-24T09:04:06.146189+0000","last_active":"2024-04-24T09:07:04.031574+0000","last_peered":"2024-04-24T09:07:04.031574+0000","last_clean":"2024-04-24T09:07:04.031574+0000","last_became_active":"2024-04-24T09:04:06.145556+0000","last_became_peered":"2024-04-24T09:04:06.145556+0000","last_unstale":"2024-04-24T09:07:04.031574+0000","last_undegraded":"2024-04-24T09:07:04.031574+0000","last_fullsized":"2024-04-24T09:07:04.031574+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:41.371948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.874786+0000","last_change":"2024-04-24T09:04:01.542185+0000","last_active":"2024-04-24T09:07:03.874786+0000","last_peered":"2024-04-24T09:07:03.874786+0000","last_clean":"2024-04-24T09:07:03.874786+0000","last_became_active":"2024-04-24T09:04:01.541240+0000","last_became_peered":"2024-04-24T09:04:01.541240+0000","last_unstale":"2024-04-24T09:07:03.874786+0000","last_undegraded":"2024-04-24T09:07:03.874786+0000","last_fullsized":"2024-04-24T09:07:03.874786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:15.103276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029990+0000","last_change":"2024-04-24T09:04:02.104054+0000","last_active":"2024-04-24T09:07:04.029990+0000","last_peered":"2024-04-24T09:07:04.029990+0000","last_clean":"2024-04-24T09:07:04.029990+0000","last_became_active":"2024-04-24T09:04:02.103673+0000","last_became_peered":"2024-04-24T09:04:02.103673+0000","last_unstale":"2024-04-24T09:07:04.029990+0000","last_undegraded":"2024-04-24T09:07:04.029990+0000","last_fullsized":"2024-04-24T09:07:04.029990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:31:58.889989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341462+0000","last_change":"2024-04-24T09:04:06.134034+0000","last_active":"2024-04-24T09:07:04.341462+0000","last_peered":"2024-04-24T09:07:04.341462+0000","last_clean":"2024-04-24T09:07:04.341462+0000","last_became_active":"2024-04-24T09:04:06.133596+0000","last_became_peered":"2024-04-24T09:04:06.133596+0000","last_unstale":"2024-04-24T09:07:04.341462+0000","last_undegraded":"2024-04-24T09:07:04.341462+0000","last_fullsized":"2024-04-24T09:07:04.341462+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:17:09.572974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880523+0000","last_change":"2024-04-24T09:04:01.543312+0000","last_active":"2024-04-24T09:07:03.880523+0000","last_peered":"2024-04-24T09:07:03.880523+0000","last_clean":"2024-04-24T09:07:03.880523+0000","last_became_active":"2024-04-24T09:04:01.542914+0000","last_became_peered":"2024-04-24T09:04:01.542914+0000","last_unstale":"2024-04-24T09:07:03.880523+0000","last_undegraded":"2024-04-24T09:07:03.880523+0000","last_fullsized":"2024-04-24T09:07:03.880523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:50.695108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.025810+0000","last_change":"2024-04-24T09:04:02.114445+0000","last_active":"2024-04-24T09:07:04.025810+0000","last_peered":"2024-04-24T09:07:04.025810+0000","last_clean":"2024-04-24T09:07:04.025810+0000","last_became_active":"2024-04-24T09:04:02.114075+0000","last_became_peered":"2024-04-24T09:04:02.114075+0000","last_unstale":"2024-04-24T09:07:04.025810+0000","last_undegraded":"2024-04-24T09:07:04.025810+0000","last_fullsized":"2024-04-24T09:07:04.025810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:35:06.933677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029729+0000","last_change":"2024-04-24T09:04:06.142368+0000","last_active":"2024-04-24T09:07:04.029729+0000","last_peered":"2024-04-24T09:07:04.029729+0000","last_clean":"2024-04-24T09:07:04.029729+0000","last_became_active":"2024-04-24T09:04:06.141872+0000","last_became_peered":"2024-04-24T09:04:06.141872+0000","last_unstale":"2024-04-24T09:07:04.029729+0000","last_undegraded":"2024-04-24T09:07:04.029729+0000","last_fullsized":"2024-04-24T09:07:04.029729+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:13:43.733662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":184,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883924+0000","last_change":"2024-04-24T09:04:01.556002+0000","last_active":"2024-04-24T09:07:03.883924+0000","last_peered":"2024-04-24T09:07:03.883924+0000","last_clean":"2024-04-24T09:07:03.883924+0000","last_became_active":"2024-04-24T09:04:01.555643+0000","last_became_peered":"2024-04-24T09:04:01.555643+0000","last_unstale":"2024-04-24T09:07:03.883924+0000","last_undegraded":"2024-04-24T09:07:03.883924+0000","last_fullsized":"2024-04-24T09:07:03.883924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:30:31.171184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":150,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342926+0000","last_change":"2024-04-24T09:04:02.104252+0000","last_active":"2024-04-24T09:07:04.342926+0000","last_peered":"2024-04-24T09:07:04.342926+0000","last_clean":"2024-04-24T09:07:04.342926+0000","last_became_active":"2024-04-24T09:04:02.099264+0000","last_became_peered":"2024-04-24T09:04:02.099264+0000","last_unstale":"2024-04-24T09:07:04.342926+0000","last_undegraded":"2024-04-24T09:07:04.342926+0000","last_fullsized":"2024-04-24T09:07:04.342926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:28.469814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.025340+0000","last_change":"2024-04-24T09:04:06.132407+0000","last_active":"2024-04-24T09:07:04.025340+0000","last_peered":"2024-04-24T09:07:04.025340+0000","last_clean":"2024-04-24T09:07:04.025340+0000","last_became_active":"2024-04-24T09:04:06.131997+0000","last_became_peered":"2024-04-24T09:04:06.131997+0000","last_unstale":"2024-04-24T09:07:04.025340+0000","last_undegraded":"2024-04-24T09:07:04.025340+0000","last_fullsized":"2024-04-24T09:07:04.025340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:08:07.369326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341438+0000","last_change":"2024-04-24T09:04:01.561039+0000","last_active":"2024-04-24T09:07:04.341438+0000","last_peered":"2024-04-24T09:07:04.341438+0000","last_clean":"2024-04-24T09:07:04.341438+0000","last_became_active":"2024-04-24T09:04:01.560543+0000","last_became_peered":"2024-04-24T09:04:01.560543+0000","last_unstale":"2024-04-24T09:07:04.341438+0000","last_undegraded":"2024-04-24T09:07:04.341438+0000","last_fullsized":"2024-04-24T09:07:04.341438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:27.706982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341945+0000","last_change":"2024-04-24T09:04:02.110850+0000","last_active":"2024-04-24T09:07:04.341945+0000","last_peered":"2024-04-24T09:07:04.341945+0000","last_clean":"2024-04-24T09:07:04.341945+0000","last_became_active":"2024-04-24T09:04:02.110512+0000","last_became_peered":"2024-04-24T09:04:02.110512+0000","last_unstale":"2024-04-24T09:07:04.341945+0000","last_undegraded":"2024-04-24T09:07:04.341945+0000","last_fullsized":"2024-04-24T09:07:04.341945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:19:16.937202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086513+0000","last_change":"2024-04-24T09:04:06.135200+0000","last_active":"2024-04-24T09:07:04.086513+0000","last_peered":"2024-04-24T09:07:04.086513+0000","last_clean":"2024-04-24T09:07:04.086513+0000","last_became_active":"2024-04-24T09:04:06.134680+0000","last_became_peered":"2024-04-24T09:04:06.134680+0000","last_unstale":"2024-04-24T09:07:04.086513+0000","last_undegraded":"2024-04-24T09:07:04.086513+0000","last_fullsized":"2024-04-24T09:07:04.086513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:48:57.697740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883243+0000","last_change":"2024-04-24T09:04:01.555069+0000","last_active":"2024-04-24T09:07:03.883243+0000","last_peered":"2024-04-24T09:07:03.883243+0000","last_clean":"2024-04-24T09:07:03.883243+0000","last_became_active":"2024-04-24T09:04:01.554737+0000","last_became_peered":"2024-04-24T09:04:01.554737+0000","last_unstale":"2024-04-24T09:07:03.883243+0000","last_undegraded":"2024-04-24T09:07:03.883243+0000","last_fullsized":"2024-04-24T09:07:03.883243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:23:00.272697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342607+0000","last_change":"2024-04-24T09:04:02.112126+0000","last_active":"2024-04-24T09:07:04.342607+0000","last_peered":"2024-04-24T09:07:04.342607+0000","last_clean":"2024-04-24T09:07:04.342607+0000","last_became_active":"2024-04-24T09:04:02.107995+0000","last_became_peered":"2024-04-24T09:04:02.107995+0000","last_unstale":"2024-04-24T09:07:04.342607+0000","last_undegraded":"2024-04-24T09:07:04.342607+0000","last_fullsized":"2024-04-24T09:07:04.342607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:13.189829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.881797+0000","last_change":"2024-04-24T09:04:24.542302+0000","last_active":"2024-04-24T09:07:03.881797+0000","last_peered":"2024-04-24T09:07:03.881797+0000","last_clean":"2024-04-24T09:07:03.881797+0000","last_became_active":"2024-04-24T09:04:24.539848+0000","last_became_peered":"2024-04-24T09:04:24.539848+0000","last_unstale":"2024-04-24T09:07:03.881797+0000","last_undegraded":"2024-04-24T09:07:03.881797+0000","last_fullsized":"2024-04-24T09:07:03.881797+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:01:52.055510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"93'46","reported_seq":389,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.442501+0000","last_change":"2024-04-24T09:04:01.548610+0000","last_active":"2024-04-24T09:07:04.442501+0000","last_peered":"2024-04-24T09:07:04.442501+0000","last_clean":"2024-04-24T09:07:04.442501+0000","last_became_active":"2024-04-24T09:04:01.548145+0000","last_became_peered":"2024-04-24T09:04:01.548145+0000","last_unstale":"2024-04-24T09:07:04.442501+0000","last_undegraded":"2024-04-24T09:07:04.442501+0000","last_fullsized":"2024-04-24T09:07:04.442501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:58:48.425187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71876,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":313,"num_read_kb":227,"num_write":47,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.922226+0000","last_change":"2024-04-24T09:04:02.109965+0000","last_active":"2024-04-24T09:07:03.922226+0000","last_peered":"2024-04-24T09:07:03.922226+0000","last_clean":"2024-04-24T09:07:03.922226+0000","last_became_active":"2024-04-24T09:04:02.108143+0000","last_became_peered":"2024-04-24T09:04:02.108143+0000","last_unstale":"2024-04-24T09:07:03.922226+0000","last_undegraded":"2024-04-24T09:07:03.922226+0000","last_fullsized":"2024-04-24T09:07:03.922226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:01:21.083039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.884532+0000","last_change":"2024-04-24T09:04:06.145369+0000","last_active":"2024-04-24T09:07:03.884532+0000","last_peered":"2024-04-24T09:07:03.884532+0000","last_clean":"2024-04-24T09:07:03.884532+0000","last_became_active":"2024-04-24T09:04:06.144884+0000","last_became_peered":"2024-04-24T09:04:06.144884+0000","last_unstale":"2024-04-24T09:07:03.884532+0000","last_undegraded":"2024-04-24T09:07:03.884532+0000","last_fullsized":"2024-04-24T09:07:03.884532+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:29:22.780846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343099+0000","last_change":"2024-04-24T09:04:01.548195+0000","last_active":"2024-04-24T09:07:04.343099+0000","last_peered":"2024-04-24T09:07:04.343099+0000","last_clean":"2024-04-24T09:07:04.343099+0000","last_became_active":"2024-04-24T09:04:01.544969+0000","last_became_peered":"2024-04-24T09:04:01.544969+0000","last_unstale":"2024-04-24T09:07:04.343099+0000","last_undegraded":"2024-04-24T09:07:04.343099+0000","last_fullsized":"2024-04-24T09:07:04.343099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:32:55.294629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.875735+0000","last_change":"2024-04-24T09:04:02.103785+0000","last_active":"2024-04-24T09:07:03.875735+0000","last_peered":"2024-04-24T09:07:03.875735+0000","last_clean":"2024-04-24T09:07:03.875735+0000","last_became_active":"2024-04-24T09:04:02.103463+0000","last_became_peered":"2024-04-24T09:04:02.103463+0000","last_unstale":"2024-04-24T09:07:03.875735+0000","last_undegraded":"2024-04-24T09:07:03.875735+0000","last_fullsized":"2024-04-24T09:07:03.875735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:56.385382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031518+0000","last_change":"2024-04-24T09:04:06.149955+0000","last_active":"2024-04-24T09:07:04.031518+0000","last_peered":"2024-04-24T09:07:04.031518+0000","last_clean":"2024-04-24T09:07:04.031518+0000","last_became_active":"2024-04-24T09:04:06.149445+0000","last_became_peered":"2024-04-24T09:04:06.149445+0000","last_unstale":"2024-04-24T09:07:04.031518+0000","last_undegraded":"2024-04-24T09:07:04.031518+0000","last_fullsized":"2024-04-24T09:07:04.031518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:30:42.527007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.882015+0000","last_change":"2024-04-24T09:04:01.548740+0000","last_active":"2024-04-24T09:07:03.882015+0000","last_peered":"2024-04-24T09:07:03.882015+0000","last_clean":"2024-04-24T09:07:03.882015+0000","last_became_active":"2024-04-24T09:04:01.546060+0000","last_became_peered":"2024-04-24T09:04:01.546060+0000","last_unstale":"2024-04-24T09:07:03.882015+0000","last_undegraded":"2024-04-24T09:07:03.882015+0000","last_fullsized":"2024-04-24T09:07:03.882015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:09:16.301150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341787+0000","last_change":"2024-04-24T09:04:24.538871+0000","last_active":"2024-04-24T09:07:04.341787+0000","last_peered":"2024-04-24T09:07:04.341787+0000","last_clean":"2024-04-24T09:07:04.341787+0000","last_became_active":"2024-04-24T09:04:24.537039+0000","last_became_peered":"2024-04-24T09:04:24.537039+0000","last_unstale":"2024-04-24T09:07:04.341787+0000","last_undegraded":"2024-04-24T09:07:04.341787+0000","last_fullsized":"2024-04-24T09:07:04.341787+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:18.520407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.026577+0000","last_change":"2024-04-24T09:04:06.136973+0000","last_active":"2024-04-24T09:07:04.026577+0000","last_peered":"2024-04-24T09:07:04.026577+0000","last_clean":"2024-04-24T09:07:04.026577+0000","last_became_active":"2024-04-24T09:04:06.136405+0000","last_became_peered":"2024-04-24T09:04:06.136405+0000","last_unstale":"2024-04-24T09:07:04.026577+0000","last_undegraded":"2024-04-24T09:07:04.026577+0000","last_fullsized":"2024-04-24T09:07:04.026577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:07.653162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":191,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086263+0000","last_change":"2024-04-24T09:04:01.546853+0000","last_active":"2024-04-24T09:07:04.086263+0000","last_peered":"2024-04-24T09:07:04.086263+0000","last_clean":"2024-04-24T09:07:04.086263+0000","last_became_active":"2024-04-24T09:04:01.546441+0000","last_became_peered":"2024-04-24T09:04:01.546441+0000","last_unstale":"2024-04-24T09:07:04.086263+0000","last_undegraded":"2024-04-24T09:07:04.086263+0000","last_fullsized":"2024-04-24T09:07:04.086263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:23:56.333410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341712+0000","last_change":"2024-04-24T09:04:02.101158+0000","last_active":"2024-04-24T09:07:04.341712+0000","last_peered":"2024-04-24T09:07:04.341712+0000","last_clean":"2024-04-24T09:07:04.341712+0000","last_became_active":"2024-04-24T09:04:02.100832+0000","last_became_peered":"2024-04-24T09:04:02.100832+0000","last_unstale":"2024-04-24T09:07:04.341712+0000","last_undegraded":"2024-04-24T09:07:04.341712+0000","last_fullsized":"2024-04-24T09:07:04.341712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:05:34.080520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.878892+0000","last_change":"2024-04-24T09:04:06.149176+0000","last_active":"2024-04-24T09:07:03.878892+0000","last_peered":"2024-04-24T09:07:03.878892+0000","last_clean":"2024-04-24T09:07:03.878892+0000","last_became_active":"2024-04-24T09:04:06.145539+0000","last_became_peered":"2024-04-24T09:04:06.145539+0000","last_unstale":"2024-04-24T09:07:03.878892+0000","last_undegraded":"2024-04-24T09:07:03.878892+0000","last_fullsized":"2024-04-24T09:07:03.878892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:38.719787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921175+0000","last_change":"2024-04-24T09:04:01.555330+0000","last_active":"2024-04-24T09:07:03.921175+0000","last_peered":"2024-04-24T09:07:03.921175+0000","last_clean":"2024-04-24T09:07:03.921175+0000","last_became_active":"2024-04-24T09:04:01.552618+0000","last_became_peered":"2024-04-24T09:04:01.552618+0000","last_unstale":"2024-04-24T09:07:03.921175+0000","last_undegraded":"2024-04-24T09:07:03.921175+0000","last_fullsized":"2024-04-24T09:07:03.921175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:30.956657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880986+0000","last_change":"2024-04-24T09:04:02.116301+0000","last_active":"2024-04-24T09:07:03.880986+0000","last_peered":"2024-04-24T09:07:03.880986+0000","last_clean":"2024-04-24T09:07:03.880986+0000","last_became_active":"2024-04-24T09:04:02.116001+0000","last_became_peered":"2024-04-24T09:04:02.116001+0000","last_unstale":"2024-04-24T09:07:03.880986+0000","last_undegraded":"2024-04-24T09:07:03.880986+0000","last_fullsized":"2024-04-24T09:07:03.880986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:16:18.995067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880626+0000","last_change":"2024-04-24T09:04:06.135814+0000","last_active":"2024-04-24T09:07:03.880626+0000","last_peered":"2024-04-24T09:07:03.880626+0000","last_clean":"2024-04-24T09:07:03.880626+0000","last_became_active":"2024-04-24T09:04:06.135054+0000","last_became_peered":"2024-04-24T09:04:06.135054+0000","last_unstale":"2024-04-24T09:07:03.880626+0000","last_undegraded":"2024-04-24T09:07:03.880626+0000","last_fullsized":"2024-04-24T09:07:03.880626+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:51:31.150589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.922651+0000","last_change":"2024-04-24T09:04:01.553144+0000","last_active":"2024-04-24T09:07:03.922651+0000","last_peered":"2024-04-24T09:07:03.922651+0000","last_clean":"2024-04-24T09:07:03.922651+0000","last_became_active":"2024-04-24T09:04:01.550599+0000","last_became_peered":"2024-04-24T09:04:01.550599+0000","last_unstale":"2024-04-24T09:07:03.922651+0000","last_undegraded":"2024-04-24T09:07:03.922651+0000","last_fullsized":"2024-04-24T09:07:03.922651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:41.950209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031628+0000","last_change":"2024-04-24T09:04:02.117064+0000","last_active":"2024-04-24T09:07:04.031628+0000","last_peered":"2024-04-24T09:07:04.031628+0000","last_clean":"2024-04-24T09:07:04.031628+0000","last_became_active":"2024-04-24T09:04:02.114528+0000","last_became_peered":"2024-04-24T09:04:02.114528+0000","last_unstale":"2024-04-24T09:07:04.031628+0000","last_undegraded":"2024-04-24T09:07:04.031628+0000","last_fullsized":"2024-04-24T09:07:04.031628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:24:53.662571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921692+0000","last_change":"2024-04-24T09:04:24.701058+0000","last_active":"2024-04-24T09:07:03.921692+0000","last_peered":"2024-04-24T09:07:03.921692+0000","last_clean":"2024-04-24T09:07:03.921692+0000","last_became_active":"2024-04-24T09:04:24.699766+0000","last_became_peered":"2024-04-24T09:04:24.699766+0000","last_unstale":"2024-04-24T09:07:03.921692+0000","last_undegraded":"2024-04-24T09:07:03.921692+0000","last_fullsized":"2024-04-24T09:07:03.921692+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:57:48.491073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.026283+0000","last_change":"2024-04-24T09:04:24.701816+0000","last_active":"2024-04-24T09:07:04.026283+0000","last_peered":"2024-04-24T09:07:04.026283+0000","last_clean":"2024-04-24T09:07:04.026283+0000","last_became_active":"2024-04-24T09:04:24.700530+0000","last_became_peered":"2024-04-24T09:04:24.700530+0000","last_unstale":"2024-04-24T09:07:04.026283+0000","last_undegraded":"2024-04-24T09:07:04.026283+0000","last_fullsized":"2024-04-24T09:07:04.026283+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:24.614043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883310+0000","last_change":"2024-04-24T09:04:02.104602+0000","last_active":"2024-04-24T09:07:03.883310+0000","last_peered":"2024-04-24T09:07:03.883310+0000","last_clean":"2024-04-24T09:07:03.883310+0000","last_became_active":"2024-04-24T09:04:02.104188+0000","last_became_peered":"2024-04-24T09:04:02.104188+0000","last_unstale":"2024-04-24T09:07:03.883310+0000","last_undegraded":"2024-04-24T09:07:03.883310+0000","last_fullsized":"2024-04-24T09:07:03.883310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:00:50.286952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.876427+0000","last_change":"2024-04-24T09:04:06.134971+0000","last_active":"2024-04-24T09:07:03.876427+0000","last_peered":"2024-04-24T09:07:03.876427+0000","last_clean":"2024-04-24T09:07:03.876427+0000","last_became_active":"2024-04-24T09:04:06.134551+0000","last_became_peered":"2024-04-24T09:04:06.134551+0000","last_unstale":"2024-04-24T09:07:03.876427+0000","last_undegraded":"2024-04-24T09:07:03.876427+0000","last_fullsized":"2024-04-24T09:07:03.876427+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:40:41.766657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341353+0000","last_change":"2024-04-24T09:04:01.551117+0000","last_active":"2024-04-24T09:07:04.341353+0000","last_peered":"2024-04-24T09:07:04.341353+0000","last_clean":"2024-04-24T09:07:04.341353+0000","last_became_active":"2024-04-24T09:04:01.550646+0000","last_became_peered":"2024-04-24T09:04:01.550646+0000","last_unstale":"2024-04-24T09:07:04.341353+0000","last_undegraded":"2024-04-24T09:07:04.341353+0000","last_fullsized":"2024-04-24T09:07:04.341353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:29:03.821300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.882733+0000","last_change":"2024-04-24T09:04:02.107111+0000","last_active":"2024-04-24T09:07:03.882733+0000","last_peered":"2024-04-24T09:07:03.882733+0000","last_clean":"2024-04-24T09:07:03.882733+0000","last_became_active":"2024-04-24T09:04:02.106185+0000","last_became_peered":"2024-04-24T09:04:02.106185+0000","last_unstale":"2024-04-24T09:07:03.882733+0000","last_undegraded":"2024-04-24T09:07:03.882733+0000","last_fullsized":"2024-04-24T09:07:03.882733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:19:03.111407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.877283+0000","last_change":"2024-04-24T09:04:06.143984+0000","last_active":"2024-04-24T09:07:03.877283+0000","last_peered":"2024-04-24T09:07:03.877283+0000","last_clean":"2024-04-24T09:07:03.877283+0000","last_became_active":"2024-04-24T09:04:06.143503+0000","last_became_peered":"2024-04-24T09:04:06.143503+0000","last_unstale":"2024-04-24T09:07:03.877283+0000","last_undegraded":"2024-04-24T09:07:03.877283+0000","last_fullsized":"2024-04-24T09:07:03.877283+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:35:19.317771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880870+0000","last_change":"2024-04-24T09:04:02.111560+0000","last_active":"2024-04-24T09:07:03.880870+0000","last_peered":"2024-04-24T09:07:03.880870+0000","last_clean":"2024-04-24T09:07:03.880870+0000","last_became_active":"2024-04-24T09:04:02.109885+0000","last_became_peered":"2024-04-24T09:04:02.109885+0000","last_unstale":"2024-04-24T09:07:03.880870+0000","last_undegraded":"2024-04-24T09:07:03.880870+0000","last_fullsized":"2024-04-24T09:07:03.880870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:35.837621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":186,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.874716+0000","last_change":"2024-04-24T09:04:01.542530+0000","last_active":"2024-04-24T09:07:03.874716+0000","last_peered":"2024-04-24T09:07:03.874716+0000","last_clean":"2024-04-24T09:07:03.874716+0000","last_became_active":"2024-04-24T09:04:01.541702+0000","last_became_peered":"2024-04-24T09:04:01.541702+0000","last_unstale":"2024-04-24T09:07:03.874716+0000","last_undegraded":"2024-04-24T09:07:03.874716+0000","last_fullsized":"2024-04-24T09:07:03.874716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:28:29.420020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":150,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341380+0000","last_change":"2024-04-24T09:04:06.143004+0000","last_active":"2024-04-24T09:07:04.341380+0000","last_peered":"2024-04-24T09:07:04.341380+0000","last_clean":"2024-04-24T09:07:04.341380+0000","last_became_active":"2024-04-24T09:04:06.142444+0000","last_became_peered":"2024-04-24T09:04:06.142444+0000","last_unstale":"2024-04-24T09:07:04.341380+0000","last_undegraded":"2024-04-24T09:07:04.341380+0000","last_fullsized":"2024-04-24T09:07:04.341380+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:13.910612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.025614+0000","last_change":"2024-04-24T09:04:24.701330+0000","last_active":"2024-04-24T09:07:04.025614+0000","last_peered":"2024-04-24T09:07:04.025614+0000","last_clean":"2024-04-24T09:07:04.025614+0000","last_became_active":"2024-04-24T09:04:24.699988+0000","last_became_peered":"2024-04-24T09:04:24.699988+0000","last_unstale":"2024-04-24T09:07:04.025614+0000","last_undegraded":"2024-04-24T09:07:04.025614+0000","last_fullsized":"2024-04-24T09:07:04.025614+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:23:24.610671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086340+0000","last_change":"2024-04-24T09:04:01.541206+0000","last_active":"2024-04-24T09:07:04.086340+0000","last_peered":"2024-04-24T09:07:04.086340+0000","last_clean":"2024-04-24T09:07:04.086340+0000","last_became_active":"2024-04-24T09:04:01.539888+0000","last_became_peered":"2024-04-24T09:04:01.539888+0000","last_unstale":"2024-04-24T09:07:04.086340+0000","last_undegraded":"2024-04-24T09:07:04.086340+0000","last_fullsized":"2024-04-24T09:07:04.086340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:01.550209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341017+0000","last_change":"2024-04-24T09:02:22.629840+0000","last_active":"2024-04-24T09:07:04.341017+0000","last_peered":"2024-04-24T09:07:04.341017+0000","last_clean":"2024-04-24T09:07:04.341017+0000","last_became_active":"2024-04-24T09:02:22.628700+0000","last_became_peered":"2024-04-24T09:02:22.628700+0000","last_unstale":"2024-04-24T09:07:04.341017+0000","last_undegraded":"2024-04-24T09:07:04.341017+0000","last_fullsized":"2024-04-24T09:07:04.341017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:57:38.349295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:57:38.349295+0000","last_clean_scrub_stamp":"2024-04-24T08:57:38.349295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:54:11.409492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883073+0000","last_change":"2024-04-24T09:04:06.149548+0000","last_active":"2024-04-24T09:07:03.883073+0000","last_peered":"2024-04-24T09:07:03.883073+0000","last_clean":"2024-04-24T09:07:03.883073+0000","last_became_active":"2024-04-24T09:04:06.149214+0000","last_became_peered":"2024-04-24T09:04:06.149214+0000","last_unstale":"2024-04-24T09:07:03.883073+0000","last_undegraded":"2024-04-24T09:07:03.883073+0000","last_fullsized":"2024-04-24T09:07:03.883073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:03.801242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.874677+0000","last_change":"2024-04-24T09:04:02.110242+0000","last_active":"2024-04-24T09:07:03.874677+0000","last_peered":"2024-04-24T09:07:03.874677+0000","last_clean":"2024-04-24T09:07:03.874677+0000","last_became_active":"2024-04-24T09:04:02.109879+0000","last_became_peered":"2024-04-24T09:04:02.109879+0000","last_unstale":"2024-04-24T09:07:03.874677+0000","last_undegraded":"2024-04-24T09:07:03.874677+0000","last_fullsized":"2024-04-24T09:07:03.874677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:16.353861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341211+0000","last_change":"2024-04-24T09:04:01.550989+0000","last_active":"2024-04-24T09:07:04.341211+0000","last_peered":"2024-04-24T09:07:04.341211+0000","last_clean":"2024-04-24T09:07:04.341211+0000","last_became_active":"2024-04-24T09:04:01.550664+0000","last_became_peered":"2024-04-24T09:04:01.550664+0000","last_unstale":"2024-04-24T09:07:04.341211+0000","last_undegraded":"2024-04-24T09:07:04.341211+0000","last_fullsized":"2024-04-24T09:07:04.341211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:33:39.521951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.030916+0000","last_change":"2024-04-24T09:04:24.538207+0000","last_active":"2024-04-24T09:07:04.030916+0000","last_peered":"2024-04-24T09:07:04.030916+0000","last_clean":"2024-04-24T09:07:04.030916+0000","last_became_active":"2024-04-24T09:04:24.536894+0000","last_became_peered":"2024-04-24T09:04:24.536894+0000","last_unstale":"2024-04-24T09:07:04.030916+0000","last_undegraded":"2024-04-24T09:07:04.030916+0000","last_fullsized":"2024-04-24T09:07:04.030916+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:32:52.860371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.922335+0000","last_change":"2024-04-24T09:04:01.539415+0000","last_active":"2024-04-24T09:07:03.922335+0000","last_peered":"2024-04-24T09:07:03.922335+0000","last_clean":"2024-04-24T09:07:03.922335+0000","last_became_active":"2024-04-24T09:04:01.539020+0000","last_became_peered":"2024-04-24T09:04:01.539020+0000","last_unstale":"2024-04-24T09:07:03.922335+0000","last_undegraded":"2024-04-24T09:07:03.922335+0000","last_fullsized":"2024-04-24T09:07:03.922335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:12:48.728029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.877295+0000","last_change":"2024-04-24T09:04:02.097560+0000","last_active":"2024-04-24T09:07:03.877295+0000","last_peered":"2024-04-24T09:07:03.877295+0000","last_clean":"2024-04-24T09:07:03.877295+0000","last_became_active":"2024-04-24T09:04:02.095612+0000","last_became_peered":"2024-04-24T09:04:02.095612+0000","last_unstale":"2024-04-24T09:07:03.877295+0000","last_undegraded":"2024-04-24T09:07:03.877295+0000","last_fullsized":"2024-04-24T09:07:03.877295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:55:47.747294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029686+0000","last_change":"2024-04-24T09:04:06.137822+0000","last_active":"2024-04-24T09:07:04.029686+0000","last_peered":"2024-04-24T09:07:04.029686+0000","last_clean":"2024-04-24T09:07:04.029686+0000","last_became_active":"2024-04-24T09:04:06.137357+0000","last_became_peered":"2024-04-24T09:04:06.137357+0000","last_unstale":"2024-04-24T09:07:04.029686+0000","last_undegraded":"2024-04-24T09:07:04.029686+0000","last_fullsized":"2024-04-24T09:07:04.029686+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:55:13.119557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'46","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.275253+0000","last_change":"2024-04-24T09:04:24.544395+0000","last_active":"2024-04-24T09:07:04.275253+0000","last_peered":"2024-04-24T09:07:04.275253+0000","last_clean":"2024-04-24T09:07:04.275253+0000","last_became_active":"2024-04-24T09:04:24.535870+0000","last_became_peered":"2024-04-24T09:04:24.535870+0000","last_unstale":"2024-04-24T09:07:04.275253+0000","last_undegraded":"2024-04-24T09:07:04.275253+0000","last_fullsized":"2024-04-24T09:07:04.275253+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:29:21.271387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031426+0000","last_change":"2024-04-24T09:04:02.106107+0000","last_active":"2024-04-24T09:07:04.031426+0000","last_peered":"2024-04-24T09:07:04.031426+0000","last_clean":"2024-04-24T09:07:04.031426+0000","last_became_active":"2024-04-24T09:04:02.105734+0000","last_became_peered":"2024-04-24T09:04:02.105734+0000","last_unstale":"2024-04-24T09:07:04.031426+0000","last_undegraded":"2024-04-24T09:07:04.031426+0000","last_fullsized":"2024-04-24T09:07:04.031426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:15.372520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342631+0000","last_change":"2024-04-24T09:04:06.139040+0000","last_active":"2024-04-24T09:07:04.342631+0000","last_peered":"2024-04-24T09:07:04.342631+0000","last_clean":"2024-04-24T09:07:04.342631+0000","last_became_active":"2024-04-24T09:04:06.138532+0000","last_became_peered":"2024-04-24T09:04:06.138532+0000","last_unstale":"2024-04-24T09:07:04.342631+0000","last_undegraded":"2024-04-24T09:07:04.342631+0000","last_fullsized":"2024-04-24T09:07:04.342631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:28.686052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.087005+0000","last_change":"2024-04-24T09:04:01.546859+0000","last_active":"2024-04-24T09:07:04.087005+0000","last_peered":"2024-04-24T09:07:04.087005+0000","last_clean":"2024-04-24T09:07:04.087005+0000","last_became_active":"2024-04-24T09:04:01.546463+0000","last_became_peered":"2024-04-24T09:04:01.546463+0000","last_unstale":"2024-04-24T09:07:04.087005+0000","last_undegraded":"2024-04-24T09:07:04.087005+0000","last_fullsized":"2024-04-24T09:07:04.087005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:31:12.985687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341595+0000","last_change":"2024-04-24T09:04:02.112487+0000","last_active":"2024-04-24T09:07:04.341595+0000","last_peered":"2024-04-24T09:07:04.341595+0000","last_clean":"2024-04-24T09:07:04.341595+0000","last_became_active":"2024-04-24T09:04:02.112078+0000","last_became_peered":"2024-04-24T09:04:02.112078+0000","last_unstale":"2024-04-24T09:07:04.341595+0000","last_undegraded":"2024-04-24T09:07:04.341595+0000","last_fullsized":"2024-04-24T09:07:04.341595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:03:47.352110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086370+0000","last_change":"2024-04-24T09:04:24.539120+0000","last_active":"2024-04-24T09:07:04.086370+0000","last_peered":"2024-04-24T09:07:04.086370+0000","last_clean":"2024-04-24T09:07:04.086370+0000","last_became_active":"2024-04-24T09:04:24.537048+0000","last_became_peered":"2024-04-24T09:04:24.537048+0000","last_unstale":"2024-04-24T09:07:04.086370+0000","last_undegraded":"2024-04-24T09:07:04.086370+0000","last_fullsized":"2024-04-24T09:07:04.086370+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:29:53.484131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342056+0000","last_change":"2024-04-24T09:04:01.545444+0000","last_active":"2024-04-24T09:07:04.342056+0000","last_peered":"2024-04-24T09:07:04.342056+0000","last_clean":"2024-04-24T09:07:04.342056+0000","last_became_active":"2024-04-24T09:04:01.543816+0000","last_became_peered":"2024-04-24T09:04:01.543816+0000","last_unstale":"2024-04-24T09:07:04.342056+0000","last_undegraded":"2024-04-24T09:07:04.342056+0000","last_fullsized":"2024-04-24T09:07:04.342056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:02:36.221797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921232+0000","last_change":"2024-04-24T09:04:02.109085+0000","last_active":"2024-04-24T09:07:03.921232+0000","last_peered":"2024-04-24T09:07:03.921232+0000","last_clean":"2024-04-24T09:07:03.921232+0000","last_became_active":"2024-04-24T09:04:02.108642+0000","last_became_peered":"2024-04-24T09:04:02.108642+0000","last_unstale":"2024-04-24T09:07:03.921232+0000","last_undegraded":"2024-04-24T09:07:03.921232+0000","last_fullsized":"2024-04-24T09:07:03.921232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:27:51.751369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.026721+0000","last_change":"2024-04-24T09:04:06.144940+0000","last_active":"2024-04-24T09:07:04.026721+0000","last_peered":"2024-04-24T09:07:04.026721+0000","last_clean":"2024-04-24T09:07:04.026721+0000","last_became_active":"2024-04-24T09:04:06.144553+0000","last_became_peered":"2024-04-24T09:04:06.144553+0000","last_unstale":"2024-04-24T09:07:04.026721+0000","last_undegraded":"2024-04-24T09:07:04.026721+0000","last_fullsized":"2024-04-24T09:07:04.026721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:17.455644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029831+0000","last_change":"2024-04-24T09:04:01.538939+0000","last_active":"2024-04-24T09:07:04.029831+0000","last_peered":"2024-04-24T09:07:04.029831+0000","last_clean":"2024-04-24T09:07:04.029831+0000","last_became_active":"2024-04-24T09:04:01.538541+0000","last_became_peered":"2024-04-24T09:04:01.538541+0000","last_unstale":"2024-04-24T09:07:04.029831+0000","last_undegraded":"2024-04-24T09:07:04.029831+0000","last_fullsized":"2024-04-24T09:07:04.029831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:08.185901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.030917+0000","last_change":"2024-04-24T09:04:02.103844+0000","last_active":"2024-04-24T09:07:04.030917+0000","last_peered":"2024-04-24T09:07:04.030917+0000","last_clean":"2024-04-24T09:07:04.030917+0000","last_became_active":"2024-04-24T09:04:02.103416+0000","last_became_peered":"2024-04-24T09:04:02.103416+0000","last_unstale":"2024-04-24T09:07:04.030917+0000","last_undegraded":"2024-04-24T09:07:04.030917+0000","last_fullsized":"2024-04-24T09:07:04.030917+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:57:47.236384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086087+0000","last_change":"2024-04-24T09:04:06.134838+0000","last_active":"2024-04-24T09:07:04.086087+0000","last_peered":"2024-04-24T09:07:04.086087+0000","last_clean":"2024-04-24T09:07:04.086087+0000","last_became_active":"2024-04-24T09:04:06.134343+0000","last_became_peered":"2024-04-24T09:04:06.134343+0000","last_unstale":"2024-04-24T09:07:04.086087+0000","last_undegraded":"2024-04-24T09:07:04.086087+0000","last_fullsized":"2024-04-24T09:07:04.086087+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:56:55.219012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343211+0000","last_change":"2024-04-24T09:04:01.557194+0000","last_active":"2024-04-24T09:07:04.343211+0000","last_peered":"2024-04-24T09:07:04.343211+0000","last_clean":"2024-04-24T09:07:04.343211+0000","last_became_active":"2024-04-24T09:04:01.556441+0000","last_became_peered":"2024-04-24T09:04:01.556441+0000","last_unstale":"2024-04-24T09:07:04.343211+0000","last_undegraded":"2024-04-24T09:07:04.343211+0000","last_fullsized":"2024-04-24T09:07:04.343211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:45:50.399096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.030826+0000","last_change":"2024-04-24T09:04:02.108076+0000","last_active":"2024-04-24T09:07:04.030826+0000","last_peered":"2024-04-24T09:07:04.030826+0000","last_clean":"2024-04-24T09:07:04.030826+0000","last_became_active":"2024-04-24T09:04:02.106829+0000","last_became_peered":"2024-04-24T09:04:02.106829+0000","last_unstale":"2024-04-24T09:07:04.030826+0000","last_undegraded":"2024-04-24T09:07:04.030826+0000","last_fullsized":"2024-04-24T09:07:04.030826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:40:23.075828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086526+0000","last_change":"2024-04-24T09:04:06.142269+0000","last_active":"2024-04-24T09:07:04.086526+0000","last_peered":"2024-04-24T09:07:04.086526+0000","last_clean":"2024-04-24T09:07:04.086526+0000","last_became_active":"2024-04-24T09:04:06.141725+0000","last_became_peered":"2024-04-24T09:04:06.141725+0000","last_unstale":"2024-04-24T09:07:04.086526+0000","last_undegraded":"2024-04-24T09:07:04.086526+0000","last_fullsized":"2024-04-24T09:07:04.086526+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:28:17.934148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921558+0000","last_change":"2024-04-24T09:04:01.553134+0000","last_active":"2024-04-24T09:07:03.921558+0000","last_peered":"2024-04-24T09:07:03.921558+0000","last_clean":"2024-04-24T09:07:03.921558+0000","last_became_active":"2024-04-24T09:04:01.552099+0000","last_became_peered":"2024-04-24T09:04:01.552099+0000","last_unstale":"2024-04-24T09:07:03.921558+0000","last_undegraded":"2024-04-24T09:07:03.921558+0000","last_fullsized":"2024-04-24T09:07:03.921558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:42:36.231405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341535+0000","last_change":"2024-04-24T09:04:02.116641+0000","last_active":"2024-04-24T09:07:04.341535+0000","last_peered":"2024-04-24T09:07:04.341535+0000","last_clean":"2024-04-24T09:07:04.341535+0000","last_became_active":"2024-04-24T09:04:02.115374+0000","last_became_peered":"2024-04-24T09:04:02.115374+0000","last_unstale":"2024-04-24T09:07:04.341535+0000","last_undegraded":"2024-04-24T09:07:04.341535+0000","last_fullsized":"2024-04-24T09:07:04.341535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:34.450990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342365+0000","last_change":"2024-04-24T09:04:06.149968+0000","last_active":"2024-04-24T09:07:04.342365+0000","last_peered":"2024-04-24T09:07:04.342365+0000","last_clean":"2024-04-24T09:07:04.342365+0000","last_became_active":"2024-04-24T09:04:06.149425+0000","last_became_peered":"2024-04-24T09:04:06.149425+0000","last_unstale":"2024-04-24T09:07:04.342365+0000","last_undegraded":"2024-04-24T09:07:04.342365+0000","last_fullsized":"2024-04-24T09:07:04.342365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:23:53.869004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343669+0000","last_change":"2024-04-24T09:04:01.553822+0000","last_active":"2024-04-24T09:07:04.343669+0000","last_peered":"2024-04-24T09:07:04.343669+0000","last_clean":"2024-04-24T09:07:04.343669+0000","last_became_active":"2024-04-24T09:04:01.551468+0000","last_became_peered":"2024-04-24T09:04:01.551468+0000","last_unstale":"2024-04-24T09:07:04.343669+0000","last_undegraded":"2024-04-24T09:07:04.343669+0000","last_fullsized":"2024-04-24T09:07:04.343669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:56.367584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342035+0000","last_change":"2024-04-24T09:04:02.115472+0000","last_active":"2024-04-24T09:07:04.342035+0000","last_peered":"2024-04-24T09:07:04.342035+0000","last_clean":"2024-04-24T09:07:04.342035+0000","last_became_active":"2024-04-24T09:04:02.114666+0000","last_became_peered":"2024-04-24T09:04:02.114666+0000","last_unstale":"2024-04-24T09:07:04.342035+0000","last_undegraded":"2024-04-24T09:07:04.342035+0000","last_fullsized":"2024-04-24T09:07:04.342035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:28:39.145108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341009+0000","last_change":"2024-04-24T09:04:06.138786+0000","last_active":"2024-04-24T09:07:04.341009+0000","last_peered":"2024-04-24T09:07:04.341009+0000","last_clean":"2024-04-24T09:07:04.341009+0000","last_became_active":"2024-04-24T09:04:06.138366+0000","last_became_peered":"2024-04-24T09:04:06.138366+0000","last_unstale":"2024-04-24T09:07:04.341009+0000","last_undegraded":"2024-04-24T09:07:04.341009+0000","last_fullsized":"2024-04-24T09:07:04.341009+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:03:16.729928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"93'39","reported_seq":76,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.265142+0000","last_change":"2024-04-24T09:04:01.547733+0000","last_active":"2024-04-24T09:07:04.265142+0000","last_peered":"2024-04-24T09:07:04.265142+0000","last_clean":"2024-04-24T09:07:04.265142+0000","last_became_active":"2024-04-24T09:04:01.544839+0000","last_became_peered":"2024-04-24T09:04:01.544839+0000","last_unstale":"2024-04-24T09:07:04.265142+0000","last_undegraded":"2024-04-24T09:07:04.265142+0000","last_fullsized":"2024-04-24T09:07:04.265142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:45:03.847092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341471+0000","last_change":"2024-04-24T09:04:02.108383+0000","last_active":"2024-04-24T09:07:04.341471+0000","last_peered":"2024-04-24T09:07:04.341471+0000","last_clean":"2024-04-24T09:07:04.341471+0000","last_became_active":"2024-04-24T09:04:02.107883+0000","last_became_peered":"2024-04-24T09:04:02.107883+0000","last_unstale":"2024-04-24T09:07:04.341471+0000","last_undegraded":"2024-04-24T09:07:04.341471+0000","last_fullsized":"2024-04-24T09:07:04.341471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:24:44.923708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880074+0000","last_change":"2024-04-24T09:04:06.140868+0000","last_active":"2024-04-24T09:07:03.880074+0000","last_peered":"2024-04-24T09:07:03.880074+0000","last_clean":"2024-04-24T09:07:03.880074+0000","last_became_active":"2024-04-24T09:04:06.139174+0000","last_became_peered":"2024-04-24T09:04:06.139174+0000","last_unstale":"2024-04-24T09:07:03.880074+0000","last_undegraded":"2024-04-24T09:07:03.880074+0000","last_fullsized":"2024-04-24T09:07:03.880074+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:17:17.334276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.878570+0000","last_change":"2024-04-24T09:04:01.534653+0000","last_active":"2024-04-24T09:07:03.878570+0000","last_peered":"2024-04-24T09:07:03.878570+0000","last_clean":"2024-04-24T09:07:03.878570+0000","last_became_active":"2024-04-24T09:04:01.534312+0000","last_became_peered":"2024-04-24T09:04:01.534312+0000","last_unstale":"2024-04-24T09:07:03.878570+0000","last_undegraded":"2024-04-24T09:07:03.878570+0000","last_fullsized":"2024-04-24T09:07:03.878570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:53:34.301007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341883+0000","last_change":"2024-04-24T09:04:02.102056+0000","last_active":"2024-04-24T09:07:04.341883+0000","last_peered":"2024-04-24T09:07:04.341883+0000","last_clean":"2024-04-24T09:07:04.341883+0000","last_became_active":"2024-04-24T09:04:02.101641+0000","last_became_peered":"2024-04-24T09:04:02.101641+0000","last_unstale":"2024-04-24T09:07:04.341883+0000","last_undegraded":"2024-04-24T09:07:04.341883+0000","last_fullsized":"2024-04-24T09:07:04.341883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:41.654024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031176+0000","last_change":"2024-04-24T09:04:02.100327+0000","last_active":"2024-04-24T09:07:04.031176+0000","last_peered":"2024-04-24T09:07:04.031176+0000","last_clean":"2024-04-24T09:07:04.031176+0000","last_became_active":"2024-04-24T09:04:02.100028+0000","last_became_peered":"2024-04-24T09:04:02.100028+0000","last_unstale":"2024-04-24T09:07:04.031176+0000","last_undegraded":"2024-04-24T09:07:04.031176+0000","last_fullsized":"2024-04-24T09:07:04.031176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:15:18.728045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343576+0000","last_change":"2024-04-24T09:04:02.101996+0000","last_active":"2024-04-24T09:07:04.343576+0000","last_peered":"2024-04-24T09:07:04.343576+0000","last_clean":"2024-04-24T09:07:04.343576+0000","last_became_active":"2024-04-24T09:04:02.101581+0000","last_became_peered":"2024-04-24T09:04:02.101581+0000","last_unstale":"2024-04-24T09:07:04.343576+0000","last_undegraded":"2024-04-24T09:07:04.343576+0000","last_fullsized":"2024-04-24T09:07:04.343576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:55.487781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342748+0000","last_change":"2024-04-24T09:04:02.107288+0000","last_active":"2024-04-24T09:07:04.342748+0000","last_peered":"2024-04-24T09:07:04.342748+0000","last_clean":"2024-04-24T09:07:04.342748+0000","last_became_active":"2024-04-24T09:04:02.106946+0000","last_became_peered":"2024-04-24T09:04:02.106946+0000","last_unstale":"2024-04-24T09:07:04.342748+0000","last_undegraded":"2024-04-24T09:07:04.342748+0000","last_fullsized":"2024-04-24T09:07:04.342748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:38:56.821285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341274+0000","last_change":"2024-04-24T09:04:02.116829+0000","last_active":"2024-04-24T09:07:04.341274+0000","last_peered":"2024-04-24T09:07:04.341274+0000","last_clean":"2024-04-24T09:07:04.341274+0000","last_became_active":"2024-04-24T09:04:02.116520+0000","last_became_peered":"2024-04-24T09:04:02.116520+0000","last_unstale":"2024-04-24T09:07:04.341274+0000","last_undegraded":"2024-04-24T09:07:04.341274+0000","last_fullsized":"2024-04-24T09:07:04.341274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:26.873826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.876882+0000","last_change":"2024-04-24T09:04:24.700595+0000","last_active":"2024-04-24T09:07:03.876882+0000","last_peered":"2024-04-24T09:07:03.876882+0000","last_clean":"2024-04-24T09:07:03.876882+0000","last_became_active":"2024-04-24T09:04:24.699395+0000","last_became_peered":"2024-04-24T09:04:24.699395+0000","last_unstale":"2024-04-24T09:07:03.876882+0000","last_undegraded":"2024-04-24T09:07:03.876882+0000","last_fullsized":"2024-04-24T09:07:03.876882+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:28.756302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.032684+0000","last_change":"2024-04-24T09:04:02.114087+0000","last_active":"2024-04-24T09:07:04.032684+0000","last_peered":"2024-04-24T09:07:04.032684+0000","last_clean":"2024-04-24T09:07:04.032684+0000","last_became_active":"2024-04-24T09:04:02.112681+0000","last_became_peered":"2024-04-24T09:04:02.112681+0000","last_unstale":"2024-04-24T09:07:04.032684+0000","last_undegraded":"2024-04-24T09:07:04.032684+0000","last_fullsized":"2024-04-24T09:07:04.032684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:50:01.762598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.344340+0000","last_change":"2024-04-24T09:04:02.102322+0000","last_active":"2024-04-24T09:07:04.344340+0000","last_peered":"2024-04-24T09:07:04.344340+0000","last_clean":"2024-04-24T09:07:04.344340+0000","last_became_active":"2024-04-24T09:04:02.101963+0000","last_became_peered":"2024-04-24T09:04:02.101963+0000","last_unstale":"2024-04-24T09:07:04.344340+0000","last_undegraded":"2024-04-24T09:07:04.344340+0000","last_fullsized":"2024-04-24T09:07:04.344340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:16.140829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031553+0000","last_change":"2024-04-24T09:04:02.110570+0000","last_active":"2024-04-24T09:07:04.031553+0000","last_peered":"2024-04-24T09:07:04.031553+0000","last_clean":"2024-04-24T09:07:04.031553+0000","last_became_active":"2024-04-24T09:04:02.110119+0000","last_became_peered":"2024-04-24T09:04:02.110119+0000","last_unstale":"2024-04-24T09:07:04.031553+0000","last_undegraded":"2024-04-24T09:07:04.031553+0000","last_fullsized":"2024-04-24T09:07:04.031553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:34.436430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880582+0000","last_change":"2024-04-24T09:04:02.108292+0000","last_active":"2024-04-24T09:07:03.880582+0000","last_peered":"2024-04-24T09:07:03.880582+0000","last_clean":"2024-04-24T09:07:03.880582+0000","last_became_active":"2024-04-24T09:04:02.107481+0000","last_became_peered":"2024-04-24T09:04:02.107481+0000","last_unstale":"2024-04-24T09:07:03.880582+0000","last_undegraded":"2024-04-24T09:07:03.880582+0000","last_fullsized":"2024-04-24T09:07:03.880582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:49.755293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029964+0000","last_change":"2024-04-24T09:04:02.102228+0000","last_active":"2024-04-24T09:07:04.029964+0000","last_peered":"2024-04-24T09:07:04.029964+0000","last_clean":"2024-04-24T09:07:04.029964+0000","last_became_active":"2024-04-24T09:04:02.101863+0000","last_became_peered":"2024-04-24T09:04:02.101863+0000","last_unstale":"2024-04-24T09:07:04.029964+0000","last_undegraded":"2024-04-24T09:07:04.029964+0000","last_fullsized":"2024-04-24T09:07:04.029964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:02:47.308116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342657+0000","last_change":"2024-04-24T09:04:02.106749+0000","last_active":"2024-04-24T09:07:04.342657+0000","last_peered":"2024-04-24T09:07:04.342657+0000","last_clean":"2024-04-24T09:07:04.342657+0000","last_became_active":"2024-04-24T09:04:02.106282+0000","last_became_peered":"2024-04-24T09:04:02.106282+0000","last_unstale":"2024-04-24T09:07:04.342657+0000","last_undegraded":"2024-04-24T09:07:04.342657+0000","last_fullsized":"2024-04-24T09:07:04.342657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:35:59.765230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.877236+0000","last_change":"2024-04-24T09:04:02.097825+0000","last_active":"2024-04-24T09:07:03.877236+0000","last_peered":"2024-04-24T09:07:03.877236+0000","last_clean":"2024-04-24T09:07:03.877236+0000","last_became_active":"2024-04-24T09:04:02.097488+0000","last_became_peered":"2024-04-24T09:04:02.097488+0000","last_unstale":"2024-04-24T09:07:03.877236+0000","last_undegraded":"2024-04-24T09:07:03.877236+0000","last_fullsized":"2024-04-24T09:07:03.877236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:05:59.962558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031865+0000","last_change":"2024-04-24T09:04:02.113819+0000","last_active":"2024-04-24T09:07:04.031865+0000","last_peered":"2024-04-24T09:07:04.031865+0000","last_clean":"2024-04-24T09:07:04.031865+0000","last_became_active":"2024-04-24T09:04:02.113463+0000","last_became_peered":"2024-04-24T09:04:02.113463+0000","last_unstale":"2024-04-24T09:07:04.031865+0000","last_undegraded":"2024-04-24T09:07:04.031865+0000","last_fullsized":"2024-04-24T09:07:04.031865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:47:05.109583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602602,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1027,"num_read_kb":628,"num_write":300,"num_write_kb":1450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2039808,"data_stored":1807806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351028,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28432,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716720,"statfs":{"total":95995035648,"available":95965921280,"internally_reserved":0,"allocated":1445888,"data_stored":910510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612667,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33200,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93711952,"statfs":{"total":95995035648,"available":95961038848,"internally_reserved":0,"allocated":1888256,"data_stored":1308281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808899,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1466368,"data_stored":914842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070537,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":517450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332176,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32784,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712368,"statfs":{"total":95995035648,"available":95961464832,"internally_reserved":0,"allocated":1646592,"data_stored":1104678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495704,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":512706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":573902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":504286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":574630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182509,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1037716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920869,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":519496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757343,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32940,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712212,"statfs":{"total":95995035648,"available":95961305088,"internally_reserved":0,"allocated":1806336,"data_stored":1241431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":206079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T09:07:08.514 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-24T09:07:08.514 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T09:07:08.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.723+0000 7fe556400640 1 -- 172.21.15.69:0/605382617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe550105450 msgr2=0x7fe5501058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:08.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.723+0000 7fe556400640 1 --2- 172.21.15.69:0/605382617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe550105450 0x7fe5501058d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009900 tx=0x7fe53c02a410 comp rx=0 tx=0).stop 2024-04-24T09:07:08.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.723+0000 7fe556400640 1 -- 172.21.15.69:0/605382617 shutdown_connections 2024-04-24T09:07:08.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.723+0000 7fe556400640 1 --2- 172.21.15.69:0/605382617 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe550105e10 0x7fe55010c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.724+0000 7fe556400640 1 --2- 172.21.15.69:0/605382617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe550105450 0x7fe5501058d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.724+0000 7fe556400640 1 --2- 172.21.15.69:0/605382617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe550104260 0x7fe550104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.724+0000 7fe556400640 1 -- 172.21.15.69:0/605382617 >> 172.21.15.69:0/605382617 conn(0x7fe5500ff7f0 msgr2=0x7fe550101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:08.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.724+0000 7fe556400640 1 -- 172.21.15.69:0/605382617 shutdown_connections 2024-04-24T09:07:08.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.724+0000 7fe556400640 1 -- 172.21.15.69:0/605382617 wait complete. 2024-04-24T09:07:08.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.724+0000 7fe556400640 1 Processor -- start 2024-04-24T09:07:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.725+0000 7fe556400640 1 -- start start 2024-04-24T09:07:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.725+0000 7fe556400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe550104260 0x7fe55019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.725+0000 7fe556400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe550105450 0x7fe55019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.725+0000 7fe556400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe550105e10 0x7fe5501a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.725+0000 7fe556400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe55010fda0 con 0x7fe550105450 2024-04-24T09:07:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.725+0000 7fe556400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe55010fc20 con 0x7fe550105e10 2024-04-24T09:07:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.725+0000 7fe556400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe55010ff20 con 0x7fe550104260 2024-04-24T09:07:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.725+0000 7fe54ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe550105450 0x7fe55019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.725+0000 7fe54ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe550105450 0x7fe55019a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46090/0 (socket says 172.21.15.69:46090) 2024-04-24T09:07:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.725+0000 7fe54ea00640 1 -- 172.21.15.69:0/220635553 learned_addr learned my addr 172.21.15.69:0/220635553 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.725+0000 7fe54f400640 1 --2- 172.21.15.69:0/220635553 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe550104260 0x7fe55019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.726+0000 7fe54fe00640 1 --2- 172.21.15.69:0/220635553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe550105e10 0x7fe5501a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.726+0000 7fe54ea00640 1 -- 172.21.15.69:0/220635553 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe550104260 msgr2=0x7fe55019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:08.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.726+0000 7fe54ea00640 1 --2- 172.21.15.69:0/220635553 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe550104260 0x7fe55019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.726+0000 7fe54ea00640 1 -- 172.21.15.69:0/220635553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe550105e10 msgr2=0x7fe5501a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:08.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.726+0000 7fe54ea00640 1 --2- 172.21.15.69:0/220635553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe550105e10 0x7fe5501a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.726+0000 7fe54ea00640 1 -- 172.21.15.69:0/220635553 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5501a2050 con 0x7fe550105450 2024-04-24T09:07:08.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.726+0000 7fe54f400640 1 --2- 172.21.15.69:0/220635553 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe550104260 0x7fe55019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:08.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.726+0000 7fe54ea00640 1 --2- 172.21.15.69:0/220635553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe550105450 0x7fe55019a970 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe53c004770 tx=0x7fe53c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:08.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.726+0000 7fe547e00640 1 -- 172.21.15.69:0/220635553 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c03f030 con 0x7fe550105450 2024-04-24T09:07:08.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.727+0000 7fe547e00640 1 -- 172.21.15.69:0/220635553 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe53c003cb0 con 0x7fe550105450 2024-04-24T09:07:08.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.727+0000 7fe556400640 1 -- 172.21.15.69:0/220635553 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5501a22e0 con 0x7fe550105450 2024-04-24T09:07:08.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.727+0000 7fe547e00640 1 -- 172.21.15.69:0/220635553 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c035070 con 0x7fe550105450 2024-04-24T09:07:08.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.727+0000 7fe556400640 1 -- 172.21.15.69:0/220635553 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5501a2820 con 0x7fe550105450 2024-04-24T09:07:08.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.728+0000 7fe556400640 1 -- 172.21.15.69:0/220635553 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe55010b580 con 0x7fe550105450 2024-04-24T09:07:08.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.730+0000 7fe547e00640 1 -- 172.21.15.69:0/220635553 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fe53c042030 con 0x7fe550105450 2024-04-24T09:07:08.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.732+0000 7fe547e00640 1 --2- 172.21.15.69:0/220635553 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe5340a3e30 0x7fe5340a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:08.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.732+0000 7fe547e00640 1 -- 172.21.15.69:0/220635553 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe53c0f19d0 con 0x7fe550105450 2024-04-24T09:07:08.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.732+0000 7fe54f400640 1 --2- 172.21.15.69:0/220635553 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe5340a3e30 0x7fe5340a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:08.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.733+0000 7fe54f400640 1 --2- 172.21.15.69:0/220635553 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe5340a3e30 0x7fe5340a62f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe5400058c0 tx=0x7fe540005850 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:08.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.734+0000 7fe547e00640 1 -- 172.21.15.69:0/220635553 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe53c0bdb60 con 0x7fe550105450 2024-04-24T09:07:08.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.925+0000 7fe556400640 1 -- 172.21.15.69:0/220635553 --> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe550062440 con 0x7fe5340a3e30 2024-04-24T09:07:08.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.953+0000 7fe547e00640 1 -- 172.21.15.69:0/220635553 <== mgr.15465 v2:172.21.15.69:6800/2822728880 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339473 (secure 0 0 0) 0x7fe550062440 con 0x7fe5340a3e30 2024-04-24T09:07:08.958 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:07:08.960 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-24T09:07:08.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.958+0000 7fe556400640 1 -- 172.21.15.69:0/220635553 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe5340a3e30 msgr2=0x7fe5340a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:08.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.958+0000 7fe556400640 1 --2- 172.21.15.69:0/220635553 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe5340a3e30 0x7fe5340a62f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe5400058c0 tx=0x7fe540005850 comp rx=0 tx=0).stop 2024-04-24T09:07:08.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.958+0000 7fe556400640 1 -- 172.21.15.69:0/220635553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe550105450 msgr2=0x7fe55019a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:08.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.958+0000 7fe556400640 1 --2- 172.21.15.69:0/220635553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe550105450 0x7fe55019a970 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe53c004770 tx=0x7fe53c0042f0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.959+0000 7fe556400640 1 -- 172.21.15.69:0/220635553 shutdown_connections 2024-04-24T09:07:08.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.959+0000 7fe556400640 1 --2- 172.21.15.69:0/220635553 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe5340a3e30 0x7fe5340a62f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.959+0000 7fe556400640 1 --2- 172.21.15.69:0/220635553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe550105e10 0x7fe5501a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.959+0000 7fe556400640 1 --2- 172.21.15.69:0/220635553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe550105450 0x7fe55019a970 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.959+0000 7fe556400640 1 --2- 172.21.15.69:0/220635553 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe550104260 0x7fe55019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:08.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.959+0000 7fe556400640 1 -- 172.21.15.69:0/220635553 >> 172.21.15.69:0/220635553 conn(0x7fe5500ff7f0 msgr2=0x7fe550108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:08.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.959+0000 7fe556400640 1 -- 172.21.15.69:0/220635553 shutdown_connections 2024-04-24T09:07:08.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:08.959+0000 7fe556400640 1 -- 172.21.15.69:0/220635553 wait complete. 2024-04-24T09:07:08.985 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-24T09:07:07.901521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193204,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1163,"num_read_kb":747,"num_write":483,"num_write_kb":4390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":877,"ondisk_log_size":877,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356972,"kb_used_data":15940,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124584852,"statfs":{"total":1151940427776,"available":1151574888448,"internally_reserved":0,"allocated":16322560,"data_stored":9719928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.983030"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031762+0000","last_change":"2024-04-24T09:04:02.107965+0000","last_active":"2024-04-24T09:07:04.031762+0000","last_peered":"2024-04-24T09:07:04.031762+0000","last_clean":"2024-04-24T09:07:04.031762+0000","last_became_active":"2024-04-24T09:04:02.106827+0000","last_became_peered":"2024-04-24T09:04:02.106827+0000","last_unstale":"2024-04-24T09:07:04.031762+0000","last_undegraded":"2024-04-24T09:07:04.031762+0000","last_fullsized":"2024-04-24T09:07:04.031762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:20:23.930599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341563+0000","last_change":"2024-04-24T09:04:02.105449+0000","last_active":"2024-04-24T09:07:04.341563+0000","last_peered":"2024-04-24T09:07:04.341563+0000","last_clean":"2024-04-24T09:07:04.341563+0000","last_became_active":"2024-04-24T09:04:02.101842+0000","last_became_peered":"2024-04-24T09:04:02.101842+0000","last_unstale":"2024-04-24T09:07:04.341563+0000","last_undegraded":"2024-04-24T09:07:04.341563+0000","last_fullsized":"2024-04-24T09:07:04.341563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:46.911768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341388+0000","last_change":"2024-04-24T09:04:02.118686+0000","last_active":"2024-04-24T09:07:04.341388+0000","last_peered":"2024-04-24T09:07:04.341388+0000","last_clean":"2024-04-24T09:07:04.341388+0000","last_became_active":"2024-04-24T09:04:02.117549+0000","last_became_peered":"2024-04-24T09:04:02.117549+0000","last_unstale":"2024-04-24T09:07:04.341388+0000","last_undegraded":"2024-04-24T09:07:04.341388+0000","last_fullsized":"2024-04-24T09:07:04.341388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:18:16.233434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086531+0000","last_change":"2024-04-24T09:04:02.120336+0000","last_active":"2024-04-24T09:07:04.086531+0000","last_peered":"2024-04-24T09:07:04.086531+0000","last_clean":"2024-04-24T09:07:04.086531+0000","last_became_active":"2024-04-24T09:04:02.116574+0000","last_became_peered":"2024-04-24T09:04:02.116574+0000","last_unstale":"2024-04-24T09:07:04.086531+0000","last_undegraded":"2024-04-24T09:07:04.086531+0000","last_fullsized":"2024-04-24T09:07:04.086531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:10:42.366531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921771+0000","last_change":"2024-04-24T09:04:02.113600+0000","last_active":"2024-04-24T09:07:03.921771+0000","last_peered":"2024-04-24T09:07:03.921771+0000","last_clean":"2024-04-24T09:07:03.921771+0000","last_became_active":"2024-04-24T09:04:02.113143+0000","last_became_peered":"2024-04-24T09:04:02.113143+0000","last_unstale":"2024-04-24T09:07:03.921771+0000","last_undegraded":"2024-04-24T09:07:03.921771+0000","last_fullsized":"2024-04-24T09:07:03.921771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:31:27.666661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.882380+0000","last_change":"2024-04-24T09:04:02.106395+0000","last_active":"2024-04-24T09:07:03.882380+0000","last_peered":"2024-04-24T09:07:03.882380+0000","last_clean":"2024-04-24T09:07:03.882380+0000","last_became_active":"2024-04-24T09:04:02.105942+0000","last_became_peered":"2024-04-24T09:04:02.105942+0000","last_unstale":"2024-04-24T09:07:03.882380+0000","last_undegraded":"2024-04-24T09:07:03.882380+0000","last_fullsized":"2024-04-24T09:07:03.882380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:20:10.655605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341117+0000","last_change":"2024-04-24T09:04:02.100279+0000","last_active":"2024-04-24T09:07:04.341117+0000","last_peered":"2024-04-24T09:07:04.341117+0000","last_clean":"2024-04-24T09:07:04.341117+0000","last_became_active":"2024-04-24T09:04:02.099745+0000","last_became_peered":"2024-04-24T09:04:02.099745+0000","last_unstale":"2024-04-24T09:07:04.341117+0000","last_undegraded":"2024-04-24T09:07:04.341117+0000","last_fullsized":"2024-04-24T09:07:04.341117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:13.150412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029846+0000","last_change":"2024-04-24T09:04:02.103294+0000","last_active":"2024-04-24T09:07:04.029846+0000","last_peered":"2024-04-24T09:07:04.029846+0000","last_clean":"2024-04-24T09:07:04.029846+0000","last_became_active":"2024-04-24T09:04:02.102114+0000","last_became_peered":"2024-04-24T09:04:02.102114+0000","last_unstale":"2024-04-24T09:07:04.029846+0000","last_undegraded":"2024-04-24T09:07:04.029846+0000","last_fullsized":"2024-04-24T09:07:04.029846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:26:26.916097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342942+0000","last_change":"2024-04-24T09:04:02.104936+0000","last_active":"2024-04-24T09:07:04.342942+0000","last_peered":"2024-04-24T09:07:04.342942+0000","last_clean":"2024-04-24T09:07:04.342942+0000","last_became_active":"2024-04-24T09:04:02.104540+0000","last_became_peered":"2024-04-24T09:04:02.104540+0000","last_unstale":"2024-04-24T09:07:04.342942+0000","last_undegraded":"2024-04-24T09:07:04.342942+0000","last_fullsized":"2024-04-24T09:07:04.342942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:36.010735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883910+0000","last_change":"2024-04-24T09:04:02.102050+0000","last_active":"2024-04-24T09:07:03.883910+0000","last_peered":"2024-04-24T09:07:03.883910+0000","last_clean":"2024-04-24T09:07:03.883910+0000","last_became_active":"2024-04-24T09:04:02.101623+0000","last_became_peered":"2024-04-24T09:04:02.101623+0000","last_unstale":"2024-04-24T09:07:03.883910+0000","last_undegraded":"2024-04-24T09:07:03.883910+0000","last_fullsized":"2024-04-24T09:07:03.883910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:13:33.606200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341928+0000","last_change":"2024-04-24T09:04:02.115804+0000","last_active":"2024-04-24T09:07:04.341928+0000","last_peered":"2024-04-24T09:07:04.341928+0000","last_clean":"2024-04-24T09:07:04.341928+0000","last_became_active":"2024-04-24T09:04:02.115093+0000","last_became_peered":"2024-04-24T09:04:02.115093+0000","last_unstale":"2024-04-24T09:07:04.341928+0000","last_undegraded":"2024-04-24T09:07:04.341928+0000","last_fullsized":"2024-04-24T09:07:04.341928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:26:56.764485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086355+0000","last_change":"2024-04-24T09:04:02.116305+0000","last_active":"2024-04-24T09:07:04.086355+0000","last_peered":"2024-04-24T09:07:04.086355+0000","last_clean":"2024-04-24T09:07:04.086355+0000","last_became_active":"2024-04-24T09:04:02.115956+0000","last_became_peered":"2024-04-24T09:04:02.115956+0000","last_unstale":"2024-04-24T09:07:04.086355+0000","last_undegraded":"2024-04-24T09:07:04.086355+0000","last_fullsized":"2024-04-24T09:07:04.086355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:38.618317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.877469+0000","last_change":"2024-04-24T09:04:02.101155+0000","last_active":"2024-04-24T09:07:03.877469+0000","last_peered":"2024-04-24T09:07:03.877469+0000","last_clean":"2024-04-24T09:07:03.877469+0000","last_became_active":"2024-04-24T09:04:02.099719+0000","last_became_peered":"2024-04-24T09:04:02.099719+0000","last_unstale":"2024-04-24T09:07:03.877469+0000","last_undegraded":"2024-04-24T09:07:03.877469+0000","last_fullsized":"2024-04-24T09:07:03.877469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:46:27.910987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.340928+0000","last_change":"2024-04-24T09:04:02.114818+0000","last_active":"2024-04-24T09:07:04.340928+0000","last_peered":"2024-04-24T09:07:04.340928+0000","last_clean":"2024-04-24T09:07:04.340928+0000","last_became_active":"2024-04-24T09:04:02.112947+0000","last_became_peered":"2024-04-24T09:04:02.112947+0000","last_unstale":"2024-04-24T09:07:04.340928+0000","last_undegraded":"2024-04-24T09:07:04.340928+0000","last_fullsized":"2024-04-24T09:07:04.340928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:16.544420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341005+0000","last_change":"2024-04-24T09:04:02.120672+0000","last_active":"2024-04-24T09:07:04.341005+0000","last_peered":"2024-04-24T09:07:04.341005+0000","last_clean":"2024-04-24T09:07:04.341005+0000","last_became_active":"2024-04-24T09:04:02.120378+0000","last_became_peered":"2024-04-24T09:04:02.120378+0000","last_unstale":"2024-04-24T09:07:04.341005+0000","last_undegraded":"2024-04-24T09:07:04.341005+0000","last_fullsized":"2024-04-24T09:07:04.341005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:17.781189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342217+0000","last_change":"2024-04-24T09:04:02.107022+0000","last_active":"2024-04-24T09:07:04.342217+0000","last_peered":"2024-04-24T09:07:04.342217+0000","last_clean":"2024-04-24T09:07:04.342217+0000","last_became_active":"2024-04-24T09:04:02.106645+0000","last_became_peered":"2024-04-24T09:04:02.106645+0000","last_unstale":"2024-04-24T09:07:04.342217+0000","last_undegraded":"2024-04-24T09:07:04.342217+0000","last_fullsized":"2024-04-24T09:07:04.342217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:47.217718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031638+0000","last_change":"2024-04-24T09:04:02.115511+0000","last_active":"2024-04-24T09:07:04.031638+0000","last_peered":"2024-04-24T09:07:04.031638+0000","last_clean":"2024-04-24T09:07:04.031638+0000","last_became_active":"2024-04-24T09:04:02.115197+0000","last_became_peered":"2024-04-24T09:04:02.115197+0000","last_unstale":"2024-04-24T09:07:04.031638+0000","last_undegraded":"2024-04-24T09:07:04.031638+0000","last_fullsized":"2024-04-24T09:07:04.031638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:18.883688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.026662+0000","last_change":"2024-04-24T09:04:02.102108+0000","last_active":"2024-04-24T09:07:04.026662+0000","last_peered":"2024-04-24T09:07:04.026662+0000","last_clean":"2024-04-24T09:07:04.026662+0000","last_became_active":"2024-04-24T09:04:02.101739+0000","last_became_peered":"2024-04-24T09:04:02.101739+0000","last_unstale":"2024-04-24T09:07:04.026662+0000","last_undegraded":"2024-04-24T09:07:04.026662+0000","last_fullsized":"2024-04-24T09:07:04.026662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:57:30.768716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343732+0000","last_change":"2024-04-24T09:04:02.108911+0000","last_active":"2024-04-24T09:07:04.343732+0000","last_peered":"2024-04-24T09:07:04.343732+0000","last_clean":"2024-04-24T09:07:04.343732+0000","last_became_active":"2024-04-24T09:04:02.105678+0000","last_became_peered":"2024-04-24T09:04:02.105678+0000","last_unstale":"2024-04-24T09:07:04.343732+0000","last_undegraded":"2024-04-24T09:07:04.343732+0000","last_fullsized":"2024-04-24T09:07:04.343732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:36:12.216271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343846+0000","last_change":"2024-04-24T09:04:06.136320+0000","last_active":"2024-04-24T09:07:04.343846+0000","last_peered":"2024-04-24T09:07:04.343846+0000","last_clean":"2024-04-24T09:07:04.343846+0000","last_became_active":"2024-04-24T09:04:06.133851+0000","last_became_peered":"2024-04-24T09:04:06.133851+0000","last_unstale":"2024-04-24T09:07:04.343846+0000","last_undegraded":"2024-04-24T09:07:04.343846+0000","last_fullsized":"2024-04-24T09:07:04.343846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:31:39.781234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.026762+0000","last_change":"2024-04-24T09:04:01.543818+0000","last_active":"2024-04-24T09:07:04.026762+0000","last_peered":"2024-04-24T09:07:04.026762+0000","last_clean":"2024-04-24T09:07:04.026762+0000","last_became_active":"2024-04-24T09:04:01.543435+0000","last_became_peered":"2024-04-24T09:04:01.543435+0000","last_unstale":"2024-04-24T09:07:04.026762+0000","last_undegraded":"2024-04-24T09:07:04.026762+0000","last_fullsized":"2024-04-24T09:07:04.026762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:08:30.176211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031206+0000","last_change":"2024-04-24T09:04:02.101646+0000","last_active":"2024-04-24T09:07:04.031206+0000","last_peered":"2024-04-24T09:07:04.031206+0000","last_clean":"2024-04-24T09:07:04.031206+0000","last_became_active":"2024-04-24T09:04:02.101284+0000","last_became_peered":"2024-04-24T09:04:02.101284+0000","last_unstale":"2024-04-24T09:07:04.031206+0000","last_undegraded":"2024-04-24T09:07:04.031206+0000","last_fullsized":"2024-04-24T09:07:04.031206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:36:32.827216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.085981+0000","last_change":"2024-04-24T09:04:06.130476+0000","last_active":"2024-04-24T09:07:04.085981+0000","last_peered":"2024-04-24T09:07:04.085981+0000","last_clean":"2024-04-24T09:07:04.085981+0000","last_became_active":"2024-04-24T09:04:06.129707+0000","last_became_peered":"2024-04-24T09:04:06.129707+0000","last_unstale":"2024-04-24T09:07:04.085981+0000","last_undegraded":"2024-04-24T09:07:04.085981+0000","last_fullsized":"2024-04-24T09:07:04.085981+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:25:36.323850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031243+0000","last_change":"2024-04-24T09:04:01.551708+0000","last_active":"2024-04-24T09:07:04.031243+0000","last_peered":"2024-04-24T09:07:04.031243+0000","last_clean":"2024-04-24T09:07:04.031243+0000","last_became_active":"2024-04-24T09:04:01.551089+0000","last_became_peered":"2024-04-24T09:04:01.551089+0000","last_unstale":"2024-04-24T09:07:04.031243+0000","last_undegraded":"2024-04-24T09:07:04.031243+0000","last_fullsized":"2024-04-24T09:07:04.031243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:18:22.515729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341702+0000","last_change":"2024-04-24T09:04:02.117356+0000","last_active":"2024-04-24T09:07:04.341702+0000","last_peered":"2024-04-24T09:07:04.341702+0000","last_clean":"2024-04-24T09:07:04.341702+0000","last_became_active":"2024-04-24T09:04:02.116972+0000","last_became_peered":"2024-04-24T09:04:02.116972+0000","last_unstale":"2024-04-24T09:07:04.341702+0000","last_undegraded":"2024-04-24T09:07:04.341702+0000","last_fullsized":"2024-04-24T09:07:04.341702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:46:29.349952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086191+0000","last_change":"2024-04-24T09:04:06.141538+0000","last_active":"2024-04-24T09:07:04.086191+0000","last_peered":"2024-04-24T09:07:04.086191+0000","last_clean":"2024-04-24T09:07:04.086191+0000","last_became_active":"2024-04-24T09:04:06.140975+0000","last_became_peered":"2024-04-24T09:04:06.140975+0000","last_unstale":"2024-04-24T09:07:04.086191+0000","last_undegraded":"2024-04-24T09:07:04.086191+0000","last_fullsized":"2024-04-24T09:07:04.086191+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:58:31.688828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.922715+0000","last_change":"2024-04-24T09:04:01.561039+0000","last_active":"2024-04-24T09:07:03.922715+0000","last_peered":"2024-04-24T09:07:03.922715+0000","last_clean":"2024-04-24T09:07:03.922715+0000","last_became_active":"2024-04-24T09:04:01.560594+0000","last_became_peered":"2024-04-24T09:04:01.560594+0000","last_unstale":"2024-04-24T09:07:03.922715+0000","last_undegraded":"2024-04-24T09:07:03.922715+0000","last_fullsized":"2024-04-24T09:07:03.922715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:59:19.707826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342953+0000","last_change":"2024-04-24T09:04:02.106007+0000","last_active":"2024-04-24T09:07:04.342953+0000","last_peered":"2024-04-24T09:07:04.342953+0000","last_clean":"2024-04-24T09:07:04.342953+0000","last_became_active":"2024-04-24T09:04:02.099786+0000","last_became_peered":"2024-04-24T09:04:02.099786+0000","last_unstale":"2024-04-24T09:07:04.342953+0000","last_undegraded":"2024-04-24T09:07:04.342953+0000","last_fullsized":"2024-04-24T09:07:04.342953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:56.288417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.085876+0000","last_change":"2024-04-24T09:04:06.143636+0000","last_active":"2024-04-24T09:07:04.085876+0000","last_peered":"2024-04-24T09:07:04.085876+0000","last_clean":"2024-04-24T09:07:04.085876+0000","last_became_active":"2024-04-24T09:04:06.143105+0000","last_became_peered":"2024-04-24T09:04:06.143105+0000","last_unstale":"2024-04-24T09:07:04.085876+0000","last_undegraded":"2024-04-24T09:07:04.085876+0000","last_fullsized":"2024-04-24T09:07:04.085876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:04:23.211067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.922078+0000","last_change":"2024-04-24T09:04:01.548403+0000","last_active":"2024-04-24T09:07:03.922078+0000","last_peered":"2024-04-24T09:07:03.922078+0000","last_clean":"2024-04-24T09:07:03.922078+0000","last_became_active":"2024-04-24T09:04:01.547447+0000","last_became_peered":"2024-04-24T09:04:01.547447+0000","last_unstale":"2024-04-24T09:07:03.922078+0000","last_undegraded":"2024-04-24T09:07:03.922078+0000","last_fullsized":"2024-04-24T09:07:03.922078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:14:08.160127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883567+0000","last_change":"2024-04-24T09:04:02.121410+0000","last_active":"2024-04-24T09:07:03.883567+0000","last_peered":"2024-04-24T09:07:03.883567+0000","last_clean":"2024-04-24T09:07:03.883567+0000","last_became_active":"2024-04-24T09:04:02.120821+0000","last_became_peered":"2024-04-24T09:04:02.120821+0000","last_unstale":"2024-04-24T09:07:03.883567+0000","last_undegraded":"2024-04-24T09:07:03.883567+0000","last_fullsized":"2024-04-24T09:07:03.883567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:39:05.525171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343281+0000","last_change":"2024-04-24T09:04:06.142349+0000","last_active":"2024-04-24T09:07:04.343281+0000","last_peered":"2024-04-24T09:07:04.343281+0000","last_clean":"2024-04-24T09:07:04.343281+0000","last_became_active":"2024-04-24T09:04:06.141911+0000","last_became_peered":"2024-04-24T09:04:06.141911+0000","last_unstale":"2024-04-24T09:07:04.343281+0000","last_undegraded":"2024-04-24T09:07:04.343281+0000","last_fullsized":"2024-04-24T09:07:04.343281+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:45:30.414686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":196,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.444417+0000","last_change":"2024-04-24T09:04:24.537349+0000","last_active":"2024-04-24T09:07:04.444417+0000","last_peered":"2024-04-24T09:07:04.444417+0000","last_clean":"2024-04-24T09:07:04.444417+0000","last_became_active":"2024-04-24T09:04:24.536738+0000","last_became_peered":"2024-04-24T09:04:24.536738+0000","last_unstale":"2024-04-24T09:07:04.444417+0000","last_undegraded":"2024-04-24T09:07:04.444417+0000","last_fullsized":"2024-04-24T09:07:04.444417+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:33.481380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1897,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":153,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921779+0000","last_change":"2024-04-24T09:04:02.111173+0000","last_active":"2024-04-24T09:07:03.921779+0000","last_peered":"2024-04-24T09:07:03.921779+0000","last_clean":"2024-04-24T09:07:03.921779+0000","last_became_active":"2024-04-24T09:04:02.110681+0000","last_became_peered":"2024-04-24T09:04:02.110681+0000","last_unstale":"2024-04-24T09:07:03.921779+0000","last_undegraded":"2024-04-24T09:07:03.921779+0000","last_fullsized":"2024-04-24T09:07:03.921779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:29.939116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343333+0000","last_change":"2024-04-24T09:04:06.137752+0000","last_active":"2024-04-24T09:07:04.343333+0000","last_peered":"2024-04-24T09:07:04.343333+0000","last_clean":"2024-04-24T09:07:04.343333+0000","last_became_active":"2024-04-24T09:04:06.137492+0000","last_became_peered":"2024-04-24T09:04:06.137492+0000","last_unstale":"2024-04-24T09:07:04.343333+0000","last_undegraded":"2024-04-24T09:07:04.343333+0000","last_fullsized":"2024-04-24T09:07:04.343333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:44:00.653864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.882216+0000","last_change":"2024-04-24T09:04:01.544009+0000","last_active":"2024-04-24T09:07:03.882216+0000","last_peered":"2024-04-24T09:07:03.882216+0000","last_clean":"2024-04-24T09:07:03.882216+0000","last_became_active":"2024-04-24T09:04:01.543651+0000","last_became_peered":"2024-04-24T09:04:01.543651+0000","last_unstale":"2024-04-24T09:07:03.882216+0000","last_undegraded":"2024-04-24T09:07:03.882216+0000","last_fullsized":"2024-04-24T09:07:03.882216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:10:21.969072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.881564+0000","last_change":"2024-04-24T09:04:02.115387+0000","last_active":"2024-04-24T09:07:03.881564+0000","last_peered":"2024-04-24T09:07:03.881564+0000","last_clean":"2024-04-24T09:07:03.881564+0000","last_became_active":"2024-04-24T09:04:02.115086+0000","last_became_peered":"2024-04-24T09:04:02.115086+0000","last_unstale":"2024-04-24T09:07:03.881564+0000","last_undegraded":"2024-04-24T09:07:03.881564+0000","last_fullsized":"2024-04-24T09:07:03.881564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:40:57.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031574+0000","last_change":"2024-04-24T09:04:06.146189+0000","last_active":"2024-04-24T09:07:04.031574+0000","last_peered":"2024-04-24T09:07:04.031574+0000","last_clean":"2024-04-24T09:07:04.031574+0000","last_became_active":"2024-04-24T09:04:06.145556+0000","last_became_peered":"2024-04-24T09:04:06.145556+0000","last_unstale":"2024-04-24T09:07:04.031574+0000","last_undegraded":"2024-04-24T09:07:04.031574+0000","last_fullsized":"2024-04-24T09:07:04.031574+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:41.371948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.874786+0000","last_change":"2024-04-24T09:04:01.542185+0000","last_active":"2024-04-24T09:07:03.874786+0000","last_peered":"2024-04-24T09:07:03.874786+0000","last_clean":"2024-04-24T09:07:03.874786+0000","last_became_active":"2024-04-24T09:04:01.541240+0000","last_became_peered":"2024-04-24T09:04:01.541240+0000","last_unstale":"2024-04-24T09:07:03.874786+0000","last_undegraded":"2024-04-24T09:07:03.874786+0000","last_fullsized":"2024-04-24T09:07:03.874786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:15.103276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029990+0000","last_change":"2024-04-24T09:04:02.104054+0000","last_active":"2024-04-24T09:07:04.029990+0000","last_peered":"2024-04-24T09:07:04.029990+0000","last_clean":"2024-04-24T09:07:04.029990+0000","last_became_active":"2024-04-24T09:04:02.103673+0000","last_became_peered":"2024-04-24T09:04:02.103673+0000","last_unstale":"2024-04-24T09:07:04.029990+0000","last_undegraded":"2024-04-24T09:07:04.029990+0000","last_fullsized":"2024-04-24T09:07:04.029990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:31:58.889989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341462+0000","last_change":"2024-04-24T09:04:06.134034+0000","last_active":"2024-04-24T09:07:04.341462+0000","last_peered":"2024-04-24T09:07:04.341462+0000","last_clean":"2024-04-24T09:07:04.341462+0000","last_became_active":"2024-04-24T09:04:06.133596+0000","last_became_peered":"2024-04-24T09:04:06.133596+0000","last_unstale":"2024-04-24T09:07:04.341462+0000","last_undegraded":"2024-04-24T09:07:04.341462+0000","last_fullsized":"2024-04-24T09:07:04.341462+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:17:09.572974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880523+0000","last_change":"2024-04-24T09:04:01.543312+0000","last_active":"2024-04-24T09:07:03.880523+0000","last_peered":"2024-04-24T09:07:03.880523+0000","last_clean":"2024-04-24T09:07:03.880523+0000","last_became_active":"2024-04-24T09:04:01.542914+0000","last_became_peered":"2024-04-24T09:04:01.542914+0000","last_unstale":"2024-04-24T09:07:03.880523+0000","last_undegraded":"2024-04-24T09:07:03.880523+0000","last_fullsized":"2024-04-24T09:07:03.880523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:50.695108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.025810+0000","last_change":"2024-04-24T09:04:02.114445+0000","last_active":"2024-04-24T09:07:04.025810+0000","last_peered":"2024-04-24T09:07:04.025810+0000","last_clean":"2024-04-24T09:07:04.025810+0000","last_became_active":"2024-04-24T09:04:02.114075+0000","last_became_peered":"2024-04-24T09:04:02.114075+0000","last_unstale":"2024-04-24T09:07:04.025810+0000","last_undegraded":"2024-04-24T09:07:04.025810+0000","last_fullsized":"2024-04-24T09:07:04.025810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:35:06.933677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029729+0000","last_change":"2024-04-24T09:04:06.142368+0000","last_active":"2024-04-24T09:07:04.029729+0000","last_peered":"2024-04-24T09:07:04.029729+0000","last_clean":"2024-04-24T09:07:04.029729+0000","last_became_active":"2024-04-24T09:04:06.141872+0000","last_became_peered":"2024-04-24T09:04:06.141872+0000","last_unstale":"2024-04-24T09:07:04.029729+0000","last_undegraded":"2024-04-24T09:07:04.029729+0000","last_fullsized":"2024-04-24T09:07:04.029729+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:13:43.733662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":184,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883924+0000","last_change":"2024-04-24T09:04:01.556002+0000","last_active":"2024-04-24T09:07:03.883924+0000","last_peered":"2024-04-24T09:07:03.883924+0000","last_clean":"2024-04-24T09:07:03.883924+0000","last_became_active":"2024-04-24T09:04:01.555643+0000","last_became_peered":"2024-04-24T09:04:01.555643+0000","last_unstale":"2024-04-24T09:07:03.883924+0000","last_undegraded":"2024-04-24T09:07:03.883924+0000","last_fullsized":"2024-04-24T09:07:03.883924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:30:31.171184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":150,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342926+0000","last_change":"2024-04-24T09:04:02.104252+0000","last_active":"2024-04-24T09:07:04.342926+0000","last_peered":"2024-04-24T09:07:04.342926+0000","last_clean":"2024-04-24T09:07:04.342926+0000","last_became_active":"2024-04-24T09:04:02.099264+0000","last_became_peered":"2024-04-24T09:04:02.099264+0000","last_unstale":"2024-04-24T09:07:04.342926+0000","last_undegraded":"2024-04-24T09:07:04.342926+0000","last_fullsized":"2024-04-24T09:07:04.342926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:28.469814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.025340+0000","last_change":"2024-04-24T09:04:06.132407+0000","last_active":"2024-04-24T09:07:04.025340+0000","last_peered":"2024-04-24T09:07:04.025340+0000","last_clean":"2024-04-24T09:07:04.025340+0000","last_became_active":"2024-04-24T09:04:06.131997+0000","last_became_peered":"2024-04-24T09:04:06.131997+0000","last_unstale":"2024-04-24T09:07:04.025340+0000","last_undegraded":"2024-04-24T09:07:04.025340+0000","last_fullsized":"2024-04-24T09:07:04.025340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:08:07.369326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341438+0000","last_change":"2024-04-24T09:04:01.561039+0000","last_active":"2024-04-24T09:07:04.341438+0000","last_peered":"2024-04-24T09:07:04.341438+0000","last_clean":"2024-04-24T09:07:04.341438+0000","last_became_active":"2024-04-24T09:04:01.560543+0000","last_became_peered":"2024-04-24T09:04:01.560543+0000","last_unstale":"2024-04-24T09:07:04.341438+0000","last_undegraded":"2024-04-24T09:07:04.341438+0000","last_fullsized":"2024-04-24T09:07:04.341438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:27.706982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341945+0000","last_change":"2024-04-24T09:04:02.110850+0000","last_active":"2024-04-24T09:07:04.341945+0000","last_peered":"2024-04-24T09:07:04.341945+0000","last_clean":"2024-04-24T09:07:04.341945+0000","last_became_active":"2024-04-24T09:04:02.110512+0000","last_became_peered":"2024-04-24T09:04:02.110512+0000","last_unstale":"2024-04-24T09:07:04.341945+0000","last_undegraded":"2024-04-24T09:07:04.341945+0000","last_fullsized":"2024-04-24T09:07:04.341945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:19:16.937202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086513+0000","last_change":"2024-04-24T09:04:06.135200+0000","last_active":"2024-04-24T09:07:04.086513+0000","last_peered":"2024-04-24T09:07:04.086513+0000","last_clean":"2024-04-24T09:07:04.086513+0000","last_became_active":"2024-04-24T09:04:06.134680+0000","last_became_peered":"2024-04-24T09:04:06.134680+0000","last_unstale":"2024-04-24T09:07:04.086513+0000","last_undegraded":"2024-04-24T09:07:04.086513+0000","last_fullsized":"2024-04-24T09:07:04.086513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:48:57.697740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883243+0000","last_change":"2024-04-24T09:04:01.555069+0000","last_active":"2024-04-24T09:07:03.883243+0000","last_peered":"2024-04-24T09:07:03.883243+0000","last_clean":"2024-04-24T09:07:03.883243+0000","last_became_active":"2024-04-24T09:04:01.554737+0000","last_became_peered":"2024-04-24T09:04:01.554737+0000","last_unstale":"2024-04-24T09:07:03.883243+0000","last_undegraded":"2024-04-24T09:07:03.883243+0000","last_fullsized":"2024-04-24T09:07:03.883243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:23:00.272697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342607+0000","last_change":"2024-04-24T09:04:02.112126+0000","last_active":"2024-04-24T09:07:04.342607+0000","last_peered":"2024-04-24T09:07:04.342607+0000","last_clean":"2024-04-24T09:07:04.342607+0000","last_became_active":"2024-04-24T09:04:02.107995+0000","last_became_peered":"2024-04-24T09:04:02.107995+0000","last_unstale":"2024-04-24T09:07:04.342607+0000","last_undegraded":"2024-04-24T09:07:04.342607+0000","last_fullsized":"2024-04-24T09:07:04.342607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:13.189829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.881797+0000","last_change":"2024-04-24T09:04:24.542302+0000","last_active":"2024-04-24T09:07:03.881797+0000","last_peered":"2024-04-24T09:07:03.881797+0000","last_clean":"2024-04-24T09:07:03.881797+0000","last_became_active":"2024-04-24T09:04:24.539848+0000","last_became_peered":"2024-04-24T09:04:24.539848+0000","last_unstale":"2024-04-24T09:07:03.881797+0000","last_undegraded":"2024-04-24T09:07:03.881797+0000","last_fullsized":"2024-04-24T09:07:03.881797+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:01:52.055510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"93'46","reported_seq":389,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.442501+0000","last_change":"2024-04-24T09:04:01.548610+0000","last_active":"2024-04-24T09:07:04.442501+0000","last_peered":"2024-04-24T09:07:04.442501+0000","last_clean":"2024-04-24T09:07:04.442501+0000","last_became_active":"2024-04-24T09:04:01.548145+0000","last_became_peered":"2024-04-24T09:04:01.548145+0000","last_unstale":"2024-04-24T09:07:04.442501+0000","last_undegraded":"2024-04-24T09:07:04.442501+0000","last_fullsized":"2024-04-24T09:07:04.442501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:58:48.425187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71876,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":313,"num_read_kb":227,"num_write":47,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.922226+0000","last_change":"2024-04-24T09:04:02.109965+0000","last_active":"2024-04-24T09:07:03.922226+0000","last_peered":"2024-04-24T09:07:03.922226+0000","last_clean":"2024-04-24T09:07:03.922226+0000","last_became_active":"2024-04-24T09:04:02.108143+0000","last_became_peered":"2024-04-24T09:04:02.108143+0000","last_unstale":"2024-04-24T09:07:03.922226+0000","last_undegraded":"2024-04-24T09:07:03.922226+0000","last_fullsized":"2024-04-24T09:07:03.922226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:01:21.083039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.884532+0000","last_change":"2024-04-24T09:04:06.145369+0000","last_active":"2024-04-24T09:07:03.884532+0000","last_peered":"2024-04-24T09:07:03.884532+0000","last_clean":"2024-04-24T09:07:03.884532+0000","last_became_active":"2024-04-24T09:04:06.144884+0000","last_became_peered":"2024-04-24T09:04:06.144884+0000","last_unstale":"2024-04-24T09:07:03.884532+0000","last_undegraded":"2024-04-24T09:07:03.884532+0000","last_fullsized":"2024-04-24T09:07:03.884532+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:29:22.780846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343099+0000","last_change":"2024-04-24T09:04:01.548195+0000","last_active":"2024-04-24T09:07:04.343099+0000","last_peered":"2024-04-24T09:07:04.343099+0000","last_clean":"2024-04-24T09:07:04.343099+0000","last_became_active":"2024-04-24T09:04:01.544969+0000","last_became_peered":"2024-04-24T09:04:01.544969+0000","last_unstale":"2024-04-24T09:07:04.343099+0000","last_undegraded":"2024-04-24T09:07:04.343099+0000","last_fullsized":"2024-04-24T09:07:04.343099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:32:55.294629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.875735+0000","last_change":"2024-04-24T09:04:02.103785+0000","last_active":"2024-04-24T09:07:03.875735+0000","last_peered":"2024-04-24T09:07:03.875735+0000","last_clean":"2024-04-24T09:07:03.875735+0000","last_became_active":"2024-04-24T09:04:02.103463+0000","last_became_peered":"2024-04-24T09:04:02.103463+0000","last_unstale":"2024-04-24T09:07:03.875735+0000","last_undegraded":"2024-04-24T09:07:03.875735+0000","last_fullsized":"2024-04-24T09:07:03.875735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:56.385382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031518+0000","last_change":"2024-04-24T09:04:06.149955+0000","last_active":"2024-04-24T09:07:04.031518+0000","last_peered":"2024-04-24T09:07:04.031518+0000","last_clean":"2024-04-24T09:07:04.031518+0000","last_became_active":"2024-04-24T09:04:06.149445+0000","last_became_peered":"2024-04-24T09:04:06.149445+0000","last_unstale":"2024-04-24T09:07:04.031518+0000","last_undegraded":"2024-04-24T09:07:04.031518+0000","last_fullsized":"2024-04-24T09:07:04.031518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:30:42.527007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.882015+0000","last_change":"2024-04-24T09:04:01.548740+0000","last_active":"2024-04-24T09:07:03.882015+0000","last_peered":"2024-04-24T09:07:03.882015+0000","last_clean":"2024-04-24T09:07:03.882015+0000","last_became_active":"2024-04-24T09:04:01.546060+0000","last_became_peered":"2024-04-24T09:04:01.546060+0000","last_unstale":"2024-04-24T09:07:03.882015+0000","last_undegraded":"2024-04-24T09:07:03.882015+0000","last_fullsized":"2024-04-24T09:07:03.882015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:09:16.301150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341787+0000","last_change":"2024-04-24T09:04:24.538871+0000","last_active":"2024-04-24T09:07:04.341787+0000","last_peered":"2024-04-24T09:07:04.341787+0000","last_clean":"2024-04-24T09:07:04.341787+0000","last_became_active":"2024-04-24T09:04:24.537039+0000","last_became_peered":"2024-04-24T09:04:24.537039+0000","last_unstale":"2024-04-24T09:07:04.341787+0000","last_undegraded":"2024-04-24T09:07:04.341787+0000","last_fullsized":"2024-04-24T09:07:04.341787+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:18.520407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.026577+0000","last_change":"2024-04-24T09:04:06.136973+0000","last_active":"2024-04-24T09:07:04.026577+0000","last_peered":"2024-04-24T09:07:04.026577+0000","last_clean":"2024-04-24T09:07:04.026577+0000","last_became_active":"2024-04-24T09:04:06.136405+0000","last_became_peered":"2024-04-24T09:04:06.136405+0000","last_unstale":"2024-04-24T09:07:04.026577+0000","last_undegraded":"2024-04-24T09:07:04.026577+0000","last_fullsized":"2024-04-24T09:07:04.026577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:07.653162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":191,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086263+0000","last_change":"2024-04-24T09:04:01.546853+0000","last_active":"2024-04-24T09:07:04.086263+0000","last_peered":"2024-04-24T09:07:04.086263+0000","last_clean":"2024-04-24T09:07:04.086263+0000","last_became_active":"2024-04-24T09:04:01.546441+0000","last_became_peered":"2024-04-24T09:04:01.546441+0000","last_unstale":"2024-04-24T09:07:04.086263+0000","last_undegraded":"2024-04-24T09:07:04.086263+0000","last_fullsized":"2024-04-24T09:07:04.086263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:23:56.333410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341712+0000","last_change":"2024-04-24T09:04:02.101158+0000","last_active":"2024-04-24T09:07:04.341712+0000","last_peered":"2024-04-24T09:07:04.341712+0000","last_clean":"2024-04-24T09:07:04.341712+0000","last_became_active":"2024-04-24T09:04:02.100832+0000","last_became_peered":"2024-04-24T09:04:02.100832+0000","last_unstale":"2024-04-24T09:07:04.341712+0000","last_undegraded":"2024-04-24T09:07:04.341712+0000","last_fullsized":"2024-04-24T09:07:04.341712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:05:34.080520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.878892+0000","last_change":"2024-04-24T09:04:06.149176+0000","last_active":"2024-04-24T09:07:03.878892+0000","last_peered":"2024-04-24T09:07:03.878892+0000","last_clean":"2024-04-24T09:07:03.878892+0000","last_became_active":"2024-04-24T09:04:06.145539+0000","last_became_peered":"2024-04-24T09:04:06.145539+0000","last_unstale":"2024-04-24T09:07:03.878892+0000","last_undegraded":"2024-04-24T09:07:03.878892+0000","last_fullsized":"2024-04-24T09:07:03.878892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:38.719787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921175+0000","last_change":"2024-04-24T09:04:01.555330+0000","last_active":"2024-04-24T09:07:03.921175+0000","last_peered":"2024-04-24T09:07:03.921175+0000","last_clean":"2024-04-24T09:07:03.921175+0000","last_became_active":"2024-04-24T09:04:01.552618+0000","last_became_peered":"2024-04-24T09:04:01.552618+0000","last_unstale":"2024-04-24T09:07:03.921175+0000","last_undegraded":"2024-04-24T09:07:03.921175+0000","last_fullsized":"2024-04-24T09:07:03.921175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:30.956657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880986+0000","last_change":"2024-04-24T09:04:02.116301+0000","last_active":"2024-04-24T09:07:03.880986+0000","last_peered":"2024-04-24T09:07:03.880986+0000","last_clean":"2024-04-24T09:07:03.880986+0000","last_became_active":"2024-04-24T09:04:02.116001+0000","last_became_peered":"2024-04-24T09:04:02.116001+0000","last_unstale":"2024-04-24T09:07:03.880986+0000","last_undegraded":"2024-04-24T09:07:03.880986+0000","last_fullsized":"2024-04-24T09:07:03.880986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:16:18.995067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880626+0000","last_change":"2024-04-24T09:04:06.135814+0000","last_active":"2024-04-24T09:07:03.880626+0000","last_peered":"2024-04-24T09:07:03.880626+0000","last_clean":"2024-04-24T09:07:03.880626+0000","last_became_active":"2024-04-24T09:04:06.135054+0000","last_became_peered":"2024-04-24T09:04:06.135054+0000","last_unstale":"2024-04-24T09:07:03.880626+0000","last_undegraded":"2024-04-24T09:07:03.880626+0000","last_fullsized":"2024-04-24T09:07:03.880626+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:51:31.150589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.922651+0000","last_change":"2024-04-24T09:04:01.553144+0000","last_active":"2024-04-24T09:07:03.922651+0000","last_peered":"2024-04-24T09:07:03.922651+0000","last_clean":"2024-04-24T09:07:03.922651+0000","last_became_active":"2024-04-24T09:04:01.550599+0000","last_became_peered":"2024-04-24T09:04:01.550599+0000","last_unstale":"2024-04-24T09:07:03.922651+0000","last_undegraded":"2024-04-24T09:07:03.922651+0000","last_fullsized":"2024-04-24T09:07:03.922651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:41.950209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031628+0000","last_change":"2024-04-24T09:04:02.117064+0000","last_active":"2024-04-24T09:07:04.031628+0000","last_peered":"2024-04-24T09:07:04.031628+0000","last_clean":"2024-04-24T09:07:04.031628+0000","last_became_active":"2024-04-24T09:04:02.114528+0000","last_became_peered":"2024-04-24T09:04:02.114528+0000","last_unstale":"2024-04-24T09:07:04.031628+0000","last_undegraded":"2024-04-24T09:07:04.031628+0000","last_fullsized":"2024-04-24T09:07:04.031628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:24:53.662571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921692+0000","last_change":"2024-04-24T09:04:24.701058+0000","last_active":"2024-04-24T09:07:03.921692+0000","last_peered":"2024-04-24T09:07:03.921692+0000","last_clean":"2024-04-24T09:07:03.921692+0000","last_became_active":"2024-04-24T09:04:24.699766+0000","last_became_peered":"2024-04-24T09:04:24.699766+0000","last_unstale":"2024-04-24T09:07:03.921692+0000","last_undegraded":"2024-04-24T09:07:03.921692+0000","last_fullsized":"2024-04-24T09:07:03.921692+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:57:48.491073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.026283+0000","last_change":"2024-04-24T09:04:24.701816+0000","last_active":"2024-04-24T09:07:04.026283+0000","last_peered":"2024-04-24T09:07:04.026283+0000","last_clean":"2024-04-24T09:07:04.026283+0000","last_became_active":"2024-04-24T09:04:24.700530+0000","last_became_peered":"2024-04-24T09:04:24.700530+0000","last_unstale":"2024-04-24T09:07:04.026283+0000","last_undegraded":"2024-04-24T09:07:04.026283+0000","last_fullsized":"2024-04-24T09:07:04.026283+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:24.614043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883310+0000","last_change":"2024-04-24T09:04:02.104602+0000","last_active":"2024-04-24T09:07:03.883310+0000","last_peered":"2024-04-24T09:07:03.883310+0000","last_clean":"2024-04-24T09:07:03.883310+0000","last_became_active":"2024-04-24T09:04:02.104188+0000","last_became_peered":"2024-04-24T09:04:02.104188+0000","last_unstale":"2024-04-24T09:07:03.883310+0000","last_undegraded":"2024-04-24T09:07:03.883310+0000","last_fullsized":"2024-04-24T09:07:03.883310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:00:50.286952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.876427+0000","last_change":"2024-04-24T09:04:06.134971+0000","last_active":"2024-04-24T09:07:03.876427+0000","last_peered":"2024-04-24T09:07:03.876427+0000","last_clean":"2024-04-24T09:07:03.876427+0000","last_became_active":"2024-04-24T09:04:06.134551+0000","last_became_peered":"2024-04-24T09:04:06.134551+0000","last_unstale":"2024-04-24T09:07:03.876427+0000","last_undegraded":"2024-04-24T09:07:03.876427+0000","last_fullsized":"2024-04-24T09:07:03.876427+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:40:41.766657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341353+0000","last_change":"2024-04-24T09:04:01.551117+0000","last_active":"2024-04-24T09:07:04.341353+0000","last_peered":"2024-04-24T09:07:04.341353+0000","last_clean":"2024-04-24T09:07:04.341353+0000","last_became_active":"2024-04-24T09:04:01.550646+0000","last_became_peered":"2024-04-24T09:04:01.550646+0000","last_unstale":"2024-04-24T09:07:04.341353+0000","last_undegraded":"2024-04-24T09:07:04.341353+0000","last_fullsized":"2024-04-24T09:07:04.341353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:29:03.821300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.882733+0000","last_change":"2024-04-24T09:04:02.107111+0000","last_active":"2024-04-24T09:07:03.882733+0000","last_peered":"2024-04-24T09:07:03.882733+0000","last_clean":"2024-04-24T09:07:03.882733+0000","last_became_active":"2024-04-24T09:04:02.106185+0000","last_became_peered":"2024-04-24T09:04:02.106185+0000","last_unstale":"2024-04-24T09:07:03.882733+0000","last_undegraded":"2024-04-24T09:07:03.882733+0000","last_fullsized":"2024-04-24T09:07:03.882733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:19:03.111407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.877283+0000","last_change":"2024-04-24T09:04:06.143984+0000","last_active":"2024-04-24T09:07:03.877283+0000","last_peered":"2024-04-24T09:07:03.877283+0000","last_clean":"2024-04-24T09:07:03.877283+0000","last_became_active":"2024-04-24T09:04:06.143503+0000","last_became_peered":"2024-04-24T09:04:06.143503+0000","last_unstale":"2024-04-24T09:07:03.877283+0000","last_undegraded":"2024-04-24T09:07:03.877283+0000","last_fullsized":"2024-04-24T09:07:03.877283+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:35:19.317771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880870+0000","last_change":"2024-04-24T09:04:02.111560+0000","last_active":"2024-04-24T09:07:03.880870+0000","last_peered":"2024-04-24T09:07:03.880870+0000","last_clean":"2024-04-24T09:07:03.880870+0000","last_became_active":"2024-04-24T09:04:02.109885+0000","last_became_peered":"2024-04-24T09:04:02.109885+0000","last_unstale":"2024-04-24T09:07:03.880870+0000","last_undegraded":"2024-04-24T09:07:03.880870+0000","last_fullsized":"2024-04-24T09:07:03.880870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:35.837621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":186,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.874716+0000","last_change":"2024-04-24T09:04:01.542530+0000","last_active":"2024-04-24T09:07:03.874716+0000","last_peered":"2024-04-24T09:07:03.874716+0000","last_clean":"2024-04-24T09:07:03.874716+0000","last_became_active":"2024-04-24T09:04:01.541702+0000","last_became_peered":"2024-04-24T09:04:01.541702+0000","last_unstale":"2024-04-24T09:07:03.874716+0000","last_undegraded":"2024-04-24T09:07:03.874716+0000","last_fullsized":"2024-04-24T09:07:03.874716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:28:29.420020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":150,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341380+0000","last_change":"2024-04-24T09:04:06.143004+0000","last_active":"2024-04-24T09:07:04.341380+0000","last_peered":"2024-04-24T09:07:04.341380+0000","last_clean":"2024-04-24T09:07:04.341380+0000","last_became_active":"2024-04-24T09:04:06.142444+0000","last_became_peered":"2024-04-24T09:04:06.142444+0000","last_unstale":"2024-04-24T09:07:04.341380+0000","last_undegraded":"2024-04-24T09:07:04.341380+0000","last_fullsized":"2024-04-24T09:07:04.341380+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:13.910612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.025614+0000","last_change":"2024-04-24T09:04:24.701330+0000","last_active":"2024-04-24T09:07:04.025614+0000","last_peered":"2024-04-24T09:07:04.025614+0000","last_clean":"2024-04-24T09:07:04.025614+0000","last_became_active":"2024-04-24T09:04:24.699988+0000","last_became_peered":"2024-04-24T09:04:24.699988+0000","last_unstale":"2024-04-24T09:07:04.025614+0000","last_undegraded":"2024-04-24T09:07:04.025614+0000","last_fullsized":"2024-04-24T09:07:04.025614+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:23:24.610671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086340+0000","last_change":"2024-04-24T09:04:01.541206+0000","last_active":"2024-04-24T09:07:04.086340+0000","last_peered":"2024-04-24T09:07:04.086340+0000","last_clean":"2024-04-24T09:07:04.086340+0000","last_became_active":"2024-04-24T09:04:01.539888+0000","last_became_peered":"2024-04-24T09:04:01.539888+0000","last_unstale":"2024-04-24T09:07:04.086340+0000","last_undegraded":"2024-04-24T09:07:04.086340+0000","last_fullsized":"2024-04-24T09:07:04.086340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:01.550209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341017+0000","last_change":"2024-04-24T09:02:22.629840+0000","last_active":"2024-04-24T09:07:04.341017+0000","last_peered":"2024-04-24T09:07:04.341017+0000","last_clean":"2024-04-24T09:07:04.341017+0000","last_became_active":"2024-04-24T09:02:22.628700+0000","last_became_peered":"2024-04-24T09:02:22.628700+0000","last_unstale":"2024-04-24T09:07:04.341017+0000","last_undegraded":"2024-04-24T09:07:04.341017+0000","last_fullsized":"2024-04-24T09:07:04.341017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:57:38.349295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:57:38.349295+0000","last_clean_scrub_stamp":"2024-04-24T08:57:38.349295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:54:11.409492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.883073+0000","last_change":"2024-04-24T09:04:06.149548+0000","last_active":"2024-04-24T09:07:03.883073+0000","last_peered":"2024-04-24T09:07:03.883073+0000","last_clean":"2024-04-24T09:07:03.883073+0000","last_became_active":"2024-04-24T09:04:06.149214+0000","last_became_peered":"2024-04-24T09:04:06.149214+0000","last_unstale":"2024-04-24T09:07:03.883073+0000","last_undegraded":"2024-04-24T09:07:03.883073+0000","last_fullsized":"2024-04-24T09:07:03.883073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:03.801242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.874677+0000","last_change":"2024-04-24T09:04:02.110242+0000","last_active":"2024-04-24T09:07:03.874677+0000","last_peered":"2024-04-24T09:07:03.874677+0000","last_clean":"2024-04-24T09:07:03.874677+0000","last_became_active":"2024-04-24T09:04:02.109879+0000","last_became_peered":"2024-04-24T09:04:02.109879+0000","last_unstale":"2024-04-24T09:07:03.874677+0000","last_undegraded":"2024-04-24T09:07:03.874677+0000","last_fullsized":"2024-04-24T09:07:03.874677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:16.353861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341211+0000","last_change":"2024-04-24T09:04:01.550989+0000","last_active":"2024-04-24T09:07:04.341211+0000","last_peered":"2024-04-24T09:07:04.341211+0000","last_clean":"2024-04-24T09:07:04.341211+0000","last_became_active":"2024-04-24T09:04:01.550664+0000","last_became_peered":"2024-04-24T09:04:01.550664+0000","last_unstale":"2024-04-24T09:07:04.341211+0000","last_undegraded":"2024-04-24T09:07:04.341211+0000","last_fullsized":"2024-04-24T09:07:04.341211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:33:39.521951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.030916+0000","last_change":"2024-04-24T09:04:24.538207+0000","last_active":"2024-04-24T09:07:04.030916+0000","last_peered":"2024-04-24T09:07:04.030916+0000","last_clean":"2024-04-24T09:07:04.030916+0000","last_became_active":"2024-04-24T09:04:24.536894+0000","last_became_peered":"2024-04-24T09:04:24.536894+0000","last_unstale":"2024-04-24T09:07:04.030916+0000","last_undegraded":"2024-04-24T09:07:04.030916+0000","last_fullsized":"2024-04-24T09:07:04.030916+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:32:52.860371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.922335+0000","last_change":"2024-04-24T09:04:01.539415+0000","last_active":"2024-04-24T09:07:03.922335+0000","last_peered":"2024-04-24T09:07:03.922335+0000","last_clean":"2024-04-24T09:07:03.922335+0000","last_became_active":"2024-04-24T09:04:01.539020+0000","last_became_peered":"2024-04-24T09:04:01.539020+0000","last_unstale":"2024-04-24T09:07:03.922335+0000","last_undegraded":"2024-04-24T09:07:03.922335+0000","last_fullsized":"2024-04-24T09:07:03.922335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:12:48.728029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.877295+0000","last_change":"2024-04-24T09:04:02.097560+0000","last_active":"2024-04-24T09:07:03.877295+0000","last_peered":"2024-04-24T09:07:03.877295+0000","last_clean":"2024-04-24T09:07:03.877295+0000","last_became_active":"2024-04-24T09:04:02.095612+0000","last_became_peered":"2024-04-24T09:04:02.095612+0000","last_unstale":"2024-04-24T09:07:03.877295+0000","last_undegraded":"2024-04-24T09:07:03.877295+0000","last_fullsized":"2024-04-24T09:07:03.877295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:55:47.747294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029686+0000","last_change":"2024-04-24T09:04:06.137822+0000","last_active":"2024-04-24T09:07:04.029686+0000","last_peered":"2024-04-24T09:07:04.029686+0000","last_clean":"2024-04-24T09:07:04.029686+0000","last_became_active":"2024-04-24T09:04:06.137357+0000","last_became_peered":"2024-04-24T09:04:06.137357+0000","last_unstale":"2024-04-24T09:07:04.029686+0000","last_undegraded":"2024-04-24T09:07:04.029686+0000","last_fullsized":"2024-04-24T09:07:04.029686+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:55:13.119557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'46","reported_seq":108,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.275253+0000","last_change":"2024-04-24T09:04:24.544395+0000","last_active":"2024-04-24T09:07:04.275253+0000","last_peered":"2024-04-24T09:07:04.275253+0000","last_clean":"2024-04-24T09:07:04.275253+0000","last_became_active":"2024-04-24T09:04:24.535870+0000","last_became_peered":"2024-04-24T09:04:24.535870+0000","last_unstale":"2024-04-24T09:07:04.275253+0000","last_undegraded":"2024-04-24T09:07:04.275253+0000","last_fullsized":"2024-04-24T09:07:04.275253+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:29:21.271387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031426+0000","last_change":"2024-04-24T09:04:02.106107+0000","last_active":"2024-04-24T09:07:04.031426+0000","last_peered":"2024-04-24T09:07:04.031426+0000","last_clean":"2024-04-24T09:07:04.031426+0000","last_became_active":"2024-04-24T09:04:02.105734+0000","last_became_peered":"2024-04-24T09:04:02.105734+0000","last_unstale":"2024-04-24T09:07:04.031426+0000","last_undegraded":"2024-04-24T09:07:04.031426+0000","last_fullsized":"2024-04-24T09:07:04.031426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:15.372520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342631+0000","last_change":"2024-04-24T09:04:06.139040+0000","last_active":"2024-04-24T09:07:04.342631+0000","last_peered":"2024-04-24T09:07:04.342631+0000","last_clean":"2024-04-24T09:07:04.342631+0000","last_became_active":"2024-04-24T09:04:06.138532+0000","last_became_peered":"2024-04-24T09:04:06.138532+0000","last_unstale":"2024-04-24T09:07:04.342631+0000","last_undegraded":"2024-04-24T09:07:04.342631+0000","last_fullsized":"2024-04-24T09:07:04.342631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:28.686052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.087005+0000","last_change":"2024-04-24T09:04:01.546859+0000","last_active":"2024-04-24T09:07:04.087005+0000","last_peered":"2024-04-24T09:07:04.087005+0000","last_clean":"2024-04-24T09:07:04.087005+0000","last_became_active":"2024-04-24T09:04:01.546463+0000","last_became_peered":"2024-04-24T09:04:01.546463+0000","last_unstale":"2024-04-24T09:07:04.087005+0000","last_undegraded":"2024-04-24T09:07:04.087005+0000","last_fullsized":"2024-04-24T09:07:04.087005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:31:12.985687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341595+0000","last_change":"2024-04-24T09:04:02.112487+0000","last_active":"2024-04-24T09:07:04.341595+0000","last_peered":"2024-04-24T09:07:04.341595+0000","last_clean":"2024-04-24T09:07:04.341595+0000","last_became_active":"2024-04-24T09:04:02.112078+0000","last_became_peered":"2024-04-24T09:04:02.112078+0000","last_unstale":"2024-04-24T09:07:04.341595+0000","last_undegraded":"2024-04-24T09:07:04.341595+0000","last_fullsized":"2024-04-24T09:07:04.341595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:03:47.352110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086370+0000","last_change":"2024-04-24T09:04:24.539120+0000","last_active":"2024-04-24T09:07:04.086370+0000","last_peered":"2024-04-24T09:07:04.086370+0000","last_clean":"2024-04-24T09:07:04.086370+0000","last_became_active":"2024-04-24T09:04:24.537048+0000","last_became_peered":"2024-04-24T09:04:24.537048+0000","last_unstale":"2024-04-24T09:07:04.086370+0000","last_undegraded":"2024-04-24T09:07:04.086370+0000","last_fullsized":"2024-04-24T09:07:04.086370+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:29:53.484131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342056+0000","last_change":"2024-04-24T09:04:01.545444+0000","last_active":"2024-04-24T09:07:04.342056+0000","last_peered":"2024-04-24T09:07:04.342056+0000","last_clean":"2024-04-24T09:07:04.342056+0000","last_became_active":"2024-04-24T09:04:01.543816+0000","last_became_peered":"2024-04-24T09:04:01.543816+0000","last_unstale":"2024-04-24T09:07:04.342056+0000","last_undegraded":"2024-04-24T09:07:04.342056+0000","last_fullsized":"2024-04-24T09:07:04.342056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:02:36.221797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921232+0000","last_change":"2024-04-24T09:04:02.109085+0000","last_active":"2024-04-24T09:07:03.921232+0000","last_peered":"2024-04-24T09:07:03.921232+0000","last_clean":"2024-04-24T09:07:03.921232+0000","last_became_active":"2024-04-24T09:04:02.108642+0000","last_became_peered":"2024-04-24T09:04:02.108642+0000","last_unstale":"2024-04-24T09:07:03.921232+0000","last_undegraded":"2024-04-24T09:07:03.921232+0000","last_fullsized":"2024-04-24T09:07:03.921232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:27:51.751369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.026721+0000","last_change":"2024-04-24T09:04:06.144940+0000","last_active":"2024-04-24T09:07:04.026721+0000","last_peered":"2024-04-24T09:07:04.026721+0000","last_clean":"2024-04-24T09:07:04.026721+0000","last_became_active":"2024-04-24T09:04:06.144553+0000","last_became_peered":"2024-04-24T09:04:06.144553+0000","last_unstale":"2024-04-24T09:07:04.026721+0000","last_undegraded":"2024-04-24T09:07:04.026721+0000","last_fullsized":"2024-04-24T09:07:04.026721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:25:17.455644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029831+0000","last_change":"2024-04-24T09:04:01.538939+0000","last_active":"2024-04-24T09:07:04.029831+0000","last_peered":"2024-04-24T09:07:04.029831+0000","last_clean":"2024-04-24T09:07:04.029831+0000","last_became_active":"2024-04-24T09:04:01.538541+0000","last_became_peered":"2024-04-24T09:04:01.538541+0000","last_unstale":"2024-04-24T09:07:04.029831+0000","last_undegraded":"2024-04-24T09:07:04.029831+0000","last_fullsized":"2024-04-24T09:07:04.029831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:08.185901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.030917+0000","last_change":"2024-04-24T09:04:02.103844+0000","last_active":"2024-04-24T09:07:04.030917+0000","last_peered":"2024-04-24T09:07:04.030917+0000","last_clean":"2024-04-24T09:07:04.030917+0000","last_became_active":"2024-04-24T09:04:02.103416+0000","last_became_peered":"2024-04-24T09:04:02.103416+0000","last_unstale":"2024-04-24T09:07:04.030917+0000","last_undegraded":"2024-04-24T09:07:04.030917+0000","last_fullsized":"2024-04-24T09:07:04.030917+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:57:47.236384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086087+0000","last_change":"2024-04-24T09:04:06.134838+0000","last_active":"2024-04-24T09:07:04.086087+0000","last_peered":"2024-04-24T09:07:04.086087+0000","last_clean":"2024-04-24T09:07:04.086087+0000","last_became_active":"2024-04-24T09:04:06.134343+0000","last_became_peered":"2024-04-24T09:04:06.134343+0000","last_unstale":"2024-04-24T09:07:04.086087+0000","last_undegraded":"2024-04-24T09:07:04.086087+0000","last_fullsized":"2024-04-24T09:07:04.086087+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:56:55.219012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343211+0000","last_change":"2024-04-24T09:04:01.557194+0000","last_active":"2024-04-24T09:07:04.343211+0000","last_peered":"2024-04-24T09:07:04.343211+0000","last_clean":"2024-04-24T09:07:04.343211+0000","last_became_active":"2024-04-24T09:04:01.556441+0000","last_became_peered":"2024-04-24T09:04:01.556441+0000","last_unstale":"2024-04-24T09:07:04.343211+0000","last_undegraded":"2024-04-24T09:07:04.343211+0000","last_fullsized":"2024-04-24T09:07:04.343211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:45:50.399096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.030826+0000","last_change":"2024-04-24T09:04:02.108076+0000","last_active":"2024-04-24T09:07:04.030826+0000","last_peered":"2024-04-24T09:07:04.030826+0000","last_clean":"2024-04-24T09:07:04.030826+0000","last_became_active":"2024-04-24T09:04:02.106829+0000","last_became_peered":"2024-04-24T09:04:02.106829+0000","last_unstale":"2024-04-24T09:07:04.030826+0000","last_undegraded":"2024-04-24T09:07:04.030826+0000","last_fullsized":"2024-04-24T09:07:04.030826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:40:23.075828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.086526+0000","last_change":"2024-04-24T09:04:06.142269+0000","last_active":"2024-04-24T09:07:04.086526+0000","last_peered":"2024-04-24T09:07:04.086526+0000","last_clean":"2024-04-24T09:07:04.086526+0000","last_became_active":"2024-04-24T09:04:06.141725+0000","last_became_peered":"2024-04-24T09:04:06.141725+0000","last_unstale":"2024-04-24T09:07:04.086526+0000","last_undegraded":"2024-04-24T09:07:04.086526+0000","last_fullsized":"2024-04-24T09:07:04.086526+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:28:17.934148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.921558+0000","last_change":"2024-04-24T09:04:01.553134+0000","last_active":"2024-04-24T09:07:03.921558+0000","last_peered":"2024-04-24T09:07:03.921558+0000","last_clean":"2024-04-24T09:07:03.921558+0000","last_became_active":"2024-04-24T09:04:01.552099+0000","last_became_peered":"2024-04-24T09:04:01.552099+0000","last_unstale":"2024-04-24T09:07:03.921558+0000","last_undegraded":"2024-04-24T09:07:03.921558+0000","last_fullsized":"2024-04-24T09:07:03.921558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:42:36.231405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341535+0000","last_change":"2024-04-24T09:04:02.116641+0000","last_active":"2024-04-24T09:07:04.341535+0000","last_peered":"2024-04-24T09:07:04.341535+0000","last_clean":"2024-04-24T09:07:04.341535+0000","last_became_active":"2024-04-24T09:04:02.115374+0000","last_became_peered":"2024-04-24T09:04:02.115374+0000","last_unstale":"2024-04-24T09:07:04.341535+0000","last_undegraded":"2024-04-24T09:07:04.341535+0000","last_fullsized":"2024-04-24T09:07:04.341535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:34.450990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342365+0000","last_change":"2024-04-24T09:04:06.149968+0000","last_active":"2024-04-24T09:07:04.342365+0000","last_peered":"2024-04-24T09:07:04.342365+0000","last_clean":"2024-04-24T09:07:04.342365+0000","last_became_active":"2024-04-24T09:04:06.149425+0000","last_became_peered":"2024-04-24T09:04:06.149425+0000","last_unstale":"2024-04-24T09:07:04.342365+0000","last_undegraded":"2024-04-24T09:07:04.342365+0000","last_fullsized":"2024-04-24T09:07:04.342365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:23:53.869004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343669+0000","last_change":"2024-04-24T09:04:01.553822+0000","last_active":"2024-04-24T09:07:04.343669+0000","last_peered":"2024-04-24T09:07:04.343669+0000","last_clean":"2024-04-24T09:07:04.343669+0000","last_became_active":"2024-04-24T09:04:01.551468+0000","last_became_peered":"2024-04-24T09:04:01.551468+0000","last_unstale":"2024-04-24T09:07:04.343669+0000","last_undegraded":"2024-04-24T09:07:04.343669+0000","last_fullsized":"2024-04-24T09:07:04.343669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:47:56.367584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342035+0000","last_change":"2024-04-24T09:04:02.115472+0000","last_active":"2024-04-24T09:07:04.342035+0000","last_peered":"2024-04-24T09:07:04.342035+0000","last_clean":"2024-04-24T09:07:04.342035+0000","last_became_active":"2024-04-24T09:04:02.114666+0000","last_became_peered":"2024-04-24T09:04:02.114666+0000","last_unstale":"2024-04-24T09:07:04.342035+0000","last_undegraded":"2024-04-24T09:07:04.342035+0000","last_fullsized":"2024-04-24T09:07:04.342035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:28:39.145108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341009+0000","last_change":"2024-04-24T09:04:06.138786+0000","last_active":"2024-04-24T09:07:04.341009+0000","last_peered":"2024-04-24T09:07:04.341009+0000","last_clean":"2024-04-24T09:07:04.341009+0000","last_became_active":"2024-04-24T09:04:06.138366+0000","last_became_peered":"2024-04-24T09:04:06.138366+0000","last_unstale":"2024-04-24T09:07:04.341009+0000","last_undegraded":"2024-04-24T09:07:04.341009+0000","last_fullsized":"2024-04-24T09:07:04.341009+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:03:16.729928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"93'39","reported_seq":76,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.265142+0000","last_change":"2024-04-24T09:04:01.547733+0000","last_active":"2024-04-24T09:07:04.265142+0000","last_peered":"2024-04-24T09:07:04.265142+0000","last_clean":"2024-04-24T09:07:04.265142+0000","last_became_active":"2024-04-24T09:04:01.544839+0000","last_became_peered":"2024-04-24T09:04:01.544839+0000","last_unstale":"2024-04-24T09:07:04.265142+0000","last_undegraded":"2024-04-24T09:07:04.265142+0000","last_fullsized":"2024-04-24T09:07:04.265142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:45:03.847092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341471+0000","last_change":"2024-04-24T09:04:02.108383+0000","last_active":"2024-04-24T09:07:04.341471+0000","last_peered":"2024-04-24T09:07:04.341471+0000","last_clean":"2024-04-24T09:07:04.341471+0000","last_became_active":"2024-04-24T09:04:02.107883+0000","last_became_peered":"2024-04-24T09:04:02.107883+0000","last_unstale":"2024-04-24T09:07:04.341471+0000","last_undegraded":"2024-04-24T09:07:04.341471+0000","last_fullsized":"2024-04-24T09:07:04.341471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:24:44.923708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880074+0000","last_change":"2024-04-24T09:04:06.140868+0000","last_active":"2024-04-24T09:07:03.880074+0000","last_peered":"2024-04-24T09:07:03.880074+0000","last_clean":"2024-04-24T09:07:03.880074+0000","last_became_active":"2024-04-24T09:04:06.139174+0000","last_became_peered":"2024-04-24T09:04:06.139174+0000","last_unstale":"2024-04-24T09:07:03.880074+0000","last_undegraded":"2024-04-24T09:07:03.880074+0000","last_fullsized":"2024-04-24T09:07:03.880074+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:05.078412+0000","last_clean_scrub_stamp":"2024-04-24T09:04:05.078412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:17:17.334276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.878570+0000","last_change":"2024-04-24T09:04:01.534653+0000","last_active":"2024-04-24T09:07:03.878570+0000","last_peered":"2024-04-24T09:07:03.878570+0000","last_clean":"2024-04-24T09:07:03.878570+0000","last_became_active":"2024-04-24T09:04:01.534312+0000","last_became_peered":"2024-04-24T09:04:01.534312+0000","last_unstale":"2024-04-24T09:07:03.878570+0000","last_undegraded":"2024-04-24T09:07:03.878570+0000","last_fullsized":"2024-04-24T09:07:03.878570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:00.500630+0000","last_clean_scrub_stamp":"2024-04-24T09:04:00.500630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:53:34.301007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341883+0000","last_change":"2024-04-24T09:04:02.102056+0000","last_active":"2024-04-24T09:07:04.341883+0000","last_peered":"2024-04-24T09:07:04.341883+0000","last_clean":"2024-04-24T09:07:04.341883+0000","last_became_active":"2024-04-24T09:04:02.101641+0000","last_became_peered":"2024-04-24T09:04:02.101641+0000","last_unstale":"2024-04-24T09:07:04.341883+0000","last_undegraded":"2024-04-24T09:07:04.341883+0000","last_fullsized":"2024-04-24T09:07:04.341883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:41.654024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031176+0000","last_change":"2024-04-24T09:04:02.100327+0000","last_active":"2024-04-24T09:07:04.031176+0000","last_peered":"2024-04-24T09:07:04.031176+0000","last_clean":"2024-04-24T09:07:04.031176+0000","last_became_active":"2024-04-24T09:04:02.100028+0000","last_became_peered":"2024-04-24T09:04:02.100028+0000","last_unstale":"2024-04-24T09:07:04.031176+0000","last_undegraded":"2024-04-24T09:07:04.031176+0000","last_fullsized":"2024-04-24T09:07:04.031176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:15:18.728045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.343576+0000","last_change":"2024-04-24T09:04:02.101996+0000","last_active":"2024-04-24T09:07:04.343576+0000","last_peered":"2024-04-24T09:07:04.343576+0000","last_clean":"2024-04-24T09:07:04.343576+0000","last_became_active":"2024-04-24T09:04:02.101581+0000","last_became_peered":"2024-04-24T09:04:02.101581+0000","last_unstale":"2024-04-24T09:07:04.343576+0000","last_undegraded":"2024-04-24T09:07:04.343576+0000","last_fullsized":"2024-04-24T09:07:04.343576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:24:55.487781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342748+0000","last_change":"2024-04-24T09:04:02.107288+0000","last_active":"2024-04-24T09:07:04.342748+0000","last_peered":"2024-04-24T09:07:04.342748+0000","last_clean":"2024-04-24T09:07:04.342748+0000","last_became_active":"2024-04-24T09:04:02.106946+0000","last_became_peered":"2024-04-24T09:04:02.106946+0000","last_unstale":"2024-04-24T09:07:04.342748+0000","last_undegraded":"2024-04-24T09:07:04.342748+0000","last_fullsized":"2024-04-24T09:07:04.342748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:38:56.821285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.341274+0000","last_change":"2024-04-24T09:04:02.116829+0000","last_active":"2024-04-24T09:07:04.341274+0000","last_peered":"2024-04-24T09:07:04.341274+0000","last_clean":"2024-04-24T09:07:04.341274+0000","last_became_active":"2024-04-24T09:04:02.116520+0000","last_became_peered":"2024-04-24T09:04:02.116520+0000","last_unstale":"2024-04-24T09:07:04.341274+0000","last_undegraded":"2024-04-24T09:07:04.341274+0000","last_fullsized":"2024-04-24T09:07:04.341274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:26.873826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.876882+0000","last_change":"2024-04-24T09:04:24.700595+0000","last_active":"2024-04-24T09:07:03.876882+0000","last_peered":"2024-04-24T09:07:03.876882+0000","last_clean":"2024-04-24T09:07:03.876882+0000","last_became_active":"2024-04-24T09:04:24.699395+0000","last_became_peered":"2024-04-24T09:04:24.699395+0000","last_unstale":"2024-04-24T09:07:03.876882+0000","last_undegraded":"2024-04-24T09:07:03.876882+0000","last_fullsized":"2024-04-24T09:07:03.876882+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:43:28.756302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.032684+0000","last_change":"2024-04-24T09:04:02.114087+0000","last_active":"2024-04-24T09:07:04.032684+0000","last_peered":"2024-04-24T09:07:04.032684+0000","last_clean":"2024-04-24T09:07:04.032684+0000","last_became_active":"2024-04-24T09:04:02.112681+0000","last_became_peered":"2024-04-24T09:04:02.112681+0000","last_unstale":"2024-04-24T09:07:04.032684+0000","last_undegraded":"2024-04-24T09:07:04.032684+0000","last_fullsized":"2024-04-24T09:07:04.032684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:50:01.762598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.344340+0000","last_change":"2024-04-24T09:04:02.102322+0000","last_active":"2024-04-24T09:07:04.344340+0000","last_peered":"2024-04-24T09:07:04.344340+0000","last_clean":"2024-04-24T09:07:04.344340+0000","last_became_active":"2024-04-24T09:04:02.101963+0000","last_became_peered":"2024-04-24T09:04:02.101963+0000","last_unstale":"2024-04-24T09:07:04.344340+0000","last_undegraded":"2024-04-24T09:07:04.344340+0000","last_fullsized":"2024-04-24T09:07:04.344340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:16.140829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031553+0000","last_change":"2024-04-24T09:04:02.110570+0000","last_active":"2024-04-24T09:07:04.031553+0000","last_peered":"2024-04-24T09:07:04.031553+0000","last_clean":"2024-04-24T09:07:04.031553+0000","last_became_active":"2024-04-24T09:04:02.110119+0000","last_became_peered":"2024-04-24T09:04:02.110119+0000","last_unstale":"2024-04-24T09:07:04.031553+0000","last_undegraded":"2024-04-24T09:07:04.031553+0000","last_fullsized":"2024-04-24T09:07:04.031553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:34.436430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.880582+0000","last_change":"2024-04-24T09:04:02.108292+0000","last_active":"2024-04-24T09:07:03.880582+0000","last_peered":"2024-04-24T09:07:03.880582+0000","last_clean":"2024-04-24T09:07:03.880582+0000","last_became_active":"2024-04-24T09:04:02.107481+0000","last_became_peered":"2024-04-24T09:04:02.107481+0000","last_unstale":"2024-04-24T09:07:03.880582+0000","last_undegraded":"2024-04-24T09:07:03.880582+0000","last_fullsized":"2024-04-24T09:07:03.880582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:49.755293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.029964+0000","last_change":"2024-04-24T09:04:02.102228+0000","last_active":"2024-04-24T09:07:04.029964+0000","last_peered":"2024-04-24T09:07:04.029964+0000","last_clean":"2024-04-24T09:07:04.029964+0000","last_became_active":"2024-04-24T09:04:02.101863+0000","last_became_peered":"2024-04-24T09:04:02.101863+0000","last_unstale":"2024-04-24T09:07:04.029964+0000","last_undegraded":"2024-04-24T09:07:04.029964+0000","last_fullsized":"2024-04-24T09:07:04.029964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:02:47.308116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.342657+0000","last_change":"2024-04-24T09:04:02.106749+0000","last_active":"2024-04-24T09:07:04.342657+0000","last_peered":"2024-04-24T09:07:04.342657+0000","last_clean":"2024-04-24T09:07:04.342657+0000","last_became_active":"2024-04-24T09:04:02.106282+0000","last_became_peered":"2024-04-24T09:04:02.106282+0000","last_unstale":"2024-04-24T09:07:04.342657+0000","last_undegraded":"2024-04-24T09:07:04.342657+0000","last_fullsized":"2024-04-24T09:07:04.342657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:35:59.765230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:03.877236+0000","last_change":"2024-04-24T09:04:02.097825+0000","last_active":"2024-04-24T09:07:03.877236+0000","last_peered":"2024-04-24T09:07:03.877236+0000","last_clean":"2024-04-24T09:07:03.877236+0000","last_became_active":"2024-04-24T09:04:02.097488+0000","last_became_peered":"2024-04-24T09:04:02.097488+0000","last_unstale":"2024-04-24T09:07:03.877236+0000","last_undegraded":"2024-04-24T09:07:03.877236+0000","last_fullsized":"2024-04-24T09:07:03.877236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:05:59.962558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-24T09:07:04.031865+0000","last_change":"2024-04-24T09:04:02.113819+0000","last_active":"2024-04-24T09:07:04.031865+0000","last_peered":"2024-04-24T09:07:04.031865+0000","last_clean":"2024-04-24T09:07:04.031865+0000","last_became_active":"2024-04-24T09:04:02.113463+0000","last_became_peered":"2024-04-24T09:04:02.113463+0000","last_unstale":"2024-04-24T09:07:04.031865+0000","last_undegraded":"2024-04-24T09:07:04.031865+0000","last_fullsized":"2024-04-24T09:07:04.031865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:04:01.503789+0000","last_clean_scrub_stamp":"2024-04-24T09:04:01.503789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:47:05.109583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602602,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1027,"num_read_kb":628,"num_write":300,"num_write_kb":1450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2039808,"data_stored":1807806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351028,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28432,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716720,"statfs":{"total":95995035648,"available":95965921280,"internally_reserved":0,"allocated":1445888,"data_stored":910510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612667,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33200,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93711952,"statfs":{"total":95995035648,"available":95961038848,"internally_reserved":0,"allocated":1888256,"data_stored":1308281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808899,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1466368,"data_stored":914842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070537,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":517450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332176,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32784,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712368,"statfs":{"total":95995035648,"available":95961464832,"internally_reserved":0,"allocated":1646592,"data_stored":1104678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495704,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":512706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":573902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":504286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":574630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182509,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1037716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920869,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":519496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757343,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32940,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712212,"statfs":{"total":95995035648,"available":95961305088,"internally_reserved":0,"allocated":1806336,"data_stored":1241431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":206079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T09:07:08.990 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:07:09.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.215+0000 7f5418200640 1 -- 172.21.15.69:0/753434136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5410107d30 msgr2=0x7f541010a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:09.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.215+0000 7f5418200640 1 --2- 172.21.15.69:0/753434136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5410107d30 0x7f541010a180 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5408009910 tx=0x7f540802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.215+0000 7f5418200640 1 -- 172.21.15.69:0/753434136 shutdown_connections 2024-04-24T09:07:09.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.215+0000 7f5418200640 1 --2- 172.21.15.69:0/753434136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5410107d30 0x7f541010a180 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.215+0000 7f5418200640 1 --2- 172.21.15.69:0/753434136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5410105400 0x7f54101077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.215+0000 7f5418200640 1 --2- 172.21.15.69:0/753434136 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5410102ad0 0x7f5410104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.215+0000 7f5418200640 1 -- 172.21.15.69:0/753434136 >> 172.21.15.69:0/753434136 conn(0x7f54100fc4e0 msgr2=0x7f54100fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:09.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.215+0000 7f5418200640 1 -- 172.21.15.69:0/753434136 shutdown_connections 2024-04-24T09:07:09.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.215+0000 7f5418200640 1 -- 172.21.15.69:0/753434136 wait complete. 2024-04-24T09:07:09.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.216+0000 7f5418200640 1 Processor -- start 2024-04-24T09:07:09.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.216+0000 7f5418200640 1 -- start start 2024-04-24T09:07:09.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.216+0000 7f5418200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5410102ad0 0x7f5410196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:09.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.216+0000 7f5418200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5410105400 0x7f5410196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:09.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.216+0000 7f5418200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5410107d30 0x7f54101a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:09.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.216+0000 7f5418200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f541010db90 con 0x7f5410102ad0 2024-04-24T09:07:09.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.216+0000 7f5418200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f541010da10 con 0x7f5410105400 2024-04-24T09:07:09.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.216+0000 7f5418200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f541010dd10 con 0x7f5410107d30 2024-04-24T09:07:09.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.217+0000 7f5415800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5410102ad0 0x7f5410196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:09.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.217+0000 7f5415800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5410102ad0 0x7f5410196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46106/0 (socket says 172.21.15.69:46106) 2024-04-24T09:07:09.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.217+0000 7f5415800640 1 -- 172.21.15.69:0/2173556614 learned_addr learned my addr 172.21.15.69:0/2173556614 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:09.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.217+0000 7f5416200640 1 --2- 172.21.15.69:0/2173556614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5410107d30 0x7f54101a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:09.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.217+0000 7f5414e00640 1 --2- 172.21.15.69:0/2173556614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5410105400 0x7f5410196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:09.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.217+0000 7f5415800640 1 -- 172.21.15.69:0/2173556614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5410107d30 msgr2=0x7f54101a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:09.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.217+0000 7f5415800640 1 --2- 172.21.15.69:0/2173556614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5410107d30 0x7f54101a5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.217+0000 7f5415800640 1 -- 172.21.15.69:0/2173556614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5410105400 msgr2=0x7f5410196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:09.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.217+0000 7f5415800640 1 --2- 172.21.15.69:0/2173556614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5410105400 0x7f5410196550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.217+0000 7f5415800640 1 -- 172.21.15.69:0/2173556614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54101a6460 con 0x7f5410102ad0 2024-04-24T09:07:09.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.217+0000 7f5415800640 1 --2- 172.21.15.69:0/2173556614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5410102ad0 0x7f5410196010 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f540000d980 tx=0x7f540000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:09.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.218+0000 7f540e000640 1 -- 172.21.15.69:0/2173556614 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5400017070 con 0x7f5410102ad0 2024-04-24T09:07:09.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.218+0000 7f540e000640 1 -- 172.21.15.69:0/2173556614 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5400009c30 con 0x7f5410102ad0 2024-04-24T09:07:09.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.218+0000 7f5418200640 1 -- 172.21.15.69:0/2173556614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54101a6750 con 0x7f5410102ad0 2024-04-24T09:07:09.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.218+0000 7f540e000640 1 -- 172.21.15.69:0/2173556614 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540000f620 con 0x7f5410102ad0 2024-04-24T09:07:09.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.218+0000 7f5418200640 1 -- 172.21.15.69:0/2173556614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54101a6cf0 con 0x7f5410102ad0 2024-04-24T09:07:09.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.219+0000 7f5418200640 1 -- 172.21.15.69:0/2173556614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5410109820 con 0x7f5410102ad0 2024-04-24T09:07:09.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.221+0000 7f540e000640 1 -- 172.21.15.69:0/2173556614 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f5400010460 con 0x7f5410102ad0 2024-04-24T09:07:09.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.223+0000 7f540e000640 1 --2- 172.21.15.69:0/2173556614 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f53f40a3e30 0x7f53f40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:09.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.223+0000 7f540e000640 1 -- 172.21.15.69:0/2173556614 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f54000d0a10 con 0x7f5410102ad0 2024-04-24T09:07:09.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.223+0000 7f5414e00640 1 --2- 172.21.15.69:0/2173556614 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f53f40a3e30 0x7f53f40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:09.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.224+0000 7f5414e00640 1 --2- 172.21.15.69:0/2173556614 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f53f40a3e30 0x7f53f40a62f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f53fc005c40 tx=0x7f53fc005bd0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:09.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.227+0000 7f540e000640 1 -- 172.21.15.69:0/2173556614 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f540009cba0 con 0x7f5410102ad0 2024-04-24T09:07:09.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:07:09.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: from='client.15606 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:07:09.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: from='client.15612 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:07:09.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-24T09:07:09.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:09 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.430+0000 7f5418200640 1 -- 172.21.15.69:0/2173556614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5410062ca0 con 0x7f5410102ad0 2024-04-24T09:07:09.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.433+0000 7f540e000640 1 -- 172.21.15.69:0/2173556614 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24941 (secure 0 0 0) 0x7f5400009de0 con 0x7f5410102ad0 2024-04-24T09:07:09.433 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:07:09.433 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":93,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:07:03.864053+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/660205566":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/1098677190":"2024-04-25T09:07:03.863940+0000","172.21.15.69:0/3145713060":"2024-04-25T09:07:03.863940+0000","172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/3008138297":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/2901345957":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/1098677190":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:07:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.435+0000 7f5418200640 1 -- 172.21.15.69:0/2173556614 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f53f40a3e30 msgr2=0x7f53f40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.435+0000 7f5418200640 1 --2- 172.21.15.69:0/2173556614 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f53f40a3e30 0x7f53f40a62f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f53fc005c40 tx=0x7f53fc005bd0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.435+0000 7f5418200640 1 -- 172.21.15.69:0/2173556614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5410102ad0 msgr2=0x7f5410196010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.435+0000 7f5418200640 1 --2- 172.21.15.69:0/2173556614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5410102ad0 0x7f5410196010 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f540000d980 tx=0x7f540000de40 comp rx=0 tx=0).stop 2024-04-24T09:07:09.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.435+0000 7f5418200640 1 -- 172.21.15.69:0/2173556614 shutdown_connections 2024-04-24T09:07:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.435+0000 7f5418200640 1 --2- 172.21.15.69:0/2173556614 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f53f40a3e30 0x7f53f40a62f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.435+0000 7f5418200640 1 --2- 172.21.15.69:0/2173556614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5410107d30 0x7f54101a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.435+0000 7f5418200640 1 --2- 172.21.15.69:0/2173556614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5410105400 0x7f5410196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.435+0000 7f5418200640 1 --2- 172.21.15.69:0/2173556614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5410102ad0 0x7f5410196010 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.436+0000 7f5418200640 1 -- 172.21.15.69:0/2173556614 >> 172.21.15.69:0/2173556614 conn(0x7f54100fc4e0 msgr2=0x7f5410102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.436+0000 7f5418200640 1 -- 172.21.15.69:0/2173556614 shutdown_connections 2024-04-24T09:07:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.436+0000 7f5418200640 1 -- 172.21.15.69:0/2173556614 wait complete. 2024-04-24T09:07:09.453 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:57:37.377599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:04:00.299827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:04:00.958498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T09:04:04.105314+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T09:07:09.454 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:07:09.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:07:09.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: from='client.15606 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:07:09.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: from='client.15612 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:07:09.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-24T09:07:09.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:09 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.645+0000 7fab4e000640 1 -- 172.21.15.69:0/3790099161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab48105330 msgr2=0x7fab481057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:09.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.645+0000 7fab4e000640 1 --2- 172.21.15.69:0/3790099161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab48105330 0x7fab481057b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fab34009900 tx=0x7fab3402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.645+0000 7fab4e000640 1 -- 172.21.15.69:0/3790099161 shutdown_connections 2024-04-24T09:07:09.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.645+0000 7fab4e000640 1 --2- 172.21.15.69:0/3790099161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab48105cf0 0x7fab4810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.645+0000 7fab4e000640 1 --2- 172.21.15.69:0/3790099161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab48105330 0x7fab481057b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.645+0000 7fab4e000640 1 --2- 172.21.15.69:0/3790099161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab48104140 0x7fab48104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.645+0000 7fab4e000640 1 -- 172.21.15.69:0/3790099161 >> 172.21.15.69:0/3790099161 conn(0x7fab480ff6b0 msgr2=0x7fab48101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:09.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.645+0000 7fab4e000640 1 -- 172.21.15.69:0/3790099161 shutdown_connections 2024-04-24T09:07:09.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.645+0000 7fab4e000640 1 -- 172.21.15.69:0/3790099161 wait complete. 2024-04-24T09:07:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.646+0000 7fab4e000640 1 Processor -- start 2024-04-24T09:07:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.646+0000 7fab4e000640 1 -- start start 2024-04-24T09:07:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab4e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab48104140 0x7fab4819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab4e000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab48105330 0x7fab4819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab4e000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab48105cf0 0x7fab481a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab4e000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fab4810fc80 con 0x7fab48104140 2024-04-24T09:07:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab4e000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fab4810fb00 con 0x7fab48105330 2024-04-24T09:07:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab4e000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fab4810fe00 con 0x7fab48105cf0 2024-04-24T09:07:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab4cc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab48104140 0x7fab4819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:09.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab4cc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab48104140 0x7fab4819a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46128/0 (socket says 172.21.15.69:46128) 2024-04-24T09:07:09.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab4cc00640 1 -- 172.21.15.69:0/927099078 learned_addr learned my addr 172.21.15.69:0/927099078 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:09.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab4d600640 1 --2- 172.21.15.69:0/927099078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab48105cf0 0x7fab481a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:09.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab47e00640 1 --2- 172.21.15.69:0/927099078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab48105330 0x7fab4819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:09.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab4cc00640 1 -- 172.21.15.69:0/927099078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab48105cf0 msgr2=0x7fab481a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:09.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab4cc00640 1 --2- 172.21.15.69:0/927099078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab48105cf0 0x7fab481a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab4cc00640 1 -- 172.21.15.69:0/927099078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab48105330 msgr2=0x7fab4819a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:09.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab4cc00640 1 --2- 172.21.15.69:0/927099078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab48105330 0x7fab4819a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab4cc00640 1 -- 172.21.15.69:0/927099078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab481a1f40 con 0x7fab48104140 2024-04-24T09:07:09.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.647+0000 7fab47e00640 1 --2- 172.21.15.69:0/927099078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab48105330 0x7fab4819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:09.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.648+0000 7fab4cc00640 1 --2- 172.21.15.69:0/927099078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab48104140 0x7fab4819a320 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00c970 tx=0x7fab3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:09.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.648+0000 7fab45600640 1 -- 172.21.15.69:0/927099078 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c004220 con 0x7fab48104140 2024-04-24T09:07:09.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.648+0000 7fab45600640 1 -- 172.21.15.69:0/927099078 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab3c0043c0 con 0x7fab48104140 2024-04-24T09:07:09.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.648+0000 7fab45600640 1 -- 172.21.15.69:0/927099078 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c00f450 con 0x7fab48104140 2024-04-24T09:07:09.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.648+0000 7fab4e000640 1 -- 172.21.15.69:0/927099078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab481a2230 con 0x7fab48104140 2024-04-24T09:07:09.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.648+0000 7fab4e000640 1 -- 172.21.15.69:0/927099078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab481a2730 con 0x7fab48104140 2024-04-24T09:07:09.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.649+0000 7fab4e000640 1 -- 172.21.15.69:0/927099078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab4810b4e0 con 0x7fab48104140 2024-04-24T09:07:09.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.651+0000 7fab45600640 1 -- 172.21.15.69:0/927099078 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fab3c0079b0 con 0x7fab48104140 2024-04-24T09:07:09.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.652+0000 7fab45600640 1 --2- 172.21.15.69:0/927099078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fab300a3e30 0x7fab300a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:09.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.653+0000 7fab45600640 1 -- 172.21.15.69:0/927099078 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fab3c0d0e70 con 0x7fab48104140 2024-04-24T09:07:09.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.653+0000 7fab47e00640 1 --2- 172.21.15.69:0/927099078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fab300a3e30 0x7fab300a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:09.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.653+0000 7fab47e00640 1 --2- 172.21.15.69:0/927099078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fab300a3e30 0x7fab300a62f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab34006010 tx=0x7fab34000bc0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:09.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.655+0000 7fab45600640 1 -- 172.21.15.69:0/927099078 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab3c014030 con 0x7fab48104140 2024-04-24T09:07:09.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:07:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: from='client.15606 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:07:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: from='client.15612 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:07:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-24T09:07:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:09 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:09.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.847+0000 7fab4e000640 1 -- 172.21.15.69:0/927099078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fab4802d890 con 0x7fab48104140 2024-04-24T09:07:09.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.848+0000 7fab45600640 1 -- 172.21.15.69:0/927099078 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fab3c09d080 con 0x7fab48104140 2024-04-24T09:07:09.848 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-24T09:07:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.850+0000 7fab4e000640 1 -- 172.21.15.69:0/927099078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fab300a3e30 msgr2=0x7fab300a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.850+0000 7fab4e000640 1 --2- 172.21.15.69:0/927099078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fab300a3e30 0x7fab300a62f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab34006010 tx=0x7fab34000bc0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.850+0000 7fab4e000640 1 -- 172.21.15.69:0/927099078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab48104140 msgr2=0x7fab4819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.850+0000 7fab4e000640 1 --2- 172.21.15.69:0/927099078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab48104140 0x7fab4819a320 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00c970 tx=0x7fab3c00ce30 comp rx=0 tx=0).stop 2024-04-24T09:07:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.850+0000 7fab4e000640 1 -- 172.21.15.69:0/927099078 shutdown_connections 2024-04-24T09:07:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.850+0000 7fab4e000640 1 --2- 172.21.15.69:0/927099078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fab300a3e30 0x7fab300a62f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.850+0000 7fab4e000640 1 --2- 172.21.15.69:0/927099078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab48105cf0 0x7fab481a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.850+0000 7fab4e000640 1 --2- 172.21.15.69:0/927099078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab48105330 0x7fab4819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.850+0000 7fab4e000640 1 --2- 172.21.15.69:0/927099078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab48104140 0x7fab4819a320 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.850+0000 7fab4e000640 1 -- 172.21.15.69:0/927099078 >> 172.21.15.69:0/927099078 conn(0x7fab480ff6b0 msgr2=0x7fab481086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.851+0000 7fab4e000640 1 -- 172.21.15.69:0/927099078 shutdown_connections 2024-04-24T09:07:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:09.851+0000 7fab4e000640 1 -- 172.21.15.69:0/927099078 wait complete. 2024-04-24T09:07:09.870 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:07:10.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.061+0000 7f1b5e600640 1 -- 172.21.15.69:0/3675895622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b58102990 msgr2=0x7f1b58104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:10.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.061+0000 7f1b5e600640 1 --2- 172.21.15.69:0/3675895622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b58102990 0x7f1b58104d80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009ca0 tx=0x7f1b4802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:07:10.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.062+0000 7f1b5e600640 1 -- 172.21.15.69:0/3675895622 shutdown_connections 2024-04-24T09:07:10.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.062+0000 7f1b5e600640 1 --2- 172.21.15.69:0/3675895622 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b58107bf0 0x7f1b5810a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:10.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.062+0000 7f1b5e600640 1 --2- 172.21.15.69:0/3675895622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b581052c0 0x7f1b581076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:10.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.062+0000 7f1b5e600640 1 --2- 172.21.15.69:0/3675895622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b58102990 0x7f1b58104d80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:10.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.062+0000 7f1b5e600640 1 -- 172.21.15.69:0/3675895622 >> 172.21.15.69:0/3675895622 conn(0x7f1b580fc3a0 msgr2=0x7f1b580fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:10.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.062+0000 7f1b5e600640 1 -- 172.21.15.69:0/3675895622 shutdown_connections 2024-04-24T09:07:10.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.062+0000 7f1b5e600640 1 -- 172.21.15.69:0/3675895622 wait complete. 2024-04-24T09:07:10.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.063+0000 7f1b5e600640 1 Processor -- start 2024-04-24T09:07:10.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.063+0000 7f1b5e600640 1 -- start start 2024-04-24T09:07:10.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.063+0000 7f1b5e600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b58102990 0x7f1b5819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:10.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.063+0000 7f1b5e600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b581052c0 0x7f1b5819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:10.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.063+0000 7f1b5e600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b58107bf0 0x7f1b581a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:10.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.063+0000 7f1b5e600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5810da80 con 0x7f1b58107bf0 2024-04-24T09:07:10.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.063+0000 7f1b5e600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5810d900 con 0x7f1b581052c0 2024-04-24T09:07:10.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.063+0000 7f1b5e600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5810dc00 con 0x7f1b58102990 2024-04-24T09:07:10.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.063+0000 7f1b5dc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b58107bf0 0x7f1b581a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:10.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.063+0000 7f1b5dc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b58107bf0 0x7f1b581a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46142/0 (socket says 172.21.15.69:46142) 2024-04-24T09:07:10.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.063+0000 7f1b5dc00640 1 -- 172.21.15.69:0/2040379490 learned_addr learned my addr 172.21.15.69:0/2040379490 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:10.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.064+0000 7f1b5d200640 1 --2- 172.21.15.69:0/2040379490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b58102990 0x7f1b5819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:10.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.064+0000 7f1b5dc00640 1 -- 172.21.15.69:0/2040379490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b58102990 msgr2=0x7f1b5819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:10.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.064+0000 7f1b57e00640 1 --2- 172.21.15.69:0/2040379490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b581052c0 0x7f1b5819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:10.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.064+0000 7f1b5dc00640 1 --2- 172.21.15.69:0/2040379490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b58102990 0x7f1b5819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:10.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.064+0000 7f1b5dc00640 1 -- 172.21.15.69:0/2040379490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b581052c0 msgr2=0x7f1b5819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:10.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.064+0000 7f1b5dc00640 1 --2- 172.21.15.69:0/2040379490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b581052c0 0x7f1b5819a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:10.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.064+0000 7f1b5dc00640 1 -- 172.21.15.69:0/2040379490 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b581a1f10 con 0x7f1b58107bf0 2024-04-24T09:07:10.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.064+0000 7f1b5dc00640 1 --2- 172.21.15.69:0/2040379490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b58107bf0 0x7f1b581a18a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000c970 tx=0x7f1b5000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:10.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.064+0000 7f1b55600640 1 -- 172.21.15.69:0/2040379490 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50007bb0 con 0x7f1b58107bf0 2024-04-24T09:07:10.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.064+0000 7f1b5e600640 1 -- 172.21.15.69:0/2040379490 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b581a2200 con 0x7f1b58107bf0 2024-04-24T09:07:10.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.064+0000 7f1b55600640 1 -- 172.21.15.69:0/2040379490 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b50007d50 con 0x7f1b58107bf0 2024-04-24T09:07:10.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.064+0000 7f1b55600640 1 -- 172.21.15.69:0/2040379490 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50018720 con 0x7f1b58107bf0 2024-04-24T09:07:10.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.064+0000 7f1b5e600640 1 -- 172.21.15.69:0/2040379490 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b581a2700 con 0x7f1b58107bf0 2024-04-24T09:07:10.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.065+0000 7f1b5e600640 1 -- 172.21.15.69:0/2040379490 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b58109740 con 0x7f1b58107bf0 2024-04-24T09:07:10.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.067+0000 7f1b55600640 1 -- 172.21.15.69:0/2040379490 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f1b5000fa20 con 0x7f1b58107bf0 2024-04-24T09:07:10.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.069+0000 7f1b55600640 1 --2- 172.21.15.69:0/2040379490 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1b3c0a3db0 0x7f1b3c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:10.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.069+0000 7f1b55600640 1 -- 172.21.15.69:0/2040379490 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f1b500d0e80 con 0x7f1b58107bf0 2024-04-24T09:07:10.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.069+0000 7f1b5d200640 1 --2- 172.21.15.69:0/2040379490 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1b3c0a3db0 0x7f1b3c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:10.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.070+0000 7f1b5d200640 1 --2- 172.21.15.69:0/2040379490 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1b3c0a3db0 0x7f1b3c0a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b48000f80 tx=0x7f1b48031a60 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:10.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.073+0000 7f1b55600640 1 -- 172.21.15.69:0/2040379490 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b50014030 con 0x7f1b58107bf0 2024-04-24T09:07:10.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.281+0000 7f1b5e600640 1 -- 172.21.15.69:0/2040379490 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1b58062c00 con 0x7f1b58107bf0 2024-04-24T09:07:10.627 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:10 smithi081 ceph-mon[30961]: from='client.15618 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:07:10.627 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:10 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi081 to 176.4M 2024-04-24T09:07:10.627 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:10 smithi081 ceph-mon[30961]: Unable to set osd_memory_target on smithi081 to 185061273: error parsing value: Value '185061273' is below minimum 939524096 2024-04-24T09:07:10.627 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:10 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:10.628 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:10 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi069 to 176.4M 2024-04-24T09:07:10.628 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:10 smithi081 ceph-mon[30961]: Unable to set osd_memory_target on smithi069 to 185003929: error parsing value: Value '185003929' is below minimum 939524096 2024-04-24T09:07:10.628 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:10 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2173556614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:07:10.628 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:10 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/927099078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:07:10.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.282+0000 7f1b55600640 1 -- 172.21.15.69:0/2040379490 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f1b5009ca70 con 0x7f1b58107bf0 2024-04-24T09:07:10.628 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:07:10.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.284+0000 7f1b5e600640 1 -- 172.21.15.69:0/2040379490 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1b3c0a3db0 msgr2=0x7f1b3c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:10.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.284+0000 7f1b5e600640 1 --2- 172.21.15.69:0/2040379490 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1b3c0a3db0 0x7f1b3c0a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b48000f80 tx=0x7f1b48031a60 comp rx=0 tx=0).stop 2024-04-24T09:07:10.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.284+0000 7f1b5e600640 1 -- 172.21.15.69:0/2040379490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b58107bf0 msgr2=0x7f1b581a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:10.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.284+0000 7f1b5e600640 1 --2- 172.21.15.69:0/2040379490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b58107bf0 0x7f1b581a18a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000c970 tx=0x7f1b5000ce30 comp rx=0 tx=0).stop 2024-04-24T09:07:10.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.284+0000 7f1b5e600640 1 -- 172.21.15.69:0/2040379490 shutdown_connections 2024-04-24T09:07:10.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.284+0000 7f1b5e600640 1 --2- 172.21.15.69:0/2040379490 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1b3c0a3db0 0x7f1b3c0a6270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:10.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.284+0000 7f1b5e600640 1 --2- 172.21.15.69:0/2040379490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b58107bf0 0x7f1b581a18a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:10.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.284+0000 7f1b5e600640 1 --2- 172.21.15.69:0/2040379490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b581052c0 0x7f1b5819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:10.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.284+0000 7f1b5e600640 1 --2- 172.21.15.69:0/2040379490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b58102990 0x7f1b5819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:10.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.284+0000 7f1b5e600640 1 -- 172.21.15.69:0/2040379490 >> 172.21.15.69:0/2040379490 conn(0x7f1b580fc3a0 msgr2=0x7f1b58102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:10.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.285+0000 7f1b5e600640 1 -- 172.21.15.69:0/2040379490 shutdown_connections 2024-04-24T09:07:10.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.285+0000 7f1b5e600640 1 -- 172.21.15.69:0/2040379490 wait complete. 2024-04-24T09:07:10.631 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:07:10.631 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:10 smithi069 ceph-mon[28170]: from='client.15618 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:07:10.631 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:10 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi081 to 176.4M 2024-04-24T09:07:10.631 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:10 smithi069 ceph-mon[28170]: Unable to set osd_memory_target on smithi081 to 185061273: error parsing value: Value '185061273' is below minimum 939524096 2024-04-24T09:07:10.632 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:10 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:10.632 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:10 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi069 to 176.4M 2024-04-24T09:07:10.632 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:10 smithi069 ceph-mon[28170]: Unable to set osd_memory_target on smithi069 to 185003929: error parsing value: Value '185003929' is below minimum 939524096 2024-04-24T09:07:10.632 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:10 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2173556614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:07:10.632 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:10 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/927099078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:07:10.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:10 smithi130 ceph-mon[31553]: from='client.15618 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:07:10.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:10 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi081 to 176.4M 2024-04-24T09:07:10.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:10 smithi130 ceph-mon[31553]: Unable to set osd_memory_target on smithi081 to 185061273: error parsing value: Value '185061273' is below minimum 939524096 2024-04-24T09:07:10.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:10 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:10.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:10 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi069 to 176.4M 2024-04-24T09:07:10.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:10 smithi130 ceph-mon[31553]: Unable to set osd_memory_target on smithi069 to 185003929: error parsing value: Value '185003929' is below minimum 939524096 2024-04-24T09:07:10.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:10 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2173556614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:07:10.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:10 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/927099078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:07:10.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.815+0000 7f4500a00640 1 -- 172.21.15.69:0/1523816024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44fc105470 msgr2=0x7f44fc1058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:10.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.815+0000 7f4500a00640 1 --2- 172.21.15.69:0/1523816024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44fc105470 0x7f44fc1058f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009900 tx=0x7f44e802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:10.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.815+0000 7f4500a00640 1 -- 172.21.15.69:0/1523816024 shutdown_connections 2024-04-24T09:07:10.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.815+0000 7f4500a00640 1 --2- 172.21.15.69:0/1523816024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44fc105e30 0x7f44fc10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:10.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.815+0000 7f4500a00640 1 --2- 172.21.15.69:0/1523816024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44fc105470 0x7f44fc1058f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:10.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.815+0000 7f4500a00640 1 --2- 172.21.15.69:0/1523816024 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44fc104280 0x7f44fc1046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:10.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.815+0000 7f4500a00640 1 -- 172.21.15.69:0/1523816024 >> 172.21.15.69:0/1523816024 conn(0x7f44fc0ff7f0 msgr2=0x7f44fc101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:10.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.816+0000 7f4500a00640 1 -- 172.21.15.69:0/1523816024 shutdown_connections 2024-04-24T09:07:10.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.816+0000 7f4500a00640 1 -- 172.21.15.69:0/1523816024 wait complete. 2024-04-24T09:07:10.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.816+0000 7f4500a00640 1 Processor -- start 2024-04-24T09:07:10.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.817+0000 7f4500a00640 1 -- start start 2024-04-24T09:07:10.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.817+0000 7f4500a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44fc104280 0x7f44fc19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:10.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.817+0000 7f4500a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44fc105470 0x7f44fc19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:10.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.817+0000 7f4500a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44fc105e30 0x7f44fc1a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:10.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.817+0000 7f4500a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc10fdc0 con 0x7f44fc105470 2024-04-24T09:07:10.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.817+0000 7f4500a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc10fc40 con 0x7f44fc105e30 2024-04-24T09:07:10.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.817+0000 7f4500a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc10ff40 con 0x7f44fc104280 2024-04-24T09:07:10.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.817+0000 7f44f9400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44fc105470 0x7f44fc19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:10.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.817+0000 7f44f9400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44fc105470 0x7f44fc19a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46162/0 (socket says 172.21.15.69:46162) 2024-04-24T09:07:10.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.817+0000 7f44f9400640 1 -- 172.21.15.69:0/2581275291 learned_addr learned my addr 172.21.15.69:0/2581275291 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:10.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.818+0000 7f44f9e00640 1 --2- 172.21.15.69:0/2581275291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44fc104280 0x7f44fc19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:10.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.818+0000 7f44fa800640 1 --2- 172.21.15.69:0/2581275291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44fc105e30 0x7f44fc1a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:10.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.818+0000 7f44f9400640 1 -- 172.21.15.69:0/2581275291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44fc104280 msgr2=0x7f44fc19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:10.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.818+0000 7f44f9400640 1 --2- 172.21.15.69:0/2581275291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44fc104280 0x7f44fc19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:10.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.818+0000 7f44f9400640 1 -- 172.21.15.69:0/2581275291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44fc105e30 msgr2=0x7f44fc1a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:10.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.818+0000 7f44f9400640 1 --2- 172.21.15.69:0/2581275291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44fc105e30 0x7f44fc1a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:10.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.818+0000 7f44f9400640 1 -- 172.21.15.69:0/2581275291 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc1a20b0 con 0x7f44fc105470 2024-04-24T09:07:10.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.818+0000 7f44fa800640 1 --2- 172.21.15.69:0/2581275291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44fc105e30 0x7f44fc1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.818+0000 7f44f9400640 1 --2- 172.21.15.69:0/2581275291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44fc105470 0x7f44fc19a9d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f44e8004770 tx=0x7f44e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.818+0000 7f44f2a00640 1 -- 172.21.15.69:0/2581275291 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e803f030 con 0x7f44fc105470 2024-04-24T09:07:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.819+0000 7f44f2a00640 1 -- 172.21.15.69:0/2581275291 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44e8003cb0 con 0x7f44fc105470 2024-04-24T09:07:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.819+0000 7f44f2a00640 1 -- 172.21.15.69:0/2581275291 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8035070 con 0x7f44fc105470 2024-04-24T09:07:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.819+0000 7f4500a00640 1 -- 172.21.15.69:0/2581275291 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44fc1a2340 con 0x7f44fc105470 2024-04-24T09:07:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.819+0000 7f4500a00640 1 -- 172.21.15.69:0/2581275291 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44fc1a2830 con 0x7f44fc105470 2024-04-24T09:07:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.820+0000 7f4500a00640 1 -- 172.21.15.69:0/2581275291 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44fc10b5a0 con 0x7f44fc105470 2024-04-24T09:07:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.822+0000 7f44f2a00640 1 -- 172.21.15.69:0/2581275291 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f44e8042030 con 0x7f44fc105470 2024-04-24T09:07:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.823+0000 7f44f2a00640 1 --2- 172.21.15.69:0/2581275291 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f44e00a3db0 0x7f44e00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.824+0000 7f44f2a00640 1 -- 172.21.15.69:0/2581275291 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f44e80f19c0 con 0x7f44fc105470 2024-04-24T09:07:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.824+0000 7f44f9e00640 1 --2- 172.21.15.69:0/2581275291 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f44e00a3db0 0x7f44e00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.825+0000 7f44f9e00640 1 --2- 172.21.15.69:0/2581275291 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f44e00a3db0 0x7f44e00a6270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f44ec005fd0 tx=0x7f44ec005850 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:10.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:10.827+0000 7f44f2a00640 1 -- 172.21.15.69:0/2581275291 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44e80bdbb0 con 0x7f44fc105470 2024-04-24T09:07:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.060+0000 7f4500a00640 1 -- 172.21.15.69:0/2581275291 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f44fc02d7b0 con 0x7f44fc105470 2024-04-24T09:07:11.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.060+0000 7f44f2a00640 1 -- 172.21.15.69:0/2581275291 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f44e80bd4b0 con 0x7f44fc105470 2024-04-24T09:07:11.060 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-24T09:07:11.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.063+0000 7f4500a00640 1 -- 172.21.15.69:0/2581275291 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f44e00a3db0 msgr2=0x7f44e00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:11.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.063+0000 7f4500a00640 1 --2- 172.21.15.69:0/2581275291 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f44e00a3db0 0x7f44e00a6270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f44ec005fd0 tx=0x7f44ec005850 comp rx=0 tx=0).stop 2024-04-24T09:07:11.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.063+0000 7f4500a00640 1 -- 172.21.15.69:0/2581275291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44fc105470 msgr2=0x7f44fc19a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:11.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.063+0000 7f4500a00640 1 --2- 172.21.15.69:0/2581275291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44fc105470 0x7f44fc19a9d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f44e8004770 tx=0x7f44e80042f0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.063+0000 7f4500a00640 1 -- 172.21.15.69:0/2581275291 shutdown_connections 2024-04-24T09:07:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.063+0000 7f4500a00640 1 --2- 172.21.15.69:0/2581275291 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f44e00a3db0 0x7f44e00a6270 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.063+0000 7f4500a00640 1 --2- 172.21.15.69:0/2581275291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44fc105e30 0x7f44fc1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.063+0000 7f4500a00640 1 --2- 172.21.15.69:0/2581275291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44fc105470 0x7f44fc19a9d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.063+0000 7f4500a00640 1 --2- 172.21.15.69:0/2581275291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44fc104280 0x7f44fc19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.063+0000 7f4500a00640 1 -- 172.21.15.69:0/2581275291 >> 172.21.15.69:0/2581275291 conn(0x7f44fc0ff7f0 msgr2=0x7f44fc108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.063+0000 7f4500a00640 1 -- 172.21.15.69:0/2581275291 shutdown_connections 2024-04-24T09:07:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.063+0000 7f4500a00640 1 -- 172.21.15.69:0/2581275291 wait complete. 2024-04-24T09:07:11.082 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T09:07:11.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.265+0000 7f4b72c00640 1 -- 172.21.15.69:0/3556234268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b6c105470 msgr2=0x7f4b6c1058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:11.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.265+0000 7f4b72c00640 1 --2- 172.21.15.69:0/3556234268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b6c105470 0x7f4b6c1058f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009900 tx=0x7f4b5402a410 comp rx=0 tx=0).stop 2024-04-24T09:07:11.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.265+0000 7f4b72c00640 1 -- 172.21.15.69:0/3556234268 shutdown_connections 2024-04-24T09:07:11.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.265+0000 7f4b72c00640 1 --2- 172.21.15.69:0/3556234268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b6c105e30 0x7f4b6c10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.265+0000 7f4b72c00640 1 --2- 172.21.15.69:0/3556234268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b6c105470 0x7f4b6c1058f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.265+0000 7f4b72c00640 1 --2- 172.21.15.69:0/3556234268 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b6c104280 0x7f4b6c1046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.265+0000 7f4b72c00640 1 -- 172.21.15.69:0/3556234268 >> 172.21.15.69:0/3556234268 conn(0x7f4b6c0ff7f0 msgr2=0x7f4b6c101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:11.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.265+0000 7f4b72c00640 1 -- 172.21.15.69:0/3556234268 shutdown_connections 2024-04-24T09:07:11.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.265+0000 7f4b72c00640 1 -- 172.21.15.69:0/3556234268 wait complete. 2024-04-24T09:07:11.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.266+0000 7f4b72c00640 1 Processor -- start 2024-04-24T09:07:11.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.266+0000 7f4b72c00640 1 -- start start 2024-04-24T09:07:11.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.266+0000 7f4b72c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b6c104280 0x7f4b6c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:11.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.266+0000 7f4b72c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b6c105470 0x7f4b6c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:11.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.266+0000 7f4b72c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b6c105e30 0x7f4b6c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:11.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.266+0000 7f4b72c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c10fdc0 con 0x7f4b6c105e30 2024-04-24T09:07:11.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.266+0000 7f4b72c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c10fc40 con 0x7f4b6c105470 2024-04-24T09:07:11.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.266+0000 7f4b72c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c10ff40 con 0x7f4b6c104280 2024-04-24T09:07:11.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.267+0000 7f4b70c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b6c105e30 0x7f4b6c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:11.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.267+0000 7f4b6b400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b6c105470 0x7f4b6c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:11.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.267+0000 7f4b6be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b6c104280 0x7f4b6c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:11.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.267+0000 7f4b70c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b6c105e30 0x7f4b6c1a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46182/0 (socket says 172.21.15.69:46182) 2024-04-24T09:07:11.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.267+0000 7f4b70c00640 1 -- 172.21.15.69:0/3829979086 learned_addr learned my addr 172.21.15.69:0/3829979086 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:11.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.267+0000 7f4b6b400640 1 -- 172.21.15.69:0/3829979086 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b6c104280 msgr2=0x7f4b6c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:11.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.267+0000 7f4b6b400640 1 --2- 172.21.15.69:0/3829979086 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b6c104280 0x7f4b6c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.267+0000 7f4b6b400640 1 -- 172.21.15.69:0/3829979086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b6c105e30 msgr2=0x7f4b6c1a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:11.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.267+0000 7f4b6b400640 1 --2- 172.21.15.69:0/3829979086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b6c105e30 0x7f4b6c1a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.267+0000 7f4b6b400640 1 -- 172.21.15.69:0/3829979086 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c1a20c0 con 0x7f4b6c105470 2024-04-24T09:07:11.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.267+0000 7f4b70c00640 1 --2- 172.21.15.69:0/3829979086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b6c105e30 0x7f4b6c1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:07:11.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.268+0000 7f4b6b400640 1 --2- 172.21.15.69:0/3829979086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b6c105470 0x7f4b6c19a9e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4b540314e0 tx=0x7f4b54004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:11.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.268+0000 7f4b68c00640 1 -- 172.21.15.69:0/3829979086 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5403f030 con 0x7f4b6c105470 2024-04-24T09:07:11.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.268+0000 7f4b72c00640 1 -- 172.21.15.69:0/3829979086 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6c1a2350 con 0x7f4b6c105470 2024-04-24T09:07:11.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.268+0000 7f4b68c00640 1 -- 172.21.15.69:0/3829979086 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b54003aa0 con 0x7f4b6c105470 2024-04-24T09:07:11.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.268+0000 7f4b68c00640 1 -- 172.21.15.69:0/3829979086 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54035070 con 0x7f4b6c105470 2024-04-24T09:07:11.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.268+0000 7f4b72c00640 1 -- 172.21.15.69:0/3829979086 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b6c1a2860 con 0x7f4b6c105470 2024-04-24T09:07:11.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.269+0000 7f4b72c00640 1 -- 172.21.15.69:0/3829979086 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b6c10b610 con 0x7f4b6c105470 2024-04-24T09:07:11.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.272+0000 7f4b68c00640 1 -- 172.21.15.69:0/3829979086 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f4b54042030 con 0x7f4b6c105470 2024-04-24T09:07:11.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.273+0000 7f4b68c00640 1 --2- 172.21.15.69:0/3829979086 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4b500a3db0 0x7f4b500a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:11.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.273+0000 7f4b68c00640 1 -- 172.21.15.69:0/3829979086 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4b540f1900 con 0x7f4b6c105470 2024-04-24T09:07:11.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.273+0000 7f4b6be00640 1 --2- 172.21.15.69:0/3829979086 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4b500a3db0 0x7f4b500a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:11.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.274+0000 7f4b6be00640 1 --2- 172.21.15.69:0/3829979086 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4b500a3db0 0x7f4b500a6270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c1052d0 tx=0x7f4b5c009500 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:11.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.277+0000 7f4b68c00640 1 -- 172.21.15.69:0/3829979086 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b540bd440 con 0x7f4b6c105470 2024-04-24T09:07:11.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:11 smithi069 ceph-mon[28170]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-24T09:07:11.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:11 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2040379490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:07:11.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:11 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:11.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:11 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:11.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:11 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:11.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:11 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:11.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:11 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:11.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:11 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:11.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:11 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi130 to 176.4M 2024-04-24T09:07:11.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:11 smithi069 ceph-mon[28170]: Unable to set osd_memory_target on smithi130 to 185046937: error parsing value: Value '185046937' is below minimum 939524096 2024-04-24T09:07:11.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:11 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:07:11.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:11 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:07:11.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:11 smithi069 ceph-mon[28170]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-24T09:07:11.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:11 smithi069 ceph-mon[28170]: Updating smithi081:/etc/ceph/ceph.conf 2024-04-24T09:07:11.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:11 smithi069 ceph-mon[28170]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-24T09:07:11.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:11 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2581275291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:07:11.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.485+0000 7f4b72c00640 1 -- 172.21.15.69:0/3829979086 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4b6c02d620 con 0x7f4b6c105470 2024-04-24T09:07:11.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.486+0000 7f4b68c00640 1 -- 172.21.15.69:0/3829979086 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f4b540c1740 con 0x7f4b6c105470 2024-04-24T09:07:11.486 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:07:11.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.489+0000 7f4b72c00640 1 -- 172.21.15.69:0/3829979086 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4b500a3db0 msgr2=0x7f4b500a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:11.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.489+0000 7f4b72c00640 1 --2- 172.21.15.69:0/3829979086 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4b500a3db0 0x7f4b500a6270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c1052d0 tx=0x7f4b5c009500 comp rx=0 tx=0).stop 2024-04-24T09:07:11.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.489+0000 7f4b72c00640 1 -- 172.21.15.69:0/3829979086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b6c105470 msgr2=0x7f4b6c19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:11.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.489+0000 7f4b72c00640 1 --2- 172.21.15.69:0/3829979086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b6c105470 0x7f4b6c19a9e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4b540314e0 tx=0x7f4b54004270 comp rx=0 tx=0).stop 2024-04-24T09:07:11.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.489+0000 7f4b72c00640 1 -- 172.21.15.69:0/3829979086 shutdown_connections 2024-04-24T09:07:11.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.489+0000 7f4b72c00640 1 --2- 172.21.15.69:0/3829979086 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4b500a3db0 0x7f4b500a6270 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.489+0000 7f4b72c00640 1 --2- 172.21.15.69:0/3829979086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b6c105e30 0x7f4b6c1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.489+0000 7f4b72c00640 1 --2- 172.21.15.69:0/3829979086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b6c105470 0x7f4b6c19a9e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.489+0000 7f4b72c00640 1 --2- 172.21.15.69:0/3829979086 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b6c104280 0x7f4b6c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.489+0000 7f4b72c00640 1 -- 172.21.15.69:0/3829979086 >> 172.21.15.69:0/3829979086 conn(0x7f4b6c0ff7f0 msgr2=0x7f4b6c1087e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:11.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.489+0000 7f4b72c00640 1 -- 172.21.15.69:0/3829979086 shutdown_connections 2024-04-24T09:07:11.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.489+0000 7f4b72c00640 1 -- 172.21.15.69:0/3829979086 wait complete. 2024-04-24T09:07:11.507 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:07:11.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:11 smithi081 ceph-mon[30961]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-24T09:07:11.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:11 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2040379490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:07:11.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:11 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:11.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:11 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:11.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:11 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:11.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:11 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:11.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:11 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:11.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:11 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:11.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:11 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi130 to 176.4M 2024-04-24T09:07:11.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:11 smithi081 ceph-mon[30961]: Unable to set osd_memory_target on smithi130 to 185046937: error parsing value: Value '185046937' is below minimum 939524096 2024-04-24T09:07:11.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:11 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:07:11.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:11 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:07:11.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:11 smithi081 ceph-mon[30961]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-24T09:07:11.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:11 smithi081 ceph-mon[30961]: Updating smithi081:/etc/ceph/ceph.conf 2024-04-24T09:07:11.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:11 smithi081 ceph-mon[30961]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-24T09:07:11.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:11 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2581275291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:07:11.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.683+0000 7faba2e00640 1 -- 172.21.15.69:0/627882984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9c105400 msgr2=0x7fab9c1077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:11.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.683+0000 7faba2e00640 1 --2- 172.21.15.69:0/627882984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9c105400 0x7fab9c1077f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fab84009900 tx=0x7fab8402a410 comp rx=0 tx=0).stop 2024-04-24T09:07:11.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.683+0000 7faba2e00640 1 -- 172.21.15.69:0/627882984 shutdown_connections 2024-04-24T09:07:11.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.683+0000 7faba2e00640 1 --2- 172.21.15.69:0/627882984 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab9c107d30 0x7fab9c10a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.683+0000 7faba2e00640 1 --2- 172.21.15.69:0/627882984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9c105400 0x7fab9c1077f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.683+0000 7faba2e00640 1 --2- 172.21.15.69:0/627882984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab9c102ad0 0x7fab9c104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.683+0000 7faba2e00640 1 -- 172.21.15.69:0/627882984 >> 172.21.15.69:0/627882984 conn(0x7fab9c0fc4e0 msgr2=0x7fab9c0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:11.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.683+0000 7faba2e00640 1 -- 172.21.15.69:0/627882984 shutdown_connections 2024-04-24T09:07:11.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.683+0000 7faba2e00640 1 -- 172.21.15.69:0/627882984 wait complete. 2024-04-24T09:07:11.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.684+0000 7faba2e00640 1 Processor -- start 2024-04-24T09:07:11.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.684+0000 7faba2e00640 1 -- start start 2024-04-24T09:07:11.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.685+0000 7faba2e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab9c102ad0 0x7fab9c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:11.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.685+0000 7faba2e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9c105400 0x7fab9c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:11.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.685+0000 7faba2e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab9c107d30 0x7fab9c1a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:11.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.685+0000 7faba2e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c10dbe0 con 0x7fab9c105400 2024-04-24T09:07:11.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.685+0000 7faba2e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c10da60 con 0x7fab9c102ad0 2024-04-24T09:07:11.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.685+0000 7faba2e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c10dd60 con 0x7fab9c107d30 2024-04-24T09:07:11.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.685+0000 7fab93e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9c105400 0x7fab9c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:11.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.685+0000 7fab93e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9c105400 0x7fab9c19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46200/0 (socket says 172.21.15.69:46200) 2024-04-24T09:07:11.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.685+0000 7fab93e00640 1 -- 172.21.15.69:0/4112191203 learned_addr learned my addr 172.21.15.69:0/4112191203 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:11.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.685+0000 7fab9be00640 1 --2- 172.21.15.69:0/4112191203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab9c102ad0 0x7fab9c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:11.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.685+0000 7faba0e00640 1 --2- 172.21.15.69:0/4112191203 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab9c107d30 0x7fab9c1a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:11.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.685+0000 7fab93e00640 1 -- 172.21.15.69:0/4112191203 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab9c107d30 msgr2=0x7fab9c1a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:11.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.685+0000 7fab93e00640 1 --2- 172.21.15.69:0/4112191203 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab9c107d30 0x7fab9c1a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.685+0000 7fab93e00640 1 -- 172.21.15.69:0/4112191203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab9c102ad0 msgr2=0x7fab9c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:11.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.685+0000 7fab93e00640 1 --2- 172.21.15.69:0/4112191203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab9c102ad0 0x7fab9c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.685+0000 7fab93e00640 1 -- 172.21.15.69:0/4112191203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9c1a2100 con 0x7fab9c105400 2024-04-24T09:07:11.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.686+0000 7fab93e00640 1 --2- 172.21.15.69:0/4112191203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9c105400 0x7fab9c19aa20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fab840314e0 tx=0x7fab84003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:11.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.686+0000 7fab99600640 1 -- 172.21.15.69:0/4112191203 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8403f070 con 0x7fab9c105400 2024-04-24T09:07:11.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.686+0000 7fab99600640 1 -- 172.21.15.69:0/4112191203 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab84031e40 con 0x7fab9c105400 2024-04-24T09:07:11.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.686+0000 7faba2e00640 1 -- 172.21.15.69:0/4112191203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9c1a2390 con 0x7fab9c105400 2024-04-24T09:07:11.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.686+0000 7fab99600640 1 -- 172.21.15.69:0/4112191203 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84035030 con 0x7fab9c105400 2024-04-24T09:07:11.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.686+0000 7faba2e00640 1 -- 172.21.15.69:0/4112191203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab9c1a2880 con 0x7fab9c105400 2024-04-24T09:07:11.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.687+0000 7faba2e00640 1 -- 172.21.15.69:0/4112191203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab9c109880 con 0x7fab9c105400 2024-04-24T09:07:11.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.689+0000 7fab99600640 1 -- 172.21.15.69:0/4112191203 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fab84042070 con 0x7fab9c105400 2024-04-24T09:07:11.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.691+0000 7fab99600640 1 --2- 172.21.15.69:0/4112191203 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fab800a3db0 0x7fab800a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:11.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.691+0000 7fab99600640 1 -- 172.21.15.69:0/4112191203 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fab840f1620 con 0x7fab9c105400 2024-04-24T09:07:11.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.691+0000 7fab9be00640 1 --2- 172.21.15.69:0/4112191203 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fab800a3db0 0x7fab800a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:11.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.692+0000 7fab9be00640 1 --2- 172.21.15.69:0/4112191203 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fab800a3db0 0x7fab800a6270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fab8c005d00 tx=0x7fab8c00b410 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:11.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.695+0000 7fab99600640 1 -- 172.21.15.69:0/4112191203 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab840bd7b0 con 0x7fab9c105400 2024-04-24T09:07:11.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:11 smithi130 ceph-mon[31553]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-24T09:07:11.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:11 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2040379490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:07:11.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:11 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:11 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:11 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:11 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:11 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:11 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:07:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:11 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi130 to 176.4M 2024-04-24T09:07:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:11 smithi130 ceph-mon[31553]: Unable to set osd_memory_target on smithi130 to 185046937: error parsing value: Value '185046937' is below minimum 939524096 2024-04-24T09:07:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:11 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:07:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:11 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:07:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:11 smithi130 ceph-mon[31553]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-24T09:07:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:11 smithi130 ceph-mon[31553]: Updating smithi081:/etc/ceph/ceph.conf 2024-04-24T09:07:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:11 smithi130 ceph-mon[31553]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-24T09:07:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:11 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2581275291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:07:11.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.979+0000 7faba2e00640 1 -- 172.21.15.69:0/4112191203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab9c02d620 con 0x7fab9c105400 2024-04-24T09:07:11.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.980+0000 7fab99600640 1 -- 172.21.15.69:0/4112191203 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11873 (secure 0 0 0) 0x7fab840bd0b0 con 0x7fab9c105400 2024-04-24T09:07:11.980 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:07:11.980 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:04:01.991559+0000","modified":"2024-04-24T09:07:02.537848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34532,"mds_1":34496,"mds_2":34460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34496":{"gid":34496,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34532":{"gid":34532,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34532,"qdb_cluster":[34532,34460,34496]},"id":1}]} 2024-04-24T09:07:11.980 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 24 2024-04-24T09:07:11.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.982+0000 7faba2e00640 1 -- 172.21.15.69:0/4112191203 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fab800a3db0 msgr2=0x7fab800a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:11.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.982+0000 7faba2e00640 1 --2- 172.21.15.69:0/4112191203 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fab800a3db0 0x7fab800a6270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fab8c005d00 tx=0x7fab8c00b410 comp rx=0 tx=0).stop 2024-04-24T09:07:11.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.982+0000 7faba2e00640 1 -- 172.21.15.69:0/4112191203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9c105400 msgr2=0x7fab9c19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:11.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.982+0000 7faba2e00640 1 --2- 172.21.15.69:0/4112191203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9c105400 0x7fab9c19aa20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fab840314e0 tx=0x7fab84003c80 comp rx=0 tx=0).stop 2024-04-24T09:07:11.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.983+0000 7faba2e00640 1 -- 172.21.15.69:0/4112191203 shutdown_connections 2024-04-24T09:07:11.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.983+0000 7faba2e00640 1 --2- 172.21.15.69:0/4112191203 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fab800a3db0 0x7fab800a6270 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.983+0000 7faba2e00640 1 --2- 172.21.15.69:0/4112191203 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab9c107d30 0x7fab9c1a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.983+0000 7faba2e00640 1 --2- 172.21.15.69:0/4112191203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9c105400 0x7fab9c19aa20 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.983+0000 7faba2e00640 1 --2- 172.21.15.69:0/4112191203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab9c102ad0 0x7fab9c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:11.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.983+0000 7faba2e00640 1 -- 172.21.15.69:0/4112191203 >> 172.21.15.69:0/4112191203 conn(0x7fab9c0fc4e0 msgr2=0x7fab9c102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:11.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.983+0000 7faba2e00640 1 -- 172.21.15.69:0/4112191203 shutdown_connections 2024-04-24T09:07:11.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:11.983+0000 7faba2e00640 1 -- 172.21.15.69:0/4112191203 wait complete. 2024-04-24T09:07:12.000 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:07:12.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.177+0000 7f6938e00640 1 -- 172.21.15.69:0/423830789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6934102900 msgr2=0x7f6934104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.177+0000 7f6938e00640 1 --2- 172.21.15.69:0/423830789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6934102900 0x7f6934104cf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6924009900 tx=0x7f692402a410 comp rx=0 tx=0).stop 2024-04-24T09:07:12.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.177+0000 7f6938e00640 1 -- 172.21.15.69:0/423830789 shutdown_connections 2024-04-24T09:07:12.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.177+0000 7f6938e00640 1 --2- 172.21.15.69:0/423830789 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6934107b60 0x7f6934109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.177+0000 7f6938e00640 1 --2- 172.21.15.69:0/423830789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6934105230 0x7f6934107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.177+0000 7f6938e00640 1 --2- 172.21.15.69:0/423830789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6934102900 0x7f6934104cf0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.177+0000 7f6938e00640 1 -- 172.21.15.69:0/423830789 >> 172.21.15.69:0/423830789 conn(0x7f69340fc310 msgr2=0x7f69340fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:12.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.177+0000 7f6938e00640 1 -- 172.21.15.69:0/423830789 shutdown_connections 2024-04-24T09:07:12.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.177+0000 7f6938e00640 1 -- 172.21.15.69:0/423830789 wait complete. 2024-04-24T09:07:12.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.178+0000 7f6938e00640 1 Processor -- start 2024-04-24T09:07:12.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.178+0000 7f6938e00640 1 -- start start 2024-04-24T09:07:12.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6938e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6934102900 0x7f6934102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6938e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6934105230 0x7f69341008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6938e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6934107b60 0x7f6934100df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6938e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f693410da20 con 0x7f6934105230 2024-04-24T09:07:12.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6938e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f693410d8a0 con 0x7f6934107b60 2024-04-24T09:07:12.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6938e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f693410dba0 con 0x7f6934102900 2024-04-24T09:07:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6932a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6934105230 0x7f69341008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6932a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6934105230 0x7f69341008b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46222/0 (socket says 172.21.15.69:46222) 2024-04-24T09:07:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6932a00640 1 -- 172.21.15.69:0/2315142519 learned_addr learned my addr 172.21.15.69:0/2315142519 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6933400640 1 --2- 172.21.15.69:0/2315142519 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6934102900 0x7f6934102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6933e00640 1 --2- 172.21.15.69:0/2315142519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6934107b60 0x7f6934100df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6932a00640 1 -- 172.21.15.69:0/2315142519 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6934102900 msgr2=0x7f6934102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6932a00640 1 --2- 172.21.15.69:0/2315142519 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6934102900 0x7f6934102270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6932a00640 1 -- 172.21.15.69:0/2315142519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6934107b60 msgr2=0x7f6934100df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6932a00640 1 --2- 172.21.15.69:0/2315142519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6934107b60 0x7f6934100df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6932a00640 1 -- 172.21.15.69:0/2315142519 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6934101680 con 0x7f6934105230 2024-04-24T09:07:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.179+0000 7f6933e00640 1 --2- 172.21.15.69:0/2315142519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6934107b60 0x7f6934100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.180+0000 7f6932a00640 1 --2- 172.21.15.69:0/2315142519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6934105230 0x7f69341008b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f692000e9c0 tx=0x7f692000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.180+0000 7f692be00640 1 -- 172.21.15.69:0/2315142519 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692000cd80 con 0x7f6934105230 2024-04-24T09:07:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.180+0000 7f692be00640 1 -- 172.21.15.69:0/2315142519 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69200054d0 con 0x7f6934105230 2024-04-24T09:07:12.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.180+0000 7f6938e00640 1 -- 172.21.15.69:0/2315142519 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69341a2210 con 0x7f6934105230 2024-04-24T09:07:12.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.180+0000 7f692be00640 1 -- 172.21.15.69:0/2315142519 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920010430 con 0x7f6934105230 2024-04-24T09:07:12.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.180+0000 7f6938e00640 1 -- 172.21.15.69:0/2315142519 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69341a2760 con 0x7f6934105230 2024-04-24T09:07:12.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.181+0000 7f6938e00640 1 -- 172.21.15.69:0/2315142519 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69340fde00 con 0x7f6934105230 2024-04-24T09:07:12.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.184+0000 7f692be00640 1 -- 172.21.15.69:0/2315142519 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f6920004fd0 con 0x7f6934105230 2024-04-24T09:07:12.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.186+0000 7f692be00640 1 --2- 172.21.15.69:0/2315142519 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69180a3db0 0x7f69180a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.186+0000 7f692be00640 1 -- 172.21.15.69:0/2315142519 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f69200d11e0 con 0x7f6934105230 2024-04-24T09:07:12.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.186+0000 7f6933400640 1 --2- 172.21.15.69:0/2315142519 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69180a3db0 0x7f69180a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.187+0000 7f6933400640 1 --2- 172.21.15.69:0/2315142519 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69180a3db0 0x7f69180a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69240314e0 tx=0x7f6924005a20 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:12.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.190+0000 7f692be00640 1 -- 172.21.15.69:0/2315142519 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f692009d3d0 con 0x7f6934105230 2024-04-24T09:07:12.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.437+0000 7f6938e00640 1 -- 172.21.15.69:0/2315142519 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f693402d870 con 0x7f6934105230 2024-04-24T09:07:12.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.439+0000 7f692be00640 1 -- 172.21.15.69:0/2315142519 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24941 (secure 0 0 0) 0x7f692009ccd0 con 0x7f6934105230 2024-04-24T09:07:12.439 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:07:12.439 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":93,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:07:03.864053+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/660205566":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/1098677190":"2024-04-25T09:07:03.863940+0000","172.21.15.69:0/3145713060":"2024-04-25T09:07:03.863940+0000","172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/3008138297":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/2901345957":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/1098677190":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:07:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.441+0000 7f6938e00640 1 -- 172.21.15.69:0/2315142519 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69180a3db0 msgr2=0x7f69180a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.441+0000 7f6938e00640 1 --2- 172.21.15.69:0/2315142519 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69180a3db0 0x7f69180a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69240314e0 tx=0x7f6924005a20 comp rx=0 tx=0).stop 2024-04-24T09:07:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.441+0000 7f6938e00640 1 -- 172.21.15.69:0/2315142519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6934105230 msgr2=0x7f69341008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.441+0000 7f6938e00640 1 --2- 172.21.15.69:0/2315142519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6934105230 0x7f69341008b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f692000e9c0 tx=0x7f692000ee80 comp rx=0 tx=0).stop 2024-04-24T09:07:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.441+0000 7f6938e00640 1 -- 172.21.15.69:0/2315142519 shutdown_connections 2024-04-24T09:07:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.441+0000 7f6938e00640 1 --2- 172.21.15.69:0/2315142519 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69180a3db0 0x7f69180a6270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.441+0000 7f6938e00640 1 --2- 172.21.15.69:0/2315142519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6934107b60 0x7f6934100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.441+0000 7f6938e00640 1 --2- 172.21.15.69:0/2315142519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6934105230 0x7f69341008b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.441+0000 7f6938e00640 1 --2- 172.21.15.69:0/2315142519 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6934102900 0x7f6934102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.441+0000 7f6938e00640 1 -- 172.21.15.69:0/2315142519 >> 172.21.15.69:0/2315142519 conn(0x7f69340fc310 msgr2=0x7f69341033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:12.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.441+0000 7f6938e00640 1 -- 172.21.15.69:0/2315142519 shutdown_connections 2024-04-24T09:07:12.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.441+0000 7f6938e00640 1 -- 172.21.15.69:0/2315142519 wait complete. 2024-04-24T09:07:12.459 DEBUG:tasks.fwd_scrub:Yielding 2024-04-24T09:07:12.459 INFO:teuthology.run_tasks:Running task exec... 2024-04-24T09:07:12.468 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T09:07:12.468 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-04-24T09:07:12.468 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-24T09:07:12.469 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-24T09:07:12.469 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:07:12.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:07:12.469 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:07:12.524 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:12 smithi069 ceph-mon[28170]: Updating smithi069:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:07:12.524 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:12 smithi069 ceph-mon[28170]: Updating smithi130:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:07:12.524 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:12 smithi069 ceph-mon[28170]: Updating smithi081:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:07:12.524 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:12 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3829979086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:07:12.524 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:12 smithi069 ceph-mon[28170]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:07:12.525 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:12 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/4112191203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:07:12.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:12 smithi081 ceph-mon[30961]: Updating smithi069:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:07:12.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:12 smithi081 ceph-mon[30961]: Updating smithi130:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:07:12.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:12 smithi081 ceph-mon[30961]: Updating smithi081:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:07:12.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:12 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3829979086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:07:12.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:12 smithi081 ceph-mon[30961]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:07:12.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:12 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/4112191203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:07:12.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.642+0000 7f3902600640 1 -- 172.21.15.69:0/1736193195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38fc104e00 msgr2=0x7f38fc108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.642+0000 7f3902600640 1 --2- 172.21.15.69:0/1736193195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38fc104e00 0x7f38fc108e40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009900 tx=0x7f38e802a410 comp rx=0 tx=0).stop 2024-04-24T09:07:12.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.643+0000 7f3902600640 1 -- 172.21.15.69:0/1736193195 shutdown_connections 2024-04-24T09:07:12.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.643+0000 7f3902600640 1 --2- 172.21.15.69:0/1736193195 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38fc109520 0x7f38fc1110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.643+0000 7f3902600640 1 --2- 172.21.15.69:0/1736193195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38fc104e00 0x7f38fc108e40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.643+0000 7f3902600640 1 --2- 172.21.15.69:0/1736193195 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38fc104450 0x7f38fc104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.643+0000 7f3902600640 1 -- 172.21.15.69:0/1736193195 >> 172.21.15.69:0/1736193195 conn(0x7f38fc0ffcf0 msgr2=0x7f38fc102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:12.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.643+0000 7f3902600640 1 -- 172.21.15.69:0/1736193195 shutdown_connections 2024-04-24T09:07:12.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.643+0000 7f3902600640 1 -- 172.21.15.69:0/1736193195 wait complete. 2024-04-24T09:07:12.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.644+0000 7f3902600640 1 Processor -- start 2024-04-24T09:07:12.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.644+0000 7f3902600640 1 -- start start 2024-04-24T09:07:12.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.645+0000 7f3902600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38fc104450 0x7f38fc1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.645+0000 7f3902600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38fc109520 0x7f38fc1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.645+0000 7f3902600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38fc19a7f0 0x7f38fc19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.645+0000 7f3902600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc1144b0 con 0x7f38fc109520 2024-04-24T09:07:12.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.645+0000 7f3902600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc114330 con 0x7f38fc104450 2024-04-24T09:07:12.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.645+0000 7f3902600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc114630 con 0x7f38fc19a7f0 2024-04-24T09:07:12.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.645+0000 7f38faa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38fc109520 0x7f38fc1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.645+0000 7f38faa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38fc109520 0x7f38fc1a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46250/0 (socket says 172.21.15.69:46250) 2024-04-24T09:07:12.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.645+0000 7f38faa00640 1 -- 172.21.15.69:0/502475073 learned_addr learned my addr 172.21.15.69:0/502475073 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:12.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.645+0000 7f38fb400640 1 --2- 172.21.15.69:0/502475073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38fc104450 0x7f38fc1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.645+0000 7f38fbe00640 1 --2- 172.21.15.69:0/502475073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38fc19a7f0 0x7f38fc19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.646+0000 7f38faa00640 1 -- 172.21.15.69:0/502475073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38fc19a7f0 msgr2=0x7f38fc19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.646+0000 7f38faa00640 1 --2- 172.21.15.69:0/502475073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38fc19a7f0 0x7f38fc19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.646+0000 7f38faa00640 1 -- 172.21.15.69:0/502475073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38fc104450 msgr2=0x7f38fc1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.646+0000 7f38faa00640 1 --2- 172.21.15.69:0/502475073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38fc104450 0x7f38fc1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.646+0000 7f38faa00640 1 -- 172.21.15.69:0/502475073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38fc19b380 con 0x7f38fc109520 2024-04-24T09:07:12.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.646+0000 7f38fbe00640 1 --2- 172.21.15.69:0/502475073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38fc19a7f0 0x7f38fc19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:12.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.646+0000 7f38faa00640 1 --2- 172.21.15.69:0/502475073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38fc109520 0x7f38fc1a0bc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f38e8005ab0 tx=0x7f38e8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:12.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.646+0000 7f38f3e00640 1 -- 172.21.15.69:0/502475073 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e803f070 con 0x7f38fc109520 2024-04-24T09:07:12.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.646+0000 7f38f3e00640 1 -- 172.21.15.69:0/502475073 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38e8031b50 con 0x7f38fc109520 2024-04-24T09:07:12.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.647+0000 7f38f3e00640 1 -- 172.21.15.69:0/502475073 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e8035030 con 0x7f38fc109520 2024-04-24T09:07:12.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.647+0000 7f3902600640 1 -- 172.21.15.69:0/502475073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38fc19b610 con 0x7f38fc109520 2024-04-24T09:07:12.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.647+0000 7f3902600640 1 -- 172.21.15.69:0/502475073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38fc1a77f0 con 0x7f38fc109520 2024-04-24T09:07:12.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.648+0000 7f3902600640 1 -- 172.21.15.69:0/502475073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38fc105920 con 0x7f38fc109520 2024-04-24T09:07:12.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.652+0000 7f38f3e00640 1 -- 172.21.15.69:0/502475073 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f38e8042070 con 0x7f38fc109520 2024-04-24T09:07:12.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.653+0000 7f38f3e00640 1 --2- 172.21.15.69:0/502475073 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f38e00a3ee0 0x7f38e00a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.654+0000 7f38f3e00640 1 -- 172.21.15.69:0/502475073 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f38e80f0f60 con 0x7f38fc109520 2024-04-24T09:07:12.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.654+0000 7f38fb400640 1 --2- 172.21.15.69:0/502475073 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f38e00a3ee0 0x7f38e00a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.655+0000 7f38fb400640 1 --2- 172.21.15.69:0/502475073 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f38e00a3ee0 0x7f38e00a63a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009bd0 tx=0x7f38ec005f90 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:12.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.658+0000 7f38f3e00640 1 -- 172.21.15.69:0/502475073 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38e80bd0f0 con 0x7f38fc109520 2024-04-24T09:07:12.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.659+0000 7efda9600640 1 -- 172.21.15.69:0/248437861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda4112eb0 msgr2=0x7efda41152a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.659+0000 7efda9600640 1 --2- 172.21.15.69:0/248437861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda4112eb0 0x7efda41152a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009ca0 tx=0x7efd9c02a950 comp rx=0 tx=0).stop 2024-04-24T09:07:12.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.659+0000 7efda9600640 1 -- 172.21.15.69:0/248437861 shutdown_connections 2024-04-24T09:07:12.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.659+0000 7efda9600640 1 --2- 172.21.15.69:0/248437861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda4112eb0 0x7efda41152a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.659+0000 7efda9600640 1 --2- 172.21.15.69:0/248437861 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efda4074cc0 0x7efda4112970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.659+0000 7efda9600640 1 --2- 172.21.15.69:0/248437861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efda40743a0 0x7efda4074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.659+0000 7efda9600640 1 -- 172.21.15.69:0/248437861 >> 172.21.15.69:0/248437861 conn(0x7efda40fd9d0 msgr2=0x7efda40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:12.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.659+0000 7efda9600640 1 -- 172.21.15.69:0/248437861 shutdown_connections 2024-04-24T09:07:12.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.659+0000 7efda9600640 1 -- 172.21.15.69:0/248437861 wait complete. 2024-04-24T09:07:12.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.660+0000 7efda9600640 1 Processor -- start 2024-04-24T09:07:12.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.660+0000 7efda9600640 1 -- start start 2024-04-24T09:07:12.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.661+0000 7efda9600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda40743a0 0x7efda41a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.661+0000 7efda9600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efda4074cc0 0x7efda41a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.661+0000 7efda9600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efda4112eb0 0x7efda419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.661+0000 7efda9600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efda4118720 con 0x7efda40743a0 2024-04-24T09:07:12.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.661+0000 7efda9600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efda41185a0 con 0x7efda4112eb0 2024-04-24T09:07:12.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.661+0000 7efda9600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efda41188a0 con 0x7efda4074cc0 2024-04-24T09:07:12.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.661+0000 7efda3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda40743a0 0x7efda41a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.661+0000 7efda3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda40743a0 0x7efda41a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46254/0 (socket says 172.21.15.69:46254) 2024-04-24T09:07:12.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.661+0000 7efda3e00640 1 -- 172.21.15.69:0/3882702027 learned_addr learned my addr 172.21.15.69:0/3882702027 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:12.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.661+0000 7efda3400640 1 --2- 172.21.15.69:0/3882702027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efda4074cc0 0x7efda41a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.661+0000 7efda8c00640 1 --2- 172.21.15.69:0/3882702027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efda4112eb0 0x7efda419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.661+0000 7efda3e00640 1 -- 172.21.15.69:0/3882702027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efda4074cc0 msgr2=0x7efda41a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.661+0000 7efda3e00640 1 --2- 172.21.15.69:0/3882702027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efda4074cc0 0x7efda41a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.662+0000 7efda3e00640 1 -- 172.21.15.69:0/3882702027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efda4112eb0 msgr2=0x7efda419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.662+0000 7efda3e00640 1 --2- 172.21.15.69:0/3882702027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efda4112eb0 0x7efda419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.662+0000 7efda3e00640 1 -- 172.21.15.69:0/3882702027 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda419f280 con 0x7efda40743a0 2024-04-24T09:07:12.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.662+0000 7efda3e00640 1 --2- 172.21.15.69:0/3882702027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda40743a0 0x7efda41a4950 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7efd9400ea60 tx=0x7efd9400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:12.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.662+0000 7efda0c00640 1 -- 172.21.15.69:0/3882702027 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9400cde0 con 0x7efda40743a0 2024-04-24T09:07:12.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.662+0000 7efda0c00640 1 -- 172.21.15.69:0/3882702027 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd940035f0 con 0x7efda40743a0 2024-04-24T09:07:12.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.662+0000 7efda0c00640 1 -- 172.21.15.69:0/3882702027 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94010690 con 0x7efda40743a0 2024-04-24T09:07:12.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.662+0000 7efda9600640 1 -- 172.21.15.69:0/3882702027 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda419f570 con 0x7efda40743a0 2024-04-24T09:07:12.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.662+0000 7efda9600640 1 -- 172.21.15.69:0/3882702027 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda41ab810 con 0x7efda40743a0 2024-04-24T09:07:12.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.666+0000 7efdad9c7740 1 Processor -- start 2024-04-24T09:07:12.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.666+0000 7efdad9c7740 1 -- start start 2024-04-24T09:07:12.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.666+0000 7efda0c00640 1 -- 172.21.15.69:0/3882702027 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7efd940037a0 con 0x7efda40743a0 2024-04-24T09:07:12.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.666+0000 7efdad9c7740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fc1270 0x55acd5fc1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.666+0000 7efdad9c7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 0x55acd5fcbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.666+0000 7efdad9c7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fcc260 0x55acd5fce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.666+0000 7efdad9c7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55acd5f26e70 con 0x55acd5fcc260 2024-04-24T09:07:12.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.666+0000 7efdad9c7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55acd5ecd380 con 0x55acd5fc1b90 2024-04-24T09:07:12.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.666+0000 7efdad9c7740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55acd5f22e30 con 0x55acd5fc1270 2024-04-24T09:07:12.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.667+0000 7efda3e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 0x55acd5fcbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.667+0000 7efda8c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fc1270 0x55acd5fc1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.667+0000 7efda3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fcc260 0x55acd5fce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.667+0000 7efda8c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fc1270 0x55acd5fc1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.69:56236/0 (socket says 172.21.15.69:56236) 2024-04-24T09:07:12.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.667+0000 7efda8c00640 1 -- 172.21.15.69:0/1312121694 learned_addr learned my addr 172.21.15.69:0/1312121694 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:12.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.667+0000 7efda3e00640 1 -- 172.21.15.69:0/1312121694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fc1270 msgr2=0x55acd5fc1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:12.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.667+0000 7efda3e00640 1 --2- 172.21.15.69:0/1312121694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fc1270 0x55acd5fc1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.667+0000 7efda3e00640 1 -- 172.21.15.69:0/1312121694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fcc260 msgr2=0x55acd5fce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:12.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.667+0000 7efda3e00640 1 --2- 172.21.15.69:0/1312121694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fcc260 0x55acd5fce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.667+0000 7efda3e00640 1 -- 172.21.15.69:0/1312121694 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acd5d017e0 con 0x55acd5fc1b90 2024-04-24T09:07:12.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.667+0000 7efda3e00640 1 --2- 172.21.15.69:0/1312121694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 0x55acd5fcbd20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efd940ce2b0 tx=0x7efd940d94b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de0aec7cc62fc762 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:12.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.668+0000 7efd99200640 1 -- 172.21.15.69:0/1312121694 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd940e4070 con 0x55acd5fc1b90 2024-04-24T09:07:12.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.668+0000 7efd99200640 1 -- 172.21.15.69:0/1312121694 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd940d9df0 con 0x55acd5fc1b90 2024-04-24T09:07:12.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.668+0000 7efd99200640 1 -- 172.21.15.69:0/1312121694 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd940df660 con 0x55acd5fc1b90 2024-04-24T09:07:12.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.669+0000 7efda0c00640 1 --2- 172.21.15.69:0/3882702027 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7efd880a3db0 0x7efd880a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.669+0000 7efdad9c7740 1 -- 172.21.15.69:0/1312121694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 msgr2=0x55acd5fcbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.669+0000 7efdad9c7740 1 --2- 172.21.15.69:0/1312121694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 0x55acd5fcbd20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efd940ce2b0 tx=0x7efd940d94b0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.669+0000 7efda0c00640 1 -- 172.21.15.69:0/3882702027 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7efd940d0f60 con 0x7efda40743a0 2024-04-24T09:07:12.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.669+0000 7efdad9c7740 1 -- 172.21.15.69:0/1312121694 shutdown_connections 2024-04-24T09:07:12.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.669+0000 7efdad9c7740 1 --2- 172.21.15.69:0/1312121694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fcc260 0x55acd5fce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.669+0000 7efdad9c7740 1 --2- 172.21.15.69:0/1312121694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 0x55acd5fcbd20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.669+0000 7efdad9c7740 1 --2- 172.21.15.69:0/1312121694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fc1270 0x55acd5fc1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.669+0000 7efda8c00640 1 --2- 172.21.15.69:0/3882702027 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7efd880a3db0 0x7efd880a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.669+0000 7efdad9c7740 1 -- 172.21.15.69:0/1312121694 >> 172.21.15.69:0/1312121694 conn(0x55acd5fbd590 msgr2=0x55acd5fbd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:12.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.669+0000 7efdad9c7740 1 -- 172.21.15.69:0/1312121694 shutdown_connections 2024-04-24T09:07:12.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.669+0000 7efdad9c7740 1 -- 172.21.15.69:0/1312121694 wait complete. 2024-04-24T09:07:12.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.670+0000 7efda8c00640 1 --2- 172.21.15.69:0/3882702027 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7efd880a3db0 0x7efd880a6270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efda406b550 tx=0x7efd9c005c50 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:12.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.670+0000 7efdad9c7740 1 -- start start 2024-04-24T09:07:12.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.670+0000 7efdad9c7740 1 Processor -- start 2024-04-24T09:07:12.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.671+0000 7efdad9c7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1270 0x55acd5fc91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.671+0000 7efdad9c7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 0x55acd5fc9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.671+0000 7efdad9c7740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fcc260 0x55acd5fc9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.671+0000 7efdad9c7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55acd5fe0060 con 0x55acd5fc1270 2024-04-24T09:07:12.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.671+0000 7efdad9c7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55acd5ee0660 con 0x55acd5fc1b90 2024-04-24T09:07:12.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.671+0000 7efdad9c7740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55acd5fe01e0 con 0x55acd5fcc260 2024-04-24T09:07:12.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.671+0000 7efdad9c7740 10 client.0 _ll_register_callbacks cb 0x55acd5eceaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:12.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.671+0000 7efda3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1270 0x55acd5fc91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.671+0000 7efda3400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 0x55acd5fc9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.671+0000 7efda3400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 0x55acd5fc9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.69:40928/0 (socket says 172.21.15.69:40928) 2024-04-24T09:07:12.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.671+0000 7efda3400640 1 -- 172.21.15.69:0/3287998499 learned_addr learned my addr 172.21.15.69:0/3287998499 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:12.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.672+0000 7efda8c00640 1 --2- 172.21.15.69:0/3287998499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fcc260 0x55acd5fc9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.672+0000 7efda3400640 1 -- 172.21.15.69:0/3287998499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fcc260 msgr2=0x55acd5fc9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:12.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.672+0000 7efda3400640 1 --2- 172.21.15.69:0/3287998499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fcc260 0x55acd5fc9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.672+0000 7efda3400640 1 -- 172.21.15.69:0/3287998499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1270 msgr2=0x55acd5fc91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:12.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.672+0000 7efda3400640 1 --2- 172.21.15.69:0/3287998499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1270 0x55acd5fc91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.672+0000 7efda3400640 1 -- 172.21.15.69:0/3287998499 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acd5d61f70 con 0x55acd5fc1b90 2024-04-24T09:07:12.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.672+0000 7efda3400640 1 --2- 172.21.15.69:0/3287998499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 0x55acd5fc9730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efd90012700 tx=0x7efd90012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e704917c1ded700 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:12.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.672+0000 7efd8d600640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-04-24T09:07:12.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.673+0000 7efd8d600640 1 -- 172.21.15.69:0/3287998499 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9000f920 con 0x55acd5fc1b90 2024-04-24T09:07:12.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.673+0000 7efd8d600640 1 -- 172.21.15.69:0/3287998499 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7efd9000fac0 con 0x55acd5fc1b90 2024-04-24T09:07:12.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.673+0000 7efdad9c7740 1 -- 172.21.15.69:0/3287998499 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd940d9df0 con 0x55acd5fc1b90 2024-04-24T09:07:12.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.673+0000 7efd8d600640 1 -- 172.21.15.69:0/3287998499 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efd90025050 con 0x55acd5fc1b90 2024-04-24T09:07:12.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.673+0000 7efdad9c7740 10 client.35159 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:12.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.673+0000 7efdad9c7740 1 -- 172.21.15.69:0/3287998499 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55acd5bf9170 con 0x55acd5fc1b90 2024-04-24T09:07:12.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.673+0000 7efd8d600640 1 -- 172.21.15.69:0/3287998499 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd940d9df0 con 0x55acd5fc1b90 2024-04-24T09:07:12.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.674+0000 7efd8d600640 1 -- 172.21.15.69:0/3287998499 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7efd90010430 con 0x55acd5fc1b90 2024-04-24T09:07:12.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.674+0000 7efdad9c7740 10 client.35159 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:12.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.674+0000 7efdad9c7740 10 client.35159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:12.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.674+0000 7efdad9c7740 20 client.35159 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:12.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.674+0000 7efdad9c7740 1 --2- 172.21.15.69:0/3287998499 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55acd5fed6a0 0x55acd5fefae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.674+0000 7efdad9c7740 4 client.35159 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:12.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.674+0000 7efdad9c7740 1 -- 172.21.15.69:0/3287998499 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acd5f34890 con 0x55acd5fed6a0 2024-04-24T09:07:12.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.674+0000 7efda3e00640 1 --2- 172.21.15.69:0/3287998499 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55acd5fed6a0 0x55acd5fefae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.675+0000 7efda3e00640 1 --2- 172.21.15.69:0/3287998499 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55acd5fed6a0 0x55acd5fefae0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=842de2c0b04e6045 server_cookie=4bbb4d52bcb6e5a0 in_seq=0 out_seq=0 2024-04-24T09:07:12.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.675+0000 7efd8d600640 10 client.35159 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:12.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.676+0000 7efd8d600640 1 -- 172.21.15.69:0/3287998499 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55acd5f34890 con 0x55acd5fed6a0 2024-04-24T09:07:12.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.676+0000 7efd8d600640 10 client.35159 handle_command_reply: tid=0 2024-04-24T09:07:12.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.676+0000 7efdad9c7740 1 client.35159 shutdown 2024-04-24T09:07:12.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.676+0000 7efdad9c7740 1 -- 172.21.15.69:0/3287998499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 msgr2=0x55acd5fc9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.676+0000 7efdad9c7740 1 --2- 172.21.15.69:0/3287998499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 0x55acd5fc9730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efd90012700 tx=0x7efd90012bc0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.676+0000 7efdad9c7740 1 -- 172.21.15.69:0/3287998499 shutdown_connections 2024-04-24T09:07:12.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.676+0000 7efdad9c7740 1 --2- 172.21.15.69:0/3287998499 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55acd5fed6a0 0x55acd5fefae0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.676+0000 7efdad9c7740 1 --2- 172.21.15.69:0/3287998499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fcc260 0x55acd5fc9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.676+0000 7efdad9c7740 1 --2- 172.21.15.69:0/3287998499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 0x55acd5fc9730 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.676+0000 7efd8d600640 0 client.35159 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:12.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.676+0000 7efdad9c7740 1 --2- 172.21.15.69:0/3287998499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1270 0x55acd5fc91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.676+0000 7efdad9c7740 1 -- 172.21.15.69:0/3287998499 >> 172.21.15.69:0/3287998499 conn(0x55acd5fbd590 msgr2=0x55acd5fc3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:12.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.676+0000 7efdad9c7740 1 -- 172.21.15.69:0/3287998499 shutdown_connections 2024-04-24T09:07:12.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.676+0000 7efdad9c7740 1 -- 172.21.15.69:0/3287998499 wait complete. 2024-04-24T09:07:12.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.676+0000 7efdad9c7740 20 client.35159 trim_cache size 0 max 16384 2024-04-24T09:07:12.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.695+0000 7efdad9c7740 1 Processor -- start 2024-04-24T09:07:12.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.695+0000 7efdad9c7740 1 -- start start 2024-04-24T09:07:12.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.695+0000 7efdad9c7740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fc1270 0x55acd5fc4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.695+0000 7efdad9c7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1b90 0x55acd5fe8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.695+0000 7efdad9c7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fcc260 0x55acd5fc39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.695+0000 7efdad9c7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efda41af870 con 0x55acd5fc1b90 2024-04-24T09:07:12.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.695+0000 7efdad9c7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efda41ad140 con 0x55acd5fcc260 2024-04-24T09:07:12.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.695+0000 7efdad9c7740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efda4073710 con 0x55acd5fc1270 2024-04-24T09:07:12.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.696+0000 7efda3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1b90 0x55acd5fe8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.696+0000 7efda3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1b90 0x55acd5fe8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46290/0 (socket says 172.21.15.69:46290) 2024-04-24T09:07:12.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.696+0000 7efda3400640 1 -- 172.21.15.69:0/3511578417 learned_addr learned my addr 172.21.15.69:0/3511578417 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:12.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.696+0000 7efda3e00640 1 --2- 172.21.15.69:0/3511578417 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fc1270 0x55acd5fc4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.696+0000 7efda8c00640 1 --2- 172.21.15.69:0/3511578417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fcc260 0x55acd5fc39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.696+0000 7efda3400640 1 -- 172.21.15.69:0/3511578417 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fc1270 msgr2=0x55acd5fc4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:12.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.696+0000 7efda3400640 1 --2- 172.21.15.69:0/3511578417 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fc1270 0x55acd5fc4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.696+0000 7efda3400640 1 -- 172.21.15.69:0/3511578417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fcc260 msgr2=0x55acd5fc39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:12.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.696+0000 7efda3400640 1 --2- 172.21.15.69:0/3511578417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fcc260 0x55acd5fc39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.696+0000 7efda3400640 1 -- 172.21.15.69:0/3511578417 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acd5fca150 con 0x55acd5fc1b90 2024-04-24T09:07:12.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.697+0000 7efda3400640 1 --2- 172.21.15.69:0/3511578417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1b90 0x55acd5fe8f20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efd90010880 tx=0x7efd900096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815094881d4ed780 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:12.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.697+0000 7efd8fe00640 1 -- 172.21.15.69:0/3511578417 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90010430 con 0x55acd5fc1b90 2024-04-24T09:07:12.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.697+0000 7efd8fe00640 1 -- 172.21.15.69:0/3511578417 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd900105d0 con 0x55acd5fc1b90 2024-04-24T09:07:12.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.697+0000 7efd8fe00640 1 -- 172.21.15.69:0/3511578417 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd900050e0 con 0x55acd5fc1b90 2024-04-24T09:07:12.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.698+0000 7efdad9c7740 1 -- 172.21.15.69:0/3511578417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1b90 msgr2=0x55acd5fe8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.698+0000 7efdad9c7740 1 --2- 172.21.15.69:0/3511578417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1b90 0x55acd5fe8f20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efd90010880 tx=0x7efd900096a0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.698+0000 7efdad9c7740 1 -- 172.21.15.69:0/3511578417 shutdown_connections 2024-04-24T09:07:12.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.698+0000 7efdad9c7740 1 --2- 172.21.15.69:0/3511578417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fcc260 0x55acd5fc39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.698+0000 7efdad9c7740 1 --2- 172.21.15.69:0/3511578417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1b90 0x55acd5fe8f20 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.698+0000 7efdad9c7740 1 --2- 172.21.15.69:0/3511578417 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fc1270 0x55acd5fc4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.698+0000 7efdad9c7740 1 -- 172.21.15.69:0/3511578417 >> 172.21.15.69:0/3511578417 conn(0x55acd5fbd590 msgr2=0x55acd5fc9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:12.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.698+0000 7efdad9c7740 1 -- 172.21.15.69:0/3511578417 shutdown_connections 2024-04-24T09:07:12.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.698+0000 7efdad9c7740 1 -- 172.21.15.69:0/3511578417 wait complete. 2024-04-24T09:07:12.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.698+0000 7efdad9c7740 1 -- start start 2024-04-24T09:07:12.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.699+0000 7efdad9c7740 1 Processor -- start 2024-04-24T09:07:12.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.699+0000 7efdad9c7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1270 0x55acd5fe15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.699+0000 7efdad9c7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 0x55acd5fe1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.699+0000 7efdad9c7740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fcc260 0x55acd6052c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.699+0000 7efdad9c7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efda4073d80 con 0x55acd5fc1270 2024-04-24T09:07:12.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.699+0000 7efdad9c7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efda41aca70 con 0x55acd5fc1b90 2024-04-24T09:07:12.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.699+0000 7efdad9c7740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efda41abcd0 con 0x55acd5fcc260 2024-04-24T09:07:12.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.699+0000 7efdad9c7740 10 client.0 _ll_register_callbacks cb 0x7efd90009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:12.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.699+0000 7efda3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1270 0x55acd5fe15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.700+0000 7efda3400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 0x55acd5fe1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.700+0000 7efda3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1270 0x55acd5fe15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46304/0 (socket says 172.21.15.69:46304) 2024-04-24T09:07:12.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.700+0000 7efda3e00640 1 -- 172.21.15.69:0/1983402539 learned_addr learned my addr 172.21.15.69:0/1983402539 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:12.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.700+0000 7efda8c00640 1 --2- 172.21.15.69:0/1983402539 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fcc260 0x55acd6052c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.700+0000 7efda3400640 1 -- 172.21.15.69:0/1983402539 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fcc260 msgr2=0x55acd6052c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:12.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.700+0000 7efda3400640 1 --2- 172.21.15.69:0/1983402539 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fcc260 0x55acd6052c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.700+0000 7efda3400640 1 -- 172.21.15.69:0/1983402539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1270 msgr2=0x55acd5fe15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.700+0000 7efda3400640 1 --2- 172.21.15.69:0/1983402539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1270 0x55acd5fe15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.700+0000 7efda3400640 1 -- 172.21.15.69:0/1983402539 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acd60531b0 con 0x55acd5fc1b90 2024-04-24T09:07:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.700+0000 7efda3e00640 1 --2- 172.21.15.69:0/1983402539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1270 0x55acd5fe15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:07:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.700+0000 7efda3400640 1 --2- 172.21.15.69:0/1983402539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 0x55acd5fe1b10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55acd5eceb20 tx=0x7efd90010c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38bf51f7ce405732 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.700+0000 7efd8d600640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-04-24T09:07:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.701+0000 7efd8d600640 1 -- 172.21.15.69:0/1983402539 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90022040 con 0x55acd5fc1b90 2024-04-24T09:07:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.701+0000 7efdad9c7740 1 -- 172.21.15.69:0/1983402539 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd900105d0 con 0x55acd5fc1b90 2024-04-24T09:07:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.701+0000 7efd8d600640 1 -- 172.21.15.69:0/1983402539 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7efd90018040 con 0x55acd5fc1b90 2024-04-24T09:07:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.701+0000 7efd8d600640 1 -- 172.21.15.69:0/1983402539 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efd9003a050 con 0x55acd5fc1b90 2024-04-24T09:07:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.701+0000 7efdad9c7740 10 client.35165 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.701+0000 7efdad9c7740 1 -- 172.21.15.69:0/1983402539 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55acd60544f0 con 0x55acd5fc1b90 2024-04-24T09:07:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.701+0000 7efd8d600640 1 -- 172.21.15.69:0/1983402539 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd900105d0 con 0x55acd5fc1b90 2024-04-24T09:07:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.702+0000 7efd8d600640 1 -- 172.21.15.69:0/1983402539 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7efd900035f0 con 0x55acd5fc1b90 2024-04-24T09:07:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.702+0000 7efdad9c7740 10 client.35165 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.702+0000 7efdad9c7740 10 client.35165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.702+0000 7efdad9c7740 20 client.35165 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.702+0000 7efdad9c7740 1 --2- 172.21.15.69:0/1983402539 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55acd6054b80 0x55acd6056fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.702+0000 7efdad9c7740 4 client.35165 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:07:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.702+0000 7efdad9c7740 1 -- 172.21.15.69:0/1983402539 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55acd5f2e700 con 0x55acd6054b80 2024-04-24T09:07:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.702+0000 7efda3e00640 1 --2- 172.21.15.69:0/1983402539 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55acd6054b80 0x55acd6056fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.703+0000 7efda3e00640 1 --2- 172.21.15.69:0/1983402539 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55acd6054b80 0x55acd6056fa0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d6beaa25a44b2f5 server_cookie=fa59f8a568426346 in_seq=0 out_seq=0 2024-04-24T09:07:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.703+0000 7efd8d600640 10 client.35165 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:12.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:12 smithi130 ceph-mon[31553]: Updating smithi069:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:07:12.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:12 smithi130 ceph-mon[31553]: Updating smithi130:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:07:12.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:12 smithi130 ceph-mon[31553]: Updating smithi081:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:07:12.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:12 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3829979086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:07:12.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:12 smithi130 ceph-mon[31553]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:07:12.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:12 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/4112191203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:07:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.705+0000 7efd8d600640 1 -- 172.21.15.69:0/1983402539 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55acd5f2e700 con 0x55acd6054b80 2024-04-24T09:07:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.705+0000 7efd8d600640 10 client.35165 handle_command_reply: tid=0 2024-04-24T09:07:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.705+0000 7efdad9c7740 1 client.35165 shutdown 2024-04-24T09:07:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.705+0000 7efdad9c7740 1 -- 172.21.15.69:0/1983402539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 msgr2=0x55acd5fe1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.705+0000 7efdad9c7740 1 --2- 172.21.15.69:0/1983402539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 0x55acd5fe1b10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55acd5eceb20 tx=0x7efd90010c20 comp rx=0 tx=0).stop 2024-04-24T09:07:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.705+0000 7efdad9c7740 1 -- 172.21.15.69:0/1983402539 shutdown_connections 2024-04-24T09:07:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.705+0000 7efdad9c7740 1 --2- 172.21.15.69:0/1983402539 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55acd6054b80 0x55acd6056fa0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.705+0000 7efdad9c7740 1 --2- 172.21.15.69:0/1983402539 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55acd5fcc260 0x55acd6052c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.705+0000 7efdad9c7740 1 --2- 172.21.15.69:0/1983402539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55acd5fc1b90 0x55acd5fe1b10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.705+0000 7efdad9c7740 1 --2- 172.21.15.69:0/1983402539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acd5fc1270 0x55acd5fe15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.705+0000 7efdad9c7740 1 -- 172.21.15.69:0/1983402539 >> 172.21.15.69:0/1983402539 conn(0x55acd5fbd590 msgr2=0x55acd5fbbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.705+0000 7efd8d600640 0 client.35165 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.705+0000 7efdad9c7740 1 -- 172.21.15.69:0/1983402539 shutdown_connections 2024-04-24T09:07:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.705+0000 7efdad9c7740 1 -- 172.21.15.69:0/1983402539 wait complete. 2024-04-24T09:07:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.705+0000 7efdad9c7740 20 client.35165 trim_cache size 0 max 16384 2024-04-24T09:07:12.707 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:07:12.707 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-04-24T09:07:12.707 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "6bc63386-c795-4e26-952f-9fb62dd68b17", 2024-04-24T09:07:12.707 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-04-24T09:07:12.707 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:07:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.706+0000 7efda9600640 1 -- 172.21.15.69:0/3882702027 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7efd880a3db0 msgr2=0x7efd880a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.706+0000 7efda9600640 1 --2- 172.21.15.69:0/3882702027 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7efd880a3db0 0x7efd880a6270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efda406b550 tx=0x7efd9c005c50 comp rx=0 tx=0).stop 2024-04-24T09:07:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.706+0000 7efda9600640 1 -- 172.21.15.69:0/3882702027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda40743a0 msgr2=0x7efda41a4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.706+0000 7efda9600640 1 --2- 172.21.15.69:0/3882702027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda40743a0 0x7efda41a4950 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7efd9400ea60 tx=0x7efd9400ef20 comp rx=0 tx=0).stop 2024-04-24T09:07:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.707+0000 7efda9600640 1 -- 172.21.15.69:0/3882702027 shutdown_connections 2024-04-24T09:07:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.707+0000 7efda9600640 1 --2- 172.21.15.69:0/3882702027 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7efd880a3db0 0x7efd880a6270 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.707+0000 7efda9600640 1 --2- 172.21.15.69:0/3882702027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efda4112eb0 0x7efda419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.707+0000 7efda9600640 1 --2- 172.21.15.69:0/3882702027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efda4074cc0 0x7efda41a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.707+0000 7efda9600640 1 --2- 172.21.15.69:0/3882702027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda40743a0 0x7efda41a4950 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.707+0000 7efda9600640 1 -- 172.21.15.69:0/3882702027 >> 172.21.15.69:0/3882702027 conn(0x7efda40fd9d0 msgr2=0x7efda40ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.707+0000 7efda9600640 1 -- 172.21.15.69:0/3882702027 shutdown_connections 2024-04-24T09:07:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.707+0000 7efda9600640 1 -- 172.21.15.69:0/3882702027 wait complete. 2024-04-24T09:07:12.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:07:12.725 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:07:12.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.904+0000 7f3902600640 1 -- 172.21.15.69:0/502475073 --> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f38fc104e00 con 0x7f38e00a3ee0 2024-04-24T09:07:12.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.950+0000 7fce8aa00640 1 -- 172.21.15.69:0/2403228872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce84102110 msgr2=0x7fce841024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.950+0000 7fce8aa00640 1 --2- 172.21.15.69:0/2403228872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce84102110 0x7fce841024f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fce78009900 tx=0x7fce7802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.950+0000 7fce8aa00640 1 -- 172.21.15.69:0/2403228872 shutdown_connections 2024-04-24T09:07:12.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.950+0000 7fce8aa00640 1 --2- 172.21.15.69:0/2403228872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce841071e0 0x7fce8410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.950+0000 7fce8aa00640 1 --2- 172.21.15.69:0/2403228872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce84102ac0 0x7fce84106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.950+0000 7fce8aa00640 1 --2- 172.21.15.69:0/2403228872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce84102110 0x7fce841024f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.950+0000 7fce8aa00640 1 -- 172.21.15.69:0/2403228872 >> 172.21.15.69:0/2403228872 conn(0x7fce840fd9d0 msgr2=0x7fce840ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:12.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.950+0000 7fce8aa00640 1 -- 172.21.15.69:0/2403228872 shutdown_connections 2024-04-24T09:07:12.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.950+0000 7fce8aa00640 1 -- 172.21.15.69:0/2403228872 wait complete. 2024-04-24T09:07:12.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.951+0000 7fce8aa00640 1 Processor -- start 2024-04-24T09:07:12.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.952+0000 7fce8aa00640 1 -- start start 2024-04-24T09:07:12.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.952+0000 7fce8aa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce84102110 0x7fce84074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.952+0000 7fce8aa00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce841071e0 0x7fce84074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.952+0000 7fce8aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce84074d80 0x7fce841a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.952+0000 7fce8aa00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fce841121a0 con 0x7fce84074d80 2024-04-24T09:07:12.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.952+0000 7fce8aa00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fce84112020 con 0x7fce841071e0 2024-04-24T09:07:12.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.952+0000 7fce8aa00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fce84112320 con 0x7fce84102110 2024-04-24T09:07:12.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.952+0000 7fce89600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce84102110 0x7fce84074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.952+0000 7fce8a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce84074d80 0x7fce841a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.952+0000 7fce89600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce84102110 0x7fce84074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.69:48596/0 (socket says 172.21.15.69:48596) 2024-04-24T09:07:12.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.952+0000 7fce89600640 1 -- 172.21.15.69:0/3917671013 learned_addr learned my addr 172.21.15.69:0/3917671013 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:12.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.952+0000 7fce88c00640 1 --2- 172.21.15.69:0/3917671013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce841071e0 0x7fce84074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.953+0000 7fce8a000640 1 -- 172.21.15.69:0/3917671013 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce84102110 msgr2=0x7fce84074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.953+0000 7fce8a000640 1 --2- 172.21.15.69:0/3917671013 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce84102110 0x7fce84074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.953+0000 7fce8a000640 1 -- 172.21.15.69:0/3917671013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce841071e0 msgr2=0x7fce84074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.953+0000 7fce8a000640 1 --2- 172.21.15.69:0/3917671013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce841071e0 0x7fce84074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.953+0000 7fce8a000640 1 -- 172.21.15.69:0/3917671013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce841a6f70 con 0x7fce84074d80 2024-04-24T09:07:12.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.953+0000 7fce8a000640 1 --2- 172.21.15.69:0/3917671013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce84074d80 0x7fce841a6870 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00e940 tx=0x7fce7c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:12.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.953+0000 7fce82000640 1 -- 172.21.15.69:0/3917671013 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c004d20 con 0x7fce84074d80 2024-04-24T09:07:12.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.953+0000 7fce82000640 1 -- 172.21.15.69:0/3917671013 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce7c005020 con 0x7fce84074d80 2024-04-24T09:07:12.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.953+0000 7fce8aa00640 1 -- 172.21.15.69:0/3917671013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce841a7260 con 0x7fce84074d80 2024-04-24T09:07:12.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.954+0000 7fce82000640 1 -- 172.21.15.69:0/3917671013 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c0116d0 con 0x7fce84074d80 2024-04-24T09:07:12.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.954+0000 7fce8aa00640 1 -- 172.21.15.69:0/3917671013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce841a7760 con 0x7fce84074d80 2024-04-24T09:07:12.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.957+0000 7fce8ee3b740 1 Processor -- start 2024-04-24T09:07:12.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.958+0000 7fce82000640 1 -- 172.21.15.69:0/3917671013 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fce7c00cb20 con 0x7fce84074d80 2024-04-24T09:07:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.958+0000 7fce8ee3b740 1 -- start start 2024-04-24T09:07:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.958+0000 7fce8ee3b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0691a30 0x55e8e0691e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.958+0000 7fce8ee3b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e0692350 0x55e8e069c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.958+0000 7fce8ee3b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e069ca20 0x55e8e069ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.958+0000 7fce8ee3b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e8e05f71f0 con 0x55e8e0691a30 2024-04-24T09:07:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.958+0000 7fce8ee3b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e8e059d350 con 0x55e8e0692350 2024-04-24T09:07:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.958+0000 7fce8ee3b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55e8e05f31b0 con 0x55e8e069ca20 2024-04-24T09:07:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.959+0000 7fce88c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0691a30 0x55e8e0691e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.959+0000 7fce88c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0691a30 0x55e8e0691e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50068/0 (socket says 172.21.15.69:50068) 2024-04-24T09:07:12.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.959+0000 7fce88c00640 1 -- 172.21.15.69:0/3025079487 learned_addr learned my addr 172.21.15.69:0/3025079487 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:12.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.959+0000 7fce88c00640 1 --2- 172.21.15.69:0/3025079487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e069ca20 0x55e8e069ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.959+0000 7fce89600640 1 --2- 172.21.15.69:0/3025079487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e0692350 0x55e8e069c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.959+0000 7fce88c00640 1 -- 172.21.15.69:0/3025079487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e069ca20 msgr2=0x55e8e069ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:12.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.959+0000 7fce88c00640 1 --2- 172.21.15.69:0/3025079487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e069ca20 0x55e8e069ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.959+0000 7fce88c00640 1 -- 172.21.15.69:0/3025079487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e0692350 msgr2=0x55e8e069c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:12.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.959+0000 7fce88c00640 1 --2- 172.21.15.69:0/3025079487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e0692350 0x55e8e069c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.959+0000 7fce88c00640 1 -- 172.21.15.69:0/3025079487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e05f1f50 con 0x55e8e0691a30 2024-04-24T09:07:12.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.959+0000 7fce82000640 1 --2- 172.21.15.69:0/3917671013 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fce6c0a3de0 0x7fce6c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.959+0000 7fce88c00640 1 --2- 172.21.15.69:0/3025079487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0691a30 0x55e8e0691e10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fce700124c0 tx=0x7fce7000b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5eaf5ecf1c6b903 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:12.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.959+0000 7fce82000640 1 -- 172.21.15.69:0/3917671013 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fce7c0d0ca0 con 0x7fce84074d80 2024-04-24T09:07:12.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.960+0000 7fce89600640 1 --2- 172.21.15.69:0/3917671013 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fce6c0a3de0 0x7fce6c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.960+0000 7fce76600640 1 -- 172.21.15.69:0/3025079487 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7001e640 con 0x55e8e0691a30 2024-04-24T09:07:12.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.960+0000 7fce76600640 1 -- 172.21.15.69:0/3025079487 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce700108e0 con 0x55e8e0691a30 2024-04-24T09:07:12.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.960+0000 7fce76600640 1 -- 172.21.15.69:0/3025079487 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7001d7b0 con 0x55e8e0691a30 2024-04-24T09:07:12.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.961+0000 7fce8ee3b740 1 -- 172.21.15.69:0/3025079487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0691a30 msgr2=0x55e8e0691e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.961+0000 7fce89600640 1 --2- 172.21.15.69:0/3917671013 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fce6c0a3de0 0x7fce6c0a62a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55e8e0692ff0 tx=0x7fce78005a20 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:12.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.961+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/3025079487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0691a30 0x55e8e0691e10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fce700124c0 tx=0x7fce7000b820 comp rx=0 tx=0).stop 2024-04-24T09:07:12.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.961+0000 7fce8ee3b740 1 -- 172.21.15.69:0/3025079487 shutdown_connections 2024-04-24T09:07:12.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.961+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/3025079487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e069ca20 0x55e8e069ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.961+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/3025079487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e0692350 0x55e8e069c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.961+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/3025079487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0691a30 0x55e8e0691e10 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.961+0000 7fce8ee3b740 1 -- 172.21.15.69:0/3025079487 >> 172.21.15.69:0/3025079487 conn(0x55e8e068dd50 msgr2=0x55e8e068e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:12.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.961+0000 7fce8ee3b740 1 -- 172.21.15.69:0/3025079487 shutdown_connections 2024-04-24T09:07:12.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.961+0000 7fce8ee3b740 1 -- 172.21.15.69:0/3025079487 wait complete. 2024-04-24T09:07:12.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.961+0000 7fce8ee3b740 1 -- start start 2024-04-24T09:07:12.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.962+0000 7fce8ee3b740 1 Processor -- start 2024-04-24T09:07:12.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.962+0000 7fce8ee3b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e0691a30 0x55e8e0699a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.962+0000 7fce8ee3b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0692350 0x55e8e0699f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.962+0000 7fce8ee3b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e069ca20 0x55e8e069a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.962+0000 7fce8ee3b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e8e069f810 con 0x55e8e0692350 2024-04-24T09:07:12.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.962+0000 7fce8ee3b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e8e05b0690 con 0x55e8e069ca20 2024-04-24T09:07:12.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.962+0000 7fce8ee3b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55e8e069f990 con 0x55e8e0691a30 2024-04-24T09:07:12.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.962+0000 7fce8ee3b740 10 client.0 _ll_register_callbacks cb 0x55e8e05ec650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:12.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.962+0000 7fce88c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0692350 0x55e8e0699f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.963+0000 7fce89600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e0691a30 0x55e8e0699a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.963+0000 7fce88c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0692350 0x55e8e0699f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50070/0 (socket says 172.21.15.69:50070) 2024-04-24T09:07:12.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.963+0000 7fce88c00640 1 -- 172.21.15.69:0/2423569061 learned_addr learned my addr 172.21.15.69:0/2423569061 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:12.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.963+0000 7fce8a000640 1 --2- 172.21.15.69:0/2423569061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e069ca20 0x55e8e069a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.963+0000 7fce88c00640 1 -- 172.21.15.69:0/2423569061 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e0691a30 msgr2=0x55e8e0699a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:12.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.963+0000 7fce88c00640 1 --2- 172.21.15.69:0/2423569061 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e0691a30 0x55e8e0699a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.963+0000 7fce88c00640 1 -- 172.21.15.69:0/2423569061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e069ca20 msgr2=0x55e8e069a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:12.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.963+0000 7fce88c00640 1 --2- 172.21.15.69:0/2423569061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e069ca20 0x55e8e069a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.963+0000 7fce88c00640 1 -- 172.21.15.69:0/2423569061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e05af920 con 0x55e8e0692350 2024-04-24T09:07:12.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.963+0000 7fce8a000640 1 --2- 172.21.15.69:0/2423569061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e069ca20 0x55e8e069a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:12.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.963+0000 7fce88c00640 1 --2- 172.21.15.69:0/2423569061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0692350 0x55e8e0699f80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fce70006010 tx=0x7fce70010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759b8b27972d30cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:12.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.963+0000 7fce6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:12.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.964+0000 7fce6aa00640 1 -- 172.21.15.69:0/2423569061 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70004250 con 0x55e8e0692350 2024-04-24T09:07:12.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.964+0000 7fce6aa00640 1 -- 172.21.15.69:0/2423569061 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fce700043f0 con 0x55e8e0692350 2024-04-24T09:07:12.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.964+0000 7fce8ee3b740 1 -- 172.21.15.69:0/2423569061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce700108e0 con 0x55e8e0692350 2024-04-24T09:07:12.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.964+0000 7fce6aa00640 1 -- 172.21.15.69:0/2423569061 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fce70034050 con 0x55e8e0692350 2024-04-24T09:07:12.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.964+0000 7fce8ee3b740 10 client.15696 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:12.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.964+0000 7fce8ee3b740 1 -- 172.21.15.69:0/2423569061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e8e03d37d0 con 0x55e8e0692350 2024-04-24T09:07:12.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.965+0000 7fce6aa00640 1 -- 172.21.15.69:0/2423569061 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce700108e0 con 0x55e8e0692350 2024-04-24T09:07:12.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.965+0000 7fce6aa00640 1 -- 172.21.15.69:0/2423569061 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fce7001c070 con 0x55e8e0692350 2024-04-24T09:07:12.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.965+0000 7fce8ee3b740 10 client.15696 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:12.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.965+0000 7fce8ee3b740 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:12.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.966+0000 7fce8ee3b740 20 client.15696 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:12.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.966+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/2423569061 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e8e06acb10 0x55e8e06aef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.966+0000 7fce8ee3b740 4 client.15696 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:12.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.966+0000 7fce8ee3b740 1 -- 172.21.15.69:0/2423569061 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8e0604c10 con 0x55e8e06acb10 2024-04-24T09:07:12.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.966+0000 7fce89600640 1 --2- 172.21.15.69:0/2423569061 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e8e06acb10 0x55e8e06aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.967+0000 7fce6aa00640 10 client.15696 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:12.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.967+0000 7fce89600640 1 --2- 172.21.15.69:0/2423569061 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e8e06acb10 0x55e8e06aef00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1788be44c6e4a8bd server_cookie=8be6626f54fc6e5 in_seq=0 out_seq=0 2024-04-24T09:07:12.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.967+0000 7fce6aa00640 1 -- 172.21.15.69:0/2423569061 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e8e0604c10 con 0x55e8e06acb10 2024-04-24T09:07:12.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.967+0000 7fce6aa00640 10 client.15696 handle_command_reply: tid=0 2024-04-24T09:07:12.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.968+0000 7fce8ee3b740 1 client.15696 shutdown 2024-04-24T09:07:12.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.968+0000 7fce8ee3b740 1 -- 172.21.15.69:0/2423569061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0692350 msgr2=0x55e8e0699f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.968+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/2423569061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0692350 0x55e8e0699f80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fce70006010 tx=0x7fce70010cc0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.968+0000 7fce8ee3b740 1 -- 172.21.15.69:0/2423569061 shutdown_connections 2024-04-24T09:07:12.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.968+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/2423569061 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e8e06acb10 0x55e8e06aef00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.968+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/2423569061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e069ca20 0x55e8e069a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.968+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/2423569061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0692350 0x55e8e0699f80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.968+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/2423569061 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e0691a30 0x55e8e0699a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.968+0000 7fce8ee3b740 1 -- 172.21.15.69:0/2423569061 >> 172.21.15.69:0/2423569061 conn(0x55e8e068dd50 msgr2=0x55e8e068cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:12.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.968+0000 7fce6aa00640 0 client.15696 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:12.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.968+0000 7fce8ee3b740 1 -- 172.21.15.69:0/2423569061 shutdown_connections 2024-04-24T09:07:12.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.968+0000 7fce8ee3b740 1 -- 172.21.15.69:0/2423569061 wait complete. 2024-04-24T09:07:12.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.968+0000 7fce8ee3b740 20 client.15696 trim_cache size 0 max 16384 2024-04-24T09:07:12.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.992+0000 7fce8ee3b740 1 Processor -- start 2024-04-24T09:07:12.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.992+0000 7fce8ee3b740 1 -- start start 2024-04-24T09:07:12.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.993+0000 7fce8ee3b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0691a30 0x55e8e06a8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.993+0000 7fce8ee3b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e0692350 0x55e8e0693890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.993+0000 7fce8ee3b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e069ca20 0x55e8e06b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:12.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.993+0000 7fce8ee3b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fce841b3c00 con 0x55e8e0691a30 2024-04-24T09:07:12.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.993+0000 7fce8ee3b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fce841b14d0 con 0x55e8e069ca20 2024-04-24T09:07:12.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.993+0000 7fce8ee3b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fce841af320 con 0x55e8e0692350 2024-04-24T09:07:12.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.993+0000 7fce89600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0691a30 0x55e8e06a8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.993+0000 7fce89600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0691a30 0x55e8e06a8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50084/0 (socket says 172.21.15.69:50084) 2024-04-24T09:07:12.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.993+0000 7fce89600640 1 -- 172.21.15.69:0/2102177448 learned_addr learned my addr 172.21.15.69:0/2102177448 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:12.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.993+0000 7fce88c00640 1 --2- 172.21.15.69:0/2102177448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e0692350 0x55e8e0693890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.993+0000 7fce8a000640 1 --2- 172.21.15.69:0/2102177448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e069ca20 0x55e8e06b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:12.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.994+0000 7fce89600640 1 -- 172.21.15.69:0/2102177448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e0692350 msgr2=0x55e8e0693890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:12.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.994+0000 7fce89600640 1 --2- 172.21.15.69:0/2102177448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e0692350 0x55e8e0693890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.994+0000 7fce89600640 1 -- 172.21.15.69:0/2102177448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e069ca20 msgr2=0x55e8e06b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:12.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.994+0000 7fce89600640 1 --2- 172.21.15.69:0/2102177448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e069ca20 0x55e8e06b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.994+0000 7fce89600640 1 -- 172.21.15.69:0/2102177448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e03093f0 con 0x55e8e0691a30 2024-04-24T09:07:12.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.994+0000 7fce89600640 1 --2- 172.21.15.69:0/2102177448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0691a30 0x55e8e06a8730 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fce78004650 tx=0x7fce7803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a4018876340628 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:12.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.994+0000 7fce75c00640 1 -- 172.21.15.69:0/2102177448 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78035040 con 0x55e8e0691a30 2024-04-24T09:07:12.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.994+0000 7fce75c00640 1 -- 172.21.15.69:0/2102177448 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce7805a070 con 0x55e8e0691a30 2024-04-24T09:07:12.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.994+0000 7fce75c00640 1 -- 172.21.15.69:0/2102177448 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78031aa0 con 0x55e8e0691a30 2024-04-24T09:07:12.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.995+0000 7fce8ee3b740 1 -- 172.21.15.69:0/2102177448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0691a30 msgr2=0x55e8e06a8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:12.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.995+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/2102177448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0691a30 0x55e8e06a8730 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fce78004650 tx=0x7fce7803f000 comp rx=0 tx=0).stop 2024-04-24T09:07:12.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.995+0000 7fce8ee3b740 1 -- 172.21.15.69:0/2102177448 shutdown_connections 2024-04-24T09:07:12.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.995+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/2102177448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e069ca20 0x55e8e06b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.995+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/2102177448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e0692350 0x55e8e0693890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.995+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/2102177448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e0691a30 0x55e8e06a8730 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:12.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.995+0000 7fce8ee3b740 1 -- 172.21.15.69:0/2102177448 >> 172.21.15.69:0/2102177448 conn(0x55e8e068dd50 msgr2=0x55e8e0693180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:12.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.995+0000 7fce8ee3b740 1 -- 172.21.15.69:0/2102177448 shutdown_connections 2024-04-24T09:07:12.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.995+0000 7fce8ee3b740 1 -- 172.21.15.69:0/2102177448 wait complete. 2024-04-24T09:07:12.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.996+0000 7fce8ee3b740 1 -- start start 2024-04-24T09:07:12.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.996+0000 7fce8ee3b740 1 Processor -- start 2024-04-24T09:07:12.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.996+0000 7f38f3e00640 1 -- 172.21.15.69:0/502475073 <== mgr.15465 v2:172.21.15.69:6800/2822728880 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f38fc104e00 con 0x7f38e00a3ee0 2024-04-24T09:07:12.997 INFO:teuthology.orchestra.run.smithi069.stdout:Schedule set for path / 2024-04-24T09:07:13.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce8ee3b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e0691a30 0x55e8e06a5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:13.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce8ee3b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e0692350 0x55e8e06a0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:13.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce8ee3b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e069ca20 0x55e8e06a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:13.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce8ee3b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fce841af990 con 0x55e8e069ca20 2024-04-24T09:07:13.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce8ee3b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fce841b0e00 con 0x55e8e0692350 2024-04-24T09:07:13.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce8ee3b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fce841b0060 con 0x55e8e0691a30 2024-04-24T09:07:13.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce8ee3b740 10 client.0 _ll_register_callbacks cb 0x7fce700123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:13.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce8a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e069ca20 0x55e8e06a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:13.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce8a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e069ca20 0x55e8e06a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50098/0 (socket says 172.21.15.69:50098) 2024-04-24T09:07:13.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce8a000640 1 -- 172.21.15.69:0/3474489938 learned_addr learned my addr 172.21.15.69:0/3474489938 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:13.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce89600640 1 --2- 172.21.15.69:0/3474489938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e0691a30 0x55e8e06a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:13.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce88c00640 1 --2- 172.21.15.69:0/3474489938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e0692350 0x55e8e06a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:13.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce8a000640 1 -- 172.21.15.69:0/3474489938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e0691a30 msgr2=0x55e8e06a5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:13.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce8a000640 1 --2- 172.21.15.69:0/3474489938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e0691a30 0x55e8e06a5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce8a000640 1 -- 172.21.15.69:0/3474489938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e0692350 msgr2=0x55e8e06a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:13.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce8a000640 1 --2- 172.21.15.69:0/3474489938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e0692350 0x55e8e06a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.997+0000 7fce8a000640 1 -- 172.21.15.69:0/3474489938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e02c90f0 con 0x55e8e069ca20 2024-04-24T09:07:13.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.998+0000 7fce8a000640 1 --2- 172.21.15.69:0/3474489938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e069ca20 0x55e8e06a08c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0ce080 tx=0x7fce7c09c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4c257ee3342ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:13.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.998+0000 7fce6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:13.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.998+0000 7fce6aa00640 1 -- 172.21.15.69:0/3474489938 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c01f040 con 0x55e8e069ca20 2024-04-24T09:07:13.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.998+0000 7fce6aa00640 1 -- 172.21.15.69:0/3474489938 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fce7c098eb0 con 0x55e8e069ca20 2024-04-24T09:07:13.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.998+0000 7fce8ee3b740 1 -- 172.21.15.69:0/3474489938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce7805a070 con 0x55e8e069ca20 2024-04-24T09:07:13.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.998+0000 7fce6aa00640 1 -- 172.21.15.69:0/3474489938 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fce7c09c840 con 0x55e8e069ca20 2024-04-24T09:07:13.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.998+0000 7fce8ee3b740 10 client.15702 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:13.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.998+0000 7fce8ee3b740 1 -- 172.21.15.69:0/3474489938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e8e05bc810 con 0x55e8e069ca20 2024-04-24T09:07:13.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.998+0000 7fce6aa00640 1 -- 172.21.15.69:0/3474489938 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce7805a070 con 0x55e8e069ca20 2024-04-24T09:07:13.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7fce6aa00640 1 -- 172.21.15.69:0/3474489938 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fce7c00f040 con 0x55e8e069ca20 2024-04-24T09:07:13.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7fce8ee3b740 10 client.15702 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:13.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7fce8ee3b740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:13.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7fce8ee3b740 20 client.15702 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:13.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7f3902600640 1 -- 172.21.15.69:0/502475073 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f38e00a3ee0 msgr2=0x7f38e00a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:13.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7f3902600640 1 --2- 172.21.15.69:0/502475073 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f38e00a3ee0 0x7f38e00a63a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009bd0 tx=0x7f38ec005f90 comp rx=0 tx=0).stop 2024-04-24T09:07:13.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7f3902600640 1 -- 172.21.15.69:0/502475073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38fc109520 msgr2=0x7f38fc1a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:13.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7f3902600640 1 --2- 172.21.15.69:0/502475073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38fc109520 0x7f38fc1a0bc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f38e8005ab0 tx=0x7f38e8003b90 comp rx=0 tx=0).stop 2024-04-24T09:07:13.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/3474489938 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e8e0713260 0x55e8e0715680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:13.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7fce8ee3b740 4 client.15702 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:07:13.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7fce8ee3b740 1 -- 172.21.15.69:0/3474489938 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e8e05fea80 con 0x55e8e0713260 2024-04-24T09:07:13.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7f3902600640 1 -- 172.21.15.69:0/502475073 shutdown_connections 2024-04-24T09:07:13.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7f3902600640 1 --2- 172.21.15.69:0/502475073 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f38e00a3ee0 0x7f38e00a63a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7f3902600640 1 --2- 172.21.15.69:0/502475073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38fc19a7f0 0x7f38fc19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7f3902600640 1 --2- 172.21.15.69:0/502475073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38fc109520 0x7f38fc1a0bc0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7f3902600640 1 --2- 172.21.15.69:0/502475073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38fc104450 0x7f38fc1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7f3902600640 1 -- 172.21.15.69:0/502475073 >> 172.21.15.69:0/502475073 conn(0x7f38fc0ffcf0 msgr2=0x7f38fc100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:13.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7f3902600640 1 -- 172.21.15.69:0/502475073 shutdown_connections 2024-04-24T09:07:13.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7f3902600640 1 -- 172.21.15.69:0/502475073 wait complete. 2024-04-24T09:07:13.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:12.999+0000 7fce89600640 1 --2- 172.21.15.69:0/3474489938 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e8e0713260 0x55e8e0715680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:13.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.000+0000 7fce89600640 1 --2- 172.21.15.69:0/3474489938 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e8e0713260 0x55e8e0715680 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde2d0b9f46253bb server_cookie=5bee283d71707f2c in_seq=0 out_seq=0 2024-04-24T09:07:13.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.000+0000 7fce6aa00640 10 client.15702 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:13.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.000+0000 7fce6aa00640 1 -- 172.21.15.69:0/3474489938 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e8e05fea80 con 0x55e8e0713260 2024-04-24T09:07:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.000+0000 7fce6aa00640 10 client.15702 handle_command_reply: tid=0 2024-04-24T09:07:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.000+0000 7fce8ee3b740 1 client.15702 shutdown 2024-04-24T09:07:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.000+0000 7fce8ee3b740 1 -- 172.21.15.69:0/3474489938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e069ca20 msgr2=0x55e8e06a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.000+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/3474489938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e069ca20 0x55e8e06a08c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0ce080 tx=0x7fce7c09c380 comp rx=0 tx=0).stop 2024-04-24T09:07:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.002+0000 7fce8ee3b740 1 -- 172.21.15.69:0/3474489938 shutdown_connections 2024-04-24T09:07:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.002+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/3474489938 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e8e0713260 0x55e8e0715680 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.002+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/3474489938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8e069ca20 0x55e8e06a08c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.002+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/3474489938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8e0692350 0x55e8e06a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.002+0000 7fce8ee3b740 1 --2- 172.21.15.69:0/3474489938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e8e0691a30 0x55e8e06a5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.002+0000 7fce6aa00640 0 client.15702 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.002+0000 7fce8ee3b740 1 -- 172.21.15.69:0/3474489938 >> 172.21.15.69:0/3474489938 conn(0x55e8e068dd50 msgr2=0x55e8e06939e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.002+0000 7fce8ee3b740 1 -- 172.21.15.69:0/3474489938 shutdown_connections 2024-04-24T09:07:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.002+0000 7fce8ee3b740 1 -- 172.21.15.69:0/3474489938 wait complete. 2024-04-24T09:07:13.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.002+0000 7fce8ee3b740 20 client.15702 trim_cache size 0 max 16384 2024-04-24T09:07:13.007 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:07:13.007 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:07:13.007 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-04-24T09:07:13.007 INFO:teuthology.orchestra.run.smithi069.stdout: "6bc63386-c795-4e26-952f-9fb62dd68b17": { 2024-04-24T09:07:13.007 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-04-24T09:07:13.007 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "6bc63386-c795-4e26-952f-9fb62dd68b17", 2024-04-24T09:07:13.007 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-04-24T09:07:13.007 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:07:13.007 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:07:13.007 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:07:13.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.002+0000 7fce8aa00640 1 -- 172.21.15.69:0/3917671013 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fce6c0a3de0 msgr2=0x7fce6c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:13.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.002+0000 7fce8aa00640 1 --2- 172.21.15.69:0/3917671013 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fce6c0a3de0 0x7fce6c0a62a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55e8e0692ff0 tx=0x7fce78005a20 comp rx=0 tx=0).stop 2024-04-24T09:07:13.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.002+0000 7fce8aa00640 1 -- 172.21.15.69:0/3917671013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce84074d80 msgr2=0x7fce841a6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:13.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.002+0000 7fce8aa00640 1 --2- 172.21.15.69:0/3917671013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce84074d80 0x7fce841a6870 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00e940 tx=0x7fce7c00ee00 comp rx=0 tx=0).stop 2024-04-24T09:07:13.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.003+0000 7fce8aa00640 1 -- 172.21.15.69:0/3917671013 shutdown_connections 2024-04-24T09:07:13.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.003+0000 7fce8aa00640 1 --2- 172.21.15.69:0/3917671013 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fce6c0a3de0 0x7fce6c0a62a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.003+0000 7fce8aa00640 1 --2- 172.21.15.69:0/3917671013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce84074d80 0x7fce841a6870 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.003+0000 7fce8aa00640 1 --2- 172.21.15.69:0/3917671013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce841071e0 0x7fce84074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.003+0000 7fce8aa00640 1 --2- 172.21.15.69:0/3917671013 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce84102110 0x7fce84074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.003+0000 7fce8aa00640 1 -- 172.21.15.69:0/3917671013 >> 172.21.15.69:0/3917671013 conn(0x7fce840fd9d0 msgr2=0x7fce840ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:13.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.003+0000 7fce8aa00640 1 -- 172.21.15.69:0/3917671013 shutdown_connections 2024-04-24T09:07:13.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.003+0000 7fce8aa00640 1 -- 172.21.15.69:0/3917671013 wait complete. 2024-04-24T09:07:13.017 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-24T09:07:13.022 INFO:tasks.cephfs.filesystem:scrub status for tag:6bc63386-c795-4e26-952f-9fb62dd68b17 - {'path': '/', 'tag': '6bc63386-c795-4e26-952f-9fb62dd68b17', 'options': 'recursive,force'} 2024-04-24T09:07:13.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.192+0000 7ff9a7200640 1 -- 172.21.15.69:0/842636801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a810e4d0 msgr2=0x7ff9a810e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:13.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.193+0000 7ff9a7200640 1 --2- 172.21.15.69:0/842636801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a810e4d0 0x7ff9a810e8b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff998009930 tx=0x7ff99802a420 comp rx=0 tx=0).stop 2024-04-24T09:07:13.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.193+0000 7ff9a7200640 1 -- 172.21.15.69:0/842636801 shutdown_connections 2024-04-24T09:07:13.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.193+0000 7ff9a7200640 1 --2- 172.21.15.69:0/842636801 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9a8108ec0 0x7ff9a8115300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.193+0000 7ff9a7200640 1 --2- 172.21.15.69:0/842636801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a8108540 0x7ff9a8108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.193+0000 7ff9a7200640 1 --2- 172.21.15.69:0/842636801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a810e4d0 0x7ff9a810e8b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.193+0000 7ff9a7200640 1 -- 172.21.15.69:0/842636801 >> 172.21.15.69:0/842636801 conn(0x7ff9a8077de0 msgr2=0x7ff9a8106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:13.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.193+0000 7ff9a7200640 1 -- 172.21.15.69:0/842636801 shutdown_connections 2024-04-24T09:07:13.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.193+0000 7ff9a7200640 1 -- 172.21.15.69:0/842636801 wait complete. 2024-04-24T09:07:13.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.194+0000 7ff9a7200640 1 Processor -- start 2024-04-24T09:07:13.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.194+0000 7ff9a7200640 1 -- start start 2024-04-24T09:07:13.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.194+0000 7ff9a7200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a8108540 0x7ff9a81a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:13.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.194+0000 7ff9a7200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a8108ec0 0x7ff9a81a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:13.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.194+0000 7ff9a7200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9a810e4d0 0x7ff9a819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:13.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.194+0000 7ff9a7200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a8118700 con 0x7ff9a8108540 2024-04-24T09:07:13.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.194+0000 7ff9a7200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a8118580 con 0x7ff9a8108ec0 2024-04-24T09:07:13.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.194+0000 7ff9a7200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a8118880 con 0x7ff9a810e4d0 2024-04-24T09:07:13.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.194+0000 7ff9a5e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a8108540 0x7ff9a81a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:13.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.194+0000 7ff9a5e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a8108540 0x7ff9a81a48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50122/0 (socket says 172.21.15.69:50122) 2024-04-24T09:07:13.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.194+0000 7ff9a5e00640 1 -- 172.21.15.69:0/1170514571 learned_addr learned my addr 172.21.15.69:0/1170514571 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:13.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.195+0000 7ff9a6800640 1 --2- 172.21.15.69:0/1170514571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9a810e4d0 0x7ff9a819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:13.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.195+0000 7ff9a5e00640 1 -- 172.21.15.69:0/1170514571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9a810e4d0 msgr2=0x7ff9a819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:13.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.195+0000 7ff9a5e00640 1 --2- 172.21.15.69:0/1170514571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9a810e4d0 0x7ff9a819ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.195+0000 7ff9a5e00640 1 -- 172.21.15.69:0/1170514571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a8108ec0 msgr2=0x7ff9a81a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:13.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.195+0000 7ff9a5e00640 1 --2- 172.21.15.69:0/1170514571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a8108ec0 0x7ff9a81a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.195+0000 7ff9a5e00640 1 -- 172.21.15.69:0/1170514571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a819f1e0 con 0x7ff9a8108540 2024-04-24T09:07:13.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.195+0000 7ff9a5e00640 1 --2- 172.21.15.69:0/1170514571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a8108540 0x7ff9a81a48b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff998005ab0 tx=0x7ff998003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:13.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.195+0000 7ff99ea00640 1 -- 172.21.15.69:0/1170514571 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99803f070 con 0x7ff9a8108540 2024-04-24T09:07:13.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.195+0000 7ff99ea00640 1 -- 172.21.15.69:0/1170514571 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff998031d60 con 0x7ff9a8108540 2024-04-24T09:07:13.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.195+0000 7ff9a7200640 1 -- 172.21.15.69:0/1170514571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a819f470 con 0x7ff9a8108540 2024-04-24T09:07:13.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.195+0000 7ff99ea00640 1 -- 172.21.15.69:0/1170514571 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998035030 con 0x7ff9a8108540 2024-04-24T09:07:13.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.196+0000 7ff9a7200640 1 -- 172.21.15.69:0/1170514571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a81ab770 con 0x7ff9a8108540 2024-04-24T09:07:13.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.197+0000 7ff9a7200640 1 -- 172.21.15.69:0/1170514571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a80505e0 con 0x7ff9a8108540 2024-04-24T09:07:13.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.198+0000 7ff99ea00640 1 -- 172.21.15.69:0/1170514571 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7ff998042070 con 0x7ff9a8108540 2024-04-24T09:07:13.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.200+0000 7ff99ea00640 1 --2- 172.21.15.69:0/1170514571 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff98c0a3e20 0x7ff98c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:13.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.200+0000 7ff99ea00640 1 -- 172.21.15.69:0/1170514571 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7ff9980f14e0 con 0x7ff9a8108540 2024-04-24T09:07:13.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.200+0000 7ff9a5400640 1 --2- 172.21.15.69:0/1170514571 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff98c0a3e20 0x7ff98c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:13.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.201+0000 7ff9a5400640 1 --2- 172.21.15.69:0/1170514571 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff98c0a3e20 0x7ff98c0a62e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff994005fd0 tx=0x7ff994005c70 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:13.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.205+0000 7ff99ea00640 1 -- 172.21.15.69:0/1170514571 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff9980bd6d0 con 0x7ff9a8108540 2024-04-24T09:07:13.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:13 smithi069 ceph-mon[28170]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:07:13.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:13 smithi069 ceph-mon[28170]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 0 B/s wr, 21 op/s 2024-04-24T09:07:13.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:13 smithi069 ceph-mon[28170]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:07:13.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:13 smithi069 ceph-mon[28170]: Updating smithi069:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T09:07:13.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:13 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2315142519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:07:13.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:13 smithi069 ceph-mon[28170]: Updating smithi081:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T09:07:13.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:13 smithi069 ceph-mon[28170]: Updating smithi130:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T09:07:13.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:13 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:13.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:13 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:13.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.439+0000 7ff9a7200640 1 -- 172.21.15.69:0/1170514571 --> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7ff9a8062440 con 0x7ff98c0a3e20 2024-04-24T09:07:13.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:13 smithi081 ceph-mon[30961]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:07:13.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:13 smithi081 ceph-mon[30961]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 0 B/s wr, 21 op/s 2024-04-24T09:07:13.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:13 smithi081 ceph-mon[30961]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:07:13.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:13 smithi081 ceph-mon[30961]: Updating smithi069:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T09:07:13.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:13 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2315142519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:07:13.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:13 smithi081 ceph-mon[30961]: Updating smithi081:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T09:07:13.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:13 smithi081 ceph-mon[30961]: Updating smithi130:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T09:07:13.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:13 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:13.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:13 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:13.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.586+0000 7ff99ea00640 1 -- 172.21.15.69:0/1170514571 <== mgr.15465 v2:172.21.15.69:6800/2822728880 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7ff9a8062440 con 0x7ff98c0a3e20 2024-04-24T09:07:13.585 INFO:teuthology.orchestra.run.smithi069.stdout:Retention added to path / 2024-04-24T09:07:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.588+0000 7ff9a7200640 1 -- 172.21.15.69:0/1170514571 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff98c0a3e20 msgr2=0x7ff98c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.588+0000 7ff9a7200640 1 --2- 172.21.15.69:0/1170514571 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff98c0a3e20 0x7ff98c0a62e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff994005fd0 tx=0x7ff994005c70 comp rx=0 tx=0).stop 2024-04-24T09:07:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.588+0000 7ff9a7200640 1 -- 172.21.15.69:0/1170514571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a8108540 msgr2=0x7ff9a81a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.588+0000 7ff9a7200640 1 --2- 172.21.15.69:0/1170514571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a8108540 0x7ff9a81a48b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff998005ab0 tx=0x7ff998003cd0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.588+0000 7ff9a7200640 1 -- 172.21.15.69:0/1170514571 shutdown_connections 2024-04-24T09:07:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.588+0000 7ff9a7200640 1 --2- 172.21.15.69:0/1170514571 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff98c0a3e20 0x7ff98c0a62e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.588+0000 7ff9a7200640 1 --2- 172.21.15.69:0/1170514571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff9a810e4d0 0x7ff9a819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.588+0000 7ff9a7200640 1 --2- 172.21.15.69:0/1170514571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a8108ec0 0x7ff9a81a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.588+0000 7ff9a7200640 1 --2- 172.21.15.69:0/1170514571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9a8108540 0x7ff9a81a48b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.588+0000 7ff9a7200640 1 -- 172.21.15.69:0/1170514571 >> 172.21.15.69:0/1170514571 conn(0x7ff9a8077de0 msgr2=0x7ff9a80784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:13.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.588+0000 7ff9a7200640 1 -- 172.21.15.69:0/1170514571 shutdown_connections 2024-04-24T09:07:13.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.588+0000 7ff9a7200640 1 -- 172.21.15.69:0/1170514571 wait complete. 2024-04-24T09:07:13.604 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-24T09:07:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:13 smithi130 ceph-mon[31553]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:07:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:13 smithi130 ceph-mon[31553]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 0 B/s wr, 21 op/s 2024-04-24T09:07:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:13 smithi130 ceph-mon[31553]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:07:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:13 smithi130 ceph-mon[31553]: Updating smithi069:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T09:07:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:13 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2315142519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:07:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:13 smithi130 ceph-mon[31553]: Updating smithi081:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T09:07:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:13 smithi130 ceph-mon[31553]: Updating smithi130:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T09:07:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:13 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:13 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:13.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.785+0000 7f352e000640 1 -- 172.21.15.69:0/2529523845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3528107140 msgr2=0x7f352810ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:13.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.785+0000 7f352e000640 1 --2- 172.21.15.69:0/2529523845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3528107140 0x7f352810ed10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3520009900 tx=0x7f352002a410 comp rx=0 tx=0).stop 2024-04-24T09:07:13.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.786+0000 7f352e000640 1 -- 172.21.15.69:0/2529523845 shutdown_connections 2024-04-24T09:07:13.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.786+0000 7f352e000640 1 --2- 172.21.15.69:0/2529523845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3528107140 0x7f352810ed10 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.786+0000 7f352e000640 1 --2- 172.21.15.69:0/2529523845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3528102a20 0x7f3528106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.786+0000 7f352e000640 1 --2- 172.21.15.69:0/2529523845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3528102070 0x7f3528102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.786+0000 7f352e000640 1 -- 172.21.15.69:0/2529523845 >> 172.21.15.69:0/2529523845 conn(0x7f35280fd950 msgr2=0x7f35280ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:13.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.786+0000 7f352e000640 1 -- 172.21.15.69:0/2529523845 shutdown_connections 2024-04-24T09:07:13.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.786+0000 7f352e000640 1 -- 172.21.15.69:0/2529523845 wait complete. 2024-04-24T09:07:13.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.787+0000 7f352e000640 1 Processor -- start 2024-04-24T09:07:13.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.787+0000 7f352e000640 1 -- start start 2024-04-24T09:07:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.787+0000 7f352e000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3528102070 0x7f35281a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.787+0000 7f352e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3528107140 0x7f35281a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.787+0000 7f352e000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f352819a680 0x7f352819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.787+0000 7f352e000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3528112100 con 0x7f3528107140 2024-04-24T09:07:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.787+0000 7f352e000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3528111f80 con 0x7f352819a680 2024-04-24T09:07:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.787+0000 7f352e000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3528112280 con 0x7f3528102070 2024-04-24T09:07:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.788+0000 7f3527e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3528107140 0x7f35281a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.788+0000 7f3527e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3528107140 0x7f35281a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50144/0 (socket says 172.21.15.69:50144) 2024-04-24T09:07:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.788+0000 7f3527e00640 1 -- 172.21.15.69:0/2985074531 learned_addr learned my addr 172.21.15.69:0/2985074531 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.788+0000 7f352cc00640 1 --2- 172.21.15.69:0/2985074531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3528102070 0x7f35281a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.788+0000 7f352d600640 1 --2- 172.21.15.69:0/2985074531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f352819a680 0x7f352819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:13.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.788+0000 7f3527e00640 1 -- 172.21.15.69:0/2985074531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3528102070 msgr2=0x7f35281a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:13.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.788+0000 7f3527e00640 1 --2- 172.21.15.69:0/2985074531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3528102070 0x7f35281a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.788+0000 7f3527e00640 1 -- 172.21.15.69:0/2985074531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f352819a680 msgr2=0x7f352819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:13.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.788+0000 7f3527e00640 1 --2- 172.21.15.69:0/2985074531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f352819a680 0x7f352819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:13.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.788+0000 7f3527e00640 1 -- 172.21.15.69:0/2985074531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352819b280 con 0x7f3528107140 2024-04-24T09:07:13.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.789+0000 7f3527e00640 1 --2- 172.21.15.69:0/2985074531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3528107140 0x7f35281a0b00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f351000d770 tx=0x7f351000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:13.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.789+0000 7f3525600640 1 -- 172.21.15.69:0/2985074531 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510017070 con 0x7f3528107140 2024-04-24T09:07:13.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.789+0000 7f3525600640 1 -- 172.21.15.69:0/2985074531 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3510009c30 con 0x7f3528107140 2024-04-24T09:07:13.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.789+0000 7f3525600640 1 -- 172.21.15.69:0/2985074531 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351000f620 con 0x7f3528107140 2024-04-24T09:07:13.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.789+0000 7f352e000640 1 -- 172.21.15.69:0/2985074531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35281a7360 con 0x7f3528107140 2024-04-24T09:07:13.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.789+0000 7f352e000640 1 -- 172.21.15.69:0/2985074531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35281a7710 con 0x7f3528107140 2024-04-24T09:07:13.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.790+0000 7f352e000640 1 -- 172.21.15.69:0/2985074531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3528103540 con 0x7f3528107140 2024-04-24T09:07:13.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.792+0000 7f3525600640 1 -- 172.21.15.69:0/2985074531 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f351001a070 con 0x7f3528107140 2024-04-24T09:07:13.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.793+0000 7f3525600640 1 --2- 172.21.15.69:0/2985074531 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f350c0a3da0 0x7f350c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:13.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.793+0000 7f3525600640 1 -- 172.21.15.69:0/2985074531 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f35100d08b0 con 0x7f3528107140 2024-04-24T09:07:13.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.794+0000 7f352cc00640 1 --2- 172.21.15.69:0/2985074531 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f350c0a3da0 0x7f350c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:13.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.794+0000 7f352cc00640 1 --2- 172.21.15.69:0/2985074531 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f350c0a3da0 0x7f350c0a6260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3518009f20 tx=0x7f35180094f0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:13.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:13.798+0000 7f3525600640 1 -- 172.21.15.69:0/2985074531 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3510009de0 con 0x7f3528107140 2024-04-24T09:07:14.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.033+0000 7f352e000640 1 -- 172.21.15.69:0/2985074531 --> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f3528102a90 con 0x7f350c0a3da0 2024-04-24T09:07:14.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.046+0000 7f3525600640 1 -- 172.21.15.69:0/2985074531 <== mgr.15465 v2:172.21.15.69:6800/2822728880 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f3528102a90 con 0x7f350c0a3da0 2024-04-24T09:07:14.045 INFO:teuthology.orchestra.run.smithi069.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-24T00:00:00", "created": "2024-04-24T09:07:12", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-24T09:07:14.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.048+0000 7f352e000640 1 -- 172.21.15.69:0/2985074531 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f350c0a3da0 msgr2=0x7f350c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:14.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.048+0000 7f352e000640 1 --2- 172.21.15.69:0/2985074531 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f350c0a3da0 0x7f350c0a6260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3518009f20 tx=0x7f35180094f0 comp rx=0 tx=0).stop 2024-04-24T09:07:14.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.048+0000 7f352e000640 1 -- 172.21.15.69:0/2985074531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3528107140 msgr2=0x7f35281a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:14.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.048+0000 7f352e000640 1 --2- 172.21.15.69:0/2985074531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3528107140 0x7f35281a0b00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f351000d770 tx=0x7f351000dc30 comp rx=0 tx=0).stop 2024-04-24T09:07:14.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.048+0000 7f352e000640 1 -- 172.21.15.69:0/2985074531 shutdown_connections 2024-04-24T09:07:14.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.048+0000 7f352e000640 1 --2- 172.21.15.69:0/2985074531 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f350c0a3da0 0x7f350c0a6260 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:14.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.048+0000 7f352e000640 1 --2- 172.21.15.69:0/2985074531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f352819a680 0x7f352819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:14.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.048+0000 7f352e000640 1 --2- 172.21.15.69:0/2985074531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3528107140 0x7f35281a0b00 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:14.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.048+0000 7f352e000640 1 --2- 172.21.15.69:0/2985074531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3528102070 0x7f35281a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:14.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.048+0000 7f352e000640 1 -- 172.21.15.69:0/2985074531 >> 172.21.15.69:0/2985074531 conn(0x7f35280fd950 msgr2=0x7f35280ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:14.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.048+0000 7f352e000640 1 -- 172.21.15.69:0/2985074531 shutdown_connections 2024-04-24T09:07:14.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.048+0000 7f352e000640 1 -- 172.21.15.69:0/2985074531 wait complete. 2024-04-24T09:07:14.064 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-24T09:07:14.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.242+0000 7f783d200640 1 -- 172.21.15.69:0/2130754331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7838108fb0 msgr2=0x7f78381154c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:14.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.242+0000 7f783d200640 1 --2- 172.21.15.69:0/2130754331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7838108fb0 0x7f78381154c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7830009cd0 tx=0x7f783002a8f0 comp rx=0 tx=0).stop 2024-04-24T09:07:14.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.242+0000 7f783d200640 1 -- 172.21.15.69:0/2130754331 shutdown_connections 2024-04-24T09:07:14.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.242+0000 7f783d200640 1 --2- 172.21.15.69:0/2130754331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7838108fb0 0x7f78381154c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:14.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.242+0000 7f783d200640 1 --2- 172.21.15.69:0/2130754331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78381085f0 0x7f7838108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:14.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.242+0000 7f783d200640 1 --2- 172.21.15.69:0/2130754331 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f783810e690 0x7f783810ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:14.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.242+0000 7f783d200640 1 -- 172.21.15.69:0/2130754331 >> 172.21.15.69:0/2130754331 conn(0x7f7838076590 msgr2=0x7f78380789b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:14.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.242+0000 7f783d200640 1 -- 172.21.15.69:0/2130754331 shutdown_connections 2024-04-24T09:07:14.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.242+0000 7f783d200640 1 -- 172.21.15.69:0/2130754331 wait complete. 2024-04-24T09:07:14.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.243+0000 7f783d200640 1 Processor -- start 2024-04-24T09:07:14.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.243+0000 7f783d200640 1 -- start start 2024-04-24T09:07:14.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.243+0000 7f783d200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78381085f0 0x7f78381a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:14.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.243+0000 7f783d200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7838108fb0 0x7f78381a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:14.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.243+0000 7f783d200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f783810e690 0x7f783819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:14.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.243+0000 7f783d200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7838118880 con 0x7f783810e690 2024-04-24T09:07:14.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.243+0000 7f783d200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7838118700 con 0x7f7838108fb0 2024-04-24T09:07:14.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.243+0000 7f783d200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7838118a00 con 0x7f78381085f0 2024-04-24T09:07:14.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.244+0000 7f7836800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f783810e690 0x7f783819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:14.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.244+0000 7f7836800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f783810e690 0x7f783819eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50162/0 (socket says 172.21.15.69:50162) 2024-04-24T09:07:14.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.244+0000 7f7836800640 1 -- 172.21.15.69:0/510380112 learned_addr learned my addr 172.21.15.69:0/510380112 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:14.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.244+0000 7f7835400640 1 --2- 172.21.15.69:0/510380112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7838108fb0 0x7f78381a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:14.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.244+0000 7f7835e00640 1 --2- 172.21.15.69:0/510380112 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78381085f0 0x7f78381a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:14.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.244+0000 7f7836800640 1 -- 172.21.15.69:0/510380112 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78381085f0 msgr2=0x7f78381a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:14.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.244+0000 7f7836800640 1 --2- 172.21.15.69:0/510380112 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78381085f0 0x7f78381a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:14.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.244+0000 7f7836800640 1 -- 172.21.15.69:0/510380112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7838108fb0 msgr2=0x7f78381a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:14.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.244+0000 7f7836800640 1 --2- 172.21.15.69:0/510380112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7838108fb0 0x7f78381a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:14.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.244+0000 7f7836800640 1 -- 172.21.15.69:0/510380112 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783819f260 con 0x7f783810e690 2024-04-24T09:07:14.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.245+0000 7f7836800640 1 --2- 172.21.15.69:0/510380112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f783810e690 0x7f783819eae0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7830005cc0 tx=0x7f78300093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:14.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.245+0000 7f782ea00640 1 -- 172.21.15.69:0/510380112 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830004040 con 0x7f783810e690 2024-04-24T09:07:14.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.245+0000 7f782ea00640 1 -- 172.21.15.69:0/510380112 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7830040070 con 0x7f783810e690 2024-04-24T09:07:14.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.245+0000 7f783d200640 1 -- 172.21.15.69:0/510380112 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783819f4f0 con 0x7f783810e690 2024-04-24T09:07:14.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.245+0000 7f782ea00640 1 -- 172.21.15.69:0/510380112 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830045b10 con 0x7f783810e690 2024-04-24T09:07:14.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.245+0000 7f783d200640 1 -- 172.21.15.69:0/510380112 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78381ab7a0 con 0x7f783810e690 2024-04-24T09:07:14.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.246+0000 7f783d200640 1 -- 172.21.15.69:0/510380112 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f783804fd90 con 0x7f783810e690 2024-04-24T09:07:14.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.248+0000 7f782ea00640 1 -- 172.21.15.69:0/510380112 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f7830045cb0 con 0x7f783810e690 2024-04-24T09:07:14.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.249+0000 7f782ea00640 1 --2- 172.21.15.69:0/510380112 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f781c0a3e60 0x7f781c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:14.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.250+0000 7f782ea00640 1 -- 172.21.15.69:0/510380112 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f78300f1d10 con 0x7f783810e690 2024-04-24T09:07:14.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.250+0000 7f7835e00640 1 --2- 172.21.15.69:0/510380112 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f781c0a3e60 0x7f781c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:14.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.251+0000 7f7835e00640 1 --2- 172.21.15.69:0/510380112 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f781c0a3e60 0x7f781c0a6320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7828005db0 tx=0x7f7828005d40 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:14.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.254+0000 7f782ea00640 1 -- 172.21.15.69:0/510380112 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78300be130 con 0x7f783810e690 2024-04-24T09:07:14.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.486+0000 7f783d200640 1 -- 172.21.15.69:0/510380112 --> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7838078260 con 0x7f781c0a3e60 2024-04-24T09:07:14.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.502+0000 7f782ea00640 1 -- 172.21.15.69:0/510380112 <== mgr.15465 v2:172.21.15.69:6800/2822728880 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7838078260 con 0x7f781c0a3e60 2024-04-24T09:07:14.502 INFO:teuthology.orchestra.run.smithi069.stdout:/ 1m 6m3h 2024-04-24T09:07:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.505+0000 7f783d200640 1 -- 172.21.15.69:0/510380112 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f781c0a3e60 msgr2=0x7f781c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.505+0000 7f783d200640 1 --2- 172.21.15.69:0/510380112 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f781c0a3e60 0x7f781c0a6320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7828005db0 tx=0x7f7828005d40 comp rx=0 tx=0).stop 2024-04-24T09:07:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.505+0000 7f783d200640 1 -- 172.21.15.69:0/510380112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f783810e690 msgr2=0x7f783819eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.505+0000 7f783d200640 1 --2- 172.21.15.69:0/510380112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f783810e690 0x7f783819eae0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7830005cc0 tx=0x7f78300093d0 comp rx=0 tx=0).stop 2024-04-24T09:07:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.506+0000 7f783d200640 1 -- 172.21.15.69:0/510380112 shutdown_connections 2024-04-24T09:07:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.506+0000 7f783d200640 1 --2- 172.21.15.69:0/510380112 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f781c0a3e60 0x7f781c0a6320 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.506+0000 7f783d200640 1 --2- 172.21.15.69:0/510380112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f783810e690 0x7f783819eae0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.506+0000 7f783d200640 1 --2- 172.21.15.69:0/510380112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7838108fb0 0x7f78381a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.506+0000 7f783d200640 1 --2- 172.21.15.69:0/510380112 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78381085f0 0x7f78381a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.506+0000 7f783d200640 1 -- 172.21.15.69:0/510380112 >> 172.21.15.69:0/510380112 conn(0x7f7838076590 msgr2=0x7f7838077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.506+0000 7f783d200640 1 -- 172.21.15.69:0/510380112 shutdown_connections 2024-04-24T09:07:14.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:14.506+0000 7f783d200640 1 -- 172.21.15.69:0/510380112 wait complete. 2024-04-24T09:07:14.524 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-24T09:07:14.608 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-24T09:07:14.618 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-24T09:07:14.689 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-04-24T09:07:14.689 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-24T09:07:14.697 INFO:teuthology.run_tasks:Running task workunit... 2024-04-24T09:07:14.721 INFO:tasks.workunit:Pulling workunits from ref 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T09:07:14.721 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-24T09:07:14.721 INFO:tasks.workunit:timeout=3h 2024-04-24T09:07:14.721 INFO:tasks.workunit:cleanup=True 2024-04-24T09:07:14.722 DEBUG:teuthology.orchestra.run.smithi069:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-24T09:07:14.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:14 smithi130 ceph-mon[31553]: scrub queued for path: / 2024-04-24T09:07:14.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:14 smithi130 ceph-mon[31553]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:14.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:14 smithi130 ceph-mon[31553]: scrub summary: active paths [/] 2024-04-24T09:07:14.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:14 smithi130 ceph-mon[31553]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:14.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:14 smithi130 ceph-mon[31553]: from='client.15666 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:07:14.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:14 smithi130 ceph-mon[31553]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:07:14.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:14 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:14 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:14 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:14 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:14 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:14 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:07:14.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:14 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:07:14.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:14 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:07:14.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:14 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:14 smithi069 ceph-mon[28170]: scrub queued for path: / 2024-04-24T09:07:14.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:14 smithi069 ceph-mon[28170]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:14.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:14 smithi069 ceph-mon[28170]: scrub summary: active paths [/] 2024-04-24T09:07:14.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:14 smithi069 ceph-mon[28170]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:14.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:14 smithi069 ceph-mon[28170]: from='client.15666 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:07:14.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:14 smithi069 ceph-mon[28170]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:07:14.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:14 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:14 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:14 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:14 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:14 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:14 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:07:14.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:14 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:07:14.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:14 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:07:14.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:14 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.768 INFO:teuthology.orchestra.run.smithi069.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-24T09:07:14.768 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-24T09:07:14.768 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 72h/114d Inode: 1 Links: 2 2024-04-24T09:07:14.769 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T09:07:14.769 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-24T09:07:14.769 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-24 09:04:35.636429783 +0000 2024-04-24T09:07:14.769 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-24 09:04:35.636429783 +0000 2024-04-24T09:07:14.769 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-24 09:07:02.091159408 +0000 2024-04-24T09:07:14.769 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: - 2024-04-24T09:07:14.769 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-24T09:07:14.769 DEBUG:teuthology.orchestra.run.smithi069:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-24T09:07:14.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:14 smithi081 ceph-mon[30961]: scrub queued for path: / 2024-04-24T09:07:14.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:14 smithi081 ceph-mon[30961]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:14.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:14 smithi081 ceph-mon[30961]: scrub summary: active paths [/] 2024-04-24T09:07:14.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:14 smithi081 ceph-mon[30961]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:14.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:14 smithi081 ceph-mon[30961]: from='client.15666 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:07:14.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:14 smithi081 ceph-mon[30961]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:07:14.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:14 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:14 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:14 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:14 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:14 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:14 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:07:14.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:14 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:07:14.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:14 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:07:14.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:14 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:07:14.898 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T09:07:14.955 INFO:tasks.workunit.client.0.smithi069.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-24T09:07:15.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:15 smithi081 ceph-mon[30961]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-24T09:07:15.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:15 smithi081 ceph-mon[30961]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:07:15.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:15 smithi081 ceph-mon[30961]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:07:15.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:15 smithi069 ceph-mon[28170]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-24T09:07:15.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:15 smithi069 ceph-mon[28170]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:07:15.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:15 smithi069 ceph-mon[28170]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:07:15.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:15 smithi130 ceph-mon[31553]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-24T09:07:15.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:15 smithi130 ceph-mon[31553]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:07:15.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:15 smithi130 ceph-mon[31553]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:07:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:17 smithi081 ceph-mon[30961]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-24T09:07:17.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:17 smithi069 ceph-mon[28170]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-24T09:07:17.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:17 smithi130 ceph-mon[31553]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-24T09:07:19.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:19 smithi081 ceph-mon[30961]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-24T09:07:19.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:19 smithi069 ceph-mon[28170]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-24T09:07:19.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:19 smithi130 ceph-mon[31553]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-24T09:07:21.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:21 smithi081 ceph-mon[30961]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-24T09:07:21.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:21 smithi069 ceph-mon[28170]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-24T09:07:21.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:21 smithi130 ceph-mon[31553]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-24T09:07:23.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:23 smithi081 ceph-mon[30961]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-24T09:07:23.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:23 smithi069 ceph-mon[28170]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-24T09:07:23.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:23 smithi130 ceph-mon[31553]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-24T09:07:25.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:25 smithi081 ceph-mon[30961]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-24T09:07:25.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:25 smithi069 ceph-mon[28170]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-24T09:07:25.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:25 smithi130 ceph-mon[31553]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-24T09:07:27.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:27 smithi081 ceph-mon[30961]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-24T09:07:27.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:27 smithi081 ceph-mon[30961]: scrub summary: idle 2024-04-24T09:07:27.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:27 smithi069 ceph-mon[28170]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-24T09:07:27.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:27 smithi069 ceph-mon[28170]: scrub summary: idle 2024-04-24T09:07:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.944+0000 7f377c600640 1 -- 172.21.15.69:0/2465411678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3774109540 msgr2=0x7f3774111110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.944+0000 7f377c600640 1 --2- 172.21.15.69:0/2465411678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3774109540 0x7f3774111110 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f376c009900 tx=0x7f376c02a410 comp rx=0 tx=0).stop 2024-04-24T09:07:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.944+0000 7f377c600640 1 -- 172.21.15.69:0/2465411678 shutdown_connections 2024-04-24T09:07:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.944+0000 7f377c600640 1 --2- 172.21.15.69:0/2465411678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3774109540 0x7f3774111110 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.944+0000 7f377c600640 1 --2- 172.21.15.69:0/2465411678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3774104e20 0x7f3774108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.944+0000 7f377c600640 1 --2- 172.21.15.69:0/2465411678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3774104470 0x7f3774104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.944+0000 7f377c600640 1 -- 172.21.15.69:0/2465411678 >> 172.21.15.69:0/2465411678 conn(0x7f37740ffd30 msgr2=0x7f3774102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.944+0000 7f377c600640 1 -- 172.21.15.69:0/2465411678 shutdown_connections 2024-04-24T09:07:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.944+0000 7f377c600640 1 -- 172.21.15.69:0/2465411678 wait complete. 2024-04-24T09:07:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.945+0000 7f377c600640 1 Processor -- start 2024-04-24T09:07:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.945+0000 7f377c600640 1 -- start start 2024-04-24T09:07:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.946+0000 7f377c600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3774104470 0x7f37741a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.946+0000 7f377c600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3774109540 0x7f37741a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.946+0000 7f377c600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377419a7b0 0x7f377419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.946+0000 7f377c600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3774114490 con 0x7f377419a7b0 2024-04-24T09:07:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.946+0000 7f377c600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3774114310 con 0x7f3774109540 2024-04-24T09:07:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.946+0000 7f377c600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3774114610 con 0x7f3774104470 2024-04-24T09:07:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.946+0000 7f377a600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377419a7b0 0x7f377419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.946+0000 7f377a600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377419a7b0 0x7f377419ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35250/0 (socket says 172.21.15.69:35250) 2024-04-24T09:07:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.946+0000 7f377a600640 1 -- 172.21.15.69:0/68228881 learned_addr learned my addr 172.21.15.69:0/68228881 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.946+0000 7f3779c00640 1 --2- 172.21.15.69:0/68228881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3774104470 0x7f37741a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.946+0000 7f3779200640 1 --2- 172.21.15.69:0/68228881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3774109540 0x7f37741a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.947+0000 7f377a600640 1 -- 172.21.15.69:0/68228881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3774104470 msgr2=0x7f37741a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.947+0000 7f377a600640 1 --2- 172.21.15.69:0/68228881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3774104470 0x7f37741a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.947+0000 7f377a600640 1 -- 172.21.15.69:0/68228881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3774109540 msgr2=0x7f37741a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.947+0000 7f377a600640 1 --2- 172.21.15.69:0/68228881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3774109540 0x7f37741a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.947+0000 7f377a600640 1 -- 172.21.15.69:0/68228881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377419b3b0 con 0x7f377419a7b0 2024-04-24T09:07:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.947+0000 7f377a600640 1 --2- 172.21.15.69:0/68228881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377419a7b0 0x7f377419ac30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f376c009a20 tx=0x7f376c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.947+0000 7f3772000640 1 -- 172.21.15.69:0/68228881 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c03f030 con 0x7f377419a7b0 2024-04-24T09:07:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.947+0000 7f3772000640 1 -- 172.21.15.69:0/68228881 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f376c0044d0 con 0x7f377419a7b0 2024-04-24T09:07:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.947+0000 7f3772000640 1 -- 172.21.15.69:0/68228881 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c035070 con 0x7f377419a7b0 2024-04-24T09:07:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.947+0000 7f377c600640 1 -- 172.21.15.69:0/68228881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377419b640 con 0x7f377419a7b0 2024-04-24T09:07:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.947+0000 7f377c600640 1 -- 172.21.15.69:0/68228881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37741a7730 con 0x7f377419a7b0 2024-04-24T09:07:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.950+0000 7f377f576740 1 Processor -- start 2024-04-24T09:07:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.951+0000 7f3772000640 1 -- 172.21.15.69:0/68228881 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f376c042030 con 0x7f377419a7b0 2024-04-24T09:07:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.951+0000 7f377f576740 1 -- start start 2024-04-24T09:07:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.951+0000 7f377f576740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fb21b0 0x562da8fb2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.951+0000 7f377f576740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb2ad0 0x562da8fbce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.951+0000 7f377f576740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fbd390 0x562da8fbf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.951+0000 7f377f576740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562da8f18900 con 0x562da8fb2ad0 2024-04-24T09:07:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.951+0000 7f377f576740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562da8ebef20 con 0x562da8fb21b0 2024-04-24T09:07:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.951+0000 7f377f576740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562da8f148c0 con 0x562da8fbd390 2024-04-24T09:07:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.952+0000 7f3779c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb2ad0 0x562da8fbce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.952+0000 7f3779c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb2ad0 0x562da8fbce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35266/0 (socket says 172.21.15.69:35266) 2024-04-24T09:07:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.952+0000 7f3779c00640 1 -- 172.21.15.69:0/3378773973 learned_addr learned my addr 172.21.15.69:0/3378773973 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.952+0000 7f3779200640 1 --2- 172.21.15.69:0/3378773973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fbd390 0x562da8fbf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.952+0000 7f3779200640 1 --2- 172.21.15.69:0/3378773973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fb21b0 0x562da8fb2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.952+0000 7f3779c00640 1 -- 172.21.15.69:0/3378773973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fbd390 msgr2=0x562da8fbf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.952+0000 7f3779c00640 1 --2- 172.21.15.69:0/3378773973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fbd390 0x562da8fbf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.952+0000 7f3779c00640 1 -- 172.21.15.69:0/3378773973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fb21b0 msgr2=0x562da8fb2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.952+0000 7f3779c00640 1 --2- 172.21.15.69:0/3378773973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fb21b0 0x562da8fb2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.952+0000 7f3779c00640 1 -- 172.21.15.69:0/3378773973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562da8ee3e10 con 0x562da8fb2ad0 2024-04-24T09:07:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.952+0000 7f3772000640 1 --2- 172.21.15.69:0/68228881 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37580a3e60 0x7f37580a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.953+0000 7f3779c00640 1 --2- 172.21.15.69:0/3378773973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb2ad0 0x562da8fbce50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f376400cbf0 tx=0x7f376400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c224c7aa70d10cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.953+0000 7f3772000640 1 -- 172.21.15.69:0/68228881 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f376c0f16f0 con 0x7f377419a7b0 2024-04-24T09:07:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.953+0000 7f3779200640 1 --2- 172.21.15.69:0/68228881 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37580a3e60 0x7f37580a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.953+0000 7f376aa00640 1 -- 172.21.15.69:0/3378773973 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764010630 con 0x562da8fb2ad0 2024-04-24T09:07:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.953+0000 7f376aa00640 1 -- 172.21.15.69:0/3378773973 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3764010c80 con 0x562da8fb2ad0 2024-04-24T09:07:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.953+0000 7f376aa00640 1 -- 172.21.15.69:0/3378773973 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764019b80 con 0x562da8fb2ad0 2024-04-24T09:07:27.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:27 smithi130 ceph-mon[31553]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-24T09:07:27.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:27 smithi130 ceph-mon[31553]: scrub summary: idle 2024-04-24T09:07:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.954+0000 7f377f576740 1 -- 172.21.15.69:0/3378773973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb2ad0 msgr2=0x562da8fbce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.954+0000 7f3779200640 1 --2- 172.21.15.69:0/68228881 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37580a3e60 0x7f37580a6320 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x562da8fafae0 tx=0x7f3760004ab0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.954+0000 7f377f576740 1 --2- 172.21.15.69:0/3378773973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb2ad0 0x562da8fbce50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f376400cbf0 tx=0x7f376400e720 comp rx=0 tx=0).stop 2024-04-24T09:07:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.954+0000 7f377f576740 1 -- 172.21.15.69:0/3378773973 shutdown_connections 2024-04-24T09:07:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.954+0000 7f377f576740 1 --2- 172.21.15.69:0/3378773973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fbd390 0x562da8fbf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.954+0000 7f377f576740 1 --2- 172.21.15.69:0/3378773973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb2ad0 0x562da8fbce50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.954+0000 7f377f576740 1 --2- 172.21.15.69:0/3378773973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fb21b0 0x562da8fb2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.954+0000 7f377f576740 1 -- 172.21.15.69:0/3378773973 >> 172.21.15.69:0/3378773973 conn(0x562da8fae6c0 msgr2=0x562da8faeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.954+0000 7f377f576740 1 -- 172.21.15.69:0/3378773973 shutdown_connections 2024-04-24T09:07:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.954+0000 7f377f576740 1 -- 172.21.15.69:0/3378773973 wait complete. 2024-04-24T09:07:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.954+0000 7f377f576740 1 -- start start 2024-04-24T09:07:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.955+0000 7f377f576740 1 Processor -- start 2024-04-24T09:07:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.955+0000 7f377f576740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fb21b0 0x562da8fba350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.955+0000 7f377f576740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fb2ad0 0x562da8fba890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.955+0000 7f377f576740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fbd390 0x562da8fbadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.955+0000 7f377f576740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562da8fbffc0 con 0x562da8fbd390 2024-04-24T09:07:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.955+0000 7f377f576740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562da8ed22c0 con 0x562da8fb2ad0 2024-04-24T09:07:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.955+0000 7f377f576740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562da8fc0140 con 0x562da8fb21b0 2024-04-24T09:07:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.955+0000 7f377f576740 10 client.0 _ll_register_callbacks cb 0x562da8c2aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.955+0000 7f377a600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fbd390 0x562da8fbadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.955+0000 7f377a600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fbd390 0x562da8fbadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35272/0 (socket says 172.21.15.69:35272) 2024-04-24T09:07:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.955+0000 7f377a600640 1 -- 172.21.15.69:0/1010254076 learned_addr learned my addr 172.21.15.69:0/1010254076 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.956+0000 7f3779c00640 1 --2- 172.21.15.69:0/1010254076 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fb21b0 0x562da8fba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.956+0000 7f3779200640 1 --2- 172.21.15.69:0/1010254076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fb2ad0 0x562da8fba890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.956+0000 7f377a600640 1 -- 172.21.15.69:0/1010254076 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fb21b0 msgr2=0x562da8fba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.956+0000 7f377a600640 1 --2- 172.21.15.69:0/1010254076 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fb21b0 0x562da8fba350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.956+0000 7f377a600640 1 -- 172.21.15.69:0/1010254076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fb2ad0 msgr2=0x562da8fba890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.956+0000 7f377a600640 1 --2- 172.21.15.69:0/1010254076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fb2ad0 0x562da8fba890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.956+0000 7f377a600640 1 -- 172.21.15.69:0/1010254076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562da8ee2fd0 con 0x562da8fbd390 2024-04-24T09:07:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.956+0000 7f3779c00640 1 --2- 172.21.15.69:0/1010254076 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fb21b0 0x562da8fba350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.956+0000 7f377a600640 1 --2- 172.21.15.69:0/1010254076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fbd390 0x562da8fbadd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f376c0c58c0 tx=0x7f376c0bcba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5027883f66b403 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.956+0000 7f375f400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.956+0000 7f375f400640 1 -- 172.21.15.69:0/1010254076 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c049040 con 0x562da8fbd390 2024-04-24T09:07:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.956+0000 7f375f400640 1 -- 172.21.15.69:0/1010254076 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f376c03bc90 con 0x562da8fbd390 2024-04-24T09:07:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.956+0000 7f377f576740 1 -- 172.21.15.69:0/1010254076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3764010c80 con 0x562da8fbd390 2024-04-24T09:07:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.957+0000 7f375f400640 1 -- 172.21.15.69:0/1010254076 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f376c0c1530 con 0x562da8fbd390 2024-04-24T09:07:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.957+0000 7f377f576740 10 client.15732 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.957+0000 7f377f576740 1 -- 172.21.15.69:0/1010254076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562da8c2b210 con 0x562da8fbd390 2024-04-24T09:07:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.957+0000 7f375f400640 1 -- 172.21.15.69:0/1010254076 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3764010c80 con 0x562da8fbd390 2024-04-24T09:07:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.957+0000 7f375f400640 1 -- 172.21.15.69:0/1010254076 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f376c0c1120 con 0x562da8fbd390 2024-04-24T09:07:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.957+0000 7f377f576740 10 client.15732 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.957+0000 7f377f576740 10 client.15732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.957+0000 7f377f576740 20 client.15732 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.957+0000 7f377f576740 1 --2- 172.21.15.69:0/1010254076 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562da8fcd4d0 0x562da8fcf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.957+0000 7f377f576740 4 client.15732 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.957+0000 7f377f576740 1 -- 172.21.15.69:0/1010254076 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562da8f26320 con 0x562da8fcd4d0 2024-04-24T09:07:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.958+0000 7f3779c00640 1 --2- 172.21.15.69:0/1010254076 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562da8fcd4d0 0x562da8fcf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.959+0000 7f3779c00640 1 --2- 172.21.15.69:0/1010254076 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562da8fcd4d0 0x562da8fcf930 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782203d7a1ce67e9 server_cookie=cac5d09e56f2c9cd in_seq=0 out_seq=0 2024-04-24T09:07:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.959+0000 7f375f400640 10 client.15732 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.961+0000 7f375f400640 1 -- 172.21.15.69:0/1010254076 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562da8f26320 con 0x562da8fcd4d0 2024-04-24T09:07:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.961+0000 7f375f400640 10 client.15732 handle_command_reply: tid=0 2024-04-24T09:07:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.961+0000 7f377f576740 1 client.15732 shutdown 2024-04-24T09:07:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.961+0000 7f377f576740 1 -- 172.21.15.69:0/1010254076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fbd390 msgr2=0x562da8fbadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.961+0000 7f377f576740 1 --2- 172.21.15.69:0/1010254076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fbd390 0x562da8fbadd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f376c0c58c0 tx=0x7f376c0bcba0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.961+0000 7f377f576740 1 -- 172.21.15.69:0/1010254076 shutdown_connections 2024-04-24T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.961+0000 7f377f576740 1 --2- 172.21.15.69:0/1010254076 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562da8fcd4d0 0x562da8fcf930 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.961+0000 7f377f576740 1 --2- 172.21.15.69:0/1010254076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fbd390 0x562da8fbadd0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.961+0000 7f377f576740 1 --2- 172.21.15.69:0/1010254076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fb2ad0 0x562da8fba890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.961+0000 7f377f576740 1 --2- 172.21.15.69:0/1010254076 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fb21b0 0x562da8fba350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.961+0000 7f375f400640 0 client.15732 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.961+0000 7f377f576740 1 -- 172.21.15.69:0/1010254076 >> 172.21.15.69:0/1010254076 conn(0x562da8fae6c0 msgr2=0x562da8fb4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.961+0000 7f377f576740 1 -- 172.21.15.69:0/1010254076 shutdown_connections 2024-04-24T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.961+0000 7f377f576740 1 -- 172.21.15.69:0/1010254076 wait complete. 2024-04-24T09:07:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.961+0000 7f377f576740 20 client.15732 trim_cache size 0 max 16384 2024-04-24T09:07:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.977+0000 7f377f576740 1 Processor -- start 2024-04-24T09:07:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.977+0000 7f377f576740 1 -- start start 2024-04-24T09:07:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.978+0000 7f377f576740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb21b0 0x562da8fc8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.978+0000 7f377f576740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fb2ad0 0x562da8fb4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.978+0000 7f377f576740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fbd390 0x562da8fd5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.978+0000 7f377f576740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37741b3c10 con 0x562da8fb21b0 2024-04-24T09:07:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.978+0000 7f377f576740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f37741b14e0 con 0x562da8fbd390 2024-04-24T09:07:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.978+0000 7f377f576740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f37741af330 con 0x562da8fb2ad0 2024-04-24T09:07:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.978+0000 7f3779c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb21b0 0x562da8fc8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.978+0000 7f3779c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb21b0 0x562da8fc8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35274/0 (socket says 172.21.15.69:35274) 2024-04-24T09:07:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.978+0000 7f3779c00640 1 -- 172.21.15.69:0/2230746669 learned_addr learned my addr 172.21.15.69:0/2230746669 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.978+0000 7f3779c00640 1 -- 172.21.15.69:0/2230746669 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fb2ad0 msgr2=0x562da8fb4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.978+0000 7f3779c00640 1 --2- 172.21.15.69:0/2230746669 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fb2ad0 0x562da8fb4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.978+0000 7f3779c00640 1 -- 172.21.15.69:0/2230746669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fbd390 msgr2=0x562da8fd5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.978+0000 7f3779c00640 1 --2- 172.21.15.69:0/2230746669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fbd390 0x562da8fd5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.978+0000 7f3779c00640 1 -- 172.21.15.69:0/2230746669 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562da8cf7620 con 0x562da8fb21b0 2024-04-24T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.979+0000 7f3779c00640 1 --2- 172.21.15.69:0/2230746669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb21b0 0x562da8fc8e80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3764000c00 tx=0x7f3764005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640e21e9a771f8df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.979+0000 7f376a000640 1 -- 172.21.15.69:0/2230746669 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37640195c0 con 0x562da8fb21b0 2024-04-24T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.979+0000 7f376a000640 1 -- 172.21.15.69:0/2230746669 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3764005040 con 0x562da8fb21b0 2024-04-24T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.979+0000 7f376a000640 1 -- 172.21.15.69:0/2230746669 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764019b50 con 0x562da8fb21b0 2024-04-24T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.979+0000 7f377f576740 1 -- 172.21.15.69:0/2230746669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb21b0 msgr2=0x562da8fc8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.979+0000 7f377f576740 1 --2- 172.21.15.69:0/2230746669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb21b0 0x562da8fc8e80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3764000c00 tx=0x7f3764005ce0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.980+0000 7f377f576740 1 -- 172.21.15.69:0/2230746669 shutdown_connections 2024-04-24T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.980+0000 7f377f576740 1 --2- 172.21.15.69:0/2230746669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fbd390 0x562da8fd5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.980+0000 7f377f576740 1 --2- 172.21.15.69:0/2230746669 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fb2ad0 0x562da8fb4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.980+0000 7f377f576740 1 --2- 172.21.15.69:0/2230746669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb21b0 0x562da8fc8e80 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.980+0000 7f377f576740 1 -- 172.21.15.69:0/2230746669 >> 172.21.15.69:0/2230746669 conn(0x562da8fae6c0 msgr2=0x562da8fbaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.980+0000 7f377f576740 1 -- 172.21.15.69:0/2230746669 shutdown_connections 2024-04-24T09:07:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.980+0000 7f377f576740 1 -- 172.21.15.69:0/2230746669 wait complete. 2024-04-24T09:07:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.980+0000 7f377f576740 1 -- start start 2024-04-24T09:07:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.980+0000 7f377f576740 1 Processor -- start 2024-04-24T09:07:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.981+0000 7f377f576740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb21b0 0x562da8fb7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.981+0000 7f377f576740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fb2ad0 0x562da8fb7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.981+0000 7f377f576740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fbd390 0x562da9032ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.981+0000 7f377f576740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37741af9a0 con 0x562da8fb21b0 2024-04-24T09:07:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.981+0000 7f377f576740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f37741b0e10 con 0x562da8fbd390 2024-04-24T09:07:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.981+0000 7f377f576740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f37741b0070 con 0x562da8fb2ad0 2024-04-24T09:07:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.981+0000 7f377f576740 10 client.0 _ll_register_callbacks cb 0x7f376c0ee870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.981+0000 7f3779c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb21b0 0x562da8fb7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.981+0000 7f3779c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb21b0 0x562da8fb7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35284/0 (socket says 172.21.15.69:35284) 2024-04-24T09:07:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.981+0000 7f3779c00640 1 -- 172.21.15.69:0/2740199625 learned_addr learned my addr 172.21.15.69:0/2740199625 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.981+0000 7f3779c00640 1 -- 172.21.15.69:0/2740199625 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fb2ad0 msgr2=0x562da8fb7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.981+0000 7f3779c00640 1 --2- 172.21.15.69:0/2740199625 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fb2ad0 0x562da8fb7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.981+0000 7f3779c00640 1 -- 172.21.15.69:0/2740199625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fbd390 msgr2=0x562da9032ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.981+0000 7f3779c00640 1 --2- 172.21.15.69:0/2740199625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fbd390 0x562da9032ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.981+0000 7f3779c00640 1 -- 172.21.15.69:0/2740199625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562da9032ff0 con 0x562da8fb21b0 2024-04-24T09:07:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.981+0000 7f3779c00640 1 --2- 172.21.15.69:0/2740199625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb21b0 0x562da8fb7660 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f376400cb40 tx=0x7f3764005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655deb76dca563f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.982+0000 7f375f400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.982+0000 7f375f400640 1 -- 172.21.15.69:0/2740199625 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764036060 con 0x562da8fb21b0 2024-04-24T09:07:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.982+0000 7f375f400640 1 -- 172.21.15.69:0/2740199625 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f37640051f0 con 0x562da8fb21b0 2024-04-24T09:07:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.982+0000 7f377f576740 1 -- 172.21.15.69:0/2740199625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3764005040 con 0x562da8fb21b0 2024-04-24T09:07:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.982+0000 7f375f400640 1 -- 172.21.15.69:0/2740199625 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f376403d050 con 0x562da8fb21b0 2024-04-24T09:07:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.982+0000 7f377f576740 10 client.15738 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.982+0000 7f377f576740 1 -- 172.21.15.69:0/2740199625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562da9034200 con 0x562da8fb21b0 2024-04-24T09:07:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.982+0000 7f375f400640 1 -- 172.21.15.69:0/2740199625 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3764005040 con 0x562da8fb21b0 2024-04-24T09:07:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.982+0000 7f375f400640 1 -- 172.21.15.69:0/2740199625 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f3764028170 con 0x562da8fb21b0 2024-04-24T09:07:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.983+0000 7f377f576740 10 client.15738 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.983+0000 7f377f576740 10 client.15738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.983+0000 7f377f576740 20 client.15738 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.983+0000 7f377f576740 1 --2- 172.21.15.69:0/2740199625 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562da90346d0 0x562da9036b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.983+0000 7f377f576740 4 client.15738 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:07:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.983+0000 7f377f576740 1 -- 172.21.15.69:0/2740199625 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562da8f20190 con 0x562da90346d0 2024-04-24T09:07:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.983+0000 7f3779200640 1 --2- 172.21.15.69:0/2740199625 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562da90346d0 0x562da9036b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.984+0000 7f3779200640 1 --2- 172.21.15.69:0/2740199625 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562da90346d0 0x562da9036b20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa3859d6aa46cff3 server_cookie=d7f389ae8bdd85f1 in_seq=0 out_seq=0 2024-04-24T09:07:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:27.984+0000 7f375f400640 10 client.15738 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.016+0000 7f375f400640 1 -- 172.21.15.69:0/2740199625 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562da8f20190 con 0x562da90346d0 2024-04-24T09:07:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.016+0000 7f375f400640 10 client.15738 handle_command_reply: tid=0 2024-04-24T09:07:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.016+0000 7f377f576740 1 client.15738 shutdown 2024-04-24T09:07:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.017+0000 7f377f576740 1 -- 172.21.15.69:0/2740199625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb21b0 msgr2=0x562da8fb7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.017+0000 7f377f576740 1 --2- 172.21.15.69:0/2740199625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb21b0 0x562da8fb7660 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f376400cb40 tx=0x7f3764005e40 comp rx=0 tx=0).stop 2024-04-24T09:07:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.017+0000 7f377f576740 1 -- 172.21.15.69:0/2740199625 shutdown_connections 2024-04-24T09:07:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.017+0000 7f377f576740 1 --2- 172.21.15.69:0/2740199625 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562da90346d0 0x562da9036b20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.017+0000 7f377f576740 1 --2- 172.21.15.69:0/2740199625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562da8fbd390 0x562da9032ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.017+0000 7f377f576740 1 --2- 172.21.15.69:0/2740199625 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562da8fb2ad0 0x562da8fb7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.017+0000 7f377f576740 1 --2- 172.21.15.69:0/2740199625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562da8fb21b0 0x562da8fb7660 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.017+0000 7f375f400640 0 client.15738 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.017+0000 7f377f576740 1 -- 172.21.15.69:0/2740199625 >> 172.21.15.69:0/2740199625 conn(0x562da8fae6c0 msgr2=0x562da8facfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.017+0000 7f377f576740 1 -- 172.21.15.69:0/2740199625 shutdown_connections 2024-04-24T09:07:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.017+0000 7f377f576740 1 -- 172.21.15.69:0/2740199625 wait complete. 2024-04-24T09:07:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.017+0000 7f377f576740 20 client.15738 trim_cache size 0 max 16384 2024-04-24T09:07:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:07:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:07:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:07:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:07:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.018+0000 7f377c600640 1 -- 172.21.15.69:0/68228881 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37580a3e60 msgr2=0x7f37580a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.018+0000 7f377c600640 1 --2- 172.21.15.69:0/68228881 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37580a3e60 0x7f37580a6320 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x562da8fafae0 tx=0x7f3760004ab0 comp rx=0 tx=0).stop 2024-04-24T09:07:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.018+0000 7f377c600640 1 -- 172.21.15.69:0/68228881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377419a7b0 msgr2=0x7f377419ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.018+0000 7f377c600640 1 --2- 172.21.15.69:0/68228881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377419a7b0 0x7f377419ac30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f376c009a20 tx=0x7f376c0042e0 comp rx=0 tx=0).stop 2024-04-24T09:07:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.018+0000 7f377c600640 1 -- 172.21.15.69:0/68228881 shutdown_connections 2024-04-24T09:07:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.018+0000 7f377c600640 1 --2- 172.21.15.69:0/68228881 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37580a3e60 0x7f37580a6320 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.018+0000 7f377c600640 1 --2- 172.21.15.69:0/68228881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377419a7b0 0x7f377419ac30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.018+0000 7f377c600640 1 --2- 172.21.15.69:0/68228881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3774109540 0x7f37741a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.018+0000 7f377c600640 1 --2- 172.21.15.69:0/68228881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3774104470 0x7f37741a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.018+0000 7f377c600640 1 -- 172.21.15.69:0/68228881 >> 172.21.15.69:0/68228881 conn(0x7f37740ffd30 msgr2=0x7f3774102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.018+0000 7f377c600640 1 -- 172.21.15.69:0/68228881 shutdown_connections 2024-04-24T09:07:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:28.018+0000 7f377c600640 1 -- 172.21.15.69:0/68228881 wait complete. 2024-04-24T09:07:29.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:29 smithi081 ceph-mon[30961]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-24T09:07:29.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:29 smithi069 ceph-mon[28170]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-24T09:07:29.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:29 smithi130 ceph-mon[31553]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-24T09:07:31.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:31 smithi081 ceph-mon[30961]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-24T09:07:31.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:31 smithi069 ceph-mon[28170]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-24T09:07:31.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:31 smithi130 ceph-mon[31553]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-24T09:07:33.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:33 smithi081 ceph-mon[30961]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-24T09:07:33.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:33 smithi069 ceph-mon[28170]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-24T09:07:33.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:33 smithi130 ceph-mon[31553]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-24T09:07:35.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:35 smithi069 ceph-mon[28170]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-24T09:07:35.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:35 smithi130 ceph-mon[31553]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-24T09:07:36.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:35 smithi081 ceph-mon[30961]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-24T09:07:37.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:37 smithi069 ceph-mon[28170]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-24T09:07:37.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:37 smithi130 ceph-mon[31553]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-24T09:07:38.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:37 smithi081 ceph-mon[30961]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-24T09:07:39.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:39 smithi069 ceph-mon[28170]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-24T09:07:39.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:39 smithi130 ceph-mon[31553]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-24T09:07:40.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:39 smithi081 ceph-mon[30961]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-24T09:07:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.159+0000 7fa5d2600640 1 -- 172.21.15.69:0/1045075733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5cc10c3e0 msgr2=0x7fa5cc10e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.159+0000 7fa5d2600640 1 --2- 172.21.15.69:0/1045075733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5cc10c3e0 0x7fa5cc10e880 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009ca0 tx=0x7fa5c402a950 comp rx=0 tx=0).stop 2024-04-24T09:07:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.159+0000 7fa5d2600640 1 -- 172.21.15.69:0/1045075733 shutdown_connections 2024-04-24T09:07:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.160+0000 7fa5d2600640 1 --2- 172.21.15.69:0/1045075733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5cc10c3e0 0x7fa5cc10e880 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.160+0000 7fa5d2600640 1 --2- 172.21.15.69:0/1045075733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc1038e0 0x7fa5cc10bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.160+0000 7fa5d2600640 1 --2- 172.21.15.69:0/1045075733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5cc102fc0 0x7fa5cc1033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.160+0000 7fa5d2600640 1 -- 172.21.15.69:0/1045075733 >> 172.21.15.69:0/1045075733 conn(0x7fa5cc0fc820 msgr2=0x7fa5cc0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.160+0000 7fa5d2600640 1 -- 172.21.15.69:0/1045075733 shutdown_connections 2024-04-24T09:07:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.160+0000 7fa5d2600640 1 -- 172.21.15.69:0/1045075733 wait complete. 2024-04-24T09:07:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.161+0000 7fa5d2600640 1 Processor -- start 2024-04-24T09:07:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.161+0000 7fa5d2600640 1 -- start start 2024-04-24T09:07:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.161+0000 7fa5d2600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc102fc0 0x7fa5cc1a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.161+0000 7fa5d2600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5cc1038e0 0x7fa5cc1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.161+0000 7fa5d2600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5cc10c3e0 0x7fa5cc19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.161+0000 7fa5d2600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1120a0 con 0x7fa5cc1038e0 2024-04-24T09:07:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.161+0000 7fa5d2600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc111f20 con 0x7fa5cc102fc0 2024-04-24T09:07:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.161+0000 7fa5d2600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc112220 con 0x7fa5cc10c3e0 2024-04-24T09:07:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.162+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5cc1038e0 0x7fa5cc1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.162+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5cc1038e0 0x7fa5cc1a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47972/0 (socket says 172.21.15.69:47972) 2024-04-24T09:07:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.162+0000 7fa5cbe00640 1 -- 172.21.15.69:0/2924878389 learned_addr learned my addr 172.21.15.69:0/2924878389 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.162+0000 7fa5d1c00640 1 --2- 172.21.15.69:0/2924878389 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5cc10c3e0 0x7fa5cc19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.162+0000 7fa5d1200640 1 --2- 172.21.15.69:0/2924878389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc102fc0 0x7fa5cc1a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.162+0000 7fa5cbe00640 1 -- 172.21.15.69:0/2924878389 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5cc10c3e0 msgr2=0x7fa5cc19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.162+0000 7fa5cbe00640 1 --2- 172.21.15.69:0/2924878389 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5cc10c3e0 0x7fa5cc19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.162+0000 7fa5cbe00640 1 -- 172.21.15.69:0/2924878389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc102fc0 msgr2=0x7fa5cc1a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.162+0000 7fa5cbe00640 1 --2- 172.21.15.69:0/2924878389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc102fc0 0x7fa5cc1a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.162+0000 7fa5cbe00640 1 -- 172.21.15.69:0/2924878389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc19afc0 con 0x7fa5cc1038e0 2024-04-24T09:07:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.162+0000 7fa5d1200640 1 --2- 172.21.15.69:0/2924878389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc102fc0 0x7fa5cc1a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.162+0000 7fa5cbe00640 1 --2- 172.21.15.69:0/2924878389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5cc1038e0 0x7fa5cc1a0a90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800d740 tx=0x7fa5b800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.163+0000 7fa5c9600640 1 -- 172.21.15.69:0/2924878389 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8017070 con 0x7fa5cc1038e0 2024-04-24T09:07:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.163+0000 7fa5c9600640 1 -- 172.21.15.69:0/2924878389 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5b800de50 con 0x7fa5cc1038e0 2024-04-24T09:07:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.163+0000 7fa5c9600640 1 -- 172.21.15.69:0/2924878389 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b800f620 con 0x7fa5cc1038e0 2024-04-24T09:07:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.163+0000 7fa5d2600640 1 -- 172.21.15.69:0/2924878389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5cc19b2b0 con 0x7fa5cc1038e0 2024-04-24T09:07:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.163+0000 7fa5d2600640 1 -- 172.21.15.69:0/2924878389 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5cc1a74d0 con 0x7fa5cc1038e0 2024-04-24T09:07:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.166+0000 7fa5d6b98740 1 Processor -- start 2024-04-24T09:07:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.166+0000 7fa5c9600640 1 -- 172.21.15.69:0/2924878389 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fa5b8009c30 con 0x7fa5cc1038e0 2024-04-24T09:07:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.166+0000 7fa5d6b98740 1 -- start start 2024-04-24T09:07:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5d6b98740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a43133c0 0x5598a43137a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5d6b98740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a431e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5d6b98740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a431e5a0 0x5598a4320990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5d6b98740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5598a4279900 con 0x5598a4313ce0 2024-04-24T09:07:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5d6b98740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5598a421ff20 con 0x5598a431e5a0 2024-04-24T09:07:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5d6b98740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5598a42758c0 con 0x5598a43133c0 2024-04-24T09:07:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5d1200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a431e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5d1200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a431e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47988/0 (socket says 172.21.15.69:47988) 2024-04-24T09:07:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5d1200640 1 -- 172.21.15.69:0/3826346318 learned_addr learned my addr 172.21.15.69:0/3826346318 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5cbe00640 1 --2- 172.21.15.69:0/3826346318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a431e5a0 0x5598a4320990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5d1c00640 1 --2- 172.21.15.69:0/3826346318 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a43133c0 0x5598a43137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5d1200640 1 -- 172.21.15.69:0/3826346318 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a43133c0 msgr2=0x5598a43137a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5d1200640 1 --2- 172.21.15.69:0/3826346318 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a43133c0 0x5598a43137a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5d1200640 1 -- 172.21.15.69:0/3826346318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a431e5a0 msgr2=0x5598a4320990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5d1200640 1 --2- 172.21.15.69:0/3826346318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a431e5a0 0x5598a4320990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5d1200640 1 -- 172.21.15.69:0/3826346318 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598a4244e10 con 0x5598a4313ce0 2024-04-24T09:07:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.167+0000 7fa5cbe00640 1 --2- 172.21.15.69:0/3826346318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a431e5a0 0x5598a4320990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.168+0000 7fa5c9600640 1 --2- 172.21.15.69:0/2924878389 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa5b40a3e20 0x7fa5b40a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.168+0000 7fa5d1200640 1 --2- 172.21.15.69:0/3826346318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a431e060 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009c50 tx=0x7fa5c000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5a11901f067364 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.168+0000 7fa5c9600640 1 -- 172.21.15.69:0/2924878389 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa5b80d0830 con 0x7fa5cc1038e0 2024-04-24T09:07:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.168+0000 7fa5be000640 1 -- 172.21.15.69:0/3826346318 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0016070 con 0x5598a4313ce0 2024-04-24T09:07:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.168+0000 7fa5d1c00640 1 --2- 172.21.15.69:0/2924878389 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa5b40a3e20 0x7fa5b40a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.168+0000 7fa5be000640 1 -- 172.21.15.69:0/3826346318 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5c00054d0 con 0x5598a4313ce0 2024-04-24T09:07:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.168+0000 7fa5be000640 1 -- 172.21.15.69:0/3826346318 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0011470 con 0x5598a4313ce0 2024-04-24T09:07:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.169+0000 7fa5d6b98740 1 -- 172.21.15.69:0/3826346318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 msgr2=0x5598a431e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.169+0000 7fa5d1c00640 1 --2- 172.21.15.69:0/2924878389 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa5b40a3e20 0x7fa5b40a62e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4005e30 tx=0x7fa5c4005d80 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.169+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/3826346318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a431e060 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009c50 tx=0x7fa5c000e6c0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.169+0000 7fa5d6b98740 1 -- 172.21.15.69:0/3826346318 shutdown_connections 2024-04-24T09:07:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.169+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/3826346318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a431e5a0 0x5598a4320990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.169+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/3826346318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a431e060 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.169+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/3826346318 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a43133c0 0x5598a43137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.169+0000 7fa5d6b98740 1 -- 172.21.15.69:0/3826346318 >> 172.21.15.69:0/3826346318 conn(0x5598a430f8d0 msgr2=0x5598a430fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.169+0000 7fa5d6b98740 1 -- 172.21.15.69:0/3826346318 shutdown_connections 2024-04-24T09:07:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.169+0000 7fa5d6b98740 1 -- 172.21.15.69:0/3826346318 wait complete. 2024-04-24T09:07:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.170+0000 7fa5d6b98740 1 -- start start 2024-04-24T09:07:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.170+0000 7fa5d6b98740 1 Processor -- start 2024-04-24T09:07:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.170+0000 7fa5d6b98740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a43133c0 0x5598a431b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.170+0000 7fa5d6b98740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a431ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.171+0000 7fa5d6b98740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a431e5a0 0x5598a431bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.171+0000 7fa5d6b98740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5598a4321140 con 0x5598a4313ce0 2024-04-24T09:07:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.171+0000 7fa5d6b98740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5598a42332c0 con 0x5598a43133c0 2024-04-24T09:07:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.171+0000 7fa5d6b98740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5598a43212c0 con 0x5598a431e5a0 2024-04-24T09:07:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.171+0000 7fa5d6b98740 10 client.0 _ll_register_callbacks cb 0x5598a3f8ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.171+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a431ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.171+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a431ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47994/0 (socket says 172.21.15.69:47994) 2024-04-24T09:07:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.171+0000 7fa5cbe00640 1 -- 172.21.15.69:0/2952586446 learned_addr learned my addr 172.21.15.69:0/2952586446 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.171+0000 7fa5d1200640 1 --2- 172.21.15.69:0/2952586446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a43133c0 0x5598a431b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.171+0000 7fa5cbe00640 1 -- 172.21.15.69:0/2952586446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a431e5a0 msgr2=0x5598a431bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.171+0000 7fa5cbe00640 1 --2- 172.21.15.69:0/2952586446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a431e5a0 0x5598a431bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.171+0000 7fa5cbe00640 1 -- 172.21.15.69:0/2952586446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a43133c0 msgr2=0x5598a431b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:41.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.171+0000 7fa5cbe00640 1 --2- 172.21.15.69:0/2952586446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a43133c0 0x5598a431b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.171+0000 7fa5cbe00640 1 -- 172.21.15.69:0/2952586446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598a4243fd0 con 0x5598a4313ce0 2024-04-24T09:07:41.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.171+0000 7fa5cbe00640 1 --2- 172.21.15.69:0/2952586446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a431ba20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa5b809c020 tx=0x7fa5b809bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a47b3cdc9becd58 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:41.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.171+0000 7fa5b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:41.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.172+0000 7fa5b2a00640 1 -- 172.21.15.69:0/2952586446 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8020070 con 0x5598a4313ce0 2024-04-24T09:07:41.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.172+0000 7fa5b2a00640 1 -- 172.21.15.69:0/2952586446 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa5b80ce030 con 0x5598a4313ce0 2024-04-24T09:07:41.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.172+0000 7fa5d6b98740 1 -- 172.21.15.69:0/2952586446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5c00054d0 con 0x5598a4313ce0 2024-04-24T09:07:41.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.172+0000 7fa5b2a00640 1 -- 172.21.15.69:0/2952586446 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa5b80a06d0 con 0x5598a4313ce0 2024-04-24T09:07:41.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.172+0000 7fa5d6b98740 10 client.15750 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:41.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.172+0000 7fa5d6b98740 1 -- 172.21.15.69:0/2952586446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5598a3f8c210 con 0x5598a4313ce0 2024-04-24T09:07:41.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.172+0000 7fa5b2a00640 1 -- 172.21.15.69:0/2952586446 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5c00054d0 con 0x5598a4313ce0 2024-04-24T09:07:41.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.172+0000 7fa5b2a00640 1 -- 172.21.15.69:0/2952586446 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fa5b80a0120 con 0x5598a4313ce0 2024-04-24T09:07:41.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.172+0000 7fa5d6b98740 10 client.15750 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:41.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.172+0000 7fa5d6b98740 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:41.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.172+0000 7fa5d6b98740 20 client.15750 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:41.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.172+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/2952586446 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5598a432e610 0x5598a4330ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.173+0000 7fa5d6b98740 4 client.15750 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:41.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.173+0000 7fa5d6b98740 1 -- 172.21.15.69:0/2952586446 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598a4287320 con 0x5598a432e610 2024-04-24T09:07:41.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.173+0000 7fa5d1200640 1 --2- 172.21.15.69:0/2952586446 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5598a432e610 0x5598a4330ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:41.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.174+0000 7fa5d1200640 1 --2- 172.21.15.69:0/2952586446 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5598a432e610 0x5598a4330ab0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de254291c7d9688f server_cookie=d0715c34c2f72c9f in_seq=0 out_seq=0 2024-04-24T09:07:41.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.174+0000 7fa5b2a00640 10 client.15750 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:41.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.176+0000 7fa5b2a00640 1 -- 172.21.15.69:0/2952586446 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598a4287320 con 0x5598a432e610 2024-04-24T09:07:41.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.176+0000 7fa5b2a00640 10 client.15750 handle_command_reply: tid=0 2024-04-24T09:07:41.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.176+0000 7fa5d6b98740 1 client.15750 shutdown 2024-04-24T09:07:41.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.176+0000 7fa5d6b98740 1 -- 172.21.15.69:0/2952586446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 msgr2=0x5598a431ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.176+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/2952586446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a431ba20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa5b809c020 tx=0x7fa5b809bd40 comp rx=0 tx=0).stop 2024-04-24T09:07:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.177+0000 7fa5d6b98740 1 -- 172.21.15.69:0/2952586446 shutdown_connections 2024-04-24T09:07:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.177+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/2952586446 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5598a432e610 0x5598a4330ab0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.177+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/2952586446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a431e5a0 0x5598a431bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.177+0000 7fa5b2a00640 0 client.15750 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.177+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/2952586446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a431ba20 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.177+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/2952586446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a43133c0 0x5598a431b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.177+0000 7fa5d6b98740 1 -- 172.21.15.69:0/2952586446 >> 172.21.15.69:0/2952586446 conn(0x5598a430f8d0 msgr2=0x5598a430ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.177+0000 7fa5d6b98740 1 -- 172.21.15.69:0/2952586446 shutdown_connections 2024-04-24T09:07:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.177+0000 7fa5d6b98740 1 -- 172.21.15.69:0/2952586446 wait complete. 2024-04-24T09:07:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.177+0000 7fa5d6b98740 20 client.15750 trim_cache size 0 max 16384 2024-04-24T09:07:41.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.194+0000 7fa5d6b98740 1 Processor -- start 2024-04-24T09:07:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.194+0000 7fa5d6b98740 1 -- start start 2024-04-24T09:07:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.194+0000 7fa5d6b98740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a43133c0 0x5598a432a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.194+0000 7fa5d6b98740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a4315d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.194+0000 7fa5d6b98740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a431e5a0 0x5598a431bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.194+0000 7fa5d6b98740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1b3bc0 con 0x5598a4313ce0 2024-04-24T09:07:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.194+0000 7fa5d6b98740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1b1490 con 0x5598a43133c0 2024-04-24T09:07:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.194+0000 7fa5d6b98740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1af2e0 con 0x5598a431e5a0 2024-04-24T09:07:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.194+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a4315d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.194+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a4315d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48000/0 (socket says 172.21.15.69:48000) 2024-04-24T09:07:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.194+0000 7fa5cbe00640 1 -- 172.21.15.69:0/3803739451 learned_addr learned my addr 172.21.15.69:0/3803739451 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.195+0000 7fa5cbe00640 1 -- 172.21.15.69:0/3803739451 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a431e5a0 msgr2=0x5598a431bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.195+0000 7fa5cbe00640 1 --2- 172.21.15.69:0/3803739451 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a431e5a0 0x5598a431bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.195+0000 7fa5cbe00640 1 -- 172.21.15.69:0/3803739451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a43133c0 msgr2=0x5598a432a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.195+0000 7fa5cbe00640 1 --2- 172.21.15.69:0/3803739451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a43133c0 0x5598a432a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.195+0000 7fa5cbe00640 1 -- 172.21.15.69:0/3803739451 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598a4058620 con 0x5598a4313ce0 2024-04-24T09:07:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.195+0000 7fa5cbe00640 1 --2- 172.21.15.69:0/3803739451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a4315d80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80a0530 tx=0x7fa5b800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9a847b96788e7f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.195+0000 7fa5bd600640 1 -- 172.21.15.69:0/3803739451 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8020070 con 0x5598a4313ce0 2024-04-24T09:07:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.195+0000 7fa5bd600640 1 -- 172.21.15.69:0/3803739451 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5b8015020 con 0x5598a4313ce0 2024-04-24T09:07:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.195+0000 7fa5bd600640 1 -- 172.21.15.69:0/3803739451 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b80ce090 con 0x5598a4313ce0 2024-04-24T09:07:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.196+0000 7fa5d6b98740 1 -- 172.21.15.69:0/3803739451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 msgr2=0x5598a4315d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.196+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/3803739451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a4315d80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80a0530 tx=0x7fa5b800b480 comp rx=0 tx=0).stop 2024-04-24T09:07:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.196+0000 7fa5d6b98740 1 -- 172.21.15.69:0/3803739451 shutdown_connections 2024-04-24T09:07:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.196+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/3803739451 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a431e5a0 0x5598a431bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.196+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/3803739451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a4315d80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.196+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/3803739451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a43133c0 0x5598a432a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.196+0000 7fa5d6b98740 1 -- 172.21.15.69:0/3803739451 >> 172.21.15.69:0/3803739451 conn(0x5598a430f8d0 msgr2=0x5598a431b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.196+0000 7fa5d6b98740 1 -- 172.21.15.69:0/3803739451 shutdown_connections 2024-04-24T09:07:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.196+0000 7fa5d6b98740 1 -- 172.21.15.69:0/3803739451 wait complete. 2024-04-24T09:07:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.196+0000 7fa5d6b98740 1 -- start start 2024-04-24T09:07:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.197+0000 7fa5d6b98740 1 Processor -- start 2024-04-24T09:07:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.197+0000 7fa5d6b98740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a43133c0 0x5598a4326a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.197+0000 7fa5d6b98740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a4317fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.197+0000 7fa5d6b98740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a431e5a0 0x5598a43185d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.197+0000 7fa5d6b98740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1af950 con 0x5598a4313ce0 2024-04-24T09:07:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.197+0000 7fa5d6b98740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1b0dc0 con 0x5598a431e5a0 2024-04-24T09:07:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.197+0000 7fa5d6b98740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1b0020 con 0x5598a43133c0 2024-04-24T09:07:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.197+0000 7fa5d6b98740 10 client.0 _ll_register_callbacks cb 0x7fa5b80cda10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.197+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a4317fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.197+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a4317fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48014/0 (socket says 172.21.15.69:48014) 2024-04-24T09:07:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.197+0000 7fa5cbe00640 1 -- 172.21.15.69:0/326412880 learned_addr learned my addr 172.21.15.69:0/326412880 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.198+0000 7fa5d1200640 1 --2- 172.21.15.69:0/326412880 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a43133c0 0x5598a4326a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.198+0000 7fa5cbe00640 1 -- 172.21.15.69:0/326412880 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a43133c0 msgr2=0x5598a4326a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.198+0000 7fa5cbe00640 1 --2- 172.21.15.69:0/326412880 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a43133c0 0x5598a4326a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.198+0000 7fa5d1c00640 1 --2- 172.21.15.69:0/326412880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a431e5a0 0x5598a43185d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.198+0000 7fa5cbe00640 1 -- 172.21.15.69:0/326412880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a431e5a0 msgr2=0x5598a43185d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.198+0000 7fa5cbe00640 1 --2- 172.21.15.69:0/326412880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a431e5a0 0x5598a43185d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.198+0000 7fa5cbe00640 1 -- 172.21.15.69:0/326412880 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598a4318e00 con 0x5598a4313ce0 2024-04-24T09:07:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.198+0000 7fa5d1200640 1 --2- 172.21.15.69:0/326412880 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a43133c0 0x5598a4326a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.198+0000 7fa5cbe00640 1 --2- 172.21.15.69:0/326412880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a4317fc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa5b809c020 tx=0x7fa5b809b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d52dfef0298441 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.198+0000 7fa5b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.198+0000 7fa5b2a00640 1 -- 172.21.15.69:0/326412880 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b80b9030 con 0x5598a4313ce0 2024-04-24T09:07:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.198+0000 7fa5b2a00640 1 -- 172.21.15.69:0/326412880 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa5b80cde30 con 0x5598a4313ce0 2024-04-24T09:07:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.198+0000 7fa5d6b98740 1 -- 172.21.15.69:0/326412880 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b8015020 con 0x5598a4313ce0 2024-04-24T09:07:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.198+0000 7fa5b2a00640 1 -- 172.21.15.69:0/326412880 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa5b8020070 con 0x5598a4313ce0 2024-04-24T09:07:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.199+0000 7fa5d6b98740 10 client.15756 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.199+0000 7fa5d6b98740 1 -- 172.21.15.69:0/326412880 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5598a4318e00 con 0x5598a4313ce0 2024-04-24T09:07:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.199+0000 7fa5b2a00640 1 -- 172.21.15.69:0/326412880 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5b8015020 con 0x5598a4313ce0 2024-04-24T09:07:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.199+0000 7fa5b2a00640 1 -- 172.21.15.69:0/326412880 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fa5b8098070 con 0x5598a4313ce0 2024-04-24T09:07:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.199+0000 7fa5d6b98740 10 client.15756 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.199+0000 7fa5d6b98740 10 client.15756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.199+0000 7fa5d6b98740 20 client.15756 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.199+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/326412880 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5598a439d850 0x5598a439fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.199+0000 7fa5d6b98740 4 client.15756 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:07:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.199+0000 7fa5d6b98740 1 -- 172.21.15.69:0/326412880 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5598a4281190 con 0x5598a439d850 2024-04-24T09:07:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.200+0000 7fa5d1200640 1 --2- 172.21.15.69:0/326412880 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5598a439d850 0x5598a439fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.200+0000 7fa5d1200640 1 --2- 172.21.15.69:0/326412880 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5598a439d850 0x5598a439fc40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=381e9d2fe349bcbd server_cookie=73be5a4a5d3bf801 in_seq=0 out_seq=0 2024-04-24T09:07:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.200+0000 7fa5b2a00640 10 client.15756 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.211+0000 7fa5b2a00640 1 -- 172.21.15.69:0/326412880 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5598a4281190 con 0x5598a439d850 2024-04-24T09:07:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.211+0000 7fa5b2a00640 10 client.15756 handle_command_reply: tid=0 2024-04-24T09:07:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.211+0000 7fa5d6b98740 1 client.15756 shutdown 2024-04-24T09:07:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.211+0000 7fa5d6b98740 1 -- 172.21.15.69:0/326412880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 msgr2=0x5598a4317fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.211+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/326412880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a4317fc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa5b809c020 tx=0x7fa5b809b530 comp rx=0 tx=0).stop 2024-04-24T09:07:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.211+0000 7fa5d6b98740 1 -- 172.21.15.69:0/326412880 shutdown_connections 2024-04-24T09:07:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.211+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/326412880 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5598a439d850 0x5598a439fc40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.211+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/326412880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598a431e5a0 0x5598a43185d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.211+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/326412880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5598a4313ce0 0x5598a4317fc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.211+0000 7fa5d6b98740 1 --2- 172.21.15.69:0/326412880 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5598a43133c0 0x5598a4326a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.211+0000 7fa5b2a00640 0 client.15756 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.211+0000 7fa5d6b98740 1 -- 172.21.15.69:0/326412880 >> 172.21.15.69:0/326412880 conn(0x5598a430f8d0 msgr2=0x5598a431bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.211+0000 7fa5d6b98740 1 -- 172.21.15.69:0/326412880 shutdown_connections 2024-04-24T09:07:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.211+0000 7fa5d6b98740 1 -- 172.21.15.69:0/326412880 wait complete. 2024-04-24T09:07:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.211+0000 7fa5d6b98740 20 client.15756 trim_cache size 0 max 16384 2024-04-24T09:07:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:07:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:07:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:07:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:07:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.212+0000 7fa5d2600640 1 -- 172.21.15.69:0/2924878389 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa5b40a3e20 msgr2=0x7fa5b40a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.212+0000 7fa5d2600640 1 --2- 172.21.15.69:0/2924878389 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa5b40a3e20 0x7fa5b40a62e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4005e30 tx=0x7fa5c4005d80 comp rx=0 tx=0).stop 2024-04-24T09:07:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.212+0000 7fa5d2600640 1 -- 172.21.15.69:0/2924878389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5cc1038e0 msgr2=0x7fa5cc1a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.212+0000 7fa5d2600640 1 --2- 172.21.15.69:0/2924878389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5cc1038e0 0x7fa5cc1a0a90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800d740 tx=0x7fa5b800dc00 comp rx=0 tx=0).stop 2024-04-24T09:07:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.212+0000 7fa5d2600640 1 -- 172.21.15.69:0/2924878389 shutdown_connections 2024-04-24T09:07:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.212+0000 7fa5d2600640 1 --2- 172.21.15.69:0/2924878389 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa5b40a3e20 0x7fa5b40a62e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.212+0000 7fa5d2600640 1 --2- 172.21.15.69:0/2924878389 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5cc10c3e0 0x7fa5cc19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.212+0000 7fa5d2600640 1 --2- 172.21.15.69:0/2924878389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5cc1038e0 0x7fa5cc1a0a90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.212+0000 7fa5d2600640 1 --2- 172.21.15.69:0/2924878389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc102fc0 0x7fa5cc1a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.212+0000 7fa5d2600640 1 -- 172.21.15.69:0/2924878389 >> 172.21.15.69:0/2924878389 conn(0x7fa5cc0fc820 msgr2=0x7fa5cc101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.212+0000 7fa5d2600640 1 -- 172.21.15.69:0/2924878389 shutdown_connections 2024-04-24T09:07:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:41.212+0000 7fa5d2600640 1 -- 172.21.15.69:0/2924878389 wait complete. 2024-04-24T09:07:41.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:41 smithi069 ceph-mon[28170]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-04-24T09:07:41.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:41 smithi130 ceph-mon[31553]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-04-24T09:07:42.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:41 smithi081 ceph-mon[30961]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-04-24T09:07:43.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:07:43.023 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:07:43.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.237+0000 7f0dd5a00640 1 -- 172.21.15.69:0/1176097275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0104e00 msgr2=0x7f0dd0108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.237+0000 7f0dd5a00640 1 --2- 172.21.15.69:0/1176097275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0104e00 0x7f0dd0108e40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009900 tx=0x7f0db802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.237+0000 7f0dd5a00640 1 -- 172.21.15.69:0/1176097275 shutdown_connections 2024-04-24T09:07:43.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.237+0000 7f0dd5a00640 1 --2- 172.21.15.69:0/1176097275 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd0109520 0x7f0dd01110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.237+0000 7f0dd5a00640 1 --2- 172.21.15.69:0/1176097275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0104e00 0x7f0dd0108e40 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.237+0000 7f0dd5a00640 1 --2- 172.21.15.69:0/1176097275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd0104450 0x7f0dd0104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.237+0000 7f0dd5a00640 1 -- 172.21.15.69:0/1176097275 >> 172.21.15.69:0/1176097275 conn(0x7f0dd00ffc70 msgr2=0x7f0dd0102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:43.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.237+0000 7f0dd5a00640 1 -- 172.21.15.69:0/1176097275 shutdown_connections 2024-04-24T09:07:43.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.237+0000 7f0dd5a00640 1 -- 172.21.15.69:0/1176097275 wait complete. 2024-04-24T09:07:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.238+0000 7f0dd5a00640 1 Processor -- start 2024-04-24T09:07:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.238+0000 7f0dd5a00640 1 -- start start 2024-04-24T09:07:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.238+0000 7f0dd5a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd0104450 0x7f0dd01a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.238+0000 7f0dd5a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0109520 0x7f0dd01a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.238+0000 7f0dd5a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd019a7b0 0x7f0dd019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.238+0000 7f0dd5a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd0114490 con 0x7f0dd0109520 2024-04-24T09:07:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.238+0000 7f0dd5a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd0114310 con 0x7f0dd0104450 2024-04-24T09:07:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.238+0000 7f0dd5a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd0114610 con 0x7f0dd019a7b0 2024-04-24T09:07:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.239+0000 7f0dc6000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0109520 0x7f0dd01a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.239+0000 7f0dc6000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0109520 0x7f0dd01a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35114/0 (socket says 172.21.15.69:35114) 2024-04-24T09:07:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.239+0000 7f0dc6000640 1 -- 172.21.15.69:0/3931185913 learned_addr learned my addr 172.21.15.69:0/3931185913 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.239+0000 7f0dcf400640 1 --2- 172.21.15.69:0/3931185913 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd019a7b0 0x7f0dd019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.239+0000 7f0dcea00640 1 --2- 172.21.15.69:0/3931185913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd0104450 0x7f0dd01a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.239+0000 7f0dc6000640 1 -- 172.21.15.69:0/3931185913 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd019a7b0 msgr2=0x7f0dd019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.239+0000 7f0dc6000640 1 --2- 172.21.15.69:0/3931185913 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd019a7b0 0x7f0dd019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.239+0000 7f0dc6000640 1 -- 172.21.15.69:0/3931185913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd0104450 msgr2=0x7f0dd01a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.239+0000 7f0dc6000640 1 --2- 172.21.15.69:0/3931185913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd0104450 0x7f0dd01a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.239+0000 7f0dc6000640 1 -- 172.21.15.69:0/3931185913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd019b3b0 con 0x7f0dd0109520 2024-04-24T09:07:43.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.239+0000 7f0dc6000640 1 --2- 172.21.15.69:0/3931185913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0109520 0x7f0dd01a0be0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0db8031440 tx=0x7f0db8003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:43.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.240+0000 7f0dc7e00640 1 -- 172.21.15.69:0/3931185913 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db803f070 con 0x7f0dd0109520 2024-04-24T09:07:43.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.240+0000 7f0dc7e00640 1 -- 172.21.15.69:0/3931185913 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0db8031d60 con 0x7f0dd0109520 2024-04-24T09:07:43.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.240+0000 7f0dd5a00640 1 -- 172.21.15.69:0/3931185913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd019b640 con 0x7f0dd0109520 2024-04-24T09:07:43.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.240+0000 7f0dc7e00640 1 -- 172.21.15.69:0/3931185913 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8035030 con 0x7f0dd0109520 2024-04-24T09:07:43.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.240+0000 7f0dd5a00640 1 -- 172.21.15.69:0/3931185913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd01a7780 con 0x7f0dd0109520 2024-04-24T09:07:43.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.244+0000 7f0dd89e2740 1 Processor -- start 2024-04-24T09:07:43.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.244+0000 7f0dd89e2740 1 -- start start 2024-04-24T09:07:43.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.244+0000 7f0dc7e00640 1 -- 172.21.15.69:0/3931185913 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f0db8042070 con 0x7f0dd0109520 2024-04-24T09:07:43.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.245+0000 7f0dd89e2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565256451860 0x565256451c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.245+0000 7f0dd89e2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565256452180 0x56525645c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.245+0000 7f0dd89e2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56525645c850 0x56525645ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.245+0000 7f0dd89e2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5652563b71f0 con 0x56525645c850 2024-04-24T09:07:43.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.245+0000 7f0dd89e2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56525635d350 con 0x565256451860 2024-04-24T09:07:43.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.245+0000 7f0dd89e2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5652563b31b0 con 0x565256452180 2024-04-24T09:07:43.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.245+0000 7f0dc6000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56525645c850 0x56525645ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.245+0000 7f0dc6000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56525645c850 0x56525645ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35122/0 (socket says 172.21.15.69:35122) 2024-04-24T09:07:43.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.245+0000 7f0dc6000640 1 -- 172.21.15.69:0/854723681 learned_addr learned my addr 172.21.15.69:0/854723681 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:43.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.245+0000 7f0dcf400640 1 --2- 172.21.15.69:0/854723681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565256451860 0x565256451c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.245+0000 7f0dcea00640 1 --2- 172.21.15.69:0/854723681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565256452180 0x56525645c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.245+0000 7f0dc6000640 1 -- 172.21.15.69:0/854723681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565256452180 msgr2=0x56525645c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:43.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.246+0000 7f0dc6000640 1 --2- 172.21.15.69:0/854723681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565256452180 0x56525645c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.246+0000 7f0dc6000640 1 -- 172.21.15.69:0/854723681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565256451860 msgr2=0x565256451c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.246+0000 7f0dc6000640 1 --2- 172.21.15.69:0/854723681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565256451860 0x565256451c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.246+0000 7f0dc6000640 1 -- 172.21.15.69:0/854723681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652563b1f50 con 0x56525645c850 2024-04-24T09:07:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.246+0000 7f0dcf400640 1 --2- 172.21.15.69:0/854723681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565256451860 0x565256451c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.246+0000 7f0dc6000640 1 --2- 172.21.15.69:0/854723681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56525645c850 0x56525645ec40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0db80ee8e0 tx=0x7f0db80f7c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6895851abe95d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.246+0000 7f0dbfe00640 1 -- 172.21.15.69:0/854723681 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8100070 con 0x56525645c850 2024-04-24T09:07:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.246+0000 7f0dbfe00640 1 -- 172.21.15.69:0/854723681 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0db80f9d00 con 0x56525645c850 2024-04-24T09:07:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.246+0000 7f0dbfe00640 1 -- 172.21.15.69:0/854723681 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db80f3750 con 0x56525645c850 2024-04-24T09:07:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.246+0000 7f0dc7e00640 1 --2- 172.21.15.69:0/3931185913 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f0db40a3e30 0x7f0db40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.246+0000 7f0dc7e00640 1 -- 172.21.15.69:0/3931185913 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f0db80f15f0 con 0x7f0dd0109520 2024-04-24T09:07:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.247+0000 7f0dcf400640 1 --2- 172.21.15.69:0/3931185913 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f0db40a3e30 0x7f0db40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.247+0000 7f0dd89e2740 1 -- 172.21.15.69:0/854723681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56525645c850 msgr2=0x56525645ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.247+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/854723681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56525645c850 0x56525645ec40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0db80ee8e0 tx=0x7f0db80f7c20 comp rx=0 tx=0).stop 2024-04-24T09:07:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.247+0000 7f0dd89e2740 1 -- 172.21.15.69:0/854723681 shutdown_connections 2024-04-24T09:07:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.247+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/854723681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56525645c850 0x56525645ec40 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.247+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/854723681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565256452180 0x56525645c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.247+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/854723681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565256451860 0x565256451c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.247+0000 7f0dd89e2740 1 -- 172.21.15.69:0/854723681 >> 172.21.15.69:0/854723681 conn(0x56525644db80 msgr2=0x56525644df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:43.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.247+0000 7f0dd89e2740 1 -- 172.21.15.69:0/854723681 shutdown_connections 2024-04-24T09:07:43.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.247+0000 7f0dcf400640 1 --2- 172.21.15.69:0/3931185913 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f0db40a3e30 0x7f0db40a62f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80073f0 tx=0x7f0dc8007380 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:43.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.248+0000 7f0dd89e2740 1 -- 172.21.15.69:0/854723681 wait complete. 2024-04-24T09:07:43.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.248+0000 7f0dd89e2740 1 -- start start 2024-04-24T09:07:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.248+0000 7f0dd89e2740 1 Processor -- start 2024-04-24T09:07:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.249+0000 7f0dd89e2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565256451860 0x565256459770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.249+0000 7f0dd89e2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565256452180 0x565256459cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.249+0000 7f0dd89e2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56525645c850 0x56525645a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.249+0000 7f0dd89e2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56525645f400 con 0x565256451860 2024-04-24T09:07:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.249+0000 7f0dd89e2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x565256370690 con 0x565256452180 2024-04-24T09:07:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.249+0000 7f0dd89e2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56525645f580 con 0x56525645c850 2024-04-24T09:07:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.249+0000 7f0dd89e2740 10 client.0 _ll_register_callbacks cb 0x5652563ac650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.249+0000 7f0dcea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565256451860 0x565256459770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.249+0000 7f0dcea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565256451860 0x565256459770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35128/0 (socket says 172.21.15.69:35128) 2024-04-24T09:07:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.249+0000 7f0dcea00640 1 -- 172.21.15.69:0/3894840807 learned_addr learned my addr 172.21.15.69:0/3894840807 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.249+0000 7f0dc6000640 1 --2- 172.21.15.69:0/3894840807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565256452180 0x565256459cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.249+0000 7f0dcea00640 1 -- 172.21.15.69:0/3894840807 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56525645c850 msgr2=0x56525645a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.249+0000 7f0dcea00640 1 --2- 172.21.15.69:0/3894840807 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56525645c850 0x56525645a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.249+0000 7f0dcea00640 1 -- 172.21.15.69:0/3894840807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565256452180 msgr2=0x565256459cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.249+0000 7f0dcea00640 1 --2- 172.21.15.69:0/3894840807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565256452180 0x565256459cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.249+0000 7f0dcea00640 1 -- 172.21.15.69:0/3894840807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56525636f920 con 0x565256451860 2024-04-24T09:07:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.250+0000 7f0dcea00640 1 --2- 172.21.15.69:0/3894840807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565256451860 0x565256459770 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0010840 tx=0x7f0dc0010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458e26dc58fb1d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.250+0000 7f0dbcc00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.250+0000 7f0dbcc00640 1 -- 172.21.15.69:0/3894840807 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc000cab0 con 0x565256451860 2024-04-24T09:07:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.250+0000 7f0dbcc00640 1 -- 172.21.15.69:0/3894840807 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f0dc000cc50 con 0x565256451860 2024-04-24T09:07:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.250+0000 7f0dd89e2740 1 -- 172.21.15.69:0/3894840807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0db80f9d00 con 0x565256451860 2024-04-24T09:07:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.250+0000 7f0dbcc00640 1 -- 172.21.15.69:0/3894840807 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0dc000ce70 con 0x565256451860 2024-04-24T09:07:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.250+0000 7f0dd89e2740 10 client.15768 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.250+0000 7f0dd89e2740 1 -- 172.21.15.69:0/3894840807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5652561937d0 con 0x565256451860 2024-04-24T09:07:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.250+0000 7f0dbcc00640 1 -- 172.21.15.69:0/3894840807 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0db80f9d00 con 0x565256451860 2024-04-24T09:07:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.251+0000 7f0dbcc00640 1 -- 172.21.15.69:0/3894840807 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0dc001a550 con 0x565256451860 2024-04-24T09:07:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.251+0000 7f0dd89e2740 10 client.15768 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.251+0000 7f0dd89e2740 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.251+0000 7f0dd89e2740 20 client.15768 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.251+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/3894840807 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x56525646c950 0x56525646edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.251+0000 7f0dd89e2740 4 client.15768 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.251+0000 7f0dd89e2740 1 -- 172.21.15.69:0/3894840807 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652563c4c10 con 0x56525646c950 2024-04-24T09:07:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.252+0000 7f0dc6000640 1 --2- 172.21.15.69:0/3894840807 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x56525646c950 0x56525646edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.252+0000 7f0dc6000640 1 --2- 172.21.15.69:0/3894840807 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x56525646c950 0x56525646edf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb283d8611211e15 server_cookie=c586216a3be241bd in_seq=0 out_seq=0 2024-04-24T09:07:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.252+0000 7f0dbcc00640 10 client.15768 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:43.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.254+0000 7f0dbcc00640 1 -- 172.21.15.69:0/3894840807 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5652563c4c10 con 0x56525646c950 2024-04-24T09:07:43.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.254+0000 7f0dbcc00640 10 client.15768 handle_command_reply: tid=0 2024-04-24T09:07:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.254+0000 7f0dd89e2740 1 client.15768 shutdown 2024-04-24T09:07:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.254+0000 7f0dd89e2740 1 -- 172.21.15.69:0/3894840807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565256451860 msgr2=0x565256459770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.254+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/3894840807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565256451860 0x565256459770 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0010840 tx=0x7f0dc0010d00 comp rx=0 tx=0).stop 2024-04-24T09:07:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.254+0000 7f0dd89e2740 1 -- 172.21.15.69:0/3894840807 shutdown_connections 2024-04-24T09:07:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.254+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/3894840807 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x56525646c950 0x56525646edf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.255+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/3894840807 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56525645c850 0x56525645a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.255+0000 7f0dbcc00640 0 client.15768 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.255+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/3894840807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565256452180 0x565256459cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.255+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/3894840807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565256451860 0x565256459770 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.255+0000 7f0dd89e2740 1 -- 172.21.15.69:0/3894840807 >> 172.21.15.69:0/3894840807 conn(0x56525644db80 msgr2=0x565256453980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.255+0000 7f0dd89e2740 1 -- 172.21.15.69:0/3894840807 shutdown_connections 2024-04-24T09:07:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.255+0000 7f0dd89e2740 1 -- 172.21.15.69:0/3894840807 wait complete. 2024-04-24T09:07:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.255+0000 7f0dd89e2740 20 client.15768 trim_cache size 0 max 16384 2024-04-24T09:07:43.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.274+0000 7f0dd89e2740 1 Processor -- start 2024-04-24T09:07:43.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.274+0000 7f0dd89e2740 1 -- start start 2024-04-24T09:07:43.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.274+0000 7f0dd89e2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565256452180 0x5652564586b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.274+0000 7f0dd89e2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56525645c850 0x565256468230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.274+0000 7f0dd89e2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565256453fc0 0x5652564746c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.274+0000 7f0dd89e2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01b3c80 con 0x56525645c850 2024-04-24T09:07:43.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.274+0000 7f0dd89e2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01b1550 con 0x565256452180 2024-04-24T09:07:43.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.274+0000 7f0dd89e2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01af3a0 con 0x565256453fc0 2024-04-24T09:07:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.275+0000 7f0dc6000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56525645c850 0x565256468230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.275+0000 7f0dc6000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56525645c850 0x565256468230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35138/0 (socket says 172.21.15.69:35138) 2024-04-24T09:07:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.275+0000 7f0dc6000640 1 -- 172.21.15.69:0/1154311973 learned_addr learned my addr 172.21.15.69:0/1154311973 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.275+0000 7f0dc6000640 1 -- 172.21.15.69:0/1154311973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565256453fc0 msgr2=0x5652564746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.275+0000 7f0dc6000640 1 --2- 172.21.15.69:0/1154311973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565256453fc0 0x5652564746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.275+0000 7f0dc6000640 1 -- 172.21.15.69:0/1154311973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565256452180 msgr2=0x5652564586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.275+0000 7f0dcea00640 1 --2- 172.21.15.69:0/1154311973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565256452180 0x5652564586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.275+0000 7f0dc6000640 1 --2- 172.21.15.69:0/1154311973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565256452180 0x5652564586b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.275+0000 7f0dc6000640 1 -- 172.21.15.69:0/1154311973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652560c93f0 con 0x56525645c850 2024-04-24T09:07:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.275+0000 7f0dc6000640 1 --2- 172.21.15.69:0/1154311973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56525645c850 0x565256468230 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0db8005ab0 tx=0x7f0db802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229b4544f554ba18 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.275+0000 7f0dbf400640 1 -- 172.21.15.69:0/1154311973 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8049040 con 0x56525645c850 2024-04-24T09:07:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.275+0000 7f0dbf400640 1 -- 172.21.15.69:0/1154311973 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0db80f7e70 con 0x56525645c850 2024-04-24T09:07:43.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.275+0000 7f0dbf400640 1 -- 172.21.15.69:0/1154311973 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8100040 con 0x56525645c850 2024-04-24T09:07:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.276+0000 7f0dd89e2740 1 -- 172.21.15.69:0/1154311973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56525645c850 msgr2=0x565256468230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.276+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/1154311973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56525645c850 0x565256468230 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0db8005ab0 tx=0x7f0db802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.276+0000 7f0dd89e2740 1 -- 172.21.15.69:0/1154311973 shutdown_connections 2024-04-24T09:07:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.276+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/1154311973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565256453fc0 0x5652564746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.276+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/1154311973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56525645c850 0x565256468230 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.276+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/1154311973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565256452180 0x5652564586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.276+0000 7f0dd89e2740 1 -- 172.21.15.69:0/1154311973 >> 172.21.15.69:0/1154311973 conn(0x56525644db80 msgr2=0x56525644cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.276+0000 7f0dd89e2740 1 -- 172.21.15.69:0/1154311973 shutdown_connections 2024-04-24T09:07:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.276+0000 7f0dd89e2740 1 -- 172.21.15.69:0/1154311973 wait complete. 2024-04-24T09:07:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.277+0000 7f0dd89e2740 1 -- start start 2024-04-24T09:07:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.277+0000 7f0dd89e2740 1 Processor -- start 2024-04-24T09:07:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.277+0000 7f0dd89e2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565256452180 0x5652564607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.277+0000 7f0dd89e2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565256453fc0 0x565256460d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.277+0000 7f0dd89e2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56525645c850 0x5652564da740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.277+0000 7f0dd89e2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01afa10 con 0x565256453fc0 2024-04-24T09:07:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.277+0000 7f0dd89e2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01b0e80 con 0x56525645c850 2024-04-24T09:07:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.277+0000 7f0dd89e2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01b00e0 con 0x565256452180 2024-04-24T09:07:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.277+0000 7f0dd89e2740 10 client.0 _ll_register_callbacks cb 0x7f0dc000e3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.278+0000 7f0dc6000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565256453fc0 0x565256460d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.278+0000 7f0dc6000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565256453fc0 0x565256460d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35154/0 (socket says 172.21.15.69:35154) 2024-04-24T09:07:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.278+0000 7f0dc6000640 1 -- 172.21.15.69:0/1489870448 learned_addr learned my addr 172.21.15.69:0/1489870448 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.278+0000 7f0dcea00640 1 --2- 172.21.15.69:0/1489870448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565256452180 0x5652564607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.278+0000 7f0dc6000640 1 -- 172.21.15.69:0/1489870448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565256452180 msgr2=0x5652564607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.278+0000 7f0dcf400640 1 --2- 172.21.15.69:0/1489870448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56525645c850 0x5652564da740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.278+0000 7f0dc6000640 1 --2- 172.21.15.69:0/1489870448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565256452180 0x5652564607f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.278+0000 7f0dc6000640 1 -- 172.21.15.69:0/1489870448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56525645c850 msgr2=0x5652564da740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.278+0000 7f0dc6000640 1 --2- 172.21.15.69:0/1489870448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56525645c850 0x5652564da740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.278+0000 7f0dc6000640 1 -- 172.21.15.69:0/1489870448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652560890f0 con 0x565256453fc0 2024-04-24T09:07:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.278+0000 7f0dcf400640 1 --2- 172.21.15.69:0/1489870448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56525645c850 0x5652564da740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.278+0000 7f0dc6000640 1 --2- 172.21.15.69:0/1489870448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565256453fc0 0x565256460d30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0db80ef000 tx=0x7f0db8031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83801edc92d613ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.278+0000 7f0dbcc00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.278+0000 7f0dbcc00640 1 -- 172.21.15.69:0/1489870448 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db80d5020 con 0x565256453fc0 2024-04-24T09:07:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.279+0000 7f0dbcc00640 1 -- 172.21.15.69:0/1489870448 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f0db80bb3e0 con 0x565256453fc0 2024-04-24T09:07:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.279+0000 7f0dd89e2740 1 -- 172.21.15.69:0/1489870448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0db80f7e70 con 0x565256453fc0 2024-04-24T09:07:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.279+0000 7f0dbcc00640 1 -- 172.21.15.69:0/1489870448 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0db8049070 con 0x565256453fc0 2024-04-24T09:07:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.279+0000 7f0dd89e2740 10 client.15774 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.279+0000 7f0dd89e2740 1 -- 172.21.15.69:0/1489870448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56525637c810 con 0x565256453fc0 2024-04-24T09:07:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.279+0000 7f0dbcc00640 1 -- 172.21.15.69:0/1489870448 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0db80f7e70 con 0x565256453fc0 2024-04-24T09:07:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.279+0000 7f0dbcc00640 1 -- 172.21.15.69:0/1489870448 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0db80b9750 con 0x565256453fc0 2024-04-24T09:07:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.279+0000 7f0dd89e2740 10 client.15774 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.279+0000 7f0dd89e2740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.279+0000 7f0dd89e2740 20 client.15774 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.279+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/1489870448 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5652564dbef0 0x5652564de2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.279+0000 7f0dd89e2740 4 client.15774 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:07:43.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.279+0000 7f0dd89e2740 1 -- 172.21.15.69:0/1489870448 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5652563bea80 con 0x5652564dbef0 2024-04-24T09:07:43.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.280+0000 7f0dcea00640 1 --2- 172.21.15.69:0/1489870448 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5652564dbef0 0x5652564de2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.281+0000 7f0dcea00640 1 --2- 172.21.15.69:0/1489870448 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5652564dbef0 0x5652564de2e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74dc867a20910d61 server_cookie=68c1f4a70843e0f9 in_seq=0 out_seq=0 2024-04-24T09:07:43.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.281+0000 7f0dbcc00640 10 client.15774 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:43.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.281+0000 7f0dbcc00640 1 -- 172.21.15.69:0/1489870448 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5652563bea80 con 0x5652564dbef0 2024-04-24T09:07:43.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.281+0000 7f0dbcc00640 10 client.15774 handle_command_reply: tid=0 2024-04-24T09:07:43.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.281+0000 7f0dd89e2740 1 client.15774 shutdown 2024-04-24T09:07:43.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.281+0000 7f0dd89e2740 1 -- 172.21.15.69:0/1489870448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565256453fc0 msgr2=0x565256460d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.281+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/1489870448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565256453fc0 0x565256460d30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0db80ef000 tx=0x7f0db8031f10 comp rx=0 tx=0).stop 2024-04-24T09:07:43.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.282+0000 7f0dd89e2740 1 -- 172.21.15.69:0/1489870448 shutdown_connections 2024-04-24T09:07:43.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.282+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/1489870448 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5652564dbef0 0x5652564de2e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.282+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/1489870448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56525645c850 0x5652564da740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.282+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/1489870448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565256453fc0 0x565256460d30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.282+0000 7f0dd89e2740 1 --2- 172.21.15.69:0/1489870448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565256452180 0x5652564607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.282+0000 7f0dd89e2740 1 -- 172.21.15.69:0/1489870448 >> 172.21.15.69:0/1489870448 conn(0x56525644db80 msgr2=0x565256452fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:43.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.282+0000 7f0dbcc00640 0 client.15774 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:43.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.282+0000 7f0dd89e2740 1 -- 172.21.15.69:0/1489870448 shutdown_connections 2024-04-24T09:07:43.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.282+0000 7f0dd89e2740 1 -- 172.21.15.69:0/1489870448 wait complete. 2024-04-24T09:07:43.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.282+0000 7f0dd89e2740 20 client.15774 trim_cache size 0 max 16384 2024-04-24T09:07:43.284 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:07:43.284 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-04-24T09:07:43.284 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-04-24T09:07:43.285 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:07:43.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.282+0000 7f0dd5a00640 1 -- 172.21.15.69:0/3931185913 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f0db40a3e30 msgr2=0x7f0db40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.282+0000 7f0dd5a00640 1 --2- 172.21.15.69:0/3931185913 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f0db40a3e30 0x7f0db40a62f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80073f0 tx=0x7f0dc8007380 comp rx=0 tx=0).stop 2024-04-24T09:07:43.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.282+0000 7f0dd5a00640 1 -- 172.21.15.69:0/3931185913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0109520 msgr2=0x7f0dd01a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.282+0000 7f0dd5a00640 1 --2- 172.21.15.69:0/3931185913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0109520 0x7f0dd01a0be0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0db8031440 tx=0x7f0db8003d50 comp rx=0 tx=0).stop 2024-04-24T09:07:43.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.283+0000 7f0dd5a00640 1 -- 172.21.15.69:0/3931185913 shutdown_connections 2024-04-24T09:07:43.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.283+0000 7f0dd5a00640 1 --2- 172.21.15.69:0/3931185913 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f0db40a3e30 0x7f0db40a62f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.283+0000 7f0dd5a00640 1 --2- 172.21.15.69:0/3931185913 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd019a7b0 0x7f0dd019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.283+0000 7f0dd5a00640 1 --2- 172.21.15.69:0/3931185913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dd0109520 0x7f0dd01a0be0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.283+0000 7f0dd5a00640 1 --2- 172.21.15.69:0/3931185913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dd0104450 0x7f0dd01a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.283+0000 7f0dd5a00640 1 -- 172.21.15.69:0/3931185913 >> 172.21.15.69:0/3931185913 conn(0x7f0dd00ffc70 msgr2=0x7f0dd0101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:43.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.283+0000 7f0dd5a00640 1 -- 172.21.15.69:0/3931185913 shutdown_connections 2024-04-24T09:07:43.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.283+0000 7f0dd5a00640 1 -- 172.21.15.69:0/3931185913 wait complete. 2024-04-24T09:07:43.298 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:07:43.299 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:07:43.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.494+0000 7f3763200640 1 -- 172.21.15.69:0/3730283480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f376410d3c0 msgr2=0x7f376410d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.494+0000 7f3763200640 1 --2- 172.21.15.69:0/3730283480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f376410d3c0 0x7f376410d7a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3754009900 tx=0x7f375402a480 comp rx=0 tx=0).stop 2024-04-24T09:07:43.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.494+0000 7f3763200640 1 -- 172.21.15.69:0/3730283480 shutdown_connections 2024-04-24T09:07:43.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.494+0000 7f3763200640 1 --2- 172.21.15.69:0/3730283480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3764108e40 0x7f37641152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.494+0000 7f3763200640 1 --2- 172.21.15.69:0/3730283480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37641084a0 0x7f3764108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.494+0000 7f3763200640 1 --2- 172.21.15.69:0/3730283480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f376410d3c0 0x7f376410d7a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.494+0000 7f3763200640 1 -- 172.21.15.69:0/3730283480 >> 172.21.15.69:0/3730283480 conn(0x7f3764077b80 msgr2=0x7f3764079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:43.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.494+0000 7f3763200640 1 -- 172.21.15.69:0/3730283480 shutdown_connections 2024-04-24T09:07:43.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.494+0000 7f3763200640 1 -- 172.21.15.69:0/3730283480 wait complete. 2024-04-24T09:07:43.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.495+0000 7f3763200640 1 Processor -- start 2024-04-24T09:07:43.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.495+0000 7f3763200640 1 -- start start 2024-04-24T09:07:43.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.495+0000 7f3763200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37641084a0 0x7f37641a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.495+0000 7f3763200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3764108e40 0x7f37641a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.495+0000 7f3763200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f376410d3c0 0x7f376419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.495+0000 7f3763200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3764118700 con 0x7f37641084a0 2024-04-24T09:07:43.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.495+0000 7f3763200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3764118580 con 0x7f376410d3c0 2024-04-24T09:07:43.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.495+0000 7f3763200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3764118880 con 0x7f3764108e40 2024-04-24T09:07:43.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.496+0000 7f3761e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37641084a0 0x7f37641a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.496+0000 7f3761e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37641084a0 0x7f37641a4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35174/0 (socket says 172.21.15.69:35174) 2024-04-24T09:07:43.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.496+0000 7f3761e00640 1 -- 172.21.15.69:0/3032986155 learned_addr learned my addr 172.21.15.69:0/3032986155 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:43.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.496+0000 7f3761400640 1 --2- 172.21.15.69:0/3032986155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3764108e40 0x7f37641a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.496+0000 7f3762800640 1 --2- 172.21.15.69:0/3032986155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f376410d3c0 0x7f376419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.496+0000 7f3761e00640 1 -- 172.21.15.69:0/3032986155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3764108e40 msgr2=0x7f37641a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.496+0000 7f3761e00640 1 --2- 172.21.15.69:0/3032986155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3764108e40 0x7f37641a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.496+0000 7f3761e00640 1 -- 172.21.15.69:0/3032986155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f376410d3c0 msgr2=0x7f376419ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.496+0000 7f3761e00640 1 --2- 172.21.15.69:0/3032986155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f376410d3c0 0x7f376419ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.496+0000 7f3761e00640 1 -- 172.21.15.69:0/3032986155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376419f1c0 con 0x7f37641084a0 2024-04-24T09:07:43.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.496+0000 7f3761e00640 1 --2- 172.21.15.69:0/3032986155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37641084a0 0x7f37641a4890 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3754005ab0 tx=0x7f3754003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:43.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.497+0000 7f375aa00640 1 -- 172.21.15.69:0/3032986155 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375403f070 con 0x7f37641084a0 2024-04-24T09:07:43.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.497+0000 7f375aa00640 1 -- 172.21.15.69:0/3032986155 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3754031d60 con 0x7f37641084a0 2024-04-24T09:07:43.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.497+0000 7f3763200640 1 -- 172.21.15.69:0/3032986155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376419f450 con 0x7f37641084a0 2024-04-24T09:07:43.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.497+0000 7f375aa00640 1 -- 172.21.15.69:0/3032986155 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754035030 con 0x7f37641084a0 2024-04-24T09:07:43.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.497+0000 7f3763200640 1 -- 172.21.15.69:0/3032986155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37641ab750 con 0x7f37641084a0 2024-04-24T09:07:43.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.498+0000 7f3763200640 1 -- 172.21.15.69:0/3032986155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3764069190 con 0x7f37641084a0 2024-04-24T09:07:43.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.500+0000 7f375aa00640 1 -- 172.21.15.69:0/3032986155 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f3754042070 con 0x7f37641084a0 2024-04-24T09:07:43.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.501+0000 7f375aa00640 1 --2- 172.21.15.69:0/3032986155 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37480a3e30 0x7f37480a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.502+0000 7f375aa00640 1 -- 172.21.15.69:0/3032986155 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f37540f18a0 con 0x7f37641084a0 2024-04-24T09:07:43.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.502+0000 7f3761400640 1 --2- 172.21.15.69:0/3032986155 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37480a3e30 0x7f37480a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.502+0000 7f3761400640 1 --2- 172.21.15.69:0/3032986155 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37480a3e30 0x7f37480a62f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f374c005fd0 tx=0x7f374c005850 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:43.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.504+0000 7f375aa00640 1 -- 172.21.15.69:0/3032986155 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f37540bda30 con 0x7f37641084a0 2024-04-24T09:07:43.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:43 smithi130 ceph-mon[31553]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-24T09:07:43.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.742+0000 7f3763200640 1 -- 172.21.15.69:0/3032986155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3764110930 con 0x7f37641084a0 2024-04-24T09:07:43.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.743+0000 7f375aa00640 1 -- 172.21.15.69:0/3032986155 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11873 (secure 0 0 0) 0x7f37540bd330 con 0x7f37641084a0 2024-04-24T09:07:43.742 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:07:43.743 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:04:01.991559+0000","modified":"2024-04-24T09:07:02.537848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34532,"mds_1":34496,"mds_2":34460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34496":{"gid":34496,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34532":{"gid":34532,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34532,"qdb_cluster":[34532,34460,34496]},"id":1}]} 2024-04-24T09:07:43.743 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 24 2024-04-24T09:07:43.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.746+0000 7f3763200640 1 -- 172.21.15.69:0/3032986155 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37480a3e30 msgr2=0x7f37480a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.746+0000 7f3763200640 1 --2- 172.21.15.69:0/3032986155 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37480a3e30 0x7f37480a62f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f374c005fd0 tx=0x7f374c005850 comp rx=0 tx=0).stop 2024-04-24T09:07:43.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.746+0000 7f3763200640 1 -- 172.21.15.69:0/3032986155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37641084a0 msgr2=0x7f37641a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.746+0000 7f3763200640 1 --2- 172.21.15.69:0/3032986155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37641084a0 0x7f37641a4890 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3754005ab0 tx=0x7f3754003cd0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.746+0000 7f3763200640 1 -- 172.21.15.69:0/3032986155 shutdown_connections 2024-04-24T09:07:43.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.746+0000 7f3763200640 1 --2- 172.21.15.69:0/3032986155 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37480a3e30 0x7f37480a62f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.746+0000 7f3763200640 1 --2- 172.21.15.69:0/3032986155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f376410d3c0 0x7f376419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.746+0000 7f3763200640 1 --2- 172.21.15.69:0/3032986155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3764108e40 0x7f37641a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.746+0000 7f3763200640 1 --2- 172.21.15.69:0/3032986155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37641084a0 0x7f37641a4890 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.746+0000 7f3763200640 1 -- 172.21.15.69:0/3032986155 >> 172.21.15.69:0/3032986155 conn(0x7f3764077b80 msgr2=0x7f3764078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:43.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.746+0000 7f3763200640 1 -- 172.21.15.69:0/3032986155 shutdown_connections 2024-04-24T09:07:43.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.746+0000 7f3763200640 1 -- 172.21.15.69:0/3032986155 wait complete. 2024-04-24T09:07:43.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:43 smithi069 ceph-mon[28170]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-24T09:07:43.762 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:07:43.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.952+0000 7f2729400640 1 -- 172.21.15.69:0/1123482290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27241093f0 msgr2=0x7f2724110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.952+0000 7f2729400640 1 --2- 172.21.15.69:0/1123482290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27241093f0 0x7f2724110fc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f271c009ca0 tx=0x7f271c02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.952+0000 7f2729400640 1 -- 172.21.15.69:0/1123482290 shutdown_connections 2024-04-24T09:07:43.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.953+0000 7f2729400640 1 --2- 172.21.15.69:0/1123482290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27241093f0 0x7f2724110fc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.953+0000 7f2729400640 1 --2- 172.21.15.69:0/1123482290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2724104cd0 0x7f2724108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.953+0000 7f2729400640 1 --2- 172.21.15.69:0/1123482290 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2724104320 0x7f2724104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.953+0000 7f2729400640 1 -- 172.21.15.69:0/1123482290 >> 172.21.15.69:0/1123482290 conn(0x7f27240ffb40 msgr2=0x7f2724101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:43.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.953+0000 7f2729400640 1 -- 172.21.15.69:0/1123482290 shutdown_connections 2024-04-24T09:07:43.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.953+0000 7f2729400640 1 -- 172.21.15.69:0/1123482290 wait complete. 2024-04-24T09:07:43.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.953+0000 7f2729400640 1 Processor -- start 2024-04-24T09:07:43.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.954+0000 7f2729400640 1 -- start start 2024-04-24T09:07:43.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.954+0000 7f2729400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2724104320 0x7f27241a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.954+0000 7f2729400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27241093f0 0x7f27241a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.954+0000 7f2729400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f272419a630 0x7f272419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.954+0000 7f2729400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2724114390 con 0x7f272419a630 2024-04-24T09:07:43.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.954+0000 7f2729400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2724114210 con 0x7f2724104320 2024-04-24T09:07:43.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.954+0000 7f2729400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2724114510 con 0x7f27241093f0 2024-04-24T09:07:43.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.954+0000 7f2728a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f272419a630 0x7f272419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.954+0000 7f2728a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f272419a630 0x7f272419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35190/0 (socket says 172.21.15.69:35190) 2024-04-24T09:07:43.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.954+0000 7f2728a00640 1 -- 172.21.15.69:0/343190516 learned_addr learned my addr 172.21.15.69:0/343190516 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:43.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.955+0000 7f2723400640 1 --2- 172.21.15.69:0/343190516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27241093f0 0x7f27241a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.955+0000 7f2723e00640 1 --2- 172.21.15.69:0/343190516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2724104320 0x7f27241a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.955+0000 7f2728a00640 1 -- 172.21.15.69:0/343190516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27241093f0 msgr2=0x7f27241a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.955+0000 7f2728a00640 1 --2- 172.21.15.69:0/343190516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27241093f0 0x7f27241a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.955+0000 7f2728a00640 1 -- 172.21.15.69:0/343190516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2724104320 msgr2=0x7f27241a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.955+0000 7f2728a00640 1 --2- 172.21.15.69:0/343190516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2724104320 0x7f27241a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.955+0000 7f2728a00640 1 -- 172.21.15.69:0/343190516 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272419b230 con 0x7f272419a630 2024-04-24T09:07:43.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.955+0000 7f2723e00640 1 --2- 172.21.15.69:0/343190516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2724104320 0x7f27241a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:43.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.955+0000 7f2723400640 1 --2- 172.21.15.69:0/343190516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27241093f0 0x7f27241a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:43.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.955+0000 7f2728a00640 1 --2- 172.21.15.69:0/343190516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f272419a630 0x7f272419aab0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f271c031040 tx=0x7f271c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:43.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.955+0000 7f2720c00640 1 -- 172.21.15.69:0/343190516 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c004040 con 0x7f272419a630 2024-04-24T09:07:43.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.955+0000 7f2720c00640 1 -- 172.21.15.69:0/343190516 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f271c004690 con 0x7f272419a630 2024-04-24T09:07:43.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.955+0000 7f2729400640 1 -- 172.21.15.69:0/343190516 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f272419b4c0 con 0x7f272419a630 2024-04-24T09:07:43.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.955+0000 7f2720c00640 1 -- 172.21.15.69:0/343190516 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c03b410 con 0x7f272419a630 2024-04-24T09:07:43.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.956+0000 7f2729400640 1 -- 172.21.15.69:0/343190516 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27241a76b0 con 0x7f272419a630 2024-04-24T09:07:43.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.959+0000 7f272d990740 1 Processor -- start 2024-04-24T09:07:43.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.960+0000 7f2720c00640 1 -- 172.21.15.69:0/343190516 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f271c0041e0 con 0x7f272419a630 2024-04-24T09:07:43.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.960+0000 7f272d990740 1 -- start start 2024-04-24T09:07:43.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.960+0000 7f272d990740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564931792b40 0x564931792f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.960+0000 7f272d990740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.960+0000 7f272d990740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56493179db30 0x56493179ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.960+0000 7f272d990740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5649316f81d0 con 0x564931793460 2024-04-24T09:07:43.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.960+0000 7f272d990740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56493169e450 con 0x564931792b40 2024-04-24T09:07:43.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.960+0000 7f272d990740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5649316f4190 con 0x56493179db30 2024-04-24T09:07:43.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.961+0000 7f2723e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.961+0000 7f2723e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35192/0 (socket says 172.21.15.69:35192) 2024-04-24T09:07:43.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.961+0000 7f2723e00640 1 -- 172.21.15.69:0/1324183487 learned_addr learned my addr 172.21.15.69:0/1324183487 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:43.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.961+0000 7f2723400640 1 --2- 172.21.15.69:0/1324183487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564931792b40 0x564931792f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.961+0000 7f2723e00640 1 -- 172.21.15.69:0/1324183487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56493179db30 msgr2=0x56493179ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:43.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.961+0000 7f2723e00640 1 --2- 172.21.15.69:0/1324183487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56493179db30 0x56493179ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.961+0000 7f2723e00640 1 -- 172.21.15.69:0/1324183487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564931792b40 msgr2=0x564931792f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:43.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.961+0000 7f2723e00640 1 --2- 172.21.15.69:0/1324183487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564931792b40 0x564931792f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.961+0000 7f2723e00640 1 -- 172.21.15.69:0/1324183487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56493175b310 con 0x564931793460 2024-04-24T09:07:43.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.961+0000 7f2723e00640 1 --2- 172.21.15.69:0/1324183487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179d5f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f271400bb70 tx=0x7f271400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910d08d5194bece6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:43.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.961+0000 7f2720c00640 1 --2- 172.21.15.69:0/343190516 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f27080a3e30 0x7f27080a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.962+0000 7f2720c00640 1 -- 172.21.15.69:0/343190516 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f271c0f22e0 con 0x7f272419a630 2024-04-24T09:07:43.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.962+0000 7f2719200640 1 -- 172.21.15.69:0/1324183487 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714017620 con 0x564931793460 2024-04-24T09:07:43.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.962+0000 7f2719200640 1 -- 172.21.15.69:0/1324183487 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2714017c70 con 0x564931793460 2024-04-24T09:07:43.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.962+0000 7f2723400640 1 --2- 172.21.15.69:0/343190516 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f27080a3e30 0x7f27080a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.962+0000 7f2719200640 1 -- 172.21.15.69:0/1324183487 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271400fb10 con 0x564931793460 2024-04-24T09:07:43.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.962+0000 7f272d990740 1 -- 172.21.15.69:0/1324183487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 msgr2=0x56493179d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.962+0000 7f272d990740 1 --2- 172.21.15.69:0/1324183487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179d5f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f271400bb70 tx=0x7f271400d710 comp rx=0 tx=0).stop 2024-04-24T09:07:43.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.962+0000 7f2723400640 1 --2- 172.21.15.69:0/343190516 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f27080a3e30 0x7f27080a62f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2710005fd0 tx=0x7f2710005f40 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:43.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.963+0000 7f272d990740 1 -- 172.21.15.69:0/1324183487 shutdown_connections 2024-04-24T09:07:43.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.963+0000 7f272d990740 1 --2- 172.21.15.69:0/1324183487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56493179db30 0x56493179ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.963+0000 7f272d990740 1 --2- 172.21.15.69:0/1324183487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179d5f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.963+0000 7f272d990740 1 --2- 172.21.15.69:0/1324183487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564931792b40 0x564931792f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.963+0000 7f272d990740 1 -- 172.21.15.69:0/1324183487 >> 172.21.15.69:0/1324183487 conn(0x56493178ee60 msgr2=0x56493178f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:43.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.963+0000 7f272d990740 1 -- 172.21.15.69:0/1324183487 shutdown_connections 2024-04-24T09:07:43.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.963+0000 7f272d990740 1 -- 172.21.15.69:0/1324183487 wait complete. 2024-04-24T09:07:43.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.963+0000 7f272d990740 1 -- start start 2024-04-24T09:07:43.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.963+0000 7f272d990740 1 Processor -- start 2024-04-24T09:07:43.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f272d990740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564931792b40 0x56493179d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f272d990740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f272d990740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56493179db30 0x56493179ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f272d990740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5649317a09c0 con 0x564931793460 2024-04-24T09:07:43.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f272d990740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5649316b1810 con 0x56493179db30 2024-04-24T09:07:43.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f272d990740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5649317a0b40 con 0x564931792b40 2024-04-24T09:07:43.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f272d990740 10 client.0 _ll_register_callbacks cb 0x56493167f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:43.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f2723400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f2723400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35206/0 (socket says 172.21.15.69:35206) 2024-04-24T09:07:43.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f2723400640 1 -- 172.21.15.69:0/3830948425 learned_addr learned my addr 172.21.15.69:0/3830948425 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:43.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f2728a00640 1 --2- 172.21.15.69:0/3830948425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56493179db30 0x56493179ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f2723e00640 1 --2- 172.21.15.69:0/3830948425 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564931792b40 0x56493179d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f2723400640 1 -- 172.21.15.69:0/3830948425 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564931792b40 msgr2=0x56493179d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:43.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f2723400640 1 --2- 172.21.15.69:0/3830948425 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564931792b40 0x56493179d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f2723400640 1 -- 172.21.15.69:0/3830948425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56493179db30 msgr2=0x56493179ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:43.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f2723400640 1 --2- 172.21.15.69:0/3830948425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56493179db30 0x56493179ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.964+0000 7f2723400640 1 -- 172.21.15.69:0/3830948425 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649316c23f0 con 0x564931793460 2024-04-24T09:07:43.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.965+0000 7f2723400640 1 --2- 172.21.15.69:0/3830948425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179a830 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f27100116f0 tx=0x7f27100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9f2af064fe47a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:43.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.965+0000 7f270d600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:43.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.965+0000 7f270d600640 1 -- 172.21.15.69:0/3830948425 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710005500 con 0x564931793460 2024-04-24T09:07:43.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.965+0000 7f270d600640 1 -- 172.21.15.69:0/3830948425 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f2710004730 con 0x564931793460 2024-04-24T09:07:43.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.965+0000 7f272d990740 1 -- 172.21.15.69:0/3830948425 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2714017c70 con 0x564931793460 2024-04-24T09:07:43.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.965+0000 7f270d600640 1 -- 172.21.15.69:0/3830948425 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f271002f050 con 0x564931793460 2024-04-24T09:07:43.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.965+0000 7f272d990740 10 client.15792 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:43.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.965+0000 7f272d990740 1 -- 172.21.15.69:0/3830948425 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56493140a3f0 con 0x564931793460 2024-04-24T09:07:43.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.966+0000 7f270d600640 1 -- 172.21.15.69:0/3830948425 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2714017c70 con 0x564931793460 2024-04-24T09:07:43.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.966+0000 7f270d600640 1 -- 172.21.15.69:0/3830948425 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f271001caa0 con 0x564931793460 2024-04-24T09:07:43.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.966+0000 7f272d990740 10 client.15792 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:43.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.966+0000 7f272d990740 10 client.15792 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:07:43.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.966+0000 7f272d990740 20 client.15792 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:43.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.966+0000 7f272d990740 1 --2- 172.21.15.69:0/3830948425 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x5649317add90 0x5649317b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.966+0000 7f272d990740 4 client.15792 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:43.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.966+0000 7f272d990740 1 -- 172.21.15.69:0/3830948425 --> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564931705bf0 con 0x5649317add90 2024-04-24T09:07:43.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.967+0000 7f2723e00640 1 --2- 172.21.15.69:0/3830948425 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x5649317add90 0x5649317b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.968+0000 7f2723e00640 1 --2- 172.21.15.69:0/3830948425 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x5649317add90 0x5649317b01d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=265b60a8d8e0d4d9 server_cookie=9da9d717fc2be234 in_seq=0 out_seq=0 2024-04-24T09:07:43.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.968+0000 7f270d600640 10 client.15792 ms_handle_connect on v2:172.21.15.81:6834/3260829578 2024-04-24T09:07:43.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.969+0000 7f270d600640 1 -- 172.21.15.69:0/3830948425 <== mds.2 v2:172.21.15.81:6834/3260829578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564931705bf0 con 0x5649317add90 2024-04-24T09:07:43.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.969+0000 7f270d600640 10 client.15792 handle_command_reply: tid=0 2024-04-24T09:07:43.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.970+0000 7f272d990740 1 client.15792 shutdown 2024-04-24T09:07:43.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.970+0000 7f272d990740 1 -- 172.21.15.69:0/3830948425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 msgr2=0x56493179a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.970+0000 7f272d990740 1 --2- 172.21.15.69:0/3830948425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179a830 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f27100116f0 tx=0x7f27100046c0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.970+0000 7f272d990740 1 -- 172.21.15.69:0/3830948425 shutdown_connections 2024-04-24T09:07:43.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.970+0000 7f272d990740 1 --2- 172.21.15.69:0/3830948425 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x5649317add90 0x5649317b01d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.970+0000 7f272d990740 1 --2- 172.21.15.69:0/3830948425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56493179db30 0x56493179ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.970+0000 7f272d990740 1 --2- 172.21.15.69:0/3830948425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179a830 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.970+0000 7f270d600640 0 client.15792 ms_handle_reset on v2:172.21.15.81:6834/3260829578 2024-04-24T09:07:43.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.970+0000 7f272d990740 1 --2- 172.21.15.69:0/3830948425 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564931792b40 0x56493179d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.970+0000 7f272d990740 1 -- 172.21.15.69:0/3830948425 >> 172.21.15.69:0/3830948425 conn(0x56493178ee60 msgr2=0x564931794b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:43.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.970+0000 7f272d990740 1 -- 172.21.15.69:0/3830948425 shutdown_connections 2024-04-24T09:07:43.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.970+0000 7f272d990740 1 -- 172.21.15.69:0/3830948425 wait complete. 2024-04-24T09:07:43.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.970+0000 7f272d990740 20 client.15792 trim_cache size 0 max 16384 2024-04-24T09:07:43.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.984+0000 7f272d990740 1 Processor -- start 2024-04-24T09:07:43.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.984+0000 7f272d990740 1 -- start start 2024-04-24T09:07:43.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.985+0000 7f272d990740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564931792b40 0x5649317a9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.985+0000 7f272d990740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.985+0000 7f272d990740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56493179db30 0x5649317b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.985+0000 7f272d990740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f27241b3b30 con 0x564931793460 2024-04-24T09:07:43.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.985+0000 7f272d990740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f27241b1400 con 0x564931792b40 2024-04-24T09:07:43.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.985+0000 7f272d990740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f27241af250 con 0x56493179db30 2024-04-24T09:07:43.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.985+0000 7f2723400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.985+0000 7f2723400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35212/0 (socket says 172.21.15.69:35212) 2024-04-24T09:07:43.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.985+0000 7f2723400640 1 -- 172.21.15.69:0/3592786402 learned_addr learned my addr 172.21.15.69:0/3592786402 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:43.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.985+0000 7f2723e00640 1 --2- 172.21.15.69:0/3592786402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564931792b40 0x5649317a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.985+0000 7f2723400640 1 -- 172.21.15.69:0/3592786402 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56493179db30 msgr2=0x5649317b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:43.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.985+0000 7f2728a00640 1 --2- 172.21.15.69:0/3592786402 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56493179db30 0x5649317b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.986+0000 7f2723400640 1 --2- 172.21.15.69:0/3592786402 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56493179db30 0x5649317b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.986+0000 7f2723400640 1 -- 172.21.15.69:0/3592786402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564931792b40 msgr2=0x5649317a9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:43.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.986+0000 7f2723400640 1 --2- 172.21.15.69:0/3592786402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564931792b40 0x5649317a9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.986+0000 7f2723400640 1 -- 172.21.15.69:0/3592786402 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649314d47d0 con 0x564931793460 2024-04-24T09:07:43.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.986+0000 7f2723400640 1 --2- 172.21.15.69:0/3592786402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179d070 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2710005940 tx=0x7f2710005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8593d4b7458c37 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:43.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.986+0000 7f270fe00640 1 -- 172.21.15.69:0/3592786402 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710005500 con 0x564931793460 2024-04-24T09:07:43.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.986+0000 7f270fe00640 1 -- 172.21.15.69:0/3592786402 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2710037030 con 0x564931793460 2024-04-24T09:07:43.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.986+0000 7f270fe00640 1 -- 172.21.15.69:0/3592786402 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710004990 con 0x564931793460 2024-04-24T09:07:43.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.987+0000 7f272d990740 1 -- 172.21.15.69:0/3592786402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 msgr2=0x56493179d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.987+0000 7f272d990740 1 --2- 172.21.15.69:0/3592786402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179d070 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2710005940 tx=0x7f2710005890 comp rx=0 tx=0).stop 2024-04-24T09:07:43.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.987+0000 7f272d990740 1 -- 172.21.15.69:0/3592786402 shutdown_connections 2024-04-24T09:07:43.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.987+0000 7f272d990740 1 --2- 172.21.15.69:0/3592786402 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56493179db30 0x5649317b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.987+0000 7f272d990740 1 --2- 172.21.15.69:0/3592786402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931793460 0x56493179d070 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.987+0000 7f272d990740 1 --2- 172.21.15.69:0/3592786402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564931792b40 0x5649317a9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.987+0000 7f272d990740 1 -- 172.21.15.69:0/3592786402 >> 172.21.15.69:0/3592786402 conn(0x56493178ee60 msgr2=0x564931795ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:43.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.987+0000 7f272d990740 1 -- 172.21.15.69:0/3592786402 shutdown_connections 2024-04-24T09:07:43.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.987+0000 7f272d990740 1 -- 172.21.15.69:0/3592786402 wait complete. 2024-04-24T09:07:43.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.987+0000 7f272d990740 1 -- start start 2024-04-24T09:07:43.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.988+0000 7f272d990740 1 Processor -- start 2024-04-24T09:07:43.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.988+0000 7f272d990740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564931792b40 0x5649317a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.988+0000 7f272d990740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56493179db30 0x5649317a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.988+0000 7f272d990740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931796aa0 0x564931796f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.988+0000 7f272d990740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f27241af8c0 con 0x564931796aa0 2024-04-24T09:07:43.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.988+0000 7f272d990740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f27241b0d30 con 0x56493179db30 2024-04-24T09:07:43.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.988+0000 7f272d990740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f27241aff90 con 0x564931792b40 2024-04-24T09:07:43.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.988+0000 7f272d990740 10 client.0 _ll_register_callbacks cb 0x7f271000d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:43.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.988+0000 7f2728a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931796aa0 0x564931796f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.988+0000 7f2728a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931796aa0 0x564931796f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35220/0 (socket says 172.21.15.69:35220) 2024-04-24T09:07:43.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.988+0000 7f2728a00640 1 -- 172.21.15.69:0/874553128 learned_addr learned my addr 172.21.15.69:0/874553128 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:43.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.988+0000 7f2723e00640 1 --2- 172.21.15.69:0/874553128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564931792b40 0x5649317a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.989+0000 7f2723400640 1 --2- 172.21.15.69:0/874553128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56493179db30 0x5649317a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.989+0000 7f2728a00640 1 -- 172.21.15.69:0/874553128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564931792b40 msgr2=0x5649317a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:43.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.989+0000 7f2728a00640 1 --2- 172.21.15.69:0/874553128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564931792b40 0x5649317a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.989+0000 7f2728a00640 1 -- 172.21.15.69:0/874553128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56493179db30 msgr2=0x5649317a62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:43.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.989+0000 7f2728a00640 1 --2- 172.21.15.69:0/874553128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56493179db30 0x5649317a62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.989+0000 7f2728a00640 1 -- 172.21.15.69:0/874553128 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564931797560 con 0x564931796aa0 2024-04-24T09:07:43.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.989+0000 7f2728a00640 1 --2- 172.21.15.69:0/874553128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931796aa0 0x564931796f20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f271c0ba850 tx=0x7f271c004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b297d2815401ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:43.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.989+0000 7f270d600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:43.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.989+0000 7f270d600640 1 -- 172.21.15.69:0/874553128 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c0bde80 con 0x564931796aa0 2024-04-24T09:07:43.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.989+0000 7f270d600640 1 -- 172.21.15.69:0/874553128 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f271c0305c0 con 0x564931796aa0 2024-04-24T09:07:43.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.989+0000 7f272d990740 1 -- 172.21.15.69:0/874553128 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2710037030 con 0x564931796aa0 2024-04-24T09:07:43.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.989+0000 7f270d600640 1 -- 172.21.15.69:0/874553128 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f271c0c2230 con 0x564931796aa0 2024-04-24T09:07:43.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.989+0000 7f272d990740 10 client.15798 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:43.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.989+0000 7f272d990740 1 -- 172.21.15.69:0/874553128 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5649317988a0 con 0x564931796aa0 2024-04-24T09:07:43.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.990+0000 7f270d600640 1 -- 172.21.15.69:0/874553128 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2710037030 con 0x564931796aa0 2024-04-24T09:07:43.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.990+0000 7f270d600640 1 -- 172.21.15.69:0/874553128 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f271c0ef880 con 0x564931796aa0 2024-04-24T09:07:43.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.990+0000 7f272d990740 10 client.15798 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:43.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.990+0000 7f272d990740 10 client.15798 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:07:43.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.990+0000 7f272d990740 20 client.15798 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:43.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.990+0000 7f272d990740 1 --2- 172.21.15.69:0/874553128 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x564931815820 0x564931817c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:43.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.990+0000 7f272d990740 4 client.15798 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:07:43.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.990+0000 7f272d990740 1 -- 172.21.15.69:0/874553128 --> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649316ffa60 con 0x564931815820 2024-04-24T09:07:43.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.990+0000 7f2723e00640 1 --2- 172.21.15.69:0/874553128 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x564931815820 0x564931817c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:43.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.991+0000 7f2723e00640 1 --2- 172.21.15.69:0/874553128 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x564931815820 0x564931817c10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a76131e09ea10546 server_cookie=d47ef5da64858cd in_seq=0 out_seq=0 2024-04-24T09:07:43.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.991+0000 7f270d600640 10 client.15798 ms_handle_connect on v2:172.21.15.81:6834/3260829578 2024-04-24T09:07:43.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.992+0000 7f270d600640 1 -- 172.21.15.69:0/874553128 <== mds.2 v2:172.21.15.81:6834/3260829578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649316ffa60 con 0x564931815820 2024-04-24T09:07:43.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.992+0000 7f270d600640 10 client.15798 handle_command_reply: tid=0 2024-04-24T09:07:43.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.992+0000 7f272d990740 1 client.15798 shutdown 2024-04-24T09:07:43.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.992+0000 7f272d990740 1 -- 172.21.15.69:0/874553128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931796aa0 msgr2=0x564931796f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.992+0000 7f272d990740 1 --2- 172.21.15.69:0/874553128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931796aa0 0x564931796f20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f271c0ba850 tx=0x7f271c004840 comp rx=0 tx=0).stop 2024-04-24T09:07:43.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.992+0000 7f272d990740 1 -- 172.21.15.69:0/874553128 shutdown_connections 2024-04-24T09:07:43.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.992+0000 7f272d990740 1 --2- 172.21.15.69:0/874553128 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x564931815820 0x564931817c10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.992+0000 7f272d990740 1 --2- 172.21.15.69:0/874553128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564931796aa0 0x564931796f20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.992+0000 7f272d990740 1 --2- 172.21.15.69:0/874553128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56493179db30 0x5649317a62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.992+0000 7f270d600640 0 client.15798 ms_handle_reset on v2:172.21.15.81:6834/3260829578 2024-04-24T09:07:43.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.992+0000 7f272d990740 1 --2- 172.21.15.69:0/874553128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564931792b40 0x5649317a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.992+0000 7f272d990740 1 -- 172.21.15.69:0/874553128 >> 172.21.15.69:0/874553128 conn(0x56493178ee60 msgr2=0x56493178d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:43.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.992+0000 7f272d990740 1 -- 172.21.15.69:0/874553128 shutdown_connections 2024-04-24T09:07:43.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.992+0000 7f272d990740 1 -- 172.21.15.69:0/874553128 wait complete. 2024-04-24T09:07:43.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.992+0000 7f272d990740 20 client.15798 trim_cache size 0 max 16384 2024-04-24T09:07:43.995 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-04-24T09:07:43.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.993+0000 7f2729400640 1 -- 172.21.15.69:0/343190516 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f27080a3e30 msgr2=0x7f27080a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.993+0000 7f2729400640 1 --2- 172.21.15.69:0/343190516 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f27080a3e30 0x7f27080a62f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2710005fd0 tx=0x7f2710005f40 comp rx=0 tx=0).stop 2024-04-24T09:07:43.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.993+0000 7f2729400640 1 -- 172.21.15.69:0/343190516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f272419a630 msgr2=0x7f272419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:43.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.993+0000 7f2729400640 1 --2- 172.21.15.69:0/343190516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f272419a630 0x7f272419aab0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f271c031040 tx=0x7f271c003ba0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.993+0000 7f2729400640 1 -- 172.21.15.69:0/343190516 shutdown_connections 2024-04-24T09:07:43.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.993+0000 7f2729400640 1 --2- 172.21.15.69:0/343190516 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f27080a3e30 0x7f27080a62f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.993+0000 7f2729400640 1 --2- 172.21.15.69:0/343190516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f272419a630 0x7f272419aab0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.993+0000 7f2729400640 1 --2- 172.21.15.69:0/343190516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27241093f0 0x7f27241a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.993+0000 7f2729400640 1 --2- 172.21.15.69:0/343190516 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2724104320 0x7f27241a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:43.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.993+0000 7f2729400640 1 -- 172.21.15.69:0/343190516 >> 172.21.15.69:0/343190516 conn(0x7f27240ffb40 msgr2=0x7f2724101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:43.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.993+0000 7f2729400640 1 -- 172.21.15.69:0/343190516 shutdown_connections 2024-04-24T09:07:43.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:43.993+0000 7f2729400640 1 -- 172.21.15.69:0/343190516 wait complete. 2024-04-24T09:07:44.009 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:07:44.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:43 smithi081 ceph-mon[30961]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-24T09:07:44.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.213+0000 7f3103a00640 1 -- 172.21.15.69:0/3925492948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc104e00 msgr2=0x7f30fc108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.213+0000 7f3103a00640 1 --2- 172.21.15.69:0/3925492948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc104e00 0x7f30fc108e40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009900 tx=0x7f30e402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.213+0000 7f3103a00640 1 -- 172.21.15.69:0/3925492948 shutdown_connections 2024-04-24T09:07:44.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.213+0000 7f3103a00640 1 --2- 172.21.15.69:0/3925492948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30fc109520 0x7f30fc1110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.213+0000 7f3103a00640 1 --2- 172.21.15.69:0/3925492948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc104e00 0x7f30fc108e40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.213+0000 7f3103a00640 1 --2- 172.21.15.69:0/3925492948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30fc104450 0x7f30fc104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.213+0000 7f3103a00640 1 -- 172.21.15.69:0/3925492948 >> 172.21.15.69:0/3925492948 conn(0x7f30fc0ffc70 msgr2=0x7f30fc102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.213+0000 7f3103a00640 1 -- 172.21.15.69:0/3925492948 shutdown_connections 2024-04-24T09:07:44.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.213+0000 7f3103a00640 1 -- 172.21.15.69:0/3925492948 wait complete. 2024-04-24T09:07:44.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.214+0000 7f3103a00640 1 Processor -- start 2024-04-24T09:07:44.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.214+0000 7f3103a00640 1 -- start start 2024-04-24T09:07:44.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.215+0000 7f3103a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc104450 0x7f30fc1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.215+0000 7f3103a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30fc109520 0x7f30fc1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.215+0000 7f3103a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30fc19a860 0x7f30fc19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.215+0000 7f3103a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc114460 con 0x7f30fc104450 2024-04-24T09:07:44.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.215+0000 7f3103a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1142e0 con 0x7f30fc19a860 2024-04-24T09:07:44.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.215+0000 7f3103a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1145e0 con 0x7f30fc109520 2024-04-24T09:07:44.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.215+0000 7f3101000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc104450 0x7f30fc1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.215+0000 7f3101000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc104450 0x7f30fc1a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35230/0 (socket says 172.21.15.69:35230) 2024-04-24T09:07:44.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.215+0000 7f3101000640 1 -- 172.21.15.69:0/460074494 learned_addr learned my addr 172.21.15.69:0/460074494 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:44.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.216+0000 7f3101000640 1 -- 172.21.15.69:0/460074494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30fc109520 msgr2=0x7f30fc1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.216+0000 7f3101000640 1 --2- 172.21.15.69:0/460074494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30fc109520 0x7f30fc1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.216+0000 7f3101000640 1 -- 172.21.15.69:0/460074494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30fc19a860 msgr2=0x7f30fc19ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.216+0000 7f3101000640 1 --2- 172.21.15.69:0/460074494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30fc19a860 0x7f30fc19ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.216+0000 7f3101000640 1 -- 172.21.15.69:0/460074494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc19b3f0 con 0x7f30fc104450 2024-04-24T09:07:44.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.216+0000 7f3101000640 1 --2- 172.21.15.69:0/460074494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc104450 0x7f30fc1a06a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00d770 tx=0x7f30ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.216+0000 7f30fa000640 1 -- 172.21.15.69:0/460074494 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec017070 con 0x7f30fc104450 2024-04-24T09:07:44.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.216+0000 7f30fa000640 1 -- 172.21.15.69:0/460074494 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30ec0054d0 con 0x7f30fc104450 2024-04-24T09:07:44.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.216+0000 7f3103a00640 1 -- 172.21.15.69:0/460074494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30fc19b6e0 con 0x7f30fc104450 2024-04-24T09:07:44.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.216+0000 7f30fa000640 1 -- 172.21.15.69:0/460074494 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec004cf0 con 0x7f30fc104450 2024-04-24T09:07:44.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.216+0000 7f3103a00640 1 -- 172.21.15.69:0/460074494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30fc1a7820 con 0x7f30fc104450 2024-04-24T09:07:44.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.220+0000 7f30fa000640 1 -- 172.21.15.69:0/460074494 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f30ec004e90 con 0x7f30fc104450 2024-04-24T09:07:44.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.220+0000 7f3106848740 1 Processor -- start 2024-04-24T09:07:44.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.220+0000 7f3106848740 1 -- start start 2024-04-24T09:07:44.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.220+0000 7f3106848740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2bfb950 0x5651a2bfbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.220+0000 7f3106848740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfc270 0x5651a2c06400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.220+0000 7f3106848740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2c06940 0x5651a2c08d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.220+0000 7f3106848740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5651a2b611d0 con 0x5651a2c06940 2024-04-24T09:07:44.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.220+0000 7f3106848740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5651a2b07450 con 0x5651a2bfc270 2024-04-24T09:07:44.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.220+0000 7f3106848740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5651a2b5d190 con 0x5651a2bfb950 2024-04-24T09:07:44.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.220+0000 7f30f3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2c06940 0x5651a2c08d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.220+0000 7f30f3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2c06940 0x5651a2c08d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35240/0 (socket says 172.21.15.69:35240) 2024-04-24T09:07:44.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.220+0000 7f30f3e00640 1 -- 172.21.15.69:0/841006109 learned_addr learned my addr 172.21.15.69:0/841006109 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:44.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.221+0000 7f3101000640 1 --2- 172.21.15.69:0/841006109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfc270 0x5651a2c06400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.221+0000 7f3101a00640 1 --2- 172.21.15.69:0/841006109 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2bfb950 0x5651a2bfbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.221+0000 7f30f3e00640 1 -- 172.21.15.69:0/841006109 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2bfb950 msgr2=0x5651a2bfbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.221+0000 7f30f3e00640 1 --2- 172.21.15.69:0/841006109 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2bfb950 0x5651a2bfbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.221+0000 7f30f3e00640 1 -- 172.21.15.69:0/841006109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfc270 msgr2=0x5651a2c06400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.221+0000 7f30f3e00640 1 --2- 172.21.15.69:0/841006109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfc270 0x5651a2c06400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.221+0000 7f30f3e00640 1 -- 172.21.15.69:0/841006109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a2bc4310 con 0x5651a2c06940 2024-04-24T09:07:44.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.221+0000 7f30fa000640 1 --2- 172.21.15.69:0/460074494 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f30dc0a3db0 0x7f30dc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.221+0000 7f30f3e00640 1 --2- 172.21.15.69:0/841006109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2c06940 0x5651a2c08d30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f30e4033000 tx=0x7f30e4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c0cf313fc34342 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.221+0000 7f30fa000640 1 -- 172.21.15.69:0/460074494 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f30ec0d0810 con 0x7f30fc104450 2024-04-24T09:07:44.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.222+0000 7f3101a00640 1 --2- 172.21.15.69:0/460074494 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f30dc0a3db0 0x7f30dc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.222+0000 7f30f2000640 1 -- 172.21.15.69:0/841006109 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4040020 con 0x5651a2c06940 2024-04-24T09:07:44.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.222+0000 7f30f2000640 1 -- 172.21.15.69:0/841006109 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30e4004040 con 0x5651a2c06940 2024-04-24T09:07:44.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.222+0000 7f30f2000640 1 -- 172.21.15.69:0/841006109 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4035040 con 0x5651a2c06940 2024-04-24T09:07:44.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.222+0000 7f3106848740 1 -- 172.21.15.69:0/841006109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2c06940 msgr2=0x5651a2c08d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.222+0000 7f3106848740 1 --2- 172.21.15.69:0/841006109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2c06940 0x5651a2c08d30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f30e4033000 tx=0x7f30e4000bc0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.222+0000 7f3101a00640 1 --2- 172.21.15.69:0/460074494 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f30dc0a3db0 0x7f30dc0a6270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f30f4005cc0 tx=0x7f30f400a330 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.223+0000 7f3106848740 1 -- 172.21.15.69:0/841006109 shutdown_connections 2024-04-24T09:07:44.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.223+0000 7f3106848740 1 --2- 172.21.15.69:0/841006109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2c06940 0x5651a2c08d30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.223+0000 7f3106848740 1 --2- 172.21.15.69:0/841006109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfc270 0x5651a2c06400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.223+0000 7f3106848740 1 --2- 172.21.15.69:0/841006109 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2bfb950 0x5651a2bfbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.223+0000 7f3106848740 1 -- 172.21.15.69:0/841006109 >> 172.21.15.69:0/841006109 conn(0x5651a2bf7c70 msgr2=0x5651a2bf8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.223+0000 7f3106848740 1 -- 172.21.15.69:0/841006109 shutdown_connections 2024-04-24T09:07:44.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.223+0000 7f3106848740 1 -- 172.21.15.69:0/841006109 wait complete. 2024-04-24T09:07:44.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.223+0000 7f3106848740 1 -- start start 2024-04-24T09:07:44.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.223+0000 7f3106848740 1 Processor -- start 2024-04-24T09:07:44.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.224+0000 7f3106848740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfb950 0x5651a2c06310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.224+0000 7f3106848740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfc270 0x5651a2c03600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.224+0000 7f3106848740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2c06940 0x5651a2c03c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.224+0000 7f3106848740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5651a2c09770 con 0x5651a2bfb950 2024-04-24T09:07:44.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.224+0000 7f3106848740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5651a2b1a810 con 0x5651a2bfc270 2024-04-24T09:07:44.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.224+0000 7f3106848740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5651a2c098f0 con 0x5651a2c06940 2024-04-24T09:07:44.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.224+0000 7f3106848740 10 client.0 _ll_register_callbacks cb 0x5651a2ae8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:44.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.224+0000 7f3101000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfb950 0x5651a2c06310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.224+0000 7f3101000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfb950 0x5651a2c06310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35246/0 (socket says 172.21.15.69:35246) 2024-04-24T09:07:44.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.224+0000 7f3101000640 1 -- 172.21.15.69:0/388500461 learned_addr learned my addr 172.21.15.69:0/388500461 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:44.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.224+0000 7f30f3e00640 1 --2- 172.21.15.69:0/388500461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfc270 0x5651a2c03600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.224+0000 7f3101000640 1 -- 172.21.15.69:0/388500461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2c06940 msgr2=0x5651a2c03c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.224+0000 7f3101000640 1 --2- 172.21.15.69:0/388500461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2c06940 0x5651a2c03c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.224+0000 7f3101000640 1 -- 172.21.15.69:0/388500461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfc270 msgr2=0x5651a2c03600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.224+0000 7f3101000640 1 --2- 172.21.15.69:0/388500461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfc270 0x5651a2c03600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.224+0000 7f3101000640 1 -- 172.21.15.69:0/388500461 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a2b2b3f0 con 0x5651a2bfb950 2024-04-24T09:07:44.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.225+0000 7f3101000640 1 --2- 172.21.15.69:0/388500461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfb950 0x5651a2c06310 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0cdbf0 tx=0x7f30ec00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72404a5beccc0f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.225+0000 7f30eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:44.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.225+0000 7f30eaa00640 1 -- 172.21.15.69:0/388500461 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec020040 con 0x5651a2bfb950 2024-04-24T09:07:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.225+0000 7f30eaa00640 1 -- 172.21.15.69:0/388500461 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f30ec09b750 con 0x5651a2bfb950 2024-04-24T09:07:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.225+0000 7f3106848740 1 -- 172.21.15.69:0/388500461 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e4004040 con 0x5651a2bfb950 2024-04-24T09:07:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.225+0000 7f30eaa00640 1 -- 172.21.15.69:0/388500461 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f30ec09bc60 con 0x5651a2bfb950 2024-04-24T09:07:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.225+0000 7f3106848740 10 client.15810 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.225+0000 7f3106848740 1 -- 172.21.15.69:0/388500461 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5651a28733f0 con 0x5651a2bfb950 2024-04-24T09:07:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.225+0000 7f30eaa00640 1 -- 172.21.15.69:0/388500461 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30e4004040 con 0x5651a2bfb950 2024-04-24T09:07:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.226+0000 7f30eaa00640 1 -- 172.21.15.69:0/388500461 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f30ec098050 con 0x5651a2bfb950 2024-04-24T09:07:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.226+0000 7f3106848740 10 client.15810 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.226+0000 7f3106848740 10 client.15810 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:07:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.226+0000 7f3106848740 20 client.15810 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.226+0000 7f3106848740 1 --2- 172.21.15.69:0/388500461 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5651a2c16ad0 0x5651a2c18f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.226+0000 7f3106848740 4 client.15810 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.226+0000 7f3106848740 1 -- 172.21.15.69:0/388500461 --> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651a2b6ebf0 con 0x5651a2c16ad0 2024-04-24T09:07:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.226+0000 7f30f3e00640 1 --2- 172.21.15.69:0/388500461 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5651a2c16ad0 0x5651a2c18f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.227+0000 7f30f3e00640 1 --2- 172.21.15.69:0/388500461 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5651a2c16ad0 0x5651a2c18f60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa4cbac27dc23f99 server_cookie=798747215a827916 in_seq=0 out_seq=0 2024-04-24T09:07:44.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.227+0000 7f30eaa00640 10 client.15810 ms_handle_connect on v2:172.21.15.81:6836/1973819593 2024-04-24T09:07:44.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.229+0000 7f30eaa00640 1 -- 172.21.15.69:0/388500461 <== mds.1 v2:172.21.15.81:6836/1973819593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651a2b6ebf0 con 0x5651a2c16ad0 2024-04-24T09:07:44.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.229+0000 7f30eaa00640 10 client.15810 handle_command_reply: tid=0 2024-04-24T09:07:44.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.229+0000 7f3106848740 1 client.15810 shutdown 2024-04-24T09:07:44.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.230+0000 7f3106848740 1 -- 172.21.15.69:0/388500461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfb950 msgr2=0x5651a2c06310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.230+0000 7f3106848740 1 --2- 172.21.15.69:0/388500461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfb950 0x5651a2c06310 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0cdbf0 tx=0x7f30ec00df80 comp rx=0 tx=0).stop 2024-04-24T09:07:44.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.230+0000 7f3106848740 1 -- 172.21.15.69:0/388500461 shutdown_connections 2024-04-24T09:07:44.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.230+0000 7f3106848740 1 --2- 172.21.15.69:0/388500461 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5651a2c16ad0 0x5651a2c18f60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.230+0000 7f3106848740 1 --2- 172.21.15.69:0/388500461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2c06940 0x5651a2c03c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.230+0000 7f3106848740 1 --2- 172.21.15.69:0/388500461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfc270 0x5651a2c03600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.230+0000 7f3106848740 1 --2- 172.21.15.69:0/388500461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfb950 0x5651a2c06310 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.230+0000 7f30eaa00640 0 client.15810 ms_handle_reset on v2:172.21.15.81:6836/1973819593 2024-04-24T09:07:44.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.230+0000 7f3106848740 1 -- 172.21.15.69:0/388500461 >> 172.21.15.69:0/388500461 conn(0x5651a2bf7c70 msgr2=0x5651a2bfda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.230+0000 7f3106848740 1 -- 172.21.15.69:0/388500461 shutdown_connections 2024-04-24T09:07:44.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.230+0000 7f3106848740 1 -- 172.21.15.69:0/388500461 wait complete. 2024-04-24T09:07:44.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.230+0000 7f3106848740 20 client.15810 trim_cache size 0 max 16384 2024-04-24T09:07:44.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.244+0000 7f3106848740 1 Processor -- start 2024-04-24T09:07:44.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.244+0000 7f3106848740 1 -- start start 2024-04-24T09:07:44.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.245+0000 7f3106848740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfb950 0x5651a2c12630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.245+0000 7f3106848740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfc270 0x5651a2c05e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.245+0000 7f3106848740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2c06940 0x5651a2c1e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.245+0000 7f3106848740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1b3c10 con 0x5651a2bfc270 2024-04-24T09:07:44.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.245+0000 7f3106848740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1b14e0 con 0x5651a2bfb950 2024-04-24T09:07:44.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.245+0000 7f3106848740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1af330 con 0x5651a2c06940 2024-04-24T09:07:44.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.245+0000 7f30f3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfc270 0x5651a2c05e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.245+0000 7f30f3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfc270 0x5651a2c05e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35248/0 (socket says 172.21.15.69:35248) 2024-04-24T09:07:44.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.245+0000 7f30f3e00640 1 -- 172.21.15.69:0/491338529 learned_addr learned my addr 172.21.15.69:0/491338529 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:44.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.245+0000 7f3101000640 1 --2- 172.21.15.69:0/491338529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfb950 0x5651a2c12630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.245+0000 7f30f3e00640 1 -- 172.21.15.69:0/491338529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2c06940 msgr2=0x5651a2c1e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.245+0000 7f30f3e00640 1 --2- 172.21.15.69:0/491338529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2c06940 0x5651a2c1e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.245+0000 7f30f3e00640 1 -- 172.21.15.69:0/491338529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfb950 msgr2=0x5651a2c12630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.245+0000 7f30f3e00640 1 --2- 172.21.15.69:0/491338529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfb950 0x5651a2c12630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.245+0000 7f30f3e00640 1 -- 172.21.15.69:0/491338529 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a293d7d0 con 0x5651a2bfc270 2024-04-24T09:07:44.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.246+0000 7f30f3e00640 1 --2- 172.21.15.69:0/491338529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfc270 0x5651a2c05e40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f30e40092a0 tx=0x7f30e4031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deefdb962eb6f22c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.246+0000 7f30f1600640 1 -- 172.21.15.69:0/491338529 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4040040 con 0x5651a2bfc270 2024-04-24T09:07:44.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.246+0000 7f30f1600640 1 -- 172.21.15.69:0/491338529 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30e403e070 con 0x5651a2bfc270 2024-04-24T09:07:44.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.246+0000 7f30f1600640 1 -- 172.21.15.69:0/491338529 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4035040 con 0x5651a2bfc270 2024-04-24T09:07:44.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.247+0000 7f3106848740 1 -- 172.21.15.69:0/491338529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfc270 msgr2=0x5651a2c05e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.247+0000 7f3106848740 1 --2- 172.21.15.69:0/491338529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfc270 0x5651a2c05e40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f30e40092a0 tx=0x7f30e4031630 comp rx=0 tx=0).stop 2024-04-24T09:07:44.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.247+0000 7f3106848740 1 -- 172.21.15.69:0/491338529 shutdown_connections 2024-04-24T09:07:44.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.247+0000 7f3106848740 1 --2- 172.21.15.69:0/491338529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2c06940 0x5651a2c1e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.247+0000 7f3106848740 1 --2- 172.21.15.69:0/491338529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfc270 0x5651a2c05e40 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.247+0000 7f3106848740 1 --2- 172.21.15.69:0/491338529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfb950 0x5651a2c12630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.247+0000 7f3106848740 1 -- 172.21.15.69:0/491338529 >> 172.21.15.69:0/491338529 conn(0x5651a2bf7c70 msgr2=0x5651a2bfe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.247+0000 7f3106848740 1 -- 172.21.15.69:0/491338529 shutdown_connections 2024-04-24T09:07:44.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.247+0000 7f3106848740 1 -- 172.21.15.69:0/491338529 wait complete. 2024-04-24T09:07:44.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.247+0000 7f3106848740 1 -- start start 2024-04-24T09:07:44.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.247+0000 7f3106848740 1 Processor -- start 2024-04-24T09:07:44.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.248+0000 7f3106848740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfb950 0x5651a2c074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.248+0000 7f3106848740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfc270 0x5651a2c079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.248+0000 7f3106848740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2c06940 0x5651a2c07f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.248+0000 7f3106848740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1af9a0 con 0x5651a2bfc270 2024-04-24T09:07:44.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.248+0000 7f3106848740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1b0e10 con 0x5651a2bfb950 2024-04-24T09:07:44.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.248+0000 7f3106848740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc1b0070 con 0x5651a2c06940 2024-04-24T09:07:44.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.248+0000 7f3106848740 10 client.0 _ll_register_callbacks cb 0x7f30ec0cd9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:44.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.248+0000 7f30f3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfc270 0x5651a2c079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.248+0000 7f30f3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfc270 0x5651a2c079f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35264/0 (socket says 172.21.15.69:35264) 2024-04-24T09:07:44.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.248+0000 7f30f3e00640 1 -- 172.21.15.69:0/1575935326 learned_addr learned my addr 172.21.15.69:0/1575935326 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:44.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.248+0000 7f30f3e00640 1 -- 172.21.15.69:0/1575935326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2c06940 msgr2=0x5651a2c07f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.248+0000 7f30f3e00640 1 --2- 172.21.15.69:0/1575935326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2c06940 0x5651a2c07f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.248+0000 7f30f3e00640 1 -- 172.21.15.69:0/1575935326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfb950 msgr2=0x5651a2c074b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.248+0000 7f30f3e00640 1 --2- 172.21.15.69:0/1575935326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfb950 0x5651a2c074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.248+0000 7f30f3e00640 1 -- 172.21.15.69:0/1575935326 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a2c0cba0 con 0x5651a2bfc270 2024-04-24T09:07:44.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.249+0000 7f30f3e00640 1 --2- 172.21.15.69:0/1575935326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfc270 0x5651a2c079f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f30e402a910 tx=0x7f30e4032b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a725edd7b9aebf6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.249+0000 7f30eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:44.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.249+0000 7f30eaa00640 1 -- 172.21.15.69:0/1575935326 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4055070 con 0x5651a2bfc270 2024-04-24T09:07:44.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.249+0000 7f30eaa00640 1 -- 172.21.15.69:0/1575935326 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f30e4050dd0 con 0x5651a2bfc270 2024-04-24T09:07:44.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.249+0000 7f3106848740 1 -- 172.21.15.69:0/1575935326 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e403e070 con 0x5651a2bfc270 2024-04-24T09:07:44.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.249+0000 7f30eaa00640 1 -- 172.21.15.69:0/1575935326 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f30e4061050 con 0x5651a2bfc270 2024-04-24T09:07:44.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.249+0000 7f3106848740 10 client.15816 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:44.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.249+0000 7f3106848740 1 -- 172.21.15.69:0/1575935326 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5651a2c7d220 con 0x5651a2bfc270 2024-04-24T09:07:44.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.250+0000 7f30eaa00640 1 -- 172.21.15.69:0/1575935326 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30e403e070 con 0x5651a2bfc270 2024-04-24T09:07:44.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.250+0000 7f30eaa00640 1 -- 172.21.15.69:0/1575935326 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f30e40046c0 con 0x5651a2bfc270 2024-04-24T09:07:44.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.250+0000 7f3106848740 10 client.15816 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:44.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.250+0000 7f3106848740 10 client.15816 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:07:44.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.250+0000 7f3106848740 20 client.15816 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:44.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.250+0000 7f3106848740 1 --2- 172.21.15.69:0/1575935326 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5651a2c7d460 0x5651a2c7f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.250+0000 7f3106848740 4 client.15816 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:07:44.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.250+0000 7f3106848740 1 -- 172.21.15.69:0/1575935326 --> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5651a2b68a60 con 0x5651a2c7d460 2024-04-24T09:07:44.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.250+0000 7f3101000640 1 --2- 172.21.15.69:0/1575935326 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5651a2c7d460 0x5651a2c7f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.251+0000 7f3101000640 1 --2- 172.21.15.69:0/1575935326 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5651a2c7d460 0x5651a2c7f850 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=441f305da6c52000 server_cookie=5e0756606f53c071 in_seq=0 out_seq=0 2024-04-24T09:07:44.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.251+0000 7f30eaa00640 10 client.15816 ms_handle_connect on v2:172.21.15.81:6836/1973819593 2024-04-24T09:07:44.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.252+0000 7f30eaa00640 1 -- 172.21.15.69:0/1575935326 <== mds.1 v2:172.21.15.81:6836/1973819593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5651a2b68a60 con 0x5651a2c7d460 2024-04-24T09:07:44.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.252+0000 7f30eaa00640 10 client.15816 handle_command_reply: tid=0 2024-04-24T09:07:44.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.252+0000 7f3106848740 1 client.15816 shutdown 2024-04-24T09:07:44.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.252+0000 7f3106848740 1 -- 172.21.15.69:0/1575935326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfc270 msgr2=0x5651a2c079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.252+0000 7f3106848740 1 --2- 172.21.15.69:0/1575935326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfc270 0x5651a2c079f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f30e402a910 tx=0x7f30e4032b80 comp rx=0 tx=0).stop 2024-04-24T09:07:44.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.252+0000 7f3106848740 1 -- 172.21.15.69:0/1575935326 shutdown_connections 2024-04-24T09:07:44.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.252+0000 7f3106848740 1 --2- 172.21.15.69:0/1575935326 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5651a2c7d460 0x5651a2c7f850 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.252+0000 7f3106848740 1 --2- 172.21.15.69:0/1575935326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5651a2c06940 0x5651a2c07f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.252+0000 7f3106848740 1 --2- 172.21.15.69:0/1575935326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651a2bfc270 0x5651a2c079f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.252+0000 7f3106848740 1 --2- 172.21.15.69:0/1575935326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5651a2bfb950 0x5651a2c074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.252+0000 7f30eaa00640 0 client.15816 ms_handle_reset on v2:172.21.15.81:6836/1973819593 2024-04-24T09:07:44.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.252+0000 7f3106848740 1 -- 172.21.15.69:0/1575935326 >> 172.21.15.69:0/1575935326 conn(0x5651a2bf7c70 msgr2=0x5651a2bf6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.252+0000 7f3106848740 1 -- 172.21.15.69:0/1575935326 shutdown_connections 2024-04-24T09:07:44.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.252+0000 7f3106848740 1 -- 172.21.15.69:0/1575935326 wait complete. 2024-04-24T09:07:44.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.252+0000 7f3106848740 20 client.15816 trim_cache size 0 max 16384 2024-04-24T09:07:44.254 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-04-24T09:07:44.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.253+0000 7f3103a00640 1 -- 172.21.15.69:0/460074494 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f30dc0a3db0 msgr2=0x7f30dc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.253+0000 7f3103a00640 1 --2- 172.21.15.69:0/460074494 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f30dc0a3db0 0x7f30dc0a6270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f30f4005cc0 tx=0x7f30f400a330 comp rx=0 tx=0).stop 2024-04-24T09:07:44.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.253+0000 7f3103a00640 1 -- 172.21.15.69:0/460074494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc104450 msgr2=0x7f30fc1a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.253+0000 7f3103a00640 1 --2- 172.21.15.69:0/460074494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc104450 0x7f30fc1a06a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00d770 tx=0x7f30ec00dc30 comp rx=0 tx=0).stop 2024-04-24T09:07:44.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.253+0000 7f3103a00640 1 -- 172.21.15.69:0/460074494 shutdown_connections 2024-04-24T09:07:44.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.253+0000 7f3103a00640 1 --2- 172.21.15.69:0/460074494 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f30dc0a3db0 0x7f30dc0a6270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.253+0000 7f3103a00640 1 --2- 172.21.15.69:0/460074494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30fc19a860 0x7f30fc19ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.253+0000 7f3103a00640 1 --2- 172.21.15.69:0/460074494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30fc109520 0x7f30fc1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.253+0000 7f3103a00640 1 --2- 172.21.15.69:0/460074494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc104450 0x7f30fc1a06a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.253+0000 7f3103a00640 1 -- 172.21.15.69:0/460074494 >> 172.21.15.69:0/460074494 conn(0x7f30fc0ffc70 msgr2=0x7f30fc101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.253+0000 7f3103a00640 1 -- 172.21.15.69:0/460074494 shutdown_connections 2024-04-24T09:07:44.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.253+0000 7f3103a00640 1 -- 172.21.15.69:0/460074494 wait complete. 2024-04-24T09:07:44.268 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:07:44.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.463+0000 7f97cf000640 1 -- 172.21.15.69:0/3825679002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c8109520 msgr2=0x7f97c81110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.463+0000 7f97cf000640 1 --2- 172.21.15.69:0/3825679002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c8109520 0x7f97c81110f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009ca0 tx=0x7f97c002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.463+0000 7f97cf000640 1 -- 172.21.15.69:0/3825679002 shutdown_connections 2024-04-24T09:07:44.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.463+0000 7f97cf000640 1 --2- 172.21.15.69:0/3825679002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c8109520 0x7f97c81110f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.463+0000 7f97cf000640 1 --2- 172.21.15.69:0/3825679002 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97c8104e00 0x7f97c8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.463+0000 7f97cf000640 1 --2- 172.21.15.69:0/3825679002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97c8104450 0x7f97c8104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.463+0000 7f97cf000640 1 -- 172.21.15.69:0/3825679002 >> 172.21.15.69:0/3825679002 conn(0x7f97c80ffc70 msgr2=0x7f97c8102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.463+0000 7f97cf000640 1 -- 172.21.15.69:0/3825679002 shutdown_connections 2024-04-24T09:07:44.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.463+0000 7f97cf000640 1 -- 172.21.15.69:0/3825679002 wait complete. 2024-04-24T09:07:44.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.464+0000 7f97cf000640 1 Processor -- start 2024-04-24T09:07:44.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.464+0000 7f97cf000640 1 -- start start 2024-04-24T09:07:44.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.465+0000 7f97cf000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c8104450 0x7f97c8076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.465+0000 7f97cf000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97c8109520 0x7f97c8076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.465+0000 7f97cf000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97c8076f90 0x7f97c81aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.465+0000 7f97cf000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f97c81144e0 con 0x7f97c8104450 2024-04-24T09:07:44.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.465+0000 7f97cf000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f97c8114360 con 0x7f97c8076f90 2024-04-24T09:07:44.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.465+0000 7f97cf000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f97c8114660 con 0x7f97c8109520 2024-04-24T09:07:44.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.465+0000 7f97c7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c8104450 0x7f97c8076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.465+0000 7f97c7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c8104450 0x7f97c8076510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35282/0 (socket says 172.21.15.69:35282) 2024-04-24T09:07:44.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.465+0000 7f97c7e00640 1 -- 172.21.15.69:0/1929646943 learned_addr learned my addr 172.21.15.69:0/1929646943 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:44.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.465+0000 7f97cd000640 1 --2- 172.21.15.69:0/1929646943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97c8076f90 0x7f97c81aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.466+0000 7f97bf400640 1 --2- 172.21.15.69:0/1929646943 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97c8109520 0x7f97c8076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.466+0000 7f97c7e00640 1 -- 172.21.15.69:0/1929646943 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97c8109520 msgr2=0x7f97c8076a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.466+0000 7f97c7e00640 1 --2- 172.21.15.69:0/1929646943 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97c8109520 0x7f97c8076a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.466+0000 7f97c7e00640 1 -- 172.21.15.69:0/1929646943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97c8076f90 msgr2=0x7f97c81aaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.466+0000 7f97c7e00640 1 --2- 172.21.15.69:0/1929646943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97c8076f90 0x7f97c81aaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.466+0000 7f97c7e00640 1 -- 172.21.15.69:0/1929646943 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c81ab490 con 0x7f97c8104450 2024-04-24T09:07:44.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.466+0000 7f97bf400640 1 --2- 172.21.15.69:0/1929646943 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97c8109520 0x7f97c8076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:44.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.466+0000 7f97c7e00640 1 --2- 172.21.15.69:0/1929646943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c8104450 0x7f97c8076510 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f97b800c920 tx=0x7f97b800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.466+0000 7f97c5600640 1 -- 172.21.15.69:0/1929646943 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8007c70 con 0x7f97c8104450 2024-04-24T09:07:44.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.466+0000 7f97c5600640 1 -- 172.21.15.69:0/1929646943 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97b8007e10 con 0x7f97c8104450 2024-04-24T09:07:44.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.466+0000 7f97cf000640 1 -- 172.21.15.69:0/1929646943 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c81ab780 con 0x7f97c8104450 2024-04-24T09:07:44.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.466+0000 7f97c5600640 1 -- 172.21.15.69:0/1929646943 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8018820 con 0x7f97c8104450 2024-04-24T09:07:44.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.466+0000 7f97cf000640 1 -- 172.21.15.69:0/1929646943 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c81abca0 con 0x7f97c8104450 2024-04-24T09:07:44.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.470+0000 7f97d1e13740 1 Processor -- start 2024-04-24T09:07:44.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.470+0000 7f97d1e13740 1 -- start start 2024-04-24T09:07:44.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.470+0000 7f97c5600640 1 -- 172.21.15.69:0/1929646943 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f97b80189c0 con 0x7f97c8104450 2024-04-24T09:07:44.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.470+0000 7f97d1e13740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f7c950 0x55f506f7cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.470+0000 7f97d1e13740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f7d270 0x55f506f87400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.470+0000 7f97d1e13740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f87940 0x55f506f89d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.470+0000 7f97d1e13740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f506ee21d0 con 0x55f506f87940 2024-04-24T09:07:44.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.470+0000 7f97d1e13740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f506e88450 con 0x55f506f7c950 2024-04-24T09:07:44.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.470+0000 7f97d1e13740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f506ede190 con 0x55f506f7d270 2024-04-24T09:07:44.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.471+0000 7f97bf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f87940 0x55f506f89d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.471+0000 7f97bf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f87940 0x55f506f89d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35290/0 (socket says 172.21.15.69:35290) 2024-04-24T09:07:44.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.471+0000 7f97bf400640 1 -- 172.21.15.69:0/4240825348 learned_addr learned my addr 172.21.15.69:0/4240825348 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:44.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.471+0000 7f97cd000640 1 --2- 172.21.15.69:0/4240825348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f7c950 0x55f506f7cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.471+0000 7f97c7e00640 1 --2- 172.21.15.69:0/4240825348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f7d270 0x55f506f87400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.471+0000 7f97bf400640 1 -- 172.21.15.69:0/4240825348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f7d270 msgr2=0x55f506f87400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.471+0000 7f97bf400640 1 --2- 172.21.15.69:0/4240825348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f7d270 0x55f506f87400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.471+0000 7f97bf400640 1 -- 172.21.15.69:0/4240825348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f7c950 msgr2=0x55f506f7cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.471+0000 7f97bf400640 1 --2- 172.21.15.69:0/4240825348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f7c950 0x55f506f7cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.471+0000 7f97bf400640 1 -- 172.21.15.69:0/4240825348 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f506f45310 con 0x55f506f87940 2024-04-24T09:07:44.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.471+0000 7f97bf400640 1 --2- 172.21.15.69:0/4240825348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f87940 0x55f506f89d30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009b30 tx=0x7f97b000e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce012f14277d00d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.472+0000 7f97bd600640 1 -- 172.21.15.69:0/4240825348 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0016070 con 0x55f506f87940 2024-04-24T09:07:44.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.472+0000 7f97bd600640 1 -- 172.21.15.69:0/4240825348 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97b000ed20 con 0x55f506f87940 2024-04-24T09:07:44.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.472+0000 7f97bd600640 1 -- 172.21.15.69:0/4240825348 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0010750 con 0x55f506f87940 2024-04-24T09:07:44.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.472+0000 7f97c5600640 1 --2- 172.21.15.69:0/1929646943 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f97ac0a3db0 0x7f97ac0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.472+0000 7f97c5600640 1 -- 172.21.15.69:0/1929646943 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f97b80d0f40 con 0x7f97c8104450 2024-04-24T09:07:44.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.472+0000 7f97cd000640 1 --2- 172.21.15.69:0/1929646943 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f97ac0a3db0 0x7f97ac0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.473+0000 7f97d1e13740 1 -- 172.21.15.69:0/4240825348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f87940 msgr2=0x55f506f89d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.473+0000 7f97d1e13740 1 --2- 172.21.15.69:0/4240825348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f87940 0x55f506f89d30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009b30 tx=0x7f97b000e2f0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.473+0000 7f97d1e13740 1 -- 172.21.15.69:0/4240825348 shutdown_connections 2024-04-24T09:07:44.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.473+0000 7f97d1e13740 1 --2- 172.21.15.69:0/4240825348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f87940 0x55f506f89d30 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.473+0000 7f97d1e13740 1 --2- 172.21.15.69:0/4240825348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f7d270 0x55f506f87400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.473+0000 7f97d1e13740 1 --2- 172.21.15.69:0/4240825348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f7c950 0x55f506f7cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.473+0000 7f97d1e13740 1 -- 172.21.15.69:0/4240825348 >> 172.21.15.69:0/4240825348 conn(0x55f506f78c70 msgr2=0x55f506f79050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.473+0000 7f97d1e13740 1 -- 172.21.15.69:0/4240825348 shutdown_connections 2024-04-24T09:07:44.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.473+0000 7f97d1e13740 1 -- 172.21.15.69:0/4240825348 wait complete. 2024-04-24T09:07:44.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.473+0000 7f97cd000640 1 --2- 172.21.15.69:0/1929646943 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f97ac0a3db0 0x7f97ac0a6270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f97c0005fe0 tx=0x7f97c0009360 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.473+0000 7f97d1e13740 1 -- start start 2024-04-24T09:07:44.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.474+0000 7f97d1e13740 1 Processor -- start 2024-04-24T09:07:44.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.474+0000 7f97d1e13740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f7c950 0x55f506f87350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.474+0000 7f97d1e13740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7d270 0x55f506f84640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.474+0000 7f97d1e13740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f87940 0x55f506f84c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.474+0000 7f97d1e13740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f506f8a7d0 con 0x55f506f7d270 2024-04-24T09:07:44.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.474+0000 7f97d1e13740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f506e9b810 con 0x55f506f87940 2024-04-24T09:07:44.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.474+0000 7f97d1e13740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f506f8a950 con 0x55f506f7c950 2024-04-24T09:07:44.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.474+0000 7f97d1e13740 10 client.0 _ll_register_callbacks cb 0x55f506e69140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:44.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.474+0000 7f97bf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7d270 0x55f506f84640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.474+0000 7f97bf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7d270 0x55f506f84640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35292/0 (socket says 172.21.15.69:35292) 2024-04-24T09:07:44.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.474+0000 7f97bf400640 1 -- 172.21.15.69:0/2392992742 learned_addr learned my addr 172.21.15.69:0/2392992742 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:44.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.475+0000 7f97c7e00640 1 --2- 172.21.15.69:0/2392992742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f7c950 0x55f506f87350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.475+0000 7f97cd000640 1 --2- 172.21.15.69:0/2392992742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f87940 0x55f506f84c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.475+0000 7f97bf400640 1 -- 172.21.15.69:0/2392992742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f7c950 msgr2=0x55f506f87350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.475+0000 7f97bf400640 1 --2- 172.21.15.69:0/2392992742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f7c950 0x55f506f87350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.475+0000 7f97bf400640 1 -- 172.21.15.69:0/2392992742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f87940 msgr2=0x55f506f84c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.475+0000 7f97bf400640 1 --2- 172.21.15.69:0/2392992742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f87940 0x55f506f84c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.475+0000 7f97bf400640 1 -- 172.21.15.69:0/2392992742 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f506eac3f0 con 0x55f506f7d270 2024-04-24T09:07:44.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.475+0000 7f97c7e00640 1 --2- 172.21.15.69:0/2392992742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f7c950 0x55f506f87350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:44.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.475+0000 7f97bf400640 1 --2- 172.21.15.69:0/2392992742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7d270 0x55f506f84640 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f97b00056e0 tx=0x7f97b0005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24349525df85f5c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.475+0000 7f97b6000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:44.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.475+0000 7f97b6000640 1 -- 172.21.15.69:0/2392992742 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0016050 con 0x55f506f7d270 2024-04-24T09:07:44.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.475+0000 7f97b6000640 1 -- 172.21.15.69:0/2392992742 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f97b0004660 con 0x55f506f7d270 2024-04-24T09:07:44.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.475+0000 7f97d1e13740 1 -- 172.21.15.69:0/2392992742 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b000ed20 con 0x55f506f7d270 2024-04-24T09:07:44.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.476+0000 7f97b6000640 1 -- 172.21.15.69:0/2392992742 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f97b0030050 con 0x55f506f7d270 2024-04-24T09:07:44.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.476+0000 7f97d1e13740 10 client.15828 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:44.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.476+0000 7f97d1e13740 1 -- 172.21.15.69:0/2392992742 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f506bf43f0 con 0x55f506f7d270 2024-04-24T09:07:44.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.476+0000 7f97b6000640 1 -- 172.21.15.69:0/2392992742 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97b000ed20 con 0x55f506f7d270 2024-04-24T09:07:44.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.476+0000 7f97b6000640 1 -- 172.21.15.69:0/2392992742 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f97b002b030 con 0x55f506f7d270 2024-04-24T09:07:44.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.476+0000 7f97d1e13740 10 client.15828 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:44.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.476+0000 7f97d1e13740 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:44.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.476+0000 7f97d1e13740 20 client.15828 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:44.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.476+0000 7f97d1e13740 1 --2- 172.21.15.69:0/2392992742 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f506f97a10 0x55f506f99e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.476+0000 7f97d1e13740 4 client.15828 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:44.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.476+0000 7f97d1e13740 1 -- 172.21.15.69:0/2392992742 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f506eefbf0 con 0x55f506f97a10 2024-04-24T09:07:44.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.477+0000 7f97c7e00640 1 --2- 172.21.15.69:0/2392992742 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f506f97a10 0x55f506f99e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.478+0000 7f97c7e00640 1 --2- 172.21.15.69:0/2392992742 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f506f97a10 0x55f506f99e00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75782fc275301676 server_cookie=aaa4a2f792fe7e45 in_seq=0 out_seq=0 2024-04-24T09:07:44.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.478+0000 7f97b6000640 10 client.15828 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:44.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.479+0000 7f97b6000640 1 -- 172.21.15.69:0/2392992742 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f506eefbf0 con 0x55f506f97a10 2024-04-24T09:07:44.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.480+0000 7f97b6000640 10 client.15828 handle_command_reply: tid=0 2024-04-24T09:07:44.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.480+0000 7f97d1e13740 1 client.15828 shutdown 2024-04-24T09:07:44.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.480+0000 7f97d1e13740 1 -- 172.21.15.69:0/2392992742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7d270 msgr2=0x55f506f84640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.480+0000 7f97d1e13740 1 --2- 172.21.15.69:0/2392992742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7d270 0x55f506f84640 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f97b00056e0 tx=0x7f97b0005710 comp rx=0 tx=0).stop 2024-04-24T09:07:44.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.480+0000 7f97d1e13740 1 -- 172.21.15.69:0/2392992742 shutdown_connections 2024-04-24T09:07:44.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.480+0000 7f97d1e13740 1 --2- 172.21.15.69:0/2392992742 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f506f97a10 0x55f506f99e00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.480+0000 7f97d1e13740 1 --2- 172.21.15.69:0/2392992742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f87940 0x55f506f84c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.480+0000 7f97d1e13740 1 --2- 172.21.15.69:0/2392992742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7d270 0x55f506f84640 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.480+0000 7f97b6000640 0 client.15828 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:44.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.480+0000 7f97d1e13740 1 --2- 172.21.15.69:0/2392992742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f7c950 0x55f506f87350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.480+0000 7f97d1e13740 1 -- 172.21.15.69:0/2392992742 >> 172.21.15.69:0/2392992742 conn(0x55f506f78c70 msgr2=0x55f506f7ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.480+0000 7f97d1e13740 1 -- 172.21.15.69:0/2392992742 shutdown_connections 2024-04-24T09:07:44.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.480+0000 7f97d1e13740 1 -- 172.21.15.69:0/2392992742 wait complete. 2024-04-24T09:07:44.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.480+0000 7f97d1e13740 20 client.15828 trim_cache size 0 max 16384 2024-04-24T09:07:44.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.495+0000 7f97d1e13740 1 Processor -- start 2024-04-24T09:07:44.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.495+0000 7f97d1e13740 1 -- start start 2024-04-24T09:07:44.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.495+0000 7f97d1e13740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f7c950 0x55f506f78570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.495+0000 7f97d1e13740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7d270 0x55f506f83620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.495+0000 7f97d1e13740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f87940 0x55f506f86e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.495+0000 7f97d1e13740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f97c81b80b0 con 0x55f506f7d270 2024-04-24T09:07:44.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.495+0000 7f97d1e13740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f97c81b5980 con 0x55f506f7c950 2024-04-24T09:07:44.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.495+0000 7f97d1e13740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f97c81b37d0 con 0x55f506f87940 2024-04-24T09:07:44.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.495+0000 7f97bf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7d270 0x55f506f83620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.495+0000 7f97bf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7d270 0x55f506f83620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35302/0 (socket says 172.21.15.69:35302) 2024-04-24T09:07:44.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.496+0000 7f97bf400640 1 -- 172.21.15.69:0/2093881240 learned_addr learned my addr 172.21.15.69:0/2093881240 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:44.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.496+0000 7f97c7e00640 1 --2- 172.21.15.69:0/2093881240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f7c950 0x55f506f78570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.496+0000 7f97cd000640 1 --2- 172.21.15.69:0/2093881240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f87940 0x55f506f86e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.496+0000 7f97bf400640 1 -- 172.21.15.69:0/2093881240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f87940 msgr2=0x55f506f86e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.496+0000 7f97bf400640 1 --2- 172.21.15.69:0/2093881240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f87940 0x55f506f86e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.496+0000 7f97bf400640 1 -- 172.21.15.69:0/2093881240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f7c950 msgr2=0x55f506f78570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.496+0000 7f97bf400640 1 --2- 172.21.15.69:0/2093881240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f7c950 0x55f506f78570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.496+0000 7f97bf400640 1 -- 172.21.15.69:0/2093881240 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f506cbe7d0 con 0x55f506f7d270 2024-04-24T09:07:44.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.496+0000 7f97bf400640 1 --2- 172.21.15.69:0/2093881240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7d270 0x55f506f83620 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f97b00105a0 tx=0x7f97b0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cebc00adf6c79ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.496+0000 7f97bcc00640 1 -- 172.21.15.69:0/2093881240 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0038070 con 0x55f506f7d270 2024-04-24T09:07:44.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.497+0000 7f97bcc00640 1 -- 172.21.15.69:0/2093881240 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97b0009730 con 0x55f506f7d270 2024-04-24T09:07:44.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.497+0000 7f97bcc00640 1 -- 172.21.15.69:0/2093881240 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0016050 con 0x55f506f7d270 2024-04-24T09:07:44.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.497+0000 7f97d1e13740 1 -- 172.21.15.69:0/2093881240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7d270 msgr2=0x55f506f83620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.497+0000 7f97d1e13740 1 --2- 172.21.15.69:0/2093881240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7d270 0x55f506f83620 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f97b00105a0 tx=0x7f97b0005c50 comp rx=0 tx=0).stop 2024-04-24T09:07:44.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.497+0000 7f97d1e13740 1 -- 172.21.15.69:0/2093881240 shutdown_connections 2024-04-24T09:07:44.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.497+0000 7f97d1e13740 1 --2- 172.21.15.69:0/2093881240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f87940 0x55f506f86e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.497+0000 7f97d1e13740 1 --2- 172.21.15.69:0/2093881240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7d270 0x55f506f83620 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.497+0000 7f97d1e13740 1 --2- 172.21.15.69:0/2093881240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f7c950 0x55f506f78570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.497+0000 7f97d1e13740 1 -- 172.21.15.69:0/2093881240 >> 172.21.15.69:0/2093881240 conn(0x55f506f78c70 msgr2=0x55f506f82f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.497+0000 7f97d1e13740 1 -- 172.21.15.69:0/2093881240 shutdown_connections 2024-04-24T09:07:44.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.497+0000 7f97d1e13740 1 -- 172.21.15.69:0/2093881240 wait complete. 2024-04-24T09:07:44.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.498+0000 7f97d1e13740 1 -- start start 2024-04-24T09:07:44.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.498+0000 7f97d1e13740 1 Processor -- start 2024-04-24T09:07:44.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.498+0000 7f97d1e13740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7c950 0x55f506f8fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.498+0000 7f97d1e13740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f7d270 0x55f506f818c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.498+0000 7f97d1e13740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f87940 0x55f506f81e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.498+0000 7f97d1e13740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f97c81b3e40 con 0x55f506f7c950 2024-04-24T09:07:44.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.498+0000 7f97d1e13740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f97c81b52b0 con 0x55f506f87940 2024-04-24T09:07:44.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.498+0000 7f97d1e13740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f97c81b4510 con 0x55f506f7d270 2024-04-24T09:07:44.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.498+0000 7f97d1e13740 10 client.0 _ll_register_callbacks cb 0x7f97c0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:44.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.498+0000 7f97c7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7c950 0x55f506f8fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.499+0000 7f97c7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7c950 0x55f506f8fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35308/0 (socket says 172.21.15.69:35308) 2024-04-24T09:07:44.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.499+0000 7f97c7e00640 1 -- 172.21.15.69:0/1024618260 learned_addr learned my addr 172.21.15.69:0/1024618260 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:44.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.499+0000 7f97bf400640 1 --2- 172.21.15.69:0/1024618260 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f7d270 0x55f506f818c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.499+0000 7f97c7e00640 1 -- 172.21.15.69:0/1024618260 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f7d270 msgr2=0x55f506f818c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.499+0000 7f97c7e00640 1 --2- 172.21.15.69:0/1024618260 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f7d270 0x55f506f818c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.499+0000 7f97c7e00640 1 -- 172.21.15.69:0/1024618260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f87940 msgr2=0x55f506f81e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.499+0000 7f97c7e00640 1 --2- 172.21.15.69:0/1024618260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f87940 0x55f506f81e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.499+0000 7f97c7e00640 1 -- 172.21.15.69:0/1024618260 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f506f82340 con 0x55f506f7c950 2024-04-24T09:07:44.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.499+0000 7f97c7e00640 1 --2- 172.21.15.69:0/1024618260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7c950 0x55f506f8fdc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f97b800c8f0 tx=0x7f97b80bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b413f14a43b9e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.499+0000 7f97b6000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:44.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.499+0000 7f97b6000640 1 -- 172.21.15.69:0/1024618260 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b800de20 con 0x55f506f7c950 2024-04-24T09:07:44.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.499+0000 7f97b6000640 1 -- 172.21.15.69:0/1024618260 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f97b80a51b0 con 0x55f506f7c950 2024-04-24T09:07:44.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.499+0000 7f97d1e13740 1 -- 172.21.15.69:0/1024618260 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b0009730 con 0x55f506f7c950 2024-04-24T09:07:44.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.500+0000 7f97b6000640 1 -- 172.21.15.69:0/1024618260 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f97b809d0d0 con 0x55f506f7c950 2024-04-24T09:07:44.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.500+0000 7f97d1e13740 10 client.15834 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:44.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.500+0000 7f97d1e13740 1 -- 172.21.15.69:0/1024618260 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f506f88f40 con 0x55f506f7c950 2024-04-24T09:07:44.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.500+0000 7f97b6000640 1 -- 172.21.15.69:0/1024618260 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97b0009730 con 0x55f506f7c950 2024-04-24T09:07:44.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.500+0000 7f97b6000640 1 -- 172.21.15.69:0/1024618260 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f97b80ce4e0 con 0x55f506f7c950 2024-04-24T09:07:44.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.500+0000 7f97d1e13740 10 client.15834 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:44.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.500+0000 7f97d1e13740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:44.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.500+0000 7f97d1e13740 20 client.15834 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:44.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.500+0000 7f97d1e13740 1 --2- 172.21.15.69:0/1024618260 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f506ffe1a0 0x55f507000600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.500+0000 7f97d1e13740 4 client.15834 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:07:44.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.500+0000 7f97d1e13740 1 -- 172.21.15.69:0/1024618260 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f506ee9a60 con 0x55f506ffe1a0 2024-04-24T09:07:44.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.501+0000 7f97bf400640 1 --2- 172.21.15.69:0/1024618260 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f506ffe1a0 0x55f507000600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.502+0000 7f97bf400640 1 --2- 172.21.15.69:0/1024618260 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f506ffe1a0 0x55f507000600 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3578f02657d9d8c7 server_cookie=48a825610bbcc22a in_seq=0 out_seq=0 2024-04-24T09:07:44.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.502+0000 7f97b6000640 10 client.15834 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:44.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.502+0000 7f97b6000640 1 -- 172.21.15.69:0/1024618260 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f506ee9a60 con 0x55f506ffe1a0 2024-04-24T09:07:44.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.502+0000 7f97b6000640 10 client.15834 handle_command_reply: tid=0 2024-04-24T09:07:44.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.502+0000 7f97d1e13740 1 client.15834 shutdown 2024-04-24T09:07:44.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.502+0000 7f97d1e13740 1 -- 172.21.15.69:0/1024618260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7c950 msgr2=0x55f506f8fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.502+0000 7f97d1e13740 1 --2- 172.21.15.69:0/1024618260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7c950 0x55f506f8fdc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f97b800c8f0 tx=0x7f97b80bd040 comp rx=0 tx=0).stop 2024-04-24T09:07:44.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.502+0000 7f97d1e13740 1 -- 172.21.15.69:0/1024618260 shutdown_connections 2024-04-24T09:07:44.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.502+0000 7f97d1e13740 1 --2- 172.21.15.69:0/1024618260 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f506ffe1a0 0x55f507000600 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.502+0000 7f97d1e13740 1 --2- 172.21.15.69:0/1024618260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f506f87940 0x55f506f81e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.502+0000 7f97d1e13740 1 --2- 172.21.15.69:0/1024618260 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f506f7d270 0x55f506f818c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.502+0000 7f97d1e13740 1 --2- 172.21.15.69:0/1024618260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f506f7c950 0x55f506f8fdc0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.502+0000 7f97b6000640 0 client.15834 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:44.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.502+0000 7f97d1e13740 1 -- 172.21.15.69:0/1024618260 >> 172.21.15.69:0/1024618260 conn(0x55f506f78c70 msgr2=0x55f506f837a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.502+0000 7f97d1e13740 1 -- 172.21.15.69:0/1024618260 shutdown_connections 2024-04-24T09:07:44.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.502+0000 7f97d1e13740 1 -- 172.21.15.69:0/1024618260 wait complete. 2024-04-24T09:07:44.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.503+0000 7f97d1e13740 20 client.15834 trim_cache size 0 max 16384 2024-04-24T09:07:44.506 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-04-24T09:07:44.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.503+0000 7f97cf000640 1 -- 172.21.15.69:0/1929646943 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f97ac0a3db0 msgr2=0x7f97ac0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.503+0000 7f97cf000640 1 --2- 172.21.15.69:0/1929646943 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f97ac0a3db0 0x7f97ac0a6270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f97c0005fe0 tx=0x7f97c0009360 comp rx=0 tx=0).stop 2024-04-24T09:07:44.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.503+0000 7f97cf000640 1 -- 172.21.15.69:0/1929646943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c8104450 msgr2=0x7f97c8076510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.503+0000 7f97cf000640 1 --2- 172.21.15.69:0/1929646943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c8104450 0x7f97c8076510 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f97b800c920 tx=0x7f97b800cde0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.503+0000 7f97cf000640 1 -- 172.21.15.69:0/1929646943 shutdown_connections 2024-04-24T09:07:44.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.503+0000 7f97cf000640 1 --2- 172.21.15.69:0/1929646943 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f97ac0a3db0 0x7f97ac0a6270 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.503+0000 7f97cf000640 1 --2- 172.21.15.69:0/1929646943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97c8076f90 0x7f97c81aaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.503+0000 7f97cf000640 1 --2- 172.21.15.69:0/1929646943 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f97c8109520 0x7f97c8076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.503+0000 7f97cf000640 1 --2- 172.21.15.69:0/1929646943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c8104450 0x7f97c8076510 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.503+0000 7f97cf000640 1 -- 172.21.15.69:0/1929646943 >> 172.21.15.69:0/1929646943 conn(0x7f97c80ffc70 msgr2=0x7f97c8101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.503+0000 7f97cf000640 1 -- 172.21.15.69:0/1929646943 shutdown_connections 2024-04-24T09:07:44.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:44.503+0000 7f97cf000640 1 -- 172.21.15.69:0/1929646943 wait complete. 2024-04-24T09:07:44.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:44 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3032986155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:07:45.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:44 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3032986155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:07:45.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:44 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3032986155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:07:45.518 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:07:45.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:07:45.519 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:07:45.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.739+0000 7f186da00640 1 -- 172.21.15.69:0/261273980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f186810c480 msgr2=0x7f186810e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:45.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.739+0000 7f186da00640 1 --2- 172.21.15.69:0/261273980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f186810c480 0x7f186810e920 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1860009900 tx=0x7f186002a480 comp rx=0 tx=0).stop 2024-04-24T09:07:45.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.739+0000 7f186da00640 1 -- 172.21.15.69:0/261273980 shutdown_connections 2024-04-24T09:07:45.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.739+0000 7f186da00640 1 --2- 172.21.15.69:0/261273980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f186810c480 0x7f186810e920 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.739+0000 7f186da00640 1 --2- 172.21.15.69:0/261273980 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1868103980 0x7f186810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.739+0000 7f186da00640 1 --2- 172.21.15.69:0/261273980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1868103060 0x7f1868103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.739+0000 7f186da00640 1 -- 172.21.15.69:0/261273980 >> 172.21.15.69:0/261273980 conn(0x7f18680fc880 msgr2=0x7f18680feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:45.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.739+0000 7f186da00640 1 -- 172.21.15.69:0/261273980 shutdown_connections 2024-04-24T09:07:45.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.739+0000 7f186da00640 1 -- 172.21.15.69:0/261273980 wait complete. 2024-04-24T09:07:45.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.740+0000 7f186da00640 1 Processor -- start 2024-04-24T09:07:45.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.740+0000 7f186da00640 1 -- start start 2024-04-24T09:07:45.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.741+0000 7f186da00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1868103060 0x7f18681a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.741+0000 7f186da00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1868103980 0x7f18681a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.741+0000 7f186da00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f186810c480 0x7f186819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.741+0000 7f186da00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18681120f0 con 0x7f186810c480 2024-04-24T09:07:45.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.741+0000 7f186da00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1868111f70 con 0x7f1868103060 2024-04-24T09:07:45.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.741+0000 7f186da00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1868112270 con 0x7f1868103980 2024-04-24T09:07:45.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.741+0000 7f186d000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f186810c480 0x7f186819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.741+0000 7f186d000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f186810c480 0x7f186819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35336/0 (socket says 172.21.15.69:35336) 2024-04-24T09:07:45.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.741+0000 7f186d000640 1 -- 172.21.15.69:0/2707126815 learned_addr learned my addr 172.21.15.69:0/2707126815 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:45.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.741+0000 7f1867e00640 1 --2- 172.21.15.69:0/2707126815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1868103060 0x7f18681a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.741+0000 7f1867400640 1 --2- 172.21.15.69:0/2707126815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1868103980 0x7f18681a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.742+0000 7f186d000640 1 -- 172.21.15.69:0/2707126815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1868103980 msgr2=0x7f18681a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:45.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.742+0000 7f186d000640 1 --2- 172.21.15.69:0/2707126815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1868103980 0x7f18681a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.742+0000 7f186d000640 1 -- 172.21.15.69:0/2707126815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1868103060 msgr2=0x7f18681a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:45.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.742+0000 7f186d000640 1 --2- 172.21.15.69:0/2707126815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1868103060 0x7f18681a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.742+0000 7f186d000640 1 -- 172.21.15.69:0/2707126815 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186819afd0 con 0x7f186810c480 2024-04-24T09:07:45.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.742+0000 7f1867400640 1 --2- 172.21.15.69:0/2707126815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1868103980 0x7f18681a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:45.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.742+0000 7f186d000640 1 --2- 172.21.15.69:0/2707126815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f186810c480 0x7f186819a710 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1860009a20 tx=0x7f18600042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:45.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.742+0000 7f1864c00640 1 -- 172.21.15.69:0/2707126815 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186003f030 con 0x7f186810c480 2024-04-24T09:07:45.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.742+0000 7f1864c00640 1 -- 172.21.15.69:0/2707126815 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18600044d0 con 0x7f186810c480 2024-04-24T09:07:45.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.742+0000 7f1864c00640 1 -- 172.21.15.69:0/2707126815 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860035070 con 0x7f186810c480 2024-04-24T09:07:45.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.742+0000 7f186da00640 1 -- 172.21.15.69:0/2707126815 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186819b260 con 0x7f186810c480 2024-04-24T09:07:45.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.742+0000 7f186da00640 1 -- 172.21.15.69:0/2707126815 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18681a7300 con 0x7f186810c480 2024-04-24T09:07:45.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.746+0000 7f1871e1e740 1 Processor -- start 2024-04-24T09:07:45.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.746+0000 7f1864c00640 1 -- 172.21.15.69:0/2707126815 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f1860042030 con 0x7f186810c480 2024-04-24T09:07:45.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.746+0000 7f1871e1e740 1 -- start start 2024-04-24T09:07:45.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.746+0000 7f1871e1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd661270 0x563ffd661650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.746+0000 7f1871e1e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd661b90 0x563ffd66bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.746+0000 7f1871e1e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd66c260 0x563ffd66e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.746+0000 7f1871e1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563ffd5c6e70 con 0x563ffd661270 2024-04-24T09:07:45.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.746+0000 7f1871e1e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563ffd56d380 con 0x563ffd66c260 2024-04-24T09:07:45.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.746+0000 7f1871e1e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563ffd5c2e30 con 0x563ffd661b90 2024-04-24T09:07:45.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.747+0000 7f1867400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd661270 0x563ffd661650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.747+0000 7f1867400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd661270 0x563ffd661650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35352/0 (socket says 172.21.15.69:35352) 2024-04-24T09:07:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.747+0000 7f1867400640 1 -- 172.21.15.69:0/1738294540 learned_addr learned my addr 172.21.15.69:0/1738294540 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.747+0000 7f1867e00640 1 --2- 172.21.15.69:0/1738294540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd661b90 0x563ffd66bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.747+0000 7f1867400640 1 -- 172.21.15.69:0/1738294540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd661b90 msgr2=0x563ffd66bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.747+0000 7f1867400640 1 --2- 172.21.15.69:0/1738294540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd661b90 0x563ffd66bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.747+0000 7f1867400640 1 -- 172.21.15.69:0/1738294540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd66c260 msgr2=0x563ffd66e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.747+0000 7f1867400640 1 --2- 172.21.15.69:0/1738294540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd66c260 0x563ffd66e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.747+0000 7f1867400640 1 -- 172.21.15.69:0/1738294540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ffd3a17e0 con 0x563ffd661270 2024-04-24T09:07:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.747+0000 7f1864c00640 1 --2- 172.21.15.69:0/2707126815 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f184c0a3e30 0x7f184c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.747+0000 7f1867400640 1 --2- 172.21.15.69:0/1738294540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd661270 0x563ffd661650 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f185000bb50 tx=0x7f185000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60d8ed59008a7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:45.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.748+0000 7f1864c00640 1 -- 172.21.15.69:0/2707126815 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f18600f1660 con 0x7f186810c480 2024-04-24T09:07:45.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.748+0000 7f1867e00640 1 --2- 172.21.15.69:0/2707126815 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f184c0a3e30 0x7f184c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.748+0000 7f185d600640 1 -- 172.21.15.69:0/1738294540 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850011670 con 0x563ffd661270 2024-04-24T09:07:45.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.748+0000 7f185d600640 1 -- 172.21.15.69:0/1738294540 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f185000be50 con 0x563ffd661270 2024-04-24T09:07:45.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.748+0000 7f185d600640 1 -- 172.21.15.69:0/1738294540 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850019b80 con 0x563ffd661270 2024-04-24T09:07:45.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.749+0000 7f1871e1e740 1 -- 172.21.15.69:0/1738294540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd661270 msgr2=0x563ffd661650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:45.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.749+0000 7f1867e00640 1 --2- 172.21.15.69:0/2707126815 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f184c0a3e30 0x7f184c0a62f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x563ffd662830 tx=0x7f18580078b0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:45.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.749+0000 7f1871e1e740 1 --2- 172.21.15.69:0/1738294540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd661270 0x563ffd661650 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f185000bb50 tx=0x7f185000f730 comp rx=0 tx=0).stop 2024-04-24T09:07:45.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.749+0000 7f1871e1e740 1 -- 172.21.15.69:0/1738294540 shutdown_connections 2024-04-24T09:07:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.749+0000 7f1871e1e740 1 --2- 172.21.15.69:0/1738294540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd66c260 0x563ffd66e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.749+0000 7f1871e1e740 1 --2- 172.21.15.69:0/1738294540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd661b90 0x563ffd66bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.749+0000 7f1871e1e740 1 --2- 172.21.15.69:0/1738294540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd661270 0x563ffd661650 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.749+0000 7f1871e1e740 1 -- 172.21.15.69:0/1738294540 >> 172.21.15.69:0/1738294540 conn(0x563ffd65d590 msgr2=0x563ffd65d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.749+0000 7f1871e1e740 1 -- 172.21.15.69:0/1738294540 shutdown_connections 2024-04-24T09:07:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.749+0000 7f1871e1e740 1 -- 172.21.15.69:0/1738294540 wait complete. 2024-04-24T09:07:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.749+0000 7f1871e1e740 1 -- start start 2024-04-24T09:07:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.749+0000 7f1871e1e740 1 Processor -- start 2024-04-24T09:07:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.750+0000 7f1871e1e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd661270 0x563ffd669270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.750+0000 7f1871e1e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd661b90 0x563ffd6697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.750+0000 7f1871e1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd66c260 0x563ffd669cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.750+0000 7f1871e1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563ffd66f050 con 0x563ffd66c260 2024-04-24T09:07:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.750+0000 7f1871e1e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563ffd580660 con 0x563ffd661b90 2024-04-24T09:07:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.750+0000 7f1871e1e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563ffd66f1d0 con 0x563ffd661270 2024-04-24T09:07:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.750+0000 7f1871e1e740 10 client.0 _ll_register_callbacks cb 0x563ffd56eaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.750+0000 7f186d000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd66c260 0x563ffd669cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.750+0000 7f186d000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd66c260 0x563ffd669cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35356/0 (socket says 172.21.15.69:35356) 2024-04-24T09:07:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.750+0000 7f186d000640 1 -- 172.21.15.69:0/1358256016 learned_addr learned my addr 172.21.15.69:0/1358256016 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.751+0000 7f1867e00640 1 --2- 172.21.15.69:0/1358256016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd661270 0x563ffd669270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.751+0000 7f1867400640 1 --2- 172.21.15.69:0/1358256016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd661b90 0x563ffd6697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.751+0000 7f186d000640 1 -- 172.21.15.69:0/1358256016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd661270 msgr2=0x563ffd669270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.751+0000 7f186d000640 1 --2- 172.21.15.69:0/1358256016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd661270 0x563ffd669270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.751+0000 7f186d000640 1 -- 172.21.15.69:0/1358256016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd661b90 msgr2=0x563ffd6697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.751+0000 7f186d000640 1 --2- 172.21.15.69:0/1358256016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd661b90 0x563ffd6697b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.751+0000 7f186d000640 1 -- 172.21.15.69:0/1358256016 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ffd401f70 con 0x563ffd66c260 2024-04-24T09:07:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.751+0000 7f186d000640 1 --2- 172.21.15.69:0/1358256016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd66c260 0x563ffd669cf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f18600b9550 tx=0x7f18600bd8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c913b8e337698a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.751+0000 7f1856000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.751+0000 7f1856000640 1 -- 172.21.15.69:0/1358256016 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860049040 con 0x563ffd66c260 2024-04-24T09:07:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.751+0000 7f1856000640 1 -- 172.21.15.69:0/1358256016 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f18600d65f0 con 0x563ffd66c260 2024-04-24T09:07:45.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.751+0000 7f1871e1e740 1 -- 172.21.15.69:0/1358256016 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f185000be50 con 0x563ffd66c260 2024-04-24T09:07:45.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.751+0000 7f1856000640 1 -- 172.21.15.69:0/1358256016 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f18600c5d60 con 0x563ffd66c260 2024-04-24T09:07:45.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.751+0000 7f1871e1e740 10 client.15846 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:45.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.751+0000 7f1871e1e740 1 -- 172.21.15.69:0/1358256016 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563ffd299170 con 0x563ffd66c260 2024-04-24T09:07:45.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.752+0000 7f1856000640 1 -- 172.21.15.69:0/1358256016 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f185000be50 con 0x563ffd66c260 2024-04-24T09:07:45.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.752+0000 7f1856000640 1 -- 172.21.15.69:0/1358256016 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f18600eec00 con 0x563ffd66c260 2024-04-24T09:07:45.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.752+0000 7f1871e1e740 10 client.15846 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:45.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.752+0000 7f1871e1e740 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:45.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.752+0000 7f1871e1e740 20 client.15846 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:45.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.752+0000 7f1871e1e740 1 --2- 172.21.15.69:0/1358256016 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x563ffd67c620 0x563ffd67ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.752+0000 7f1871e1e740 4 client.15846 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:45.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.752+0000 7f1871e1e740 1 -- 172.21.15.69:0/1358256016 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ffd5d4890 con 0x563ffd67c620 2024-04-24T09:07:45.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.753+0000 7f1867e00640 1 --2- 172.21.15.69:0/1358256016 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x563ffd67c620 0x563ffd67ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.753+0000 7f1867e00640 1 --2- 172.21.15.69:0/1358256016 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x563ffd67c620 0x563ffd67ea80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d68fe2e2a6b52b32 server_cookie=32c54daa23c72884 in_seq=0 out_seq=0 2024-04-24T09:07:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.753+0000 7f1856000640 10 client.15846 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.755+0000 7f1856000640 1 -- 172.21.15.69:0/1358256016 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ffd5d4890 con 0x563ffd67c620 2024-04-24T09:07:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.755+0000 7f1856000640 10 client.15846 handle_command_reply: tid=0 2024-04-24T09:07:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.755+0000 7f1871e1e740 1 client.15846 shutdown 2024-04-24T09:07:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.756+0000 7f1871e1e740 1 -- 172.21.15.69:0/1358256016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd66c260 msgr2=0x563ffd669cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.756+0000 7f1871e1e740 1 --2- 172.21.15.69:0/1358256016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd66c260 0x563ffd669cf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f18600b9550 tx=0x7f18600bd8b0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.756+0000 7f1871e1e740 1 -- 172.21.15.69:0/1358256016 shutdown_connections 2024-04-24T09:07:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.756+0000 7f1871e1e740 1 --2- 172.21.15.69:0/1358256016 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x563ffd67c620 0x563ffd67ea80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.756+0000 7f1871e1e740 1 --2- 172.21.15.69:0/1358256016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd66c260 0x563ffd669cf0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.756+0000 7f1871e1e740 1 --2- 172.21.15.69:0/1358256016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd661b90 0x563ffd6697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.756+0000 7f1871e1e740 1 --2- 172.21.15.69:0/1358256016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd661270 0x563ffd669270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.756+0000 7f1856000640 0 client.15846 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.756+0000 7f1871e1e740 1 -- 172.21.15.69:0/1358256016 >> 172.21.15.69:0/1358256016 conn(0x563ffd65d590 msgr2=0x563ffd663290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.756+0000 7f1871e1e740 1 -- 172.21.15.69:0/1358256016 shutdown_connections 2024-04-24T09:07:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.756+0000 7f1871e1e740 1 -- 172.21.15.69:0/1358256016 wait complete. 2024-04-24T09:07:45.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.756+0000 7f1871e1e740 20 client.15846 trim_cache size 0 max 16384 2024-04-24T09:07:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.776+0000 7f1871e1e740 1 Processor -- start 2024-04-24T09:07:45.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.776+0000 7f1871e1e740 1 -- start start 2024-04-24T09:07:45.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.776+0000 7f1871e1e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd661270 0x563ffd677f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.776+0000 7f1871e1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd661b90 0x563ffd6639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.776+0000 7f1871e1e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd66c260 0x563ffd684390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.776+0000 7f1871e1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18681b3a00 con 0x563ffd661b90 2024-04-24T09:07:45.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.776+0000 7f1871e1e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f18681b12d0 con 0x563ffd661270 2024-04-24T09:07:45.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.776+0000 7f1871e1e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f18681af120 con 0x563ffd66c260 2024-04-24T09:07:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.776+0000 7f1867400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd661b90 0x563ffd6639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.776+0000 7f1867400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd661b90 0x563ffd6639d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35370/0 (socket says 172.21.15.69:35370) 2024-04-24T09:07:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.776+0000 7f1867400640 1 -- 172.21.15.69:0/1224785906 learned_addr learned my addr 172.21.15.69:0/1224785906 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.777+0000 7f1867e00640 1 --2- 172.21.15.69:0/1224785906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd661270 0x563ffd677f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.777+0000 7f186d000640 1 --2- 172.21.15.69:0/1224785906 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd66c260 0x563ffd684390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.777+0000 7f1867400640 1 -- 172.21.15.69:0/1224785906 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd66c260 msgr2=0x563ffd684390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.777+0000 7f1867400640 1 --2- 172.21.15.69:0/1224785906 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd66c260 0x563ffd684390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.777+0000 7f1867400640 1 -- 172.21.15.69:0/1224785906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd661270 msgr2=0x563ffd677f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.777+0000 7f1867400640 1 --2- 172.21.15.69:0/1224785906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd661270 0x563ffd677f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.777+0000 7f1867400640 1 -- 172.21.15.69:0/1224785906 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ffd67f100 con 0x563ffd661b90 2024-04-24T09:07:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.777+0000 7f1867400640 1 --2- 172.21.15.69:0/1224785906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd661b90 0x563ffd6639d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f185000ffd0 tx=0x7f1850004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c75f15f549c656c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.777+0000 7f185cc00640 1 -- 172.21.15.69:0/1224785906 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185000b790 con 0x563ffd661b90 2024-04-24T09:07:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.778+0000 7f185cc00640 1 -- 172.21.15.69:0/1224785906 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1850011920 con 0x563ffd661b90 2024-04-24T09:07:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.778+0000 7f185cc00640 1 -- 172.21.15.69:0/1224785906 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850019980 con 0x563ffd661b90 2024-04-24T09:07:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.778+0000 7f1871e1e740 1 -- 172.21.15.69:0/1224785906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd661b90 msgr2=0x563ffd6639d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.778+0000 7f1871e1e740 1 --2- 172.21.15.69:0/1224785906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd661b90 0x563ffd6639d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f185000ffd0 tx=0x7f1850004720 comp rx=0 tx=0).stop 2024-04-24T09:07:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.778+0000 7f1871e1e740 1 -- 172.21.15.69:0/1224785906 shutdown_connections 2024-04-24T09:07:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.778+0000 7f1871e1e740 1 --2- 172.21.15.69:0/1224785906 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd66c260 0x563ffd684390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.778+0000 7f1871e1e740 1 --2- 172.21.15.69:0/1224785906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd661b90 0x563ffd6639d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.778+0000 7f1871e1e740 1 --2- 172.21.15.69:0/1224785906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd661270 0x563ffd677f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.778+0000 7f1871e1e740 1 -- 172.21.15.69:0/1224785906 >> 172.21.15.69:0/1224785906 conn(0x563ffd65d590 msgr2=0x563ffd65c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.778+0000 7f1871e1e740 1 -- 172.21.15.69:0/1224785906 shutdown_connections 2024-04-24T09:07:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.778+0000 7f1871e1e740 1 -- 172.21.15.69:0/1224785906 wait complete. 2024-04-24T09:07:45.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.779+0000 7f1871e1e740 1 -- start start 2024-04-24T09:07:45.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.779+0000 7f1871e1e740 1 Processor -- start 2024-04-24T09:07:45.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f1871e1e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd661270 0x563ffd674800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f1871e1e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd661b90 0x563ffd670b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f1871e1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd66c260 0x563ffd6710c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f1871e1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18681af790 con 0x563ffd66c260 2024-04-24T09:07:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f1871e1e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f18681b0c00 con 0x563ffd661270 2024-04-24T09:07:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f1871e1e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f18681afe60 con 0x563ffd661b90 2024-04-24T09:07:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f1871e1e740 10 client.0 _ll_register_callbacks cb 0x7f18600ee7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f186d000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd66c260 0x563ffd6710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f186d000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd66c260 0x563ffd6710c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35372/0 (socket says 172.21.15.69:35372) 2024-04-24T09:07:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f186d000640 1 -- 172.21.15.69:0/3461432337 learned_addr learned my addr 172.21.15.69:0/3461432337 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f1867e00640 1 --2- 172.21.15.69:0/3461432337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd661270 0x563ffd674800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f1867400640 1 --2- 172.21.15.69:0/3461432337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd661b90 0x563ffd670b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f186d000640 1 -- 172.21.15.69:0/3461432337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd661b90 msgr2=0x563ffd670b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f186d000640 1 --2- 172.21.15.69:0/3461432337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd661b90 0x563ffd670b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f186d000640 1 -- 172.21.15.69:0/3461432337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd661270 msgr2=0x563ffd674800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f186d000640 1 --2- 172.21.15.69:0/3461432337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd661270 0x563ffd674800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f186d000640 1 -- 172.21.15.69:0/3461432337 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ffd6718f0 con 0x563ffd66c260 2024-04-24T09:07:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.780+0000 7f1867e00640 1 --2- 172.21.15.69:0/3461432337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd661270 0x563ffd674800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.781+0000 7f186d000640 1 --2- 172.21.15.69:0/3461432337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd66c260 0x563ffd6710c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f18600bd080 tx=0x7f18600bcc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6294f8aaba89bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.781+0000 7f1856000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.781+0000 7f1856000640 1 -- 172.21.15.69:0/3461432337 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860049040 con 0x563ffd66c260 2024-04-24T09:07:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.781+0000 7f1871e1e740 1 -- 172.21.15.69:0/3461432337 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1850011920 con 0x563ffd66c260 2024-04-24T09:07:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.781+0000 7f1856000640 1 -- 172.21.15.69:0/3461432337 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f18600c58d0 con 0x563ffd66c260 2024-04-24T09:07:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.781+0000 7f1856000640 1 -- 172.21.15.69:0/3461432337 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f18600c5ad0 con 0x563ffd66c260 2024-04-24T09:07:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.781+0000 7f1871e1e740 10 client.15852 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.781+0000 7f1871e1e740 1 -- 172.21.15.69:0/3461432337 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563ffd6e3c40 con 0x563ffd66c260 2024-04-24T09:07:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.781+0000 7f1856000640 1 -- 172.21.15.69:0/3461432337 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1850011920 con 0x563ffd66c260 2024-04-24T09:07:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.782+0000 7f1856000640 1 -- 172.21.15.69:0/3461432337 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f18600d1b90 con 0x563ffd66c260 2024-04-24T09:07:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.782+0000 7f1871e1e740 10 client.15852 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.782+0000 7f1871e1e740 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.782+0000 7f1871e1e740 20 client.15852 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.782+0000 7f1871e1e740 1 --2- 172.21.15.69:0/3461432337 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x563ffd6e4220 0x563ffd6e6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.782+0000 7f1871e1e740 4 client.15852 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:07:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.782+0000 7f1871e1e740 1 -- 172.21.15.69:0/3461432337 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563ffd5ce700 con 0x563ffd6e4220 2024-04-24T09:07:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.782+0000 7f1867e00640 1 --2- 172.21.15.69:0/3461432337 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x563ffd6e4220 0x563ffd6e6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.783+0000 7f1867e00640 1 --2- 172.21.15.69:0/3461432337 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x563ffd6e4220 0x563ffd6e6610 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a26304459bd18059 server_cookie=a5078ed29247b6d1 in_seq=0 out_seq=0 2024-04-24T09:07:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.783+0000 7f1856000640 10 client.15852 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.786+0000 7f1856000640 1 -- 172.21.15.69:0/3461432337 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563ffd5ce700 con 0x563ffd6e4220 2024-04-24T09:07:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.786+0000 7f1856000640 10 client.15852 handle_command_reply: tid=0 2024-04-24T09:07:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.786+0000 7f1871e1e740 1 client.15852 shutdown 2024-04-24T09:07:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.786+0000 7f1871e1e740 1 -- 172.21.15.69:0/3461432337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd66c260 msgr2=0x563ffd6710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.786+0000 7f1871e1e740 1 --2- 172.21.15.69:0/3461432337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd66c260 0x563ffd6710c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f18600bd080 tx=0x7f18600bcc30 comp rx=0 tx=0).stop 2024-04-24T09:07:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.786+0000 7f1871e1e740 1 -- 172.21.15.69:0/3461432337 shutdown_connections 2024-04-24T09:07:45.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.786+0000 7f1871e1e740 1 --2- 172.21.15.69:0/3461432337 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x563ffd6e4220 0x563ffd6e6610 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.786+0000 7f1871e1e740 1 --2- 172.21.15.69:0/3461432337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ffd66c260 0x563ffd6710c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.786+0000 7f1871e1e740 1 --2- 172.21.15.69:0/3461432337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563ffd661b90 0x563ffd670b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.786+0000 7f1871e1e740 1 --2- 172.21.15.69:0/3461432337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563ffd661270 0x563ffd674800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.786+0000 7f1871e1e740 1 -- 172.21.15.69:0/3461432337 >> 172.21.15.69:0/3461432337 conn(0x563ffd65d590 msgr2=0x563ffd669230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:45.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.786+0000 7f1856000640 0 client.15852 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:45.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.786+0000 7f1871e1e740 1 -- 172.21.15.69:0/3461432337 shutdown_connections 2024-04-24T09:07:45.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.786+0000 7f1871e1e740 1 -- 172.21.15.69:0/3461432337 wait complete. 2024-04-24T09:07:45.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.786+0000 7f1871e1e740 20 client.15852 trim_cache size 0 max 16384 2024-04-24T09:07:45.786 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:07:45.787 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-04-24T09:07:45.787 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "26238a16-ab65-41ad-8c22-678242d4fdbe", 2024-04-24T09:07:45.787 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-04-24T09:07:45.787 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:07:45.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.787+0000 7f186da00640 1 -- 172.21.15.69:0/2707126815 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f184c0a3e30 msgr2=0x7f184c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.787+0000 7f186da00640 1 --2- 172.21.15.69:0/2707126815 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f184c0a3e30 0x7f184c0a62f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x563ffd662830 tx=0x7f18580078b0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.787+0000 7f186da00640 1 -- 172.21.15.69:0/2707126815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f186810c480 msgr2=0x7f186819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.787+0000 7f186da00640 1 --2- 172.21.15.69:0/2707126815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f186810c480 0x7f186819a710 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1860009a20 tx=0x7f18600042e0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.787+0000 7f186da00640 1 -- 172.21.15.69:0/2707126815 shutdown_connections 2024-04-24T09:07:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.787+0000 7f186da00640 1 --2- 172.21.15.69:0/2707126815 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f184c0a3e30 0x7f184c0a62f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.787+0000 7f186da00640 1 --2- 172.21.15.69:0/2707126815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f186810c480 0x7f186819a710 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.787+0000 7f186da00640 1 --2- 172.21.15.69:0/2707126815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1868103980 0x7f18681a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.787+0000 7f186da00640 1 --2- 172.21.15.69:0/2707126815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1868103060 0x7f18681a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.787+0000 7f186da00640 1 -- 172.21.15.69:0/2707126815 >> 172.21.15.69:0/2707126815 conn(0x7f18680fc880 msgr2=0x7f1868101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.787+0000 7f186da00640 1 -- 172.21.15.69:0/2707126815 shutdown_connections 2024-04-24T09:07:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:45.787+0000 7f186da00640 1 -- 172.21.15.69:0/2707126815 wait complete. 2024-04-24T09:07:45.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:07:45.804 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:07:45.865 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:45 smithi069 ceph-mon[28170]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:07:45.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:45 smithi130 ceph-mon[31553]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:07:46.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.002+0000 7fbfe6600640 1 -- 172.21.15.69:0/3788785387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfe0108f40 msgr2=0x7fbfe01153b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:46.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.002+0000 7fbfe6600640 1 --2- 172.21.15.69:0/3788785387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfe0108f40 0x7fbfe01153b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009cd0 tx=0x7fbfd802a8e0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.003+0000 7fbfe6600640 1 -- 172.21.15.69:0/3788785387 shutdown_connections 2024-04-24T09:07:46.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.003+0000 7fbfe6600640 1 --2- 172.21.15.69:0/3788785387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfe0108f40 0x7fbfe01153b0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.003+0000 7fbfe6600640 1 --2- 172.21.15.69:0/3788785387 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbfe0108580 0x7fbfe0108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.003+0000 7fbfe6600640 1 --2- 172.21.15.69:0/3788785387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfe010d500 0x7fbfe010d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.003+0000 7fbfe6600640 1 -- 172.21.15.69:0/3788785387 >> 172.21.15.69:0/3788785387 conn(0x7fbfe0077e20 msgr2=0x7fbfe0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:46.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.003+0000 7fbfe6600640 1 -- 172.21.15.69:0/3788785387 shutdown_connections 2024-04-24T09:07:46.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.003+0000 7fbfe6600640 1 -- 172.21.15.69:0/3788785387 wait complete. 2024-04-24T09:07:46.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.004+0000 7fbfe6600640 1 Processor -- start 2024-04-24T09:07:46.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.004+0000 7fbfe6600640 1 -- start start 2024-04-24T09:07:46.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.004+0000 7fbfe6600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfe0108580 0x7fbfe01a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.004+0000 7fbfe6600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbfe0108f40 0x7fbfe01a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.004+0000 7fbfe6600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfe010d500 0x7fbfe019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.004+0000 7fbfe6600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe01187d0 con 0x7fbfe010d500 2024-04-24T09:07:46.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.004+0000 7fbfe6600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe0118650 con 0x7fbfe0108580 2024-04-24T09:07:46.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.004+0000 7fbfe6600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe0118950 con 0x7fbfe0108f40 2024-04-24T09:07:46.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.005+0000 7fbfe5c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfe010d500 0x7fbfe019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.005+0000 7fbfe5c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfe010d500 0x7fbfe019ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35398/0 (socket says 172.21.15.69:35398) 2024-04-24T09:07:46.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.005+0000 7fbfe5c00640 1 -- 172.21.15.69:0/2205736461 learned_addr learned my addr 172.21.15.69:0/2205736461 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:46.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.005+0000 7fbfe5200640 1 --2- 172.21.15.69:0/2205736461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfe0108580 0x7fbfe01a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.005+0000 7fbfdfe00640 1 --2- 172.21.15.69:0/2205736461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbfe0108f40 0x7fbfe01a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.005+0000 7fbfe5c00640 1 -- 172.21.15.69:0/2205736461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbfe0108f40 msgr2=0x7fbfe01a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:46.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.005+0000 7fbfe5c00640 1 --2- 172.21.15.69:0/2205736461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbfe0108f40 0x7fbfe01a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.005+0000 7fbfe5c00640 1 -- 172.21.15.69:0/2205736461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfe0108580 msgr2=0x7fbfe01a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:46.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.005+0000 7fbfe5c00640 1 --2- 172.21.15.69:0/2205736461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfe0108580 0x7fbfe01a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.005+0000 7fbfe5c00640 1 -- 172.21.15.69:0/2205736461 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe019f1f0 con 0x7fbfe010d500 2024-04-24T09:07:46.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.005+0000 7fbfe5c00640 1 --2- 172.21.15.69:0/2205736461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfe010d500 0x7fbfe019ea70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8005e30 tx=0x7fbfd8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:46.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.006+0000 7fbfdd600640 1 -- 172.21.15.69:0/2205736461 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8004240 con 0x7fbfe010d500 2024-04-24T09:07:46.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.006+0000 7fbfdd600640 1 -- 172.21.15.69:0/2205736461 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfd8031d00 con 0x7fbfe010d500 2024-04-24T09:07:46.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.006+0000 7fbfe6600640 1 -- 172.21.15.69:0/2205736461 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe019f480 con 0x7fbfe010d500 2024-04-24T09:07:46.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.006+0000 7fbfdd600640 1 -- 172.21.15.69:0/2205736461 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8030e30 con 0x7fbfe010d500 2024-04-24T09:07:46.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.006+0000 7fbfe6600640 1 -- 172.21.15.69:0/2205736461 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe01ab780 con 0x7fbfe010d500 2024-04-24T09:07:46.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.009+0000 7fbfdd600640 1 -- 172.21.15.69:0/2205736461 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fbfd80043e0 con 0x7fbfe010d500 2024-04-24T09:07:46.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.009+0000 7fbfeaa7f740 1 Processor -- start 2024-04-24T09:07:46.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.009+0000 7fbfeaa7f740 1 -- start start 2024-04-24T09:07:46.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.010+0000 7fbfeaa7f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaad0a30 0x55e7aaad0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.010+0000 7fbfeaa7f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad1350 0x55e7aaadb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.010+0000 7fbfeaa7f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaadba20 0x55e7aaadde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.010+0000 7fbfeaa7f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e7aaa361f0 con 0x55e7aaad0a30 2024-04-24T09:07:46.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.010+0000 7fbfeaa7f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e7aa9dc350 con 0x55e7aaad1350 2024-04-24T09:07:46.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.010+0000 7fbfeaa7f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55e7aaa321b0 con 0x55e7aaadba20 2024-04-24T09:07:46.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.010+0000 7fbfdfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaad0a30 0x55e7aaad0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.010+0000 7fbfdfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaad0a30 0x55e7aaad0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35400/0 (socket says 172.21.15.69:35400) 2024-04-24T09:07:46.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.010+0000 7fbfdfe00640 1 -- 172.21.15.69:0/3154547428 learned_addr learned my addr 172.21.15.69:0/3154547428 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.010+0000 7fbfe5200640 1 --2- 172.21.15.69:0/3154547428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad1350 0x55e7aaadb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.011+0000 7fbfdfe00640 1 -- 172.21.15.69:0/3154547428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaadba20 msgr2=0x55e7aaadde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.011+0000 7fbfdfe00640 1 --2- 172.21.15.69:0/3154547428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaadba20 0x55e7aaadde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.011+0000 7fbfdfe00640 1 -- 172.21.15.69:0/3154547428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad1350 msgr2=0x55e7aaadb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.011+0000 7fbfdfe00640 1 --2- 172.21.15.69:0/3154547428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad1350 0x55e7aaadb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.011+0000 7fbfdfe00640 1 -- 172.21.15.69:0/3154547428 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7aaa30f50 con 0x55e7aaad0a30 2024-04-24T09:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.011+0000 7fbfe5200640 1 --2- 172.21.15.69:0/3154547428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad1350 0x55e7aaadb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.011+0000 7fbfdd600640 1 --2- 172.21.15.69:0/2205736461 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fbfc80a3e30 0x7fbfc80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.011+0000 7fbfdfe00640 1 --2- 172.21.15.69:0/3154547428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaad0a30 0x55e7aaad0e10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000bd10 tx=0x7fbfd0011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cac45ec719624e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.011+0000 7fbfdd600640 1 -- 172.21.15.69:0/2205736461 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fbfd802e030 con 0x7fbfe010d500 2024-04-24T09:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.011+0000 7fbfe5200640 1 --2- 172.21.15.69:0/2205736461 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fbfc80a3e30 0x7fbfc80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.011+0000 7fbfce000640 1 -- 172.21.15.69:0/3154547428 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd001a400 con 0x55e7aaad0a30 2024-04-24T09:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.011+0000 7fbfce000640 1 -- 172.21.15.69:0/3154547428 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfd001aa50 con 0x55e7aaad0a30 2024-04-24T09:07:46.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.011+0000 7fbfce000640 1 -- 172.21.15.69:0/3154547428 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd00129d0 con 0x55e7aaad0a30 2024-04-24T09:07:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.012+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/3154547428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaad0a30 msgr2=0x55e7aaad0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.012+0000 7fbfe5200640 1 --2- 172.21.15.69:0/2205736461 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fbfc80a3e30 0x7fbfc80a62f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55e7aaad1ff0 tx=0x7fbfd4005f70 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.012+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/3154547428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaad0a30 0x55e7aaad0e10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000bd10 tx=0x7fbfd0011750 comp rx=0 tx=0).stop 2024-04-24T09:07:46.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.012+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/3154547428 shutdown_connections 2024-04-24T09:07:46.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.012+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/3154547428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaadba20 0x55e7aaadde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.012+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/3154547428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad1350 0x55e7aaadb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.012+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/3154547428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaad0a30 0x55e7aaad0e10 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.012+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/3154547428 >> 172.21.15.69:0/3154547428 conn(0x55e7aaaccd50 msgr2=0x55e7aaacd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:46.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.012+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/3154547428 shutdown_connections 2024-04-24T09:07:46.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.012+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/3154547428 wait complete. 2024-04-24T09:07:46.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.013+0000 7fbfeaa7f740 1 -- start start 2024-04-24T09:07:46.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.013+0000 7fbfeaa7f740 1 Processor -- start 2024-04-24T09:07:46.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.013+0000 7fbfeaa7f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad0a30 0x55e7aaad89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.013+0000 7fbfeaa7f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaad1350 0x55e7aaad8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.013+0000 7fbfeaa7f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaadba20 0x55e7aaad9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.014+0000 7fbfeaa7f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e7aaade810 con 0x55e7aaad1350 2024-04-24T09:07:46.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.014+0000 7fbfeaa7f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e7aa9ef690 con 0x55e7aaad0a30 2024-04-24T09:07:46.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.014+0000 7fbfeaa7f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55e7aaade990 con 0x55e7aaadba20 2024-04-24T09:07:46.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.014+0000 7fbfeaa7f740 10 client.0 _ll_register_callbacks cb 0x55e7aaa2b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:46.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.014+0000 7fbfdfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaad1350 0x55e7aaad8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.014+0000 7fbfdfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaad1350 0x55e7aaad8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35404/0 (socket says 172.21.15.69:35404) 2024-04-24T09:07:46.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.014+0000 7fbfdfe00640 1 -- 172.21.15.69:0/2485497930 learned_addr learned my addr 172.21.15.69:0/2485497930 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:46.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.014+0000 7fbfe5200640 1 --2- 172.21.15.69:0/2485497930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad0a30 0x55e7aaad89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.014+0000 7fbfe5c00640 1 --2- 172.21.15.69:0/2485497930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaadba20 0x55e7aaad9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.014+0000 7fbfdfe00640 1 -- 172.21.15.69:0/2485497930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaadba20 msgr2=0x55e7aaad9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:46.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.014+0000 7fbfdfe00640 1 --2- 172.21.15.69:0/2485497930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaadba20 0x55e7aaad9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.014+0000 7fbfdfe00640 1 -- 172.21.15.69:0/2485497930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad0a30 msgr2=0x55e7aaad89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:46.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.014+0000 7fbfdfe00640 1 --2- 172.21.15.69:0/2485497930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad0a30 0x55e7aaad89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.014+0000 7fbfdfe00640 1 -- 172.21.15.69:0/2485497930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7aa9ee920 con 0x55e7aaad1350 2024-04-24T09:07:46.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.015+0000 7fbfdfe00640 1 --2- 172.21.15.69:0/2485497930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaad1350 0x55e7aaad8ef0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00125b0 tx=0x7fbfd001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c489e1715e47a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:46.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.015+0000 7fbfc2a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:46.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.015+0000 7fbfc2a00640 1 -- 172.21.15.69:0/2485497930 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd001a810 con 0x55e7aaad1350 2024-04-24T09:07:46.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.015+0000 7fbfc2a00640 1 -- 172.21.15.69:0/2485497930 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fbfd0005030 con 0x55e7aaad1350 2024-04-24T09:07:46.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.015+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/2485497930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfd001aa50 con 0x55e7aaad1350 2024-04-24T09:07:46.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.015+0000 7fbfc2a00640 1 -- 172.21.15.69:0/2485497930 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbfd0005230 con 0x55e7aaad1350 2024-04-24T09:07:46.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.015+0000 7fbfeaa7f740 10 client.15864 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:46.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.015+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/2485497930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e7aa8127d0 con 0x55e7aaad1350 2024-04-24T09:07:46.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.016+0000 7fbfc2a00640 1 -- 172.21.15.69:0/2485497930 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfd001aa50 con 0x55e7aaad1350 2024-04-24T09:07:46.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.016+0000 7fbfc2a00640 1 -- 172.21.15.69:0/2485497930 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fbfd0004730 con 0x55e7aaad1350 2024-04-24T09:07:46.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.016+0000 7fbfeaa7f740 10 client.15864 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:46.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.016+0000 7fbfeaa7f740 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:46.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.016+0000 7fbfeaa7f740 20 client.15864 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:46.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.016+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/2485497930 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e7aaaebd10 0x55e7aaaee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.016+0000 7fbfeaa7f740 4 client.15864 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:46.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.016+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/2485497930 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7aaa43c10 con 0x55e7aaaebd10 2024-04-24T09:07:46.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.017+0000 7fbfe5200640 1 --2- 172.21.15.69:0/2485497930 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e7aaaebd10 0x55e7aaaee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.018+0000 7fbfe5200640 1 --2- 172.21.15.69:0/2485497930 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e7aaaebd10 0x55e7aaaee100 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b006f0b66f44f5b server_cookie=5dec7bc9a9b58d55 in_seq=0 out_seq=0 2024-04-24T09:07:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.018+0000 7fbfc2a00640 10 client.15864 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.019+0000 7fbfc2a00640 1 -- 172.21.15.69:0/2485497930 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7aaa43c10 con 0x55e7aaaebd10 2024-04-24T09:07:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.019+0000 7fbfc2a00640 10 client.15864 handle_command_reply: tid=0 2024-04-24T09:07:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.019+0000 7fbfeaa7f740 1 client.15864 shutdown 2024-04-24T09:07:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.020+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/2485497930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaad1350 msgr2=0x55e7aaad8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.020+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/2485497930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaad1350 0x55e7aaad8ef0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00125b0 tx=0x7fbfd001ac00 comp rx=0 tx=0).stop 2024-04-24T09:07:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.020+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/2485497930 shutdown_connections 2024-04-24T09:07:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.020+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/2485497930 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e7aaaebd10 0x55e7aaaee100 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.020+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/2485497930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaadba20 0x55e7aaad9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.020+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/2485497930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaad1350 0x55e7aaad8ef0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.020+0000 7fbfc2a00640 0 client.15864 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.020+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/2485497930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad0a30 0x55e7aaad89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.020+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/2485497930 >> 172.21.15.69:0/2485497930 conn(0x55e7aaaccd50 msgr2=0x55e7aaacbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:46.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.020+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/2485497930 shutdown_connections 2024-04-24T09:07:46.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.020+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/2485497930 wait complete. 2024-04-24T09:07:46.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.020+0000 7fbfeaa7f740 20 client.15864 trim_cache size 0 max 16384 2024-04-24T09:07:46.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.039+0000 7fbfeaa7f740 1 Processor -- start 2024-04-24T09:07:46.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.039+0000 7fbfeaa7f740 1 -- start start 2024-04-24T09:07:46.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.040+0000 7fbfeaa7f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad0a30 0x55e7aaac7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.040+0000 7fbfeaa7f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaad1350 0x55e7aaae7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.040+0000 7fbfeaa7f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaadba20 0x55e7aaaf3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.040+0000 7fbfeaa7f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe01b7e60 con 0x55e7aaadba20 2024-04-24T09:07:46.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.040+0000 7fbfeaa7f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe01b5730 con 0x55e7aaad0a30 2024-04-24T09:07:46.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.040+0000 7fbfeaa7f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe01b3580 con 0x55e7aaad1350 2024-04-24T09:07:46.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.040+0000 7fbfe5c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaadba20 0x55e7aaaf3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.040+0000 7fbfe5c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaadba20 0x55e7aaaf3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35406/0 (socket says 172.21.15.69:35406) 2024-04-24T09:07:46.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.040+0000 7fbfe5c00640 1 -- 172.21.15.69:0/1886904361 learned_addr learned my addr 172.21.15.69:0/1886904361 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:46.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.040+0000 7fbfe5200640 1 --2- 172.21.15.69:0/1886904361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad0a30 0x55e7aaac7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.041+0000 7fbfdfe00640 1 --2- 172.21.15.69:0/1886904361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaad1350 0x55e7aaae7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.041+0000 7fbfe5c00640 1 -- 172.21.15.69:0/1886904361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaad1350 msgr2=0x55e7aaae7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:46.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.041+0000 7fbfe5c00640 1 --2- 172.21.15.69:0/1886904361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaad1350 0x55e7aaae7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.041+0000 7fbfe5c00640 1 -- 172.21.15.69:0/1886904361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad0a30 msgr2=0x55e7aaac7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:46.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.041+0000 7fbfe5c00640 1 --2- 172.21.15.69:0/1886904361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad0a30 0x55e7aaac7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.041+0000 7fbfe5c00640 1 -- 172.21.15.69:0/1886904361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7aaad7980 con 0x55e7aaadba20 2024-04-24T09:07:46.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.041+0000 7fbfdfe00640 1 --2- 172.21.15.69:0/1886904361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaad1350 0x55e7aaae7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:46.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.041+0000 7fbfe5c00640 1 --2- 172.21.15.69:0/1886904361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaadba20 0x55e7aaaf3840 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8006010 tx=0x7fbfd80b9ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83f0415a1518c9b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:46.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.041+0000 7fbfcd600640 1 -- 172.21.15.69:0/1886904361 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8045ea0 con 0x55e7aaadba20 2024-04-24T09:07:46.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.041+0000 7fbfcd600640 1 -- 172.21.15.69:0/1886904361 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfd8036040 con 0x55e7aaadba20 2024-04-24T09:07:46.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.041+0000 7fbfcd600640 1 -- 172.21.15.69:0/1886904361 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80c67b0 con 0x55e7aaadba20 2024-04-24T09:07:46.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.042+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/1886904361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaadba20 msgr2=0x55e7aaaf3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:46.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.042+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/1886904361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaadba20 0x55e7aaaf3840 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8006010 tx=0x7fbfd80b9ed0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.042+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/1886904361 shutdown_connections 2024-04-24T09:07:46.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.042+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/1886904361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaadba20 0x55e7aaaf3840 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.042+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/1886904361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaad1350 0x55e7aaae7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.042+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/1886904361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad0a30 0x55e7aaac7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.042+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/1886904361 >> 172.21.15.69:0/1886904361 conn(0x55e7aaaccd50 msgr2=0x55e7aaad2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:46.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.042+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/1886904361 shutdown_connections 2024-04-24T09:07:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.042+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/1886904361 wait complete. 2024-04-24T09:07:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.043+0000 7fbfeaa7f740 1 -- start start 2024-04-24T09:07:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.043+0000 7fbfeaa7f740 1 Processor -- start 2024-04-24T09:07:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.043+0000 7fbfeaa7f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad0a30 0x55e7aaad5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.043+0000 7fbfeaa7f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaad1350 0x55e7aaad5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.043+0000 7fbfeaa7f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaadba20 0x55e7aaad62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.044+0000 7fbfeaa7f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe01b3bf0 con 0x55e7aaadba20 2024-04-24T09:07:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.044+0000 7fbfeaa7f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe01b5060 con 0x55e7aaad0a30 2024-04-24T09:07:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.044+0000 7fbfeaa7f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe01b42c0 con 0x55e7aaad1350 2024-04-24T09:07:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.044+0000 7fbfeaa7f740 10 client.0 _ll_register_callbacks cb 0x7fbfd000bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.044+0000 7fbfe5c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaadba20 0x55e7aaad62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.044+0000 7fbfe5c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaadba20 0x55e7aaad62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35414/0 (socket says 172.21.15.69:35414) 2024-04-24T09:07:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.044+0000 7fbfe5c00640 1 -- 172.21.15.69:0/56347881 learned_addr learned my addr 172.21.15.69:0/56347881 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.044+0000 7fbfdfe00640 1 --2- 172.21.15.69:0/56347881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaad1350 0x55e7aaad5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.044+0000 7fbfe5200640 1 --2- 172.21.15.69:0/56347881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad0a30 0x55e7aaad5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.044+0000 7fbfe5c00640 1 -- 172.21.15.69:0/56347881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaad1350 msgr2=0x55e7aaad5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:46.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.044+0000 7fbfe5c00640 1 --2- 172.21.15.69:0/56347881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaad1350 0x55e7aaad5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.044+0000 7fbfe5c00640 1 -- 172.21.15.69:0/56347881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad0a30 msgr2=0x55e7aaad5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:46.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.044+0000 7fbfe5c00640 1 --2- 172.21.15.69:0/56347881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad0a30 0x55e7aaad5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.044+0000 7fbfe5c00640 1 -- 172.21.15.69:0/56347881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7aa7483f0 con 0x55e7aaadba20 2024-04-24T09:07:46.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.044+0000 7fbfdfe00640 1 --2- 172.21.15.69:0/56347881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaad1350 0x55e7aaad5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:46.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.044+0000 7fbfe5c00640 1 --2- 172.21.15.69:0/56347881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaadba20 0x55e7aaad62c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80bdad0 tx=0x7fbfd80093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805027e89b99d1de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:46.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.045+0000 7fbfc2a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:46.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.045+0000 7fbfc2a00640 1 -- 172.21.15.69:0/56347881 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80bdb40 con 0x55e7aaadba20 2024-04-24T09:07:46.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.045+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/56347881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfd8036040 con 0x55e7aaadba20 2024-04-24T09:07:46.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.045+0000 7fbfc2a00640 1 -- 172.21.15.69:0/56347881 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fbfd80ba1b0 con 0x55e7aaadba20 2024-04-24T09:07:46.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.045+0000 7fbfc2a00640 1 -- 172.21.15.69:0/56347881 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbfd80098c0 con 0x55e7aaadba20 2024-04-24T09:07:46.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.045+0000 7fbfeaa7f740 10 client.15870 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:46.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.045+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/56347881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e7aa7080f0 con 0x55e7aaadba20 2024-04-24T09:07:46.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.045+0000 7fbfc2a00640 1 -- 172.21.15.69:0/56347881 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfd8036040 con 0x55e7aaadba20 2024-04-24T09:07:46.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.045+0000 7fbfc2a00640 1 -- 172.21.15.69:0/56347881 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fbfd8030070 con 0x55e7aaadba20 2024-04-24T09:07:46.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.045+0000 7fbfeaa7f740 10 client.15870 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:46.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.045+0000 7fbfeaa7f740 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:46.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.045+0000 7fbfeaa7f740 20 client.15870 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:46.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.045+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/56347881 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e7aab52800 0x55e7aab54c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:46.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.045+0000 7fbfeaa7f740 4 client.15870 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:07:46.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.045+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/56347881 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e7aaa3da80 con 0x55e7aab52800 2024-04-24T09:07:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.046+0000 7fbfe5200640 1 --2- 172.21.15.69:0/56347881 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e7aab52800 0x55e7aab54c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.047+0000 7fbfe5200640 1 --2- 172.21.15.69:0/56347881 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e7aab52800 0x55e7aab54c20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ba55e091cee7672 server_cookie=66e23b3381b32fd in_seq=0 out_seq=0 2024-04-24T09:07:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.047+0000 7fbfc2a00640 10 client.15870 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.047+0000 7fbfc2a00640 1 -- 172.21.15.69:0/56347881 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e7aaa3da80 con 0x55e7aab52800 2024-04-24T09:07:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.047+0000 7fbfc2a00640 10 client.15870 handle_command_reply: tid=0 2024-04-24T09:07:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.048+0000 7fbfeaa7f740 1 client.15870 shutdown 2024-04-24T09:07:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.048+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/56347881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaadba20 msgr2=0x55e7aaad62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.048+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/56347881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaadba20 0x55e7aaad62c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80bdad0 tx=0x7fbfd80093f0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.048+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/56347881 shutdown_connections 2024-04-24T09:07:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.048+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/56347881 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e7aab52800 0x55e7aab54c20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.048+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/56347881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7aaadba20 0x55e7aaad62c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.048+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/56347881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e7aaad1350 0x55e7aaad5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.048+0000 7fbfeaa7f740 1 --2- 172.21.15.69:0/56347881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e7aaad0a30 0x55e7aaad5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.048+0000 7fbfc2a00640 0 client.15870 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.048+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/56347881 >> 172.21.15.69:0/56347881 conn(0x55e7aaaccd50 msgr2=0x55e7aaad87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.048+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/56347881 shutdown_connections 2024-04-24T09:07:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.048+0000 7fbfeaa7f740 1 -- 172.21.15.69:0/56347881 wait complete. 2024-04-24T09:07:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.048+0000 7fbfeaa7f740 20 client.15870 trim_cache size 0 max 16384 2024-04-24T09:07:46.050 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:07:46.050 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:07:46.050 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-04-24T09:07:46.050 INFO:teuthology.orchestra.run.smithi069.stdout: "26238a16-ab65-41ad-8c22-678242d4fdbe": { 2024-04-24T09:07:46.050 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-04-24T09:07:46.050 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "26238a16-ab65-41ad-8c22-678242d4fdbe", 2024-04-24T09:07:46.050 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-04-24T09:07:46.050 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:07:46.050 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:07:46.050 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:07:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.049+0000 7fbfe6600640 1 -- 172.21.15.69:0/2205736461 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fbfc80a3e30 msgr2=0x7fbfc80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.049+0000 7fbfe6600640 1 --2- 172.21.15.69:0/2205736461 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fbfc80a3e30 0x7fbfc80a62f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55e7aaad1ff0 tx=0x7fbfd4005f70 comp rx=0 tx=0).stop 2024-04-24T09:07:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.049+0000 7fbfe6600640 1 -- 172.21.15.69:0/2205736461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfe010d500 msgr2=0x7fbfe019ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.049+0000 7fbfe6600640 1 --2- 172.21.15.69:0/2205736461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfe010d500 0x7fbfe019ea70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8005e30 tx=0x7fbfd8003ba0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.049+0000 7fbfe6600640 1 -- 172.21.15.69:0/2205736461 shutdown_connections 2024-04-24T09:07:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.049+0000 7fbfe6600640 1 --2- 172.21.15.69:0/2205736461 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fbfc80a3e30 0x7fbfc80a62f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.049+0000 7fbfe6600640 1 --2- 172.21.15.69:0/2205736461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfe010d500 0x7fbfe019ea70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.049+0000 7fbfe6600640 1 --2- 172.21.15.69:0/2205736461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbfe0108f40 0x7fbfe01a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.049+0000 7fbfe6600640 1 --2- 172.21.15.69:0/2205736461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfe0108580 0x7fbfe01a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:46.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.049+0000 7fbfe6600640 1 -- 172.21.15.69:0/2205736461 >> 172.21.15.69:0/2205736461 conn(0x7fbfe0077e20 msgr2=0x7fbfe0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:46.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.049+0000 7fbfe6600640 1 -- 172.21.15.69:0/2205736461 shutdown_connections 2024-04-24T09:07:46.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:07:46.049+0000 7fbfe6600640 1 -- 172.21.15.69:0/2205736461 wait complete. 2024-04-24T09:07:46.064 INFO:tasks.cephfs.filesystem:scrub status for tag:26238a16-ab65-41ad-8c22-678242d4fdbe - {'path': '/', 'tag': '26238a16-ab65-41ad-8c22-678242d4fdbe', 'options': 'recursive,force'} 2024-04-24T09:07:46.066 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:45 smithi081 ceph-mon[30961]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:07:46.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:46 smithi130 ceph-mon[31553]: scrub queued for path: / 2024-04-24T09:07:46.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:46 smithi130 ceph-mon[31553]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:46.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:46 smithi130 ceph-mon[31553]: scrub summary: active paths [/] 2024-04-24T09:07:46.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:46 smithi130 ceph-mon[31553]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:47.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:46 smithi081 ceph-mon[30961]: scrub queued for path: / 2024-04-24T09:07:47.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:46 smithi081 ceph-mon[30961]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:47.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:46 smithi081 ceph-mon[30961]: scrub summary: active paths [/] 2024-04-24T09:07:47.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:46 smithi081 ceph-mon[30961]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:47.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:46 smithi069 ceph-mon[28170]: scrub queued for path: / 2024-04-24T09:07:47.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:46 smithi069 ceph-mon[28170]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:47.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:46 smithi069 ceph-mon[28170]: scrub summary: active paths [/] 2024-04-24T09:07:47.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:46 smithi069 ceph-mon[28170]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:47.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:47 smithi130 ceph-mon[31553]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T09:07:47.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:47 smithi130 ceph-mon[31553]: scrub summary: idle 2024-04-24T09:07:48.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:47 smithi081 ceph-mon[30961]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T09:07:48.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:47 smithi081 ceph-mon[30961]: scrub summary: idle 2024-04-24T09:07:48.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:47 smithi069 ceph-mon[28170]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T09:07:48.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:47 smithi069 ceph-mon[28170]: scrub summary: idle 2024-04-24T09:07:49.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:49 smithi130 ceph-mon[31553]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:07:50.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:49 smithi081 ceph-mon[30961]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:07:50.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:49 smithi069 ceph-mon[28170]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:07:51.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:51 smithi130 ceph-mon[31553]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T09:07:52.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:51 smithi081 ceph-mon[30961]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T09:07:52.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:51 smithi069 ceph-mon[28170]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T09:07:53.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:53 smithi130 ceph-mon[31553]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:07:54.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:53 smithi081 ceph-mon[30961]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:07:54.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:53 smithi069 ceph-mon[28170]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:07:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.354+0000 7f964a800640 1 -- 172.21.15.69:0/3176964952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9644109530 msgr2=0x7f9644111100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.354+0000 7f964a800640 1 --2- 172.21.15.69:0/3176964952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9644109530 0x7f9644111100 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f963c009900 tx=0x7f963c02a410 comp rx=0 tx=0).stop 2024-04-24T09:07:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.354+0000 7f964a800640 1 -- 172.21.15.69:0/3176964952 shutdown_connections 2024-04-24T09:07:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.354+0000 7f964a800640 1 --2- 172.21.15.69:0/3176964952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9644109530 0x7f9644111100 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.354+0000 7f964a800640 1 --2- 172.21.15.69:0/3176964952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9644104e10 0x7f9644108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.354+0000 7f964a800640 1 --2- 172.21.15.69:0/3176964952 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9644104460 0x7f9644104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.354+0000 7f964a800640 1 -- 172.21.15.69:0/3176964952 >> 172.21.15.69:0/3176964952 conn(0x7f96440ffd00 msgr2=0x7f9644102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.354+0000 7f964a800640 1 -- 172.21.15.69:0/3176964952 shutdown_connections 2024-04-24T09:07:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.354+0000 7f964a800640 1 -- 172.21.15.69:0/3176964952 wait complete. 2024-04-24T09:07:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.355+0000 7f964a800640 1 Processor -- start 2024-04-24T09:07:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.355+0000 7f964a800640 1 -- start start 2024-04-24T09:07:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.356+0000 7f964a800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9644104460 0x7f96441a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.356+0000 7f964a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9644109530 0x7f96441a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.356+0000 7f964a800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f964419a740 0x7f964419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.356+0000 7f964a800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9644114480 con 0x7f9644109530 2024-04-24T09:07:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.356+0000 7f964a800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9644114300 con 0x7f9644104460 2024-04-24T09:07:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.356+0000 7f964a800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9644114600 con 0x7f964419a740 2024-04-24T09:07:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.356+0000 7f9642a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9644109530 0x7f96441a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.356+0000 7f9642a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9644109530 0x7f96441a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43916/0 (socket says 172.21.15.69:43916) 2024-04-24T09:07:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.356+0000 7f9642a00640 1 -- 172.21.15.69:0/2342957075 learned_addr learned my addr 172.21.15.69:0/2342957075 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.356+0000 7f9643400640 1 --2- 172.21.15.69:0/2342957075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9644104460 0x7f96441a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.356+0000 7f9643e00640 1 --2- 172.21.15.69:0/2342957075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f964419a740 0x7f964419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.357+0000 7f9642a00640 1 -- 172.21.15.69:0/2342957075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f964419a740 msgr2=0x7f964419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.357+0000 7f9642a00640 1 --2- 172.21.15.69:0/2342957075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f964419a740 0x7f964419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.357+0000 7f9642a00640 1 -- 172.21.15.69:0/2342957075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9644104460 msgr2=0x7f96441a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.357+0000 7f9642a00640 1 --2- 172.21.15.69:0/2342957075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9644104460 0x7f96441a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.357+0000 7f9642a00640 1 -- 172.21.15.69:0/2342957075 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964419b340 con 0x7f9644109530 2024-04-24T09:07:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.357+0000 7f9643e00640 1 --2- 172.21.15.69:0/2342957075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f964419a740 0x7f964419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.357+0000 7f9643400640 1 --2- 172.21.15.69:0/2342957075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9644104460 0x7f96441a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.357+0000 7f9642a00640 1 --2- 172.21.15.69:0/2342957075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9644109530 0x7f96441a0bc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f962c00d770 tx=0x7f962c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.357+0000 7f963be00640 1 -- 172.21.15.69:0/2342957075 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c017070 con 0x7f9644109530 2024-04-24T09:07:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.357+0000 7f963be00640 1 -- 172.21.15.69:0/2342957075 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f962c009c30 con 0x7f9644109530 2024-04-24T09:07:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.357+0000 7f963be00640 1 -- 172.21.15.69:0/2342957075 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c00f620 con 0x7f9644109530 2024-04-24T09:07:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.357+0000 7f964a800640 1 -- 172.21.15.69:0/2342957075 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96441a7420 con 0x7f9644109530 2024-04-24T09:07:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.357+0000 7f964a800640 1 -- 172.21.15.69:0/2342957075 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96441a7880 con 0x7f9644109530 2024-04-24T09:07:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.360+0000 7f964d765740 1 Processor -- start 2024-04-24T09:07:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.361+0000 7f964d765740 1 -- start start 2024-04-24T09:07:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.361+0000 7f963be00640 1 -- 172.21.15.69:0/2342957075 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f962c01a070 con 0x7f9644109530 2024-04-24T09:07:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.361+0000 7f964d765740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f36590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.361+0000 7f964d765740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f36ad0 0x560ed6f40e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.361+0000 7f964d765740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f41390 0x560ed6f43780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.361+0000 7f964d765740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560ed6e9c900 con 0x560ed6f361b0 2024-04-24T09:07:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.361+0000 7f964d765740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x560ed6e42f20 con 0x560ed6f36ad0 2024-04-24T09:07:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.361+0000 7f964d765740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x560ed6e988c0 con 0x560ed6f41390 2024-04-24T09:07:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.361+0000 7f9643e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f36590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.361+0000 7f9643e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f36590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43928/0 (socket says 172.21.15.69:43928) 2024-04-24T09:07:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.361+0000 7f9643e00640 1 -- 172.21.15.69:0/1885208808 learned_addr learned my addr 172.21.15.69:0/1885208808 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.362+0000 7f9642a00640 1 --2- 172.21.15.69:0/1885208808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f41390 0x560ed6f43780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.362+0000 7f9643e00640 1 -- 172.21.15.69:0/1885208808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f41390 msgr2=0x560ed6f43780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.362+0000 7f9643400640 1 --2- 172.21.15.69:0/1885208808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f36ad0 0x560ed6f40e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.362+0000 7f9643e00640 1 --2- 172.21.15.69:0/1885208808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f41390 0x560ed6f43780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.362+0000 7f9643e00640 1 -- 172.21.15.69:0/1885208808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f36ad0 msgr2=0x560ed6f40e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.362+0000 7f9643e00640 1 --2- 172.21.15.69:0/1885208808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f36ad0 0x560ed6f40e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.362+0000 7f9643e00640 1 -- 172.21.15.69:0/1885208808 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ed6e67e10 con 0x560ed6f361b0 2024-04-24T09:07:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.362+0000 7f9642a00640 1 --2- 172.21.15.69:0/1885208808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f41390 0x560ed6f43780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.362+0000 7f9643400640 1 --2- 172.21.15.69:0/1885208808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f36ad0 0x560ed6f40e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.362+0000 7f9643e00640 1 --2- 172.21.15.69:0/1885208808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f36590 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f963c031440 tx=0x7f963c032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d472ebbc3aa22 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.362+0000 7f963be00640 1 --2- 172.21.15.69:0/2342957075 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f96280a3ee0 0x7f96280a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.363+0000 7f9638c00640 1 -- 172.21.15.69:0/1885208808 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c03f070 con 0x560ed6f361b0 2024-04-24T09:07:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.363+0000 7f963be00640 1 -- 172.21.15.69:0/2342957075 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f962c0d07f0 con 0x7f9644109530 2024-04-24T09:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.363+0000 7f9638c00640 1 -- 172.21.15.69:0/1885208808 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f963c0316a0 con 0x560ed6f361b0 2024-04-24T09:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.363+0000 7f9638c00640 1 -- 172.21.15.69:0/1885208808 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c035070 con 0x560ed6f361b0 2024-04-24T09:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.363+0000 7f9643400640 1 --2- 172.21.15.69:0/2342957075 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f96280a3ee0 0x7f96280a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.364+0000 7f964d765740 1 -- 172.21.15.69:0/1885208808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 msgr2=0x560ed6f36590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.364+0000 7f9643400640 1 --2- 172.21.15.69:0/2342957075 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f96280a3ee0 0x7f96280a63a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x560ed6f37750 tx=0x7f963400a500 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.364+0000 7f964d765740 1 --2- 172.21.15.69:0/1885208808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f36590 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f963c031440 tx=0x7f963c032c60 comp rx=0 tx=0).stop 2024-04-24T09:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.364+0000 7f964d765740 1 -- 172.21.15.69:0/1885208808 shutdown_connections 2024-04-24T09:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.364+0000 7f964d765740 1 --2- 172.21.15.69:0/1885208808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f41390 0x560ed6f43780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.364+0000 7f964d765740 1 --2- 172.21.15.69:0/1885208808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f36ad0 0x560ed6f40e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.364+0000 7f964d765740 1 --2- 172.21.15.69:0/1885208808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f36590 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.364+0000 7f964d765740 1 -- 172.21.15.69:0/1885208808 >> 172.21.15.69:0/1885208808 conn(0x560ed6f326c0 msgr2=0x560ed6f32aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.364+0000 7f964d765740 1 -- 172.21.15.69:0/1885208808 shutdown_connections 2024-04-24T09:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.364+0000 7f964d765740 1 -- 172.21.15.69:0/1885208808 wait complete. 2024-04-24T09:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.364+0000 7f964d765740 1 -- start start 2024-04-24T09:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.365+0000 7f964d765740 1 Processor -- start 2024-04-24T09:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.365+0000 7f964d765740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f361b0 0x560ed6f3e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.365+0000 7f964d765740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f36ad0 0x560ed6f3e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.365+0000 7f964d765740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f41390 0x560ed6f3ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.365+0000 7f964d765740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560ed6f43f60 con 0x560ed6f36ad0 2024-04-24T09:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.365+0000 7f964d765740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x560ed6e562c0 con 0x560ed6f361b0 2024-04-24T09:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.365+0000 7f964d765740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x560ed6f440e0 con 0x560ed6f41390 2024-04-24T09:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.365+0000 7f964d765740 10 client.0 _ll_register_callbacks cb 0x560ed6baea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.366+0000 7f9642a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f36ad0 0x560ed6f3e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.366+0000 7f9642a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f36ad0 0x560ed6f3e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43934/0 (socket says 172.21.15.69:43934) 2024-04-24T09:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.366+0000 7f9642a00640 1 -- 172.21.15.69:0/2663143020 learned_addr learned my addr 172.21.15.69:0/2663143020 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.366+0000 7f9643400640 1 --2- 172.21.15.69:0/2663143020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f361b0 0x560ed6f3e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.366+0000 7f9643e00640 1 --2- 172.21.15.69:0/2663143020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f41390 0x560ed6f3ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.366+0000 7f9642a00640 1 -- 172.21.15.69:0/2663143020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f41390 msgr2=0x560ed6f3ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.366+0000 7f9642a00640 1 --2- 172.21.15.69:0/2663143020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f41390 0x560ed6f3ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.366+0000 7f9642a00640 1 -- 172.21.15.69:0/2663143020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f361b0 msgr2=0x560ed6f3e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.366+0000 7f9642a00640 1 --2- 172.21.15.69:0/2663143020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f361b0 0x560ed6f3e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.366+0000 7f9642a00640 1 -- 172.21.15.69:0/2663143020 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ed6e66fd0 con 0x560ed6f36ad0 2024-04-24T09:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.367+0000 7f9642a00640 1 --2- 172.21.15.69:0/2663143020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f36ad0 0x560ed6f3e780 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f962c009610 tx=0x7f962c09c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1272a07dfd43f80 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.367+0000 7f9631600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.367+0000 7f9631600640 1 -- 172.21.15.69:0/2663143020 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c021070 con 0x560ed6f36ad0 2024-04-24T09:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.367+0000 7f9631600640 1 -- 172.21.15.69:0/2663143020 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f962c0d03f0 con 0x560ed6f36ad0 2024-04-24T09:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.367+0000 7f964d765740 1 -- 172.21.15.69:0/2663143020 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f963c0316a0 con 0x560ed6f36ad0 2024-04-24T09:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.367+0000 7f9631600640 1 -- 172.21.15.69:0/2663143020 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f962c0d05f0 con 0x560ed6f36ad0 2024-04-24T09:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.367+0000 7f964d765740 10 client.15882 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.367+0000 7f964d765740 1 -- 172.21.15.69:0/2663143020 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560ed6baf210 con 0x560ed6f36ad0 2024-04-24T09:07:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.368+0000 7f9631600640 1 -- 172.21.15.69:0/2663143020 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f963c0316a0 con 0x560ed6f36ad0 2024-04-24T09:07:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.368+0000 7f9631600640 1 -- 172.21.15.69:0/2663143020 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f962c0cf070 con 0x560ed6f36ad0 2024-04-24T09:07:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.368+0000 7f964d765740 10 client.15882 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.368+0000 7f964d765740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.368+0000 7f964d765740 20 client.15882 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.368+0000 7f964d765740 1 --2- 172.21.15.69:0/2663143020 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x560ed6f513a0 0x560ed6f53790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.368+0000 7f964d765740 4 client.15882 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.368+0000 7f964d765740 1 -- 172.21.15.69:0/2663143020 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ed6eaa320 con 0x560ed6f513a0 2024-04-24T09:07:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.369+0000 7f9643400640 1 --2- 172.21.15.69:0/2663143020 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x560ed6f513a0 0x560ed6f53790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.369+0000 7f9643400640 1 --2- 172.21.15.69:0/2663143020 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x560ed6f513a0 0x560ed6f53790 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de309e721b25d900 server_cookie=4d8532119120c3ac in_seq=0 out_seq=0 2024-04-24T09:07:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.369+0000 7f9631600640 10 client.15882 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.371+0000 7f9631600640 1 -- 172.21.15.69:0/2663143020 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ed6eaa320 con 0x560ed6f513a0 2024-04-24T09:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.371+0000 7f9631600640 10 client.15882 handle_command_reply: tid=0 2024-04-24T09:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.371+0000 7f964d765740 1 client.15882 shutdown 2024-04-24T09:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.371+0000 7f964d765740 1 -- 172.21.15.69:0/2663143020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f36ad0 msgr2=0x560ed6f3e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.371+0000 7f964d765740 1 --2- 172.21.15.69:0/2663143020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f36ad0 0x560ed6f3e780 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f962c009610 tx=0x7f962c09c140 comp rx=0 tx=0).stop 2024-04-24T09:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.371+0000 7f964d765740 1 -- 172.21.15.69:0/2663143020 shutdown_connections 2024-04-24T09:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.371+0000 7f964d765740 1 --2- 172.21.15.69:0/2663143020 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x560ed6f513a0 0x560ed6f53790 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.371+0000 7f964d765740 1 --2- 172.21.15.69:0/2663143020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f41390 0x560ed6f3ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.371+0000 7f964d765740 1 --2- 172.21.15.69:0/2663143020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f36ad0 0x560ed6f3e780 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.371+0000 7f964d765740 1 --2- 172.21.15.69:0/2663143020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f361b0 0x560ed6f3e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.371+0000 7f9631600640 0 client.15882 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.372+0000 7f964d765740 1 -- 172.21.15.69:0/2663143020 >> 172.21.15.69:0/2663143020 conn(0x560ed6f326c0 msgr2=0x560ed6f38440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.372+0000 7f964d765740 1 -- 172.21.15.69:0/2663143020 shutdown_connections 2024-04-24T09:07:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.372+0000 7f964d765740 1 -- 172.21.15.69:0/2663143020 wait complete. 2024-04-24T09:07:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.372+0000 7f964d765740 20 client.15882 trim_cache size 0 max 16384 2024-04-24T09:07:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.388+0000 7f964d765740 1 Processor -- start 2024-04-24T09:07:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.388+0000 7f964d765740 1 -- start start 2024-04-24T09:07:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.388+0000 7f964d765740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f3db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.388+0000 7f964d765740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f36ad0 0x560ed6f59030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.388+0000 7f964d765740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f41390 0x560ed6f59570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.388+0000 7f964d765740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f96441b3bb0 con 0x560ed6f361b0 2024-04-24T09:07:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.388+0000 7f964d765740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f96441b1480 con 0x560ed6f36ad0 2024-04-24T09:07:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.388+0000 7f964d765740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f96441af2d0 con 0x560ed6f41390 2024-04-24T09:07:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.389+0000 7f9643400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f3db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.389+0000 7f9643400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f3db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43946/0 (socket says 172.21.15.69:43946) 2024-04-24T09:07:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.389+0000 7f9643400640 1 -- 172.21.15.69:0/2887235899 learned_addr learned my addr 172.21.15.69:0/2887235899 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.389+0000 7f9643400640 1 -- 172.21.15.69:0/2887235899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f41390 msgr2=0x560ed6f59570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.389+0000 7f9643400640 1 --2- 172.21.15.69:0/2887235899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f41390 0x560ed6f59570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.389+0000 7f9642a00640 1 --2- 172.21.15.69:0/2887235899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f36ad0 0x560ed6f59030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.389+0000 7f9643400640 1 -- 172.21.15.69:0/2887235899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f36ad0 msgr2=0x560ed6f59030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.389+0000 7f9643400640 1 --2- 172.21.15.69:0/2887235899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f36ad0 0x560ed6f59030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.389+0000 7f9643400640 1 -- 172.21.15.69:0/2887235899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ed6f53b00 con 0x560ed6f361b0 2024-04-24T09:07:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.389+0000 7f9642a00640 1 --2- 172.21.15.69:0/2887235899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f36ad0 0x560ed6f59030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.389+0000 7f9643400640 1 --2- 172.21.15.69:0/2887235899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f3db40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f963400afd0 tx=0x7f9634013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af935932a73f6162 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.390+0000 7f9633e00640 1 -- 172.21.15.69:0/2887235899 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96340200d0 con 0x560ed6f361b0 2024-04-24T09:07:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.390+0000 7f9633e00640 1 -- 172.21.15.69:0/2887235899 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9634020270 con 0x560ed6f361b0 2024-04-24T09:07:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.390+0000 7f9633e00640 1 -- 172.21.15.69:0/2887235899 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9634020bb0 con 0x560ed6f361b0 2024-04-24T09:07:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.391+0000 7f964d765740 1 -- 172.21.15.69:0/2887235899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 msgr2=0x560ed6f3db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.391+0000 7f964d765740 1 --2- 172.21.15.69:0/2887235899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f3db40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f963400afd0 tx=0x7f9634013000 comp rx=0 tx=0).stop 2024-04-24T09:07:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.391+0000 7f964d765740 1 -- 172.21.15.69:0/2887235899 shutdown_connections 2024-04-24T09:07:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.391+0000 7f964d765740 1 --2- 172.21.15.69:0/2887235899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f41390 0x560ed6f59570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.391+0000 7f964d765740 1 --2- 172.21.15.69:0/2887235899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f36ad0 0x560ed6f59030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.391+0000 7f964d765740 1 --2- 172.21.15.69:0/2887235899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f3db40 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.391+0000 7f964d765740 1 -- 172.21.15.69:0/2887235899 >> 172.21.15.69:0/2887235899 conn(0x560ed6f326c0 msgr2=0x560ed6f40300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.391+0000 7f964d765740 1 -- 172.21.15.69:0/2887235899 shutdown_connections 2024-04-24T09:07:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.391+0000 7f964d765740 1 -- 172.21.15.69:0/2887235899 wait complete. 2024-04-24T09:07:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.391+0000 7f964d765740 1 -- start start 2024-04-24T09:07:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.392+0000 7f964d765740 1 Processor -- start 2024-04-24T09:07:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.392+0000 7f964d765740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f49600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.392+0000 7f964d765740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f36ad0 0x560ed6f3ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.392+0000 7f964d765740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f41390 0x560ed6f3b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.392+0000 7f964d765740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f96441af940 con 0x560ed6f361b0 2024-04-24T09:07:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.392+0000 7f964d765740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f96441b0db0 con 0x560ed6f41390 2024-04-24T09:07:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.392+0000 7f964d765740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f96441b0010 con 0x560ed6f36ad0 2024-04-24T09:07:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.392+0000 7f964d765740 10 client.0 _ll_register_callbacks cb 0x7f962c0cd970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.393+0000 7f9643400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f49600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.393+0000 7f9643400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f49600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43958/0 (socket says 172.21.15.69:43958) 2024-04-24T09:07:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.393+0000 7f9643400640 1 -- 172.21.15.69:0/966679988 learned_addr learned my addr 172.21.15.69:0/966679988 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:07:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.393+0000 7f9642a00640 1 --2- 172.21.15.69:0/966679988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f36ad0 0x560ed6f3ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.393+0000 7f9643e00640 1 --2- 172.21.15.69:0/966679988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f41390 0x560ed6f3b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.393+0000 7f9643400640 1 -- 172.21.15.69:0/966679988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f36ad0 msgr2=0x560ed6f3ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.393+0000 7f9643400640 1 --2- 172.21.15.69:0/966679988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f36ad0 0x560ed6f3ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.393+0000 7f9643400640 1 -- 172.21.15.69:0/966679988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f41390 msgr2=0x560ed6f3b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.393+0000 7f9643400640 1 --2- 172.21.15.69:0/966679988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f41390 0x560ed6f3b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.393+0000 7f9643400640 1 -- 172.21.15.69:0/966679988 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ed6c7b620 con 0x560ed6f361b0 2024-04-24T09:07:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.393+0000 7f9643400640 1 --2- 172.21.15.69:0/966679988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f49600 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9634020210 tx=0x7f9634020890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736dc8960b627c01 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.393+0000 7f9631600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:07:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.394+0000 7f9631600640 1 -- 172.21.15.69:0/966679988 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9634020c90 con 0x560ed6f361b0 2024-04-24T09:07:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.394+0000 7f9631600640 1 -- 172.21.15.69:0/966679988 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9634026040 con 0x560ed6f361b0 2024-04-24T09:07:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.394+0000 7f964d765740 1 -- 172.21.15.69:0/966679988 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9634020270 con 0x560ed6f361b0 2024-04-24T09:07:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.394+0000 7f9631600640 1 -- 172.21.15.69:0/966679988 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9634004b60 con 0x560ed6f361b0 2024-04-24T09:07:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.394+0000 7f964d765740 10 client.15888 fetch_fsmap learned FSMap version 24 2024-04-24T09:07:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.394+0000 7f964d765740 1 -- 172.21.15.69:0/966679988 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560ed6fb7530 con 0x560ed6f361b0 2024-04-24T09:07:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.394+0000 7f9631600640 1 -- 172.21.15.69:0/966679988 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9634020270 con 0x560ed6f361b0 2024-04-24T09:07:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.394+0000 7f9631600640 1 -- 172.21.15.69:0/966679988 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f96340051a0 con 0x560ed6f361b0 2024-04-24T09:07:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.395+0000 7f964d765740 10 client.15888 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:07:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.395+0000 7f964d765740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.395+0000 7f964d765740 20 client.15888 populate_metadata read hostname 'smithi069' 2024-04-24T09:07:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.395+0000 7f964d765740 1 --2- 172.21.15.69:0/966679988 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x560ed6fb7be0 0x560ed6fba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.395+0000 7f964d765740 4 client.15888 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:07:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.395+0000 7f964d765740 1 -- 172.21.15.69:0/966679988 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ed6ea4190 con 0x560ed6fb7be0 2024-04-24T09:07:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.395+0000 7f9642a00640 1 --2- 172.21.15.69:0/966679988 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x560ed6fb7be0 0x560ed6fba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.396+0000 7f9642a00640 1 --2- 172.21.15.69:0/966679988 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x560ed6fb7be0 0x560ed6fba000 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=570707d7c10d2c4e server_cookie=a102f3230d4866d in_seq=0 out_seq=0 2024-04-24T09:07:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.396+0000 7f9631600640 10 client.15888 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.406+0000 7f9631600640 1 -- 172.21.15.69:0/966679988 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560ed6ea4190 con 0x560ed6fb7be0 2024-04-24T09:07:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.406+0000 7f9631600640 10 client.15888 handle_command_reply: tid=0 2024-04-24T09:07:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.406+0000 7f964d765740 1 client.15888 shutdown 2024-04-24T09:07:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.407+0000 7f964d765740 1 -- 172.21.15.69:0/966679988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 msgr2=0x560ed6f49600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.407+0000 7f964d765740 1 --2- 172.21.15.69:0/966679988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f49600 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9634020210 tx=0x7f9634020890 comp rx=0 tx=0).stop 2024-04-24T09:07:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.407+0000 7f964d765740 1 -- 172.21.15.69:0/966679988 shutdown_connections 2024-04-24T09:07:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.407+0000 7f964d765740 1 --2- 172.21.15.69:0/966679988 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x560ed6fb7be0 0x560ed6fba000 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.407+0000 7f964d765740 1 --2- 172.21.15.69:0/966679988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed6f41390 0x560ed6f3b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.407+0000 7f964d765740 1 --2- 172.21.15.69:0/966679988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560ed6f36ad0 0x560ed6f3ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.407+0000 7f964d765740 1 --2- 172.21.15.69:0/966679988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ed6f361b0 0x560ed6f49600 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.407+0000 7f964d765740 1 -- 172.21.15.69:0/966679988 >> 172.21.15.69:0/966679988 conn(0x560ed6f326c0 msgr2=0x560ed6f3e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.407+0000 7f9631600640 0 client.15888 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.407+0000 7f964d765740 1 -- 172.21.15.69:0/966679988 shutdown_connections 2024-04-24T09:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.407+0000 7f964d765740 1 -- 172.21.15.69:0/966679988 wait complete. 2024-04-24T09:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.407+0000 7f964d765740 20 client.15888 trim_cache size 0 max 16384 2024-04-24T09:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:07:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.408+0000 7f964a800640 1 -- 172.21.15.69:0/2342957075 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f96280a3ee0 msgr2=0x7f96280a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.408+0000 7f964a800640 1 --2- 172.21.15.69:0/2342957075 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f96280a3ee0 0x7f96280a63a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x560ed6f37750 tx=0x7f963400a500 comp rx=0 tx=0).stop 2024-04-24T09:07:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.408+0000 7f964a800640 1 -- 172.21.15.69:0/2342957075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9644109530 msgr2=0x7f96441a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.408+0000 7f964a800640 1 --2- 172.21.15.69:0/2342957075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9644109530 0x7f96441a0bc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f962c00d770 tx=0x7f962c00dc30 comp rx=0 tx=0).stop 2024-04-24T09:07:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.408+0000 7f964a800640 1 -- 172.21.15.69:0/2342957075 shutdown_connections 2024-04-24T09:07:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.408+0000 7f964a800640 1 --2- 172.21.15.69:0/2342957075 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f96280a3ee0 0x7f96280a63a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.408+0000 7f964a800640 1 --2- 172.21.15.69:0/2342957075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f964419a740 0x7f964419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.408+0000 7f964a800640 1 --2- 172.21.15.69:0/2342957075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9644109530 0x7f96441a0bc0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.408+0000 7f964a800640 1 --2- 172.21.15.69:0/2342957075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9644104460 0x7f96441a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.408+0000 7f964a800640 1 -- 172.21.15.69:0/2342957075 >> 172.21.15.69:0/2342957075 conn(0x7f96440ffd00 msgr2=0x7f9644101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.408+0000 7f964a800640 1 -- 172.21.15.69:0/2342957075 shutdown_connections 2024-04-24T09:07:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:07:54.408+0000 7f964a800640 1 -- 172.21.15.69:0/2342957075 wait complete. 2024-04-24T09:07:55.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:55 smithi130 ceph-mon[31553]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:07:56.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:55 smithi081 ceph-mon[30961]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:07:56.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:55 smithi069 ceph-mon[28170]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:07:57.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:57 smithi130 ceph-mon[31553]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-24T09:07:58.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:57 smithi081 ceph-mon[30961]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-24T09:07:58.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:57 smithi069 ceph-mon[28170]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-24T09:07:59.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:07:59 smithi130 ceph-mon[31553]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:08:00.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:07:59 smithi081 ceph-mon[30961]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:08:00.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:07:59 smithi069 ceph-mon[28170]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:08:01.091 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:08:00 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:08:00.609+0000 7fe38ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:08:01.091 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:08:00 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:08:00.609+0000 7fe38ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:08:01.092 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:08:00 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:08:00.609+0000 7fe38ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:08:01.092 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:08:00 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:08:00.609+0000 7fe38ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:08:01.092 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:08:00 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:08:00.609+0000 7fe38ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:08:01.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:01 smithi130 ceph-mon[31553]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T09:08:02.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:01 smithi081 ceph-mon[30961]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T09:08:02.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:01 smithi069 ceph-mon[28170]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T09:08:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:03 smithi130 ceph-mon[31553]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:08:03.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:03 smithi130 ceph-mon[31553]: mgrmap e25: x(active, since 58s), standbys: z, y 2024-04-24T09:08:04.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:03 smithi081 ceph-mon[30961]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:08:04.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:03 smithi081 ceph-mon[30961]: mgrmap e25: x(active, since 58s), standbys: z, y 2024-04-24T09:08:04.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:03 smithi069 ceph-mon[28170]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:08:04.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:03 smithi069 ceph-mon[28170]: mgrmap e25: x(active, since 58s), standbys: z, y 2024-04-24T09:08:05.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:05 smithi130 ceph-mon[31553]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:08:06.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:05 smithi081 ceph-mon[30961]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:08:06.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:05 smithi069 ceph-mon[28170]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:08:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.541+0000 7f4051800640 1 -- 172.21.15.69:0/1936616977 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f404c102080 msgr2=0x7f404c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.541+0000 7f4051800640 1 --2- 172.21.15.69:0/1936616977 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f404c102080 0x7f404c102460 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f403c009900 tx=0x7f403c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.541+0000 7f4051800640 1 -- 172.21.15.69:0/1936616977 shutdown_connections 2024-04-24T09:08:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.541+0000 7f4051800640 1 --2- 172.21.15.69:0/1936616977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c107150 0x7f404c10ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.541+0000 7f4051800640 1 --2- 172.21.15.69:0/1936616977 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f404c102a30 0x7f404c106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.541+0000 7f4051800640 1 --2- 172.21.15.69:0/1936616977 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f404c102080 0x7f404c102460 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.541+0000 7f4051800640 1 -- 172.21.15.69:0/1936616977 >> 172.21.15.69:0/1936616977 conn(0x7f404c0fd980 msgr2=0x7f404c0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.541+0000 7f4051800640 1 -- 172.21.15.69:0/1936616977 shutdown_connections 2024-04-24T09:08:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.541+0000 7f4051800640 1 -- 172.21.15.69:0/1936616977 wait complete. 2024-04-24T09:08:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.542+0000 7f4051800640 1 Processor -- start 2024-04-24T09:08:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.542+0000 7f4051800640 1 -- start start 2024-04-24T09:08:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.543+0000 7f4051800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c102080 0x7f404c1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.543+0000 7f4051800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f404c107150 0x7f404c1a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.543+0000 7f4051800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f404c19a6a0 0x7f404c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.543+0000 7f4051800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f404c112100 con 0x7f404c102080 2024-04-24T09:08:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.543+0000 7f4051800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f404c111f80 con 0x7f404c19a6a0 2024-04-24T09:08:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.543+0000 7f4051800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f404c112280 con 0x7f404c107150 2024-04-24T09:08:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.543+0000 7f404be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c102080 0x7f404c1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.543+0000 7f404be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c102080 0x7f404c1a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48274/0 (socket says 172.21.15.69:48274) 2024-04-24T09:08:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.543+0000 7f404be00640 1 -- 172.21.15.69:0/731914659 learned_addr learned my addr 172.21.15.69:0/731914659 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.543+0000 7f4050e00640 1 --2- 172.21.15.69:0/731914659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f404c19a6a0 0x7f404c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.543+0000 7f404b400640 1 --2- 172.21.15.69:0/731914659 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f404c107150 0x7f404c1a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.543+0000 7f404be00640 1 -- 172.21.15.69:0/731914659 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f404c107150 msgr2=0x7f404c1a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.543+0000 7f404be00640 1 --2- 172.21.15.69:0/731914659 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f404c107150 0x7f404c1a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.543+0000 7f404be00640 1 -- 172.21.15.69:0/731914659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f404c19a6a0 msgr2=0x7f404c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.543+0000 7f404be00640 1 --2- 172.21.15.69:0/731914659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f404c19a6a0 0x7f404c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.544+0000 7f404be00640 1 -- 172.21.15.69:0/731914659 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c19b2a0 con 0x7f404c102080 2024-04-24T09:08:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.544+0000 7f404b400640 1 --2- 172.21.15.69:0/731914659 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f404c107150 0x7f404c1a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.544+0000 7f404be00640 1 --2- 172.21.15.69:0/731914659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c102080 0x7f404c1a05e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f403c005ab0 tx=0x7f403c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.544+0000 7f4048c00640 1 -- 172.21.15.69:0/731914659 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c03f070 con 0x7f404c102080 2024-04-24T09:08:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.544+0000 7f4051800640 1 -- 172.21.15.69:0/731914659 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404c19b530 con 0x7f404c102080 2024-04-24T09:08:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.544+0000 7f4048c00640 1 -- 172.21.15.69:0/731914659 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f403c031b60 con 0x7f404c102080 2024-04-24T09:08:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.544+0000 7f4048c00640 1 -- 172.21.15.69:0/731914659 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c035030 con 0x7f404c102080 2024-04-24T09:08:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.544+0000 7f4051800640 1 -- 172.21.15.69:0/731914659 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f404c1a76c0 con 0x7f404c102080 2024-04-24T09:08:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.549+0000 7f4055df2740 1 Processor -- start 2024-04-24T09:08:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.549+0000 7f4055df2740 1 -- start start 2024-04-24T09:08:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.549+0000 7f4048c00640 1 -- 172.21.15.69:0/731914659 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f403c042070 con 0x7f404c102080 2024-04-24T09:08:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.549+0000 7f4055df2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b153c0 0x55af70b157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.549+0000 7f4055df2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b15ce0 0x55af70b20060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.549+0000 7f4055df2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b205a0 0x55af70b22990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.549+0000 7f4055df2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55af70a7b900 con 0x55af70b15ce0 2024-04-24T09:08:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.549+0000 7f4055df2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55af70a21f20 con 0x55af70b205a0 2024-04-24T09:08:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.549+0000 7f4055df2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55af70a778c0 con 0x55af70b153c0 2024-04-24T09:08:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.549+0000 7f404be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b15ce0 0x55af70b20060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.550+0000 7f404be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b15ce0 0x55af70b20060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48278/0 (socket says 172.21.15.69:48278) 2024-04-24T09:08:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.550+0000 7f404be00640 1 -- 172.21.15.69:0/1269881122 learned_addr learned my addr 172.21.15.69:0/1269881122 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.550+0000 7f4050e00640 1 --2- 172.21.15.69:0/1269881122 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b153c0 0x55af70b157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.550+0000 7f404b400640 1 --2- 172.21.15.69:0/1269881122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b205a0 0x55af70b22990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.550+0000 7f404be00640 1 -- 172.21.15.69:0/1269881122 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b153c0 msgr2=0x55af70b157a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.550+0000 7f404be00640 1 --2- 172.21.15.69:0/1269881122 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b153c0 0x55af70b157a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.550+0000 7f404be00640 1 -- 172.21.15.69:0/1269881122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b205a0 msgr2=0x55af70b22990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.550+0000 7f404be00640 1 --2- 172.21.15.69:0/1269881122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b205a0 0x55af70b22990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.550+0000 7f404be00640 1 -- 172.21.15.69:0/1269881122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af70a46e10 con 0x55af70b15ce0 2024-04-24T09:08:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.550+0000 7f404be00640 1 --2- 172.21.15.69:0/1269881122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b15ce0 0x55af70b20060 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f403c0ee7e0 tx=0x7f403c0f7750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce28605a410e38d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.551+0000 7f4041600640 1 -- 172.21.15.69:0/1269881122 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c0f1d40 con 0x55af70b15ce0 2024-04-24T09:08:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.551+0000 7f4041600640 1 -- 172.21.15.69:0/1269881122 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f403c0fe030 con 0x55af70b15ce0 2024-04-24T09:08:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.551+0000 7f4041600640 1 -- 172.21.15.69:0/1269881122 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c103980 con 0x55af70b15ce0 2024-04-24T09:08:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.551+0000 7f4048c00640 1 --2- 172.21.15.69:0/731914659 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f40300a3e30 0x7f40300a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.551+0000 7f4048c00640 1 -- 172.21.15.69:0/731914659 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f403c0f1480 con 0x7f404c102080 2024-04-24T09:08:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.551+0000 7f4050e00640 1 --2- 172.21.15.69:0/731914659 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f40300a3e30 0x7f40300a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.551+0000 7f4055df2740 1 -- 172.21.15.69:0/1269881122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b15ce0 msgr2=0x55af70b20060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.552+0000 7f4055df2740 1 --2- 172.21.15.69:0/1269881122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b15ce0 0x55af70b20060 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f403c0ee7e0 tx=0x7f403c0f7750 comp rx=0 tx=0).stop 2024-04-24T09:08:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.552+0000 7f4055df2740 1 -- 172.21.15.69:0/1269881122 shutdown_connections 2024-04-24T09:08:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.552+0000 7f4055df2740 1 --2- 172.21.15.69:0/1269881122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b205a0 0x55af70b22990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.552+0000 7f4055df2740 1 --2- 172.21.15.69:0/1269881122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b15ce0 0x55af70b20060 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.552+0000 7f4055df2740 1 --2- 172.21.15.69:0/1269881122 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b153c0 0x55af70b157a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.552+0000 7f4055df2740 1 -- 172.21.15.69:0/1269881122 >> 172.21.15.69:0/1269881122 conn(0x55af70b118d0 msgr2=0x55af70b11cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.552+0000 7f4055df2740 1 -- 172.21.15.69:0/1269881122 shutdown_connections 2024-04-24T09:08:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.552+0000 7f4055df2740 1 -- 172.21.15.69:0/1269881122 wait complete. 2024-04-24T09:08:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.552+0000 7f4050e00640 1 --2- 172.21.15.69:0/731914659 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f40300a3e30 0x7f40300a62f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f40440099a0 tx=0x7f4044009500 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.552+0000 7f4055df2740 1 -- start start 2024-04-24T09:08:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.553+0000 7f4055df2740 1 Processor -- start 2024-04-24T09:08:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.553+0000 7f4055df2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b153c0 0x55af70b1d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.553+0000 7f4055df2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b15ce0 0x55af70b1da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.553+0000 7f4055df2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b205a0 0x55af70b1df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.553+0000 7f4055df2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55af70b22f90 con 0x55af70b205a0 2024-04-24T09:08:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.553+0000 7f4055df2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55af70a352c0 con 0x55af70b153c0 2024-04-24T09:08:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.553+0000 7f4055df2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55af70b23110 con 0x55af70b15ce0 2024-04-24T09:08:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.553+0000 7f4055df2740 10 client.0 _ll_register_callbacks cb 0x55af7078da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.553+0000 7f4050e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b205a0 0x55af70b1df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.553+0000 7f4050e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b205a0 0x55af70b1df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48292/0 (socket says 172.21.15.69:48292) 2024-04-24T09:08:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.553+0000 7f4050e00640 1 -- 172.21.15.69:0/468789821 learned_addr learned my addr 172.21.15.69:0/468789821 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.554+0000 7f404b400640 1 --2- 172.21.15.69:0/468789821 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b15ce0 0x55af70b1da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.554+0000 7f404be00640 1 --2- 172.21.15.69:0/468789821 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b153c0 0x55af70b1d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.554+0000 7f4050e00640 1 -- 172.21.15.69:0/468789821 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b15ce0 msgr2=0x55af70b1da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.554+0000 7f4050e00640 1 --2- 172.21.15.69:0/468789821 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b15ce0 0x55af70b1da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.554+0000 7f4050e00640 1 -- 172.21.15.69:0/468789821 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b153c0 msgr2=0x55af70b1d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.554+0000 7f4050e00640 1 --2- 172.21.15.69:0/468789821 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b153c0 0x55af70b1d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.554+0000 7f4050e00640 1 -- 172.21.15.69:0/468789821 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af70a45fd0 con 0x55af70b205a0 2024-04-24T09:08:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.554+0000 7f404b400640 1 --2- 172.21.15.69:0/468789821 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b15ce0 0x55af70b1da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.554+0000 7f4050e00640 1 --2- 172.21.15.69:0/468789821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b205a0 0x55af70b1df80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4044010d40 tx=0x7f4044004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6072771e743d6ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.554+0000 7f4036000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.555+0000 7f4036000640 1 -- 172.21.15.69:0/468789821 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044004b20 con 0x55af70b205a0 2024-04-24T09:08:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.555+0000 7f4036000640 1 -- 172.21.15.69:0/468789821 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4044004cc0 con 0x55af70b205a0 2024-04-24T09:08:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.555+0000 7f4055df2740 1 -- 172.21.15.69:0/468789821 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f403c0fe030 con 0x55af70b205a0 2024-04-24T09:08:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.555+0000 7f4036000640 1 -- 172.21.15.69:0/468789821 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4044030050 con 0x55af70b205a0 2024-04-24T09:08:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.555+0000 7f4055df2740 10 client.15906 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.555+0000 7f4055df2740 1 -- 172.21.15.69:0/468789821 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55af7078e210 con 0x55af70b205a0 2024-04-24T09:08:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.555+0000 7f4036000640 1 -- 172.21.15.69:0/468789821 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f403c0fe030 con 0x55af70b205a0 2024-04-24T09:08:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.556+0000 7f4036000640 1 -- 172.21.15.69:0/468789821 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4044019040 con 0x55af70b205a0 2024-04-24T09:08:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.556+0000 7f4055df2740 10 client.15906 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.556+0000 7f4055df2740 10 client.15906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.556+0000 7f4055df2740 20 client.15906 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.556+0000 7f4055df2740 1 --2- 172.21.15.69:0/468789821 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55af70b304f0 0x55af70b328e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.556+0000 7f4055df2740 4 client.15906 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.556+0000 7f4055df2740 1 -- 172.21.15.69:0/468789821 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af70a89320 con 0x55af70b304f0 2024-04-24T09:08:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.556+0000 7f404be00640 1 --2- 172.21.15.69:0/468789821 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55af70b304f0 0x55af70b328e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.557+0000 7f404be00640 1 --2- 172.21.15.69:0/468789821 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55af70b304f0 0x55af70b328e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473d20725f47069f server_cookie=f069ab5e11406be9 in_seq=0 out_seq=0 2024-04-24T09:08:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.557+0000 7f4036000640 10 client.15906 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.559+0000 7f4036000640 1 -- 172.21.15.69:0/468789821 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af70a89320 con 0x55af70b304f0 2024-04-24T09:08:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.559+0000 7f4036000640 10 client.15906 handle_command_reply: tid=0 2024-04-24T09:08:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.559+0000 7f4055df2740 1 client.15906 shutdown 2024-04-24T09:08:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.559+0000 7f4055df2740 1 -- 172.21.15.69:0/468789821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b205a0 msgr2=0x55af70b1df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.559+0000 7f4055df2740 1 --2- 172.21.15.69:0/468789821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b205a0 0x55af70b1df80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4044010d40 tx=0x7f4044004a40 comp rx=0 tx=0).stop 2024-04-24T09:08:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.559+0000 7f4055df2740 1 -- 172.21.15.69:0/468789821 shutdown_connections 2024-04-24T09:08:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.559+0000 7f4055df2740 1 --2- 172.21.15.69:0/468789821 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55af70b304f0 0x55af70b328e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.559+0000 7f4055df2740 1 --2- 172.21.15.69:0/468789821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b205a0 0x55af70b1df80 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.559+0000 7f4055df2740 1 --2- 172.21.15.69:0/468789821 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b15ce0 0x55af70b1da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.559+0000 7f4055df2740 1 --2- 172.21.15.69:0/468789821 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b153c0 0x55af70b1d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.559+0000 7f4055df2740 1 -- 172.21.15.69:0/468789821 >> 172.21.15.69:0/468789821 conn(0x55af70b118d0 msgr2=0x55af70b10a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.559+0000 7f4036000640 0 client.15906 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.560+0000 7f4055df2740 1 -- 172.21.15.69:0/468789821 shutdown_connections 2024-04-24T09:08:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.560+0000 7f4055df2740 1 -- 172.21.15.69:0/468789821 wait complete. 2024-04-24T09:08:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.560+0000 7f4055df2740 20 client.15906 trim_cache size 0 max 16384 2024-04-24T09:08:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.576+0000 7f4055df2740 1 Processor -- start 2024-04-24T09:08:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.576+0000 7f4055df2740 1 -- start start 2024-04-24T09:08:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.576+0000 7f4055df2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b153c0 0x55af70b2bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.576+0000 7f4055df2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b15ce0 0x55af70b17480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.576+0000 7f4055df2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b205a0 0x55af70b381f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.576+0000 7f4055df2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f404c1b3c10 con 0x55af70b153c0 2024-04-24T09:08:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.576+0000 7f4055df2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f404c1b14e0 con 0x55af70b205a0 2024-04-24T09:08:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.576+0000 7f4055df2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f404c1af330 con 0x55af70b15ce0 2024-04-24T09:08:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.576+0000 7f404be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b153c0 0x55af70b2bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.576+0000 7f404be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b153c0 0x55af70b2bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48302/0 (socket says 172.21.15.69:48302) 2024-04-24T09:08:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.576+0000 7f404be00640 1 -- 172.21.15.69:0/343559687 learned_addr learned my addr 172.21.15.69:0/343559687 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.577+0000 7f404b400640 1 --2- 172.21.15.69:0/343559687 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b15ce0 0x55af70b17480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.577+0000 7f4050e00640 1 --2- 172.21.15.69:0/343559687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b205a0 0x55af70b381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.577+0000 7f404be00640 1 -- 172.21.15.69:0/343559687 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b15ce0 msgr2=0x55af70b17480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.577+0000 7f404be00640 1 --2- 172.21.15.69:0/343559687 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b15ce0 0x55af70b17480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.577+0000 7f404be00640 1 -- 172.21.15.69:0/343559687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b205a0 msgr2=0x55af70b381f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.577+0000 7f404be00640 1 --2- 172.21.15.69:0/343559687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b205a0 0x55af70b381f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.577+0000 7f404be00640 1 -- 172.21.15.69:0/343559687 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af70b22d00 con 0x55af70b153c0 2024-04-24T09:08:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.577+0000 7f404be00640 1 --2- 172.21.15.69:0/343559687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b153c0 0x55af70b2bd80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f403c02a9b0 tx=0x7f403c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b55258e38e152b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.577+0000 7f4040c00640 1 -- 172.21.15.69:0/343559687 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c049040 con 0x55af70b153c0 2024-04-24T09:08:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.577+0000 7f4040c00640 1 -- 172.21.15.69:0/343559687 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f403c0fe050 con 0x55af70b153c0 2024-04-24T09:08:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.577+0000 7f4040c00640 1 -- 172.21.15.69:0/343559687 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c0bd6a0 con 0x55af70b153c0 2024-04-24T09:08:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.578+0000 7f4055df2740 1 -- 172.21.15.69:0/343559687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b153c0 msgr2=0x55af70b2bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.578+0000 7f4055df2740 1 --2- 172.21.15.69:0/343559687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b153c0 0x55af70b2bd80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f403c02a9b0 tx=0x7f403c009260 comp rx=0 tx=0).stop 2024-04-24T09:08:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.578+0000 7f4055df2740 1 -- 172.21.15.69:0/343559687 shutdown_connections 2024-04-24T09:08:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.578+0000 7f4055df2740 1 --2- 172.21.15.69:0/343559687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b205a0 0x55af70b381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.578+0000 7f4055df2740 1 --2- 172.21.15.69:0/343559687 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b15ce0 0x55af70b17480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.578+0000 7f4055df2740 1 --2- 172.21.15.69:0/343559687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b153c0 0x55af70b2bd80 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.578+0000 7f4055df2740 1 -- 172.21.15.69:0/343559687 >> 172.21.15.69:0/343559687 conn(0x55af70b118d0 msgr2=0x55af70b16d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.578+0000 7f4055df2740 1 -- 172.21.15.69:0/343559687 shutdown_connections 2024-04-24T09:08:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.578+0000 7f4055df2740 1 -- 172.21.15.69:0/343559687 wait complete. 2024-04-24T09:08:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.579+0000 7f4055df2740 1 -- start start 2024-04-24T09:08:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.579+0000 7f4055df2740 1 Processor -- start 2024-04-24T09:08:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.579+0000 7f4055df2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b153c0 0x55af70b28720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.579+0000 7f4055df2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b15ce0 0x55af70b23a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.579+0000 7f4055df2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b205a0 0x55af70b240a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.579+0000 7f4055df2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f404c1af9a0 con 0x55af70b15ce0 2024-04-24T09:08:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.579+0000 7f4055df2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f404c1b0e10 con 0x55af70b205a0 2024-04-24T09:08:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.579+0000 7f4055df2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f404c1b0070 con 0x55af70b153c0 2024-04-24T09:08:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.580+0000 7f4055df2740 10 client.0 _ll_register_callbacks cb 0x7f4044009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.580+0000 7f404b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b15ce0 0x55af70b23a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.580+0000 7f404b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b15ce0 0x55af70b23a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48312/0 (socket says 172.21.15.69:48312) 2024-04-24T09:08:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.580+0000 7f404b400640 1 -- 172.21.15.69:0/2775347018 learned_addr learned my addr 172.21.15.69:0/2775347018 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.580+0000 7f404be00640 1 --2- 172.21.15.69:0/2775347018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b153c0 0x55af70b28720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.580+0000 7f4050e00640 1 --2- 172.21.15.69:0/2775347018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b205a0 0x55af70b240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.580+0000 7f404b400640 1 -- 172.21.15.69:0/2775347018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b153c0 msgr2=0x55af70b28720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.580+0000 7f404b400640 1 --2- 172.21.15.69:0/2775347018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b153c0 0x55af70b28720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.580+0000 7f404b400640 1 -- 172.21.15.69:0/2775347018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b205a0 msgr2=0x55af70b240a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.580+0000 7f404b400640 1 --2- 172.21.15.69:0/2775347018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b205a0 0x55af70b240a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.580+0000 7f404b400640 1 -- 172.21.15.69:0/2775347018 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af7085a620 con 0x55af70b15ce0 2024-04-24T09:08:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.580+0000 7f404be00640 1 --2- 172.21.15.69:0/2775347018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b153c0 0x55af70b28720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.580+0000 7f404b400640 1 --2- 172.21.15.69:0/2775347018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b15ce0 0x55af70b23a90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f404c19bef0 tx=0x7f4038012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b17cf58c3fe413c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.580+0000 7f4036000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.581+0000 7f4036000640 1 -- 172.21.15.69:0/2775347018 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038021060 con 0x55af70b15ce0 2024-04-24T09:08:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.581+0000 7f4036000640 1 -- 172.21.15.69:0/2775347018 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4038004250 con 0x55af70b15ce0 2024-04-24T09:08:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.581+0000 7f4055df2740 1 -- 172.21.15.69:0/2775347018 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f403c0fe050 con 0x55af70b15ce0 2024-04-24T09:08:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.581+0000 7f4036000640 1 -- 172.21.15.69:0/2775347018 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4038028050 con 0x55af70b15ce0 2024-04-24T09:08:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.581+0000 7f4055df2740 10 client.15912 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.581+0000 7f4055df2740 1 -- 172.21.15.69:0/2775347018 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55af70b969f0 con 0x55af70b15ce0 2024-04-24T09:08:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.581+0000 7f4036000640 1 -- 172.21.15.69:0/2775347018 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f403c0fe050 con 0x55af70b15ce0 2024-04-24T09:08:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.581+0000 7f4036000640 1 -- 172.21.15.69:0/2775347018 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f403801b030 con 0x55af70b15ce0 2024-04-24T09:08:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.582+0000 7f4055df2740 10 client.15912 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.582+0000 7f4055df2740 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.582+0000 7f4055df2740 20 client.15912 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.582+0000 7f4055df2740 1 --2- 172.21.15.69:0/2775347018 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55af70b971d0 0x55af70b995f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.582+0000 7f4055df2740 4 client.15912 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:08:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.582+0000 7f4055df2740 1 -- 172.21.15.69:0/2775347018 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af70a83190 con 0x55af70b971d0 2024-04-24T09:08:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.582+0000 7f404be00640 1 --2- 172.21.15.69:0/2775347018 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55af70b971d0 0x55af70b995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.583+0000 7f404be00640 1 --2- 172.21.15.69:0/2775347018 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55af70b971d0 0x55af70b995f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14753482e3c08d81 server_cookie=9281dd974df0f61e in_seq=0 out_seq=0 2024-04-24T09:08:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.583+0000 7f4036000640 10 client.15912 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.598+0000 7f4036000640 1 -- 172.21.15.69:0/2775347018 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af70a83190 con 0x55af70b971d0 2024-04-24T09:08:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.598+0000 7f4036000640 10 client.15912 handle_command_reply: tid=0 2024-04-24T09:08:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.598+0000 7f4055df2740 1 client.15912 shutdown 2024-04-24T09:08:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.598+0000 7f4055df2740 1 -- 172.21.15.69:0/2775347018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b15ce0 msgr2=0x55af70b23a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.598+0000 7f4055df2740 1 --2- 172.21.15.69:0/2775347018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b15ce0 0x55af70b23a90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f404c19bef0 tx=0x7f4038012b60 comp rx=0 tx=0).stop 2024-04-24T09:08:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.598+0000 7f4055df2740 1 -- 172.21.15.69:0/2775347018 shutdown_connections 2024-04-24T09:08:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.598+0000 7f4055df2740 1 --2- 172.21.15.69:0/2775347018 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55af70b971d0 0x55af70b995f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.598+0000 7f4055df2740 1 --2- 172.21.15.69:0/2775347018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af70b205a0 0x55af70b240a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.598+0000 7f4055df2740 1 --2- 172.21.15.69:0/2775347018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af70b15ce0 0x55af70b23a90 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.598+0000 7f4055df2740 1 --2- 172.21.15.69:0/2775347018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af70b153c0 0x55af70b28720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.598+0000 7f4036000640 0 client.15912 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.598+0000 7f4055df2740 1 -- 172.21.15.69:0/2775347018 >> 172.21.15.69:0/2775347018 conn(0x55af70b118d0 msgr2=0x55af70b17600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.599+0000 7f4055df2740 1 -- 172.21.15.69:0/2775347018 shutdown_connections 2024-04-24T09:08:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.599+0000 7f4055df2740 1 -- 172.21.15.69:0/2775347018 wait complete. 2024-04-24T09:08:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.599+0000 7f4055df2740 20 client.15912 trim_cache size 0 max 16384 2024-04-24T09:08:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:08:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:08:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:08:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:08:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.599+0000 7f4051800640 1 -- 172.21.15.69:0/731914659 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f40300a3e30 msgr2=0x7f40300a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.599+0000 7f4051800640 1 --2- 172.21.15.69:0/731914659 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f40300a3e30 0x7f40300a62f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f40440099a0 tx=0x7f4044009500 comp rx=0 tx=0).stop 2024-04-24T09:08:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.599+0000 7f4051800640 1 -- 172.21.15.69:0/731914659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c102080 msgr2=0x7f404c1a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.599+0000 7f4051800640 1 --2- 172.21.15.69:0/731914659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c102080 0x7f404c1a05e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f403c005ab0 tx=0x7f403c003c10 comp rx=0 tx=0).stop 2024-04-24T09:08:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.599+0000 7f4051800640 1 -- 172.21.15.69:0/731914659 shutdown_connections 2024-04-24T09:08:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.599+0000 7f4051800640 1 --2- 172.21.15.69:0/731914659 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f40300a3e30 0x7f40300a62f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.599+0000 7f4051800640 1 --2- 172.21.15.69:0/731914659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f404c19a6a0 0x7f404c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.599+0000 7f4051800640 1 --2- 172.21.15.69:0/731914659 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f404c107150 0x7f404c1a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.599+0000 7f4051800640 1 --2- 172.21.15.69:0/731914659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c102080 0x7f404c1a05e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.600+0000 7f4051800640 1 -- 172.21.15.69:0/731914659 >> 172.21.15.69:0/731914659 conn(0x7f404c0fd980 msgr2=0x7f404c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.600+0000 7f4051800640 1 -- 172.21.15.69:0/731914659 shutdown_connections 2024-04-24T09:08:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:07.600+0000 7f4051800640 1 -- 172.21.15.69:0/731914659 wait complete. 2024-04-24T09:08:08.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:07 smithi081 ceph-mon[30961]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T09:08:08.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:07 smithi069 ceph-mon[28170]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T09:08:08.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:07 smithi130 ceph-mon[31553]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T09:08:10.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:09 smithi081 ceph-mon[30961]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T09:08:10.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:09 smithi069 ceph-mon[28170]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T09:08:10.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:09 smithi130 ceph-mon[31553]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T09:08:12.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:11 smithi081 ceph-mon[30961]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T09:08:12.090 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:11 smithi069 ceph-mon[28170]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T09:08:12.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:11 smithi130 ceph-mon[31553]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T09:08:14.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:13 smithi081 ceph-mon[30961]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-24T09:08:14.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:13 smithi069 ceph-mon[28170]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-24T09:08:14.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:13 smithi130 ceph-mon[31553]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-24T09:08:15.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:14 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:08:15.090 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:14 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:08:15.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:14 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:08:16.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:08:16.066 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:08:16.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:15 smithi081 ceph-mon[30961]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-24T09:08:16.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:15 smithi069 ceph-mon[28170]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-24T09:08:16.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:15 smithi130 ceph-mon[31553]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-24T09:08:16.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.347+0000 7f1c85a00640 1 -- 172.21.15.69:0/2599905917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c801071e0 msgr2=0x7f1c8010edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:16.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.347+0000 7f1c85a00640 1 --2- 172.21.15.69:0/2599905917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c801071e0 0x7f1c8010edb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009ca0 tx=0x7f1c7802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.347+0000 7f1c85a00640 1 -- 172.21.15.69:0/2599905917 shutdown_connections 2024-04-24T09:08:16.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.347+0000 7f1c85a00640 1 --2- 172.21.15.69:0/2599905917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c801071e0 0x7f1c8010edb0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.347+0000 7f1c85a00640 1 --2- 172.21.15.69:0/2599905917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c80102ac0 0x7f1c80106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.347+0000 7f1c85a00640 1 --2- 172.21.15.69:0/2599905917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c80102110 0x7f1c801024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.347+0000 7f1c85a00640 1 -- 172.21.15.69:0/2599905917 >> 172.21.15.69:0/2599905917 conn(0x7f1c800fd9d0 msgr2=0x7f1c800ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:16.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.347+0000 7f1c85a00640 1 -- 172.21.15.69:0/2599905917 shutdown_connections 2024-04-24T09:08:16.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.347+0000 7f1c85a00640 1 -- 172.21.15.69:0/2599905917 wait complete. 2024-04-24T09:08:16.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.349+0000 7f1c85a00640 1 Processor -- start 2024-04-24T09:08:16.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.349+0000 7f1c85a00640 1 -- start start 2024-04-24T09:08:16.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.350+0000 7f1c85a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c80102110 0x7f1c801a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.350+0000 7f1c85a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c801071e0 0x7f1c801a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.350+0000 7f1c85a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c8019a780 0x7f1c8019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.350+0000 7f1c85a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1c801121a0 con 0x7f1c8019a780 2024-04-24T09:08:16.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.350+0000 7f1c85a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1c80112020 con 0x7f1c80102110 2024-04-24T09:08:16.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.350+0000 7f1c85a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1c80112320 con 0x7f1c801071e0 2024-04-24T09:08:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.350+0000 7f1c85000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c8019a780 0x7f1c8019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.350+0000 7f1c85000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c8019a780 0x7f1c8019ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55256/0 (socket says 172.21.15.69:55256) 2024-04-24T09:08:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.350+0000 7f1c85000640 1 -- 172.21.15.69:0/3297472094 learned_addr learned my addr 172.21.15.69:0/3297472094 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.350+0000 7f1c7fe00640 1 --2- 172.21.15.69:0/3297472094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c80102110 0x7f1c801a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.351+0000 7f1c7f400640 1 --2- 172.21.15.69:0/3297472094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c801071e0 0x7f1c801a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.351+0000 7f1c85000640 1 -- 172.21.15.69:0/3297472094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c801071e0 msgr2=0x7f1c801a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.351+0000 7f1c85000640 1 --2- 172.21.15.69:0/3297472094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c801071e0 0x7f1c801a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.351+0000 7f1c85000640 1 -- 172.21.15.69:0/3297472094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c80102110 msgr2=0x7f1c801a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.351+0000 7f1c85000640 1 --2- 172.21.15.69:0/3297472094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c80102110 0x7f1c801a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.351+0000 7f1c85000640 1 -- 172.21.15.69:0/3297472094 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8019b310 con 0x7f1c8019a780 2024-04-24T09:08:16.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.351+0000 7f1c85000640 1 --2- 172.21.15.69:0/3297472094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c8019a780 0x7f1c8019ac30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1c78031040 tx=0x7f1c78003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:16.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.352+0000 7f1c7cc00640 1 -- 172.21.15.69:0/3297472094 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78003ce0 con 0x7f1c8019a780 2024-04-24T09:08:16.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.352+0000 7f1c7cc00640 1 -- 172.21.15.69:0/3297472094 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c780044f0 con 0x7f1c8019a780 2024-04-24T09:08:16.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.352+0000 7f1c85a00640 1 -- 172.21.15.69:0/3297472094 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c8019b5a0 con 0x7f1c8019a780 2024-04-24T09:08:16.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.352+0000 7f1c85a00640 1 -- 172.21.15.69:0/3297472094 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c801a7780 con 0x7f1c8019a780 2024-04-24T09:08:16.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.353+0000 7f1c7cc00640 1 -- 172.21.15.69:0/3297472094 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78030c90 con 0x7f1c8019a780 2024-04-24T09:08:16.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.356+0000 7f1c89d7d740 1 Processor -- start 2024-04-24T09:08:16.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.357+0000 7f1c7cc00640 1 -- 172.21.15.69:0/3297472094 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f1c7804e430 con 0x7f1c8019a780 2024-04-24T09:08:16.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.357+0000 7f1c89d7d740 1 -- start start 2024-04-24T09:08:16.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.358+0000 7f1c89d7d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc881a30 0x55dbfc881e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.358+0000 7f1c89d7d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc88c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.358+0000 7f1c89d7d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc88ca20 0x55dbfc88ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.358+0000 7f1c89d7d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55dbfc7e71f0 con 0x55dbfc882350 2024-04-24T09:08:16.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.358+0000 7f1c89d7d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55dbfc78d350 con 0x55dbfc881a30 2024-04-24T09:08:16.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.358+0000 7f1c89d7d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55dbfc7e31b0 con 0x55dbfc88ca20 2024-04-24T09:08:16.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.358+0000 7f1c7fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc88c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.359+0000 7f1c7fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc88c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55272/0 (socket says 172.21.15.69:55272) 2024-04-24T09:08:16.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.359+0000 7f1c7fe00640 1 -- 172.21.15.69:0/4238126294 learned_addr learned my addr 172.21.15.69:0/4238126294 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:16.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.359+0000 7f1c7f400640 1 --2- 172.21.15.69:0/4238126294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc881a30 0x55dbfc881e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.359+0000 7f1c7f400640 1 --2- 172.21.15.69:0/4238126294 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc88ca20 0x55dbfc88ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.359+0000 7f1c7fe00640 1 -- 172.21.15.69:0/4238126294 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc88ca20 msgr2=0x55dbfc88ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:16.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.359+0000 7f1c7fe00640 1 --2- 172.21.15.69:0/4238126294 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc88ca20 0x55dbfc88ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.359+0000 7f1c7fe00640 1 -- 172.21.15.69:0/4238126294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc881a30 msgr2=0x55dbfc881e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:16.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.359+0000 7f1c7fe00640 1 --2- 172.21.15.69:0/4238126294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc881a30 0x55dbfc881e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.359+0000 7f1c7fe00640 1 -- 172.21.15.69:0/4238126294 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbfc7e1f50 con 0x55dbfc882350 2024-04-24T09:08:16.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.359+0000 7f1c7cc00640 1 --2- 172.21.15.69:0/3297472094 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1c640a3e40 0x7f1c640a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.360+0000 7f1c7fe00640 1 --2- 172.21.15.69:0/4238126294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc88c4e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000bcb0 tx=0x7f1c7000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed2757b5af5f288 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:16.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.360+0000 7f1c7cc00640 1 -- 172.21.15.69:0/3297472094 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f1c780f2420 con 0x7f1c8019a780 2024-04-24T09:08:16.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.360+0000 7f1c7f400640 1 --2- 172.21.15.69:0/3297472094 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1c640a3e40 0x7f1c640a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.360+0000 7f1c75600640 1 -- 172.21.15.69:0/4238126294 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70018be0 con 0x55dbfc882350 2024-04-24T09:08:16.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.360+0000 7f1c75600640 1 -- 172.21.15.69:0/4238126294 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c70018d80 con 0x55dbfc882350 2024-04-24T09:08:16.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.360+0000 7f1c75600640 1 -- 172.21.15.69:0/4238126294 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7001f5a0 con 0x55dbfc882350 2024-04-24T09:08:16.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.361+0000 7f1c7f400640 1 --2- 172.21.15.69:0/3297472094 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1c640a3e40 0x7f1c640a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55dbfc87eee0 tx=0x7f1c6c016040 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:16.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.361+0000 7f1c89d7d740 1 -- 172.21.15.69:0/4238126294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 msgr2=0x55dbfc88c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:16.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.361+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/4238126294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc88c4e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000bcb0 tx=0x7f1c7000f730 comp rx=0 tx=0).stop 2024-04-24T09:08:16.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.362+0000 7f1c89d7d740 1 -- 172.21.15.69:0/4238126294 shutdown_connections 2024-04-24T09:08:16.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.362+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/4238126294 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc88ca20 0x55dbfc88ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.362+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/4238126294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc88c4e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.362+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/4238126294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc881a30 0x55dbfc881e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.362+0000 7f1c89d7d740 1 -- 172.21.15.69:0/4238126294 >> 172.21.15.69:0/4238126294 conn(0x55dbfc87dd50 msgr2=0x55dbfc87e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:16.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.362+0000 7f1c89d7d740 1 -- 172.21.15.69:0/4238126294 shutdown_connections 2024-04-24T09:08:16.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.362+0000 7f1c89d7d740 1 -- 172.21.15.69:0/4238126294 wait complete. 2024-04-24T09:08:16.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.362+0000 7f1c89d7d740 1 -- start start 2024-04-24T09:08:16.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.363+0000 7f1c89d7d740 1 Processor -- start 2024-04-24T09:08:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.363+0000 7f1c89d7d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc881a30 0x55dbfc8899f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.363+0000 7f1c89d7d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc882350 0x55dbfc889f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.363+0000 7f1c89d7d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc88ca20 0x55dbfc88a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.363+0000 7f1c89d7d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55dbfc88f810 con 0x55dbfc88ca20 2024-04-24T09:08:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.364+0000 7f1c89d7d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55dbfc7a0690 con 0x55dbfc881a30 2024-04-24T09:08:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.364+0000 7f1c89d7d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55dbfc88f990 con 0x55dbfc882350 2024-04-24T09:08:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.364+0000 7f1c89d7d740 10 client.0 _ll_register_callbacks cb 0x55dbfc7dc650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.364+0000 7f1c85000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc88ca20 0x55dbfc88a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.364+0000 7f1c85000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc88ca20 0x55dbfc88a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55286/0 (socket says 172.21.15.69:55286) 2024-04-24T09:08:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.364+0000 7f1c85000640 1 -- 172.21.15.69:0/1795875169 learned_addr learned my addr 172.21.15.69:0/1795875169 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:16.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.364+0000 7f1c7f400640 1 --2- 172.21.15.69:0/1795875169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc882350 0x55dbfc889f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.364+0000 7f1c7fe00640 1 --2- 172.21.15.69:0/1795875169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc881a30 0x55dbfc8899f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.364+0000 7f1c85000640 1 -- 172.21.15.69:0/1795875169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc882350 msgr2=0x55dbfc889f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:16.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.364+0000 7f1c85000640 1 --2- 172.21.15.69:0/1795875169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc882350 0x55dbfc889f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.364+0000 7f1c85000640 1 -- 172.21.15.69:0/1795875169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc881a30 msgr2=0x55dbfc8899f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:16.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.364+0000 7f1c85000640 1 --2- 172.21.15.69:0/1795875169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc881a30 0x55dbfc8899f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.364+0000 7f1c85000640 1 -- 172.21.15.69:0/1795875169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbfc79f920 con 0x55dbfc88ca20 2024-04-24T09:08:16.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.365+0000 7f1c85000640 1 --2- 172.21.15.69:0/1795875169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc88ca20 0x55dbfc88a470 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1c78005e30 tx=0x7f1c78030e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddbe4aa07967aca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:16.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.365+0000 7f1c6a000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.365+0000 7f1c6a000640 1 -- 172.21.15.69:0/1795875169 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c780be640 con 0x55dbfc88ca20 2024-04-24T09:08:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.365+0000 7f1c6a000640 1 -- 172.21.15.69:0/1795875169 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f1c780046a0 con 0x55dbfc88ca20 2024-04-24T09:08:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.365+0000 7f1c89d7d740 1 -- 172.21.15.69:0/1795875169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c70018d80 con 0x55dbfc88ca20 2024-04-24T09:08:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.366+0000 7f1c6a000640 1 -- 172.21.15.69:0/1795875169 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1c780bdf40 con 0x55dbfc88ca20 2024-04-24T09:08:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.366+0000 7f1c89d7d740 10 client.15924 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.366+0000 7f1c89d7d740 1 -- 172.21.15.69:0/1795875169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dbfc5c37d0 con 0x55dbfc88ca20 2024-04-24T09:08:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.366+0000 7f1c6a000640 1 -- 172.21.15.69:0/1795875169 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c70018d80 con 0x55dbfc88ca20 2024-04-24T09:08:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.366+0000 7f1c6a000640 1 -- 172.21.15.69:0/1795875169 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f1c780c6720 con 0x55dbfc88ca20 2024-04-24T09:08:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.367+0000 7f1c89d7d740 10 client.15924 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.367+0000 7f1c89d7d740 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.367+0000 7f1c89d7d740 20 client.15924 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.367+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/1795875169 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55dbfc89cde0 0x55dbfc89f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.367+0000 7f1c89d7d740 4 client.15924 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.367+0000 7f1c89d7d740 1 -- 172.21.15.69:0/1795875169 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbfc7f4c10 con 0x55dbfc89cde0 2024-04-24T09:08:16.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.367+0000 7f1c7fe00640 1 --2- 172.21.15.69:0/1795875169 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55dbfc89cde0 0x55dbfc89f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.369+0000 7f1c7fe00640 1 --2- 172.21.15.69:0/1795875169 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55dbfc89cde0 0x55dbfc89f280 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=118bccc99c34a265 server_cookie=a9799e14b90eea74 in_seq=0 out_seq=0 2024-04-24T09:08:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.369+0000 7f1c6a000640 10 client.15924 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:16.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.370+0000 7f1c6a000640 1 -- 172.21.15.69:0/1795875169 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dbfc7f4c10 con 0x55dbfc89cde0 2024-04-24T09:08:16.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.370+0000 7f1c6a000640 10 client.15924 handle_command_reply: tid=0 2024-04-24T09:08:16.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.370+0000 7f1c89d7d740 1 client.15924 shutdown 2024-04-24T09:08:16.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.371+0000 7f1c89d7d740 1 -- 172.21.15.69:0/1795875169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc88ca20 msgr2=0x55dbfc88a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:16.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.371+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/1795875169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc88ca20 0x55dbfc88a470 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1c78005e30 tx=0x7f1c78030e30 comp rx=0 tx=0).stop 2024-04-24T09:08:16.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.371+0000 7f1c89d7d740 1 -- 172.21.15.69:0/1795875169 shutdown_connections 2024-04-24T09:08:16.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.371+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/1795875169 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55dbfc89cde0 0x55dbfc89f280 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.371+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/1795875169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc88ca20 0x55dbfc88a470 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.371+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/1795875169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc882350 0x55dbfc889f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.371+0000 7f1c6a000640 0 client.15924 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.371+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/1795875169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc881a30 0x55dbfc8899f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.371+0000 7f1c89d7d740 1 -- 172.21.15.69:0/1795875169 >> 172.21.15.69:0/1795875169 conn(0x55dbfc87dd50 msgr2=0x55dbfc87cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.371+0000 7f1c89d7d740 1 -- 172.21.15.69:0/1795875169 shutdown_connections 2024-04-24T09:08:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.371+0000 7f1c89d7d740 1 -- 172.21.15.69:0/1795875169 wait complete. 2024-04-24T09:08:16.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.372+0000 7f1c89d7d740 20 client.15924 trim_cache size 0 max 16384 2024-04-24T09:08:16.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.406+0000 7f1c89d7d740 1 Processor -- start 2024-04-24T09:08:16.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.406+0000 7f1c89d7d740 1 -- start start 2024-04-24T09:08:16.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.407+0000 7f1c89d7d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc881a30 0x55dbfc8986d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.407+0000 7f1c89d7d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc88a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.407+0000 7f1c89d7d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc88ca20 0x55dbfc8a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.407+0000 7f1c89d7d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1c801af7b0 con 0x55dbfc882350 2024-04-24T09:08:16.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.407+0000 7f1c89d7d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1c801ad080 con 0x55dbfc881a30 2024-04-24T09:08:16.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.407+0000 7f1c89d7d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1c801aaed0 con 0x55dbfc88ca20 2024-04-24T09:08:16.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.407+0000 7f1c7f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc88a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.407+0000 7f1c7f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc88a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55292/0 (socket says 172.21.15.69:55292) 2024-04-24T09:08:16.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.407+0000 7f1c7f400640 1 -- 172.21.15.69:0/296892892 learned_addr learned my addr 172.21.15.69:0/296892892 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.407+0000 7f1c7fe00640 1 --2- 172.21.15.69:0/296892892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc881a30 0x55dbfc8986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.408+0000 7f1c85000640 1 --2- 172.21.15.69:0/296892892 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc88ca20 0x55dbfc8a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.408+0000 7f1c7f400640 1 -- 172.21.15.69:0/296892892 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc88ca20 msgr2=0x55dbfc8a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.408+0000 7f1c7f400640 1 --2- 172.21.15.69:0/296892892 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc88ca20 0x55dbfc8a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.408+0000 7f1c7f400640 1 -- 172.21.15.69:0/296892892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc881a30 msgr2=0x55dbfc8986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.408+0000 7f1c7f400640 1 --2- 172.21.15.69:0/296892892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc881a30 0x55dbfc8986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.408+0000 7f1c7f400640 1 -- 172.21.15.69:0/296892892 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbfc88f180 con 0x55dbfc882350 2024-04-24T09:08:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.408+0000 7f1c7fe00640 1 --2- 172.21.15.69:0/296892892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc881a30 0x55dbfc8986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.408+0000 7f1c7f400640 1 --2- 172.21.15.69:0/296892892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc88a240 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c0116d0 tx=0x7f1c6c00c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5badabd85f4ea77 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.409+0000 7f1c74c00640 1 -- 172.21.15.69:0/296892892 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00f790 con 0x55dbfc882350 2024-04-24T09:08:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.409+0000 7f1c74c00640 1 -- 172.21.15.69:0/296892892 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c6c025070 con 0x55dbfc882350 2024-04-24T09:08:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.409+0000 7f1c74c00640 1 -- 172.21.15.69:0/296892892 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c029c40 con 0x55dbfc882350 2024-04-24T09:08:16.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.410+0000 7f1c89d7d740 1 -- 172.21.15.69:0/296892892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 msgr2=0x55dbfc88a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.410+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/296892892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc88a240 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c0116d0 tx=0x7f1c6c00c3b0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.410+0000 7f1c89d7d740 1 -- 172.21.15.69:0/296892892 shutdown_connections 2024-04-24T09:08:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.410+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/296892892 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc88ca20 0x55dbfc8a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.410+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/296892892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc88a240 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.410+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/296892892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc881a30 0x55dbfc8986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.410+0000 7f1c89d7d740 1 -- 172.21.15.69:0/296892892 >> 172.21.15.69:0/296892892 conn(0x55dbfc87dd50 msgr2=0x55dbfc883180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.410+0000 7f1c89d7d740 1 -- 172.21.15.69:0/296892892 shutdown_connections 2024-04-24T09:08:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.410+0000 7f1c89d7d740 1 -- 172.21.15.69:0/296892892 wait complete. 2024-04-24T09:08:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.411+0000 7f1c89d7d740 1 -- start start 2024-04-24T09:08:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.411+0000 7f1c89d7d740 1 Processor -- start 2024-04-24T09:08:16.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.411+0000 7f1c89d7d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc881a30 0x55dbfc894be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.411+0000 7f1c89d7d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc895120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.411+0000 7f1c89d7d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc88ca20 0x55dbfc891bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.412+0000 7f1c89d7d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1c801ab540 con 0x55dbfc882350 2024-04-24T09:08:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.412+0000 7f1c89d7d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1c801ac9b0 con 0x55dbfc88ca20 2024-04-24T09:08:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.412+0000 7f1c89d7d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1c801abc10 con 0x55dbfc881a30 2024-04-24T09:08:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.412+0000 7f1c89d7d740 10 client.0 _ll_register_callbacks cb 0x7f1c780b9ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.412+0000 7f1c7f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc895120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.412+0000 7f1c7f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc895120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55294/0 (socket says 172.21.15.69:55294) 2024-04-24T09:08:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.412+0000 7f1c7f400640 1 -- 172.21.15.69:0/1205747394 learned_addr learned my addr 172.21.15.69:0/1205747394 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.412+0000 7f1c7fe00640 1 --2- 172.21.15.69:0/1205747394 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc881a30 0x55dbfc894be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.412+0000 7f1c85000640 1 --2- 172.21.15.69:0/1205747394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc88ca20 0x55dbfc891bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.413+0000 7f1c7f400640 1 -- 172.21.15.69:0/1205747394 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc881a30 msgr2=0x55dbfc894be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.413+0000 7f1c7f400640 1 --2- 172.21.15.69:0/1205747394 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc881a30 0x55dbfc894be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.413+0000 7f1c7f400640 1 -- 172.21.15.69:0/1205747394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc88ca20 msgr2=0x55dbfc891bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.413+0000 7f1c7f400640 1 --2- 172.21.15.69:0/1205747394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc88ca20 0x55dbfc891bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.413+0000 7f1c7f400640 1 -- 172.21.15.69:0/1205747394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbfc4f93f0 con 0x55dbfc882350 2024-04-24T09:08:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.413+0000 7f1c7f400640 1 --2- 172.21.15.69:0/1205747394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc895120 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00cf60 tx=0x7f1c6c029bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47166e0b86cd0157 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.413+0000 7f1c6a000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.413+0000 7f1c6a000640 1 -- 172.21.15.69:0/1205747394 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c037060 con 0x55dbfc882350 2024-04-24T09:08:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.414+0000 7f1c6a000640 1 -- 172.21.15.69:0/1205747394 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f1c6c005040 con 0x55dbfc882350 2024-04-24T09:08:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.414+0000 7f1c89d7d740 1 -- 172.21.15.69:0/1205747394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c6c025070 con 0x55dbfc882350 2024-04-24T09:08:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.414+0000 7f1c6a000640 1 -- 172.21.15.69:0/1205747394 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1c6c005240 con 0x55dbfc882350 2024-04-24T09:08:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.414+0000 7f1c89d7d740 10 client.15930 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.414+0000 7f1c89d7d740 1 -- 172.21.15.69:0/1205747394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dbfc4b90f0 con 0x55dbfc882350 2024-04-24T09:08:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.415+0000 7f1c6a000640 1 -- 172.21.15.69:0/1205747394 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c6c025070 con 0x55dbfc882350 2024-04-24T09:08:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.415+0000 7f1c6a000640 1 -- 172.21.15.69:0/1205747394 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f1c6c01e030 con 0x55dbfc882350 2024-04-24T09:08:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.415+0000 7f1c89d7d740 10 client.15930 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.415+0000 7f1c89d7d740 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.415+0000 7f1c89d7d740 20 client.15930 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.415+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/1205747394 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55dbfc904130 0x55dbfc906550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.415+0000 7f1c89d7d740 4 client.15930 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:08:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.415+0000 7f1c89d7d740 1 -- 172.21.15.69:0/1205747394 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dbfc7eea80 con 0x55dbfc904130 2024-04-24T09:08:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.416+0000 7f1c7fe00640 1 --2- 172.21.15.69:0/1205747394 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55dbfc904130 0x55dbfc906550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.417+0000 7f1c7fe00640 1 --2- 172.21.15.69:0/1205747394 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55dbfc904130 0x55dbfc906550 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b839d7b6da36eba server_cookie=91e1600dff4f4f3a in_seq=0 out_seq=0 2024-04-24T09:08:16.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.417+0000 7f1c6a000640 10 client.15930 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:16.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.417+0000 7f1c6a000640 1 -- 172.21.15.69:0/1205747394 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dbfc7eea80 con 0x55dbfc904130 2024-04-24T09:08:16.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.417+0000 7f1c6a000640 10 client.15930 handle_command_reply: tid=0 2024-04-24T09:08:16.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.417+0000 7f1c89d7d740 1 client.15930 shutdown 2024-04-24T09:08:16.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.418+0000 7f1c89d7d740 1 -- 172.21.15.69:0/1205747394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 msgr2=0x55dbfc895120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:16.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.418+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/1205747394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc895120 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00cf60 tx=0x7f1c6c029bb0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.418+0000 7f1c89d7d740 1 -- 172.21.15.69:0/1205747394 shutdown_connections 2024-04-24T09:08:16.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.418+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/1205747394 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55dbfc904130 0x55dbfc906550 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.418+0000 7f1c6a000640 0 client.15930 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:16.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.418+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/1205747394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dbfc88ca20 0x55dbfc891bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.418+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/1205747394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dbfc882350 0x55dbfc895120 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.418+0000 7f1c89d7d740 1 --2- 172.21.15.69:0/1205747394 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dbfc881a30 0x55dbfc894be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.418+0000 7f1c89d7d740 1 -- 172.21.15.69:0/1205747394 >> 172.21.15.69:0/1205747394 conn(0x55dbfc87dd50 msgr2=0x55dbfc88a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:16.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.418+0000 7f1c89d7d740 1 -- 172.21.15.69:0/1205747394 shutdown_connections 2024-04-24T09:08:16.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.418+0000 7f1c89d7d740 1 -- 172.21.15.69:0/1205747394 wait complete. 2024-04-24T09:08:16.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.418+0000 7f1c89d7d740 20 client.15930 trim_cache size 0 max 16384 2024-04-24T09:08:16.419 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:08:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-04-24T09:08:16.420 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-04-24T09:08:16.420 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:08:16.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.419+0000 7f1c85a00640 1 -- 172.21.15.69:0/3297472094 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1c640a3e40 msgr2=0x7f1c640a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:16.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.419+0000 7f1c85a00640 1 --2- 172.21.15.69:0/3297472094 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1c640a3e40 0x7f1c640a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55dbfc87eee0 tx=0x7f1c6c016040 comp rx=0 tx=0).stop 2024-04-24T09:08:16.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.419+0000 7f1c85a00640 1 -- 172.21.15.69:0/3297472094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c8019a780 msgr2=0x7f1c8019ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:16.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.419+0000 7f1c85a00640 1 --2- 172.21.15.69:0/3297472094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c8019a780 0x7f1c8019ac30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1c78031040 tx=0x7f1c78003970 comp rx=0 tx=0).stop 2024-04-24T09:08:16.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.419+0000 7f1c85a00640 1 -- 172.21.15.69:0/3297472094 shutdown_connections 2024-04-24T09:08:16.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.419+0000 7f1c85a00640 1 --2- 172.21.15.69:0/3297472094 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1c640a3e40 0x7f1c640a6300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.419+0000 7f1c85a00640 1 --2- 172.21.15.69:0/3297472094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c8019a780 0x7f1c8019ac30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.419+0000 7f1c85a00640 1 --2- 172.21.15.69:0/3297472094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c801071e0 0x7f1c801a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.419+0000 7f1c85a00640 1 --2- 172.21.15.69:0/3297472094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c80102110 0x7f1c801a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.419+0000 7f1c85a00640 1 -- 172.21.15.69:0/3297472094 >> 172.21.15.69:0/3297472094 conn(0x7f1c800fd9d0 msgr2=0x7f1c800ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:16.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.419+0000 7f1c85a00640 1 -- 172.21.15.69:0/3297472094 shutdown_connections 2024-04-24T09:08:16.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.420+0000 7f1c85a00640 1 -- 172.21.15.69:0/3297472094 wait complete. 2024-04-24T09:08:16.442 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:08:16.442 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:08:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.664+0000 7f842b000640 1 -- 172.21.15.69:0/2834926069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8424104270 msgr2=0x7f8424104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.664+0000 7f842b000640 1 --2- 172.21.15.69:0/2834926069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8424104270 0x7f8424104650 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8418009900 tx=0x7f841802a410 comp rx=0 tx=0).stop 2024-04-24T09:08:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.664+0000 7f842b000640 1 -- 172.21.15.69:0/2834926069 shutdown_connections 2024-04-24T09:08:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.664+0000 7f842b000640 1 --2- 172.21.15.69:0/2834926069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8424109340 0x7f8424110f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.664+0000 7f842b000640 1 --2- 172.21.15.69:0/2834926069 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8424104c20 0x7f8424108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.664+0000 7f842b000640 1 --2- 172.21.15.69:0/2834926069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8424104270 0x7f8424104650 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.664+0000 7f842b000640 1 -- 172.21.15.69:0/2834926069 >> 172.21.15.69:0/2834926069 conn(0x7f84240ffad0 msgr2=0x7f8424101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.664+0000 7f842b000640 1 -- 172.21.15.69:0/2834926069 shutdown_connections 2024-04-24T09:08:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.664+0000 7f842b000640 1 -- 172.21.15.69:0/2834926069 wait complete. 2024-04-24T09:08:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.666+0000 7f842b000640 1 Processor -- start 2024-04-24T09:08:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.666+0000 7f842b000640 1 -- start start 2024-04-24T09:08:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.666+0000 7f842b000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8424104270 0x7f84241a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.666+0000 7f842b000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8424109340 0x7f84241a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.666+0000 7f842b000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f842419a6b0 0x7f842419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.667+0000 7f842b000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f84241142f0 con 0x7f842419a6b0 2024-04-24T09:08:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.667+0000 7f842b000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8424114170 con 0x7f8424109340 2024-04-24T09:08:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.667+0000 7f842b000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8424114470 con 0x7f8424104270 2024-04-24T09:08:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.667+0000 7f8429c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8424104270 0x7f84241a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.667+0000 7f8429c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8424104270 0x7f84241a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.69:45090/0 (socket says 172.21.15.69:45090) 2024-04-24T09:08:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.667+0000 7f8429c00640 1 -- 172.21.15.69:0/3074690135 learned_addr learned my addr 172.21.15.69:0/3074690135 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.667+0000 7f8429200640 1 --2- 172.21.15.69:0/3074690135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8424109340 0x7f84241a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.668+0000 7f842a600640 1 --2- 172.21.15.69:0/3074690135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f842419a6b0 0x7f842419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.668+0000 7f8429c00640 1 -- 172.21.15.69:0/3074690135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8424109340 msgr2=0x7f84241a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.668+0000 7f8429c00640 1 --2- 172.21.15.69:0/3074690135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8424109340 0x7f84241a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.668+0000 7f8429c00640 1 -- 172.21.15.69:0/3074690135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f842419a6b0 msgr2=0x7f842419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.668+0000 7f8429c00640 1 --2- 172.21.15.69:0/3074690135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f842419a6b0 0x7f842419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.668+0000 7f8429c00640 1 -- 172.21.15.69:0/3074690135 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842419b2b0 con 0x7f8424104270 2024-04-24T09:08:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.669+0000 7f8429c00640 1 --2- 172.21.15.69:0/3074690135 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8424104270 0x7f84241a05f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8418005ab0 tx=0x7f8418003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.669+0000 7f8422000640 1 -- 172.21.15.69:0/3074690135 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841803f070 con 0x7f8424104270 2024-04-24T09:08:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.670+0000 7f842b000640 1 -- 172.21.15.69:0/3074690135 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842419b540 con 0x7f8424104270 2024-04-24T09:08:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.670+0000 7f842b000640 1 -- 172.21.15.69:0/3074690135 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84241a76d0 con 0x7f8424104270 2024-04-24T09:08:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.670+0000 7f8422000640 1 -- 172.21.15.69:0/3074690135 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8418031d60 con 0x7f8424104270 2024-04-24T09:08:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.670+0000 7f8422000640 1 -- 172.21.15.69:0/3074690135 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8418035030 con 0x7f8424104270 2024-04-24T09:08:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.671+0000 7f842b000640 1 -- 172.21.15.69:0/3074690135 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8404005290 con 0x7f8424104270 2024-04-24T09:08:16.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.676+0000 7f8422000640 1 -- 172.21.15.69:0/3074690135 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f8418042070 con 0x7f8424104270 2024-04-24T09:08:16.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.678+0000 7f8422000640 1 --2- 172.21.15.69:0/3074690135 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f840c0a3e40 0x7f840c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:16.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.678+0000 7f8422000640 1 -- 172.21.15.69:0/3074690135 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f84180f1820 con 0x7f8424104270 2024-04-24T09:08:16.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.678+0000 7f8429200640 1 --2- 172.21.15.69:0/3074690135 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f840c0a3e40 0x7f840c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:16.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.679+0000 7f8429200640 1 --2- 172.21.15.69:0/3074690135 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f840c0a3e40 0x7f840c0a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f842419bf00 tx=0x7f8410005f90 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:16.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.690+0000 7f8422000640 1 -- 172.21.15.69:0/3074690135 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8418049030 con 0x7f8424104270 2024-04-24T09:08:16.735 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:16 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:08:16.735 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:16 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:08:16.736 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:16 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:08:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.933+0000 7f842b000640 1 -- 172.21.15.69:0/3074690135 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8404005580 con 0x7f8424104270 2024-04-24T09:08:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.935+0000 7f8422000640 1 -- 172.21.15.69:0/3074690135 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11873 (secure 0 0 0) 0x7f84180bd9a0 con 0x7f8424104270 2024-04-24T09:08:16.935 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:08:16.935 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:04:01.991559+0000","modified":"2024-04-24T09:07:02.537848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34532,"mds_1":34496,"mds_2":34460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34496":{"gid":34496,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34532":{"gid":34532,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34532,"qdb_cluster":[34532,34460,34496]},"id":1}]} 2024-04-24T09:08:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 24 2024-04-24T09:08:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.938+0000 7f842b000640 1 -- 172.21.15.69:0/3074690135 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f840c0a3e40 msgr2=0x7f840c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.938+0000 7f842b000640 1 --2- 172.21.15.69:0/3074690135 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f840c0a3e40 0x7f840c0a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f842419bf00 tx=0x7f8410005f90 comp rx=0 tx=0).stop 2024-04-24T09:08:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.938+0000 7f842b000640 1 -- 172.21.15.69:0/3074690135 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8424104270 msgr2=0x7f84241a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.938+0000 7f842b000640 1 --2- 172.21.15.69:0/3074690135 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8424104270 0x7f84241a05f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8418005ab0 tx=0x7f8418003c80 comp rx=0 tx=0).stop 2024-04-24T09:08:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.938+0000 7f842b000640 1 -- 172.21.15.69:0/3074690135 shutdown_connections 2024-04-24T09:08:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.938+0000 7f842b000640 1 --2- 172.21.15.69:0/3074690135 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f840c0a3e40 0x7f840c0a6300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.938+0000 7f842b000640 1 --2- 172.21.15.69:0/3074690135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f842419a6b0 0x7f842419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.938+0000 7f842b000640 1 --2- 172.21.15.69:0/3074690135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8424109340 0x7f84241a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.938+0000 7f842b000640 1 --2- 172.21.15.69:0/3074690135 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8424104270 0x7f84241a05f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.938+0000 7f842b000640 1 -- 172.21.15.69:0/3074690135 >> 172.21.15.69:0/3074690135 conn(0x7f84240ffad0 msgr2=0x7f8424100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.938+0000 7f842b000640 1 -- 172.21.15.69:0/3074690135 shutdown_connections 2024-04-24T09:08:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:16.938+0000 7f842b000640 1 -- 172.21.15.69:0/3074690135 wait complete. 2024-04-24T09:08:16.962 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:08:17.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:16 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:08:17.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:16 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:08:17.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:16 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:08:17.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.138+0000 7fc5f3200640 1 -- 172.21.15.69:0/2608785963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f410c480 msgr2=0x7fc5f410e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.138+0000 7fc5f3200640 1 --2- 172.21.15.69:0/2608785963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f410c480 0x7fc5f410e920 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009ca0 tx=0x7fc5ec02a950 comp rx=0 tx=0).stop 2024-04-24T09:08:17.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.138+0000 7fc5f3200640 1 -- 172.21.15.69:0/2608785963 shutdown_connections 2024-04-24T09:08:17.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.138+0000 7fc5f3200640 1 --2- 172.21.15.69:0/2608785963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f410c480 0x7fc5f410e920 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.138+0000 7fc5f3200640 1 --2- 172.21.15.69:0/2608785963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5f4103980 0x7fc5f410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.138+0000 7fc5f3200640 1 --2- 172.21.15.69:0/2608785963 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5f4103060 0x7fc5f4103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.138+0000 7fc5f3200640 1 -- 172.21.15.69:0/2608785963 >> 172.21.15.69:0/2608785963 conn(0x7fc5f40fc880 msgr2=0x7fc5f40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.138+0000 7fc5f3200640 1 -- 172.21.15.69:0/2608785963 shutdown_connections 2024-04-24T09:08:17.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.138+0000 7fc5f3200640 1 -- 172.21.15.69:0/2608785963 wait complete. 2024-04-24T09:08:17.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.139+0000 7fc5f3200640 1 Processor -- start 2024-04-24T09:08:17.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.140+0000 7fc5f3200640 1 -- start start 2024-04-24T09:08:17.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.140+0000 7fc5f3200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5f4103060 0x7fc5f4100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.140+0000 7fc5f3200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5f4103980 0x7fc5f41011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.140+0000 7fc5f3200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f410c480 0x7fc5f4101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.140+0000 7fc5f3200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f4112160 con 0x7fc5f410c480 2024-04-24T09:08:17.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.140+0000 7fc5f3200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f4111fe0 con 0x7fc5f4103060 2024-04-24T09:08:17.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.140+0000 7fc5f3200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f41122e0 con 0x7fc5f4103980 2024-04-24T09:08:17.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.140+0000 7fc5f2800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f410c480 0x7fc5f4101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.141+0000 7fc5f2800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f410c480 0x7fc5f4101730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55330/0 (socket says 172.21.15.69:55330) 2024-04-24T09:08:17.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.141+0000 7fc5f2800640 1 -- 172.21.15.69:0/3862971952 learned_addr learned my addr 172.21.15.69:0/3862971952 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:17.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.141+0000 7fc5f1e00640 1 --2- 172.21.15.69:0/3862971952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5f4103060 0x7fc5f4100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.141+0000 7fc5f2800640 1 -- 172.21.15.69:0/3862971952 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5f4103980 msgr2=0x7fc5f41011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.141+0000 7fc5f1400640 1 --2- 172.21.15.69:0/3862971952 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5f4103980 0x7fc5f41011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.141+0000 7fc5f2800640 1 --2- 172.21.15.69:0/3862971952 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5f4103980 0x7fc5f41011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.141+0000 7fc5f2800640 1 -- 172.21.15.69:0/3862971952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5f4103060 msgr2=0x7fc5f4100cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.141+0000 7fc5f2800640 1 --2- 172.21.15.69:0/3862971952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5f4103060 0x7fc5f4100cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.141+0000 7fc5f2800640 1 -- 172.21.15.69:0/3862971952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f41a6ac0 con 0x7fc5f410c480 2024-04-24T09:08:17.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.141+0000 7fc5f1400640 1 --2- 172.21.15.69:0/3862971952 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5f4103980 0x7fc5f41011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:17.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.142+0000 7fc5f2800640 1 --2- 172.21.15.69:0/3862971952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f410c480 0x7fc5f4101730 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009dc0 tx=0x7fc5ec003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.142+0000 7fc5eaa00640 1 -- 172.21.15.69:0/3862971952 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec034040 con 0x7fc5f410c480 2024-04-24T09:08:17.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.142+0000 7fc5eaa00640 1 -- 172.21.15.69:0/3862971952 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5ec030850 con 0x7fc5f410c480 2024-04-24T09:08:17.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.142+0000 7fc5f3200640 1 -- 172.21.15.69:0/3862971952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f41a6d50 con 0x7fc5f410c480 2024-04-24T09:08:17.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.142+0000 7fc5eaa00640 1 -- 172.21.15.69:0/3862971952 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0047f0 con 0x7fc5f410c480 2024-04-24T09:08:17.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.142+0000 7fc5f3200640 1 -- 172.21.15.69:0/3862971952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f41a7240 con 0x7fc5f410c480 2024-04-24T09:08:17.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.146+0000 7fc5eaa00640 1 -- 172.21.15.69:0/3862971952 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fc5ec037020 con 0x7fc5f410c480 2024-04-24T09:08:17.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.146+0000 7fc5fbfce740 1 Processor -- start 2024-04-24T09:08:17.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.147+0000 7fc5fbfce740 1 -- start start 2024-04-24T09:08:17.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.147+0000 7fc5fbfce740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a3f350 0x55a428a3f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.147+0000 7fc5fbfce740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 0x55a428a49e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.147+0000 7fc5fbfce740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a4a340 0x55a428a4c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.147+0000 7fc5fbfce740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a42899c1d0 con 0x55a428a4a340 2024-04-24T09:08:17.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.147+0000 7fc5fbfce740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a428942450 con 0x55a428a3fc70 2024-04-24T09:08:17.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.147+0000 7fc5fbfce740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55a428998190 con 0x55a428a3f350 2024-04-24T09:08:17.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.148+0000 7fc5f1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a4a340 0x55a428a4c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.148+0000 7fc5f1400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a3f350 0x55a428a3f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.148+0000 7fc5f1400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a3f350 0x55a428a3f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.69:45118/0 (socket says 172.21.15.69:45118) 2024-04-24T09:08:17.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.148+0000 7fc5f1e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 0x55a428a49e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.148+0000 7fc5f1400640 1 -- 172.21.15.69:0/3539023127 learned_addr learned my addr 172.21.15.69:0/3539023127 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:17.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.148+0000 7fc5eaa00640 1 --2- 172.21.15.69:0/3862971952 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc5d80a3e70 0x7fc5d80a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.149+0000 7fc5f1400640 1 -- 172.21.15.69:0/3539023127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a3f350 msgr2=0x55a428a3f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.149+0000 7fc5f1400640 1 --2- 172.21.15.69:0/3539023127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a3f350 0x55a428a3f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.149+0000 7fc5f1400640 1 -- 172.21.15.69:0/3539023127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 msgr2=0x55a428a49e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.149+0000 7fc5f1400640 1 --2- 172.21.15.69:0/3539023127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 0x55a428a49e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.149+0000 7fc5eaa00640 1 -- 172.21.15.69:0/3862971952 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc5ec0f1d20 con 0x7fc5f410c480 2024-04-24T09:08:17.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.149+0000 7fc5f1400640 1 -- 172.21.15.69:0/3539023127 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4289ff310 con 0x55a428a4a340 2024-04-24T09:08:17.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.149+0000 7fc5f2800640 1 --2- 172.21.15.69:0/3862971952 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc5d80a3e70 0x7fc5d80a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.149+0000 7fc5f1400640 1 --2- 172.21.15.69:0/3539023127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a4a340 0x55a428a4c730 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0013910 tx=0x7fc5e0005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec1e688fd0929d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.150+0000 7fc5df400640 1 -- 172.21.15.69:0/3539023127 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e000b860 con 0x55a428a4a340 2024-04-24T09:08:17.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.150+0000 7fc5df400640 1 -- 172.21.15.69:0/3539023127 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5e000f3f0 con 0x55a428a4a340 2024-04-24T09:08:17.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.150+0000 7fc5df400640 1 -- 172.21.15.69:0/3539023127 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0011640 con 0x55a428a4a340 2024-04-24T09:08:17.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.150+0000 7fc5fbfce740 1 -- 172.21.15.69:0/3539023127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a4a340 msgr2=0x55a428a4c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.150+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/3539023127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a4a340 0x55a428a4c730 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0013910 tx=0x7fc5e0005ee0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.150+0000 7fc5f2800640 1 --2- 172.21.15.69:0/3862971952 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc5d80a3e70 0x7fc5d80a6330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0ef0c0 tx=0x7fc5ec02a950 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.151+0000 7fc5fbfce740 1 -- 172.21.15.69:0/3539023127 shutdown_connections 2024-04-24T09:08:17.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.151+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/3539023127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a4a340 0x55a428a4c730 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.151+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/3539023127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 0x55a428a49e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.151+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/3539023127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a3f350 0x55a428a3f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.151+0000 7fc5fbfce740 1 -- 172.21.15.69:0/3539023127 >> 172.21.15.69:0/3539023127 conn(0x55a428a3b670 msgr2=0x55a428a3ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.151+0000 7fc5fbfce740 1 -- 172.21.15.69:0/3539023127 shutdown_connections 2024-04-24T09:08:17.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.151+0000 7fc5fbfce740 1 -- 172.21.15.69:0/3539023127 wait complete. 2024-04-24T09:08:17.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.152+0000 7fc5fbfce740 1 -- start start 2024-04-24T09:08:17.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.152+0000 7fc5fbfce740 1 Processor -- start 2024-04-24T09:08:17.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.153+0000 7fc5fbfce740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a3f350 0x55a428a47100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.153+0000 7fc5fbfce740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 0x55a428a47640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.153+0000 7fc5fbfce740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a4a340 0x55a428a47b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.153+0000 7fc5fbfce740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a428a4d1d0 con 0x55a428a3f350 2024-04-24T09:08:17.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.153+0000 7fc5fbfce740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a428955810 con 0x55a428a3fc70 2024-04-24T09:08:17.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.153+0000 7fc5fbfce740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55a428a4d350 con 0x55a428a4a340 2024-04-24T09:08:17.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.153+0000 7fc5fbfce740 10 client.0 _ll_register_callbacks cb 0x55a428923140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.153+0000 7fc5f1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a3f350 0x55a428a47100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.153+0000 7fc5f1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a3f350 0x55a428a47100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55354/0 (socket says 172.21.15.69:55354) 2024-04-24T09:08:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.153+0000 7fc5f1400640 1 -- 172.21.15.69:0/1859061625 learned_addr learned my addr 172.21.15.69:0/1859061625 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.154+0000 7fc5f1e00640 1 --2- 172.21.15.69:0/1859061625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 0x55a428a47640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.154+0000 7fc5f1400640 1 --2- 172.21.15.69:0/1859061625 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a4a340 0x55a428a47b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.154+0000 7fc5f1400640 1 -- 172.21.15.69:0/1859061625 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a4a340 msgr2=0x55a428a47b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.154+0000 7fc5f1400640 1 --2- 172.21.15.69:0/1859061625 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a4a340 0x55a428a47b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.154+0000 7fc5f1400640 1 -- 172.21.15.69:0/1859061625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 msgr2=0x55a428a47640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.154+0000 7fc5f1400640 1 --2- 172.21.15.69:0/1859061625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 0x55a428a47640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.154+0000 7fc5f1400640 1 -- 172.21.15.69:0/1859061625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4289663f0 con 0x55a428a3f350 2024-04-24T09:08:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.154+0000 7fc5f1e00640 1 --2- 172.21.15.69:0/1859061625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 0x55a428a47640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.154+0000 7fc5f1400640 1 --2- 172.21.15.69:0/1859061625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a3f350 0x55a428a47100 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000dca0 tx=0x7fc5e000d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa783d3dee2ffff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.154+0000 7fc5d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:17.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.155+0000 7fc5d3e00640 1 -- 172.21.15.69:0/1859061625 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0004220 con 0x55a428a3f350 2024-04-24T09:08:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.155+0000 7fc5d3e00640 1 -- 172.21.15.69:0/1859061625 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc5e00043c0 con 0x55a428a3f350 2024-04-24T09:08:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.155+0000 7fc5fbfce740 1 -- 172.21.15.69:0/1859061625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5e000f3f0 con 0x55a428a3f350 2024-04-24T09:08:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.155+0000 7fc5d3e00640 1 -- 172.21.15.69:0/1859061625 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc5e000f9f0 con 0x55a428a3f350 2024-04-24T09:08:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.155+0000 7fc5fbfce740 10 client.15948 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:17.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.155+0000 7fc5fbfce740 1 -- 172.21.15.69:0/1859061625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a4286ae3f0 con 0x55a428a3f350 2024-04-24T09:08:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.155+0000 7fc5d3e00640 1 -- 172.21.15.69:0/1859061625 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5e000f3f0 con 0x55a428a3f350 2024-04-24T09:08:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.156+0000 7fc5d3e00640 1 -- 172.21.15.69:0/1859061625 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc5e001d070 con 0x55a428a3f350 2024-04-24T09:08:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.156+0000 7fc5fbfce740 10 client.15948 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.156+0000 7fc5fbfce740 10 client.15948 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:08:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.156+0000 7fc5fbfce740 20 client.15948 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.156+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/1859061625 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55a428a5a450 0x55a428a5c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.156+0000 7fc5fbfce740 4 client.15948 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.156+0000 7fc5fbfce740 1 -- 172.21.15.69:0/1859061625 --> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4289a9bf0 con 0x55a428a5a450 2024-04-24T09:08:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.156+0000 7fc5f1e00640 1 --2- 172.21.15.69:0/1859061625 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55a428a5a450 0x55a428a5c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.157+0000 7fc5f1e00640 1 --2- 172.21.15.69:0/1859061625 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55a428a5a450 0x55a428a5c840 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7726b65cc0841059 server_cookie=9fce82605e6b80dc in_seq=0 out_seq=0 2024-04-24T09:08:17.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.157+0000 7fc5d3e00640 10 client.15948 ms_handle_connect on v2:172.21.15.81:6834/3260829578 2024-04-24T09:08:17.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.159+0000 7fc5d3e00640 1 -- 172.21.15.69:0/1859061625 <== mds.2 v2:172.21.15.81:6834/3260829578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4289a9bf0 con 0x55a428a5a450 2024-04-24T09:08:17.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.159+0000 7fc5d3e00640 10 client.15948 handle_command_reply: tid=0 2024-04-24T09:08:17.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.159+0000 7fc5fbfce740 1 client.15948 shutdown 2024-04-24T09:08:17.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.160+0000 7fc5fbfce740 1 -- 172.21.15.69:0/1859061625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a3f350 msgr2=0x55a428a47100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.160+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/1859061625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a3f350 0x55a428a47100 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000dca0 tx=0x7fc5e000d370 comp rx=0 tx=0).stop 2024-04-24T09:08:17.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.160+0000 7fc5fbfce740 1 -- 172.21.15.69:0/1859061625 shutdown_connections 2024-04-24T09:08:17.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.160+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/1859061625 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55a428a5a450 0x55a428a5c840 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.160+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/1859061625 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a4a340 0x55a428a47b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.160+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/1859061625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 0x55a428a47640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.160+0000 7fc5d3e00640 0 client.15948 ms_handle_reset on v2:172.21.15.81:6834/3260829578 2024-04-24T09:08:17.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.160+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/1859061625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a3f350 0x55a428a47100 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.160+0000 7fc5fbfce740 1 -- 172.21.15.69:0/1859061625 >> 172.21.15.69:0/1859061625 conn(0x55a428a3b670 msgr2=0x55a428a41470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.160+0000 7fc5fbfce740 1 -- 172.21.15.69:0/1859061625 shutdown_connections 2024-04-24T09:08:17.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.160+0000 7fc5fbfce740 1 -- 172.21.15.69:0/1859061625 wait complete. 2024-04-24T09:08:17.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.160+0000 7fc5fbfce740 20 client.15948 trim_cache size 0 max 16384 2024-04-24T09:08:17.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.187+0000 7fc5fbfce740 1 Processor -- start 2024-04-24T09:08:17.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.187+0000 7fc5fbfce740 1 -- start start 2024-04-24T09:08:17.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.187+0000 7fc5fbfce740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 0x55a428a560c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.187+0000 7fc5fbfce740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a4a340 0x55a428a46370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.188+0000 7fc5fbfce740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a41ba0 0x55a428a61e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.188+0000 7fc5fbfce740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f41af6c0 con 0x55a428a4a340 2024-04-24T09:08:17.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.188+0000 7fc5fbfce740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f41acf90 con 0x55a428a3fc70 2024-04-24T09:08:17.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.188+0000 7fc5fbfce740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f41aade0 con 0x55a428a41ba0 2024-04-24T09:08:17.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.188+0000 7fc5f1e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a4a340 0x55a428a46370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.188+0000 7fc5f1e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a4a340 0x55a428a46370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55368/0 (socket says 172.21.15.69:55368) 2024-04-24T09:08:17.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.188+0000 7fc5f1e00640 1 -- 172.21.15.69:0/1347194661 learned_addr learned my addr 172.21.15.69:0/1347194661 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:17.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.188+0000 7fc5f1400640 1 --2- 172.21.15.69:0/1347194661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 0x55a428a560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.188+0000 7fc5f1400640 1 --2- 172.21.15.69:0/1347194661 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a41ba0 0x55a428a61e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.188+0000 7fc5f1e00640 1 -- 172.21.15.69:0/1347194661 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a41ba0 msgr2=0x55a428a61e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.188+0000 7fc5f1e00640 1 --2- 172.21.15.69:0/1347194661 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a41ba0 0x55a428a61e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.188+0000 7fc5f1e00640 1 -- 172.21.15.69:0/1347194661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 msgr2=0x55a428a560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.188+0000 7fc5f1e00640 1 --2- 172.21.15.69:0/1347194661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 0x55a428a560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.188+0000 7fc5f1e00640 1 -- 172.21.15.69:0/1347194661 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a428a3faa0 con 0x55a428a4a340 2024-04-24T09:08:17.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.188+0000 7fc5f1400640 1 --2- 172.21.15.69:0/1347194661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 0x55a428a560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:17.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.189+0000 7fc5f1e00640 1 --2- 172.21.15.69:0/1347194661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a4a340 0x55a428a46370 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55a428a46960 tx=0x7fc5e40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e23055621dc35aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.189+0000 7fc5dea00640 1 -- 172.21.15.69:0/1347194661 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4021030 con 0x55a428a4a340 2024-04-24T09:08:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.189+0000 7fc5dea00640 1 -- 172.21.15.69:0/1347194661 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5e401d8d0 con 0x55a428a4a340 2024-04-24T09:08:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.189+0000 7fc5dea00640 1 -- 172.21.15.69:0/1347194661 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e40047a0 con 0x55a428a4a340 2024-04-24T09:08:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.190+0000 7fc5fbfce740 1 -- 172.21.15.69:0/1347194661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a4a340 msgr2=0x55a428a46370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.190+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/1347194661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a4a340 0x55a428a46370 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55a428a46960 tx=0x7fc5e40039c0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.190+0000 7fc5fbfce740 1 -- 172.21.15.69:0/1347194661 shutdown_connections 2024-04-24T09:08:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.190+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/1347194661 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a41ba0 0x55a428a61e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.190+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/1347194661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a4a340 0x55a428a46370 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.190+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/1347194661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a3fc70 0x55a428a560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.190+0000 7fc5fbfce740 1 -- 172.21.15.69:0/1347194661 >> 172.21.15.69:0/1347194661 conn(0x55a428a3b670 msgr2=0x55a428a31f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.190+0000 7fc5fbfce740 1 -- 172.21.15.69:0/1347194661 shutdown_connections 2024-04-24T09:08:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.190+0000 7fc5fbfce740 1 -- 172.21.15.69:0/1347194661 wait complete. 2024-04-24T09:08:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.191+0000 7fc5fbfce740 1 -- start start 2024-04-24T09:08:17.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.191+0000 7fc5fbfce740 1 Processor -- start 2024-04-24T09:08:17.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.192+0000 7fc5fbfce740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a3fc70 0x55a428a4b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.192+0000 7fc5fbfce740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a41ba0 0x55a428a4b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.192+0000 7fc5fbfce740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a4a340 0x55a428a50ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.192+0000 7fc5fbfce740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f41ab450 con 0x55a428a3fc70 2024-04-24T09:08:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.192+0000 7fc5fbfce740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f41ac8c0 con 0x55a428a41ba0 2024-04-24T09:08:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.192+0000 7fc5fbfce740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f41abb20 con 0x55a428a4a340 2024-04-24T09:08:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.192+0000 7fc5fbfce740 10 client.0 _ll_register_callbacks cb 0x7fc5e4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.192+0000 7fc5f1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a3fc70 0x55a428a4b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.192+0000 7fc5f1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a3fc70 0x55a428a4b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55382/0 (socket says 172.21.15.69:55382) 2024-04-24T09:08:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.192+0000 7fc5f1400640 1 -- 172.21.15.69:0/2804310970 learned_addr learned my addr 172.21.15.69:0/2804310970 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.192+0000 7fc5f1e00640 1 --2- 172.21.15.69:0/2804310970 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a41ba0 0x55a428a4b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.192+0000 7fc5f1400640 1 -- 172.21.15.69:0/2804310970 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a4a340 msgr2=0x55a428a50ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.192+0000 7fc5f1400640 1 --2- 172.21.15.69:0/2804310970 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a4a340 0x55a428a50ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.192+0000 7fc5f1400640 1 -- 172.21.15.69:0/2804310970 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a41ba0 msgr2=0x55a428a4b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.192+0000 7fc5f1400640 1 --2- 172.21.15.69:0/2804310970 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a41ba0 0x55a428a4b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.192+0000 7fc5f1400640 1 -- 172.21.15.69:0/2804310970 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4287787d0 con 0x55a428a3fc70 2024-04-24T09:08:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.193+0000 7fc5f1400640 1 --2- 172.21.15.69:0/2804310970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a3fc70 0x55a428a4b450 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x55a428a48210 tx=0x7fc5e000d6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7518ff5ca3e7e698 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.193+0000 7fc5d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.193+0000 7fc5d3e00640 1 -- 172.21.15.69:0/2804310970 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e000f3f0 con 0x55a428a3fc70 2024-04-24T09:08:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.193+0000 7fc5d3e00640 1 -- 172.21.15.69:0/2804310970 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc5e0004890 con 0x55a428a3fc70 2024-04-24T09:08:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.193+0000 7fc5fbfce740 1 -- 172.21.15.69:0/2804310970 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5e401d8d0 con 0x55a428a3fc70 2024-04-24T09:08:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.193+0000 7fc5d3e00640 1 -- 172.21.15.69:0/2804310970 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc5e0043050 con 0x55a428a3fc70 2024-04-24T09:08:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.193+0000 7fc5fbfce740 10 client.15954 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.193+0000 7fc5fbfce740 1 -- 172.21.15.69:0/2804310970 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a428ac0a50 con 0x55a428a3fc70 2024-04-24T09:08:17.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.194+0000 7fc5d3e00640 1 -- 172.21.15.69:0/2804310970 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5e401d8d0 con 0x55a428a3fc70 2024-04-24T09:08:17.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.194+0000 7fc5d3e00640 1 -- 172.21.15.69:0/2804310970 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc5e001d040 con 0x55a428a3fc70 2024-04-24T09:08:17.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.194+0000 7fc5fbfce740 10 client.15954 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:17.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.194+0000 7fc5fbfce740 10 client.15954 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:08:17.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.194+0000 7fc5fbfce740 20 client.15954 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:17.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.194+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/2804310970 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55a428ac1170 0x55a428ac3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.194+0000 7fc5fbfce740 4 client.15954 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:17.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.194+0000 7fc5fbfce740 1 -- 172.21.15.69:0/2804310970 --> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a4289a3a60 con 0x55a428ac1170 2024-04-24T09:08:17.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.195+0000 7fc5f1e00640 1 --2- 172.21.15.69:0/2804310970 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55a428ac1170 0x55a428ac3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.196+0000 7fc5f1e00640 1 --2- 172.21.15.69:0/2804310970 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55a428ac1170 0x55a428ac3590 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9cabd732b8cdb9fe server_cookie=63e3e997806bbb00 in_seq=0 out_seq=0 2024-04-24T09:08:17.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.196+0000 7fc5d3e00640 10 client.15954 ms_handle_connect on v2:172.21.15.81:6834/3260829578 2024-04-24T09:08:17.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.196+0000 7fc5d3e00640 1 -- 172.21.15.69:0/2804310970 <== mds.2 v2:172.21.15.81:6834/3260829578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a4289a3a60 con 0x55a428ac1170 2024-04-24T09:08:17.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.196+0000 7fc5d3e00640 10 client.15954 handle_command_reply: tid=0 2024-04-24T09:08:17.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.196+0000 7fc5fbfce740 1 client.15954 shutdown 2024-04-24T09:08:17.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.196+0000 7fc5fbfce740 1 -- 172.21.15.69:0/2804310970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a3fc70 msgr2=0x55a428a4b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.197+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/2804310970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a3fc70 0x55a428a4b450 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x55a428a48210 tx=0x7fc5e000d6c0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.197+0000 7fc5fbfce740 1 -- 172.21.15.69:0/2804310970 shutdown_connections 2024-04-24T09:08:17.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.197+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/2804310970 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55a428ac1170 0x55a428ac3590 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.197+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/2804310970 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a428a4a340 0x55a428a50ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.197+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/2804310970 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a428a41ba0 0x55a428a4b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.197+0000 7fc5fbfce740 1 --2- 172.21.15.69:0/2804310970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a428a3fc70 0x55a428a4b450 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.197+0000 7fc5fbfce740 1 -- 172.21.15.69:0/2804310970 >> 172.21.15.69:0/2804310970 conn(0x55a428a3b670 msgr2=0x55a428a40aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.197+0000 7fc5d3e00640 0 client.15954 ms_handle_reset on v2:172.21.15.81:6834/3260829578 2024-04-24T09:08:17.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.197+0000 7fc5fbfce740 1 -- 172.21.15.69:0/2804310970 shutdown_connections 2024-04-24T09:08:17.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.197+0000 7fc5fbfce740 1 -- 172.21.15.69:0/2804310970 wait complete. 2024-04-24T09:08:17.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.197+0000 7fc5fbfce740 20 client.15954 trim_cache size 0 max 16384 2024-04-24T09:08:17.199 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-04-24T09:08:17.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.198+0000 7fc5f3200640 1 -- 172.21.15.69:0/3862971952 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc5d80a3e70 msgr2=0x7fc5d80a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.198+0000 7fc5f3200640 1 --2- 172.21.15.69:0/3862971952 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc5d80a3e70 0x7fc5d80a6330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0ef0c0 tx=0x7fc5ec02a950 comp rx=0 tx=0).stop 2024-04-24T09:08:17.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.198+0000 7fc5f3200640 1 -- 172.21.15.69:0/3862971952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f410c480 msgr2=0x7fc5f4101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.198+0000 7fc5f3200640 1 --2- 172.21.15.69:0/3862971952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f410c480 0x7fc5f4101730 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009dc0 tx=0x7fc5ec003f10 comp rx=0 tx=0).stop 2024-04-24T09:08:17.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.198+0000 7fc5f3200640 1 -- 172.21.15.69:0/3862971952 shutdown_connections 2024-04-24T09:08:17.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.198+0000 7fc5f3200640 1 --2- 172.21.15.69:0/3862971952 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc5d80a3e70 0x7fc5d80a6330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.198+0000 7fc5f3200640 1 --2- 172.21.15.69:0/3862971952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f410c480 0x7fc5f4101730 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.198+0000 7fc5f3200640 1 --2- 172.21.15.69:0/3862971952 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5f4103980 0x7fc5f41011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.198+0000 7fc5f3200640 1 --2- 172.21.15.69:0/3862971952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc5f4103060 0x7fc5f4100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.198+0000 7fc5f3200640 1 -- 172.21.15.69:0/3862971952 >> 172.21.15.69:0/3862971952 conn(0x7fc5f40fc880 msgr2=0x7fc5f40fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.198+0000 7fc5f3200640 1 -- 172.21.15.69:0/3862971952 shutdown_connections 2024-04-24T09:08:17.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.198+0000 7fc5f3200640 1 -- 172.21.15.69:0/3862971952 wait complete. 2024-04-24T09:08:17.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:16 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:08:17.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:16 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:08:17.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:16 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:08:17.215 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:08:17.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.383+0000 7f20b3200640 1 -- 172.21.15.69:0/110246871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20b4100e50 msgr2=0x7f20b4101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.383+0000 7f20b3200640 1 --2- 172.21.15.69:0/110246871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20b4100e50 0x7f20b4101230 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009900 tx=0x7f20a402a470 comp rx=0 tx=0).stop 2024-04-24T09:08:17.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.383+0000 7f20b3200640 1 -- 172.21.15.69:0/110246871 shutdown_connections 2024-04-24T09:08:17.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.383+0000 7f20b3200640 1 --2- 172.21.15.69:0/110246871 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20b4106250 0x7f20b41066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.383+0000 7f20b3200640 1 --2- 172.21.15.69:0/110246871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20b4101770 0x7f20b4105c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.383+0000 7f20b3200640 1 --2- 172.21.15.69:0/110246871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20b4100e50 0x7f20b4101230 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.383+0000 7f20b3200640 1 -- 172.21.15.69:0/110246871 >> 172.21.15.69:0/110246871 conn(0x7f20b40fc880 msgr2=0x7f20b40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.383+0000 7f20b3200640 1 -- 172.21.15.69:0/110246871 shutdown_connections 2024-04-24T09:08:17.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.383+0000 7f20b3200640 1 -- 172.21.15.69:0/110246871 wait complete. 2024-04-24T09:08:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.384+0000 7f20b3200640 1 Processor -- start 2024-04-24T09:08:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.384+0000 7f20b3200640 1 -- start start 2024-04-24T09:08:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.385+0000 7f20b3200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20b4100e50 0x7f20b41a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.385+0000 7f20b3200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20b4101770 0x7f20b41a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.385+0000 7f20b3200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20b4106250 0x7f20b419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.385+0000 7f20b3200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4118770 con 0x7f20b4101770 2024-04-24T09:08:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.385+0000 7f20b3200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41185f0 con 0x7f20b4100e50 2024-04-24T09:08:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.385+0000 7f20b3200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41188f0 con 0x7f20b4106250 2024-04-24T09:08:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.385+0000 7f20b1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20b4101770 0x7f20b41a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.385+0000 7f20b1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20b4101770 0x7f20b41a4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55398/0 (socket says 172.21.15.69:55398) 2024-04-24T09:08:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.385+0000 7f20b1e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20b4100e50 0x7f20b41a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.386+0000 7f20b1400640 1 -- 172.21.15.69:0/1561959226 learned_addr learned my addr 172.21.15.69:0/1561959226 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.386+0000 7f20b2800640 1 --2- 172.21.15.69:0/1561959226 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20b4106250 0x7f20b419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.386+0000 7f20b1400640 1 -- 172.21.15.69:0/1561959226 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20b4106250 msgr2=0x7f20b419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.386+0000 7f20b1400640 1 --2- 172.21.15.69:0/1561959226 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20b4106250 0x7f20b419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.386+0000 7f20b1400640 1 -- 172.21.15.69:0/1561959226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20b4100e50 msgr2=0x7f20b41a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.386+0000 7f20b1400640 1 --2- 172.21.15.69:0/1561959226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20b4100e50 0x7f20b41a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.386+0000 7f20b1400640 1 -- 172.21.15.69:0/1561959226 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b419f3d0 con 0x7f20b4101770 2024-04-24T09:08:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.387+0000 7f20b1400640 1 --2- 172.21.15.69:0/1561959226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20b4101770 0x7f20b41a4f90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f20a000d740 tx=0x7f20a000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.387+0000 7f20aaa00640 1 -- 172.21.15.69:0/1561959226 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0017070 con 0x7f20b4101770 2024-04-24T09:08:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.387+0000 7f20b3200640 1 -- 172.21.15.69:0/1561959226 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b419f6c0 con 0x7f20b4101770 2024-04-24T09:08:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.387+0000 7f20aaa00640 1 -- 172.21.15.69:0/1561959226 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20a000de50 con 0x7f20b4101770 2024-04-24T09:08:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.387+0000 7f20aaa00640 1 -- 172.21.15.69:0/1561959226 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a000f620 con 0x7f20b4101770 2024-04-24T09:08:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.387+0000 7f20b3200640 1 -- 172.21.15.69:0/1561959226 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b41ab9d0 con 0x7f20b4101770 2024-04-24T09:08:17.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.392+0000 7f20bbe37740 1 Processor -- start 2024-04-24T09:08:17.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.392+0000 7f20bbe37740 1 -- start start 2024-04-24T09:08:17.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.392+0000 7f20bbe37740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 0x55794234ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.392+0000 7f20bbe37740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55794234f460 0x5579423595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.392+0000 7f20bbe37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557942359b30 0x55794235bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.392+0000 7f20bbe37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5579422b41d0 con 0x557942359b30 2024-04-24T09:08:17.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.392+0000 7f20bbe37740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55794225a450 con 0x55794234f460 2024-04-24T09:08:17.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.392+0000 7f20bbe37740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5579422b0190 con 0x55794234eb40 2024-04-24T09:08:17.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.393+0000 7f20b2800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 0x55794234ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.393+0000 7f20b2800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 0x55794234ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.69:45186/0 (socket says 172.21.15.69:45186) 2024-04-24T09:08:17.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.393+0000 7f20b2800640 1 -- 172.21.15.69:0/1609116685 learned_addr learned my addr 172.21.15.69:0/1609116685 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:17.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.393+0000 7f20aaa00640 1 -- 172.21.15.69:0/1561959226 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f20a0009c30 con 0x7f20b4101770 2024-04-24T09:08:17.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.393+0000 7f20b1e00640 1 --2- 172.21.15.69:0/1609116685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55794234f460 0x5579423595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.394+0000 7f20b1e00640 1 -- 172.21.15.69:0/1609116685 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 msgr2=0x55794234ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.394+0000 7f20b1e00640 1 --2- 172.21.15.69:0/1609116685 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 0x55794234ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.394+0000 7f20b1e00640 1 -- 172.21.15.69:0/1609116685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557942359b30 msgr2=0x55794235bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.394+0000 7f20b1400640 1 --2- 172.21.15.69:0/1609116685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557942359b30 0x55794235bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.394+0000 7f20b1e00640 1 --2- 172.21.15.69:0/1609116685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557942359b30 0x55794235bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.394+0000 7f20b1e00640 1 -- 172.21.15.69:0/1609116685 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557942317310 con 0x55794234f460 2024-04-24T09:08:17.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.394+0000 7f20b1400640 1 --2- 172.21.15.69:0/1609116685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557942359b30 0x55794235bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:17.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.395+0000 7f20b1e00640 1 --2- 172.21.15.69:0/1609116685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55794234f460 0x5579423595f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20a402aa60 tx=0x7f20a4004190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e48fe537b5521 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.395+0000 7f20aaa00640 1 --2- 172.21.15.69:0/1561959226 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f20980a3ec0 0x7f20980a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.395+0000 7f209f400640 1 -- 172.21.15.69:0/1609116685 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4035040 con 0x55794234f460 2024-04-24T09:08:17.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.395+0000 7f20aaa00640 1 -- 172.21.15.69:0/1561959226 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f20a00d01f0 con 0x7f20b4101770 2024-04-24T09:08:17.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.396+0000 7f209f400640 1 -- 172.21.15.69:0/1609116685 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20a40054f0 con 0x55794234f460 2024-04-24T09:08:17.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.396+0000 7f209f400640 1 -- 172.21.15.69:0/1609116685 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4003850 con 0x55794234f460 2024-04-24T09:08:17.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.396+0000 7f20b2800640 1 --2- 172.21.15.69:0/1561959226 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f20980a3ec0 0x7f20980a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.397+0000 7f20bbe37740 1 -- 172.21.15.69:0/1609116685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55794234f460 msgr2=0x5579423595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.397+0000 7f20b2800640 1 --2- 172.21.15.69:0/1561959226 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f20980a3ec0 0x7f20980a6380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f20ac013570 tx=0x7f20ac009600 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.397+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1609116685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55794234f460 0x5579423595f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20a402aa60 tx=0x7f20a4004190 comp rx=0 tx=0).stop 2024-04-24T09:08:17.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.397+0000 7f20bbe37740 1 -- 172.21.15.69:0/1609116685 shutdown_connections 2024-04-24T09:08:17.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.397+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1609116685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557942359b30 0x55794235bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.397+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1609116685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55794234f460 0x5579423595f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.397+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1609116685 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 0x55794234ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.397+0000 7f20bbe37740 1 -- 172.21.15.69:0/1609116685 >> 172.21.15.69:0/1609116685 conn(0x55794234ae60 msgr2=0x55794234b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.397+0000 7f20bbe37740 1 -- 172.21.15.69:0/1609116685 shutdown_connections 2024-04-24T09:08:17.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.397+0000 7f20bbe37740 1 -- 172.21.15.69:0/1609116685 wait complete. 2024-04-24T09:08:17.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.397+0000 7f20bbe37740 1 -- start start 2024-04-24T09:08:17.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.398+0000 7f20bbe37740 1 Processor -- start 2024-04-24T09:08:17.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.398+0000 7f20bbe37740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 0x5579423595e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.398+0000 7f20bbe37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234f460 0x5579423568d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.398+0000 7f20bbe37740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557942359b30 0x557942357010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.398+0000 7f20bbe37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55794235c960 con 0x55794234f460 2024-04-24T09:08:17.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.398+0000 7f20bbe37740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55794226d810 con 0x557942359b30 2024-04-24T09:08:17.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.398+0000 7f20bbe37740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55794235cae0 con 0x55794234eb40 2024-04-24T09:08:17.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.398+0000 7f20bbe37740 10 client.0 _ll_register_callbacks cb 0x55794223b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:17.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.399+0000 7f20b1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234f460 0x5579423568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.399+0000 7f20b1e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 0x5579423595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.399+0000 7f20b1e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 0x5579423595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.69:45200/0 (socket says 172.21.15.69:45200) 2024-04-24T09:08:17.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.399+0000 7f20b1e00640 1 -- 172.21.15.69:0/1899983920 learned_addr learned my addr 172.21.15.69:0/1899983920 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:17.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.399+0000 7f20b2800640 1 --2- 172.21.15.69:0/1899983920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557942359b30 0x557942357010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.399+0000 7f20b1400640 1 -- 172.21.15.69:0/1899983920 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 msgr2=0x5579423595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.399+0000 7f20b1400640 1 --2- 172.21.15.69:0/1899983920 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 0x5579423595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.399+0000 7f20b1400640 1 -- 172.21.15.69:0/1899983920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557942359b30 msgr2=0x557942357010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.399+0000 7f20b1400640 1 --2- 172.21.15.69:0/1899983920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557942359b30 0x557942357010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.399+0000 7f20b1400640 1 -- 172.21.15.69:0/1899983920 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55794227e3f0 con 0x55794234f460 2024-04-24T09:08:17.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.399+0000 7f20b2800640 1 --2- 172.21.15.69:0/1899983920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557942359b30 0x557942357010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:17.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.400+0000 7f20b1400640 1 --2- 172.21.15.69:0/1899983920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234f460 0x5579423568d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f20a00cd420 tx=0x7f20a009bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3bd82866ed908 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.400+0000 7f2093e00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:17.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.400+0000 7f2093e00640 1 -- 172.21.15.69:0/1899983920 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0020070 con 0x55794234f460 2024-04-24T09:08:17.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.400+0000 7f2093e00640 1 -- 172.21.15.69:0/1899983920 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f20a00cd700 con 0x55794234f460 2024-04-24T09:08:17.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.400+0000 7f20bbe37740 1 -- 172.21.15.69:0/1899983920 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20a40054f0 con 0x55794234f460 2024-04-24T09:08:17.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.400+0000 7f2093e00640 1 -- 172.21.15.69:0/1899983920 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f20a00cd900 con 0x55794234f460 2024-04-24T09:08:17.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.401+0000 7f20bbe37740 10 client.15963 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:17.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.401+0000 7f20bbe37740 1 -- 172.21.15.69:0/1899983920 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557941fc63f0 con 0x55794234f460 2024-04-24T09:08:17.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.401+0000 7f2093e00640 1 -- 172.21.15.69:0/1899983920 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20a40054f0 con 0x55794234f460 2024-04-24T09:08:17.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.401+0000 7f2093e00640 1 -- 172.21.15.69:0/1899983920 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f20a009ab10 con 0x55794234f460 2024-04-24T09:08:17.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.401+0000 7f20bbe37740 10 client.15963 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:17.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.401+0000 7f20bbe37740 10 client.15963 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:08:17.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.401+0000 7f20bbe37740 20 client.15963 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:17.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.401+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1899983920 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x557942369c70 0x55794236c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.401+0000 7f20bbe37740 4 client.15963 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:17.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.401+0000 7f20bbe37740 1 -- 172.21.15.69:0/1899983920 --> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579422c1bf0 con 0x557942369c70 2024-04-24T09:08:17.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.402+0000 7f20b1e00640 1 --2- 172.21.15.69:0/1899983920 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x557942369c70 0x55794236c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.403+0000 7f20b1e00640 1 --2- 172.21.15.69:0/1899983920 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x557942369c70 0x55794236c060 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=996f9198f1c3ca28 server_cookie=bdec76a76714d2ec in_seq=0 out_seq=0 2024-04-24T09:08:17.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.403+0000 7f2093e00640 10 client.15963 ms_handle_connect on v2:172.21.15.81:6836/1973819593 2024-04-24T09:08:17.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.404+0000 7f2093e00640 1 -- 172.21.15.69:0/1899983920 <== mds.1 v2:172.21.15.81:6836/1973819593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579422c1bf0 con 0x557942369c70 2024-04-24T09:08:17.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.404+0000 7f2093e00640 10 client.15963 handle_command_reply: tid=0 2024-04-24T09:08:17.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.405+0000 7f20bbe37740 1 client.15963 shutdown 2024-04-24T09:08:17.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.405+0000 7f20bbe37740 1 -- 172.21.15.69:0/1899983920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234f460 msgr2=0x5579423568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.405+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1899983920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234f460 0x5579423568d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f20a00cd420 tx=0x7f20a009bc50 comp rx=0 tx=0).stop 2024-04-24T09:08:17.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.405+0000 7f20bbe37740 1 -- 172.21.15.69:0/1899983920 shutdown_connections 2024-04-24T09:08:17.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.405+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1899983920 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x557942369c70 0x55794236c060 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.405+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1899983920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557942359b30 0x557942357010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.405+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1899983920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234f460 0x5579423568d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.405+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1899983920 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 0x5579423595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.405+0000 7f20bbe37740 1 -- 172.21.15.69:0/1899983920 >> 172.21.15.69:0/1899983920 conn(0x55794234ae60 msgr2=0x557942350c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.405+0000 7f2093e00640 0 client.15963 ms_handle_reset on v2:172.21.15.81:6836/1973819593 2024-04-24T09:08:17.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.405+0000 7f20bbe37740 1 -- 172.21.15.69:0/1899983920 shutdown_connections 2024-04-24T09:08:17.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.405+0000 7f20bbe37740 1 -- 172.21.15.69:0/1899983920 wait complete. 2024-04-24T09:08:17.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.405+0000 7f20bbe37740 20 client.15963 trim_cache size 0 max 16384 2024-04-24T09:08:17.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.433+0000 7f20bbe37740 1 Processor -- start 2024-04-24T09:08:17.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.433+0000 7f20bbe37740 1 -- start start 2024-04-24T09:08:17.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.433+0000 7f20bbe37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234eb40 0x557942349dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.433+0000 7f20bbe37740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234f460 0x557942359110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.433+0000 7f20bbe37740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557942359b30 0x557942371910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.433+0000 7f20bbe37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41b8070 con 0x55794234eb40 2024-04-24T09:08:17.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.433+0000 7f20bbe37740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41b5940 con 0x557942359b30 2024-04-24T09:08:17.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.433+0000 7f20bbe37740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41b3790 con 0x55794234f460 2024-04-24T09:08:17.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.434+0000 7f20b1e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234eb40 0x557942349dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.434+0000 7f20b1e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234eb40 0x557942349dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55438/0 (socket says 172.21.15.69:55438) 2024-04-24T09:08:17.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.434+0000 7f20b1e00640 1 -- 172.21.15.69:0/1653919482 learned_addr learned my addr 172.21.15.69:0/1653919482 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:17.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.434+0000 7f20b1e00640 1 -- 172.21.15.69:0/1653919482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234f460 msgr2=0x557942359110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.434+0000 7f20b2800640 1 --2- 172.21.15.69:0/1653919482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557942359b30 0x557942371910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.434+0000 7f20b1400640 1 --2- 172.21.15.69:0/1653919482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234f460 0x557942359110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.434+0000 7f20b1e00640 1 --2- 172.21.15.69:0/1653919482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234f460 0x557942359110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.434+0000 7f20b1e00640 1 -- 172.21.15.69:0/1653919482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557942359b30 msgr2=0x557942371910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.434+0000 7f20b1e00640 1 --2- 172.21.15.69:0/1653919482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557942359b30 0x557942371910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.434+0000 7f20b1e00640 1 -- 172.21.15.69:0/1653919482 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557942357380 con 0x55794234eb40 2024-04-24T09:08:17.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.435+0000 7f20b1e00640 1 --2- 172.21.15.69:0/1653919482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234eb40 0x557942349dd0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f20a402aa60 tx=0x7f20a4004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6ca6949ca9848d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.435+0000 7f209ea00640 1 -- 172.21.15.69:0/1653919482 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4035040 con 0x55794234eb40 2024-04-24T09:08:17.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.435+0000 7f209ea00640 1 -- 172.21.15.69:0/1653919482 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20a40525d0 con 0x55794234eb40 2024-04-24T09:08:17.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.435+0000 7f209ea00640 1 -- 172.21.15.69:0/1653919482 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a40052d0 con 0x55794234eb40 2024-04-24T09:08:17.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.436+0000 7f20bbe37740 1 -- 172.21.15.69:0/1653919482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234eb40 msgr2=0x557942349dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.436+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1653919482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234eb40 0x557942349dd0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f20a402aa60 tx=0x7f20a4004cd0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.436+0000 7f20bbe37740 1 -- 172.21.15.69:0/1653919482 shutdown_connections 2024-04-24T09:08:17.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.436+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1653919482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557942359b30 0x557942371910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.436+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1653919482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234f460 0x557942359110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.436+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1653919482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234eb40 0x557942349dd0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.436+0000 7f20bbe37740 1 -- 172.21.15.69:0/1653919482 >> 172.21.15.69:0/1653919482 conn(0x55794234ae60 msgr2=0x5579423496c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.436+0000 7f20bbe37740 1 -- 172.21.15.69:0/1653919482 shutdown_connections 2024-04-24T09:08:17.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.437+0000 7f20bbe37740 1 -- 172.21.15.69:0/1653919482 wait complete. 2024-04-24T09:08:17.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.437+0000 7f20bbe37740 1 -- start start 2024-04-24T09:08:17.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.438+0000 7f20bbe37740 1 Processor -- start 2024-04-24T09:08:17.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.438+0000 7f20bbe37740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 0x557942361f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.438+0000 7f20bbe37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234f460 0x557942352b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.438+0000 7f20bbe37740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557942359b30 0x557942353080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.438+0000 7f20bbe37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41b3e00 con 0x55794234f460 2024-04-24T09:08:17.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.439+0000 7f20bbe37740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41b5270 con 0x557942359b30 2024-04-24T09:08:17.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.439+0000 7f20bbe37740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41b44d0 con 0x55794234eb40 2024-04-24T09:08:17.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.439+0000 7f20b1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234f460 0x557942352b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.439+0000 7f20bbe37740 10 client.0 _ll_register_callbacks cb 0x7f20a009b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:17.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.439+0000 7f20b1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234f460 0x557942352b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55452/0 (socket says 172.21.15.69:55452) 2024-04-24T09:08:17.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.439+0000 7f20b1400640 1 -- 172.21.15.69:0/1079707435 learned_addr learned my addr 172.21.15.69:0/1079707435 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:17.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.439+0000 7f20b2800640 1 --2- 172.21.15.69:0/1079707435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557942359b30 0x557942353080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.439+0000 7f20b1e00640 1 --2- 172.21.15.69:0/1079707435 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 0x557942361f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.439+0000 7f20b1400640 1 -- 172.21.15.69:0/1079707435 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 msgr2=0x557942361f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.439+0000 7f20b1400640 1 --2- 172.21.15.69:0/1079707435 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 0x557942361f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.439+0000 7f20b1400640 1 -- 172.21.15.69:0/1079707435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557942359b30 msgr2=0x557942353080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.439+0000 7f20b1400640 1 --2- 172.21.15.69:0/1079707435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557942359b30 0x557942353080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.439+0000 7f20b1400640 1 -- 172.21.15.69:0/1079707435 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579420907d0 con 0x55794234f460 2024-04-24T09:08:17.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.439+0000 7f20b1e00640 1 --2- 172.21.15.69:0/1079707435 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 0x557942361f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:17.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.440+0000 7f20b1400640 1 --2- 172.21.15.69:0/1079707435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234f460 0x557942352b40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f20a00a0110 tx=0x7f20a0010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9d5accaaf54064 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.440+0000 7f2093e00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:17.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.440+0000 7f2093e00640 1 -- 172.21.15.69:0/1079707435 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0020070 con 0x55794234f460 2024-04-24T09:08:17.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.440+0000 7f2093e00640 1 -- 172.21.15.69:0/1079707435 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f20a009b0a0 con 0x55794234f460 2024-04-24T09:08:17.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.440+0000 7f20bbe37740 1 -- 172.21.15.69:0/1079707435 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20a40525d0 con 0x55794234f460 2024-04-24T09:08:17.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.440+0000 7f2093e00640 1 -- 172.21.15.69:0/1079707435 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f20a009ba20 con 0x55794234f460 2024-04-24T09:08:17.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.441+0000 7f20bbe37740 10 client.15969 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:17.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.441+0000 7f20bbe37740 1 -- 172.21.15.69:0/1079707435 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557942354660 con 0x55794234f460 2024-04-24T09:08:17.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.441+0000 7f2093e00640 1 -- 172.21.15.69:0/1079707435 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20a40525d0 con 0x55794234f460 2024-04-24T09:08:17.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.441+0000 7f2093e00640 1 -- 172.21.15.69:0/1079707435 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f20a00a10e0 con 0x55794234f460 2024-04-24T09:08:17.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.441+0000 7f20bbe37740 10 client.15969 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:17.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.441+0000 7f20bbe37740 10 client.15969 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:08:17.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.441+0000 7f20bbe37740 20 client.15969 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:17.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.441+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1079707435 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5579423d12f0 0x5579423d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.441+0000 7f20bbe37740 4 client.15969 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:17.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.441+0000 7f20bbe37740 1 -- 172.21.15.69:0/1079707435 --> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579422bba60 con 0x5579423d12f0 2024-04-24T09:08:17.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.442+0000 7f20b1e00640 1 --2- 172.21.15.69:0/1079707435 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5579423d12f0 0x5579423d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.443+0000 7f2093e00640 10 client.15969 ms_handle_connect on v2:172.21.15.81:6836/1973819593 2024-04-24T09:08:17.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.443+0000 7f20b1e00640 1 --2- 172.21.15.69:0/1079707435 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5579423d12f0 0x5579423d36e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96b6cfbc9d27e820 server_cookie=914632e5ea58c90b in_seq=0 out_seq=0 2024-04-24T09:08:17.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.444+0000 7f2093e00640 1 -- 172.21.15.69:0/1079707435 <== mds.1 v2:172.21.15.81:6836/1973819593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579422bba60 con 0x5579423d12f0 2024-04-24T09:08:17.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.444+0000 7f2093e00640 10 client.15969 handle_command_reply: tid=0 2024-04-24T09:08:17.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.444+0000 7f20bbe37740 1 client.15969 shutdown 2024-04-24T09:08:17.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.444+0000 7f20bbe37740 1 -- 172.21.15.69:0/1079707435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234f460 msgr2=0x557942352b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.444+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1079707435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234f460 0x557942352b40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f20a00a0110 tx=0x7f20a0010c50 comp rx=0 tx=0).stop 2024-04-24T09:08:17.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.444+0000 7f20bbe37740 1 -- 172.21.15.69:0/1079707435 shutdown_connections 2024-04-24T09:08:17.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.444+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1079707435 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5579423d12f0 0x5579423d36e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.444+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1079707435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557942359b30 0x557942353080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.444+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1079707435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794234f460 0x557942352b40 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.444+0000 7f20bbe37740 1 --2- 172.21.15.69:0/1079707435 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55794234eb40 0x557942361f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.444+0000 7f20bbe37740 1 -- 172.21.15.69:0/1079707435 >> 172.21.15.69:0/1079707435 conn(0x55794234ae60 msgr2=0x557942349f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.444+0000 7f2093e00640 0 client.15969 ms_handle_reset on v2:172.21.15.81:6836/1973819593 2024-04-24T09:08:17.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.444+0000 7f20bbe37740 1 -- 172.21.15.69:0/1079707435 shutdown_connections 2024-04-24T09:08:17.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.444+0000 7f20bbe37740 1 -- 172.21.15.69:0/1079707435 wait complete. 2024-04-24T09:08:17.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.444+0000 7f20bbe37740 20 client.15969 trim_cache size 0 max 16384 2024-04-24T09:08:17.445 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-04-24T09:08:17.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.445+0000 7f20b3200640 1 -- 172.21.15.69:0/1561959226 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f20980a3ec0 msgr2=0x7f20980a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.445+0000 7f20b3200640 1 --2- 172.21.15.69:0/1561959226 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f20980a3ec0 0x7f20980a6380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f20ac013570 tx=0x7f20ac009600 comp rx=0 tx=0).stop 2024-04-24T09:08:17.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.445+0000 7f20b3200640 1 -- 172.21.15.69:0/1561959226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20b4101770 msgr2=0x7f20b41a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.445+0000 7f20b3200640 1 --2- 172.21.15.69:0/1561959226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20b4101770 0x7f20b41a4f90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f20a000d740 tx=0x7f20a000dc00 comp rx=0 tx=0).stop 2024-04-24T09:08:17.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.446+0000 7f20b3200640 1 -- 172.21.15.69:0/1561959226 shutdown_connections 2024-04-24T09:08:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.446+0000 7f20b3200640 1 --2- 172.21.15.69:0/1561959226 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f20980a3ec0 0x7f20980a6380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.446+0000 7f20b3200640 1 --2- 172.21.15.69:0/1561959226 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20b4106250 0x7f20b419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.446+0000 7f20b3200640 1 --2- 172.21.15.69:0/1561959226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20b4101770 0x7f20b41a4f90 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.446+0000 7f20b3200640 1 --2- 172.21.15.69:0/1561959226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20b4100e50 0x7f20b41a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.446+0000 7f20b3200640 1 -- 172.21.15.69:0/1561959226 >> 172.21.15.69:0/1561959226 conn(0x7f20b40fc880 msgr2=0x7f20b40fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.446+0000 7f20b3200640 1 -- 172.21.15.69:0/1561959226 shutdown_connections 2024-04-24T09:08:17.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.446+0000 7f20b3200640 1 -- 172.21.15.69:0/1561959226 wait complete. 2024-04-24T09:08:17.462 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:08:17.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.635+0000 7f2683200640 1 -- 172.21.15.69:0/1700452086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26841019b0 msgr2=0x7f26841059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.635+0000 7f2683200640 1 --2- 172.21.15.69:0/1700452086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26841019b0 0x7f26841059f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2670009900 tx=0x7f267002a410 comp rx=0 tx=0).stop 2024-04-24T09:08:17.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.635+0000 7f2683200640 1 -- 172.21.15.69:0/1700452086 shutdown_connections 2024-04-24T09:08:17.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.635+0000 7f2683200640 1 --2- 172.21.15.69:0/1700452086 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26841060d0 0x7f268410dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.635+0000 7f2683200640 1 --2- 172.21.15.69:0/1700452086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26841019b0 0x7f26841059f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.635+0000 7f2683200640 1 --2- 172.21.15.69:0/1700452086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2684101000 0x7f26841013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.635+0000 7f2683200640 1 -- 172.21.15.69:0/1700452086 >> 172.21.15.69:0/1700452086 conn(0x7f26840fc880 msgr2=0x7f26840feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.635+0000 7f2683200640 1 -- 172.21.15.69:0/1700452086 shutdown_connections 2024-04-24T09:08:17.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.635+0000 7f2683200640 1 -- 172.21.15.69:0/1700452086 wait complete. 2024-04-24T09:08:17.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.637+0000 7f2683200640 1 Processor -- start 2024-04-24T09:08:17.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.637+0000 7f2683200640 1 -- start start 2024-04-24T09:08:17.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.637+0000 7f2683200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2684101000 0x7f268410e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.637+0000 7f2683200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26841060d0 0x7f268410eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.637+0000 7f2683200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f268410f0b0 0x7f26841a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.637+0000 7f2683200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2684112150 con 0x7f26841060d0 2024-04-24T09:08:17.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.637+0000 7f2683200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2684111fd0 con 0x7f2684101000 2024-04-24T09:08:17.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.637+0000 7f2683200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f26841122d0 con 0x7f268410f0b0 2024-04-24T09:08:17.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.638+0000 7f2681400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26841060d0 0x7f268410eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.638+0000 7f2681400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26841060d0 0x7f268410eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55468/0 (socket says 172.21.15.69:55468) 2024-04-24T09:08:17.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.638+0000 7f2681400640 1 -- 172.21.15.69:0/3786428416 learned_addr learned my addr 172.21.15.69:0/3786428416 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:17.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.638+0000 7f2681e00640 1 --2- 172.21.15.69:0/3786428416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2684101000 0x7f268410e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.638+0000 7f2682800640 1 --2- 172.21.15.69:0/3786428416 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f268410f0b0 0x7f26841a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.638+0000 7f2681400640 1 -- 172.21.15.69:0/3786428416 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f268410f0b0 msgr2=0x7f26841a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.638+0000 7f2681400640 1 --2- 172.21.15.69:0/3786428416 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f268410f0b0 0x7f26841a68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.638+0000 7f2681400640 1 -- 172.21.15.69:0/3786428416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2684101000 msgr2=0x7f268410e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.638+0000 7f2681400640 1 --2- 172.21.15.69:0/3786428416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2684101000 0x7f268410e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.638+0000 7f2681400640 1 -- 172.21.15.69:0/3786428416 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26841a6fa0 con 0x7f26841060d0 2024-04-24T09:08:17.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.638+0000 7f2681e00640 1 --2- 172.21.15.69:0/3786428416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2684101000 0x7f268410e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:17.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.639+0000 7f2681400640 1 --2- 172.21.15.69:0/3786428416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26841060d0 0x7f268410eb70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f26700058c0 tx=0x7f2670031a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.639+0000 7f267aa00640 1 -- 172.21.15.69:0/3786428416 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670041020 con 0x7f26841060d0 2024-04-24T09:08:17.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.639+0000 7f2683200640 1 -- 172.21.15.69:0/3786428416 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26841a7230 con 0x7f26841060d0 2024-04-24T09:08:17.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.639+0000 7f2683200640 1 -- 172.21.15.69:0/3786428416 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26841a7770 con 0x7f26841060d0 2024-04-24T09:08:17.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.639+0000 7f267aa00640 1 -- 172.21.15.69:0/3786428416 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26700053c0 con 0x7f26841060d0 2024-04-24T09:08:17.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.639+0000 7f267aa00640 1 -- 172.21.15.69:0/3786428416 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670036040 con 0x7f26841060d0 2024-04-24T09:08:17.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.643+0000 7f268be0b740 1 Processor -- start 2024-04-24T09:08:17.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.643+0000 7f268be0b740 1 -- start start 2024-04-24T09:08:17.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.644+0000 7f268be0b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee7fb40 0x56050ee7ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.644+0000 7f268be0b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee80460 0x56050ee8a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.644+0000 7f268be0b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee8ab30 0x56050ee8cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.644+0000 7f268be0b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56050ede51d0 con 0x56050ee7fb40 2024-04-24T09:08:17.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.644+0000 7f268be0b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56050ed8b450 con 0x56050ee80460 2024-04-24T09:08:17.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.644+0000 7f268be0b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56050ede1190 con 0x56050ee8ab30 2024-04-24T09:08:17.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.644+0000 7f2682800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee7fb40 0x56050ee7ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.645+0000 7f2682800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee7fb40 0x56050ee7ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55470/0 (socket says 172.21.15.69:55470) 2024-04-24T09:08:17.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.645+0000 7f2682800640 1 -- 172.21.15.69:0/215480242 learned_addr learned my addr 172.21.15.69:0/215480242 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:17.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.645+0000 7f2681e00640 1 --2- 172.21.15.69:0/215480242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee80460 0x56050ee8a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.645+0000 7f2682800640 1 -- 172.21.15.69:0/215480242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee8ab30 msgr2=0x56050ee8cf20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:08:17.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.645+0000 7f2682800640 1 --2- 172.21.15.69:0/215480242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee8ab30 0x56050ee8cf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.645+0000 7f2682800640 1 -- 172.21.15.69:0/215480242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee80460 msgr2=0x56050ee8a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.645+0000 7f2682800640 1 --2- 172.21.15.69:0/215480242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee80460 0x56050ee8a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.645+0000 7f2682800640 1 -- 172.21.15.69:0/215480242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050ee48310 con 0x56050ee7fb40 2024-04-24T09:08:17.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.645+0000 7f2682800640 1 --2- 172.21.15.69:0/215480242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee7fb40 0x56050ee7ff20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f267c009e60 tx=0x7f267c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6e20a3728c7bf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.646+0000 7f266f400640 1 -- 172.21.15.69:0/215480242 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c016070 con 0x56050ee7fb40 2024-04-24T09:08:17.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.646+0000 7f266f400640 1 -- 172.21.15.69:0/215480242 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f267c004d20 con 0x56050ee7fb40 2024-04-24T09:08:17.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.646+0000 7f266f400640 1 -- 172.21.15.69:0/215480242 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c0126b0 con 0x56050ee7fb40 2024-04-24T09:08:17.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.647+0000 7f268be0b740 1 -- 172.21.15.69:0/215480242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee7fb40 msgr2=0x56050ee7ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.647+0000 7f268be0b740 1 --2- 172.21.15.69:0/215480242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee7fb40 0x56050ee7ff20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f267c009e60 tx=0x7f267c010740 comp rx=0 tx=0).stop 2024-04-24T09:08:17.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.647+0000 7f268be0b740 1 -- 172.21.15.69:0/215480242 shutdown_connections 2024-04-24T09:08:17.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.647+0000 7f268be0b740 1 --2- 172.21.15.69:0/215480242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee8ab30 0x56050ee8cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.647+0000 7f268be0b740 1 --2- 172.21.15.69:0/215480242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee80460 0x56050ee8a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.647+0000 7f268be0b740 1 --2- 172.21.15.69:0/215480242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee7fb40 0x56050ee7ff20 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.647+0000 7f268be0b740 1 -- 172.21.15.69:0/215480242 >> 172.21.15.69:0/215480242 conn(0x56050ee7be60 msgr2=0x56050ee7c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.648+0000 7f267aa00640 1 -- 172.21.15.69:0/3786428416 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f2670044020 con 0x7f26841060d0 2024-04-24T09:08:17.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.648+0000 7f268be0b740 1 -- 172.21.15.69:0/215480242 shutdown_connections 2024-04-24T09:08:17.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.648+0000 7f268be0b740 1 -- 172.21.15.69:0/215480242 wait complete. 2024-04-24T09:08:17.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.648+0000 7f268be0b740 1 -- start start 2024-04-24T09:08:17.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.649+0000 7f268be0b740 1 Processor -- start 2024-04-24T09:08:17.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.649+0000 7f267aa00640 1 --2- 172.21.15.69:0/3786428416 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f26680a3e00 0x7f26680a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.649+0000 7f268be0b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee7fb40 0x56050ee879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.649+0000 7f267aa00640 1 -- 172.21.15.69:0/3786428416 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f26700f1830 con 0x7f26841060d0 2024-04-24T09:08:17.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.650+0000 7f268be0b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee80460 0x56050ee87f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.650+0000 7f268be0b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee88460 0x56050ee96e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.650+0000 7f268be0b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56050ee8d9c0 con 0x56050ee7fb40 2024-04-24T09:08:17.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.650+0000 7f268be0b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56050ed9e810 con 0x56050ee88460 2024-04-24T09:08:17.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.650+0000 7f268be0b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56050ee8db40 con 0x56050ee80460 2024-04-24T09:08:17.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.650+0000 7f268be0b740 10 client.0 _ll_register_callbacks cb 0x56050ed6c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:17.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.650+0000 7f2682800640 1 --2- 172.21.15.69:0/3786428416 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f26680a3e00 0x7f26680a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.650+0000 7f2681400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee80460 0x56050ee87f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.650+0000 7f2681400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee80460 0x56050ee87f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.69:45246/0 (socket says 172.21.15.69:45246) 2024-04-24T09:08:17.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.650+0000 7f2681400640 1 -- 172.21.15.69:0/4091057544 learned_addr learned my addr 172.21.15.69:0/4091057544 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:17.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.651+0000 7f2681e00640 1 --2- 172.21.15.69:0/4091057544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee7fb40 0x56050ee879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.651+0000 7f2682800640 1 --2- 172.21.15.69:0/4091057544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee88460 0x56050ee96e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.651+0000 7f2682800640 1 --2- 172.21.15.69:0/3786428416 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f26680a3e00 0x7f26680a62c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f267c005970 tx=0x7f267c0059d0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.651+0000 7f2681400640 1 -- 172.21.15.69:0/4091057544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee88460 msgr2=0x56050ee96e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.651+0000 7f2681400640 1 --2- 172.21.15.69:0/4091057544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee88460 0x56050ee96e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.651+0000 7f2681400640 1 -- 172.21.15.69:0/4091057544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee7fb40 msgr2=0x56050ee879e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.651+0000 7f2681400640 1 --2- 172.21.15.69:0/4091057544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee7fb40 0x56050ee879e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.651+0000 7f2681400640 1 -- 172.21.15.69:0/4091057544 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050edaf3f0 con 0x56050ee80460 2024-04-24T09:08:17.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.651+0000 7f2681e00640 1 --2- 172.21.15.69:0/4091057544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee7fb40 0x56050ee879e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:17.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.652+0000 7f2681400640 1 --2- 172.21.15.69:0/4091057544 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee80460 0x56050ee87f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f26700ef240 tx=0x7f26700bda10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7731f98b87264755 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.652+0000 7f2663e00640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-24T09:08:17.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.652+0000 7f2663e00640 1 -- 172.21.15.69:0/4091057544 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26700c5f80 con 0x56050ee80460 2024-04-24T09:08:17.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.652+0000 7f2663e00640 1 -- 172.21.15.69:0/4091057544 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f26700b9980 con 0x56050ee80460 2024-04-24T09:08:17.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.652+0000 7f268be0b740 1 -- 172.21.15.69:0/4091057544 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267c004d20 con 0x56050ee80460 2024-04-24T09:08:17.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.653+0000 7f2663e00640 1 -- 172.21.15.69:0/4091057544 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f26700bd570 con 0x56050ee80460 2024-04-24T09:08:17.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.653+0000 7f268be0b740 10 client.25330 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:17.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.653+0000 7f268be0b740 1 -- 172.21.15.69:0/4091057544 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56050eaf73f0 con 0x56050ee80460 2024-04-24T09:08:17.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.653+0000 7f2663e00640 1 -- 172.21.15.69:0/4091057544 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f267c004d20 con 0x56050ee80460 2024-04-24T09:08:17.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.653+0000 7f2663e00640 1 -- 172.21.15.69:0/4091057544 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f26700eeda0 con 0x56050ee80460 2024-04-24T09:08:17.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.654+0000 7f268be0b740 10 client.25330 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:17.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.654+0000 7f268be0b740 10 client.25330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:17.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.654+0000 7f268be0b740 20 client.25330 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:17.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.654+0000 7f268be0b740 1 --2- 172.21.15.69:0/4091057544 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x56050ee9b170 0x56050ee9d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.654+0000 7f268be0b740 4 client.25330 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:17.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.654+0000 7f268be0b740 1 -- 172.21.15.69:0/4091057544 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56050edf2bf0 con 0x56050ee9b170 2024-04-24T09:08:17.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.654+0000 7f2681e00640 1 --2- 172.21.15.69:0/4091057544 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x56050ee9b170 0x56050ee9d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.655+0000 7f2681e00640 1 --2- 172.21.15.69:0/4091057544 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x56050ee9b170 0x56050ee9d560 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa0e0a09773595e0 server_cookie=bf530e134e38dd94 in_seq=0 out_seq=0 2024-04-24T09:08:17.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.655+0000 7f2663e00640 10 client.25330 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:17.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.657+0000 7f2663e00640 1 -- 172.21.15.69:0/4091057544 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56050edf2bf0 con 0x56050ee9b170 2024-04-24T09:08:17.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.657+0000 7f2663e00640 10 client.25330 handle_command_reply: tid=0 2024-04-24T09:08:17.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.657+0000 7f268be0b740 1 client.25330 shutdown 2024-04-24T09:08:17.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.657+0000 7f268be0b740 1 -- 172.21.15.69:0/4091057544 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee80460 msgr2=0x56050ee87f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.657+0000 7f268be0b740 1 --2- 172.21.15.69:0/4091057544 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee80460 0x56050ee87f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f26700ef240 tx=0x7f26700bda10 comp rx=0 tx=0).stop 2024-04-24T09:08:17.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.658+0000 7f268be0b740 1 -- 172.21.15.69:0/4091057544 shutdown_connections 2024-04-24T09:08:17.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.658+0000 7f268be0b740 1 --2- 172.21.15.69:0/4091057544 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x56050ee9b170 0x56050ee9d560 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.658+0000 7f268be0b740 1 --2- 172.21.15.69:0/4091057544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee88460 0x56050ee96e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.658+0000 7f268be0b740 1 --2- 172.21.15.69:0/4091057544 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee80460 0x56050ee87f20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.658+0000 7f268be0b740 1 --2- 172.21.15.69:0/4091057544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee7fb40 0x56050ee879e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.658+0000 7f268be0b740 1 -- 172.21.15.69:0/4091057544 >> 172.21.15.69:0/4091057544 conn(0x56050ee7be60 msgr2=0x56050ee81be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.658+0000 7f2663e00640 0 client.25330 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:17.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.658+0000 7f268be0b740 1 -- 172.21.15.69:0/4091057544 shutdown_connections 2024-04-24T09:08:17.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.658+0000 7f268be0b740 1 -- 172.21.15.69:0/4091057544 wait complete. 2024-04-24T09:08:17.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.658+0000 7f268be0b740 20 client.25330 trim_cache size 0 max 16384 2024-04-24T09:08:17.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.685+0000 7f268be0b740 1 Processor -- start 2024-04-24T09:08:17.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.686+0000 7f268be0b740 1 -- start start 2024-04-24T09:08:17.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.686+0000 7f268be0b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee7fb40 0x56050ee819e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.686+0000 7f268be0b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee80460 0x56050eea2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.686+0000 7f268be0b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee88460 0x56050eea3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.686+0000 7f268be0b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f26841b3ac0 con 0x56050ee88460 2024-04-24T09:08:17.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.686+0000 7f268be0b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f26841b1390 con 0x56050ee7fb40 2024-04-24T09:08:17.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.686+0000 7f268be0b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f26841af1e0 con 0x56050ee80460 2024-04-24T09:08:17.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.687+0000 7f2682800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee88460 0x56050eea3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.687+0000 7f2682800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee88460 0x56050eea3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55490/0 (socket says 172.21.15.69:55490) 2024-04-24T09:08:17.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.687+0000 7f2682800640 1 -- 172.21.15.69:0/2927673200 learned_addr learned my addr 172.21.15.69:0/2927673200 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:17.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.687+0000 7f2681e00640 1 --2- 172.21.15.69:0/2927673200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee7fb40 0x56050ee819e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.687+0000 7f2682800640 1 -- 172.21.15.69:0/2927673200 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee80460 msgr2=0x56050eea2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.687+0000 7f2682800640 1 --2- 172.21.15.69:0/2927673200 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee80460 0x56050eea2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.687+0000 7f2682800640 1 -- 172.21.15.69:0/2927673200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee7fb40 msgr2=0x56050ee819e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.687+0000 7f2682800640 1 --2- 172.21.15.69:0/2927673200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee7fb40 0x56050ee819e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.687+0000 7f2682800640 1 -- 172.21.15.69:0/2927673200 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050ee86990 con 0x56050ee88460 2024-04-24T09:08:17.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.688+0000 7f2682800640 1 --2- 172.21.15.69:0/2927673200 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee88460 0x56050eea3370 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f267c010c40 tx=0x7f267c025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969c0dd0e4305772 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.688+0000 7f266ea00640 1 -- 172.21.15.69:0/2927673200 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c016040 con 0x56050ee88460 2024-04-24T09:08:17.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.688+0000 7f266ea00640 1 -- 172.21.15.69:0/2927673200 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f267c012df0 con 0x56050ee88460 2024-04-24T09:08:17.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.688+0000 7f266ea00640 1 -- 172.21.15.69:0/2927673200 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c01a630 con 0x56050ee88460 2024-04-24T09:08:17.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.689+0000 7f268be0b740 1 -- 172.21.15.69:0/2927673200 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee88460 msgr2=0x56050eea3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.689+0000 7f268be0b740 1 --2- 172.21.15.69:0/2927673200 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee88460 0x56050eea3370 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f267c010c40 tx=0x7f267c025000 comp rx=0 tx=0).stop 2024-04-24T09:08:17.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.689+0000 7f268be0b740 1 -- 172.21.15.69:0/2927673200 shutdown_connections 2024-04-24T09:08:17.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.689+0000 7f268be0b740 1 --2- 172.21.15.69:0/2927673200 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee88460 0x56050eea3370 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.689+0000 7f268be0b740 1 --2- 172.21.15.69:0/2927673200 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee80460 0x56050eea2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.689+0000 7f268be0b740 1 --2- 172.21.15.69:0/2927673200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee7fb40 0x56050ee819e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.689+0000 7f268be0b740 1 -- 172.21.15.69:0/2927673200 >> 172.21.15.69:0/2927673200 conn(0x56050ee7be60 msgr2=0x56050ee812d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.689+0000 7f268be0b740 1 -- 172.21.15.69:0/2927673200 shutdown_connections 2024-04-24T09:08:17.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.689+0000 7f268be0b740 1 -- 172.21.15.69:0/2927673200 wait complete. 2024-04-24T09:08:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.689+0000 7f268be0b740 1 -- start start 2024-04-24T09:08:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.690+0000 7f268be0b740 1 Processor -- start 2024-04-24T09:08:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.690+0000 7f268be0b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee7fb40 0x56050ee931c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.690+0000 7f268be0b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee80460 0x56050ee8b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.690+0000 7f268be0b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee88460 0x56050ee8bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.690+0000 7f268be0b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f26841af850 con 0x56050ee88460 2024-04-24T09:08:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.690+0000 7f268be0b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f26841b0cc0 con 0x56050ee80460 2024-04-24T09:08:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.690+0000 7f268be0b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f26841aff20 con 0x56050ee7fb40 2024-04-24T09:08:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.690+0000 7f268be0b740 10 client.0 _ll_register_callbacks cb 0x7f26700bce50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.691+0000 7f2682800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee88460 0x56050ee8bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.691+0000 7f2682800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee88460 0x56050ee8bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55494/0 (socket says 172.21.15.69:55494) 2024-04-24T09:08:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.691+0000 7f2682800640 1 -- 172.21.15.69:0/3270448763 learned_addr learned my addr 172.21.15.69:0/3270448763 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.691+0000 7f2681e00640 1 --2- 172.21.15.69:0/3270448763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee7fb40 0x56050ee931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.691+0000 7f2681400640 1 --2- 172.21.15.69:0/3270448763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee80460 0x56050ee8b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.691+0000 7f2682800640 1 -- 172.21.15.69:0/3270448763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee7fb40 msgr2=0x56050ee931c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.691+0000 7f2682800640 1 --2- 172.21.15.69:0/3270448763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee7fb40 0x56050ee931c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.691+0000 7f2682800640 1 -- 172.21.15.69:0/3270448763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee80460 msgr2=0x56050ee8b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.691+0000 7f2682800640 1 --2- 172.21.15.69:0/3270448763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee80460 0x56050ee8b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.691+0000 7f2682800640 1 -- 172.21.15.69:0/3270448763 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050ebc17d0 con 0x56050ee88460 2024-04-24T09:08:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.691+0000 7f2681400640 1 --2- 172.21.15.69:0/3270448763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee80460 0x56050ee8b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:17.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.692+0000 7f2682800640 1 --2- 172.21.15.69:0/3270448763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee88460 0x56050ee8bbc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f267c010780 tx=0x7f267c0276d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa6f6c0d68bef29 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:17.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.692+0000 7f2663e00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:17.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.692+0000 7f2663e00640 1 -- 172.21.15.69:0/3270448763 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c016050 con 0x56050ee88460 2024-04-24T09:08:17.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.692+0000 7f2663e00640 1 -- 172.21.15.69:0/3270448763 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f267c030470 con 0x56050ee88460 2024-04-24T09:08:17.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.692+0000 7f268be0b740 1 -- 172.21.15.69:0/3270448763 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267c012df0 con 0x56050ee88460 2024-04-24T09:08:17.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.693+0000 7f2663e00640 1 -- 172.21.15.69:0/3270448763 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f267c043050 con 0x56050ee88460 2024-04-24T09:08:17.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.693+0000 7f268be0b740 10 client.15987 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:17.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.693+0000 7f268be0b740 1 -- 172.21.15.69:0/3270448763 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56050ef09eb0 con 0x56050ee88460 2024-04-24T09:08:17.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.693+0000 7f2663e00640 1 -- 172.21.15.69:0/3270448763 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f267c012df0 con 0x56050ee88460 2024-04-24T09:08:17.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.693+0000 7f2663e00640 1 -- 172.21.15.69:0/3270448763 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f267c00f040 con 0x56050ee88460 2024-04-24T09:08:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.694+0000 7f268be0b740 10 client.15987 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.694+0000 7f268be0b740 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.694+0000 7f268be0b740 20 client.15987 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.694+0000 7f268be0b740 1 --2- 172.21.15.69:0/3270448763 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x56050ef0a2b0 0x56050ef0c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.694+0000 7f268be0b740 4 client.15987 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.694+0000 7f268be0b740 1 -- 172.21.15.69:0/3270448763 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56050edeca60 con 0x56050ef0a2b0 2024-04-24T09:08:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.695+0000 7f2681e00640 1 --2- 172.21.15.69:0/3270448763 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x56050ef0a2b0 0x56050ef0c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:17.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.696+0000 7f2663e00640 10 client.15987 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.696+0000 7f2681e00640 1 --2- 172.21.15.69:0/3270448763 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x56050ef0a2b0 0x56050ef0c6d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ee4aaeef1e1d9 server_cookie=2e3c11971a5e0d5f in_seq=0 out_seq=0 2024-04-24T09:08:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.696+0000 7f2663e00640 1 -- 172.21.15.69:0/3270448763 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56050edeca60 con 0x56050ef0a2b0 2024-04-24T09:08:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.696+0000 7f2663e00640 10 client.15987 handle_command_reply: tid=0 2024-04-24T09:08:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.696+0000 7f268be0b740 1 client.15987 shutdown 2024-04-24T09:08:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.696+0000 7f268be0b740 1 -- 172.21.15.69:0/3270448763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee88460 msgr2=0x56050ee8bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.696+0000 7f268be0b740 1 --2- 172.21.15.69:0/3270448763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee88460 0x56050ee8bbc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f267c010780 tx=0x7f267c0276d0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.697+0000 7f268be0b740 1 -- 172.21.15.69:0/3270448763 shutdown_connections 2024-04-24T09:08:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.697+0000 7f268be0b740 1 --2- 172.21.15.69:0/3270448763 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x56050ef0a2b0 0x56050ef0c6d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.697+0000 7f268be0b740 1 --2- 172.21.15.69:0/3270448763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56050ee88460 0x56050ee8bbc0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.697+0000 7f268be0b740 1 --2- 172.21.15.69:0/3270448763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56050ee80460 0x56050ee8b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.697+0000 7f268be0b740 1 --2- 172.21.15.69:0/3270448763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56050ee7fb40 0x56050ee931c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.697+0000 7f268be0b740 1 -- 172.21.15.69:0/3270448763 >> 172.21.15.69:0/3270448763 conn(0x56050ee7be60 msgr2=0x56050ee7af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.697+0000 7f2663e00640 0 client.15987 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.697+0000 7f268be0b740 1 -- 172.21.15.69:0/3270448763 shutdown_connections 2024-04-24T09:08:17.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.697+0000 7f268be0b740 1 -- 172.21.15.69:0/3270448763 wait complete. 2024-04-24T09:08:17.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.697+0000 7f268be0b740 20 client.15987 trim_cache size 0 max 16384 2024-04-24T09:08:17.698 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-04-24T09:08:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.698+0000 7f2683200640 1 -- 172.21.15.69:0/3786428416 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f26680a3e00 msgr2=0x7f26680a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.698+0000 7f2683200640 1 --2- 172.21.15.69:0/3786428416 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f26680a3e00 0x7f26680a62c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f267c005970 tx=0x7f267c0059d0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.698+0000 7f2683200640 1 -- 172.21.15.69:0/3786428416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26841060d0 msgr2=0x7f268410eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.698+0000 7f2683200640 1 --2- 172.21.15.69:0/3786428416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26841060d0 0x7f268410eb70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f26700058c0 tx=0x7f2670031a80 comp rx=0 tx=0).stop 2024-04-24T09:08:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.698+0000 7f2683200640 1 -- 172.21.15.69:0/3786428416 shutdown_connections 2024-04-24T09:08:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.698+0000 7f2683200640 1 --2- 172.21.15.69:0/3786428416 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f26680a3e00 0x7f26680a62c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.698+0000 7f2683200640 1 --2- 172.21.15.69:0/3786428416 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f268410f0b0 0x7f26841a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.698+0000 7f2683200640 1 --2- 172.21.15.69:0/3786428416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26841060d0 0x7f268410eb70 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.698+0000 7f2683200640 1 --2- 172.21.15.69:0/3786428416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2684101000 0x7f268410e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.698+0000 7f2683200640 1 -- 172.21.15.69:0/3786428416 >> 172.21.15.69:0/3786428416 conn(0x7f26840fc880 msgr2=0x7f26840fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:17.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.698+0000 7f2683200640 1 -- 172.21.15.69:0/3786428416 shutdown_connections 2024-04-24T09:08:17.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:17.698+0000 7f2683200640 1 -- 172.21.15.69:0/3786428416 wait complete. 2024-04-24T09:08:18.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:17 smithi081 ceph-mon[30961]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T09:08:18.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:17 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3074690135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:08:18.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:17 smithi069 ceph-mon[28170]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T09:08:18.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:17 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3074690135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:08:18.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:17 smithi130 ceph-mon[31553]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T09:08:18.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:17 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3074690135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:08:18.715 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:08:18.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:08:18.715 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:08:18.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.894+0000 7f1763e00640 1 -- 172.21.15.69:0/1299746713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1764074430 msgr2=0x7f1764074810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:18.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.894+0000 7f1763e00640 1 --2- 172.21.15.69:0/1299746713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1764074430 0x7f1764074810 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1754009900 tx=0x7f175402a480 comp rx=0 tx=0).stop 2024-04-24T09:08:18.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.894+0000 7f1763e00640 1 -- 172.21.15.69:0/1299746713 shutdown_connections 2024-04-24T09:08:18.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.894+0000 7f1763e00640 1 --2- 172.21.15.69:0/1299746713 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f176410e6e0 0x7f1764110b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.894+0000 7f1763e00640 1 --2- 172.21.15.69:0/1299746713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1764074d50 0x7f176410e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.894+0000 7f1763e00640 1 --2- 172.21.15.69:0/1299746713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1764074430 0x7f1764074810 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.894+0000 7f1763e00640 1 -- 172.21.15.69:0/1299746713 >> 172.21.15.69:0/1299746713 conn(0x7f17640fd9d0 msgr2=0x7f17640ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:18.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.895+0000 7f1763e00640 1 -- 172.21.15.69:0/1299746713 shutdown_connections 2024-04-24T09:08:18.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.895+0000 7f1763e00640 1 -- 172.21.15.69:0/1299746713 wait complete. 2024-04-24T09:08:18.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.895+0000 7f1763e00640 1 Processor -- start 2024-04-24T09:08:18.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.896+0000 7f1763e00640 1 -- start start 2024-04-24T09:08:18.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.896+0000 7f1763e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1764074430 0x7f17641a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.896+0000 7f1763e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1764074d50 0x7f17641a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.896+0000 7f1763e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f176410e6e0 0x7f176419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.896+0000 7f1763e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1764114360 con 0x7f176410e6e0 2024-04-24T09:08:18.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.896+0000 7f1763e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f17641141e0 con 0x7f1764074430 2024-04-24T09:08:18.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.896+0000 7f1763e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f17641144e0 con 0x7f1764074d50 2024-04-24T09:08:18.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.896+0000 7f1763400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f176410e6e0 0x7f176419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:18.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.896+0000 7f1763400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f176410e6e0 0x7f176419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55502/0 (socket says 172.21.15.69:55502) 2024-04-24T09:08:18.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.896+0000 7f1763400640 1 -- 172.21.15.69:0/675745354 learned_addr learned my addr 172.21.15.69:0/675745354 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:18.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.897+0000 7f1762a00640 1 --2- 172.21.15.69:0/675745354 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1764074430 0x7f17641a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:18.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.897+0000 7f175be00640 1 --2- 172.21.15.69:0/675745354 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1764074d50 0x7f17641a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:18.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.897+0000 7f1763400640 1 -- 172.21.15.69:0/675745354 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1764074d50 msgr2=0x7f17641a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:18.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.897+0000 7f1763400640 1 --2- 172.21.15.69:0/675745354 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1764074d50 0x7f17641a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.897+0000 7f1763400640 1 -- 172.21.15.69:0/675745354 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1764074430 msgr2=0x7f17641a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:18.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.897+0000 7f1763400640 1 --2- 172.21.15.69:0/675745354 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1764074430 0x7f17641a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.897+0000 7f1763400640 1 -- 172.21.15.69:0/675745354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176419b000 con 0x7f176410e6e0 2024-04-24T09:08:18.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.897+0000 7f1763400640 1 --2- 172.21.15.69:0/675745354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f176410e6e0 0x7f176419a740 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f175c00cad0 tx=0x7f175c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:18.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.898+0000 7f175b400640 1 -- 172.21.15.69:0/675745354 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c00ae20 con 0x7f176410e6e0 2024-04-24T09:08:18.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.898+0000 7f175b400640 1 -- 172.21.15.69:0/675745354 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f175c005020 con 0x7f176410e6e0 2024-04-24T09:08:18.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.898+0000 7f175b400640 1 -- 172.21.15.69:0/675745354 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c00f6b0 con 0x7f176410e6e0 2024-04-24T09:08:18.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.898+0000 7f1763e00640 1 -- 172.21.15.69:0/675745354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176419b2f0 con 0x7f176410e6e0 2024-04-24T09:08:18.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.898+0000 7f1763e00640 1 -- 172.21.15.69:0/675745354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17641a75b0 con 0x7f176410e6e0 2024-04-24T09:08:18.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.902+0000 7f176c43c740 1 Processor -- start 2024-04-24T09:08:18.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.903+0000 7f175b400640 1 -- 172.21.15.69:0/675745354 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f175c0051d0 con 0x7f176410e6e0 2024-04-24T09:08:18.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.903+0000 7f176c43c740 1 -- start start 2024-04-24T09:08:18.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.903+0000 7f176c43c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb1150a80 0x555cb1150e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.903+0000 7f176c43c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb11513a0 0x555cb115b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.903+0000 7f176c43c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb115ba70 0x555cb115de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.903+0000 7f176c43c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555cb10ade70 con 0x555cb115ba70 2024-04-24T09:08:18.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.903+0000 7f176c43c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x555cb1054380 con 0x555cb1150a80 2024-04-24T09:08:18.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.903+0000 7f176c43c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555cb10a9e30 con 0x555cb11513a0 2024-04-24T09:08:18.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.904+0000 7f175be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb115ba70 0x555cb115de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:18.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.904+0000 7f1762a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb11513a0 0x555cb115b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:18.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.904+0000 7f175be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb115ba70 0x555cb115de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55510/0 (socket says 172.21.15.69:55510) 2024-04-24T09:08:18.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.904+0000 7f175be00640 1 -- 172.21.15.69:0/1036875849 learned_addr learned my addr 172.21.15.69:0/1036875849 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:18.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.904+0000 7f175be00640 1 -- 172.21.15.69:0/1036875849 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb11513a0 msgr2=0x555cb115b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:18.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.904+0000 7f175be00640 1 --2- 172.21.15.69:0/1036875849 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb11513a0 0x555cb115b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.905+0000 7f175be00640 1 -- 172.21.15.69:0/1036875849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb1150a80 msgr2=0x555cb1150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:18.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.905+0000 7f175be00640 1 --2- 172.21.15.69:0/1036875849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb1150a80 0x555cb1150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.905+0000 7f175be00640 1 -- 172.21.15.69:0/1036875849 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cb0e887e0 con 0x555cb115ba70 2024-04-24T09:08:18.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.905+0000 7f175b400640 1 --2- 172.21.15.69:0/675745354 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f17480a3ec0 0x7f17480a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.905+0000 7f175b400640 1 -- 172.21.15.69:0/675745354 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f175c0d0ed0 con 0x7f176410e6e0 2024-04-24T09:08:18.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.905+0000 7f175be00640 1 --2- 172.21.15.69:0/1036875849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb115ba70 0x555cb115de60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f174c00eca0 tx=0x7f174c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abdb61f2f0adbd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:18.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.905+0000 7f1762a00640 1 --2- 172.21.15.69:0/675745354 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f17480a3ec0 0x7f17480a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:18.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.906+0000 7f1753400640 1 -- 172.21.15.69:0/1036875849 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c005220 con 0x555cb115ba70 2024-04-24T09:08:18.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.906+0000 7f1753400640 1 -- 172.21.15.69:0/1036875849 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f174c015070 con 0x555cb115ba70 2024-04-24T09:08:18.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.906+0000 7f1753400640 1 -- 172.21.15.69:0/1036875849 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c0117a0 con 0x555cb115ba70 2024-04-24T09:08:18.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.906+0000 7f176c43c740 1 -- 172.21.15.69:0/1036875849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb115ba70 msgr2=0x555cb115de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:18.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.906+0000 7f176c43c740 1 --2- 172.21.15.69:0/1036875849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb115ba70 0x555cb115de60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f174c00eca0 tx=0x7f174c0107a0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.906+0000 7f1762a00640 1 --2- 172.21.15.69:0/675745354 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f17480a3ec0 0x7f17480a6380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1754006010 tx=0x7f1754031400 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:18.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.907+0000 7f176c43c740 1 -- 172.21.15.69:0/1036875849 shutdown_connections 2024-04-24T09:08:18.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.907+0000 7f176c43c740 1 --2- 172.21.15.69:0/1036875849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb115ba70 0x555cb115de60 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.907+0000 7f176c43c740 1 --2- 172.21.15.69:0/1036875849 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb11513a0 0x555cb115b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.907+0000 7f176c43c740 1 --2- 172.21.15.69:0/1036875849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb1150a80 0x555cb1150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.907+0000 7f176c43c740 1 -- 172.21.15.69:0/1036875849 >> 172.21.15.69:0/1036875849 conn(0x555cb114cda0 msgr2=0x555cb114d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:18.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.907+0000 7f176c43c740 1 -- 172.21.15.69:0/1036875849 shutdown_connections 2024-04-24T09:08:18.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.907+0000 7f176c43c740 1 -- 172.21.15.69:0/1036875849 wait complete. 2024-04-24T09:08:18.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.907+0000 7f176c43c740 1 -- start start 2024-04-24T09:08:18.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.907+0000 7f176c43c740 1 Processor -- start 2024-04-24T09:08:18.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.908+0000 7f176c43c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb1150a80 0x555cb1158a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.908+0000 7f176c43c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb11513a0 0x555cb1158f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.908+0000 7f176c43c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb115ba70 0x555cb11594d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.908+0000 7f176c43c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555cb115e860 con 0x555cb1150a80 2024-04-24T09:08:18.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.908+0000 7f176c43c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x555cb1067660 con 0x555cb115ba70 2024-04-24T09:08:18.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.908+0000 7f176c43c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555cb115e9e0 con 0x555cb11513a0 2024-04-24T09:08:18.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.908+0000 7f176c43c740 10 client.0 _ll_register_callbacks cb 0x555cb1055aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:18.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.908+0000 7f1762a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb1150a80 0x555cb1158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:18.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.908+0000 7f1762a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb1150a80 0x555cb1158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55516/0 (socket says 172.21.15.69:55516) 2024-04-24T09:08:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.908+0000 7f1762a00640 1 -- 172.21.15.69:0/2966574963 learned_addr learned my addr 172.21.15.69:0/2966574963 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.909+0000 7f175be00640 1 --2- 172.21.15.69:0/2966574963 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb11513a0 0x555cb1158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.909+0000 7f1762a00640 1 -- 172.21.15.69:0/2966574963 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb11513a0 msgr2=0x555cb1158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.909+0000 7f1762a00640 1 --2- 172.21.15.69:0/2966574963 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb11513a0 0x555cb1158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.909+0000 7f1762a00640 1 -- 172.21.15.69:0/2966574963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb115ba70 msgr2=0x555cb11594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.909+0000 7f1762a00640 1 --2- 172.21.15.69:0/2966574963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb115ba70 0x555cb11594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.909+0000 7f1762a00640 1 -- 172.21.15.69:0/2966574963 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cb0ee8f70 con 0x555cb1150a80 2024-04-24T09:08:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.909+0000 7f175be00640 1 --2- 172.21.15.69:0/2966574963 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb11513a0 0x555cb1158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.909+0000 7f1762a00640 1 --2- 172.21.15.69:0/2966574963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb1150a80 0x555cb1158a50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1754033000 tx=0x7f175402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edc593a2d5b1bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.909+0000 7f1747e00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.909+0000 7f1747e00640 1 -- 172.21.15.69:0/2966574963 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175404d020 con 0x555cb1150a80 2024-04-24T09:08:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.909+0000 7f176c43c740 1 -- 172.21.15.69:0/2966574963 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f174c015070 con 0x555cb1150a80 2024-04-24T09:08:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.909+0000 7f1747e00640 1 -- 172.21.15.69:0/2966574963 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f1754004ad0 con 0x555cb1150a80 2024-04-24T09:08:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.910+0000 7f1747e00640 1 -- 172.21.15.69:0/2966574963 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1754058050 con 0x555cb1150a80 2024-04-24T09:08:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.910+0000 7f176c43c740 10 client.15999 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.910+0000 7f176c43c740 1 -- 172.21.15.69:0/2966574963 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555cb0d80170 con 0x555cb1150a80 2024-04-24T09:08:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.910+0000 7f1747e00640 1 -- 172.21.15.69:0/2966574963 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f174c015070 con 0x555cb1150a80 2024-04-24T09:08:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.910+0000 7f1747e00640 1 -- 172.21.15.69:0/2966574963 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f17540324b0 con 0x555cb1150a80 2024-04-24T09:08:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.910+0000 7f176c43c740 10 client.15999 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.910+0000 7f176c43c740 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.910+0000 7f176c43c740 20 client.15999 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.910+0000 7f176c43c740 1 --2- 172.21.15.69:0/2966574963 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x555cb116bf00 0x555cb116e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.910+0000 7f176c43c740 4 client.15999 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.910+0000 7f176c43c740 1 -- 172.21.15.69:0/2966574963 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cb10bb890 con 0x555cb116bf00 2024-04-24T09:08:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.911+0000 7f175be00640 1 --2- 172.21.15.69:0/2966574963 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x555cb116bf00 0x555cb116e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.912+0000 7f175be00640 1 --2- 172.21.15.69:0/2966574963 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x555cb116bf00 0x555cb116e3a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cad706b0aaaa597 server_cookie=83546f562b63831d in_seq=0 out_seq=0 2024-04-24T09:08:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.912+0000 7f1747e00640 10 client.15999 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.913+0000 7f1747e00640 1 -- 172.21.15.69:0/2966574963 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555cb10bb890 con 0x555cb116bf00 2024-04-24T09:08:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.913+0000 7f1747e00640 10 client.15999 handle_command_reply: tid=0 2024-04-24T09:08:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.914+0000 7f176c43c740 1 client.15999 shutdown 2024-04-24T09:08:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.914+0000 7f176c43c740 1 -- 172.21.15.69:0/2966574963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb1150a80 msgr2=0x555cb1158a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.914+0000 7f176c43c740 1 --2- 172.21.15.69:0/2966574963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb1150a80 0x555cb1158a50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1754033000 tx=0x7f175402a480 comp rx=0 tx=0).stop 2024-04-24T09:08:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.914+0000 7f176c43c740 1 -- 172.21.15.69:0/2966574963 shutdown_connections 2024-04-24T09:08:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.914+0000 7f176c43c740 1 --2- 172.21.15.69:0/2966574963 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x555cb116bf00 0x555cb116e3a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.914+0000 7f176c43c740 1 --2- 172.21.15.69:0/2966574963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb115ba70 0x555cb11594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.914+0000 7f176c43c740 1 --2- 172.21.15.69:0/2966574963 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb11513a0 0x555cb1158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.914+0000 7f176c43c740 1 --2- 172.21.15.69:0/2966574963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb1150a80 0x555cb1158a50 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.914+0000 7f176c43c740 1 -- 172.21.15.69:0/2966574963 >> 172.21.15.69:0/2966574963 conn(0x555cb114cda0 msgr2=0x555cb1152ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:18.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.914+0000 7f1747e00640 0 client.15999 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:18.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.914+0000 7f176c43c740 1 -- 172.21.15.69:0/2966574963 shutdown_connections 2024-04-24T09:08:18.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.914+0000 7f176c43c740 1 -- 172.21.15.69:0/2966574963 wait complete. 2024-04-24T09:08:18.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.914+0000 7f176c43c740 20 client.15999 trim_cache size 0 max 16384 2024-04-24T09:08:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.932+0000 7f176c43c740 1 Processor -- start 2024-04-24T09:08:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.933+0000 7f176c43c740 1 -- start start 2024-04-24T09:08:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.933+0000 7f176c43c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb1150a80 0x555cb1167780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.933+0000 7f176c43c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb11513a0 0x555cb1173cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.933+0000 7f176c43c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb115ba70 0x555cb11741f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.933+0000 7f176c43c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17641af730 con 0x555cb1150a80 2024-04-24T09:08:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.933+0000 7f176c43c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f17641ad000 con 0x555cb115ba70 2024-04-24T09:08:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.933+0000 7f176c43c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f17641aae70 con 0x555cb11513a0 2024-04-24T09:08:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.933+0000 7f1762a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb1150a80 0x555cb1167780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.933+0000 7f1762a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb1150a80 0x555cb1167780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55520/0 (socket says 172.21.15.69:55520) 2024-04-24T09:08:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.933+0000 7f1762a00640 1 -- 172.21.15.69:0/332595393 learned_addr learned my addr 172.21.15.69:0/332595393 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:18.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.933+0000 7f1763400640 1 --2- 172.21.15.69:0/332595393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb115ba70 0x555cb11741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:18.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.934+0000 7f1762a00640 1 -- 172.21.15.69:0/332595393 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb11513a0 msgr2=0x555cb1173cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:18.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.934+0000 7f1762a00640 1 --2- 172.21.15.69:0/332595393 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb11513a0 0x555cb1173cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.934+0000 7f1762a00640 1 -- 172.21.15.69:0/332595393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb115ba70 msgr2=0x555cb11741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:18.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.934+0000 7f1762a00640 1 --2- 172.21.15.69:0/332595393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb115ba70 0x555cb11741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.934+0000 7f1762a00640 1 -- 172.21.15.69:0/332595393 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cb113fe80 con 0x555cb1150a80 2024-04-24T09:08:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.934+0000 7f1763400640 1 --2- 172.21.15.69:0/332595393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb115ba70 0x555cb11741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.934+0000 7f1762a00640 1 --2- 172.21.15.69:0/332595393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb1150a80 0x555cb1167780 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f175402a9a0 tx=0x7f1754032f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1245ceb000f22345 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.934+0000 7f1752a00640 1 -- 172.21.15.69:0/332595393 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175404d070 con 0x555cb1150a80 2024-04-24T09:08:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.934+0000 7f1752a00640 1 -- 172.21.15.69:0/332595393 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1754060020 con 0x555cb1150a80 2024-04-24T09:08:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.934+0000 7f1752a00640 1 -- 172.21.15.69:0/332595393 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1754035040 con 0x555cb1150a80 2024-04-24T09:08:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.935+0000 7f176c43c740 1 -- 172.21.15.69:0/332595393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb1150a80 msgr2=0x555cb1167780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.935+0000 7f176c43c740 1 --2- 172.21.15.69:0/332595393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb1150a80 0x555cb1167780 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f175402a9a0 tx=0x7f1754032f10 comp rx=0 tx=0).stop 2024-04-24T09:08:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.935+0000 7f176c43c740 1 -- 172.21.15.69:0/332595393 shutdown_connections 2024-04-24T09:08:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.935+0000 7f176c43c740 1 --2- 172.21.15.69:0/332595393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb115ba70 0x555cb11741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.935+0000 7f176c43c740 1 --2- 172.21.15.69:0/332595393 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb11513a0 0x555cb1173cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.935+0000 7f176c43c740 1 --2- 172.21.15.69:0/332595393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb1150a80 0x555cb1167780 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.935+0000 7f176c43c740 1 -- 172.21.15.69:0/332595393 >> 172.21.15.69:0/332595393 conn(0x555cb114cda0 msgr2=0x555cb11436f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.935+0000 7f176c43c740 1 -- 172.21.15.69:0/332595393 shutdown_connections 2024-04-24T09:08:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.935+0000 7f176c43c740 1 -- 172.21.15.69:0/332595393 wait complete. 2024-04-24T09:08:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.936+0000 7f176c43c740 1 -- start start 2024-04-24T09:08:18.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.936+0000 7f176c43c740 1 Processor -- start 2024-04-24T09:08:18.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.936+0000 7f176c43c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb1150a80 0x555cb1163c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.936+0000 7f176c43c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb11513a0 0x555cb11641d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.936+0000 7f176c43c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb115ba70 0x555cb1160e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.936+0000 7f176c43c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17641ab4c0 con 0x555cb11513a0 2024-04-24T09:08:18.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.936+0000 7f176c43c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f17641ac930 con 0x555cb115ba70 2024-04-24T09:08:18.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.936+0000 7f176c43c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f17641abb90 con 0x555cb1150a80 2024-04-24T09:08:18.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.936+0000 7f176c43c740 10 client.0 _ll_register_callbacks cb 0x7f1754040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:18.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.937+0000 7f175be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb11513a0 0x555cb11641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:18.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.937+0000 7f175be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb11513a0 0x555cb11641d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55530/0 (socket says 172.21.15.69:55530) 2024-04-24T09:08:18.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.937+0000 7f175be00640 1 -- 172.21.15.69:0/1764228110 learned_addr learned my addr 172.21.15.69:0/1764228110 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.937+0000 7f1762a00640 1 --2- 172.21.15.69:0/1764228110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb1150a80 0x555cb1163c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.937+0000 7f1763400640 1 --2- 172.21.15.69:0/1764228110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb115ba70 0x555cb1160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.937+0000 7f175be00640 1 -- 172.21.15.69:0/1764228110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb1150a80 msgr2=0x555cb1163c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.937+0000 7f175be00640 1 --2- 172.21.15.69:0/1764228110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb1150a80 0x555cb1163c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.937+0000 7f175be00640 1 -- 172.21.15.69:0/1764228110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb115ba70 msgr2=0x555cb1160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.937+0000 7f175be00640 1 --2- 172.21.15.69:0/1764228110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb115ba70 0x555cb1160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.937+0000 7f175be00640 1 -- 172.21.15.69:0/1764228110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cb115de60 con 0x555cb11513a0 2024-04-24T09:08:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.937+0000 7f1763400640 1 --2- 172.21.15.69:0/1764228110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb115ba70 0x555cb1160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.937+0000 7f175be00640 1 --2- 172.21.15.69:0/1764228110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb11513a0 0x555cb11641d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x555cb114df30 tx=0x7f174c00e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb74be2f6d151c4c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.937+0000 7f1747e00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.938+0000 7f1747e00640 1 -- 172.21.15.69:0/1764228110 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c02d060 con 0x555cb11513a0 2024-04-24T09:08:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.938+0000 7f1747e00640 1 -- 172.21.15.69:0/1764228110 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f174c027170 con 0x555cb11513a0 2024-04-24T09:08:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.938+0000 7f176c43c740 1 -- 172.21.15.69:0/1764228110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1754060020 con 0x555cb11513a0 2024-04-24T09:08:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.938+0000 7f1747e00640 1 -- 172.21.15.69:0/1764228110 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f174c005610 con 0x555cb11513a0 2024-04-24T09:08:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.938+0000 7f176c43c740 10 client.16005 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.938+0000 7f176c43c740 1 -- 172.21.15.69:0/1764228110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555cb11d2f50 con 0x555cb11513a0 2024-04-24T09:08:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.938+0000 7f1747e00640 1 -- 172.21.15.69:0/1764228110 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1754060020 con 0x555cb11513a0 2024-04-24T09:08:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.939+0000 7f1747e00640 1 -- 172.21.15.69:0/1764228110 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f174c004af0 con 0x555cb11513a0 2024-04-24T09:08:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.939+0000 7f176c43c740 10 client.16005 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.939+0000 7f176c43c740 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.939+0000 7f176c43c740 20 client.16005 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.939+0000 7f176c43c740 1 --2- 172.21.15.69:0/1764228110 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x555cb11d3350 0x555cb11d5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.939+0000 7f176c43c740 4 client.16005 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:08:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.939+0000 7f176c43c740 1 -- 172.21.15.69:0/1764228110 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555cb10b5700 con 0x555cb11d3350 2024-04-24T09:08:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.939+0000 7f1762a00640 1 --2- 172.21.15.69:0/1764228110 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x555cb11d3350 0x555cb11d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:18.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.940+0000 7f1762a00640 1 --2- 172.21.15.69:0/1764228110 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x555cb11d3350 0x555cb11d5740 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6204dd18528cddb9 server_cookie=1998cc7016c8c448 in_seq=0 out_seq=0 2024-04-24T09:08:18.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.940+0000 7f1747e00640 10 client.16005 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:18.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.943+0000 7f1747e00640 1 -- 172.21.15.69:0/1764228110 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555cb10b5700 con 0x555cb11d3350 2024-04-24T09:08:18.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.943+0000 7f1747e00640 10 client.16005 handle_command_reply: tid=0 2024-04-24T09:08:18.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.943+0000 7f176c43c740 1 client.16005 shutdown 2024-04-24T09:08:18.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.943+0000 7f176c43c740 1 -- 172.21.15.69:0/1764228110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb11513a0 msgr2=0x555cb11641d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:18.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.943+0000 7f176c43c740 1 --2- 172.21.15.69:0/1764228110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb11513a0 0x555cb11641d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x555cb114df30 tx=0x7f174c00e420 comp rx=0 tx=0).stop 2024-04-24T09:08:18.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.943+0000 7f176c43c740 1 -- 172.21.15.69:0/1764228110 shutdown_connections 2024-04-24T09:08:18.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.943+0000 7f176c43c740 1 --2- 172.21.15.69:0/1764228110 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x555cb11d3350 0x555cb11d5740 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.943+0000 7f176c43c740 1 --2- 172.21.15.69:0/1764228110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555cb115ba70 0x555cb1160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.943+0000 7f176c43c740 1 --2- 172.21.15.69:0/1764228110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555cb11513a0 0x555cb11641d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.943+0000 7f176c43c740 1 --2- 172.21.15.69:0/1764228110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555cb1150a80 0x555cb1163c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.944+0000 7f176c43c740 1 -- 172.21.15.69:0/1764228110 >> 172.21.15.69:0/1764228110 conn(0x555cb114cda0 msgr2=0x555cb1142e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:18.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.944+0000 7f1747e00640 0 client.16005 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:18.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.944+0000 7f176c43c740 1 -- 172.21.15.69:0/1764228110 shutdown_connections 2024-04-24T09:08:18.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.944+0000 7f176c43c740 1 -- 172.21.15.69:0/1764228110 wait complete. 2024-04-24T09:08:18.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.944+0000 7f176c43c740 20 client.16005 trim_cache size 0 max 16384 2024-04-24T09:08:18.945 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:08:18.945 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-04-24T09:08:18.945 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "ba9693f0-7e96-405c-bc00-91191fd795cc", 2024-04-24T09:08:18.945 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-04-24T09:08:18.945 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:08:18.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.945+0000 7f1763e00640 1 -- 172.21.15.69:0/675745354 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f17480a3ec0 msgr2=0x7f17480a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:18.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.945+0000 7f1763e00640 1 --2- 172.21.15.69:0/675745354 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f17480a3ec0 0x7f17480a6380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1754006010 tx=0x7f1754031400 comp rx=0 tx=0).stop 2024-04-24T09:08:18.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.945+0000 7f1763e00640 1 -- 172.21.15.69:0/675745354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f176410e6e0 msgr2=0x7f176419a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:18.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.945+0000 7f1763e00640 1 --2- 172.21.15.69:0/675745354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f176410e6e0 0x7f176419a740 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f175c00cad0 tx=0x7f175c00cf90 comp rx=0 tx=0).stop 2024-04-24T09:08:18.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.945+0000 7f1763e00640 1 -- 172.21.15.69:0/675745354 shutdown_connections 2024-04-24T09:08:18.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.945+0000 7f1763e00640 1 --2- 172.21.15.69:0/675745354 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f17480a3ec0 0x7f17480a6380 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.945+0000 7f1763e00640 1 --2- 172.21.15.69:0/675745354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f176410e6e0 0x7f176419a740 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.945+0000 7f1763e00640 1 --2- 172.21.15.69:0/675745354 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1764074d50 0x7f17641a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.945+0000 7f1763e00640 1 --2- 172.21.15.69:0/675745354 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1764074430 0x7f17641a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:18.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.945+0000 7f1763e00640 1 -- 172.21.15.69:0/675745354 >> 172.21.15.69:0/675745354 conn(0x7f17640fd9d0 msgr2=0x7f17640fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:18.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.945+0000 7f1763e00640 1 -- 172.21.15.69:0/675745354 shutdown_connections 2024-04-24T09:08:18.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:18.945+0000 7f1763e00640 1 -- 172.21.15.69:0/675745354 wait complete. 2024-04-24T09:08:18.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:08:18.961 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:08:19.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:18 smithi069 ceph-mon[28170]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-24T09:08:19.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:18 smithi081 ceph-mon[30961]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-24T09:08:19.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.179+0000 7f19e1600640 1 -- 172.21.15.69:0/1914772549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19dc102110 msgr2=0x7f19dc1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:19.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.179+0000 7f19e1600640 1 --2- 172.21.15.69:0/1914772549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19dc102110 0x7f19dc1024f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009900 tx=0x7f19cc02a410 comp rx=0 tx=0).stop 2024-04-24T09:08:19.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.179+0000 7f19e1600640 1 -- 172.21.15.69:0/1914772549 shutdown_connections 2024-04-24T09:08:19.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.179+0000 7f19e1600640 1 --2- 172.21.15.69:0/1914772549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19dc1071e0 0x7f19dc10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.179+0000 7f19e1600640 1 --2- 172.21.15.69:0/1914772549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19dc102ac0 0x7f19dc106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.179+0000 7f19e1600640 1 --2- 172.21.15.69:0/1914772549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19dc102110 0x7f19dc1024f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.179+0000 7f19e1600640 1 -- 172.21.15.69:0/1914772549 >> 172.21.15.69:0/1914772549 conn(0x7f19dc0fd9d0 msgr2=0x7f19dc0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:19.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.179+0000 7f19e1600640 1 -- 172.21.15.69:0/1914772549 shutdown_connections 2024-04-24T09:08:19.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.179+0000 7f19e1600640 1 -- 172.21.15.69:0/1914772549 wait complete. 2024-04-24T09:08:19.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.181+0000 7f19e1600640 1 Processor -- start 2024-04-24T09:08:19.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.181+0000 7f19e1600640 1 -- start start 2024-04-24T09:08:19.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.181+0000 7f19e1600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19dc102110 0x7f19dc198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.181+0000 7f19e1600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19dc1071e0 0x7f19dc198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.181+0000 7f19e1600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19dc19cf90 0x7f19dc19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.182+0000 7f19e1600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc1121a0 con 0x7f19dc102110 2024-04-24T09:08:19.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.182+0000 7f19e1600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc112020 con 0x7f19dc1071e0 2024-04-24T09:08:19.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.182+0000 7f19e1600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc112320 con 0x7f19dc19cf90 2024-04-24T09:08:19.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.182+0000 7f19dbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19dc102110 0x7f19dc198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.182+0000 7f19dbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19dc102110 0x7f19dc198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55556/0 (socket says 172.21.15.69:55556) 2024-04-24T09:08:19.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.182+0000 7f19dbe00640 1 -- 172.21.15.69:0/1134113592 learned_addr learned my addr 172.21.15.69:0/1134113592 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:19.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.182+0000 7f19db400640 1 --2- 172.21.15.69:0/1134113592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19dc1071e0 0x7f19dc198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.182+0000 7f19dbe00640 1 -- 172.21.15.69:0/1134113592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19dc19cf90 msgr2=0x7f19dc19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:19.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.182+0000 7f19dbe00640 1 --2- 172.21.15.69:0/1134113592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19dc19cf90 0x7f19dc19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.182+0000 7f19dbe00640 1 -- 172.21.15.69:0/1134113592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19dc1071e0 msgr2=0x7f19dc198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:19.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.182+0000 7f19dbe00640 1 --2- 172.21.15.69:0/1134113592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19dc1071e0 0x7f19dc198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.182+0000 7f19dbe00640 1 -- 172.21.15.69:0/1134113592 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc19da80 con 0x7f19dc102110 2024-04-24T09:08:19.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.182+0000 7f19db400640 1 --2- 172.21.15.69:0/1134113592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19dc1071e0 0x7f19dc198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:19.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.183+0000 7f19dbe00640 1 --2- 172.21.15.69:0/1134113592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19dc102110 0x7f19dc198720 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f19cc005ab0 tx=0x7f19cc003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:19.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.183+0000 7f19d8c00640 1 -- 172.21.15.69:0/1134113592 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc03f070 con 0x7f19dc102110 2024-04-24T09:08:19.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.183+0000 7f19d8c00640 1 -- 172.21.15.69:0/1134113592 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19cc031d60 con 0x7f19dc102110 2024-04-24T09:08:19.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.183+0000 7f19d8c00640 1 -- 172.21.15.69:0/1134113592 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc035030 con 0x7f19dc102110 2024-04-24T09:08:19.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.183+0000 7f19e1600640 1 -- 172.21.15.69:0/1134113592 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19dc19dd10 con 0x7f19dc102110 2024-04-24T09:08:19.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.183+0000 7f19e1600640 1 -- 172.21.15.69:0/1134113592 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19dc1a5480 con 0x7f19dc102110 2024-04-24T09:08:19.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.189+0000 7f19e5986740 1 Processor -- start 2024-04-24T09:08:19.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.189+0000 7f19d8c00640 1 -- 172.21.15.69:0/1134113592 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f19cc042070 con 0x7f19dc102110 2024-04-24T09:08:19.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.189+0000 7f19e5986740 1 -- start start 2024-04-24T09:08:19.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.190+0000 7f19e5986740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9319a30 0x55d2f9319e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.190+0000 7f19e5986740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f931a350 0x55d2f93244e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.190+0000 7f19e5986740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f9324a20 0x55d2f9326e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.190+0000 7f19e5986740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d2f927f1f0 con 0x55d2f9319a30 2024-04-24T09:08:19.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.190+0000 7f19e0c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9319a30 0x55d2f9319e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.190+0000 7f19e0c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9319a30 0x55d2f9319e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55570/0 (socket says 172.21.15.69:55570) 2024-04-24T09:08:19.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.191+0000 7f19e0c00640 1 -- 172.21.15.69:0/3692474858 learned_addr learned my addr 172.21.15.69:0/3692474858 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:19.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.191+0000 7f19e5986740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55d2f9225350 con 0x55d2f931a350 2024-04-24T09:08:19.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.191+0000 7f19dbe00640 1 --2- 172.21.15.69:0/3692474858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f931a350 0x55d2f93244e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.191+0000 7f19e5986740 1 -- 172.21.15.69:0/3692474858 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d2f927b1b0 con 0x55d2f9324a20 2024-04-24T09:08:19.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.191+0000 7f19db400640 1 --2- 172.21.15.69:0/3692474858 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f9324a20 0x55d2f9326e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.191+0000 7f19d8c00640 1 --2- 172.21.15.69:0/1134113592 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f19c00a3e40 0x7f19c00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.191+0000 7f19e0c00640 1 -- 172.21.15.69:0/3692474858 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f9324a20 msgr2=0x55d2f9326e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:19.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.191+0000 7f19e0c00640 1 --2- 172.21.15.69:0/3692474858 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f9324a20 0x55d2f9326e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.191+0000 7f19e0c00640 1 -- 172.21.15.69:0/3692474858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f931a350 msgr2=0x55d2f93244e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:19.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.191+0000 7f19e0c00640 1 --2- 172.21.15.69:0/3692474858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f931a350 0x55d2f93244e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.191+0000 7f19e0c00640 1 -- 172.21.15.69:0/3692474858 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2f9279f50 con 0x55d2f9319a30 2024-04-24T09:08:19.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.192+0000 7f19d8c00640 1 -- 172.21.15.69:0/1134113592 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f19cc0f1720 con 0x7f19dc102110 2024-04-24T09:08:19.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.192+0000 7f19e0c00640 1 --2- 172.21.15.69:0/3692474858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9319a30 0x55d2f9319e10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009d10 tx=0x7f19d400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13be0604720260b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.193+0000 7f19e0c00640 1 --2- 172.21.15.69:0/1134113592 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f19c00a3e40 0x7f19c00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.193+0000 7f19d1200640 1 -- 172.21.15.69:0/3692474858 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4016070 con 0x55d2f9319a30 2024-04-24T09:08:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.193+0000 7f19d1200640 1 -- 172.21.15.69:0/3692474858 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19d4011b20 con 0x55d2f9319a30 2024-04-24T09:08:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.193+0000 7f19d1200640 1 -- 172.21.15.69:0/3692474858 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d401bd10 con 0x55d2f9319a30 2024-04-24T09:08:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.194+0000 7f19e5986740 1 -- 172.21.15.69:0/3692474858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9319a30 msgr2=0x55d2f9319e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.194+0000 7f19e5986740 1 --2- 172.21.15.69:0/3692474858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9319a30 0x55d2f9319e10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009d10 tx=0x7f19d400e6c0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.194+0000 7f19e0c00640 1 --2- 172.21.15.69:0/1134113592 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f19c00a3e40 0x7f19c00a6300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f19d4018000 tx=0x7f19d401bf90 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:19.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.194+0000 7f19e5986740 1 -- 172.21.15.69:0/3692474858 shutdown_connections 2024-04-24T09:08:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.194+0000 7f19e5986740 1 --2- 172.21.15.69:0/3692474858 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f9324a20 0x55d2f9326e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.194+0000 7f19e5986740 1 --2- 172.21.15.69:0/3692474858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f931a350 0x55d2f93244e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.194+0000 7f19e5986740 1 --2- 172.21.15.69:0/3692474858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9319a30 0x55d2f9319e10 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.194+0000 7f19e5986740 1 -- 172.21.15.69:0/3692474858 >> 172.21.15.69:0/3692474858 conn(0x55d2f9315d50 msgr2=0x55d2f9316130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.194+0000 7f19e5986740 1 -- 172.21.15.69:0/3692474858 shutdown_connections 2024-04-24T09:08:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.194+0000 7f19e5986740 1 -- 172.21.15.69:0/3692474858 wait complete. 2024-04-24T09:08:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.195+0000 7f19e5986740 1 -- start start 2024-04-24T09:08:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.195+0000 7f19e5986740 1 Processor -- start 2024-04-24T09:08:19.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.196+0000 7f19e5986740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f9319a30 0x55d2f9321970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.196+0000 7f19e5986740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f931a350 0x55d2f9321eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.196+0000 7f19e5986740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f9324a20 0x55d2f93223f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.196+0000 7f19e5986740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d2f9327810 con 0x55d2f931a350 2024-04-24T09:08:19.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.196+0000 7f19e5986740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55d2f9238690 con 0x55d2f9324a20 2024-04-24T09:08:19.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.196+0000 7f19e5986740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d2f9327990 con 0x55d2f9319a30 2024-04-24T09:08:19.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.196+0000 7f19e5986740 10 client.0 _ll_register_callbacks cb 0x55d2f9274650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:19.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.196+0000 7f19db400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f931a350 0x55d2f9321eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.196+0000 7f19db400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f931a350 0x55d2f9321eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55574/0 (socket says 172.21.15.69:55574) 2024-04-24T09:08:19.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.196+0000 7f19db400640 1 -- 172.21.15.69:0/3621654755 learned_addr learned my addr 172.21.15.69:0/3621654755 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:19.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.196+0000 7f19dbe00640 1 --2- 172.21.15.69:0/3621654755 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f9319a30 0x55d2f9321970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.197+0000 7f19e0c00640 1 --2- 172.21.15.69:0/3621654755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f9324a20 0x55d2f93223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.197+0000 7f19db400640 1 -- 172.21.15.69:0/3621654755 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f9319a30 msgr2=0x55d2f9321970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:19.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.197+0000 7f19db400640 1 --2- 172.21.15.69:0/3621654755 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f9319a30 0x55d2f9321970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.197+0000 7f19db400640 1 -- 172.21.15.69:0/3621654755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f9324a20 msgr2=0x55d2f93223f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:19.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.197+0000 7f19db400640 1 --2- 172.21.15.69:0/3621654755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f9324a20 0x55d2f93223f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.197+0000 7f19db400640 1 -- 172.21.15.69:0/3621654755 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2f9237920 con 0x55d2f931a350 2024-04-24T09:08:19.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.197+0000 7f19e0c00640 1 --2- 172.21.15.69:0/3621654755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f9324a20 0x55d2f93223f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:19.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.197+0000 7f19db400640 1 --2- 172.21.15.69:0/3621654755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f931a350 0x55d2f9321eb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f19c8010670 tx=0x7f19c8010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff39f16c6d61a42 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:19.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.197+0000 7f19c5600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:19.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.197+0000 7f19c5600640 1 -- 172.21.15.69:0/3621654755 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8004220 con 0x55d2f931a350 2024-04-24T09:08:19.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.197+0000 7f19c5600640 1 -- 172.21.15.69:0/3621654755 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f19c80043c0 con 0x55d2f931a350 2024-04-24T09:08:19.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.197+0000 7f19e5986740 1 -- 172.21.15.69:0/3621654755 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19d4011b20 con 0x55d2f931a350 2024-04-24T09:08:19.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.197+0000 7f19c5600640 1 -- 172.21.15.69:0/3621654755 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f19c8023050 con 0x55d2f931a350 2024-04-24T09:08:19.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.197+0000 7f19e5986740 10 client.16017 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.198+0000 7f19e5986740 1 -- 172.21.15.69:0/3621654755 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d2f905b7d0 con 0x55d2f931a350 2024-04-24T09:08:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.198+0000 7f19c5600640 1 -- 172.21.15.69:0/3621654755 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19d4011b20 con 0x55d2f931a350 2024-04-24T09:08:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.198+0000 7f19c5600640 1 -- 172.21.15.69:0/3621654755 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f19c800f040 con 0x55d2f931a350 2024-04-24T09:08:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.198+0000 7f19e5986740 10 client.16017 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.198+0000 7f19e5986740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.198+0000 7f19e5986740 20 client.16017 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.198+0000 7f19e5986740 1 --2- 172.21.15.69:0/3621654755 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55d2f93391e0 0x55d2f933b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.198+0000 7f19e5986740 4 client.16017 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.198+0000 7f19e5986740 1 -- 172.21.15.69:0/3621654755 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2f928cc10 con 0x55d2f93391e0 2024-04-24T09:08:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.199+0000 7f19dbe00640 1 --2- 172.21.15.69:0/3621654755 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55d2f93391e0 0x55d2f933b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.200+0000 7f19dbe00640 1 --2- 172.21.15.69:0/3621654755 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55d2f93391e0 0x55d2f933b640 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=802cbd4571cb53f6 server_cookie=c3a3b5afd34eac65 in_seq=0 out_seq=0 2024-04-24T09:08:19.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.200+0000 7f19c5600640 10 client.16017 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:19.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.202+0000 7f19c5600640 1 -- 172.21.15.69:0/3621654755 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2f928cc10 con 0x55d2f93391e0 2024-04-24T09:08:19.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.202+0000 7f19c5600640 10 client.16017 handle_command_reply: tid=0 2024-04-24T09:08:19.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.202+0000 7f19e5986740 1 client.16017 shutdown 2024-04-24T09:08:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.202+0000 7f19e5986740 1 -- 172.21.15.69:0/3621654755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f931a350 msgr2=0x55d2f9321eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.202+0000 7f19e5986740 1 --2- 172.21.15.69:0/3621654755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f931a350 0x55d2f9321eb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f19c8010670 tx=0x7f19c8010b30 comp rx=0 tx=0).stop 2024-04-24T09:08:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.203+0000 7f19e5986740 1 -- 172.21.15.69:0/3621654755 shutdown_connections 2024-04-24T09:08:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.203+0000 7f19e5986740 1 --2- 172.21.15.69:0/3621654755 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55d2f93391e0 0x55d2f933b640 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.203+0000 7f19e5986740 1 --2- 172.21.15.69:0/3621654755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f9324a20 0x55d2f93223f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.203+0000 7f19e5986740 1 --2- 172.21.15.69:0/3621654755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f931a350 0x55d2f9321eb0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.203+0000 7f19e5986740 1 --2- 172.21.15.69:0/3621654755 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f9319a30 0x55d2f9321970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.203+0000 7f19e5986740 1 -- 172.21.15.69:0/3621654755 >> 172.21.15.69:0/3621654755 conn(0x55d2f9315d50 msgr2=0x55d2f9314fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.203+0000 7f19c5600640 0 client.16017 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.203+0000 7f19e5986740 1 -- 172.21.15.69:0/3621654755 shutdown_connections 2024-04-24T09:08:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.203+0000 7f19e5986740 1 -- 172.21.15.69:0/3621654755 wait complete. 2024-04-24T09:08:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.203+0000 7f19e5986740 20 client.16017 trim_cache size 0 max 16384 2024-04-24T09:08:19.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:18 smithi130 ceph-mon[31553]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-24T09:08:19.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.239+0000 7f19e5986740 1 Processor -- start 2024-04-24T09:08:19.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.239+0000 7f19e5986740 1 -- start start 2024-04-24T09:08:19.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.239+0000 7f19e5986740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f9319a30 0x55d2f9330700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.239+0000 7f19e5986740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9324a20 0x55d2f9320ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.239+0000 7f19e5986740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f93241b0 0x55d2f9322040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.239+0000 7f19e5986740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc1af730 con 0x55d2f9324a20 2024-04-24T09:08:19.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.239+0000 7f19e5986740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc1ad000 con 0x55d2f93241b0 2024-04-24T09:08:19.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.239+0000 7f19e5986740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc1aae50 con 0x55d2f9319a30 2024-04-24T09:08:19.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.240+0000 7f19db400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9324a20 0x55d2f9320ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.240+0000 7f19db400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9324a20 0x55d2f9320ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55590/0 (socket says 172.21.15.69:55590) 2024-04-24T09:08:19.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.240+0000 7f19db400640 1 -- 172.21.15.69:0/2021348993 learned_addr learned my addr 172.21.15.69:0/2021348993 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:19.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.240+0000 7f19dbe00640 1 --2- 172.21.15.69:0/2021348993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f9319a30 0x55d2f9330700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.240+0000 7f19e0c00640 1 --2- 172.21.15.69:0/2021348993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f93241b0 0x55d2f9322040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.240+0000 7f19db400640 1 -- 172.21.15.69:0/2021348993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f9319a30 msgr2=0x55d2f9330700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:19.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.240+0000 7f19db400640 1 --2- 172.21.15.69:0/2021348993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f9319a30 0x55d2f9330700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.240+0000 7f19db400640 1 -- 172.21.15.69:0/2021348993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f93241b0 msgr2=0x55d2f9322040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:19.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.240+0000 7f19db400640 1 --2- 172.21.15.69:0/2021348993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f93241b0 0x55d2f9322040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.240+0000 7f19db400640 1 -- 172.21.15.69:0/2021348993 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2f8f913f0 con 0x55d2f9324a20 2024-04-24T09:08:19.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.241+0000 7f19db400640 1 --2- 172.21.15.69:0/2021348993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9324a20 0x55d2f9320ad0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f19c8003bf0 tx=0x7f19c8009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e728674a7f5cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:19.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.241+0000 7f19c7e00640 1 -- 172.21.15.69:0/2021348993 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c802c070 con 0x55d2f9324a20 2024-04-24T09:08:19.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.241+0000 7f19c7e00640 1 -- 172.21.15.69:0/2021348993 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19c800e6c0 con 0x55d2f9324a20 2024-04-24T09:08:19.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.241+0000 7f19c7e00640 1 -- 172.21.15.69:0/2021348993 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8004040 con 0x55d2f9324a20 2024-04-24T09:08:19.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.242+0000 7f19e5986740 1 -- 172.21.15.69:0/2021348993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9324a20 msgr2=0x55d2f9320ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:19.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.242+0000 7f19e5986740 1 --2- 172.21.15.69:0/2021348993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9324a20 0x55d2f9320ad0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f19c8003bf0 tx=0x7f19c8009e00 comp rx=0 tx=0).stop 2024-04-24T09:08:19.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.242+0000 7f19e5986740 1 -- 172.21.15.69:0/2021348993 shutdown_connections 2024-04-24T09:08:19.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.242+0000 7f19e5986740 1 --2- 172.21.15.69:0/2021348993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f93241b0 0x55d2f9322040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.242+0000 7f19e5986740 1 --2- 172.21.15.69:0/2021348993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9324a20 0x55d2f9320ad0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.242+0000 7f19e5986740 1 --2- 172.21.15.69:0/2021348993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f9319a30 0x55d2f9330700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.242+0000 7f19e5986740 1 -- 172.21.15.69:0/2021348993 >> 172.21.15.69:0/2021348993 conn(0x55d2f9315d50 msgr2=0x55d2f9321930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:19.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.242+0000 7f19e5986740 1 -- 172.21.15.69:0/2021348993 shutdown_connections 2024-04-24T09:08:19.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.242+0000 7f19e5986740 1 -- 172.21.15.69:0/2021348993 wait complete. 2024-04-24T09:08:19.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.243+0000 7f19e5986740 1 -- start start 2024-04-24T09:08:19.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.243+0000 7f19e5986740 1 Processor -- start 2024-04-24T09:08:19.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.244+0000 7f19e5986740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f9319a30 0x55d2f932cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.244+0000 7f19e5986740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f93241b0 0x55d2f932d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.244+0000 7f19e5986740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9324a20 0x55d2f9329be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.244+0000 7f19e5986740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc1ab4c0 con 0x55d2f9324a20 2024-04-24T09:08:19.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.244+0000 7f19e5986740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc1ac930 con 0x55d2f9319a30 2024-04-24T09:08:19.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.244+0000 7f19e5986740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc1abb90 con 0x55d2f93241b0 2024-04-24T09:08:19.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.244+0000 7f19e5986740 10 client.0 _ll_register_callbacks cb 0x7f19c8009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:19.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.244+0000 7f19e0c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9324a20 0x55d2f9329be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.244+0000 7f19e0c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9324a20 0x55d2f9329be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55596/0 (socket says 172.21.15.69:55596) 2024-04-24T09:08:19.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.244+0000 7f19e0c00640 1 -- 172.21.15.69:0/1745286639 learned_addr learned my addr 172.21.15.69:0/1745286639 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:19.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.244+0000 7f19dbe00640 1 --2- 172.21.15.69:0/1745286639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f9319a30 0x55d2f932cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.245+0000 7f19db400640 1 --2- 172.21.15.69:0/1745286639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f93241b0 0x55d2f932d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.245+0000 7f19e0c00640 1 -- 172.21.15.69:0/1745286639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f93241b0 msgr2=0x55d2f932d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:19.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.245+0000 7f19e0c00640 1 --2- 172.21.15.69:0/1745286639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f93241b0 0x55d2f932d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.245+0000 7f19e0c00640 1 -- 172.21.15.69:0/1745286639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f9319a30 msgr2=0x55d2f932cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:19.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.245+0000 7f19e0c00640 1 --2- 172.21.15.69:0/1745286639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f9319a30 0x55d2f932cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.245+0000 7f19e0c00640 1 -- 172.21.15.69:0/1745286639 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2f8f510f0 con 0x55d2f9324a20 2024-04-24T09:08:19.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.245+0000 7f19e0c00640 1 --2- 172.21.15.69:0/1745286639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9324a20 0x55d2f9329be0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f19d40092f0 tx=0x7f19d4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a02581f57cc2bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:19.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.245+0000 7f19c5600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:19.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.246+0000 7f19c5600640 1 -- 172.21.15.69:0/1745286639 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4016040 con 0x55d2f9324a20 2024-04-24T09:08:19.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.246+0000 7f19c5600640 1 -- 172.21.15.69:0/1745286639 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f19d4003970 con 0x55d2f9324a20 2024-04-24T09:08:19.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.246+0000 7f19e5986740 1 -- 172.21.15.69:0/1745286639 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19c800e6c0 con 0x55d2f9324a20 2024-04-24T09:08:19.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.246+0000 7f19c5600640 1 -- 172.21.15.69:0/1745286639 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f19d4003b70 con 0x55d2f9324a20 2024-04-24T09:08:19.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.246+0000 7f19e5986740 10 client.16023 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:19.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.246+0000 7f19e5986740 1 -- 172.21.15.69:0/1745286639 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d2f9244810 con 0x55d2f9324a20 2024-04-24T09:08:19.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.246+0000 7f19c5600640 1 -- 172.21.15.69:0/1745286639 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19c800e6c0 con 0x55d2f9324a20 2024-04-24T09:08:19.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.247+0000 7f19c5600640 1 -- 172.21.15.69:0/1745286639 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f19d400f040 con 0x55d2f9324a20 2024-04-24T09:08:19.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.247+0000 7f19e5986740 10 client.16023 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:19.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.247+0000 7f19e5986740 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:19.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.247+0000 7f19e5986740 20 client.16023 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:19.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.247+0000 7f19e5986740 1 --2- 172.21.15.69:0/1745286639 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55d2f93a05a0 0x55d2f93a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:19.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.247+0000 7f19e5986740 4 client.16023 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:08:19.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.247+0000 7f19e5986740 1 -- 172.21.15.69:0/1745286639 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d2f9286a80 con 0x55d2f93a05a0 2024-04-24T09:08:19.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.248+0000 7f19dbe00640 1 --2- 172.21.15.69:0/1745286639 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55d2f93a05a0 0x55d2f93a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:19.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.249+0000 7f19c5600640 10 client.16023 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:19.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.249+0000 7f19dbe00640 1 --2- 172.21.15.69:0/1745286639 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55d2f93a05a0 0x55d2f93a29c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2058ece8daf0192f server_cookie=3bb550850a9e15fd in_seq=0 out_seq=0 2024-04-24T09:08:19.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.249+0000 7f19c5600640 1 -- 172.21.15.69:0/1745286639 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d2f9286a80 con 0x55d2f93a05a0 2024-04-24T09:08:19.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.249+0000 7f19c5600640 10 client.16023 handle_command_reply: tid=0 2024-04-24T09:08:19.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.249+0000 7f19e5986740 1 client.16023 shutdown 2024-04-24T09:08:19.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.250+0000 7f19e5986740 1 -- 172.21.15.69:0/1745286639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9324a20 msgr2=0x55d2f9329be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:19.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.250+0000 7f19e5986740 1 --2- 172.21.15.69:0/1745286639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9324a20 0x55d2f9329be0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f19d40092f0 tx=0x7f19d4019000 comp rx=0 tx=0).stop 2024-04-24T09:08:19.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.250+0000 7f19e5986740 1 -- 172.21.15.69:0/1745286639 shutdown_connections 2024-04-24T09:08:19.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.250+0000 7f19e5986740 1 --2- 172.21.15.69:0/1745286639 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55d2f93a05a0 0x55d2f93a29c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.250+0000 7f19e5986740 1 --2- 172.21.15.69:0/1745286639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2f9324a20 0x55d2f9329be0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.250+0000 7f19e5986740 1 --2- 172.21.15.69:0/1745286639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2f93241b0 0x55d2f932d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.250+0000 7f19e5986740 1 --2- 172.21.15.69:0/1745286639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f9319a30 0x55d2f932cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.250+0000 7f19e5986740 1 -- 172.21.15.69:0/1745286639 >> 172.21.15.69:0/1745286639 conn(0x55d2f9315d50 msgr2=0x55d2f9323a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:19.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.250+0000 7f19c5600640 0 client.16023 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:19.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.250+0000 7f19e5986740 1 -- 172.21.15.69:0/1745286639 shutdown_connections 2024-04-24T09:08:19.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.250+0000 7f19e5986740 1 -- 172.21.15.69:0/1745286639 wait complete. 2024-04-24T09:08:19.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.250+0000 7f19e5986740 20 client.16023 trim_cache size 0 max 16384 2024-04-24T09:08:19.251 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:08:19.251 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:08:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-04-24T09:08:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: "ba9693f0-7e96-405c-bc00-91191fd795cc": { 2024-04-24T09:08:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-04-24T09:08:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "ba9693f0-7e96-405c-bc00-91191fd795cc", 2024-04-24T09:08:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-04-24T09:08:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:08:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:08:19.252 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:08:19.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.251+0000 7f19e1600640 1 -- 172.21.15.69:0/1134113592 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f19c00a3e40 msgr2=0x7f19c00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:19.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.251+0000 7f19e1600640 1 --2- 172.21.15.69:0/1134113592 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f19c00a3e40 0x7f19c00a6300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f19d4018000 tx=0x7f19d401bf90 comp rx=0 tx=0).stop 2024-04-24T09:08:19.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.251+0000 7f19e1600640 1 -- 172.21.15.69:0/1134113592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19dc102110 msgr2=0x7f19dc198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:19.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.251+0000 7f19e1600640 1 --2- 172.21.15.69:0/1134113592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19dc102110 0x7f19dc198720 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f19cc005ab0 tx=0x7f19cc003cd0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.251+0000 7f19e1600640 1 -- 172.21.15.69:0/1134113592 shutdown_connections 2024-04-24T09:08:19.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.251+0000 7f19e1600640 1 --2- 172.21.15.69:0/1134113592 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f19c00a3e40 0x7f19c00a6300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.251+0000 7f19e1600640 1 --2- 172.21.15.69:0/1134113592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19dc19cf90 0x7f19dc19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.251+0000 7f19e1600640 1 --2- 172.21.15.69:0/1134113592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19dc1071e0 0x7f19dc198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.251+0000 7f19e1600640 1 --2- 172.21.15.69:0/1134113592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19dc102110 0x7f19dc198720 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:19.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.251+0000 7f19e1600640 1 -- 172.21.15.69:0/1134113592 >> 172.21.15.69:0/1134113592 conn(0x7f19dc0fd9d0 msgr2=0x7f19dc0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:19.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.252+0000 7f19e1600640 1 -- 172.21.15.69:0/1134113592 shutdown_connections 2024-04-24T09:08:19.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:19.252+0000 7f19e1600640 1 -- 172.21.15.69:0/1134113592 wait complete. 2024-04-24T09:08:19.267 INFO:tasks.cephfs.filesystem:scrub status for tag:ba9693f0-7e96-405c-bc00-91191fd795cc - {'path': '/', 'tag': 'ba9693f0-7e96-405c-bc00-91191fd795cc', 'options': 'recursive,force'} 2024-04-24T09:08:20.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:19 smithi081 ceph-mon[30961]: scrub queued for path: / 2024-04-24T09:08:20.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:19 smithi081 ceph-mon[30961]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:20.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:19 smithi081 ceph-mon[30961]: scrub summary: active paths [/] 2024-04-24T09:08:20.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:19 smithi081 ceph-mon[30961]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:20.090 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:19 smithi069 ceph-mon[28170]: scrub queued for path: / 2024-04-24T09:08:20.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:19 smithi069 ceph-mon[28170]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:20.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:19 smithi069 ceph-mon[28170]: scrub summary: active paths [/] 2024-04-24T09:08:20.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:19 smithi069 ceph-mon[28170]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:20.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:19 smithi130 ceph-mon[31553]: scrub queued for path: / 2024-04-24T09:08:20.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:19 smithi130 ceph-mon[31553]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:20.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:19 smithi130 ceph-mon[31553]: scrub summary: active paths [/] 2024-04-24T09:08:20.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:19 smithi130 ceph-mon[31553]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.784+0000 7f845c800640 1 -- 172.21.15.69:0/3107220936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8454109520 msgr2=0x7f84541110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.784+0000 7f845c800640 1 --2- 172.21.15.69:0/3107220936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8454109520 0x7f84541110f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8450009ca0 tx=0x7f845002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.784+0000 7f845c800640 1 -- 172.21.15.69:0/3107220936 shutdown_connections 2024-04-24T09:08:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.784+0000 7f845c800640 1 --2- 172.21.15.69:0/3107220936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8454109520 0x7f84541110f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.784+0000 7f845c800640 1 --2- 172.21.15.69:0/3107220936 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8454104e00 0x7f8454108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.784+0000 7f845c800640 1 --2- 172.21.15.69:0/3107220936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8454104450 0x7f8454104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.784+0000 7f845c800640 1 -- 172.21.15.69:0/3107220936 >> 172.21.15.69:0/3107220936 conn(0x7f84540ffcf0 msgr2=0x7f8454102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.785+0000 7f845c800640 1 -- 172.21.15.69:0/3107220936 shutdown_connections 2024-04-24T09:08:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.785+0000 7f845c800640 1 -- 172.21.15.69:0/3107220936 wait complete. 2024-04-24T09:08:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.786+0000 7f845c800640 1 Processor -- start 2024-04-24T09:08:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.786+0000 7f845c800640 1 -- start start 2024-04-24T09:08:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.787+0000 7f845c800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8454104450 0x7f84541a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.787+0000 7f845c800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8454109520 0x7f84541a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.787+0000 7f845c800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f845419a710 0x7f845419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.787+0000 7f845c800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8454114490 con 0x7f8454109520 2024-04-24T09:08:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.787+0000 7f845c800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8454114310 con 0x7f845419a710 2024-04-24T09:08:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.787+0000 7f845c800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8454114610 con 0x7f8454104450 2024-04-24T09:08:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.787+0000 7f8459400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8454109520 0x7f84541a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.787+0000 7f8459400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8454109520 0x7f84541a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55620/0 (socket says 172.21.15.69:55620) 2024-04-24T09:08:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.787+0000 7f8459400640 1 -- 172.21.15.69:0/1803535732 learned_addr learned my addr 172.21.15.69:0/1803535732 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.788+0000 7f8459e00640 1 --2- 172.21.15.69:0/1803535732 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8454104450 0x7f84541a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.788+0000 7f845a800640 1 --2- 172.21.15.69:0/1803535732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f845419a710 0x7f845419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.788+0000 7f8459400640 1 -- 172.21.15.69:0/1803535732 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8454104450 msgr2=0x7f84541a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.788+0000 7f8459400640 1 --2- 172.21.15.69:0/1803535732 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8454104450 0x7f84541a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.788+0000 7f8459400640 1 -- 172.21.15.69:0/1803535732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f845419a710 msgr2=0x7f845419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.788+0000 7f8459400640 1 --2- 172.21.15.69:0/1803535732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f845419a710 0x7f845419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.788+0000 7f8459400640 1 -- 172.21.15.69:0/1803535732 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845419b310 con 0x7f8454109520 2024-04-24T09:08:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.788+0000 7f845a800640 1 --2- 172.21.15.69:0/1803535732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f845419a710 0x7f845419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.788+0000 7f8459400640 1 --2- 172.21.15.69:0/1803535732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8454109520 0x7f84541a0b90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f844400d740 tx=0x7f844400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.789+0000 7f844ea00640 1 -- 172.21.15.69:0/1803535732 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444017070 con 0x7f8454109520 2024-04-24T09:08:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.789+0000 7f844ea00640 1 -- 172.21.15.69:0/1803535732 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f844400de50 con 0x7f8454109520 2024-04-24T09:08:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.789+0000 7f844ea00640 1 -- 172.21.15.69:0/1803535732 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444004d60 con 0x7f8454109520 2024-04-24T09:08:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.789+0000 7f845c800640 1 -- 172.21.15.69:0/1803535732 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84541a73f0 con 0x7f8454109520 2024-04-24T09:08:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.789+0000 7f845c800640 1 -- 172.21.15.69:0/1803535732 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84541a77f0 con 0x7f8454109520 2024-04-24T09:08:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.794+0000 7f845f662740 1 Processor -- start 2024-04-24T09:08:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.794+0000 7f845f662740 1 -- start start 2024-04-24T09:08:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.795+0000 7f844ea00640 1 -- 172.21.15.69:0/1803535732 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f8444004f00 con 0x7f8454109520 2024-04-24T09:08:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.795+0000 7f845f662740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c3e0f0 0x5596f6c3e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.795+0000 7f845f662740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c3ea10 0x5596f6c51660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.795+0000 7f845f662740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c51ba0 0x5596f6c53f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.795+0000 7f845f662740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5596f6ba4900 con 0x5596f6c51ba0 2024-04-24T09:08:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.795+0000 7f845f662740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5596f6b4af20 con 0x5596f6c3e0f0 2024-04-24T09:08:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.795+0000 7f845f662740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5596f6ba08c0 con 0x5596f6c3ea10 2024-04-24T09:08:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.795+0000 7f8459400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c51ba0 0x5596f6c53f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.796+0000 7f845a800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c3e0f0 0x5596f6c3e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.796+0000 7f8459400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c51ba0 0x5596f6c53f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55626/0 (socket says 172.21.15.69:55626) 2024-04-24T09:08:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.796+0000 7f8459400640 1 -- 172.21.15.69:0/2420324436 learned_addr learned my addr 172.21.15.69:0/2420324436 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.796+0000 7f8459e00640 1 --2- 172.21.15.69:0/2420324436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c3ea10 0x5596f6c51660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.796+0000 7f8459400640 1 -- 172.21.15.69:0/2420324436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c3ea10 msgr2=0x5596f6c51660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.796+0000 7f8459400640 1 --2- 172.21.15.69:0/2420324436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c3ea10 0x5596f6c51660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.796+0000 7f8459400640 1 -- 172.21.15.69:0/2420324436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c3e0f0 msgr2=0x5596f6c3e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.796+0000 7f8459400640 1 --2- 172.21.15.69:0/2420324436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c3e0f0 0x5596f6c3e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.796+0000 7f8459400640 1 -- 172.21.15.69:0/2420324436 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f6b6fe10 con 0x5596f6c51ba0 2024-04-24T09:08:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.797+0000 7f8459400640 1 --2- 172.21.15.69:0/2420324436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c51ba0 0x5596f6c53f90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f84440cd960 tx=0x7f84440d8670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f98f9f10a225aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.797+0000 7f8443000640 1 -- 172.21.15.69:0/2420324436 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84440df070 con 0x5596f6c51ba0 2024-04-24T09:08:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.797+0000 7f8443000640 1 -- 172.21.15.69:0/2420324436 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84440d0d80 con 0x5596f6c51ba0 2024-04-24T09:08:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.797+0000 7f8443000640 1 -- 172.21.15.69:0/2420324436 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84440db680 con 0x5596f6c51ba0 2024-04-24T09:08:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.798+0000 7f844ea00640 1 --2- 172.21.15.69:0/1803535732 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f843c0a3fb0 0x7f843c0a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.798+0000 7f844ea00640 1 -- 172.21.15.69:0/1803535732 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f84440d0790 con 0x7f8454109520 2024-04-24T09:08:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.798+0000 7f845a800640 1 --2- 172.21.15.69:0/1803535732 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f843c0a3fb0 0x7f843c0a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.798+0000 7f845f662740 1 -- 172.21.15.69:0/2420324436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c51ba0 msgr2=0x5596f6c53f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.798+0000 7f845f662740 1 --2- 172.21.15.69:0/2420324436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c51ba0 0x5596f6c53f90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f84440cd960 tx=0x7f84440d8670 comp rx=0 tx=0).stop 2024-04-24T09:08:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.799+0000 7f845f662740 1 -- 172.21.15.69:0/2420324436 shutdown_connections 2024-04-24T09:08:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.799+0000 7f845f662740 1 --2- 172.21.15.69:0/2420324436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c51ba0 0x5596f6c53f90 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.799+0000 7f845f662740 1 --2- 172.21.15.69:0/2420324436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c3ea10 0x5596f6c51660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.799+0000 7f845f662740 1 --2- 172.21.15.69:0/2420324436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c3e0f0 0x5596f6c3e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.799+0000 7f845f662740 1 -- 172.21.15.69:0/2420324436 >> 172.21.15.69:0/2420324436 conn(0x5596f6c3a6c0 msgr2=0x5596f6c3aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.799+0000 7f845f662740 1 -- 172.21.15.69:0/2420324436 shutdown_connections 2024-04-24T09:08:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.799+0000 7f845f662740 1 -- 172.21.15.69:0/2420324436 wait complete. 2024-04-24T09:08:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.799+0000 7f845a800640 1 --2- 172.21.15.69:0/1803535732 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f843c0a3fb0 0x7f843c0a6470 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f845002f3a0 tx=0x7f8450009800 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.799+0000 7f845f662740 1 -- start start 2024-04-24T09:08:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.800+0000 7f845f662740 1 Processor -- start 2024-04-24T09:08:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.800+0000 7f845f662740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3e0f0 0x5596f6c4eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.800+0000 7f845f662740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c3ea10 0x5596f6c4eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.800+0000 7f845f662740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c51ba0 0x5596f6c4f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.800+0000 7f845f662740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5596f6c54770 con 0x5596f6c3e0f0 2024-04-24T09:08:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.800+0000 7f845f662740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5596f6b5e2c0 con 0x5596f6c51ba0 2024-04-24T09:08:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.800+0000 7f845f662740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5596f6c548f0 con 0x5596f6c3ea10 2024-04-24T09:08:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.801+0000 7f845f662740 10 client.0 _ll_register_callbacks cb 0x5596f68b6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.801+0000 7f8459e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3e0f0 0x5596f6c4eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.801+0000 7f8459e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3e0f0 0x5596f6c4eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55640/0 (socket says 172.21.15.69:55640) 2024-04-24T09:08:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.801+0000 7f8459e00640 1 -- 172.21.15.69:0/4227608324 learned_addr learned my addr 172.21.15.69:0/4227608324 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.801+0000 7f8459400640 1 --2- 172.21.15.69:0/4227608324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c3ea10 0x5596f6c4eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.801+0000 7f845a800640 1 --2- 172.21.15.69:0/4227608324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c51ba0 0x5596f6c4f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.801+0000 7f8459e00640 1 -- 172.21.15.69:0/4227608324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c3ea10 msgr2=0x5596f6c4eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.801+0000 7f8459e00640 1 --2- 172.21.15.69:0/4227608324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c3ea10 0x5596f6c4eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.801+0000 7f8459e00640 1 -- 172.21.15.69:0/4227608324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c51ba0 msgr2=0x5596f6c4f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.801+0000 7f8459e00640 1 --2- 172.21.15.69:0/4227608324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c51ba0 0x5596f6c4f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.801+0000 7f8459e00640 1 -- 172.21.15.69:0/4227608324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f6b6efd0 con 0x5596f6c3e0f0 2024-04-24T09:08:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.801+0000 7f845a800640 1 --2- 172.21.15.69:0/4227608324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c51ba0 0x5596f6c4f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.802+0000 7f8459e00640 1 --2- 172.21.15.69:0/4227608324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3e0f0 0x5596f6c4eab0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f844800be70 tx=0x7f84480117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed07e8659094b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.802+0000 7f8437400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.802+0000 7f8437400640 1 -- 172.21.15.69:0/4227608324 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8448004220 con 0x5596f6c3e0f0 2024-04-24T09:08:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.802+0000 7f8437400640 1 -- 172.21.15.69:0/4227608324 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f84480043c0 con 0x5596f6c3e0f0 2024-04-24T09:08:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.802+0000 7f845f662740 1 -- 172.21.15.69:0/4227608324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84440d0d80 con 0x5596f6c3e0f0 2024-04-24T09:08:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.802+0000 7f8437400640 1 -- 172.21.15.69:0/4227608324 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8448023050 con 0x5596f6c3e0f0 2024-04-24T09:08:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.802+0000 7f845f662740 10 client.16035 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.802+0000 7f845f662740 1 -- 172.21.15.69:0/4227608324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5596f68b7210 con 0x5596f6c3e0f0 2024-04-24T09:08:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.803+0000 7f8437400640 1 -- 172.21.15.69:0/4227608324 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84440d0d80 con 0x5596f6c3e0f0 2024-04-24T09:08:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.803+0000 7f8437400640 1 -- 172.21.15.69:0/4227608324 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f84480038f0 con 0x5596f6c3e0f0 2024-04-24T09:08:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.803+0000 7f845f662740 10 client.16035 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.803+0000 7f845f662740 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.803+0000 7f845f662740 20 client.16035 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.803+0000 7f845f662740 1 --2- 172.21.15.69:0/4227608324 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5596f6c61d80 0x5596f6c64170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.803+0000 7f845f662740 4 client.16035 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.803+0000 7f845f662740 1 -- 172.21.15.69:0/4227608324 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596f6bb2320 con 0x5596f6c61d80 2024-04-24T09:08:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.804+0000 7f8459400640 1 --2- 172.21.15.69:0/4227608324 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5596f6c61d80 0x5596f6c64170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.805+0000 7f8459400640 1 --2- 172.21.15.69:0/4227608324 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5596f6c61d80 0x5596f6c64170 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af4dd8b5826845f9 server_cookie=2bb2f3d3b48065c2 in_seq=0 out_seq=0 2024-04-24T09:08:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.805+0000 7f8437400640 10 client.16035 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:20.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.806+0000 7f8437400640 1 -- 172.21.15.69:0/4227608324 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5596f6bb2320 con 0x5596f6c61d80 2024-04-24T09:08:20.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.806+0000 7f8437400640 10 client.16035 handle_command_reply: tid=0 2024-04-24T09:08:20.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.807+0000 7f845f662740 1 client.16035 shutdown 2024-04-24T09:08:20.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.807+0000 7f845f662740 1 -- 172.21.15.69:0/4227608324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3e0f0 msgr2=0x5596f6c4eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:20.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.807+0000 7f845f662740 1 --2- 172.21.15.69:0/4227608324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3e0f0 0x5596f6c4eab0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f844800be70 tx=0x7f84480117b0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.807+0000 7f845f662740 1 -- 172.21.15.69:0/4227608324 shutdown_connections 2024-04-24T09:08:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.807+0000 7f845f662740 1 --2- 172.21.15.69:0/4227608324 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5596f6c61d80 0x5596f6c64170 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.807+0000 7f845f662740 1 --2- 172.21.15.69:0/4227608324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c51ba0 0x5596f6c4f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.807+0000 7f845f662740 1 --2- 172.21.15.69:0/4227608324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c3ea10 0x5596f6c4eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.807+0000 7f845f662740 1 --2- 172.21.15.69:0/4227608324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3e0f0 0x5596f6c4eab0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.807+0000 7f845f662740 1 -- 172.21.15.69:0/4227608324 >> 172.21.15.69:0/4227608324 conn(0x5596f6c3a6c0 msgr2=0x5596f6c40440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.807+0000 7f8437400640 0 client.16035 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.807+0000 7f845f662740 1 -- 172.21.15.69:0/4227608324 shutdown_connections 2024-04-24T09:08:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.807+0000 7f845f662740 1 -- 172.21.15.69:0/4227608324 wait complete. 2024-04-24T09:08:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.808+0000 7f845f662740 20 client.16035 trim_cache size 0 max 16384 2024-04-24T09:08:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.823+0000 7f845f662740 1 Processor -- start 2024-04-24T09:08:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.823+0000 7f845f662740 1 -- start start 2024-04-24T09:08:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.823+0000 7f845f662740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c3e0f0 0x5596f6c5d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.823+0000 7f845f662740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3ea10 0x5596f6c396c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.823+0000 7f845f662740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c51ba0 0x5596f6c69a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.823+0000 7f845f662740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f84541af880 con 0x5596f6c3ea10 2024-04-24T09:08:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.823+0000 7f845f662740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f84541ad150 con 0x5596f6c51ba0 2024-04-24T09:08:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.823+0000 7f845f662740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f84541aafa0 con 0x5596f6c3e0f0 2024-04-24T09:08:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.824+0000 7f8459400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3ea10 0x5596f6c396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.824+0000 7f8459400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3ea10 0x5596f6c396c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55644/0 (socket says 172.21.15.69:55644) 2024-04-24T09:08:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.824+0000 7f8459400640 1 -- 172.21.15.69:0/1606324501 learned_addr learned my addr 172.21.15.69:0/1606324501 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.824+0000 7f8459e00640 1 --2- 172.21.15.69:0/1606324501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c3e0f0 0x5596f6c5d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.824+0000 7f845a800640 1 --2- 172.21.15.69:0/1606324501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c51ba0 0x5596f6c69a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.824+0000 7f8459400640 1 -- 172.21.15.69:0/1606324501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c3e0f0 msgr2=0x5596f6c5d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.825+0000 7f8459400640 1 --2- 172.21.15.69:0/1606324501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c3e0f0 0x5596f6c5d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.825+0000 7f8459400640 1 -- 172.21.15.69:0/1606324501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c51ba0 msgr2=0x5596f6c69a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.825+0000 7f8459400640 1 --2- 172.21.15.69:0/1606324501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c51ba0 0x5596f6c69a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.825+0000 7f8459400640 1 -- 172.21.15.69:0/1606324501 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f6c35ea0 con 0x5596f6c3ea10 2024-04-24T09:08:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.825+0000 7f8459400640 1 --2- 172.21.15.69:0/1606324501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3ea10 0x5596f6c396c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f844409c0d0 tx=0x7f84440d2870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ab29d591e3e3e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.825+0000 7f8442600640 1 -- 172.21.15.69:0/1606324501 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444020040 con 0x5596f6c3ea10 2024-04-24T09:08:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.825+0000 7f8442600640 1 -- 172.21.15.69:0/1606324501 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84440046c0 con 0x5596f6c3ea10 2024-04-24T09:08:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.825+0000 7f8442600640 1 -- 172.21.15.69:0/1606324501 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84440df040 con 0x5596f6c3ea10 2024-04-24T09:08:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.826+0000 7f845f662740 1 -- 172.21.15.69:0/1606324501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3ea10 msgr2=0x5596f6c396c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.826+0000 7f845f662740 1 --2- 172.21.15.69:0/1606324501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3ea10 0x5596f6c396c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f844409c0d0 tx=0x7f84440d2870 comp rx=0 tx=0).stop 2024-04-24T09:08:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.826+0000 7f845f662740 1 -- 172.21.15.69:0/1606324501 shutdown_connections 2024-04-24T09:08:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.826+0000 7f845f662740 1 --2- 172.21.15.69:0/1606324501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c51ba0 0x5596f6c69a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.826+0000 7f845f662740 1 --2- 172.21.15.69:0/1606324501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3ea10 0x5596f6c396c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.826+0000 7f845f662740 1 --2- 172.21.15.69:0/1606324501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c3e0f0 0x5596f6c5d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.826+0000 7f845f662740 1 -- 172.21.15.69:0/1606324501 >> 172.21.15.69:0/1606324501 conn(0x5596f6c3a6c0 msgr2=0x5596f6c38fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.826+0000 7f845f662740 1 -- 172.21.15.69:0/1606324501 shutdown_connections 2024-04-24T09:08:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.826+0000 7f845f662740 1 -- 172.21.15.69:0/1606324501 wait complete. 2024-04-24T09:08:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.827+0000 7f845f662740 1 -- start start 2024-04-24T09:08:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.827+0000 7f845f662740 1 Processor -- start 2024-04-24T09:08:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.827+0000 7f845f662740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3e0f0 0x5596f6c4baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.827+0000 7f845f662740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c3ea10 0x5596f6c4c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.827+0000 7f845f662740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c51ba0 0x5596f6cc72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.827+0000 7f845f662740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f84541ab610 con 0x5596f6c3e0f0 2024-04-24T09:08:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.827+0000 7f845f662740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f84541aca80 con 0x5596f6c3ea10 2024-04-24T09:08:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.827+0000 7f845f662740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f84541abce0 con 0x5596f6c51ba0 2024-04-24T09:08:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.827+0000 7f845f662740 10 client.0 _ll_register_callbacks cb 0x7f844800b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.828+0000 7f8459e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3e0f0 0x5596f6c4baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.828+0000 7f8459e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3e0f0 0x5596f6c4baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55656/0 (socket says 172.21.15.69:55656) 2024-04-24T09:08:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.828+0000 7f8459e00640 1 -- 172.21.15.69:0/1849123027 learned_addr learned my addr 172.21.15.69:0/1849123027 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.828+0000 7f8459400640 1 --2- 172.21.15.69:0/1849123027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c3ea10 0x5596f6c4c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.828+0000 7f845a800640 1 --2- 172.21.15.69:0/1849123027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c51ba0 0x5596f6cc72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.829+0000 7f8459e00640 1 -- 172.21.15.69:0/1849123027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c51ba0 msgr2=0x5596f6cc72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.829+0000 7f8459e00640 1 --2- 172.21.15.69:0/1849123027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c51ba0 0x5596f6cc72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.829+0000 7f8459e00640 1 -- 172.21.15.69:0/1849123027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c3ea10 msgr2=0x5596f6c4c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.829+0000 7f8459e00640 1 --2- 172.21.15.69:0/1849123027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c3ea10 0x5596f6c4c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.829+0000 7f8459e00640 1 -- 172.21.15.69:0/1849123027 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f6c3e840 con 0x5596f6c3e0f0 2024-04-24T09:08:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.829+0000 7f8459400640 1 --2- 172.21.15.69:0/1849123027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c3ea10 0x5596f6c4c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.829+0000 7f8459e00640 1 --2- 172.21.15.69:0/1849123027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3e0f0 0x5596f6c4baf0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f844800be40 tx=0x7f844800b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe909711131d76d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.829+0000 7f8437400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.830+0000 7f8437400640 1 -- 172.21.15.69:0/1849123027 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84480038c0 con 0x5596f6c3e0f0 2024-04-24T09:08:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.830+0000 7f8437400640 1 -- 172.21.15.69:0/1849123027 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f8448020040 con 0x5596f6c3e0f0 2024-04-24T09:08:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.830+0000 7f845f662740 1 -- 172.21.15.69:0/1849123027 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5596f6bac190 con 0x5596f6c3e0f0 2024-04-24T09:08:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.830+0000 7f8437400640 1 -- 172.21.15.69:0/1849123027 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8448038050 con 0x5596f6c3e0f0 2024-04-24T09:08:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.830+0000 7f845f662740 10 client.16041 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.830+0000 7f845f662740 1 -- 172.21.15.69:0/1849123027 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5596f6983620 con 0x5596f6c3e0f0 2024-04-24T09:08:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.830+0000 7f8437400640 1 -- 172.21.15.69:0/1849123027 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5596f6bac190 con 0x5596f6c3e0f0 2024-04-24T09:08:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.831+0000 7f8437400640 1 -- 172.21.15.69:0/1849123027 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f84480053a0 con 0x5596f6c3e0f0 2024-04-24T09:08:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.831+0000 7f845f662740 10 client.16041 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.831+0000 7f845f662740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.831+0000 7f845f662740 20 client.16041 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.831+0000 7f845f662740 1 --2- 172.21.15.69:0/1849123027 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5596f6cc8d60 0x5596f6ccb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.831+0000 7f845f662740 4 client.16041 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:08:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.831+0000 7f845f662740 1 -- 172.21.15.69:0/1849123027 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596f6b284b0 con 0x5596f6cc8d60 2024-04-24T09:08:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.831+0000 7f8459400640 1 --2- 172.21.15.69:0/1849123027 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5596f6cc8d60 0x5596f6ccb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.832+0000 7f8459400640 1 --2- 172.21.15.69:0/1849123027 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5596f6cc8d60 0x5596f6ccb180 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce0e43bf5ccdf7b5 server_cookie=646fe0d8486735b4 in_seq=0 out_seq=0 2024-04-24T09:08:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.832+0000 7f8437400640 10 client.16041 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.841+0000 7f8437400640 1 -- 172.21.15.69:0/1849123027 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5596f6b284b0 con 0x5596f6cc8d60 2024-04-24T09:08:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.841+0000 7f8437400640 10 client.16041 handle_command_reply: tid=0 2024-04-24T09:08:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.841+0000 7f845f662740 1 client.16041 shutdown 2024-04-24T09:08:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.841+0000 7f845f662740 1 -- 172.21.15.69:0/1849123027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3e0f0 msgr2=0x5596f6c4baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.841+0000 7f845f662740 1 --2- 172.21.15.69:0/1849123027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3e0f0 0x5596f6c4baf0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f844800be40 tx=0x7f844800b770 comp rx=0 tx=0).stop 2024-04-24T09:08:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.841+0000 7f845f662740 1 -- 172.21.15.69:0/1849123027 shutdown_connections 2024-04-24T09:08:20.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.841+0000 7f845f662740 1 --2- 172.21.15.69:0/1849123027 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5596f6cc8d60 0x5596f6ccb180 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.841+0000 7f845f662740 1 --2- 172.21.15.69:0/1849123027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5596f6c51ba0 0x5596f6cc72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.841+0000 7f8437400640 0 client.16041 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.841+0000 7f845f662740 1 --2- 172.21.15.69:0/1849123027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596f6c3ea10 0x5596f6c4c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.841+0000 7f845f662740 1 --2- 172.21.15.69:0/1849123027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596f6c3e0f0 0x5596f6c4baf0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.841+0000 7f845f662740 1 -- 172.21.15.69:0/1849123027 >> 172.21.15.69:0/1849123027 conn(0x5596f6c3a6c0 msgr2=0x5596f6c4f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:20.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.842+0000 7f845f662740 1 -- 172.21.15.69:0/1849123027 shutdown_connections 2024-04-24T09:08:20.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.842+0000 7f845f662740 1 -- 172.21.15.69:0/1849123027 wait complete. 2024-04-24T09:08:20.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.842+0000 7f845f662740 20 client.16041 trim_cache size 0 max 16384 2024-04-24T09:08:20.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:08:20.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:08:20.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:08:20.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:08:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.843+0000 7f845c800640 1 -- 172.21.15.69:0/1803535732 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f843c0a3fb0 msgr2=0x7f843c0a6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.843+0000 7f845c800640 1 --2- 172.21.15.69:0/1803535732 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f843c0a3fb0 0x7f843c0a6470 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f845002f3a0 tx=0x7f8450009800 comp rx=0 tx=0).stop 2024-04-24T09:08:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.843+0000 7f845c800640 1 -- 172.21.15.69:0/1803535732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8454109520 msgr2=0x7f84541a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.843+0000 7f845c800640 1 --2- 172.21.15.69:0/1803535732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8454109520 0x7f84541a0b90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f844400d740 tx=0x7f844400dc00 comp rx=0 tx=0).stop 2024-04-24T09:08:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.843+0000 7f845c800640 1 -- 172.21.15.69:0/1803535732 shutdown_connections 2024-04-24T09:08:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.843+0000 7f845c800640 1 --2- 172.21.15.69:0/1803535732 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f843c0a3fb0 0x7f843c0a6470 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.843+0000 7f845c800640 1 --2- 172.21.15.69:0/1803535732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f845419a710 0x7f845419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.843+0000 7f845c800640 1 --2- 172.21.15.69:0/1803535732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8454109520 0x7f84541a0b90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.843+0000 7f845c800640 1 --2- 172.21.15.69:0/1803535732 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8454104450 0x7f84541a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.843+0000 7f845c800640 1 -- 172.21.15.69:0/1803535732 >> 172.21.15.69:0/1803535732 conn(0x7f84540ffcf0 msgr2=0x7f84541008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.843+0000 7f845c800640 1 -- 172.21.15.69:0/1803535732 shutdown_connections 2024-04-24T09:08:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:20.843+0000 7f845c800640 1 -- 172.21.15.69:0/1803535732 wait complete. 2024-04-24T09:08:21.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:20 smithi081 ceph-mon[30961]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-24T09:08:21.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:20 smithi069 ceph-mon[28170]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-24T09:08:21.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:20 smithi130 ceph-mon[31553]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-24T09:08:23.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:22 smithi081 ceph-mon[30961]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-24T09:08:23.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:22 smithi081 ceph-mon[30961]: scrub summary: idle 2024-04-24T09:08:23.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:22 smithi069 ceph-mon[28170]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-24T09:08:23.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:22 smithi069 ceph-mon[28170]: scrub summary: idle 2024-04-24T09:08:23.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:22 smithi130 ceph-mon[31553]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-24T09:08:23.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:22 smithi130 ceph-mon[31553]: scrub summary: idle 2024-04-24T09:08:25.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:24 smithi081 ceph-mon[30961]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-24T09:08:25.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:24 smithi069 ceph-mon[28170]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-24T09:08:25.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:24 smithi130 ceph-mon[31553]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-24T09:08:27.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:27 smithi081 ceph-mon[30961]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T09:08:27.340 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:27 smithi069 ceph-mon[28170]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T09:08:27.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:27 smithi130 ceph-mon[31553]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T09:08:29.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:29 smithi081 ceph-mon[30961]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:08:29.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:29 smithi069 ceph-mon[28170]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:08:29.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:29 smithi130 ceph-mon[31553]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T09:08:31.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:31 smithi081 ceph-mon[30961]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T09:08:31.340 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:31 smithi069 ceph-mon[28170]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T09:08:31.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:31 smithi130 ceph-mon[31553]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T09:08:33.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:33 smithi081 ceph-mon[30961]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:08:33.340 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:33 smithi069 ceph-mon[28170]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:08:33.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:33 smithi130 ceph-mon[31553]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:08:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.028+0000 7f4d0f000640 1 -- 172.21.15.69:0/869405073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d08101910 msgr2=0x7f4d08105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.028+0000 7f4d0f000640 1 --2- 172.21.15.69:0/869405073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d08101910 0x7f4d08105950 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009900 tx=0x7f4cf402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.029+0000 7f4d0f000640 1 -- 172.21.15.69:0/869405073 shutdown_connections 2024-04-24T09:08:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.029+0000 7f4d0f000640 1 --2- 172.21.15.69:0/869405073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d08106030 0x7f4d0810dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.029+0000 7f4d0f000640 1 --2- 172.21.15.69:0/869405073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d08101910 0x7f4d08105950 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.029+0000 7f4d0f000640 1 --2- 172.21.15.69:0/869405073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d08100f60 0x7f4d08101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.029+0000 7f4d0f000640 1 -- 172.21.15.69:0/869405073 >> 172.21.15.69:0/869405073 conn(0x7f4d080fc820 msgr2=0x7f4d080fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.029+0000 7f4d0f000640 1 -- 172.21.15.69:0/869405073 shutdown_connections 2024-04-24T09:08:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.029+0000 7f4d0f000640 1 -- 172.21.15.69:0/869405073 wait complete. 2024-04-24T09:08:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.030+0000 7f4d0f000640 1 Processor -- start 2024-04-24T09:08:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.030+0000 7f4d0f000640 1 -- start start 2024-04-24T09:08:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.030+0000 7f4d0f000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d08100f60 0x7f4d081a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.030+0000 7f4d0f000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d08106030 0x7f4d081a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.030+0000 7f4d0f000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d0819eb20 0x7f4d0819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.030+0000 7f4d0f000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0810fea0 con 0x7f4d0819eb20 2024-04-24T09:08:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.030+0000 7f4d0f000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0810fd20 con 0x7f4d08106030 2024-04-24T09:08:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.030+0000 7f4d0f000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4d08110020 con 0x7f4d08100f60 2024-04-24T09:08:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.031+0000 7f4d0e600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d0819eb20 0x7f4d0819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.031+0000 7f4d0e600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d0819eb20 0x7f4d0819efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44894/0 (socket says 172.21.15.69:44894) 2024-04-24T09:08:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.031+0000 7f4d0e600640 1 -- 172.21.15.69:0/2078939313 learned_addr learned my addr 172.21.15.69:0/2078939313 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.031+0000 7f4d0d200640 1 --2- 172.21.15.69:0/2078939313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d08106030 0x7f4d081a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.031+0000 7f4d0dc00640 1 --2- 172.21.15.69:0/2078939313 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d08100f60 0x7f4d081a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.031+0000 7f4d0e600640 1 -- 172.21.15.69:0/2078939313 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d08100f60 msgr2=0x7f4d081a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.031+0000 7f4d0e600640 1 --2- 172.21.15.69:0/2078939313 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d08100f60 0x7f4d081a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.031+0000 7f4d0e600640 1 -- 172.21.15.69:0/2078939313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d08106030 msgr2=0x7f4d081a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.031+0000 7f4d0e600640 1 --2- 172.21.15.69:0/2078939313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d08106030 0x7f4d081a4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.031+0000 7f4d0e600640 1 -- 172.21.15.69:0/2078939313 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0819f640 con 0x7f4d0819eb20 2024-04-24T09:08:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.032+0000 7f4d0dc00640 1 --2- 172.21.15.69:0/2078939313 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d08100f60 0x7f4d081a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.032+0000 7f4d0e600640 1 --2- 172.21.15.69:0/2078939313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d0819eb20 0x7f4d0819efd0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000eaa0 tx=0x7f4d0000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.032+0000 7f4d06000640 1 -- 172.21.15.69:0/2078939313 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000cdb0 con 0x7f4d0819eb20 2024-04-24T09:08:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.032+0000 7f4d0f000640 1 -- 172.21.15.69:0/2078939313 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0819f930 con 0x7f4d0819eb20 2024-04-24T09:08:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.032+0000 7f4d0f000640 1 -- 172.21.15.69:0/2078939313 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d081abb80 con 0x7f4d0819eb20 2024-04-24T09:08:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.032+0000 7f4d06000640 1 -- 172.21.15.69:0/2078939313 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d00005020 con 0x7f4d0819eb20 2024-04-24T09:08:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.032+0000 7f4d06000640 1 -- 172.21.15.69:0/2078939313 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d000116d0 con 0x7f4d0819eb20 2024-04-24T09:08:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.038+0000 7f4d06000640 1 -- 172.21.15.69:0/2078939313 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f4d00011870 con 0x7f4d0819eb20 2024-04-24T09:08:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.039+0000 7f4d1347c740 1 Processor -- start 2024-04-24T09:08:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.039+0000 7f4d1347c740 1 -- start start 2024-04-24T09:08:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.039+0000 7f4d1347c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f73c0 0x5591f44f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.040+0000 7f4d1347c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f44f7ce0 0x5591f4502060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.040+0000 7f4d1347c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f45025a0 0x5591f4504990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.040+0000 7f4d1347c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5591f445d900 con 0x5591f44f73c0 2024-04-24T09:08:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.040+0000 7f4d1347c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5591f4403f20 con 0x5591f44f7ce0 2024-04-24T09:08:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.040+0000 7f4d1347c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5591f44598c0 con 0x5591f45025a0 2024-04-24T09:08:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.040+0000 7f4d06000640 1 --2- 172.21.15.69:0/2078939313 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4cf00a3ea0 0x7f4cf00a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.040+0000 7f4d0d200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f73c0 0x5591f44f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.040+0000 7f4d0d200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f73c0 0x5591f44f77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44910/0 (socket says 172.21.15.69:44910) 2024-04-24T09:08:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.040+0000 7f4d0d200640 1 -- 172.21.15.69:0/1006565097 learned_addr learned my addr 172.21.15.69:0/1006565097 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.040+0000 7f4d06000640 1 -- 172.21.15.69:0/2078939313 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4d000d1ec0 con 0x7f4d0819eb20 2024-04-24T09:08:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.040+0000 7f4d0e600640 1 --2- 172.21.15.69:0/2078939313 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4cf00a3ea0 0x7f4cf00a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.040+0000 7f4d0dc00640 1 --2- 172.21.15.69:0/1006565097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f44f7ce0 0x5591f4502060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.041+0000 7f4d0d200640 1 --2- 172.21.15.69:0/1006565097 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f45025a0 0x5591f4504990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.041+0000 7f4d0d200640 1 -- 172.21.15.69:0/1006565097 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f45025a0 msgr2=0x5591f4504990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.041+0000 7f4d0d200640 1 --2- 172.21.15.69:0/1006565097 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f45025a0 0x5591f4504990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.041+0000 7f4d0d200640 1 -- 172.21.15.69:0/1006565097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f44f7ce0 msgr2=0x5591f4502060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.041+0000 7f4d0d200640 1 --2- 172.21.15.69:0/1006565097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f44f7ce0 0x5591f4502060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.041+0000 7f4d0d200640 1 -- 172.21.15.69:0/1006565097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591f4428e10 con 0x5591f44f73c0 2024-04-24T09:08:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.042+0000 7f4d0d200640 1 --2- 172.21.15.69:0/1006565097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f73c0 0x5591f44f77a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4032fd0 tx=0x7f4cf40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbe5b2455aacabc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.042+0000 7f4d0e600640 1 --2- 172.21.15.69:0/2078939313 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4cf00a3ea0 0x7f4cf00a6360 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4d000cf1b0 tx=0x7f4d00009270 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.042+0000 7f4cfaa00640 1 -- 172.21.15.69:0/1006565097 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4035030 con 0x5591f44f73c0 2024-04-24T09:08:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.042+0000 7f4cfaa00640 1 -- 172.21.15.69:0/1006565097 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cf40044d0 con 0x5591f44f73c0 2024-04-24T09:08:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.042+0000 7f4cfaa00640 1 -- 172.21.15.69:0/1006565097 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4004920 con 0x5591f44f73c0 2024-04-24T09:08:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.043+0000 7f4d1347c740 1 -- 172.21.15.69:0/1006565097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f73c0 msgr2=0x5591f44f77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.043+0000 7f4d1347c740 1 --2- 172.21.15.69:0/1006565097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f73c0 0x5591f44f77a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4032fd0 tx=0x7f4cf40058b0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.043+0000 7f4d1347c740 1 -- 172.21.15.69:0/1006565097 shutdown_connections 2024-04-24T09:08:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.043+0000 7f4d1347c740 1 --2- 172.21.15.69:0/1006565097 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f45025a0 0x5591f4504990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.043+0000 7f4d1347c740 1 --2- 172.21.15.69:0/1006565097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f44f7ce0 0x5591f4502060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.043+0000 7f4d1347c740 1 --2- 172.21.15.69:0/1006565097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f73c0 0x5591f44f77a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.043+0000 7f4d1347c740 1 -- 172.21.15.69:0/1006565097 >> 172.21.15.69:0/1006565097 conn(0x5591f44f38d0 msgr2=0x5591f44f3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.043+0000 7f4d1347c740 1 -- 172.21.15.69:0/1006565097 shutdown_connections 2024-04-24T09:08:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.043+0000 7f4d1347c740 1 -- 172.21.15.69:0/1006565097 wait complete. 2024-04-24T09:08:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.044+0000 7f4d1347c740 1 -- start start 2024-04-24T09:08:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.044+0000 7f4d1347c740 1 Processor -- start 2024-04-24T09:08:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.045+0000 7f4d1347c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f44f73c0 0x5591f44ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.045+0000 7f4d1347c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f44f7ce0 0x5591f44ffb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.045+0000 7f4d1347c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f45025a0 0x5591f4500040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.045+0000 7f4d1347c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5591f4505080 con 0x5591f45025a0 2024-04-24T09:08:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.045+0000 7f4d1347c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5591f44172c0 con 0x5591f44f73c0 2024-04-24T09:08:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.045+0000 7f4d1347c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5591f4505200 con 0x5591f44f7ce0 2024-04-24T09:08:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.045+0000 7f4d1347c740 10 client.0 _ll_register_callbacks cb 0x5591f416fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.045+0000 7f4d0d200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f45025a0 0x5591f4500040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.045+0000 7f4d0d200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f45025a0 0x5591f4500040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44914/0 (socket says 172.21.15.69:44914) 2024-04-24T09:08:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.045+0000 7f4d0d200640 1 -- 172.21.15.69:0/1922702124 learned_addr learned my addr 172.21.15.69:0/1922702124 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.045+0000 7f4d0d200640 1 --2- 172.21.15.69:0/1922702124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f44f73c0 0x5591f44ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.045+0000 7f4d0dc00640 1 --2- 172.21.15.69:0/1922702124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f44f7ce0 0x5591f44ffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.046+0000 7f4d0d200640 1 -- 172.21.15.69:0/1922702124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f44f7ce0 msgr2=0x5591f44ffb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.046+0000 7f4d0d200640 1 --2- 172.21.15.69:0/1922702124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f44f7ce0 0x5591f44ffb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.046+0000 7f4d0d200640 1 -- 172.21.15.69:0/1922702124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f44f73c0 msgr2=0x5591f44ff5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.046+0000 7f4d0d200640 1 --2- 172.21.15.69:0/1922702124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f44f73c0 0x5591f44ff5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.046+0000 7f4d0d200640 1 -- 172.21.15.69:0/1922702124 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591f4427fd0 con 0x5591f45025a0 2024-04-24T09:08:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.046+0000 7f4d0d200640 1 --2- 172.21.15.69:0/1922702124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f45025a0 0x5591f4500040 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4005660 tx=0x7f4cf4031530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67bb1206b689373 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.046+0000 7f4cef400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.046+0000 7f4cef400640 1 -- 172.21.15.69:0/1922702124 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4035040 con 0x5591f45025a0 2024-04-24T09:08:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.047+0000 7f4cef400640 1 -- 172.21.15.69:0/1922702124 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4cf4005260 con 0x5591f45025a0 2024-04-24T09:08:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.047+0000 7f4d1347c740 1 -- 172.21.15.69:0/1922702124 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cf40044d0 con 0x5591f45025a0 2024-04-24T09:08:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.047+0000 7f4cef400640 1 -- 172.21.15.69:0/1922702124 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4cf40046c0 con 0x5591f45025a0 2024-04-24T09:08:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.048+0000 7f4d1347c740 10 client.16053 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.048+0000 7f4d1347c740 1 -- 172.21.15.69:0/1922702124 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5591f4170210 con 0x5591f45025a0 2024-04-24T09:08:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.048+0000 7f4cef400640 1 -- 172.21.15.69:0/1922702124 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cf40044d0 con 0x5591f45025a0 2024-04-24T09:08:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.049+0000 7f4cef400640 1 -- 172.21.15.69:0/1922702124 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4cf40038f0 con 0x5591f45025a0 2024-04-24T09:08:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.049+0000 7f4d1347c740 10 client.16053 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.049+0000 7f4d1347c740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.049+0000 7f4d1347c740 20 client.16053 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.049+0000 7f4d1347c740 1 --2- 172.21.15.69:0/1922702124 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5591f4512580 0x5591f4514970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.049+0000 7f4d1347c740 4 client.16053 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.049+0000 7f4d1347c740 1 -- 172.21.15.69:0/1922702124 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591f446b320 con 0x5591f4512580 2024-04-24T09:08:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.050+0000 7f4d0dc00640 1 --2- 172.21.15.69:0/1922702124 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5591f4512580 0x5591f4514970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.051+0000 7f4cef400640 10 client.16053 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.051+0000 7f4d0dc00640 1 --2- 172.21.15.69:0/1922702124 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5591f4512580 0x5591f4514970 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f155c297a5618ffa server_cookie=45fdef6caced0941 in_seq=0 out_seq=0 2024-04-24T09:08:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.052+0000 7f4cef400640 1 -- 172.21.15.69:0/1922702124 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591f446b320 con 0x5591f4512580 2024-04-24T09:08:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.052+0000 7f4cef400640 10 client.16053 handle_command_reply: tid=0 2024-04-24T09:08:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.052+0000 7f4d1347c740 1 client.16053 shutdown 2024-04-24T09:08:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.053+0000 7f4d1347c740 1 -- 172.21.15.69:0/1922702124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f45025a0 msgr2=0x5591f4500040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.053+0000 7f4d1347c740 1 --2- 172.21.15.69:0/1922702124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f45025a0 0x5591f4500040 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4005660 tx=0x7f4cf4031530 comp rx=0 tx=0).stop 2024-04-24T09:08:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.053+0000 7f4d1347c740 1 -- 172.21.15.69:0/1922702124 shutdown_connections 2024-04-24T09:08:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.053+0000 7f4d1347c740 1 --2- 172.21.15.69:0/1922702124 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5591f4512580 0x5591f4514970 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.053+0000 7f4d1347c740 1 --2- 172.21.15.69:0/1922702124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f45025a0 0x5591f4500040 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.053+0000 7f4d1347c740 1 --2- 172.21.15.69:0/1922702124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f44f7ce0 0x5591f44ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.053+0000 7f4d1347c740 1 --2- 172.21.15.69:0/1922702124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f44f73c0 0x5591f44ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.053+0000 7f4d1347c740 1 -- 172.21.15.69:0/1922702124 >> 172.21.15.69:0/1922702124 conn(0x5591f44f38d0 msgr2=0x5591f44f9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.053+0000 7f4cef400640 0 client.16053 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.053+0000 7f4d1347c740 1 -- 172.21.15.69:0/1922702124 shutdown_connections 2024-04-24T09:08:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.053+0000 7f4d1347c740 1 -- 172.21.15.69:0/1922702124 wait complete. 2024-04-24T09:08:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.053+0000 7f4d1347c740 20 client.16053 trim_cache size 0 max 16384 2024-04-24T09:08:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.069+0000 7f4d1347c740 1 Processor -- start 2024-04-24T09:08:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.069+0000 7f4d1347c740 1 -- start start 2024-04-24T09:08:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.069+0000 7f4d1347c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f73c0 0x5591f44f9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.069+0000 7f4d1347c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f44f7ce0 0x5591f44ffc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.069+0000 7f4d1347c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f45025a0 0x5591f44fc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.069+0000 7f4d1347c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce8012630 con 0x5591f44f73c0 2024-04-24T09:08:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.069+0000 7f4d1347c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce800ff00 con 0x5591f45025a0 2024-04-24T09:08:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.069+0000 7f4d1347c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce800dd50 con 0x5591f44f7ce0 2024-04-24T09:08:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.070+0000 7f4d0d200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f73c0 0x5591f44f9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.070+0000 7f4d0d200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f73c0 0x5591f44f9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44928/0 (socket says 172.21.15.69:44928) 2024-04-24T09:08:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.070+0000 7f4d0d200640 1 -- 172.21.15.69:0/669199322 learned_addr learned my addr 172.21.15.69:0/669199322 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.070+0000 7f4d0dc00640 1 --2- 172.21.15.69:0/669199322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f44f7ce0 0x5591f44ffc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.070+0000 7f4d0d200640 1 -- 172.21.15.69:0/669199322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f44f7ce0 msgr2=0x5591f44ffc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.070+0000 7f4d0d200640 1 --2- 172.21.15.69:0/669199322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f44f7ce0 0x5591f44ffc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.070+0000 7f4d0d200640 1 -- 172.21.15.69:0/669199322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f45025a0 msgr2=0x5591f44fc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.070+0000 7f4d0d200640 1 --2- 172.21.15.69:0/669199322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f45025a0 0x5591f44fc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.070+0000 7f4d0d200640 1 -- 172.21.15.69:0/669199322 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591f423c620 con 0x5591f44f73c0 2024-04-24T09:08:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.071+0000 7f4d0d200640 1 --2- 172.21.15.69:0/669199322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f73c0 0x5591f44f9d80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4000c00 tx=0x7f4cf4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d82828d70c1421 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.071+0000 7f4cfa000640 1 -- 172.21.15.69:0/669199322 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4035070 con 0x5591f44f73c0 2024-04-24T09:08:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.071+0000 7f4cfa000640 1 -- 172.21.15.69:0/669199322 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cf40044c0 con 0x5591f44f73c0 2024-04-24T09:08:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.071+0000 7f4cfa000640 1 -- 172.21.15.69:0/669199322 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4003aa0 con 0x5591f44f73c0 2024-04-24T09:08:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.072+0000 7f4d1347c740 1 -- 172.21.15.69:0/669199322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f73c0 msgr2=0x5591f44f9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.072+0000 7f4d1347c740 1 --2- 172.21.15.69:0/669199322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f73c0 0x5591f44f9d80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4000c00 tx=0x7f4cf4009780 comp rx=0 tx=0).stop 2024-04-24T09:08:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.072+0000 7f4d1347c740 1 -- 172.21.15.69:0/669199322 shutdown_connections 2024-04-24T09:08:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.072+0000 7f4d1347c740 1 --2- 172.21.15.69:0/669199322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f45025a0 0x5591f44fc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.072+0000 7f4d1347c740 1 --2- 172.21.15.69:0/669199322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f44f7ce0 0x5591f44ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.072+0000 7f4d1347c740 1 --2- 172.21.15.69:0/669199322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f73c0 0x5591f44f9d80 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.072+0000 7f4d1347c740 1 -- 172.21.15.69:0/669199322 >> 172.21.15.69:0/669199322 conn(0x5591f44f38d0 msgr2=0x5591f44ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.072+0000 7f4d1347c740 1 -- 172.21.15.69:0/669199322 shutdown_connections 2024-04-24T09:08:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.072+0000 7f4d1347c740 1 -- 172.21.15.69:0/669199322 wait complete. 2024-04-24T09:08:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.072+0000 7f4d1347c740 1 -- start start 2024-04-24T09:08:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.072+0000 7f4d1347c740 1 Processor -- start 2024-04-24T09:08:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.073+0000 7f4d1347c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f44f73c0 0x5591f4515150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.073+0000 7f4d1347c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f7ce0 0x5591f4515690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.073+0000 7f4d1347c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f45025a0 0x5591f4515bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.073+0000 7f4d1347c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce800e3c0 con 0x5591f44f7ce0 2024-04-24T09:08:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.073+0000 7f4d1347c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce800f830 con 0x5591f44f73c0 2024-04-24T09:08:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.073+0000 7f4d1347c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce800ea90 con 0x5591f45025a0 2024-04-24T09:08:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.073+0000 7f4d1347c740 10 client.0 _ll_register_callbacks cb 0x7f4cfc00c390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.073+0000 7f4d0dc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f7ce0 0x5591f4515690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.073+0000 7f4d0dc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f7ce0 0x5591f4515690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44938/0 (socket says 172.21.15.69:44938) 2024-04-24T09:08:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.073+0000 7f4d0dc00640 1 -- 172.21.15.69:0/1195938947 learned_addr learned my addr 172.21.15.69:0/1195938947 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.073+0000 7f4d0d200640 1 --2- 172.21.15.69:0/1195938947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f44f73c0 0x5591f4515150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.074+0000 7f4d0d200640 1 --2- 172.21.15.69:0/1195938947 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f45025a0 0x5591f4515bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.074+0000 7f4d0dc00640 1 -- 172.21.15.69:0/1195938947 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f45025a0 msgr2=0x5591f4515bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.074+0000 7f4d0dc00640 1 --2- 172.21.15.69:0/1195938947 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f45025a0 0x5591f4515bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.074+0000 7f4d0dc00640 1 -- 172.21.15.69:0/1195938947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f44f73c0 msgr2=0x5591f4515150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.074+0000 7f4d0dc00640 1 --2- 172.21.15.69:0/1195938947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f44f73c0 0x5591f4515150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.074+0000 7f4d0dc00640 1 -- 172.21.15.69:0/1195938947 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591f45778c0 con 0x5591f44f7ce0 2024-04-24T09:08:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.074+0000 7f4d0d200640 1 --2- 172.21.15.69:0/1195938947 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f45025a0 0x5591f4515bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.074+0000 7f4d0dc00640 1 --2- 172.21.15.69:0/1195938947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f7ce0 0x5591f4515690 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00d010 tx=0x7f4cfc010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fcf62457148fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.074+0000 7f4cef400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.074+0000 7f4cef400640 1 -- 172.21.15.69:0/1195938947 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc020030 con 0x5591f44f7ce0 2024-04-24T09:08:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.074+0000 7f4cef400640 1 -- 172.21.15.69:0/1195938947 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4cfc01c420 con 0x5591f44f7ce0 2024-04-24T09:08:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.074+0000 7f4d1347c740 1 -- 172.21.15.69:0/1195938947 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5591f4465190 con 0x5591f44f7ce0 2024-04-24T09:08:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.075+0000 7f4cef400640 1 -- 172.21.15.69:0/1195938947 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4cfc01c620 con 0x5591f44f7ce0 2024-04-24T09:08:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.075+0000 7f4d1347c740 10 client.16059 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.075+0000 7f4d1347c740 1 -- 172.21.15.69:0/1195938947 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5591f4578c40 con 0x5591f44f7ce0 2024-04-24T09:08:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.075+0000 7f4cef400640 1 -- 172.21.15.69:0/1195938947 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5591f4465190 con 0x5591f44f7ce0 2024-04-24T09:08:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.075+0000 7f4cef400640 1 -- 172.21.15.69:0/1195938947 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4cfc005280 con 0x5591f44f7ce0 2024-04-24T09:08:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.075+0000 7f4d1347c740 10 client.16059 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.075+0000 7f4d1347c740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.075+0000 7f4d1347c740 20 client.16059 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.076+0000 7f4d1347c740 1 --2- 172.21.15.69:0/1195938947 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5591f45793f0 0x5591f457b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.076+0000 7f4d1347c740 4 client.16059 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:08:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.076+0000 7f4d1347c740 1 -- 172.21.15.69:0/1195938947 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591f43e14b0 con 0x5591f45793f0 2024-04-24T09:08:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.076+0000 7f4d0d200640 1 --2- 172.21.15.69:0/1195938947 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5591f45793f0 0x5591f457b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.077+0000 7f4d0d200640 1 --2- 172.21.15.69:0/1195938947 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5591f45793f0 0x5591f457b810 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f8855e0c15d132a server_cookie=3dfe95cc323fb581 in_seq=0 out_seq=0 2024-04-24T09:08:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.077+0000 7f4cef400640 10 client.16059 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.086+0000 7f4cef400640 1 -- 172.21.15.69:0/1195938947 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5591f43e14b0 con 0x5591f45793f0 2024-04-24T09:08:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.086+0000 7f4cef400640 10 client.16059 handle_command_reply: tid=0 2024-04-24T09:08:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.086+0000 7f4d1347c740 1 client.16059 shutdown 2024-04-24T09:08:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.086+0000 7f4d1347c740 1 -- 172.21.15.69:0/1195938947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f7ce0 msgr2=0x5591f4515690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.086+0000 7f4d1347c740 1 --2- 172.21.15.69:0/1195938947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f7ce0 0x5591f4515690 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00d010 tx=0x7f4cfc010b40 comp rx=0 tx=0).stop 2024-04-24T09:08:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.086+0000 7f4d1347c740 1 -- 172.21.15.69:0/1195938947 shutdown_connections 2024-04-24T09:08:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.086+0000 7f4d1347c740 1 --2- 172.21.15.69:0/1195938947 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5591f45793f0 0x5591f457b810 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.086+0000 7f4d1347c740 1 --2- 172.21.15.69:0/1195938947 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5591f45025a0 0x5591f4515bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.086+0000 7f4d1347c740 1 --2- 172.21.15.69:0/1195938947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591f44f7ce0 0x5591f4515690 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.086+0000 7f4d1347c740 1 --2- 172.21.15.69:0/1195938947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5591f44f73c0 0x5591f4515150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.086+0000 7f4d1347c740 1 -- 172.21.15.69:0/1195938947 >> 172.21.15.69:0/1195938947 conn(0x5591f44f38d0 msgr2=0x5591f44ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.086+0000 7f4cef400640 0 client.16059 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.087+0000 7f4d1347c740 1 -- 172.21.15.69:0/1195938947 shutdown_connections 2024-04-24T09:08:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.087+0000 7f4d1347c740 1 -- 172.21.15.69:0/1195938947 wait complete. 2024-04-24T09:08:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.087+0000 7f4d1347c740 20 client.16059 trim_cache size 0 max 16384 2024-04-24T09:08:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:08:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:08:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:08:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:08:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.087+0000 7f4d0f000640 1 -- 172.21.15.69:0/2078939313 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4cf00a3ea0 msgr2=0x7f4cf00a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.087+0000 7f4d0f000640 1 --2- 172.21.15.69:0/2078939313 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4cf00a3ea0 0x7f4cf00a6360 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4d000cf1b0 tx=0x7f4d00009270 comp rx=0 tx=0).stop 2024-04-24T09:08:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.087+0000 7f4d0f000640 1 -- 172.21.15.69:0/2078939313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d0819eb20 msgr2=0x7f4d0819efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.087+0000 7f4d0f000640 1 --2- 172.21.15.69:0/2078939313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d0819eb20 0x7f4d0819efd0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000eaa0 tx=0x7f4d0000ef60 comp rx=0 tx=0).stop 2024-04-24T09:08:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.088+0000 7f4d0f000640 1 -- 172.21.15.69:0/2078939313 shutdown_connections 2024-04-24T09:08:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.088+0000 7f4d0f000640 1 --2- 172.21.15.69:0/2078939313 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4cf00a3ea0 0x7f4cf00a6360 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.088+0000 7f4d0f000640 1 --2- 172.21.15.69:0/2078939313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d0819eb20 0x7f4d0819efd0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.088+0000 7f4d0f000640 1 --2- 172.21.15.69:0/2078939313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d08106030 0x7f4d081a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.088+0000 7f4d0f000640 1 --2- 172.21.15.69:0/2078939313 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d08100f60 0x7f4d081a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.088+0000 7f4d0f000640 1 -- 172.21.15.69:0/2078939313 >> 172.21.15.69:0/2078939313 conn(0x7f4d080fc820 msgr2=0x7f4d080fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.088+0000 7f4d0f000640 1 -- 172.21.15.69:0/2078939313 shutdown_connections 2024-04-24T09:08:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:34.088+0000 7f4d0f000640 1 -- 172.21.15.69:0/2078939313 wait complete. 2024-04-24T09:08:35.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:35 smithi081 ceph-mon[30961]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:08:35.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:35 smithi069 ceph-mon[28170]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:08:35.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:35 smithi130 ceph-mon[31553]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T09:08:37.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:37 smithi081 ceph-mon[30961]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-24T09:08:37.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:37 smithi081 ceph-mon[30961]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-24T09:08:37.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:37 smithi069 ceph-mon[28170]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-24T09:08:37.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:37 smithi069 ceph-mon[28170]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-24T09:08:37.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:37 smithi130 ceph-mon[31553]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-24T09:08:37.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:37 smithi130 ceph-mon[31553]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-24T09:08:39.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:39 smithi081 ceph-mon[30961]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T09:08:39.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:39 smithi069 ceph-mon[28170]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T09:08:39.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:39 smithi130 ceph-mon[31553]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T09:08:41.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:41 smithi081 ceph-mon[30961]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T09:08:41.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:41 smithi069 ceph-mon[28170]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T09:08:41.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:41 smithi130 ceph-mon[31553]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T09:08:42.611 INFO:tasks.workunit.client.0.smithi069.stderr:Updating files: 62% (7712/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-24T09:08:43.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:43 smithi081 ceph-mon[30961]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T09:08:43.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:43 smithi069 ceph-mon[28170]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T09:08:43.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:43 smithi130 ceph-mon[31553]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T09:08:43.472 INFO:tasks.workunit.client.0.smithi069.stderr:Note: switching to '1f81fb397ae98da7563d451a78c61574c8f4e6e0'. 2024-04-24T09:08:43.472 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-24T09:08:43.472 INFO:tasks.workunit.client.0.smithi069.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T09:08:43.472 INFO:tasks.workunit.client.0.smithi069.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T09:08:43.472 INFO:tasks.workunit.client.0.smithi069.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T09:08:43.472 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-24T09:08:43.472 INFO:tasks.workunit.client.0.smithi069.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T09:08:43.472 INFO:tasks.workunit.client.0.smithi069.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T09:08:43.473 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-24T09:08:43.473 INFO:tasks.workunit.client.0.smithi069.stderr: git switch -c 2024-04-24T09:08:43.473 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-24T09:08:43.473 INFO:tasks.workunit.client.0.smithi069.stderr:Or undo this operation with: 2024-04-24T09:08:43.473 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-24T09:08:43.473 INFO:tasks.workunit.client.0.smithi069.stderr: git switch - 2024-04-24T09:08:43.473 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-24T09:08:43.473 INFO:tasks.workunit.client.0.smithi069.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T09:08:43.473 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-24T09:08:43.473 INFO:tasks.workunit.client.0.smithi069.stderr:HEAD is now at 1f81fb397ae DNM: trigger builds 2024-04-24T09:08:43.486 DEBUG:teuthology.orchestra.run.smithi069:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-24T09:08:43.592 INFO:tasks.workunit.client.0.smithi069.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T09:08:43.599 INFO:tasks.workunit.client.0.smithi069.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T09:08:43.599 INFO:tasks.workunit.client.0.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T09:08:44.553 INFO:tasks.workunit.client.0.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T09:08:44.663 INFO:tasks.workunit.client.0.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T09:08:44.749 INFO:tasks.workunit.client.0.smithi069.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T09:08:44.754 INFO:tasks.workunit.client.0.smithi069.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T09:08:44.754 INFO:tasks.workunit.client.0.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T09:08:44.839 INFO:tasks.workunit.client.0.smithi069.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T09:08:44.849 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T09:08:44.849 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-24T09:08:44.900 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-04-24T09:08:44.901 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-04-24T09:08:44.902 DEBUG:teuthology.orchestra.run.smithi069:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1f81fb397ae98da7563d451a78c61574c8f4e6e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-04-24T09:08:44.990 INFO:tasks.workunit.client.0.smithi069.stderr:+ set -e 2024-04-24T09:08:44.990 INFO:tasks.workunit.client.0.smithi069.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-04-24T09:08:45.006 INFO:tasks.workunit.client.0.smithi069.stderr:Cloning into 'xfstests-dev'... 2024-04-24T09:08:45.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:45 smithi081 ceph-mon[30961]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T09:08:45.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:45 smithi069 ceph-mon[28170]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T09:08:45.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:45 smithi130 ceph-mon[31553]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T09:08:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.232+0000 7fbdd6600640 1 -- 172.21.15.69:0/3924181752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdd0104e20 msgr2=0x7fbdd0108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.232+0000 7fbdd6600640 1 --2- 172.21.15.69:0/3924181752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdd0104e20 0x7fbdd0108e60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009900 tx=0x7fbdb802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.232+0000 7fbdd6600640 1 -- 172.21.15.69:0/3924181752 shutdown_connections 2024-04-24T09:08:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.232+0000 7fbdd6600640 1 --2- 172.21.15.69:0/3924181752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdd0109540 0x7fbdd0111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.232+0000 7fbdd6600640 1 --2- 172.21.15.69:0/3924181752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdd0104e20 0x7fbdd0108e60 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.232+0000 7fbdd6600640 1 --2- 172.21.15.69:0/3924181752 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbdd0104470 0x7fbdd0104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.233+0000 7fbdd6600640 1 -- 172.21.15.69:0/3924181752 >> 172.21.15.69:0/3924181752 conn(0x7fbdd00ffd30 msgr2=0x7fbdd0102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.233+0000 7fbdd6600640 1 -- 172.21.15.69:0/3924181752 shutdown_connections 2024-04-24T09:08:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.233+0000 7fbdd6600640 1 -- 172.21.15.69:0/3924181752 wait complete. 2024-04-24T09:08:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.234+0000 7fbdd6600640 1 Processor -- start 2024-04-24T09:08:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.234+0000 7fbdd6600640 1 -- start start 2024-04-24T09:08:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.234+0000 7fbdd6600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdd0104470 0x7fbdd01a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.234+0000 7fbdd6600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbdd0109540 0x7fbdd01a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.234+0000 7fbdd6600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdd019a830 0x7fbdd019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.234+0000 7fbdd6600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01144d0 con 0x7fbdd019a830 2024-04-24T09:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.234+0000 7fbdd6600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0114350 con 0x7fbdd0104470 2024-04-24T09:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.234+0000 7fbdd6600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0114650 con 0x7fbdd0109540 2024-04-24T09:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.234+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdd019a830 0x7fbdd019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.234+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdd019a830 0x7fbdd019ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33314/0 (socket says 172.21.15.69:33314) 2024-04-24T09:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.234+0000 7fbdcfe00640 1 -- 172.21.15.69:0/3841018864 learned_addr learned my addr 172.21.15.69:0/3841018864 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.235+0000 7fbdcf400640 1 --2- 172.21.15.69:0/3841018864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdd0104470 0x7fbdd01a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.235+0000 7fbdcfe00640 1 -- 172.21.15.69:0/3841018864 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbdd0109540 msgr2=0x7fbdd01a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.235+0000 7fbdcfe00640 1 --2- 172.21.15.69:0/3841018864 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbdd0109540 0x7fbdd01a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.235+0000 7fbdcfe00640 1 -- 172.21.15.69:0/3841018864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdd0104470 msgr2=0x7fbdd01a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.235+0000 7fbdcfe00640 1 --2- 172.21.15.69:0/3841018864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdd0104470 0x7fbdd01a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.235+0000 7fbdcfe00640 1 -- 172.21.15.69:0/3841018864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd019b3c0 con 0x7fbdd019a830 2024-04-24T09:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.235+0000 7fbdcf400640 1 --2- 172.21.15.69:0/3841018864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdd0104470 0x7fbdd01a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.235+0000 7fbdcfe00640 1 --2- 172.21.15.69:0/3841018864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdd019a830 0x7fbdd019ace0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800eaa0 tx=0x7fbdc800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.235+0000 7fbdccc00640 1 -- 172.21.15.69:0/3841018864 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc800cdb0 con 0x7fbdd019a830 2024-04-24T09:08:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.235+0000 7fbdd6600640 1 -- 172.21.15.69:0/3841018864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd019b6b0 con 0x7fbdd019a830 2024-04-24T09:08:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.235+0000 7fbdccc00640 1 -- 172.21.15.69:0/3841018864 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdc8005020 con 0x7fbdd019a830 2024-04-24T09:08:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.235+0000 7fbdccc00640 1 -- 172.21.15.69:0/3841018864 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc80116d0 con 0x7fbdd019a830 2024-04-24T09:08:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.236+0000 7fbdd6600640 1 -- 172.21.15.69:0/3841018864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd01a7840 con 0x7fbdd019a830 2024-04-24T09:08:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.239+0000 7fbdd9438740 1 Processor -- start 2024-04-24T09:08:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.239+0000 7fbdccc00640 1 -- 172.21.15.69:0/3841018864 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbdc80051d0 con 0x7fbdd019a830 2024-04-24T09:08:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.239+0000 7fbdd9438740 1 -- start start 2024-04-24T09:08:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.239+0000 7fbdd9438740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236dda1b0 0x558236dda590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.239+0000 7fbdd9438740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236ddaad0 0x558236de4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.240+0000 7fbdd9438740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236de5390 0x558236de7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.240+0000 7fbdd9438740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558236d40900 con 0x558236ddaad0 2024-04-24T09:08:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.240+0000 7fbdd9438740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x558236ce6f20 con 0x558236de5390 2024-04-24T09:08:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.240+0000 7fbdd9438740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558236d3c8c0 con 0x558236dda1b0 2024-04-24T09:08:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.240+0000 7fbdcf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236ddaad0 0x558236de4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.240+0000 7fbdcf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236ddaad0 0x558236de4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33328/0 (socket says 172.21.15.69:33328) 2024-04-24T09:08:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.240+0000 7fbdcf400640 1 -- 172.21.15.69:0/3160167704 learned_addr learned my addr 172.21.15.69:0/3160167704 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.240+0000 7fbdc6a00640 1 --2- 172.21.15.69:0/3160167704 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236dda1b0 0x558236dda590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.240+0000 7fbdcf400640 1 -- 172.21.15.69:0/3160167704 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236dda1b0 msgr2=0x558236dda590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.240+0000 7fbdcf400640 1 --2- 172.21.15.69:0/3160167704 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236dda1b0 0x558236dda590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.240+0000 7fbdcf400640 1 -- 172.21.15.69:0/3160167704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236de5390 msgr2=0x558236de7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.240+0000 7fbdcf400640 1 --2- 172.21.15.69:0/3160167704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236de5390 0x558236de7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.240+0000 7fbdcf400640 1 -- 172.21.15.69:0/3160167704 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558236d0be10 con 0x558236ddaad0 2024-04-24T09:08:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.240+0000 7fbdc6a00640 1 --2- 172.21.15.69:0/3160167704 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236dda1b0 0x558236dda590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.241+0000 7fbdcf400640 1 --2- 172.21.15.69:0/3160167704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236ddaad0 0x558236de4e50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009c50 tx=0x7fbdc000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16015159090266f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.241+0000 7fbdccc00640 1 --2- 172.21.15.69:0/3841018864 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fbdb40a3e40 0x7fbdb40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.241+0000 7fbdc4c00640 1 -- 172.21.15.69:0/3160167704 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0016070 con 0x558236ddaad0 2024-04-24T09:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.241+0000 7fbdccc00640 1 -- 172.21.15.69:0/3841018864 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fbdc80d0d80 con 0x7fbdd019a830 2024-04-24T09:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.241+0000 7fbdc4c00640 1 -- 172.21.15.69:0/3160167704 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdc0011920 con 0x558236ddaad0 2024-04-24T09:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.241+0000 7fbdc4c00640 1 -- 172.21.15.69:0/3160167704 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0010830 con 0x558236ddaad0 2024-04-24T09:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.241+0000 7fbdc6a00640 1 --2- 172.21.15.69:0/3841018864 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fbdb40a3e40 0x7fbdb40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.242+0000 7fbdd9438740 1 -- 172.21.15.69:0/3160167704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236ddaad0 msgr2=0x558236de4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.242+0000 7fbdd9438740 1 --2- 172.21.15.69:0/3160167704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236ddaad0 0x558236de4e50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009c50 tx=0x7fbdc000e6c0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.242+0000 7fbdc6a00640 1 --2- 172.21.15.69:0/3841018864 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fbdb40a3e40 0x7fbdb40a6300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009230 tx=0x7fbdb8000bc0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.242+0000 7fbdd9438740 1 -- 172.21.15.69:0/3160167704 shutdown_connections 2024-04-24T09:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.242+0000 7fbdd9438740 1 --2- 172.21.15.69:0/3160167704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236de5390 0x558236de7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.242+0000 7fbdd9438740 1 --2- 172.21.15.69:0/3160167704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236ddaad0 0x558236de4e50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.242+0000 7fbdd9438740 1 --2- 172.21.15.69:0/3160167704 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236dda1b0 0x558236dda590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.242+0000 7fbdd9438740 1 -- 172.21.15.69:0/3160167704 >> 172.21.15.69:0/3160167704 conn(0x558236dd66c0 msgr2=0x558236dd6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.242+0000 7fbdd9438740 1 -- 172.21.15.69:0/3160167704 shutdown_connections 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.242+0000 7fbdd9438740 1 -- 172.21.15.69:0/3160167704 wait complete. 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.242+0000 7fbdd9438740 1 -- start start 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.243+0000 7fbdd9438740 1 Processor -- start 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.243+0000 7fbdd9438740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236dda1b0 0x558236de22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.243+0000 7fbdd9438740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236ddaad0 0x558236de2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.243+0000 7fbdd9438740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236de5390 0x558236de2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.243+0000 7fbdd9438740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558236de7fc0 con 0x558236dda1b0 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.243+0000 7fbdd9438740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x558236cfa2c0 con 0x558236de5390 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.243+0000 7fbdd9438740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558236de8140 con 0x558236ddaad0 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.243+0000 7fbdd9438740 10 client.0 _ll_register_callbacks cb 0x558236a52a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.243+0000 7fbdcf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236dda1b0 0x558236de22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.243+0000 7fbdcf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236dda1b0 0x558236de22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33340/0 (socket says 172.21.15.69:33340) 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.243+0000 7fbdcf400640 1 -- 172.21.15.69:0/1797827280 learned_addr learned my addr 172.21.15.69:0/1797827280 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.244+0000 7fbdc6a00640 1 --2- 172.21.15.69:0/1797827280 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236ddaad0 0x558236de2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.244+0000 7fbdcf400640 1 -- 172.21.15.69:0/1797827280 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236ddaad0 msgr2=0x558236de2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.244+0000 7fbdcf400640 1 --2- 172.21.15.69:0/1797827280 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236ddaad0 0x558236de2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.244+0000 7fbdcf400640 1 -- 172.21.15.69:0/1797827280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236de5390 msgr2=0x558236de2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.244+0000 7fbdcf400640 1 --2- 172.21.15.69:0/1797827280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236de5390 0x558236de2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.244+0000 7fbdcf400640 1 -- 172.21.15.69:0/1797827280 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558236d0afd0 con 0x558236dda1b0 2024-04-24T09:08:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.244+0000 7fbdcf400640 1 --2- 172.21.15.69:0/1797827280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236dda1b0 0x558236de22d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009fa0 tx=0x7fbdc0009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7383c63dd28a851 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.244+0000 7fbdbd600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.244+0000 7fbdbd600640 1 -- 172.21.15.69:0/1797827280 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0028060 con 0x558236dda1b0 2024-04-24T09:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.244+0000 7fbdbd600640 1 -- 172.21.15.69:0/1797827280 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fbdc0005040 con 0x558236dda1b0 2024-04-24T09:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.244+0000 7fbdd9438740 1 -- 172.21.15.69:0/1797827280 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc0011920 con 0x558236dda1b0 2024-04-24T09:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.245+0000 7fbdbd600640 1 -- 172.21.15.69:0/1797827280 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbdc0016070 con 0x558236dda1b0 2024-04-24T09:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.245+0000 7fbdd9438740 10 client.16071 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.245+0000 7fbdd9438740 1 -- 172.21.15.69:0/1797827280 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558236a53210 con 0x558236dda1b0 2024-04-24T09:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.245+0000 7fbdbd600640 1 -- 172.21.15.69:0/1797827280 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdc0011920 con 0x558236dda1b0 2024-04-24T09:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.245+0000 7fbdbd600640 1 -- 172.21.15.69:0/1797827280 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fbdc000f040 con 0x558236dda1b0 2024-04-24T09:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.245+0000 7fbdd9438740 10 client.16071 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.245+0000 7fbdd9438740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.245+0000 7fbdd9438740 20 client.16071 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.245+0000 7fbdd9438740 1 --2- 172.21.15.69:0/1797827280 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x558236df5390 0x558236df7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.245+0000 7fbdd9438740 4 client.16071 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.245+0000 7fbdd9438740 1 -- 172.21.15.69:0/1797827280 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558236d4e320 con 0x558236df5390 2024-04-24T09:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.246+0000 7fbdc6a00640 1 --2- 172.21.15.69:0/1797827280 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x558236df5390 0x558236df7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.247+0000 7fbdc6a00640 1 --2- 172.21.15.69:0/1797827280 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x558236df5390 0x558236df7780 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d45d27e20e5997a6 server_cookie=ae3173382843b60e in_seq=0 out_seq=0 2024-04-24T09:08:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.247+0000 7fbdbd600640 10 client.16071 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.249+0000 7fbdbd600640 1 -- 172.21.15.69:0/1797827280 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558236d4e320 con 0x558236df5390 2024-04-24T09:08:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.249+0000 7fbdbd600640 10 client.16071 handle_command_reply: tid=0 2024-04-24T09:08:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.249+0000 7fbdd9438740 1 client.16071 shutdown 2024-04-24T09:08:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.249+0000 7fbdd9438740 1 -- 172.21.15.69:0/1797827280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236dda1b0 msgr2=0x558236de22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.249+0000 7fbdd9438740 1 --2- 172.21.15.69:0/1797827280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236dda1b0 0x558236de22d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009fa0 tx=0x7fbdc0009fd0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.249+0000 7fbdd9438740 1 -- 172.21.15.69:0/1797827280 shutdown_connections 2024-04-24T09:08:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.249+0000 7fbdd9438740 1 --2- 172.21.15.69:0/1797827280 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x558236df5390 0x558236df7780 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.249+0000 7fbdd9438740 1 --2- 172.21.15.69:0/1797827280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236de5390 0x558236de2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.249+0000 7fbdd9438740 1 --2- 172.21.15.69:0/1797827280 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236ddaad0 0x558236de2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.249+0000 7fbdbd600640 0 client.16071 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.249+0000 7fbdd9438740 1 --2- 172.21.15.69:0/1797827280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236dda1b0 0x558236de22d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.249+0000 7fbdd9438740 1 -- 172.21.15.69:0/1797827280 >> 172.21.15.69:0/1797827280 conn(0x558236dd66c0 msgr2=0x558236dd5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.249+0000 7fbdd9438740 1 -- 172.21.15.69:0/1797827280 shutdown_connections 2024-04-24T09:08:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.250+0000 7fbdd9438740 1 -- 172.21.15.69:0/1797827280 wait complete. 2024-04-24T09:08:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.250+0000 7fbdd9438740 20 client.16071 trim_cache size 0 max 16384 2024-04-24T09:08:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.266+0000 7fbdd9438740 1 Processor -- start 2024-04-24T09:08:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.266+0000 7fbdd9438740 1 -- start start 2024-04-24T09:08:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.266+0000 7fbdd9438740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236dda1b0 0x558236df0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.266+0000 7fbdd9438740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236ddaad0 0x558236dfcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.266+0000 7fbdd9438740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236de5390 0x558236dfd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.266+0000 7fbdd9438740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01b3de0 con 0x558236dda1b0 2024-04-24T09:08:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.266+0000 7fbdd9438740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01b16b0 con 0x558236de5390 2024-04-24T09:08:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.266+0000 7fbdd9438740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01af500 con 0x558236ddaad0 2024-04-24T09:08:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.266+0000 7fbdcf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236dda1b0 0x558236df0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.266+0000 7fbdcf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236dda1b0 0x558236df0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33352/0 (socket says 172.21.15.69:33352) 2024-04-24T09:08:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.266+0000 7fbdcf400640 1 -- 172.21.15.69:0/1811964396 learned_addr learned my addr 172.21.15.69:0/1811964396 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.267+0000 7fbdc6a00640 1 --2- 172.21.15.69:0/1811964396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236ddaad0 0x558236dfcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.267+0000 7fbdcf400640 1 -- 172.21.15.69:0/1811964396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236ddaad0 msgr2=0x558236dfcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.267+0000 7fbdcf400640 1 --2- 172.21.15.69:0/1811964396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236ddaad0 0x558236dfcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.267+0000 7fbdcfe00640 1 --2- 172.21.15.69:0/1811964396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236de5390 0x558236dfd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.267+0000 7fbdcf400640 1 -- 172.21.15.69:0/1811964396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236de5390 msgr2=0x558236dfd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.267+0000 7fbdcf400640 1 --2- 172.21.15.69:0/1811964396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236de5390 0x558236dfd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.267+0000 7fbdcf400640 1 -- 172.21.15.69:0/1811964396 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558236b1f620 con 0x558236dda1b0 2024-04-24T09:08:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.267+0000 7fbdc6a00640 1 --2- 172.21.15.69:0/1811964396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236ddaad0 0x558236dfcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.267+0000 7fbdcf400640 1 --2- 172.21.15.69:0/1811964396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236dda1b0 0x558236df0e80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000ebc0 tx=0x7fbdc000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2349064e9156a5af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.267+0000 7fbdbfe00640 1 -- 172.21.15.69:0/1811964396 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0028070 con 0x558236dda1b0 2024-04-24T09:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.267+0000 7fbdbfe00640 1 -- 172.21.15.69:0/1811964396 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdc0011780 con 0x558236dda1b0 2024-04-24T09:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.267+0000 7fbdbfe00640 1 -- 172.21.15.69:0/1811964396 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0005600 con 0x558236dda1b0 2024-04-24T09:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.268+0000 7fbdd9438740 1 -- 172.21.15.69:0/1811964396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236dda1b0 msgr2=0x558236df0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.268+0000 7fbdd9438740 1 --2- 172.21.15.69:0/1811964396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236dda1b0 0x558236df0e80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000ebc0 tx=0x7fbdc000ecf0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.268+0000 7fbdd9438740 1 -- 172.21.15.69:0/1811964396 shutdown_connections 2024-04-24T09:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.268+0000 7fbdd9438740 1 --2- 172.21.15.69:0/1811964396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236de5390 0x558236dfd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.268+0000 7fbdd9438740 1 --2- 172.21.15.69:0/1811964396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236ddaad0 0x558236dfcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.268+0000 7fbdd9438740 1 --2- 172.21.15.69:0/1811964396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236dda1b0 0x558236df0e80 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.268+0000 7fbdd9438740 1 -- 172.21.15.69:0/1811964396 >> 172.21.15.69:0/1811964396 conn(0x558236dd66c0 msgr2=0x558236ddaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.268+0000 7fbdd9438740 1 -- 172.21.15.69:0/1811964396 shutdown_connections 2024-04-24T09:08:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.268+0000 7fbdd9438740 1 -- 172.21.15.69:0/1811964396 wait complete. 2024-04-24T09:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.269+0000 7fbdd9438740 1 -- start start 2024-04-24T09:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.269+0000 7fbdd9438740 1 Processor -- start 2024-04-24T09:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.269+0000 7fbdd9438740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236dda1b0 0x558236ded8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.269+0000 7fbdd9438740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236ddaad0 0x558236ddeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.269+0000 7fbdd9438740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236de5390 0x558236ddf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.269+0000 7fbdd9438740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01afb70 con 0x558236de5390 2024-04-24T09:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.269+0000 7fbdd9438740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01b0fe0 con 0x558236ddaad0 2024-04-24T09:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.269+0000 7fbdd9438740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01b0240 con 0x558236dda1b0 2024-04-24T09:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.269+0000 7fbdd9438740 10 client.0 _ll_register_callbacks cb 0x7fbdc0009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.269+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236de5390 0x558236ddf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.270+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236de5390 0x558236ddf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33356/0 (socket says 172.21.15.69:33356) 2024-04-24T09:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.270+0000 7fbdcfe00640 1 -- 172.21.15.69:0/2101726806 learned_addr learned my addr 172.21.15.69:0/2101726806 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.270+0000 7fbdcf400640 1 --2- 172.21.15.69:0/2101726806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236dda1b0 0x558236ded8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.270+0000 7fbdc6a00640 1 --2- 172.21.15.69:0/2101726806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236ddaad0 0x558236ddeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.270+0000 7fbdcfe00640 1 -- 172.21.15.69:0/2101726806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236dda1b0 msgr2=0x558236ded8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.270+0000 7fbdcfe00640 1 --2- 172.21.15.69:0/2101726806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236dda1b0 0x558236ded8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.270+0000 7fbdcfe00640 1 -- 172.21.15.69:0/2101726806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236ddaad0 msgr2=0x558236ddeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.270+0000 7fbdcfe00640 1 --2- 172.21.15.69:0/2101726806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236ddaad0 0x558236ddeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.270+0000 7fbdcfe00640 1 -- 172.21.15.69:0/2101726806 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558236e62e30 con 0x558236de5390 2024-04-24T09:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.270+0000 7fbdcfe00640 1 --2- 172.21.15.69:0/2101726806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236de5390 0x558236ddf4e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbdc809c680 tx=0x7fbdc809d1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae61136e9503e980 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.270+0000 7fbdbd600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.270+0000 7fbdbd600640 1 -- 172.21.15.69:0/2101726806 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc801f040 con 0x558236de5390 2024-04-24T09:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.270+0000 7fbdbd600640 1 -- 172.21.15.69:0/2101726806 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fbdc80870c0 con 0x558236de5390 2024-04-24T09:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.270+0000 7fbdd9438740 1 -- 172.21.15.69:0/2101726806 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc0011780 con 0x558236de5390 2024-04-24T09:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.271+0000 7fbdbd600640 1 -- 172.21.15.69:0/2101726806 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbdc80872c0 con 0x558236de5390 2024-04-24T09:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.271+0000 7fbdd9438740 10 client.16077 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.271+0000 7fbdd9438740 1 -- 172.21.15.69:0/2101726806 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558236e63f70 con 0x558236de5390 2024-04-24T09:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.271+0000 7fbdbd600640 1 -- 172.21.15.69:0/2101726806 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdc0011780 con 0x558236de5390 2024-04-24T09:08:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.271+0000 7fbdbd600640 1 -- 172.21.15.69:0/2101726806 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fbdc800f040 con 0x558236de5390 2024-04-24T09:08:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.271+0000 7fbdd9438740 10 client.16077 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.271+0000 7fbdd9438740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.271+0000 7fbdd9438740 20 client.16077 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.271+0000 7fbdd9438740 1 --2- 172.21.15.69:0/2101726806 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x558236e646f0 0x558236e66b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.271+0000 7fbdd9438740 4 client.16077 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:08:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.271+0000 7fbdd9438740 1 -- 172.21.15.69:0/2101726806 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558236d48190 con 0x558236e646f0 2024-04-24T09:08:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.272+0000 7fbdcf400640 1 --2- 172.21.15.69:0/2101726806 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x558236e646f0 0x558236e66b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.273+0000 7fbdcf400640 1 --2- 172.21.15.69:0/2101726806 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x558236e646f0 0x558236e66b10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a1aa2b64cfdc5a server_cookie=84da0a31fa40dc45 in_seq=0 out_seq=0 2024-04-24T09:08:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.273+0000 7fbdbd600640 10 client.16077 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:47.316 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:47 smithi081 ceph-mon[30961]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T09:08:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.335+0000 7fbdbd600640 1 -- 172.21.15.69:0/2101726806 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558236d48190 con 0x558236e646f0 2024-04-24T09:08:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.335+0000 7fbdbd600640 10 client.16077 handle_command_reply: tid=0 2024-04-24T09:08:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.335+0000 7fbdd9438740 1 client.16077 shutdown 2024-04-24T09:08:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.335+0000 7fbdd9438740 1 -- 172.21.15.69:0/2101726806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236de5390 msgr2=0x558236ddf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.335+0000 7fbdd9438740 1 --2- 172.21.15.69:0/2101726806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236de5390 0x558236ddf4e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbdc809c680 tx=0x7fbdc809d1c0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.336+0000 7fbdd9438740 1 -- 172.21.15.69:0/2101726806 shutdown_connections 2024-04-24T09:08:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.336+0000 7fbdd9438740 1 --2- 172.21.15.69:0/2101726806 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x558236e646f0 0x558236e66b10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.336+0000 7fbdd9438740 1 --2- 172.21.15.69:0/2101726806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558236de5390 0x558236ddf4e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.336+0000 7fbdd9438740 1 --2- 172.21.15.69:0/2101726806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558236ddaad0 0x558236ddeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.336+0000 7fbdd9438740 1 --2- 172.21.15.69:0/2101726806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558236dda1b0 0x558236ded8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.336+0000 7fbdbd600640 0 client.16077 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.336+0000 7fbdd9438740 1 -- 172.21.15.69:0/2101726806 >> 172.21.15.69:0/2101726806 conn(0x558236dd66c0 msgr2=0x558236dd4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.336+0000 7fbdd9438740 1 -- 172.21.15.69:0/2101726806 shutdown_connections 2024-04-24T09:08:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.336+0000 7fbdd9438740 1 -- 172.21.15.69:0/2101726806 wait complete. 2024-04-24T09:08:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.336+0000 7fbdd9438740 20 client.16077 trim_cache size 0 max 16384 2024-04-24T09:08:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:08:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:08:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:08:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:08:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.337+0000 7fbdd6600640 1 -- 172.21.15.69:0/3841018864 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fbdb40a3e40 msgr2=0x7fbdb40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.337+0000 7fbdd6600640 1 --2- 172.21.15.69:0/3841018864 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fbdb40a3e40 0x7fbdb40a6300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009230 tx=0x7fbdb8000bc0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.337+0000 7fbdd6600640 1 -- 172.21.15.69:0/3841018864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdd019a830 msgr2=0x7fbdd019ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.337+0000 7fbdd6600640 1 --2- 172.21.15.69:0/3841018864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdd019a830 0x7fbdd019ace0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800eaa0 tx=0x7fbdc800ef60 comp rx=0 tx=0).stop 2024-04-24T09:08:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.337+0000 7fbdd6600640 1 -- 172.21.15.69:0/3841018864 shutdown_connections 2024-04-24T09:08:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.337+0000 7fbdd6600640 1 --2- 172.21.15.69:0/3841018864 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fbdb40a3e40 0x7fbdb40a6300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.337+0000 7fbdd6600640 1 --2- 172.21.15.69:0/3841018864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdd019a830 0x7fbdd019ace0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.337+0000 7fbdd6600640 1 --2- 172.21.15.69:0/3841018864 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbdd0109540 0x7fbdd01a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.337+0000 7fbdd6600640 1 --2- 172.21.15.69:0/3841018864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbdd0104470 0x7fbdd01a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.337+0000 7fbdd6600640 1 -- 172.21.15.69:0/3841018864 >> 172.21.15.69:0/3841018864 conn(0x7fbdd00ffd30 msgr2=0x7fbdd01020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.337+0000 7fbdd6600640 1 -- 172.21.15.69:0/3841018864 shutdown_connections 2024-04-24T09:08:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:08:47.337+0000 7fbdd6600640 1 -- 172.21.15.69:0/3841018864 wait complete. 2024-04-24T09:08:47.340 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:47 smithi069 ceph-mon[28170]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T09:08:47.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:47 smithi130 ceph-mon[31553]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T09:08:49.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:08:49.268 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:08:49.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:49 smithi069 ceph-mon[28170]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T09:08:49.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:49 smithi130 ceph-mon[31553]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T09:08:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.499+0000 7fa198a00640 1 -- 172.21.15.69:0/3995674065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa19410c480 msgr2=0x7fa19410e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.499+0000 7fa198a00640 1 --2- 172.21.15.69:0/3995674065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa19410c480 0x7fa19410e920 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009900 tx=0x7fa18c02a480 comp rx=0 tx=0).stop 2024-04-24T09:08:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.499+0000 7fa198a00640 1 -- 172.21.15.69:0/3995674065 shutdown_connections 2024-04-24T09:08:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.499+0000 7fa198a00640 1 --2- 172.21.15.69:0/3995674065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa19410c480 0x7fa19410e920 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.499+0000 7fa198a00640 1 --2- 172.21.15.69:0/3995674065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa194103980 0x7fa19410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.499+0000 7fa198a00640 1 --2- 172.21.15.69:0/3995674065 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa194103060 0x7fa194103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.499+0000 7fa198a00640 1 -- 172.21.15.69:0/3995674065 >> 172.21.15.69:0/3995674065 conn(0x7fa1940fc880 msgr2=0x7fa1940feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.499+0000 7fa198a00640 1 -- 172.21.15.69:0/3995674065 shutdown_connections 2024-04-24T09:08:49.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.499+0000 7fa198a00640 1 -- 172.21.15.69:0/3995674065 wait complete. 2024-04-24T09:08:49.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.500+0000 7fa198a00640 1 Processor -- start 2024-04-24T09:08:49.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.501+0000 7fa198a00640 1 -- start start 2024-04-24T09:08:49.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.501+0000 7fa198a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa194103060 0x7fa1941a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.501+0000 7fa198a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa194103980 0x7fa1941a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.501+0000 7fa198a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa19410c480 0x7fa19419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.501+0000 7fa198a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa194112160 con 0x7fa194103060 2024-04-24T09:08:49.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.501+0000 7fa198a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa194111fe0 con 0x7fa19410c480 2024-04-24T09:08:49.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.501+0000 7fa198a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa1941122e0 con 0x7fa194103980 2024-04-24T09:08:49.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.501+0000 7fa192800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa194103060 0x7fa1941a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.501+0000 7fa192800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa194103060 0x7fa1941a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33384/0 (socket says 172.21.15.69:33384) 2024-04-24T09:08:49.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.501+0000 7fa192800640 1 -- 172.21.15.69:0/2632429611 learned_addr learned my addr 172.21.15.69:0/2632429611 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:49.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.502+0000 7fa191e00640 1 --2- 172.21.15.69:0/2632429611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa194103980 0x7fa1941a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.502+0000 7fa193200640 1 --2- 172.21.15.69:0/2632429611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa19410c480 0x7fa19419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.502+0000 7fa192800640 1 -- 172.21.15.69:0/2632429611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa194103980 msgr2=0x7fa1941a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:49.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.502+0000 7fa192800640 1 --2- 172.21.15.69:0/2632429611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa194103980 0x7fa1941a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.502+0000 7fa192800640 1 -- 172.21.15.69:0/2632429611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa19410c480 msgr2=0x7fa19419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:49.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.502+0000 7fa192800640 1 --2- 172.21.15.69:0/2632429611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa19410c480 0x7fa19419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.502+0000 7fa192800640 1 -- 172.21.15.69:0/2632429611 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19419af60 con 0x7fa194103060 2024-04-24T09:08:49.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.502+0000 7fa193200640 1 --2- 172.21.15.69:0/2632429611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa19410c480 0x7fa19419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:49.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.502+0000 7fa192800640 1 --2- 172.21.15.69:0/2632429611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa194103060 0x7fa1941a05e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa18400eb90 tx=0x7fa18400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:49.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.503+0000 7fa18b400640 1 -- 172.21.15.69:0/2632429611 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184004d20 con 0x7fa194103060 2024-04-24T09:08:49.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.503+0000 7fa18b400640 1 -- 172.21.15.69:0/2632429611 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa18400ee90 con 0x7fa194103060 2024-04-24T09:08:49.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.503+0000 7fa198a00640 1 -- 172.21.15.69:0/2632429611 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19419b250 con 0x7fa194103060 2024-04-24T09:08:49.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.503+0000 7fa18b400640 1 -- 172.21.15.69:0/2632429611 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184010670 con 0x7fa194103060 2024-04-24T09:08:49.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.503+0000 7fa198a00640 1 -- 172.21.15.69:0/2632429611 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1941a7560 con 0x7fa194103060 2024-04-24T09:08:49.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.506+0000 7fa18b400640 1 -- 172.21.15.69:0/2632429611 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa184005020 con 0x7fa194103060 2024-04-24T09:08:49.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.506+0000 7fa19c210740 1 Processor -- start 2024-04-24T09:08:49.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.507+0000 7fa19c210740 1 -- start start 2024-04-24T09:08:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.507+0000 7fa19c210740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 0x55ad8c3dae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.507+0000 7fa19c210740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3db350 0x55ad8c3e54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.507+0000 7fa19c210740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3e7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.507+0000 7fa19c210740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ad8c3401f0 con 0x55ad8c3e5a20 2024-04-24T09:08:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.507+0000 7fa19c210740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ad8c2e6350 con 0x55ad8c3db350 2024-04-24T09:08:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.507+0000 7fa19c210740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ad8c33c1b0 con 0x55ad8c3daa30 2024-04-24T09:08:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.508+0000 7fa191e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3e7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.508+0000 7fa191e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3e7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33386/0 (socket says 172.21.15.69:33386) 2024-04-24T09:08:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.508+0000 7fa191e00640 1 -- 172.21.15.69:0/1866830375 learned_addr learned my addr 172.21.15.69:0/1866830375 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.508+0000 7fa193200640 1 --2- 172.21.15.69:0/1866830375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 0x55ad8c3dae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.508+0000 7fa192800640 1 --2- 172.21.15.69:0/1866830375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3db350 0x55ad8c3e54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.508+0000 7fa191e00640 1 -- 172.21.15.69:0/1866830375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 msgr2=0x55ad8c3dae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.508+0000 7fa191e00640 1 --2- 172.21.15.69:0/1866830375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 0x55ad8c3dae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.508+0000 7fa191e00640 1 -- 172.21.15.69:0/1866830375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3db350 msgr2=0x55ad8c3e54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.508+0000 7fa191e00640 1 --2- 172.21.15.69:0/1866830375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3db350 0x55ad8c3e54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.508+0000 7fa191e00640 1 -- 172.21.15.69:0/1866830375 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad8c33af50 con 0x55ad8c3e5a20 2024-04-24T09:08:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.508+0000 7fa193200640 1 --2- 172.21.15.69:0/1866830375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 0x55ad8c3dae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.508+0000 7fa18b400640 1 --2- 172.21.15.69:0/2632429611 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa1780a3da0 0x7fa1780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.508+0000 7fa191e00640 1 --2- 172.21.15.69:0/1866830375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3e7e10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00bbf0 tx=0x7fa17c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877e7d5ee4ba30fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.509+0000 7fa18b400640 1 -- 172.21.15.69:0/2632429611 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa1840d12c0 con 0x7fa194103060 2024-04-24T09:08:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.509+0000 7fa183400640 1 -- 172.21.15.69:0/1866830375 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c015070 con 0x55ad8c3e5a20 2024-04-24T09:08:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.509+0000 7fa183400640 1 -- 172.21.15.69:0/1866830375 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa17c01a910 con 0x55ad8c3e5a20 2024-04-24T09:08:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.509+0000 7fa183400640 1 -- 172.21.15.69:0/1866830375 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0118e0 con 0x55ad8c3e5a20 2024-04-24T09:08:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.509+0000 7fa193200640 1 --2- 172.21.15.69:0/2632429611 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa1780a3da0 0x7fa1780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.510+0000 7fa19c210740 1 -- 172.21.15.69:0/1866830375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3e5a20 msgr2=0x55ad8c3e7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.510+0000 7fa19c210740 1 --2- 172.21.15.69:0/1866830375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3e7e10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00bbf0 tx=0x7fa17c00f730 comp rx=0 tx=0).stop 2024-04-24T09:08:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.510+0000 7fa193200640 1 --2- 172.21.15.69:0/2632429611 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa1780a3da0 0x7fa1780a6260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa18c031440 tx=0x7fa18c005b70 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.510+0000 7fa19c210740 1 -- 172.21.15.69:0/1866830375 shutdown_connections 2024-04-24T09:08:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.510+0000 7fa19c210740 1 --2- 172.21.15.69:0/1866830375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3e7e10 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.510+0000 7fa19c210740 1 --2- 172.21.15.69:0/1866830375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3db350 0x55ad8c3e54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.510+0000 7fa19c210740 1 --2- 172.21.15.69:0/1866830375 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 0x55ad8c3dae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.510+0000 7fa19c210740 1 -- 172.21.15.69:0/1866830375 >> 172.21.15.69:0/1866830375 conn(0x55ad8c3d6d50 msgr2=0x55ad8c3d7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.510+0000 7fa19c210740 1 -- 172.21.15.69:0/1866830375 shutdown_connections 2024-04-24T09:08:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.510+0000 7fa19c210740 1 -- 172.21.15.69:0/1866830375 wait complete. 2024-04-24T09:08:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.511+0000 7fa19c210740 1 -- start start 2024-04-24T09:08:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.511+0000 7fa19c210740 1 Processor -- start 2024-04-24T09:08:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.511+0000 7fa19c210740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3daa30 0x55ad8c3e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.511+0000 7fa19c210740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3db350 0x55ad8c3e2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.511+0000 7fa19c210740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3e3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.511+0000 7fa19c210740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ad8c3e8810 con 0x55ad8c3daa30 2024-04-24T09:08:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.511+0000 7fa19c210740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ad8c2f9690 con 0x55ad8c3db350 2024-04-24T09:08:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.511+0000 7fa19c210740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ad8c3e8990 con 0x55ad8c3e5a20 2024-04-24T09:08:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.512+0000 7fa19c210740 10 client.0 _ll_register_callbacks cb 0x55ad8c335650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.512+0000 7fa192800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3daa30 0x55ad8c3e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.512+0000 7fa192800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3daa30 0x55ad8c3e2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33390/0 (socket says 172.21.15.69:33390) 2024-04-24T09:08:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.512+0000 7fa192800640 1 -- 172.21.15.69:0/1870019296 learned_addr learned my addr 172.21.15.69:0/1870019296 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.512+0000 7fa191e00640 1 --2- 172.21.15.69:0/1870019296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3db350 0x55ad8c3e2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.512+0000 7fa193200640 1 --2- 172.21.15.69:0/1870019296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3e3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.512+0000 7fa192800640 1 -- 172.21.15.69:0/1870019296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3e5a20 msgr2=0x55ad8c3e3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.512+0000 7fa192800640 1 --2- 172.21.15.69:0/1870019296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3e3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.512+0000 7fa192800640 1 -- 172.21.15.69:0/1870019296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3db350 msgr2=0x55ad8c3e2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.512+0000 7fa192800640 1 --2- 172.21.15.69:0/1870019296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3db350 0x55ad8c3e2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.512+0000 7fa192800640 1 -- 172.21.15.69:0/1870019296 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad8c2f8920 con 0x55ad8c3daa30 2024-04-24T09:08:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.513+0000 7fa192800640 1 --2- 172.21.15.69:0/1870019296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3daa30 0x55ad8c3e2a10 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa1840049d0 tx=0x7fa18409cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9291b198efb595 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.513+0000 7fa177e00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.513+0000 7fa177e00640 1 -- 172.21.15.69:0/1870019296 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18409cec0 con 0x55ad8c3daa30 2024-04-24T09:08:49.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.513+0000 7fa177e00640 1 -- 172.21.15.69:0/1870019296 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa184005560 con 0x55ad8c3daa30 2024-04-24T09:08:49.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.513+0000 7fa19c210740 1 -- 172.21.15.69:0/1870019296 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa17c01a910 con 0x55ad8c3daa30 2024-04-24T09:08:49.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.513+0000 7fa177e00640 1 -- 172.21.15.69:0/1870019296 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa1840a1270 con 0x55ad8c3daa30 2024-04-24T09:08:49.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.513+0000 7fa19c210740 10 client.16089 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:49.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.513+0000 7fa19c210740 1 -- 172.21.15.69:0/1870019296 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ad8c11c7d0 con 0x55ad8c3daa30 2024-04-24T09:08:49.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.513+0000 7fa177e00640 1 -- 172.21.15.69:0/1870019296 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa17c01a910 con 0x55ad8c3daa30 2024-04-24T09:08:49.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.514+0000 7fa177e00640 1 -- 172.21.15.69:0/1870019296 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fa18407bce0 con 0x55ad8c3daa30 2024-04-24T09:08:49.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.514+0000 7fa19c210740 10 client.16089 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:49.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.514+0000 7fa19c210740 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:49.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.514+0000 7fa19c210740 20 client.16089 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:49.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.514+0000 7fa19c210740 1 --2- 172.21.15.69:0/1870019296 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55ad8c3f5dd0 0x55ad8c3f8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.514+0000 7fa19c210740 4 client.16089 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:49.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.514+0000 7fa19c210740 1 -- 172.21.15.69:0/1870019296 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad8c34dc10 con 0x55ad8c3f5dd0 2024-04-24T09:08:49.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.515+0000 7fa191e00640 1 --2- 172.21.15.69:0/1870019296 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55ad8c3f5dd0 0x55ad8c3f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.516+0000 7fa191e00640 1 --2- 172.21.15.69:0/1870019296 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55ad8c3f5dd0 0x55ad8c3f8260 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e41425783d04411 server_cookie=dc4f441c3ec7b836 in_seq=0 out_seq=0 2024-04-24T09:08:49.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.516+0000 7fa177e00640 10 client.16089 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:49.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.523+0000 7fa177e00640 1 -- 172.21.15.69:0/1870019296 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad8c34dc10 con 0x55ad8c3f5dd0 2024-04-24T09:08:49.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.523+0000 7fa177e00640 10 client.16089 handle_command_reply: tid=0 2024-04-24T09:08:49.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.523+0000 7fa19c210740 1 client.16089 shutdown 2024-04-24T09:08:49.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.523+0000 7fa19c210740 1 -- 172.21.15.69:0/1870019296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3daa30 msgr2=0x55ad8c3e2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:49.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.523+0000 7fa19c210740 1 --2- 172.21.15.69:0/1870019296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3daa30 0x55ad8c3e2a10 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa1840049d0 tx=0x7fa18409cd50 comp rx=0 tx=0).stop 2024-04-24T09:08:49.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.523+0000 7fa19c210740 1 -- 172.21.15.69:0/1870019296 shutdown_connections 2024-04-24T09:08:49.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.523+0000 7fa19c210740 1 --2- 172.21.15.69:0/1870019296 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55ad8c3f5dd0 0x55ad8c3f8260 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.523+0000 7fa19c210740 1 --2- 172.21.15.69:0/1870019296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3e3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.523+0000 7fa19c210740 1 --2- 172.21.15.69:0/1870019296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3db350 0x55ad8c3e2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.523+0000 7fa19c210740 1 --2- 172.21.15.69:0/1870019296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3daa30 0x55ad8c3e2a10 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.523+0000 7fa177e00640 0 client.16089 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:49.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.523+0000 7fa19c210740 1 -- 172.21.15.69:0/1870019296 >> 172.21.15.69:0/1870019296 conn(0x55ad8c3d6d50 msgr2=0x55ad8c3d5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:49.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.523+0000 7fa19c210740 1 -- 172.21.15.69:0/1870019296 shutdown_connections 2024-04-24T09:08:49.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.523+0000 7fa19c210740 1 -- 172.21.15.69:0/1870019296 wait complete. 2024-04-24T09:08:49.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.524+0000 7fa19c210740 20 client.16089 trim_cache size 0 max 16384 2024-04-24T09:08:49.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.550+0000 7fa19c210740 1 Processor -- start 2024-04-24T09:08:49.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.550+0000 7fa19c210740 1 -- start start 2024-04-24T09:08:49.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.550+0000 7fa19c210740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 0x55ad8c3f1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.550+0000 7fa19c210740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3db350 0x55ad8c3e3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.550+0000 7fa19c210740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3fdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.550+0000 7fa19c210740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1941b3b70 con 0x55ad8c3db350 2024-04-24T09:08:49.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.550+0000 7fa19c210740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa1941b1440 con 0x55ad8c3e5a20 2024-04-24T09:08:49.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.550+0000 7fa19c210740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa1941af290 con 0x55ad8c3daa30 2024-04-24T09:08:49.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.551+0000 7fa191e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3db350 0x55ad8c3e3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.551+0000 7fa191e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3db350 0x55ad8c3e3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33404/0 (socket says 172.21.15.69:33404) 2024-04-24T09:08:49.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.551+0000 7fa191e00640 1 -- 172.21.15.69:0/2906298427 learned_addr learned my addr 172.21.15.69:0/2906298427 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:49.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.551+0000 7fa193200640 1 --2- 172.21.15.69:0/2906298427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3fdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.551+0000 7fa192800640 1 --2- 172.21.15.69:0/2906298427 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 0x55ad8c3f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.551+0000 7fa191e00640 1 -- 172.21.15.69:0/2906298427 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 msgr2=0x55ad8c3f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:49.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.551+0000 7fa191e00640 1 --2- 172.21.15.69:0/2906298427 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 0x55ad8c3f1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.551+0000 7fa191e00640 1 -- 172.21.15.69:0/2906298427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3e5a20 msgr2=0x55ad8c3fdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:49.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.551+0000 7fa191e00640 1 --2- 172.21.15.69:0/2906298427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3fdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.551+0000 7fa191e00640 1 -- 172.21.15.69:0/2906298427 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad8c0523f0 con 0x55ad8c3db350 2024-04-24T09:08:49.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.551+0000 7fa192800640 1 --2- 172.21.15.69:0/2906298427 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 0x55ad8c3f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:49.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.551+0000 7fa193200640 1 --2- 172.21.15.69:0/2906298427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3fdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:49.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.552+0000 7fa191e00640 1 --2- 172.21.15.69:0/2906298427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3db350 0x55ad8c3e3260 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00b9b0 tx=0x7fa17c00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627a45bd8909ce89 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:49.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.552+0000 7fa182a00640 1 -- 172.21.15.69:0/2906298427 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c015040 con 0x55ad8c3db350 2024-04-24T09:08:49.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.552+0000 7fa182a00640 1 -- 172.21.15.69:0/2906298427 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa17c0054f0 con 0x55ad8c3db350 2024-04-24T09:08:49.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.552+0000 7fa182a00640 1 -- 172.21.15.69:0/2906298427 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c004ae0 con 0x55ad8c3db350 2024-04-24T09:08:49.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.553+0000 7fa19c210740 1 -- 172.21.15.69:0/2906298427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3db350 msgr2=0x55ad8c3e3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:49.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.553+0000 7fa19c210740 1 --2- 172.21.15.69:0/2906298427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3db350 0x55ad8c3e3260 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00b9b0 tx=0x7fa17c00bf20 comp rx=0 tx=0).stop 2024-04-24T09:08:49.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.553+0000 7fa19c210740 1 -- 172.21.15.69:0/2906298427 shutdown_connections 2024-04-24T09:08:49.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.553+0000 7fa19c210740 1 --2- 172.21.15.69:0/2906298427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3fdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.553+0000 7fa19c210740 1 --2- 172.21.15.69:0/2906298427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3db350 0x55ad8c3e3260 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.553+0000 7fa19c210740 1 --2- 172.21.15.69:0/2906298427 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 0x55ad8c3f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.553+0000 7fa19c210740 1 -- 172.21.15.69:0/2906298427 >> 172.21.15.69:0/2906298427 conn(0x55ad8c3d6d50 msgr2=0x55ad8c3e1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:49.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.553+0000 7fa19c210740 1 -- 172.21.15.69:0/2906298427 shutdown_connections 2024-04-24T09:08:49.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.553+0000 7fa19c210740 1 -- 172.21.15.69:0/2906298427 wait complete. 2024-04-24T09:08:49.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.554+0000 7fa19c210740 1 -- start start 2024-04-24T09:08:49.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.554+0000 7fa19c210740 1 Processor -- start 2024-04-24T09:08:49.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.554+0000 7fa19c210740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 0x55ad8c3df9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.554+0000 7fa19c210740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3db350 0x55ad8c3dfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.555+0000 7fa19c210740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3e0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.555+0000 7fa19c210740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1941af900 con 0x55ad8c3e5a20 2024-04-24T09:08:49.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.555+0000 7fa19c210740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa1941b0d70 con 0x55ad8c3db350 2024-04-24T09:08:49.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.555+0000 7fa19c210740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa1941affd0 con 0x55ad8c3daa30 2024-04-24T09:08:49.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.555+0000 7fa19c210740 10 client.0 _ll_register_callbacks cb 0x7fa1840bd040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:49.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.555+0000 7fa193200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3e0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.555+0000 7fa193200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3e0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33418/0 (socket says 172.21.15.69:33418) 2024-04-24T09:08:49.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.555+0000 7fa193200640 1 -- 172.21.15.69:0/131935575 learned_addr learned my addr 172.21.15.69:0/131935575 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:49.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.555+0000 7fa192800640 1 --2- 172.21.15.69:0/131935575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 0x55ad8c3df9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.555+0000 7fa191e00640 1 --2- 172.21.15.69:0/131935575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3db350 0x55ad8c3dfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.555+0000 7fa193200640 1 -- 172.21.15.69:0/131935575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 msgr2=0x55ad8c3df9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:49.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.555+0000 7fa193200640 1 --2- 172.21.15.69:0/131935575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 0x55ad8c3df9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.555+0000 7fa193200640 1 -- 172.21.15.69:0/131935575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3db350 msgr2=0x55ad8c3dfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:49.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.555+0000 7fa193200640 1 --2- 172.21.15.69:0/131935575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3db350 0x55ad8c3dfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.555+0000 7fa193200640 1 -- 172.21.15.69:0/131935575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad8c0120f0 con 0x55ad8c3e5a20 2024-04-24T09:08:49.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.556+0000 7fa191e00640 1 --2- 172.21.15.69:0/131935575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3db350 0x55ad8c3dfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:49.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.556+0000 7fa193200640 1 --2- 172.21.15.69:0/131935575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3e0420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa18c02a9a0 tx=0x7fa18c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54d458d52eff881 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:49.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.556+0000 7fa177e00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:49.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.556+0000 7fa177e00640 1 -- 172.21.15.69:0/131935575 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c035050 con 0x55ad8c3e5a20 2024-04-24T09:08:49.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.556+0000 7fa177e00640 1 -- 172.21.15.69:0/131935575 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa18c004ad0 con 0x55ad8c3e5a20 2024-04-24T09:08:49.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.556+0000 7fa19c210740 1 -- 172.21.15.69:0/131935575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa17c0054f0 con 0x55ad8c3e5a20 2024-04-24T09:08:49.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.557+0000 7fa177e00640 1 -- 172.21.15.69:0/131935575 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa18c05b050 con 0x55ad8c3e5a20 2024-04-24T09:08:49.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.557+0000 7fa19c210740 10 client.16095 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:49.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.557+0000 7fa19c210740 1 -- 172.21.15.69:0/131935575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ad8c305810 con 0x55ad8c3e5a20 2024-04-24T09:08:49.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.557+0000 7fa177e00640 1 -- 172.21.15.69:0/131935575 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa17c0054f0 con 0x55ad8c3e5a20 2024-04-24T09:08:49.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.557+0000 7fa177e00640 1 -- 172.21.15.69:0/131935575 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fa18c0324b0 con 0x55ad8c3e5a20 2024-04-24T09:08:49.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.557+0000 7fa19c210740 10 client.16095 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:49.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.557+0000 7fa19c210740 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:49.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.557+0000 7fa19c210740 20 client.16095 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:49.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.557+0000 7fa19c210740 1 --2- 172.21.15.69:0/131935575 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55ad8c45ccc0 0x55ad8c45f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.558+0000 7fa19c210740 4 client.16095 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:08:49.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.558+0000 7fa19c210740 1 -- 172.21.15.69:0/131935575 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ad8c347a80 con 0x55ad8c45ccc0 2024-04-24T09:08:49.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.558+0000 7fa192800640 1 --2- 172.21.15.69:0/131935575 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55ad8c45ccc0 0x55ad8c45f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.559+0000 7fa192800640 1 --2- 172.21.15.69:0/131935575 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55ad8c45ccc0 0x55ad8c45f0e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc765a578a73b24c server_cookie=2b615851da562d80 in_seq=0 out_seq=0 2024-04-24T09:08:49.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.559+0000 7fa177e00640 10 client.16095 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:49.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.560+0000 7fa177e00640 1 -- 172.21.15.69:0/131935575 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ad8c347a80 con 0x55ad8c45ccc0 2024-04-24T09:08:49.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.560+0000 7fa177e00640 10 client.16095 handle_command_reply: tid=0 2024-04-24T09:08:49.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.560+0000 7fa19c210740 1 client.16095 shutdown 2024-04-24T09:08:49.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.560+0000 7fa19c210740 1 -- 172.21.15.69:0/131935575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3e5a20 msgr2=0x55ad8c3e0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:49.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.560+0000 7fa19c210740 1 --2- 172.21.15.69:0/131935575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3e0420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa18c02a9a0 tx=0x7fa18c02a480 comp rx=0 tx=0).stop 2024-04-24T09:08:49.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.560+0000 7fa19c210740 1 -- 172.21.15.69:0/131935575 shutdown_connections 2024-04-24T09:08:49.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.560+0000 7fa19c210740 1 --2- 172.21.15.69:0/131935575 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55ad8c45ccc0 0x55ad8c45f0e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.560+0000 7fa19c210740 1 --2- 172.21.15.69:0/131935575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad8c3e5a20 0x55ad8c3e0420 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.560+0000 7fa19c210740 1 --2- 172.21.15.69:0/131935575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ad8c3db350 0x55ad8c3dfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.560+0000 7fa177e00640 0 client.16095 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:49.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.560+0000 7fa19c210740 1 --2- 172.21.15.69:0/131935575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ad8c3daa30 0x55ad8c3df9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.560+0000 7fa19c210740 1 -- 172.21.15.69:0/131935575 >> 172.21.15.69:0/131935575 conn(0x55ad8c3d6d50 msgr2=0x55ad8c3dca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:49.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.560+0000 7fa19c210740 1 -- 172.21.15.69:0/131935575 shutdown_connections 2024-04-24T09:08:49.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.560+0000 7fa19c210740 1 -- 172.21.15.69:0/131935575 wait complete. 2024-04-24T09:08:49.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.561+0000 7fa19c210740 20 client.16095 trim_cache size 0 max 16384 2024-04-24T09:08:49.562 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:08:49.562 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-04-24T09:08:49.563 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-04-24T09:08:49.563 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:08:49.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.561+0000 7fa198a00640 1 -- 172.21.15.69:0/2632429611 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa1780a3da0 msgr2=0x7fa1780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:49.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.561+0000 7fa198a00640 1 --2- 172.21.15.69:0/2632429611 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa1780a3da0 0x7fa1780a6260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa18c031440 tx=0x7fa18c005b70 comp rx=0 tx=0).stop 2024-04-24T09:08:49.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.561+0000 7fa198a00640 1 -- 172.21.15.69:0/2632429611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa194103060 msgr2=0x7fa1941a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:49.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.561+0000 7fa198a00640 1 --2- 172.21.15.69:0/2632429611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa194103060 0x7fa1941a05e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa18400eb90 tx=0x7fa18400c6a0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.562+0000 7fa198a00640 1 -- 172.21.15.69:0/2632429611 shutdown_connections 2024-04-24T09:08:49.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.562+0000 7fa198a00640 1 --2- 172.21.15.69:0/2632429611 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa1780a3da0 0x7fa1780a6260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.562+0000 7fa198a00640 1 --2- 172.21.15.69:0/2632429611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa19410c480 0x7fa19419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.562+0000 7fa198a00640 1 --2- 172.21.15.69:0/2632429611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa194103980 0x7fa1941a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.562+0000 7fa198a00640 1 --2- 172.21.15.69:0/2632429611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa194103060 0x7fa1941a05e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.562+0000 7fa198a00640 1 -- 172.21.15.69:0/2632429611 >> 172.21.15.69:0/2632429611 conn(0x7fa1940fc880 msgr2=0x7fa194100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:49.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.562+0000 7fa198a00640 1 -- 172.21.15.69:0/2632429611 shutdown_connections 2024-04-24T09:08:49.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.562+0000 7fa198a00640 1 -- 172.21.15.69:0/2632429611 wait complete. 2024-04-24T09:08:49.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:49 smithi081 ceph-mon[30961]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T09:08:49.581 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:08:49.581 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:08:49.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.780+0000 7f4832600640 1 -- 172.21.15.69:0/1144188349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f482c074cf0 msgr2=0x7f482c10e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:49.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.780+0000 7f4832600640 1 --2- 172.21.15.69:0/1144188349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f482c074cf0 0x7f482c10e370 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4818009900 tx=0x7f481802a480 comp rx=0 tx=0).stop 2024-04-24T09:08:49.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.780+0000 7f4832600640 1 -- 172.21.15.69:0/1144188349 shutdown_connections 2024-04-24T09:08:49.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.780+0000 7f4832600640 1 --2- 172.21.15.69:0/1144188349 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f482c10e8b0 0x7f482c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.780+0000 7f4832600640 1 --2- 172.21.15.69:0/1144188349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f482c074cf0 0x7f482c10e370 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.780+0000 7f4832600640 1 --2- 172.21.15.69:0/1144188349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f482c0743d0 0x7f482c0747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.780+0000 7f4832600640 1 -- 172.21.15.69:0/1144188349 >> 172.21.15.69:0/1144188349 conn(0x7f482c079490 msgr2=0x7f482c0798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:49.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.780+0000 7f4832600640 1 -- 172.21.15.69:0/1144188349 shutdown_connections 2024-04-24T09:08:49.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.780+0000 7f4832600640 1 -- 172.21.15.69:0/1144188349 wait complete. 2024-04-24T09:08:49.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.781+0000 7f4832600640 1 Processor -- start 2024-04-24T09:08:49.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.782+0000 7f4832600640 1 -- start start 2024-04-24T09:08:49.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.782+0000 7f4832600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f482c0743d0 0x7f482c1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.782+0000 7f4832600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f482c074cf0 0x7f482c1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.782+0000 7f4832600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f482c10e8b0 0x7f482c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.782+0000 7f4832600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f482c114530 con 0x7f482c10e8b0 2024-04-24T09:08:49.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.782+0000 7f4832600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1143b0 con 0x7f482c0743d0 2024-04-24T09:08:49.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.782+0000 7f4832600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1146b0 con 0x7f482c074cf0 2024-04-24T09:08:49.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.782+0000 7f482be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f482c10e8b0 0x7f482c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.783+0000 7f482be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f482c10e8b0 0x7f482c19a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33434/0 (socket says 172.21.15.69:33434) 2024-04-24T09:08:49.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.783+0000 7f482be00640 1 -- 172.21.15.69:0/3359343443 learned_addr learned my addr 172.21.15.69:0/3359343443 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:49.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.783+0000 7f482aa00640 1 --2- 172.21.15.69:0/3359343443 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f482c074cf0 0x7f482c1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.783+0000 7f482b400640 1 --2- 172.21.15.69:0/3359343443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f482c0743d0 0x7f482c1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.783+0000 7f482be00640 1 -- 172.21.15.69:0/3359343443 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f482c074cf0 msgr2=0x7f482c1a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:49.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.783+0000 7f482be00640 1 --2- 172.21.15.69:0/3359343443 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f482c074cf0 0x7f482c1a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.783+0000 7f482be00640 1 -- 172.21.15.69:0/3359343443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f482c0743d0 msgr2=0x7f482c1a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:49.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.783+0000 7f482be00640 1 --2- 172.21.15.69:0/3359343443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f482c0743d0 0x7f482c1a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:49.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.783+0000 7f482be00640 1 -- 172.21.15.69:0/3359343443 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482c19b0b0 con 0x7f482c10e8b0 2024-04-24T09:08:49.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.783+0000 7f482aa00640 1 --2- 172.21.15.69:0/3359343443 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f482c074cf0 0x7f482c1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:49.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.783+0000 7f482be00640 1 --2- 172.21.15.69:0/3359343443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f482c10e8b0 0x7f482c19a930 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f482400ea80 tx=0x7f482400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:49.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.784+0000 7f4823e00640 1 -- 172.21.15.69:0/3359343443 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482400ce20 con 0x7f482c10e8b0 2024-04-24T09:08:49.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.784+0000 7f4823e00640 1 -- 172.21.15.69:0/3359343443 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4824005020 con 0x7f482c10e8b0 2024-04-24T09:08:49.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.784+0000 7f4832600640 1 -- 172.21.15.69:0/3359343443 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482c19b3a0 con 0x7f482c10e8b0 2024-04-24T09:08:49.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.784+0000 7f4823e00640 1 -- 172.21.15.69:0/3359343443 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824010690 con 0x7f482c10e8b0 2024-04-24T09:08:49.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.784+0000 7f4832600640 1 -- 172.21.15.69:0/3359343443 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f482c1a7640 con 0x7f482c10e8b0 2024-04-24T09:08:49.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.785+0000 7f4832600640 1 -- 172.21.15.69:0/3359343443 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f482c068590 con 0x7f482c10e8b0 2024-04-24T09:08:49.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.787+0000 7f4823e00640 1 -- 172.21.15.69:0/3359343443 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f48240051d0 con 0x7f482c10e8b0 2024-04-24T09:08:49.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.789+0000 7f4823e00640 1 --2- 172.21.15.69:0/3359343443 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f48100a3da0 0x7f48100a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:49.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.789+0000 7f4823e00640 1 -- 172.21.15.69:0/3359343443 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f48240d1220 con 0x7f482c10e8b0 2024-04-24T09:08:49.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.789+0000 7f482b400640 1 --2- 172.21.15.69:0/3359343443 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f48100a3da0 0x7f48100a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:49.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.790+0000 7f482b400640 1 --2- 172.21.15.69:0/3359343443 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f48100a3da0 0x7f48100a6260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f482c19ba70 tx=0x7f481c005f90 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:49.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:49.794+0000 7f4823e00640 1 -- 172.21.15.69:0/3359343443 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f482409d580 con 0x7f482c10e8b0 2024-04-24T09:08:50.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.071+0000 7f4832600640 1 -- 172.21.15.69:0/3359343443 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f482c19be60 con 0x7f482c10e8b0 2024-04-24T09:08:50.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.072+0000 7f4823e00640 1 -- 172.21.15.69:0/3359343443 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11873 (secure 0 0 0) 0x7f482c19be60 con 0x7f482c10e8b0 2024-04-24T09:08:50.073 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:08:50.073 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:04:01.991559+0000","modified":"2024-04-24T09:07:02.537848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34532,"mds_1":34496,"mds_2":34460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34496":{"gid":34496,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34532":{"gid":34532,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34532,"qdb_cluster":[34532,34460,34496]},"id":1}]} 2024-04-24T09:08:50.073 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 24 2024-04-24T09:08:50.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.075+0000 7f4832600640 1 -- 172.21.15.69:0/3359343443 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f48100a3da0 msgr2=0x7f48100a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.075+0000 7f4832600640 1 --2- 172.21.15.69:0/3359343443 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f48100a3da0 0x7f48100a6260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f482c19ba70 tx=0x7f481c005f90 comp rx=0 tx=0).stop 2024-04-24T09:08:50.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.075+0000 7f4832600640 1 -- 172.21.15.69:0/3359343443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f482c10e8b0 msgr2=0x7f482c19a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.075+0000 7f4832600640 1 --2- 172.21.15.69:0/3359343443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f482c10e8b0 0x7f482c19a930 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f482400ea80 tx=0x7f482400ef40 comp rx=0 tx=0).stop 2024-04-24T09:08:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.076+0000 7f4832600640 1 -- 172.21.15.69:0/3359343443 shutdown_connections 2024-04-24T09:08:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.076+0000 7f4832600640 1 --2- 172.21.15.69:0/3359343443 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f48100a3da0 0x7f48100a6260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.076+0000 7f4832600640 1 --2- 172.21.15.69:0/3359343443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f482c10e8b0 0x7f482c19a930 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.076+0000 7f4832600640 1 --2- 172.21.15.69:0/3359343443 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f482c074cf0 0x7f482c1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.076+0000 7f4832600640 1 --2- 172.21.15.69:0/3359343443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f482c0743d0 0x7f482c1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.076+0000 7f4832600640 1 -- 172.21.15.69:0/3359343443 >> 172.21.15.69:0/3359343443 conn(0x7f482c079490 msgr2=0x7f482c100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.076+0000 7f4832600640 1 -- 172.21.15.69:0/3359343443 shutdown_connections 2024-04-24T09:08:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.076+0000 7f4832600640 1 -- 172.21.15.69:0/3359343443 wait complete. 2024-04-24T09:08:50.096 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:08:50.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.299+0000 7f0286800640 1 -- 172.21.15.69:0/18003530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f028010e9b0 msgr2=0x7f0280110da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.299+0000 7f0286800640 1 --2- 172.21.15.69:0/18003530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f028010e9b0 0x7f0280110da0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0278009900 tx=0x7f027802a410 comp rx=0 tx=0).stop 2024-04-24T09:08:50.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.299+0000 7f0286800640 1 -- 172.21.15.69:0/18003530 shutdown_connections 2024-04-24T09:08:50.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.299+0000 7f0286800640 1 --2- 172.21.15.69:0/18003530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f028010e9b0 0x7f0280110da0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.299+0000 7f0286800640 1 --2- 172.21.15.69:0/18003530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0280101720 0x7f028010e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.299+0000 7f0286800640 1 --2- 172.21.15.69:0/18003530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0280100e00 0x7f02801011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.299+0000 7f0286800640 1 -- 172.21.15.69:0/18003530 >> 172.21.15.69:0/18003530 conn(0x7f0280078000 msgr2=0x7f0280078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.299+0000 7f0286800640 1 -- 172.21.15.69:0/18003530 shutdown_connections 2024-04-24T09:08:50.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.299+0000 7f0286800640 1 -- 172.21.15.69:0/18003530 wait complete. 2024-04-24T09:08:50.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.300+0000 7f0286800640 1 Processor -- start 2024-04-24T09:08:50.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.300+0000 7f0286800640 1 -- start start 2024-04-24T09:08:50.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.301+0000 7f0286800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0280100e00 0x7f02801a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.301+0000 7f0286800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0280101720 0x7f02801a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.301+0000 7f0286800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f028010e9b0 0x7f028019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.301+0000 7f0286800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0280114470 con 0x7f0280101720 2024-04-24T09:08:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.301+0000 7f0286800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f02801142f0 con 0x7f0280100e00 2024-04-24T09:08:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.301+0000 7f0286800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f02801145f0 con 0x7f028010e9b0 2024-04-24T09:08:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.301+0000 7f027ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0280101720 0x7f02801a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.301+0000 7f027ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0280101720 0x7f02801a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33456/0 (socket says 172.21.15.69:33456) 2024-04-24T09:08:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.301+0000 7f027ea00640 1 -- 172.21.15.69:0/3286175529 learned_addr learned my addr 172.21.15.69:0/3286175529 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.301+0000 7f027f400640 1 --2- 172.21.15.69:0/3286175529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0280100e00 0x7f02801a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.301+0000 7f027fe00640 1 --2- 172.21.15.69:0/3286175529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f028010e9b0 0x7f028019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.302+0000 7f027ea00640 1 -- 172.21.15.69:0/3286175529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f028010e9b0 msgr2=0x7f028019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.302+0000 7f027ea00640 1 --2- 172.21.15.69:0/3286175529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f028010e9b0 0x7f028019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.302+0000 7f027ea00640 1 -- 172.21.15.69:0/3286175529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0280100e00 msgr2=0x7f02801a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.302+0000 7f027ea00640 1 --2- 172.21.15.69:0/3286175529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0280100e00 0x7f02801a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.302+0000 7f027ea00640 1 -- 172.21.15.69:0/3286175529 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028019afa0 con 0x7f0280101720 2024-04-24T09:08:50.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.302+0000 7f027f400640 1 --2- 172.21.15.69:0/3286175529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0280100e00 0x7f02801a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:50.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.302+0000 7f027ea00640 1 --2- 172.21.15.69:0/3286175529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0280101720 0x7f02801a0ba0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f026c00d770 tx=0x7f026c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.302+0000 7f0277e00640 1 -- 172.21.15.69:0/3286175529 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c017070 con 0x7f0280101720 2024-04-24T09:08:50.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.302+0000 7f0277e00640 1 -- 172.21.15.69:0/3286175529 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f026c0054d0 con 0x7f0280101720 2024-04-24T09:08:50.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.302+0000 7f0277e00640 1 -- 172.21.15.69:0/3286175529 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c004cf0 con 0x7f0280101720 2024-04-24T09:08:50.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.302+0000 7f0286800640 1 -- 172.21.15.69:0/3286175529 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028019b290 con 0x7f0280101720 2024-04-24T09:08:50.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.303+0000 7f0286800640 1 -- 172.21.15.69:0/3286175529 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02801a74b0 con 0x7f0280101720 2024-04-24T09:08:50.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.306+0000 7f028961a740 1 Processor -- start 2024-04-24T09:08:50.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.307+0000 7f028961a740 1 -- start start 2024-04-24T09:08:50.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.307+0000 7f028961a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804ed8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.307+0000 7f028961a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ed9270 0x563804ee3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.307+0000 7f028961a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ee3940 0x563804ee5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.307+0000 7f028961a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563804e3e1d0 con 0x563804ed9270 2024-04-24T09:08:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.307+0000 7f028961a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563804de4450 con 0x563804ee3940 2024-04-24T09:08:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.307+0000 7f028961a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563804e3a190 con 0x563804ed8950 2024-04-24T09:08:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.307+0000 7f027f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ed9270 0x563804ee3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.307+0000 7f0277e00640 1 -- 172.21.15.69:0/3286175529 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f026c004e90 con 0x7f0280101720 2024-04-24T09:08:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.307+0000 7f027f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ed9270 0x563804ee3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33470/0 (socket says 172.21.15.69:33470) 2024-04-24T09:08:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.307+0000 7f027f400640 1 -- 172.21.15.69:0/1853620414 learned_addr learned my addr 172.21.15.69:0/1853620414 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.308+0000 7f027fe00640 1 --2- 172.21.15.69:0/1853620414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804ed8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.308+0000 7f027f400640 1 -- 172.21.15.69:0/1853620414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 msgr2=0x563804ed8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.308+0000 7f027f400640 1 --2- 172.21.15.69:0/1853620414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804ed8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.308+0000 7f027f400640 1 -- 172.21.15.69:0/1853620414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ee3940 msgr2=0x563804ee5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.308+0000 7f027f400640 1 --2- 172.21.15.69:0/1853620414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ee3940 0x563804ee5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.308+0000 7f027f400640 1 -- 172.21.15.69:0/1853620414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563804ea1310 con 0x563804ed9270 2024-04-24T09:08:50.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.308+0000 7f027f400640 1 --2- 172.21.15.69:0/1853620414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ed9270 0x563804ee3400 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0270009f20 tx=0x7f027000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff9be608aa02146 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.309+0000 7f0274c00640 1 -- 172.21.15.69:0/1853620414 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270003b70 con 0x563804ed9270 2024-04-24T09:08:50.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.309+0000 7f0274c00640 1 -- 172.21.15.69:0/1853620414 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0270003d10 con 0x563804ed9270 2024-04-24T09:08:50.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.309+0000 7f0274c00640 1 -- 172.21.15.69:0/1853620414 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700118c0 con 0x563804ed9270 2024-04-24T09:08:50.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.309+0000 7f0277e00640 1 --2- 172.21.15.69:0/3286175529 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f02640a3da0 0x7f02640a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.309+0000 7f028961a740 1 -- 172.21.15.69:0/1853620414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ed9270 msgr2=0x563804ee3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.309+0000 7f0277e00640 1 -- 172.21.15.69:0/3286175529 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f026c0d0200 con 0x7f0280101720 2024-04-24T09:08:50.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.309+0000 7f028961a740 1 --2- 172.21.15.69:0/1853620414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ed9270 0x563804ee3400 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0270009f20 tx=0x7f027000f7f0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.310+0000 7f027fe00640 1 --2- 172.21.15.69:0/3286175529 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f02640a3da0 0x7f02640a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.310+0000 7f028961a740 1 -- 172.21.15.69:0/1853620414 shutdown_connections 2024-04-24T09:08:50.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.310+0000 7f028961a740 1 --2- 172.21.15.69:0/1853620414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ee3940 0x563804ee5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.310+0000 7f028961a740 1 --2- 172.21.15.69:0/1853620414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ed9270 0x563804ee3400 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.310+0000 7f028961a740 1 --2- 172.21.15.69:0/1853620414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804ed8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.310+0000 7f028961a740 1 -- 172.21.15.69:0/1853620414 >> 172.21.15.69:0/1853620414 conn(0x563804ed4c70 msgr2=0x563804ed5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.310+0000 7f028961a740 1 -- 172.21.15.69:0/1853620414 shutdown_connections 2024-04-24T09:08:50.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.310+0000 7f028961a740 1 -- 172.21.15.69:0/1853620414 wait complete. 2024-04-24T09:08:50.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.310+0000 7f028961a740 1 -- start start 2024-04-24T09:08:50.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.311+0000 7f027fe00640 1 --2- 172.21.15.69:0/3286175529 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f02640a3da0 0x7f02640a6260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0278031440 tx=0x7f02780058b0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.311+0000 7f028961a740 1 Processor -- start 2024-04-24T09:08:50.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.311+0000 7f028961a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804ee3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.311+0000 7f028961a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ed9270 0x563804ee0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.311+0000 7f028961a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804ee0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.311+0000 7f028961a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563804ee67d0 con 0x563804ee3940 2024-04-24T09:08:50.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.311+0000 7f028961a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563804df7810 con 0x563804ed9270 2024-04-24T09:08:50.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.311+0000 7f028961a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563804ee6950 con 0x563804ed8950 2024-04-24T09:08:50.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.311+0000 7f028961a740 10 client.0 _ll_register_callbacks cb 0x563804dc5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:50.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.312+0000 7f027fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804ee0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.312+0000 7f027fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804ee0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33480/0 (socket says 172.21.15.69:33480) 2024-04-24T09:08:50.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.312+0000 7f027fe00640 1 -- 172.21.15.69:0/3466315524 learned_addr learned my addr 172.21.15.69:0/3466315524 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:50.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.312+0000 7f027f400640 1 --2- 172.21.15.69:0/3466315524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804ee3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.312+0000 7f027ea00640 1 --2- 172.21.15.69:0/3466315524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ed9270 0x563804ee0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.312+0000 7f027fe00640 1 -- 172.21.15.69:0/3466315524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 msgr2=0x563804ee3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.312+0000 7f027fe00640 1 --2- 172.21.15.69:0/3466315524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804ee3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.312+0000 7f027fe00640 1 -- 172.21.15.69:0/3466315524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ed9270 msgr2=0x563804ee0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.312+0000 7f027fe00640 1 --2- 172.21.15.69:0/3466315524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ed9270 0x563804ee0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.312+0000 7f027fe00640 1 -- 172.21.15.69:0/3466315524 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563804e083f0 con 0x563804ee3940 2024-04-24T09:08:50.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.313+0000 7f027fe00640 1 --2- 172.21.15.69:0/3466315524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804ee0c50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0278033000 tx=0x7f0278009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ca470f44d6326e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.313+0000 7f0269600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:50.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.313+0000 7f0269600640 1 -- 172.21.15.69:0/3466315524 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027804b070 con 0x563804ee3940 2024-04-24T09:08:50.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.313+0000 7f0269600640 1 -- 172.21.15.69:0/3466315524 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f0278005460 con 0x563804ee3940 2024-04-24T09:08:50.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.313+0000 7f028961a740 1 -- 172.21.15.69:0/3466315524 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0270003d10 con 0x563804ee3940 2024-04-24T09:08:50.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.313+0000 7f0269600640 1 -- 172.21.15.69:0/3466315524 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0278057050 con 0x563804ee3940 2024-04-24T09:08:50.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.313+0000 7f028961a740 10 client.16113 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:50.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.313+0000 7f028961a740 1 -- 172.21.15.69:0/3466315524 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563804b503f0 con 0x563804ee3940 2024-04-24T09:08:50.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.314+0000 7f0269600640 1 -- 172.21.15.69:0/3466315524 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0270003d10 con 0x563804ee3940 2024-04-24T09:08:50.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.314+0000 7f0269600640 1 -- 172.21.15.69:0/3466315524 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f02780035f0 con 0x563804ee3940 2024-04-24T09:08:50.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.314+0000 7f028961a740 10 client.16113 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:50.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.314+0000 7f028961a740 10 client.16113 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:08:50.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.314+0000 7f028961a740 20 client.16113 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:50.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.314+0000 7f028961a740 1 --2- 172.21.15.69:0/3466315524 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x563804ef3a70 0x563804ef5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.314+0000 7f028961a740 4 client.16113 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:50.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.314+0000 7f028961a740 1 -- 172.21.15.69:0/3466315524 --> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563804e4bbf0 con 0x563804ef3a70 2024-04-24T09:08:50.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.315+0000 7f027f400640 1 --2- 172.21.15.69:0/3466315524 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x563804ef3a70 0x563804ef5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.316+0000 7f0269600640 10 client.16113 ms_handle_connect on v2:172.21.15.81:6834/3260829578 2024-04-24T09:08:50.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.316+0000 7f027f400640 1 --2- 172.21.15.69:0/3466315524 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x563804ef3a70 0x563804ef5ec0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ac5a0c1fe6573d8 server_cookie=95b4c174c5412fed in_seq=0 out_seq=0 2024-04-24T09:08:50.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.317+0000 7f0269600640 1 -- 172.21.15.69:0/3466315524 <== mds.2 v2:172.21.15.81:6834/3260829578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563804e4bbf0 con 0x563804ef3a70 2024-04-24T09:08:50.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.317+0000 7f0269600640 10 client.16113 handle_command_reply: tid=0 2024-04-24T09:08:50.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.318+0000 7f028961a740 1 client.16113 shutdown 2024-04-24T09:08:50.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.318+0000 7f028961a740 1 -- 172.21.15.69:0/3466315524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 msgr2=0x563804ee0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.318+0000 7f028961a740 1 --2- 172.21.15.69:0/3466315524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804ee0c50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0278033000 tx=0x7f0278009260 comp rx=0 tx=0).stop 2024-04-24T09:08:50.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.318+0000 7f028961a740 1 -- 172.21.15.69:0/3466315524 shutdown_connections 2024-04-24T09:08:50.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.318+0000 7f028961a740 1 --2- 172.21.15.69:0/3466315524 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x563804ef3a70 0x563804ef5ec0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.318+0000 7f028961a740 1 --2- 172.21.15.69:0/3466315524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804ee0c50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.318+0000 7f028961a740 1 --2- 172.21.15.69:0/3466315524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ed9270 0x563804ee0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.318+0000 7f028961a740 1 --2- 172.21.15.69:0/3466315524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804ee3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.318+0000 7f028961a740 1 -- 172.21.15.69:0/3466315524 >> 172.21.15.69:0/3466315524 conn(0x563804ed4c70 msgr2=0x563804ed3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.318+0000 7f0269600640 0 client.16113 ms_handle_reset on v2:172.21.15.81:6834/3260829578 2024-04-24T09:08:50.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.318+0000 7f028961a740 1 -- 172.21.15.69:0/3466315524 shutdown_connections 2024-04-24T09:08:50.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.318+0000 7f028961a740 1 -- 172.21.15.69:0/3466315524 wait complete. 2024-04-24T09:08:50.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.318+0000 7f028961a740 20 client.16113 trim_cache size 0 max 16384 2024-04-24T09:08:50.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.336+0000 7f028961a740 1 Processor -- start 2024-04-24T09:08:50.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.336+0000 7f028961a740 1 -- start start 2024-04-24T09:08:50.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.337+0000 7f028961a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804eef6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.337+0000 7f028961a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ed9270 0x563804ee2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.337+0000 7f028961a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804eda7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.337+0000 7f028961a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f02801b3d70 con 0x563804ee3940 2024-04-24T09:08:50.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.337+0000 7f028961a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f02801b1640 con 0x563804ed9270 2024-04-24T09:08:50.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.337+0000 7f028961a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f02801af490 con 0x563804ed8950 2024-04-24T09:08:50.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.337+0000 7f027fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804eda7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.337+0000 7f027fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804eda7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33490/0 (socket says 172.21.15.69:33490) 2024-04-24T09:08:50.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.337+0000 7f027fe00640 1 -- 172.21.15.69:0/3131584914 learned_addr learned my addr 172.21.15.69:0/3131584914 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:50.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.337+0000 7f027ea00640 1 --2- 172.21.15.69:0/3131584914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ed9270 0x563804ee2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.338+0000 7f027f400640 1 --2- 172.21.15.69:0/3131584914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804eef6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.338+0000 7f027fe00640 1 -- 172.21.15.69:0/3131584914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 msgr2=0x563804eef6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.338+0000 7f027fe00640 1 --2- 172.21.15.69:0/3131584914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804eef6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.338+0000 7f027fe00640 1 -- 172.21.15.69:0/3131584914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ed9270 msgr2=0x563804ee2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.338+0000 7f027fe00640 1 --2- 172.21.15.69:0/3131584914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ed9270 0x563804ee2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.338+0000 7f027fe00640 1 -- 172.21.15.69:0/3131584914 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563804c1a7d0 con 0x563804ee3940 2024-04-24T09:08:50.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.338+0000 7f027fe00640 1 --2- 172.21.15.69:0/3131584914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804eda7f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f02780319d0 tx=0x7f027802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5d54f5df762989 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.338+0000 7f026be00640 1 -- 172.21.15.69:0/3131584914 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027804b070 con 0x563804ee3940 2024-04-24T09:08:50.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.339+0000 7f026be00640 1 -- 172.21.15.69:0/3131584914 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0278060070 con 0x563804ee3940 2024-04-24T09:08:50.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.339+0000 7f026be00640 1 -- 172.21.15.69:0/3131584914 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278035050 con 0x563804ee3940 2024-04-24T09:08:50.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.339+0000 7f028961a740 1 -- 172.21.15.69:0/3131584914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 msgr2=0x563804eda7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.339+0000 7f028961a740 1 --2- 172.21.15.69:0/3131584914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804eda7f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f02780319d0 tx=0x7f027802a410 comp rx=0 tx=0).stop 2024-04-24T09:08:50.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.340+0000 7f028961a740 1 -- 172.21.15.69:0/3131584914 shutdown_connections 2024-04-24T09:08:50.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.340+0000 7f028961a740 1 --2- 172.21.15.69:0/3131584914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804eda7f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.340+0000 7f028961a740 1 --2- 172.21.15.69:0/3131584914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ed9270 0x563804ee2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.340+0000 7f028961a740 1 --2- 172.21.15.69:0/3131584914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804eef6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.340+0000 7f028961a740 1 -- 172.21.15.69:0/3131584914 >> 172.21.15.69:0/3131584914 conn(0x563804ed4c70 msgr2=0x563804eda0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.340+0000 7f028961a740 1 -- 172.21.15.69:0/3131584914 shutdown_connections 2024-04-24T09:08:50.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.340+0000 7f028961a740 1 -- 172.21.15.69:0/3131584914 wait complete. 2024-04-24T09:08:50.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.340+0000 7f028961a740 1 -- start start 2024-04-24T09:08:50.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.340+0000 7f028961a740 1 Processor -- start 2024-04-24T09:08:50.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.341+0000 7f028961a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804eebbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.341+0000 7f028961a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ed9270 0x563804eec110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.343 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:50 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3359343443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:08:50.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.341+0000 7f028961a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804ee4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.341+0000 7f028961a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f02801afb00 con 0x563804ee3940 2024-04-24T09:08:50.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.341+0000 7f028961a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f02801b0f70 con 0x563804ed9270 2024-04-24T09:08:50.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.341+0000 7f028961a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f02801b01d0 con 0x563804ed8950 2024-04-24T09:08:50.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.341+0000 7f028961a740 10 client.0 _ll_register_callbacks cb 0x7f0278009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:50.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.341+0000 7f027fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804ee4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.341+0000 7f027fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804ee4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33496/0 (socket says 172.21.15.69:33496) 2024-04-24T09:08:50.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.341+0000 7f027fe00640 1 -- 172.21.15.69:0/2889655955 learned_addr learned my addr 172.21.15.69:0/2889655955 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:50.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.342+0000 7f027f400640 1 --2- 172.21.15.69:0/2889655955 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804eebbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.342+0000 7f027ea00640 1 --2- 172.21.15.69:0/2889655955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ed9270 0x563804eec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.342+0000 7f027fe00640 1 -- 172.21.15.69:0/2889655955 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 msgr2=0x563804eebbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.342+0000 7f027fe00640 1 --2- 172.21.15.69:0/2889655955 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804eebbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.342+0000 7f027fe00640 1 -- 172.21.15.69:0/2889655955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ed9270 msgr2=0x563804eec110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.342+0000 7f027fe00640 1 --2- 172.21.15.69:0/2889655955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ed9270 0x563804eec110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.342+0000 7f027fe00640 1 -- 172.21.15.69:0/2889655955 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563804ee5170 con 0x563804ee3940 2024-04-24T09:08:50.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.342+0000 7f027f400640 1 --2- 172.21.15.69:0/2889655955 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804eebbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:50.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.342+0000 7f027fe00640 1 --2- 172.21.15.69:0/2889655955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804ee4c30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0278031930 tx=0x7f0278031960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521ff1a795e5ebc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.342+0000 7f0269600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:50.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.342+0000 7f0269600640 1 -- 172.21.15.69:0/2889655955 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278035040 con 0x563804ee3940 2024-04-24T09:08:50.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.343+0000 7f0269600640 1 -- 172.21.15.69:0/2889655955 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f0278005260 con 0x563804ee3940 2024-04-24T09:08:50.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.343+0000 7f028961a740 1 -- 172.21.15.69:0/2889655955 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0278060070 con 0x563804ee3940 2024-04-24T09:08:50.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.343+0000 7f0269600640 1 -- 172.21.15.69:0/2889655955 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f027804b040 con 0x563804ee3940 2024-04-24T09:08:50.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.343+0000 7f028961a740 10 client.16119 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:50.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.343+0000 7f028961a740 1 -- 172.21.15.69:0/2889655955 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563804f5a790 con 0x563804ee3940 2024-04-24T09:08:50.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.343+0000 7f0269600640 1 -- 172.21.15.69:0/2889655955 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0278060070 con 0x563804ee3940 2024-04-24T09:08:50.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.343+0000 7f0269600640 1 -- 172.21.15.69:0/2889655955 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f027806c030 con 0x563804ee3940 2024-04-24T09:08:50.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.343+0000 7f028961a740 10 client.16119 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:50.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.343+0000 7f028961a740 10 client.16119 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:08:50.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.343+0000 7f028961a740 20 client.16119 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:50.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.343+0000 7f028961a740 1 --2- 172.21.15.69:0/2889655955 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x563804f5ae50 0x563804f5d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.343+0000 7f028961a740 4 client.16119 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:50.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.343+0000 7f028961a740 1 -- 172.21.15.69:0/2889655955 --> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563804e45a60 con 0x563804f5ae50 2024-04-24T09:08:50.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.344+0000 7f027f400640 1 --2- 172.21.15.69:0/2889655955 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x563804f5ae50 0x563804f5d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.345+0000 7f027f400640 1 --2- 172.21.15.69:0/2889655955 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x563804f5ae50 0x563804f5d270 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36ae3bdd770061ea server_cookie=95ea0f479b7c667c in_seq=0 out_seq=0 2024-04-24T09:08:50.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.345+0000 7f0269600640 10 client.16119 ms_handle_connect on v2:172.21.15.81:6834/3260829578 2024-04-24T09:08:50.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.345+0000 7f0269600640 1 -- 172.21.15.69:0/2889655955 <== mds.2 v2:172.21.15.81:6834/3260829578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563804e45a60 con 0x563804f5ae50 2024-04-24T09:08:50.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.345+0000 7f0269600640 10 client.16119 handle_command_reply: tid=0 2024-04-24T09:08:50.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.345+0000 7f028961a740 1 client.16119 shutdown 2024-04-24T09:08:50.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.346+0000 7f028961a740 1 -- 172.21.15.69:0/2889655955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 msgr2=0x563804ee4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.346+0000 7f028961a740 1 --2- 172.21.15.69:0/2889655955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804ee4c30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0278031930 tx=0x7f0278031960 comp rx=0 tx=0).stop 2024-04-24T09:08:50.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.346+0000 7f028961a740 1 -- 172.21.15.69:0/2889655955 shutdown_connections 2024-04-24T09:08:50.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.346+0000 7f028961a740 1 --2- 172.21.15.69:0/2889655955 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x563804f5ae50 0x563804f5d270 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.346+0000 7f028961a740 1 --2- 172.21.15.69:0/2889655955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563804ee3940 0x563804ee4c30 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.346+0000 7f028961a740 1 --2- 172.21.15.69:0/2889655955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563804ed9270 0x563804eec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.346+0000 7f028961a740 1 --2- 172.21.15.69:0/2889655955 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563804ed8950 0x563804eebbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.346+0000 7f0269600640 0 client.16119 ms_handle_reset on v2:172.21.15.81:6834/3260829578 2024-04-24T09:08:50.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.346+0000 7f028961a740 1 -- 172.21.15.69:0/2889655955 >> 172.21.15.69:0/2889655955 conn(0x563804ed4c70 msgr2=0x563804eda930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.346+0000 7f028961a740 1 -- 172.21.15.69:0/2889655955 shutdown_connections 2024-04-24T09:08:50.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.346+0000 7f028961a740 1 -- 172.21.15.69:0/2889655955 wait complete. 2024-04-24T09:08:50.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.346+0000 7f028961a740 20 client.16119 trim_cache size 0 max 16384 2024-04-24T09:08:50.351 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-04-24T09:08:50.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.347+0000 7f0286800640 1 -- 172.21.15.69:0/3286175529 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f02640a3da0 msgr2=0x7f02640a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.347+0000 7f0286800640 1 --2- 172.21.15.69:0/3286175529 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f02640a3da0 0x7f02640a6260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0278031440 tx=0x7f02780058b0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.347+0000 7f0286800640 1 -- 172.21.15.69:0/3286175529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0280101720 msgr2=0x7f02801a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.347+0000 7f0286800640 1 --2- 172.21.15.69:0/3286175529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0280101720 0x7f02801a0ba0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f026c00d770 tx=0x7f026c00dc30 comp rx=0 tx=0).stop 2024-04-24T09:08:50.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.347+0000 7f0286800640 1 -- 172.21.15.69:0/3286175529 shutdown_connections 2024-04-24T09:08:50.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.347+0000 7f0286800640 1 --2- 172.21.15.69:0/3286175529 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f02640a3da0 0x7f02640a6260 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.347+0000 7f0286800640 1 --2- 172.21.15.69:0/3286175529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f028010e9b0 0x7f028019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.347+0000 7f0286800640 1 --2- 172.21.15.69:0/3286175529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0280101720 0x7f02801a0ba0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.347+0000 7f0286800640 1 --2- 172.21.15.69:0/3286175529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0280100e00 0x7f02801a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.347+0000 7f0286800640 1 -- 172.21.15.69:0/3286175529 >> 172.21.15.69:0/3286175529 conn(0x7f0280078000 msgr2=0x7f028010ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.347+0000 7f0286800640 1 -- 172.21.15.69:0/3286175529 shutdown_connections 2024-04-24T09:08:50.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.347+0000 7f0286800640 1 -- 172.21.15.69:0/3286175529 wait complete. 2024-04-24T09:08:50.364 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:08:50.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:50 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3359343443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:08:50.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:50 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3359343443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:08:50.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.588+0000 7f001f000640 1 -- 172.21.15.69:0/454553527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0018101610 msgr2=0x7f001810e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.588+0000 7f001f000640 1 --2- 172.21.15.69:0/454553527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0018101610 0x7f001810e2d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0004009900 tx=0x7f000402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.588+0000 7f001f000640 1 -- 172.21.15.69:0/454553527 shutdown_connections 2024-04-24T09:08:50.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.588+0000 7f001f000640 1 --2- 172.21.15.69:0/454553527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f001810e810 0x7f0018110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.588+0000 7f001f000640 1 --2- 172.21.15.69:0/454553527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0018101610 0x7f001810e2d0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.588+0000 7f001f000640 1 --2- 172.21.15.69:0/454553527 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0018100c60 0x7f0018101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.588+0000 7f001f000640 1 -- 172.21.15.69:0/454553527 >> 172.21.15.69:0/454553527 conn(0x7f00180fc880 msgr2=0x7f00180feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.588+0000 7f001f000640 1 -- 172.21.15.69:0/454553527 shutdown_connections 2024-04-24T09:08:50.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.588+0000 7f001f000640 1 -- 172.21.15.69:0/454553527 wait complete. 2024-04-24T09:08:50.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.590+0000 7f001f000640 1 Processor -- start 2024-04-24T09:08:50.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.590+0000 7f001f000640 1 -- start start 2024-04-24T09:08:50.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.590+0000 7f001f000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0018100c60 0x7f00181a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.590+0000 7f001f000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0018101610 0x7f00181a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.590+0000 7f001f000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f001810e810 0x7f001819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.590+0000 7f001f000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f00181143a0 con 0x7f0018101610 2024-04-24T09:08:50.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.590+0000 7f001f000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0018114220 con 0x7f001810e810 2024-04-24T09:08:50.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.590+0000 7f001f000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0018114520 con 0x7f0018100c60 2024-04-24T09:08:50.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.590+0000 7f001d200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0018101610 0x7f00181a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.591+0000 7f001d200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0018101610 0x7f00181a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33522/0 (socket says 172.21.15.69:33522) 2024-04-24T09:08:50.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.591+0000 7f001d200640 1 -- 172.21.15.69:0/3581295163 learned_addr learned my addr 172.21.15.69:0/3581295163 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:50.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.591+0000 7f001dc00640 1 --2- 172.21.15.69:0/3581295163 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0018100c60 0x7f00181a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.591+0000 7f001e600640 1 --2- 172.21.15.69:0/3581295163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f001810e810 0x7f001819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.591+0000 7f001d200640 1 -- 172.21.15.69:0/3581295163 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0018100c60 msgr2=0x7f00181a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.591+0000 7f001d200640 1 --2- 172.21.15.69:0/3581295163 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0018100c60 0x7f00181a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.591+0000 7f001d200640 1 -- 172.21.15.69:0/3581295163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f001810e810 msgr2=0x7f001819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.591+0000 7f001d200640 1 --2- 172.21.15.69:0/3581295163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f001810e810 0x7f001819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.591+0000 7f001d200640 1 -- 172.21.15.69:0/3581295163 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001819b000 con 0x7f0018101610 2024-04-24T09:08:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.592+0000 7f001d200640 1 --2- 172.21.15.69:0/3581295163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0018101610 0x7f00181a0bc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0004005ab0 tx=0x7f0004003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.592+0000 7f0016000640 1 -- 172.21.15.69:0/3581295163 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000403f070 con 0x7f0018101610 2024-04-24T09:08:50.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.592+0000 7f001f000640 1 -- 172.21.15.69:0/3581295163 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001819b290 con 0x7f0018101610 2024-04-24T09:08:50.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.592+0000 7f0016000640 1 -- 172.21.15.69:0/3581295163 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0004031d60 con 0x7f0018101610 2024-04-24T09:08:50.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.592+0000 7f0016000640 1 -- 172.21.15.69:0/3581295163 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004035030 con 0x7f0018101610 2024-04-24T09:08:50.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.592+0000 7f001f000640 1 -- 172.21.15.69:0/3581295163 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00181a7420 con 0x7f0018101610 2024-04-24T09:08:50.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.596+0000 7f0023409740 1 Processor -- start 2024-04-24T09:08:50.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.597+0000 7f0023409740 1 -- start start 2024-04-24T09:08:50.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.597+0000 7f0016000640 1 -- 172.21.15.69:0/3581295163 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0004042070 con 0x7f0018101610 2024-04-24T09:08:50.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.597+0000 7f0023409740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee858c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.597+0000 7f0023409740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee859240 0x5559ee86be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.597+0000 7f0023409740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.597+0000 7f0023409740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5559ee7be1d0 con 0x5559ee86c340 2024-04-24T09:08:50.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.597+0000 7f0023409740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5559ee764450 con 0x5559ee858890 2024-04-24T09:08:50.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.597+0000 7f0023409740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5559ee7ba190 con 0x5559ee859240 2024-04-24T09:08:50.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.597+0000 7f001d200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.598+0000 7f001d200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33524/0 (socket says 172.21.15.69:33524) 2024-04-24T09:08:50.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.598+0000 7f001e600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee858c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.598+0000 7f001d200640 1 -- 172.21.15.69:0/2767967048 learned_addr learned my addr 172.21.15.69:0/2767967048 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:50.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.598+0000 7f001dc00640 1 --2- 172.21.15.69:0/2767967048 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee859240 0x5559ee86be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.598+0000 7f001d200640 1 -- 172.21.15.69:0/2767967048 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee859240 msgr2=0x5559ee86be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.598+0000 7f001d200640 1 --2- 172.21.15.69:0/2767967048 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee859240 0x5559ee86be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.598+0000 7f001d200640 1 -- 172.21.15.69:0/2767967048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 msgr2=0x5559ee858c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.598+0000 7f001d200640 1 --2- 172.21.15.69:0/2767967048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee858c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.598+0000 7f001d200640 1 -- 172.21.15.69:0/2767967048 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ee821310 con 0x5559ee86c340 2024-04-24T09:08:50.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.599+0000 7f001d200640 1 --2- 172.21.15.69:0/2767967048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86e730 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f00040ee3f0 tx=0x7f00040f93b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0c38ce669ce64c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.599+0000 7f0016000640 1 --2- 172.21.15.69:0/3581295163 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f00000a3d20 0x7f00000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.599+0000 7f000aa00640 1 -- 172.21.15.69:0/2767967048 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00040b94e0 con 0x5559ee86c340 2024-04-24T09:08:50.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.599+0000 7f000aa00640 1 -- 172.21.15.69:0/2767967048 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00040fe030 con 0x5559ee86c340 2024-04-24T09:08:50.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.599+0000 7f000aa00640 1 -- 172.21.15.69:0/2767967048 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00040bc630 con 0x5559ee86c340 2024-04-24T09:08:50.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.599+0000 7f0016000640 1 -- 172.21.15.69:0/3581295163 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f00040f1070 con 0x7f0018101610 2024-04-24T09:08:50.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.599+0000 7f001e600640 1 --2- 172.21.15.69:0/3581295163 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f00000a3d20 0x7f00000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.600+0000 7f0023409740 1 -- 172.21.15.69:0/2767967048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 msgr2=0x5559ee86e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.600+0000 7f0023409740 1 --2- 172.21.15.69:0/2767967048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86e730 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f00040ee3f0 tx=0x7f00040f93b0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.600+0000 7f001e600640 1 --2- 172.21.15.69:0/3581295163 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f00000a3d20 0x7f00000a61e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f001806a270 tx=0x7f0010008570 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.600+0000 7f0023409740 1 -- 172.21.15.69:0/2767967048 shutdown_connections 2024-04-24T09:08:50.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.600+0000 7f0023409740 1 --2- 172.21.15.69:0/2767967048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86e730 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.600+0000 7f0023409740 1 --2- 172.21.15.69:0/2767967048 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee859240 0x5559ee86be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.600+0000 7f0023409740 1 --2- 172.21.15.69:0/2767967048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee858c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.600+0000 7f0023409740 1 -- 172.21.15.69:0/2767967048 >> 172.21.15.69:0/2767967048 conn(0x5559ee854e60 msgr2=0x5559ee855240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.601+0000 7f0023409740 1 -- 172.21.15.69:0/2767967048 shutdown_connections 2024-04-24T09:08:50.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.601+0000 7f0023409740 1 -- 172.21.15.69:0/2767967048 wait complete. 2024-04-24T09:08:50.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.601+0000 7f0023409740 1 -- start start 2024-04-24T09:08:50.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.601+0000 7f0023409740 1 Processor -- start 2024-04-24T09:08:50.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.602+0000 7f0023409740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee865b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.602+0000 7f0023409740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee859240 0x5559ee8660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.602+0000 7f0023409740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.602+0000 7f0023409740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5559ee85b750 con 0x5559ee86c340 2024-04-24T09:08:50.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.602+0000 7f0023409740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5559ee777810 con 0x5559ee858890 2024-04-24T09:08:50.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.602+0000 7f0023409740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5559ee85b8d0 con 0x5559ee859240 2024-04-24T09:08:50.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.602+0000 7f0023409740 10 client.0 _ll_register_callbacks cb 0x5559ee745140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:50.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.602+0000 7f001e600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.602+0000 7f001e600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33526/0 (socket says 172.21.15.69:33526) 2024-04-24T09:08:50.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.602+0000 7f001e600640 1 -- 172.21.15.69:0/3855152277 learned_addr learned my addr 172.21.15.69:0/3855152277 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:50.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.602+0000 7f001dc00640 1 --2- 172.21.15.69:0/3855152277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee865b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.602+0000 7f001d200640 1 --2- 172.21.15.69:0/3855152277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee859240 0x5559ee8660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.603+0000 7f001e600640 1 -- 172.21.15.69:0/3855152277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee859240 msgr2=0x5559ee8660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.603+0000 7f001e600640 1 --2- 172.21.15.69:0/3855152277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee859240 0x5559ee8660c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.603+0000 7f001e600640 1 -- 172.21.15.69:0/3855152277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 msgr2=0x5559ee865b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.603+0000 7f001e600640 1 --2- 172.21.15.69:0/3855152277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee865b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.603+0000 7f001e600640 1 -- 172.21.15.69:0/3855152277 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559ee7883f0 con 0x5559ee86c340 2024-04-24T09:08:50.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.603+0000 7f001e600640 1 --2- 172.21.15.69:0/3855152277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86b8a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f001001ce00 tx=0x7f0010004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7305050cd10b1cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.603+0000 7effff400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:50.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.603+0000 7effff400640 1 -- 172.21.15.69:0/3855152277 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010029060 con 0x5559ee86c340 2024-04-24T09:08:50.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.603+0000 7effff400640 1 -- 172.21.15.69:0/3855152277 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f001001a460 con 0x5559ee86c340 2024-04-24T09:08:50.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.603+0000 7f0023409740 1 -- 172.21.15.69:0/3855152277 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00040fe030 con 0x5559ee86c340 2024-04-24T09:08:50.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.604+0000 7effff400640 1 -- 172.21.15.69:0/3855152277 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0010030050 con 0x5559ee86c340 2024-04-24T09:08:50.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.604+0000 7f0023409740 10 client.16131 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:50.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.604+0000 7f0023409740 1 -- 172.21.15.69:0/3855152277 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5559ee4d03f0 con 0x5559ee86c340 2024-04-24T09:08:50.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.604+0000 7effff400640 1 -- 172.21.15.69:0/3855152277 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00040fe030 con 0x5559ee86c340 2024-04-24T09:08:50.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.604+0000 7effff400640 1 -- 172.21.15.69:0/3855152277 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0010018040 con 0x5559ee86c340 2024-04-24T09:08:50.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.604+0000 7f0023409740 10 client.16131 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:50.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.604+0000 7f0023409740 10 client.16131 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:08:50.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.604+0000 7f0023409740 20 client.16131 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:50.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.604+0000 7f0023409740 1 --2- 172.21.15.69:0/3855152277 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5559ee884b10 0x5559ee886f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.604+0000 7f0023409740 4 client.16131 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:50.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.604+0000 7f0023409740 1 -- 172.21.15.69:0/3855152277 --> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559ee7cbbf0 con 0x5559ee884b10 2024-04-24T09:08:50.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.605+0000 7f001dc00640 1 --2- 172.21.15.69:0/3855152277 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5559ee884b10 0x5559ee886f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.606+0000 7f001dc00640 1 --2- 172.21.15.69:0/3855152277 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5559ee884b10 0x5559ee886f40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=395b4624c947f3d4 server_cookie=d7d81019f4b5e6c in_seq=0 out_seq=0 2024-04-24T09:08:50.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.606+0000 7effff400640 10 client.16131 ms_handle_connect on v2:172.21.15.81:6836/1973819593 2024-04-24T09:08:50.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.608+0000 7effff400640 1 -- 172.21.15.69:0/3855152277 <== mds.1 v2:172.21.15.81:6836/1973819593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5559ee7cbbf0 con 0x5559ee884b10 2024-04-24T09:08:50.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.608+0000 7effff400640 10 client.16131 handle_command_reply: tid=0 2024-04-24T09:08:50.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.608+0000 7f0023409740 1 client.16131 shutdown 2024-04-24T09:08:50.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.608+0000 7f0023409740 1 -- 172.21.15.69:0/3855152277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 msgr2=0x5559ee86b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.608+0000 7f0023409740 1 --2- 172.21.15.69:0/3855152277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86b8a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f001001ce00 tx=0x7f0010004a40 comp rx=0 tx=0).stop 2024-04-24T09:08:50.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.609+0000 7f0023409740 1 -- 172.21.15.69:0/3855152277 shutdown_connections 2024-04-24T09:08:50.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.609+0000 7f0023409740 1 --2- 172.21.15.69:0/3855152277 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5559ee884b10 0x5559ee886f40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.609+0000 7f0023409740 1 --2- 172.21.15.69:0/3855152277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86b8a0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.609+0000 7f0023409740 1 --2- 172.21.15.69:0/3855152277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee859240 0x5559ee8660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.609+0000 7f0023409740 1 --2- 172.21.15.69:0/3855152277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee865b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.609+0000 7f0023409740 1 -- 172.21.15.69:0/3855152277 >> 172.21.15.69:0/3855152277 conn(0x5559ee854e60 msgr2=0x5559ee85ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.609+0000 7effff400640 0 client.16131 ms_handle_reset on v2:172.21.15.81:6836/1973819593 2024-04-24T09:08:50.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.609+0000 7f0023409740 1 -- 172.21.15.69:0/3855152277 shutdown_connections 2024-04-24T09:08:50.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.609+0000 7f0023409740 1 -- 172.21.15.69:0/3855152277 wait complete. 2024-04-24T09:08:50.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.609+0000 7f0023409740 20 client.16131 trim_cache size 0 max 16384 2024-04-24T09:08:50.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.624+0000 7f0023409740 1 Processor -- start 2024-04-24T09:08:50.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.624+0000 7f0023409740 1 -- start start 2024-04-24T09:08:50.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.624+0000 7f0023409740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee88c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.624+0000 7f0023409740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee859240 0x5559ee88cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.624+0000 7f0023409740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.624+0000 7f0023409740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f00181af7f0 con 0x5559ee86c340 2024-04-24T09:08:50.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.624+0000 7f0023409740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f00181ad0c0 con 0x5559ee858890 2024-04-24T09:08:50.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.624+0000 7f0023409740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f00181aaf10 con 0x5559ee859240 2024-04-24T09:08:50.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.625+0000 7f001e600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.625+0000 7f001e600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33536/0 (socket says 172.21.15.69:33536) 2024-04-24T09:08:50.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.625+0000 7f001e600640 1 -- 172.21.15.69:0/705788566 learned_addr learned my addr 172.21.15.69:0/705788566 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:50.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.625+0000 7f001d200640 1 --2- 172.21.15.69:0/705788566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee859240 0x5559ee88cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.625+0000 7f001dc00640 1 --2- 172.21.15.69:0/705788566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee88c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.625+0000 7f001e600640 1 -- 172.21.15.69:0/705788566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee859240 msgr2=0x5559ee88cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.625+0000 7f001e600640 1 --2- 172.21.15.69:0/705788566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee859240 0x5559ee88cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.625+0000 7f001e600640 1 -- 172.21.15.69:0/705788566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 msgr2=0x5559ee88c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.625+0000 7f001e600640 1 --2- 172.21.15.69:0/705788566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee88c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.625+0000 7f001e600640 1 -- 172.21.15.69:0/705788566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ee858fe0 con 0x5559ee86c340 2024-04-24T09:08:50.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.626+0000 7f001e600640 1 --2- 172.21.15.69:0/705788566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86d730 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0010016040 tx=0x7f001001cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419cad64a405d673 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.626+0000 7f000a000640 1 -- 172.21.15.69:0/705788566 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010024a90 con 0x5559ee86c340 2024-04-24T09:08:50.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.626+0000 7f000a000640 1 -- 172.21.15.69:0/705788566 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00100243e0 con 0x5559ee86c340 2024-04-24T09:08:50.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.626+0000 7f000a000640 1 -- 172.21.15.69:0/705788566 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001001a460 con 0x5559ee86c340 2024-04-24T09:08:50.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.627+0000 7f0023409740 1 -- 172.21.15.69:0/705788566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 msgr2=0x5559ee86d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.627+0000 7f0023409740 1 --2- 172.21.15.69:0/705788566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86d730 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0010016040 tx=0x7f001001cae0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.627+0000 7f0023409740 1 -- 172.21.15.69:0/705788566 shutdown_connections 2024-04-24T09:08:50.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.627+0000 7f0023409740 1 --2- 172.21.15.69:0/705788566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee86c340 0x5559ee86d730 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.627+0000 7f0023409740 1 --2- 172.21.15.69:0/705788566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee859240 0x5559ee88cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.627+0000 7f0023409740 1 --2- 172.21.15.69:0/705788566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee88c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.627+0000 7f0023409740 1 -- 172.21.15.69:0/705788566 >> 172.21.15.69:0/705788566 conn(0x5559ee854e60 msgr2=0x5559ee85ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.627+0000 7f0023409740 1 -- 172.21.15.69:0/705788566 shutdown_connections 2024-04-24T09:08:50.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.627+0000 7f0023409740 1 -- 172.21.15.69:0/705788566 wait complete. 2024-04-24T09:08:50.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.627+0000 7f0023409740 1 -- start start 2024-04-24T09:08:50.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.628+0000 7f0023409740 1 Processor -- start 2024-04-24T09:08:50.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.628+0000 7f0023409740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee897e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.628+0000 7f0023409740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee859240 0x5559ee898350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.628+0000 7f0023409740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee86c340 0x5559ee892dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.628+0000 7f0023409740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f00181ab580 con 0x5559ee859240 2024-04-24T09:08:50.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.628+0000 7f0023409740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f00181ac9f0 con 0x5559ee858890 2024-04-24T09:08:50.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.628+0000 7f0023409740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f00181abc50 con 0x5559ee86c340 2024-04-24T09:08:50.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.628+0000 7f0023409740 10 client.0 _ll_register_callbacks cb 0x7f0010008e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:50.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.628+0000 7f001d200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee859240 0x5559ee898350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.628+0000 7f001d200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee859240 0x5559ee898350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33548/0 (socket says 172.21.15.69:33548) 2024-04-24T09:08:50.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.628+0000 7f001d200640 1 -- 172.21.15.69:0/4275921885 learned_addr learned my addr 172.21.15.69:0/4275921885 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:50.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.629+0000 7f001dc00640 1 --2- 172.21.15.69:0/4275921885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee897e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.629+0000 7f001d200640 1 -- 172.21.15.69:0/4275921885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee86c340 msgr2=0x5559ee892dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.629+0000 7f001e600640 1 --2- 172.21.15.69:0/4275921885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee86c340 0x5559ee892dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.629+0000 7f001d200640 1 --2- 172.21.15.69:0/4275921885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee86c340 0x5559ee892dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.629+0000 7f001d200640 1 -- 172.21.15.69:0/4275921885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 msgr2=0x5559ee897e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.629+0000 7f001d200640 1 --2- 172.21.15.69:0/4275921885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee897e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.629+0000 7f001d200640 1 -- 172.21.15.69:0/4275921885 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559ee59a7d0 con 0x5559ee859240 2024-04-24T09:08:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.629+0000 7f001dc00640 1 --2- 172.21.15.69:0/4275921885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee897e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.629+0000 7f001e600640 1 --2- 172.21.15.69:0/4275921885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee86c340 0x5559ee892dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.629+0000 7f001d200640 1 --2- 172.21.15.69:0/4275921885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee859240 0x5559ee898350 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f00040ee530 tx=0x7f00040ca470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc2c42490adbaf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.629+0000 7effff400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.629+0000 7effff400640 1 -- 172.21.15.69:0/4275921885 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00040c9ed0 con 0x5559ee859240 2024-04-24T09:08:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.630+0000 7effff400640 1 -- 172.21.15.69:0/4275921885 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f00040d6110 con 0x5559ee859240 2024-04-24T09:08:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.630+0000 7f0023409740 1 -- 172.21.15.69:0/4275921885 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00100243e0 con 0x5559ee859240 2024-04-24T09:08:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.630+0000 7effff400640 1 -- 172.21.15.69:0/4275921885 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f00040ca0d0 con 0x5559ee859240 2024-04-24T09:08:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.630+0000 7f0023409740 10 client.16137 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.630+0000 7f0023409740 1 -- 172.21.15.69:0/4275921885 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5559ee8eb440 con 0x5559ee859240 2024-04-24T09:08:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.630+0000 7effff400640 1 -- 172.21.15.69:0/4275921885 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00100243e0 con 0x5559ee859240 2024-04-24T09:08:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.630+0000 7effff400640 1 -- 172.21.15.69:0/4275921885 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f000404c060 con 0x5559ee859240 2024-04-24T09:08:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.630+0000 7f0023409740 10 client.16137 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.630+0000 7f0023409740 10 client.16137 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:08:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.630+0000 7f0023409740 20 client.16137 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.630+0000 7f0023409740 1 --2- 172.21.15.69:0/4275921885 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5559ee8eb770 0x5559ee8edb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.630+0000 7f0023409740 4 client.16137 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.630+0000 7f0023409740 1 -- 172.21.15.69:0/4275921885 --> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559ee7c5a60 con 0x5559ee8eb770 2024-04-24T09:08:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.631+0000 7f001dc00640 1 --2- 172.21.15.69:0/4275921885 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5559ee8eb770 0x5559ee8edb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.632+0000 7f001dc00640 1 --2- 172.21.15.69:0/4275921885 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5559ee8eb770 0x5559ee8edb60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b5d08e7e475a58f server_cookie=84af2f14ffbdaeb7 in_seq=0 out_seq=0 2024-04-24T09:08:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.632+0000 7effff400640 10 client.16137 ms_handle_connect on v2:172.21.15.81:6836/1973819593 2024-04-24T09:08:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.632+0000 7effff400640 1 -- 172.21.15.69:0/4275921885 <== mds.1 v2:172.21.15.81:6836/1973819593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559ee7c5a60 con 0x5559ee8eb770 2024-04-24T09:08:50.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.632+0000 7effff400640 10 client.16137 handle_command_reply: tid=0 2024-04-24T09:08:50.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.632+0000 7f0023409740 1 client.16137 shutdown 2024-04-24T09:08:50.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.633+0000 7f0023409740 1 -- 172.21.15.69:0/4275921885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee859240 msgr2=0x5559ee898350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.633+0000 7f0023409740 1 --2- 172.21.15.69:0/4275921885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee859240 0x5559ee898350 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f00040ee530 tx=0x7f00040ca470 comp rx=0 tx=0).stop 2024-04-24T09:08:50.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.633+0000 7f0023409740 1 -- 172.21.15.69:0/4275921885 shutdown_connections 2024-04-24T09:08:50.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.633+0000 7f0023409740 1 --2- 172.21.15.69:0/4275921885 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5559ee8eb770 0x5559ee8edb60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.633+0000 7f0023409740 1 --2- 172.21.15.69:0/4275921885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5559ee86c340 0x5559ee892dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.633+0000 7f0023409740 1 --2- 172.21.15.69:0/4275921885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559ee859240 0x5559ee898350 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.633+0000 7effff400640 0 client.16137 ms_handle_reset on v2:172.21.15.81:6836/1973819593 2024-04-24T09:08:50.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.633+0000 7f0023409740 1 --2- 172.21.15.69:0/4275921885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559ee858890 0x5559ee897e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.633+0000 7f0023409740 1 -- 172.21.15.69:0/4275921885 >> 172.21.15.69:0/4275921885 conn(0x5559ee854e60 msgr2=0x5559ee85a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.633+0000 7f0023409740 1 -- 172.21.15.69:0/4275921885 shutdown_connections 2024-04-24T09:08:50.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.633+0000 7f0023409740 1 -- 172.21.15.69:0/4275921885 wait complete. 2024-04-24T09:08:50.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.633+0000 7f0023409740 20 client.16137 trim_cache size 0 max 16384 2024-04-24T09:08:50.636 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-04-24T09:08:50.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.634+0000 7f001f000640 1 -- 172.21.15.69:0/3581295163 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f00000a3d20 msgr2=0x7f00000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.634+0000 7f001f000640 1 --2- 172.21.15.69:0/3581295163 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f00000a3d20 0x7f00000a61e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f001806a270 tx=0x7f0010008570 comp rx=0 tx=0).stop 2024-04-24T09:08:50.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.634+0000 7f001f000640 1 -- 172.21.15.69:0/3581295163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0018101610 msgr2=0x7f00181a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.634+0000 7f001f000640 1 --2- 172.21.15.69:0/3581295163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0018101610 0x7f00181a0bc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0004005ab0 tx=0x7f0004003cd0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.634+0000 7f001f000640 1 -- 172.21.15.69:0/3581295163 shutdown_connections 2024-04-24T09:08:50.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.634+0000 7f001f000640 1 --2- 172.21.15.69:0/3581295163 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f00000a3d20 0x7f00000a61e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.634+0000 7f001f000640 1 --2- 172.21.15.69:0/3581295163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f001810e810 0x7f001819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.634+0000 7f001f000640 1 --2- 172.21.15.69:0/3581295163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0018101610 0x7f00181a0bc0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.634+0000 7f001f000640 1 --2- 172.21.15.69:0/3581295163 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0018100c60 0x7f00181a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.634+0000 7f001f000640 1 -- 172.21.15.69:0/3581295163 >> 172.21.15.69:0/3581295163 conn(0x7f00180fc880 msgr2=0x7f00180fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.634+0000 7f001f000640 1 -- 172.21.15.69:0/3581295163 shutdown_connections 2024-04-24T09:08:50.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.634+0000 7f001f000640 1 -- 172.21.15.69:0/3581295163 wait complete. 2024-04-24T09:08:50.654 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:08:50.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.882+0000 7fad7ae00640 1 -- 172.21.15.69:0/2142588976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad74104cd0 msgr2=0x7fad74108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.882+0000 7fad7ae00640 1 --2- 172.21.15.69:0/2142588976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad74104cd0 0x7fad74108d10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fad60009900 tx=0x7fad6002a400 comp rx=0 tx=0).stop 2024-04-24T09:08:50.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.882+0000 7fad7ae00640 1 -- 172.21.15.69:0/2142588976 shutdown_connections 2024-04-24T09:08:50.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.882+0000 7fad7ae00640 1 --2- 172.21.15.69:0/2142588976 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad741093f0 0x7fad74110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.882+0000 7fad7ae00640 1 --2- 172.21.15.69:0/2142588976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad74104cd0 0x7fad74108d10 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.882+0000 7fad7ae00640 1 --2- 172.21.15.69:0/2142588976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad74104320 0x7fad74104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.882+0000 7fad7ae00640 1 -- 172.21.15.69:0/2142588976 >> 172.21.15.69:0/2142588976 conn(0x7fad740ffb40 msgr2=0x7fad74101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.882+0000 7fad7ae00640 1 -- 172.21.15.69:0/2142588976 shutdown_connections 2024-04-24T09:08:50.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.882+0000 7fad7ae00640 1 -- 172.21.15.69:0/2142588976 wait complete. 2024-04-24T09:08:50.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.883+0000 7fad7ae00640 1 Processor -- start 2024-04-24T09:08:50.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.883+0000 7fad7ae00640 1 -- start start 2024-04-24T09:08:50.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.884+0000 7fad7ae00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad74104320 0x7fad741a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.884+0000 7fad7ae00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad741093f0 0x7fad741a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.884+0000 7fad7ae00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7419a6b0 0x7fad7419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.884+0000 7fad7ae00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fad74114310 con 0x7fad7419a6b0 2024-04-24T09:08:50.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.884+0000 7fad7ae00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fad74114190 con 0x7fad74104320 2024-04-24T09:08:50.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.884+0000 7fad7ae00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fad74114490 con 0x7fad741093f0 2024-04-24T09:08:50.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.884+0000 7fad7a400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7419a6b0 0x7fad7419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.884+0000 7fad7a400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7419a6b0 0x7fad7419ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33560/0 (socket says 172.21.15.69:33560) 2024-04-24T09:08:50.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.884+0000 7fad7a400640 1 -- 172.21.15.69:0/17983853 learned_addr learned my addr 172.21.15.69:0/17983853 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:50.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.885+0000 7fad79a00640 1 --2- 172.21.15.69:0/17983853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad74104320 0x7fad741a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.885+0000 7fad79000640 1 --2- 172.21.15.69:0/17983853 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad741093f0 0x7fad741a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.885+0000 7fad7a400640 1 -- 172.21.15.69:0/17983853 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad741093f0 msgr2=0x7fad741a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.885+0000 7fad7a400640 1 --2- 172.21.15.69:0/17983853 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad741093f0 0x7fad741a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.885+0000 7fad7a400640 1 -- 172.21.15.69:0/17983853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad74104320 msgr2=0x7fad741a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.885+0000 7fad7a400640 1 --2- 172.21.15.69:0/17983853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad74104320 0x7fad741a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.885+0000 7fad7a400640 1 -- 172.21.15.69:0/17983853 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7419b2b0 con 0x7fad7419a6b0 2024-04-24T09:08:50.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.885+0000 7fad7a400640 1 --2- 172.21.15.69:0/17983853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7419a6b0 0x7fad7419ab30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00cce0 tx=0x7fad6c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.886+0000 7fad72000640 1 -- 172.21.15.69:0/17983853 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c00f450 con 0x7fad7419a6b0 2024-04-24T09:08:50.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.886+0000 7fad72000640 1 -- 172.21.15.69:0/17983853 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad6c00faa0 con 0x7fad7419a6b0 2024-04-24T09:08:50.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.886+0000 7fad72000640 1 -- 172.21.15.69:0/17983853 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c018b70 con 0x7fad7419a6b0 2024-04-24T09:08:50.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.886+0000 7fad7ae00640 1 -- 172.21.15.69:0/17983853 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad741a7390 con 0x7fad7419a6b0 2024-04-24T09:08:50.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.886+0000 7fad7ae00640 1 -- 172.21.15.69:0/17983853 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad741a7740 con 0x7fad7419a6b0 2024-04-24T09:08:50.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.889+0000 7fad7f272740 1 Processor -- start 2024-04-24T09:08:50.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.890+0000 7fad72000640 1 -- 172.21.15.69:0/17983853 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fad6c018d10 con 0x7fad7419a6b0 2024-04-24T09:08:50.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.890+0000 7fad7f272740 1 -- start start 2024-04-24T09:08:50.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.890+0000 7fad7f272740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565293249b40 0x565293249f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.891+0000 7fad7f272740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56529324a460 0x5652932545f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.891+0000 7fad7f272740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565293254b30 0x565293256f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.891+0000 7fad7f272740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5652931af1d0 con 0x56529324a460 2024-04-24T09:08:50.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.891+0000 7fad7f272740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x565293155450 con 0x565293249b40 2024-04-24T09:08:50.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.891+0000 7fad7f272740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5652931ab190 con 0x565293254b30 2024-04-24T09:08:50.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.891+0000 7fad79a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56529324a460 0x5652932545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.891+0000 7fad79a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56529324a460 0x5652932545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33562/0 (socket says 172.21.15.69:33562) 2024-04-24T09:08:50.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.891+0000 7fad79a00640 1 -- 172.21.15.69:0/2639867020 learned_addr learned my addr 172.21.15.69:0/2639867020 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:50.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.891+0000 7fad79000640 1 --2- 172.21.15.69:0/2639867020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565293249b40 0x565293249f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.891+0000 7fad79000640 1 --2- 172.21.15.69:0/2639867020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565293254b30 0x565293256f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.891+0000 7fad79a00640 1 -- 172.21.15.69:0/2639867020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565293254b30 msgr2=0x565293256f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.892+0000 7fad79a00640 1 --2- 172.21.15.69:0/2639867020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565293254b30 0x565293256f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.892+0000 7fad79a00640 1 -- 172.21.15.69:0/2639867020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565293249b40 msgr2=0x565293249f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.892+0000 7fad79a00640 1 --2- 172.21.15.69:0/2639867020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565293249b40 0x565293249f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.892+0000 7fad79a00640 1 -- 172.21.15.69:0/2639867020 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565293212310 con 0x56529324a460 2024-04-24T09:08:50.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.892+0000 7fad79000640 1 --2- 172.21.15.69:0/2639867020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565293254b30 0x565293256f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:50.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.892+0000 7fad72000640 1 --2- 172.21.15.69:0/17983853 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fad5c0a3d20 0x7fad5c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.892+0000 7fad79a00640 1 --2- 172.21.15.69:0/2639867020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56529324a460 0x5652932545f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fad6800bcb0 tx=0x7fad6800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf8d04ac885b6bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.892+0000 7fad72000640 1 -- 172.21.15.69:0/17983853 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fad6c0d0fa0 con 0x7fad7419a6b0 2024-04-24T09:08:50.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.892+0000 7fad66a00640 1 -- 172.21.15.69:0/2639867020 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68015070 con 0x56529324a460 2024-04-24T09:08:50.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.892+0000 7fad79000640 1 --2- 172.21.15.69:0/17983853 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fad5c0a3d20 0x7fad5c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.892+0000 7fad66a00640 1 -- 172.21.15.69:0/2639867020 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad6801a910 con 0x56529324a460 2024-04-24T09:08:50.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.892+0000 7fad66a00640 1 -- 172.21.15.69:0/2639867020 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68011990 con 0x56529324a460 2024-04-24T09:08:50.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.893+0000 7fad7f272740 1 -- 172.21.15.69:0/2639867020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56529324a460 msgr2=0x5652932545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.893+0000 7fad7f272740 1 --2- 172.21.15.69:0/2639867020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56529324a460 0x5652932545f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fad6800bcb0 tx=0x7fad6800f730 comp rx=0 tx=0).stop 2024-04-24T09:08:50.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.893+0000 7fad79000640 1 --2- 172.21.15.69:0/17983853 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fad5c0a3d20 0x7fad5c0a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad60000c00 tx=0x7fad600058b0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.894+0000 7fad7f272740 1 -- 172.21.15.69:0/2639867020 shutdown_connections 2024-04-24T09:08:50.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.894+0000 7fad7f272740 1 --2- 172.21.15.69:0/2639867020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565293254b30 0x565293256f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.894+0000 7fad7f272740 1 --2- 172.21.15.69:0/2639867020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56529324a460 0x5652932545f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.894+0000 7fad7f272740 1 --2- 172.21.15.69:0/2639867020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565293249b40 0x565293249f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.894+0000 7fad7f272740 1 -- 172.21.15.69:0/2639867020 >> 172.21.15.69:0/2639867020 conn(0x565293245e60 msgr2=0x565293246240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.894+0000 7fad7f272740 1 -- 172.21.15.69:0/2639867020 shutdown_connections 2024-04-24T09:08:50.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.894+0000 7fad7f272740 1 -- 172.21.15.69:0/2639867020 wait complete. 2024-04-24T09:08:50.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.894+0000 7fad7f272740 1 -- start start 2024-04-24T09:08:50.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.894+0000 7fad7f272740 1 Processor -- start 2024-04-24T09:08:50.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.895+0000 7fad7f272740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565293249b40 0x565293251910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.895+0000 7fad7f272740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56529324a460 0x565293251e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.895+0000 7fad7f272740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293254b30 0x565293252390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.895+0000 7fad7f272740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5652932579c0 con 0x565293254b30 2024-04-24T09:08:50.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.895+0000 7fad7f272740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x565293168810 con 0x565293249b40 2024-04-24T09:08:50.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.895+0000 7fad7f272740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x565293257b40 con 0x56529324a460 2024-04-24T09:08:50.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.895+0000 7fad7f272740 10 client.0 _ll_register_callbacks cb 0x565293136140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:50.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.895+0000 7fad7a400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293254b30 0x565293252390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.896+0000 7fad7a400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293254b30 0x565293252390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33572/0 (socket says 172.21.15.69:33572) 2024-04-24T09:08:50.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.896+0000 7fad7a400640 1 -- 172.21.15.69:0/2382540924 learned_addr learned my addr 172.21.15.69:0/2382540924 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:50.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.896+0000 7fad79a00640 1 --2- 172.21.15.69:0/2382540924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565293249b40 0x565293251910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.896+0000 7fad79000640 1 --2- 172.21.15.69:0/2382540924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56529324a460 0x565293251e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.896+0000 7fad7a400640 1 -- 172.21.15.69:0/2382540924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56529324a460 msgr2=0x565293251e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.896+0000 7fad7a400640 1 --2- 172.21.15.69:0/2382540924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56529324a460 0x565293251e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.896+0000 7fad7a400640 1 -- 172.21.15.69:0/2382540924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565293249b40 msgr2=0x565293251910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.896+0000 7fad7a400640 1 --2- 172.21.15.69:0/2382540924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565293249b40 0x565293251910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.896+0000 7fad7a400640 1 -- 172.21.15.69:0/2382540924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652931793f0 con 0x565293254b30 2024-04-24T09:08:50.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.896+0000 7fad79000640 1 --2- 172.21.15.69:0/2382540924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56529324a460 0x565293251e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:50.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.896+0000 7fad7a400640 1 --2- 172.21.15.69:0/2382540924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293254b30 0x565293252390 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fad6c099060 tx=0x7fad6c09c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f218aed09fc872 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.896+0000 7fad5b400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:50.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.897+0000 7fad5b400640 1 -- 172.21.15.69:0/2382540924 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0b6060 con 0x565293254b30 2024-04-24T09:08:50.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.897+0000 7fad5b400640 1 -- 172.21.15.69:0/2382540924 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fad6c09ce60 con 0x565293254b30 2024-04-24T09:08:50.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.897+0000 7fad7f272740 1 -- 172.21.15.69:0/2382540924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad6801a910 con 0x565293254b30 2024-04-24T09:08:50.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.897+0000 7fad5b400640 1 -- 172.21.15.69:0/2382540924 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fad6c0a0fb0 con 0x565293254b30 2024-04-24T09:08:50.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.897+0000 7fad7f272740 10 client.16149 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:50.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.897+0000 7fad7f272740 1 -- 172.21.15.69:0/2382540924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x565292ec13f0 con 0x565293254b30 2024-04-24T09:08:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.897+0000 7fad5b400640 1 -- 172.21.15.69:0/2382540924 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad6801a910 con 0x565293254b30 2024-04-24T09:08:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.897+0000 7fad5b400640 1 -- 172.21.15.69:0/2382540924 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fad6c0a53e0 con 0x565293254b30 2024-04-24T09:08:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.898+0000 7fad7f272740 10 client.16149 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.898+0000 7fad7f272740 10 client.16149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.898+0000 7fad7f272740 20 client.16149 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.898+0000 7fad7f272740 1 --2- 172.21.15.69:0/2382540924 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x565293264d70 0x565293267180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.898+0000 7fad7f272740 4 client.16149 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.898+0000 7fad7f272740 1 -- 172.21.15.69:0/2382540924 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652931bcbf0 con 0x565293264d70 2024-04-24T09:08:50.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.898+0000 7fad79a00640 1 --2- 172.21.15.69:0/2382540924 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x565293264d70 0x565293267180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.900+0000 7fad79a00640 1 --2- 172.21.15.69:0/2382540924 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x565293264d70 0x565293267180 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc4ccaf43a109684 server_cookie=d0f7e4534182217a in_seq=0 out_seq=0 2024-04-24T09:08:50.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.900+0000 7fad5b400640 10 client.16149 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:50.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.902+0000 7fad5b400640 1 -- 172.21.15.69:0/2382540924 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5652931bcbf0 con 0x565293264d70 2024-04-24T09:08:50.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.902+0000 7fad5b400640 10 client.16149 handle_command_reply: tid=0 2024-04-24T09:08:50.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.902+0000 7fad7f272740 1 client.16149 shutdown 2024-04-24T09:08:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.902+0000 7fad7f272740 1 -- 172.21.15.69:0/2382540924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293254b30 msgr2=0x565293252390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.902+0000 7fad7f272740 1 --2- 172.21.15.69:0/2382540924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293254b30 0x565293252390 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fad6c099060 tx=0x7fad6c09c620 comp rx=0 tx=0).stop 2024-04-24T09:08:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.902+0000 7fad7f272740 1 -- 172.21.15.69:0/2382540924 shutdown_connections 2024-04-24T09:08:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.902+0000 7fad7f272740 1 --2- 172.21.15.69:0/2382540924 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x565293264d70 0x565293267180 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.902+0000 7fad7f272740 1 --2- 172.21.15.69:0/2382540924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293254b30 0x565293252390 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.902+0000 7fad7f272740 1 --2- 172.21.15.69:0/2382540924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56529324a460 0x565293251e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.902+0000 7fad7f272740 1 --2- 172.21.15.69:0/2382540924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565293249b40 0x565293251910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.902+0000 7fad7f272740 1 -- 172.21.15.69:0/2382540924 >> 172.21.15.69:0/2382540924 conn(0x565293245e60 msgr2=0x56529324bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.902+0000 7fad5b400640 0 client.16149 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.903+0000 7fad7f272740 1 -- 172.21.15.69:0/2382540924 shutdown_connections 2024-04-24T09:08:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.903+0000 7fad7f272740 1 -- 172.21.15.69:0/2382540924 wait complete. 2024-04-24T09:08:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.903+0000 7fad7f272740 20 client.16149 trim_cache size 0 max 16384 2024-04-24T09:08:50.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.923+0000 7fad7f272740 1 Processor -- start 2024-04-24T09:08:50.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.923+0000 7fad7f272740 1 -- start start 2024-04-24T09:08:50.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.923+0000 7fad7f272740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565293249b40 0x565293260880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.923+0000 7fad7f272740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56529324a460 0x56529326ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.923+0000 7fad7f272740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293254b30 0x56529326cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.923+0000 7fad7f272740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fad741b3a60 con 0x565293254b30 2024-04-24T09:08:50.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.923+0000 7fad7f272740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fad741b1330 con 0x56529324a460 2024-04-24T09:08:50.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.923+0000 7fad7f272740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fad741af180 con 0x565293249b40 2024-04-24T09:08:50.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.924+0000 7fad7a400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293254b30 0x56529326cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.924+0000 7fad7a400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293254b30 0x56529326cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33582/0 (socket says 172.21.15.69:33582) 2024-04-24T09:08:50.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.924+0000 7fad7a400640 1 -- 172.21.15.69:0/2598913989 learned_addr learned my addr 172.21.15.69:0/2598913989 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:50.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.924+0000 7fad79a00640 1 --2- 172.21.15.69:0/2598913989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565293249b40 0x565293260880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.924+0000 7fad79000640 1 --2- 172.21.15.69:0/2598913989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56529324a460 0x56529326ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.924+0000 7fad7a400640 1 -- 172.21.15.69:0/2598913989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565293249b40 msgr2=0x565293260880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.924+0000 7fad7a400640 1 --2- 172.21.15.69:0/2598913989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565293249b40 0x565293260880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.924+0000 7fad7a400640 1 -- 172.21.15.69:0/2598913989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56529324a460 msgr2=0x56529326ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.924+0000 7fad7a400640 1 --2- 172.21.15.69:0/2598913989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56529324a460 0x56529326ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.924+0000 7fad7a400640 1 -- 172.21.15.69:0/2598913989 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565292f8b7d0 con 0x565293254b30 2024-04-24T09:08:50.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.925+0000 7fad7a400640 1 --2- 172.21.15.69:0/2598913989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293254b30 0x56529326cf90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fad6c09cb90 tx=0x7fad6c09c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65762abacc3f612 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.925+0000 7fad66000640 1 -- 172.21.15.69:0/2598913989 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0a53e0 con 0x565293254b30 2024-04-24T09:08:50.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.925+0000 7fad66000640 1 -- 172.21.15.69:0/2598913989 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad6c0bd0e0 con 0x565293254b30 2024-04-24T09:08:50.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.925+0000 7fad66000640 1 -- 172.21.15.69:0/2598913989 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c09d0c0 con 0x565293254b30 2024-04-24T09:08:50.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.926+0000 7fad7f272740 1 -- 172.21.15.69:0/2598913989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293254b30 msgr2=0x56529326cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.926+0000 7fad7f272740 1 --2- 172.21.15.69:0/2598913989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293254b30 0x56529326cf90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fad6c09cb90 tx=0x7fad6c09c690 comp rx=0 tx=0).stop 2024-04-24T09:08:50.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.926+0000 7fad7f272740 1 -- 172.21.15.69:0/2598913989 shutdown_connections 2024-04-24T09:08:50.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.926+0000 7fad7f272740 1 --2- 172.21.15.69:0/2598913989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293254b30 0x56529326cf90 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.926+0000 7fad7f272740 1 --2- 172.21.15.69:0/2598913989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56529324a460 0x56529326ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.926+0000 7fad7f272740 1 --2- 172.21.15.69:0/2598913989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x565293249b40 0x565293260880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.926+0000 7fad7f272740 1 -- 172.21.15.69:0/2598913989 >> 172.21.15.69:0/2598913989 conn(0x565293245e60 msgr2=0x565293244f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.926+0000 7fad7f272740 1 -- 172.21.15.69:0/2598913989 shutdown_connections 2024-04-24T09:08:50.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.926+0000 7fad7f272740 1 -- 172.21.15.69:0/2598913989 wait complete. 2024-04-24T09:08:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.927+0000 7fad7f272740 1 -- start start 2024-04-24T09:08:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.927+0000 7fad7f272740 1 Processor -- start 2024-04-24T09:08:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.927+0000 7fad7f272740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293249b40 0x56529324e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.927+0000 7fad7f272740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56529324a460 0x56529324ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.927+0000 7fad7f272740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565293254b30 0x56529325b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.927+0000 7fad7f272740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fad741af7f0 con 0x565293249b40 2024-04-24T09:08:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.927+0000 7fad7f272740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fad741b0c60 con 0x565293254b30 2024-04-24T09:08:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.927+0000 7fad7f272740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fad741afec0 con 0x56529324a460 2024-04-24T09:08:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.927+0000 7fad7f272740 10 client.0 _ll_register_callbacks cb 0x7fad6c0077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.928+0000 7fad79a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293249b40 0x56529324e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.928+0000 7fad79a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293249b40 0x56529324e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33586/0 (socket says 172.21.15.69:33586) 2024-04-24T09:08:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.928+0000 7fad79a00640 1 -- 172.21.15.69:0/1511539645 learned_addr learned my addr 172.21.15.69:0/1511539645 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.928+0000 7fad79000640 1 --2- 172.21.15.69:0/1511539645 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56529324a460 0x56529324ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.928+0000 7fad7a400640 1 --2- 172.21.15.69:0/1511539645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565293254b30 0x56529325b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.928+0000 7fad79a00640 1 -- 172.21.15.69:0/1511539645 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56529324a460 msgr2=0x56529324ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.928+0000 7fad79a00640 1 --2- 172.21.15.69:0/1511539645 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56529324a460 0x56529324ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.928+0000 7fad79a00640 1 -- 172.21.15.69:0/1511539645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565293254b30 msgr2=0x56529325b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.928+0000 7fad79a00640 1 --2- 172.21.15.69:0/1511539645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565293254b30 0x56529325b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.928+0000 7fad79a00640 1 -- 172.21.15.69:0/1511539645 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56529325b890 con 0x565293249b40 2024-04-24T09:08:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.929+0000 7fad79a00640 1 --2- 172.21.15.69:0/1511539645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293249b40 0x56529324e9c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fad6800bcb0 tx=0x7fad68005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702db2e9cfc530f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.929+0000 7fad5b400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.929+0000 7fad5b400640 1 -- 172.21.15.69:0/1511539645 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6802b070 con 0x565293249b40 2024-04-24T09:08:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.929+0000 7fad5b400640 1 -- 172.21.15.69:0/1511539645 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fad68004450 con 0x565293249b40 2024-04-24T09:08:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.929+0000 7fad7f272740 1 -- 172.21.15.69:0/1511539645 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad6c0bd0e0 con 0x565293249b40 2024-04-24T09:08:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.929+0000 7fad5b400640 1 -- 172.21.15.69:0/1511539645 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fad68036050 con 0x565293249b40 2024-04-24T09:08:50.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.929+0000 7fad7f272740 10 client.16155 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:50.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.929+0000 7fad7f272740 1 -- 172.21.15.69:0/1511539645 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5652932cb9a0 con 0x565293249b40 2024-04-24T09:08:50.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.930+0000 7fad5b400640 1 -- 172.21.15.69:0/1511539645 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad6c0bd0e0 con 0x565293249b40 2024-04-24T09:08:50.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.930+0000 7fad5b400640 1 -- 172.21.15.69:0/1511539645 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fad68004c20 con 0x565293249b40 2024-04-24T09:08:50.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.930+0000 7fad7f272740 10 client.16155 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:50.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.930+0000 7fad7f272740 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:50.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.930+0000 7fad7f272740 20 client.16155 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:50.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.930+0000 7fad7f272740 1 --2- 172.21.15.69:0/1511539645 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5652932cbd40 0x5652932ce130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:50.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.930+0000 7fad7f272740 4 client.16155 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:50.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.930+0000 7fad7f272740 1 -- 172.21.15.69:0/1511539645 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652931b6a60 con 0x5652932cbd40 2024-04-24T09:08:50.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.931+0000 7fad79000640 1 --2- 172.21.15.69:0/1511539645 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5652932cbd40 0x5652932ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:50.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.932+0000 7fad5b400640 10 client.16155 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:50.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.932+0000 7fad79000640 1 --2- 172.21.15.69:0/1511539645 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5652932cbd40 0x5652932ce130 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e6aad1dc1a5617 server_cookie=f8b54cd82abed155 in_seq=0 out_seq=0 2024-04-24T09:08:50.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.932+0000 7fad5b400640 1 -- 172.21.15.69:0/1511539645 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652931b6a60 con 0x5652932cbd40 2024-04-24T09:08:50.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.932+0000 7fad5b400640 10 client.16155 handle_command_reply: tid=0 2024-04-24T09:08:50.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.932+0000 7fad7f272740 1 client.16155 shutdown 2024-04-24T09:08:50.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.932+0000 7fad7f272740 1 -- 172.21.15.69:0/1511539645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293249b40 msgr2=0x56529324e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.932+0000 7fad7f272740 1 --2- 172.21.15.69:0/1511539645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293249b40 0x56529324e9c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fad6800bcb0 tx=0x7fad68005850 comp rx=0 tx=0).stop 2024-04-24T09:08:50.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.932+0000 7fad7f272740 1 -- 172.21.15.69:0/1511539645 shutdown_connections 2024-04-24T09:08:50.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.933+0000 7fad7f272740 1 --2- 172.21.15.69:0/1511539645 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5652932cbd40 0x5652932ce130 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.933+0000 7fad7f272740 1 --2- 172.21.15.69:0/1511539645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565293254b30 0x56529325b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.933+0000 7fad7f272740 1 --2- 172.21.15.69:0/1511539645 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56529324a460 0x56529324ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.933+0000 7fad7f272740 1 --2- 172.21.15.69:0/1511539645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x565293249b40 0x56529324e9c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.933+0000 7fad5b400640 0 client.16155 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:50.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.933+0000 7fad7f272740 1 -- 172.21.15.69:0/1511539645 >> 172.21.15.69:0/1511539645 conn(0x565293245e60 msgr2=0x5652932518d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.933+0000 7fad7f272740 1 -- 172.21.15.69:0/1511539645 shutdown_connections 2024-04-24T09:08:50.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.933+0000 7fad7f272740 1 -- 172.21.15.69:0/1511539645 wait complete. 2024-04-24T09:08:50.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.933+0000 7fad7f272740 20 client.16155 trim_cache size 0 max 16384 2024-04-24T09:08:50.934 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-04-24T09:08:50.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.934+0000 7fad7ae00640 1 -- 172.21.15.69:0/17983853 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fad5c0a3d20 msgr2=0x7fad5c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.934+0000 7fad7ae00640 1 --2- 172.21.15.69:0/17983853 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fad5c0a3d20 0x7fad5c0a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad60000c00 tx=0x7fad600058b0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.934+0000 7fad7ae00640 1 -- 172.21.15.69:0/17983853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7419a6b0 msgr2=0x7fad7419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:50.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.934+0000 7fad7ae00640 1 --2- 172.21.15.69:0/17983853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7419a6b0 0x7fad7419ab30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00cce0 tx=0x7fad6c00a620 comp rx=0 tx=0).stop 2024-04-24T09:08:50.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.934+0000 7fad7ae00640 1 -- 172.21.15.69:0/17983853 shutdown_connections 2024-04-24T09:08:50.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.934+0000 7fad7ae00640 1 --2- 172.21.15.69:0/17983853 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fad5c0a3d20 0x7fad5c0a61e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.934+0000 7fad7ae00640 1 --2- 172.21.15.69:0/17983853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7419a6b0 0x7fad7419ab30 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.934+0000 7fad7ae00640 1 --2- 172.21.15.69:0/17983853 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad741093f0 0x7fad741a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.934+0000 7fad7ae00640 1 --2- 172.21.15.69:0/17983853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad74104320 0x7fad741a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:50.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.934+0000 7fad7ae00640 1 -- 172.21.15.69:0/17983853 >> 172.21.15.69:0/17983853 conn(0x7fad740ffb40 msgr2=0x7fad74100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:50.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.934+0000 7fad7ae00640 1 -- 172.21.15.69:0/17983853 shutdown_connections 2024-04-24T09:08:50.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:50.934+0000 7fad7ae00640 1 -- 172.21.15.69:0/17983853 wait complete. 2024-04-24T09:08:51.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:51 smithi069 ceph-mon[28170]: pgmap v56: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 KiB/s wr, 27 op/s 2024-04-24T09:08:51.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:51 smithi130 ceph-mon[31553]: pgmap v56: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 KiB/s wr, 27 op/s 2024-04-24T09:08:51.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:51 smithi081 ceph-mon[30961]: pgmap v56: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 KiB/s wr, 27 op/s 2024-04-24T09:08:51.954 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:08:51.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:08:51.955 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:08:52.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.181+0000 7fb65b200640 1 -- 172.21.15.69:0/4157149494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb65c100e50 msgr2=0x7fb65c101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.181+0000 7fb65b200640 1 --2- 172.21.15.69:0/4157149494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb65c100e50 0x7fb65c101230 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009900 tx=0x7fb64c02a470 comp rx=0 tx=0).stop 2024-04-24T09:08:52.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.182+0000 7fb65b200640 1 -- 172.21.15.69:0/4157149494 shutdown_connections 2024-04-24T09:08:52.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.182+0000 7fb65b200640 1 --2- 172.21.15.69:0/4157149494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb65c106250 0x7fb65c1066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.182+0000 7fb65b200640 1 --2- 172.21.15.69:0/4157149494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb65c101770 0x7fb65c105c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.182+0000 7fb65b200640 1 --2- 172.21.15.69:0/4157149494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb65c100e50 0x7fb65c101230 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.182+0000 7fb65b200640 1 -- 172.21.15.69:0/4157149494 >> 172.21.15.69:0/4157149494 conn(0x7fb65c0fc880 msgr2=0x7fb65c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:52.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.182+0000 7fb65b200640 1 -- 172.21.15.69:0/4157149494 shutdown_connections 2024-04-24T09:08:52.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.182+0000 7fb65b200640 1 -- 172.21.15.69:0/4157149494 wait complete. 2024-04-24T09:08:52.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.183+0000 7fb65b200640 1 Processor -- start 2024-04-24T09:08:52.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.183+0000 7fb65b200640 1 -- start start 2024-04-24T09:08:52.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.184+0000 7fb65b200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb65c100e50 0x7fb65c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.184+0000 7fb65b200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb65c101770 0x7fb65c19f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.184+0000 7fb65b200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb65c106250 0x7fb65c1a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.184+0000 7fb65b200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c118720 con 0x7fb65c100e50 2024-04-24T09:08:52.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.184+0000 7fb65b200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1185a0 con 0x7fb65c101770 2024-04-24T09:08:52.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.184+0000 7fb65b200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1188a0 con 0x7fb65c106250 2024-04-24T09:08:52.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.184+0000 7fb659e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb65c100e50 0x7fb65c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.184+0000 7fb659e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb65c100e50 0x7fb65c19ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33604/0 (socket says 172.21.15.69:33604) 2024-04-24T09:08:52.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.184+0000 7fb659e00640 1 -- 172.21.15.69:0/1582919214 learned_addr learned my addr 172.21.15.69:0/1582919214 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:52.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.184+0000 7fb659400640 1 --2- 172.21.15.69:0/1582919214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb65c101770 0x7fb65c19f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.185+0000 7fb65a800640 1 --2- 172.21.15.69:0/1582919214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb65c106250 0x7fb65c1a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.185+0000 7fb659e00640 1 -- 172.21.15.69:0/1582919214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb65c106250 msgr2=0x7fb65c1a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.185+0000 7fb659e00640 1 --2- 172.21.15.69:0/1582919214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb65c106250 0x7fb65c1a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.185+0000 7fb659e00640 1 -- 172.21.15.69:0/1582919214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb65c101770 msgr2=0x7fb65c19f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.185+0000 7fb659e00640 1 --2- 172.21.15.69:0/1582919214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb65c101770 0x7fb65c19f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.185+0000 7fb659e00640 1 -- 172.21.15.69:0/1582919214 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65c1a3d30 con 0x7fb65c100e50 2024-04-24T09:08:52.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.185+0000 7fb659400640 1 --2- 172.21.15.69:0/1582919214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb65c101770 0x7fb65c19f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:52.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.185+0000 7fb659e00640 1 --2- 172.21.15.69:0/1582919214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb65c100e50 0x7fb65c19ed40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb64c005940 tx=0x7fb64c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:52.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.186+0000 7fb652a00640 1 -- 172.21.15.69:0/1582919214 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c03f030 con 0x7fb65c100e50 2024-04-24T09:08:52.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.186+0000 7fb652a00640 1 -- 172.21.15.69:0/1582919214 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb64c003aa0 con 0x7fb65c100e50 2024-04-24T09:08:52.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.186+0000 7fb652a00640 1 -- 172.21.15.69:0/1582919214 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c035070 con 0x7fb65c100e50 2024-04-24T09:08:52.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.186+0000 7fb65b200640 1 -- 172.21.15.69:0/1582919214 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65c1a3fc0 con 0x7fb65c100e50 2024-04-24T09:08:52.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.186+0000 7fb65b200640 1 -- 172.21.15.69:0/1582919214 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb65c1ab8b0 con 0x7fb65c100e50 2024-04-24T09:08:52.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.191+0000 7fb6639ec740 1 Processor -- start 2024-04-24T09:08:52.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.191+0000 7fb652a00640 1 -- 172.21.15.69:0/1582919214 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb64c042030 con 0x7fb65c100e50 2024-04-24T09:08:52.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.191+0000 7fb6639ec740 1 -- start start 2024-04-24T09:08:52.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.192+0000 7fb6639ec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a71b270 0x55852a71b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.192+0000 7fb6639ec740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 0x55852a725d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.192+0000 7fb6639ec740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a726260 0x55852a728650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.193+0000 7fb65a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a71b270 0x55852a71b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.193+0000 7fb6639ec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55852a680e70 con 0x55852a71b270 2024-04-24T09:08:52.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.193+0000 7fb6639ec740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55852a627380 con 0x55852a71bb90 2024-04-24T09:08:52.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.193+0000 7fb65a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a71b270 0x55852a71b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33610/0 (socket says 172.21.15.69:33610) 2024-04-24T09:08:52.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.193+0000 7fb6639ec740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55852a67ce30 con 0x55852a726260 2024-04-24T09:08:52.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.193+0000 7fb65a800640 1 -- 172.21.15.69:0/1985113495 learned_addr learned my addr 172.21.15.69:0/1985113495 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:52.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.193+0000 7fb659e00640 1 --2- 172.21.15.69:0/1985113495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 0x55852a725d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.193+0000 7fb659400640 1 --2- 172.21.15.69:0/1985113495 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a726260 0x55852a728650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.193+0000 7fb65a800640 1 -- 172.21.15.69:0/1985113495 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a726260 msgr2=0x55852a728650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:52.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.193+0000 7fb65a800640 1 --2- 172.21.15.69:0/1985113495 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a726260 0x55852a728650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.194+0000 7fb65a800640 1 -- 172.21.15.69:0/1985113495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 msgr2=0x55852a725d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:52.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.194+0000 7fb65a800640 1 --2- 172.21.15.69:0/1985113495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 0x55852a725d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.194+0000 7fb65a800640 1 -- 172.21.15.69:0/1985113495 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55852a45b7e0 con 0x55852a71b270 2024-04-24T09:08:52.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.194+0000 7fb652a00640 1 --2- 172.21.15.69:0/1582919214 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb6400a3d20 0x7fb6400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.194+0000 7fb65a800640 1 --2- 172.21.15.69:0/1985113495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a71b270 0x55852a71b650 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb654009f70 tx=0x7fb654010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f415e78bc61e50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:52.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.194+0000 7fb652a00640 1 -- 172.21.15.69:0/1582919214 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb64c0f10f0 con 0x7fb65c100e50 2024-04-24T09:08:52.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.195+0000 7fb659e00640 1 --2- 172.21.15.69:0/1582919214 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb6400a3d20 0x7fb6400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.195+0000 7fb647000640 1 -- 172.21.15.69:0/1985113495 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654016070 con 0x55852a71b270 2024-04-24T09:08:52.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.195+0000 7fb647000640 1 -- 172.21.15.69:0/1985113495 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb654004d20 con 0x55852a71b270 2024-04-24T09:08:52.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.195+0000 7fb647000640 1 -- 172.21.15.69:0/1985113495 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654012890 con 0x55852a71b270 2024-04-24T09:08:52.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.196+0000 7fb6639ec740 1 -- 172.21.15.69:0/1985113495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a71b270 msgr2=0x55852a71b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.196+0000 7fb659e00640 1 --2- 172.21.15.69:0/1582919214 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb6400a3d20 0x7fb6400a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0ee5b0 tx=0x7fb64c02a470 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:52.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.196+0000 7fb6639ec740 1 --2- 172.21.15.69:0/1985113495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a71b270 0x55852a71b650 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb654009f70 tx=0x7fb654010800 comp rx=0 tx=0).stop 2024-04-24T09:08:52.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.197+0000 7fb6639ec740 1 -- 172.21.15.69:0/1985113495 shutdown_connections 2024-04-24T09:08:52.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.197+0000 7fb6639ec740 1 --2- 172.21.15.69:0/1985113495 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a726260 0x55852a728650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.197+0000 7fb6639ec740 1 --2- 172.21.15.69:0/1985113495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 0x55852a725d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.197+0000 7fb6639ec740 1 --2- 172.21.15.69:0/1985113495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a71b270 0x55852a71b650 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.197+0000 7fb6639ec740 1 -- 172.21.15.69:0/1985113495 >> 172.21.15.69:0/1985113495 conn(0x55852a717590 msgr2=0x55852a717970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:52.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.197+0000 7fb6639ec740 1 -- 172.21.15.69:0/1985113495 shutdown_connections 2024-04-24T09:08:52.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.197+0000 7fb6639ec740 1 -- 172.21.15.69:0/1985113495 wait complete. 2024-04-24T09:08:52.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.197+0000 7fb6639ec740 1 -- start start 2024-04-24T09:08:52.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.198+0000 7fb6639ec740 1 Processor -- start 2024-04-24T09:08:52.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.198+0000 7fb6639ec740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a71b270 0x55852a723240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.198+0000 7fb6639ec740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 0x55852a723780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.198+0000 7fb6639ec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a726260 0x55852a723cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.198+0000 7fb6639ec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55852a729050 con 0x55852a726260 2024-04-24T09:08:52.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.198+0000 7fb6639ec740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55852a63a660 con 0x55852a71bb90 2024-04-24T09:08:52.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.198+0000 7fb6639ec740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55852a7291d0 con 0x55852a71b270 2024-04-24T09:08:52.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.198+0000 7fb6639ec740 10 client.0 _ll_register_callbacks cb 0x55852a628aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:52.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.199+0000 7fb65a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a726260 0x55852a723cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.199+0000 7fb659400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 0x55852a723780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.199+0000 7fb65a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a726260 0x55852a723cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33624/0 (socket says 172.21.15.69:33624) 2024-04-24T09:08:52.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.199+0000 7fb65a800640 1 -- 172.21.15.69:0/3541423128 learned_addr learned my addr 172.21.15.69:0/3541423128 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:52.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.199+0000 7fb659400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 0x55852a723780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.69:40576/0 (socket says 172.21.15.69:40576) 2024-04-24T09:08:52.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.200+0000 7fb659400640 1 -- 172.21.15.69:0/3541423128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a71b270 msgr2=0x55852a723240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:08:52.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.200+0000 7fb659400640 1 --2- 172.21.15.69:0/3541423128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a71b270 0x55852a723240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.200+0000 7fb659400640 1 -- 172.21.15.69:0/3541423128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a726260 msgr2=0x55852a723cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:52.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.200+0000 7fb659400640 1 --2- 172.21.15.69:0/3541423128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a726260 0x55852a723cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.200+0000 7fb659400640 1 -- 172.21.15.69:0/3541423128 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55852a4bbf70 con 0x55852a71bb90 2024-04-24T09:08:52.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.200+0000 7fb65a800640 1 --2- 172.21.15.69:0/3541423128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a726260 0x55852a723cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.200+0000 7fb659400640 1 --2- 172.21.15.69:0/3541423128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 0x55852a723780 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb65c110130 tx=0x7fb6480117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb26cc01739f90b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.200+0000 7fb63b400640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-04-24T09:08:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.200+0000 7fb63b400640 1 -- 172.21.15.69:0/3541423128 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648005220 con 0x55852a71bb90 2024-04-24T09:08:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.201+0000 7fb63b400640 1 -- 172.21.15.69:0/3541423128 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb6480053c0 con 0x55852a71bb90 2024-04-24T09:08:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.201+0000 7fb6639ec740 1 -- 172.21.15.69:0/3541423128 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb654004d20 con 0x55852a71bb90 2024-04-24T09:08:52.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.201+0000 7fb63b400640 1 -- 172.21.15.69:0/3541423128 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb6480055c0 con 0x55852a71bb90 2024-04-24T09:08:52.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.201+0000 7fb6639ec740 10 client.35477 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:52.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.201+0000 7fb6639ec740 1 -- 172.21.15.69:0/3541423128 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55852a353170 con 0x55852a71bb90 2024-04-24T09:08:52.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.202+0000 7fb63b400640 1 -- 172.21.15.69:0/3541423128 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb648004c50 con 0x55852a71bb90 2024-04-24T09:08:52.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.202+0000 7fb63b400640 1 -- 172.21.15.69:0/3541423128 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb648012c00 con 0x55852a71bb90 2024-04-24T09:08:52.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.202+0000 7fb6639ec740 10 client.35477 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:52.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.202+0000 7fb6639ec740 10 client.35477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:52.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.202+0000 7fb6639ec740 20 client.35477 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:52.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.203+0000 7fb6639ec740 1 --2- 172.21.15.69:0/3541423128 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55852a736690 0x55852a738ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.203+0000 7fb6639ec740 4 client.35477 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:52.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.203+0000 7fb6639ec740 1 -- 172.21.15.69:0/3541423128 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb654004d20 con 0x55852a736690 2024-04-24T09:08:52.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.203+0000 7fb65a800640 1 --2- 172.21.15.69:0/3541423128 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55852a736690 0x55852a738ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.204+0000 7fb65a800640 1 --2- 172.21.15.69:0/3541423128 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55852a736690 0x55852a738ac0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85c1b35a48c73ac2 server_cookie=9902a58595799195 in_seq=0 out_seq=0 2024-04-24T09:08:52.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.204+0000 7fb63b400640 10 client.35477 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:52.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.206+0000 7fb63b400640 1 -- 172.21.15.69:0/3541423128 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fb654004d20 con 0x55852a736690 2024-04-24T09:08:52.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.206+0000 7fb63b400640 10 client.35477 handle_command_reply: tid=0 2024-04-24T09:08:52.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.206+0000 7fb6639ec740 1 client.35477 shutdown 2024-04-24T09:08:52.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.207+0000 7fb6639ec740 1 -- 172.21.15.69:0/3541423128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 msgr2=0x55852a723780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.207+0000 7fb6639ec740 1 --2- 172.21.15.69:0/3541423128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 0x55852a723780 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb65c110130 tx=0x7fb6480117b0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.207+0000 7fb6639ec740 1 -- 172.21.15.69:0/3541423128 shutdown_connections 2024-04-24T09:08:52.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.207+0000 7fb6639ec740 1 --2- 172.21.15.69:0/3541423128 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55852a736690 0x55852a738ac0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.207+0000 7fb6639ec740 1 --2- 172.21.15.69:0/3541423128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a726260 0x55852a723cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.207+0000 7fb6639ec740 1 --2- 172.21.15.69:0/3541423128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 0x55852a723780 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.207+0000 7fb6639ec740 1 --2- 172.21.15.69:0/3541423128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a71b270 0x55852a723240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.207+0000 7fb6639ec740 1 -- 172.21.15.69:0/3541423128 >> 172.21.15.69:0/3541423128 conn(0x55852a717590 msgr2=0x55852a71d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:52.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.207+0000 7fb63b400640 0 client.35477 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:52.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.207+0000 7fb6639ec740 1 -- 172.21.15.69:0/3541423128 shutdown_connections 2024-04-24T09:08:52.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.207+0000 7fb6639ec740 1 -- 172.21.15.69:0/3541423128 wait complete. 2024-04-24T09:08:52.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.207+0000 7fb6639ec740 20 client.35477 trim_cache size 0 max 16384 2024-04-24T09:08:52.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.237+0000 7fb6639ec740 1 Processor -- start 2024-04-24T09:08:52.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.237+0000 7fb6639ec740 1 -- start start 2024-04-24T09:08:52.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.238+0000 7fb6639ec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a71b270 0x55852a722180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.238+0000 7fb6639ec740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 0x55852a731f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.238+0000 7fb6639ec740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a726260 0x55852a71d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.238+0000 7fb6639ec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1b7f90 con 0x55852a71b270 2024-04-24T09:08:52.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.238+0000 7fb6639ec740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1b5860 con 0x55852a71bb90 2024-04-24T09:08:52.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.238+0000 7fb6639ec740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1b36b0 con 0x55852a726260 2024-04-24T09:08:52.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.238+0000 7fb65a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a71b270 0x55852a722180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.238+0000 7fb65a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a71b270 0x55852a722180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33626/0 (socket says 172.21.15.69:33626) 2024-04-24T09:08:52.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.239+0000 7fb65a800640 1 -- 172.21.15.69:0/2898509914 learned_addr learned my addr 172.21.15.69:0/2898509914 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:52.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.239+0000 7fb659400640 1 --2- 172.21.15.69:0/2898509914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 0x55852a731f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.239+0000 7fb65a800640 1 --2- 172.21.15.69:0/2898509914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a726260 0x55852a71d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.239+0000 7fb65a800640 1 -- 172.21.15.69:0/2898509914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a726260 msgr2=0x55852a71d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:52.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.239+0000 7fb65a800640 1 --2- 172.21.15.69:0/2898509914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a726260 0x55852a71d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.239+0000 7fb65a800640 1 -- 172.21.15.69:0/2898509914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 msgr2=0x55852a731f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:52.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.239+0000 7fb65a800640 1 --2- 172.21.15.69:0/2898509914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 0x55852a731f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.239+0000 7fb65a800640 1 -- 172.21.15.69:0/2898509914 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55852a71b9c0 con 0x55852a71b270 2024-04-24T09:08:52.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.239+0000 7fb65a800640 1 --2- 172.21.15.69:0/2898509914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a71b270 0x55852a722180 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb654005ac0 tx=0x7fb6540059e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b056c6e860e13ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:52.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.240+0000 7fb646600640 1 -- 172.21.15.69:0/2898509914 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654016040 con 0x55852a71b270 2024-04-24T09:08:52.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.240+0000 7fb646600640 1 -- 172.21.15.69:0/2898509914 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb654004f60 con 0x55852a71b270 2024-04-24T09:08:52.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.240+0000 7fb646600640 1 -- 172.21.15.69:0/2898509914 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6540127c0 con 0x55852a71b270 2024-04-24T09:08:52.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.241+0000 7fb6639ec740 1 -- 172.21.15.69:0/2898509914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a71b270 msgr2=0x55852a722180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.241+0000 7fb6639ec740 1 --2- 172.21.15.69:0/2898509914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a71b270 0x55852a722180 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb654005ac0 tx=0x7fb6540059e0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.241+0000 7fb6639ec740 1 -- 172.21.15.69:0/2898509914 shutdown_connections 2024-04-24T09:08:52.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.241+0000 7fb6639ec740 1 --2- 172.21.15.69:0/2898509914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a726260 0x55852a71d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.241+0000 7fb6639ec740 1 --2- 172.21.15.69:0/2898509914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71bb90 0x55852a731f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.241+0000 7fb6639ec740 1 --2- 172.21.15.69:0/2898509914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a71b270 0x55852a722180 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.241+0000 7fb6639ec740 1 -- 172.21.15.69:0/2898509914 >> 172.21.15.69:0/2898509914 conn(0x55852a717590 msgr2=0x55852a723a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:52.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.241+0000 7fb6639ec740 1 -- 172.21.15.69:0/2898509914 shutdown_connections 2024-04-24T09:08:52.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.241+0000 7fb6639ec740 1 -- 172.21.15.69:0/2898509914 wait complete. 2024-04-24T09:08:52.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.242+0000 7fb6639ec740 1 -- start start 2024-04-24T09:08:52.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.242+0000 7fb6639ec740 1 Processor -- start 2024-04-24T09:08:52.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.243+0000 7fb6639ec740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71b270 0x55852a72a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.243+0000 7fb6639ec740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a71bb90 0x55852a72a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.243+0000 7fb6639ec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a726260 0x55852a79bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.243+0000 7fb6639ec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1b3d20 con 0x55852a726260 2024-04-24T09:08:52.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.243+0000 7fb6639ec740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1b5190 con 0x55852a71b270 2024-04-24T09:08:52.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.243+0000 7fb6639ec740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c1b43f0 con 0x55852a71bb90 2024-04-24T09:08:52.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.243+0000 7fb6639ec740 10 client.0 _ll_register_callbacks cb 0x7fb65c06a1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:52.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.243+0000 7fb65a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a726260 0x55852a79bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.243+0000 7fb65a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a726260 0x55852a79bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33638/0 (socket says 172.21.15.69:33638) 2024-04-24T09:08:52.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.243+0000 7fb65a800640 1 -- 172.21.15.69:0/2452017258 learned_addr learned my addr 172.21.15.69:0/2452017258 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:52.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.243+0000 7fb65a800640 1 --2- 172.21.15.69:0/2452017258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71b270 0x55852a72a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.243+0000 7fb659400640 1 --2- 172.21.15.69:0/2452017258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a71bb90 0x55852a72a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.244+0000 7fb65a800640 1 -- 172.21.15.69:0/2452017258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a71bb90 msgr2=0x55852a72a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:52.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.244+0000 7fb65a800640 1 --2- 172.21.15.69:0/2452017258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a71bb90 0x55852a72a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.244+0000 7fb65a800640 1 -- 172.21.15.69:0/2452017258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71b270 msgr2=0x55852a72a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:52.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.244+0000 7fb65a800640 1 --2- 172.21.15.69:0/2452017258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71b270 0x55852a72a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.244+0000 7fb65a800640 1 -- 172.21.15.69:0/2452017258 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55852a79c180 con 0x55852a726260 2024-04-24T09:08:52.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.244+0000 7fb65a800640 1 --2- 172.21.15.69:0/2452017258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a726260 0x55852a79bc40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb654009a80 tx=0x7fb654009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92824ee2880b16cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:52.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.244+0000 7fb63b400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:52.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.245+0000 7fb63b400640 1 -- 172.21.15.69:0/2452017258 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654016040 con 0x55852a726260 2024-04-24T09:08:52.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.245+0000 7fb6639ec740 1 -- 172.21.15.69:0/2452017258 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb654004f60 con 0x55852a726260 2024-04-24T09:08:52.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.245+0000 7fb63b400640 1 -- 172.21.15.69:0/2452017258 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb654028760 con 0x55852a726260 2024-04-24T09:08:52.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.245+0000 7fb63b400640 1 -- 172.21.15.69:0/2452017258 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb654012770 con 0x55852a726260 2024-04-24T09:08:52.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.245+0000 7fb6639ec740 10 client.16173 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:52.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.245+0000 7fb6639ec740 1 -- 172.21.15.69:0/2452017258 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55852a79d3f0 con 0x55852a726260 2024-04-24T09:08:52.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.246+0000 7fb63b400640 1 -- 172.21.15.69:0/2452017258 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb654004f60 con 0x55852a726260 2024-04-24T09:08:52.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.246+0000 7fb63b400640 1 -- 172.21.15.69:0/2452017258 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb65400f040 con 0x55852a726260 2024-04-24T09:08:52.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.246+0000 7fb6639ec740 10 client.16173 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:52.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.246+0000 7fb6639ec740 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:52.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.246+0000 7fb6639ec740 20 client.16173 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:52.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.246+0000 7fb6639ec740 1 --2- 172.21.15.69:0/2452017258 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55852a79d860 0x55852a79fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.246+0000 7fb6639ec740 4 client.16173 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:08:52.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.246+0000 7fb6639ec740 1 -- 172.21.15.69:0/2452017258 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55852a68e890 con 0x55852a79d860 2024-04-24T09:08:52.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.247+0000 7fb659400640 1 --2- 172.21.15.69:0/2452017258 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55852a79d860 0x55852a79fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.248+0000 7fb659400640 1 --2- 172.21.15.69:0/2452017258 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55852a79d860 0x55852a79fc80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28187c559eb45da4 server_cookie=e302e9a91c6cc0d8 in_seq=0 out_seq=0 2024-04-24T09:08:52.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.248+0000 7fb63b400640 10 client.16173 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:52.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.250+0000 7fb63b400640 1 -- 172.21.15.69:0/2452017258 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55852a68e890 con 0x55852a79d860 2024-04-24T09:08:52.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.250+0000 7fb63b400640 10 client.16173 handle_command_reply: tid=0 2024-04-24T09:08:52.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.250+0000 7fb6639ec740 1 client.16173 shutdown 2024-04-24T09:08:52.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.250+0000 7fb6639ec740 1 -- 172.21.15.69:0/2452017258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a726260 msgr2=0x55852a79bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.250+0000 7fb6639ec740 1 --2- 172.21.15.69:0/2452017258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a726260 0x55852a79bc40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb654009a80 tx=0x7fb654009ca0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.251+0000 7fb6639ec740 1 -- 172.21.15.69:0/2452017258 shutdown_connections 2024-04-24T09:08:52.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.251+0000 7fb6639ec740 1 --2- 172.21.15.69:0/2452017258 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55852a79d860 0x55852a79fc80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.251+0000 7fb6639ec740 1 --2- 172.21.15.69:0/2452017258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55852a726260 0x55852a79bc40 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.251+0000 7fb6639ec740 1 --2- 172.21.15.69:0/2452017258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55852a71bb90 0x55852a72a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.251+0000 7fb6639ec740 1 --2- 172.21.15.69:0/2452017258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852a71b270 0x55852a72a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.251+0000 7fb63b400640 0 client.16173 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:52.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.251+0000 7fb6639ec740 1 -- 172.21.15.69:0/2452017258 >> 172.21.15.69:0/2452017258 conn(0x55852a717590 msgr2=0x55852a715e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:52.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.251+0000 7fb6639ec740 1 -- 172.21.15.69:0/2452017258 shutdown_connections 2024-04-24T09:08:52.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.251+0000 7fb6639ec740 1 -- 172.21.15.69:0/2452017258 wait complete. 2024-04-24T09:08:52.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.251+0000 7fb6639ec740 20 client.16173 trim_cache size 0 max 16384 2024-04-24T09:08:52.252 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:08:52.252 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-04-24T09:08:52.252 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "13b2203e-5f5b-4beb-8fed-d83f1ed5c012", 2024-04-24T09:08:52.252 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-04-24T09:08:52.252 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:08:52.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.252+0000 7fb65b200640 1 -- 172.21.15.69:0/1582919214 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb6400a3d20 msgr2=0x7fb6400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.252+0000 7fb65b200640 1 --2- 172.21.15.69:0/1582919214 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb6400a3d20 0x7fb6400a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0ee5b0 tx=0x7fb64c02a470 comp rx=0 tx=0).stop 2024-04-24T09:08:52.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.252+0000 7fb65b200640 1 -- 172.21.15.69:0/1582919214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb65c100e50 msgr2=0x7fb65c19ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.252+0000 7fb65b200640 1 --2- 172.21.15.69:0/1582919214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb65c100e50 0x7fb65c19ed40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb64c005940 tx=0x7fb64c004270 comp rx=0 tx=0).stop 2024-04-24T09:08:52.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.252+0000 7fb65b200640 1 -- 172.21.15.69:0/1582919214 shutdown_connections 2024-04-24T09:08:52.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.252+0000 7fb65b200640 1 --2- 172.21.15.69:0/1582919214 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb6400a3d20 0x7fb6400a61e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.252+0000 7fb65b200640 1 --2- 172.21.15.69:0/1582919214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb65c106250 0x7fb65c1a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.252+0000 7fb65b200640 1 --2- 172.21.15.69:0/1582919214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb65c101770 0x7fb65c19f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.252+0000 7fb65b200640 1 --2- 172.21.15.69:0/1582919214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb65c100e50 0x7fb65c19ed40 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.252+0000 7fb65b200640 1 -- 172.21.15.69:0/1582919214 >> 172.21.15.69:0/1582919214 conn(0x7fb65c0fc880 msgr2=0x7fb65c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:52.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.252+0000 7fb65b200640 1 -- 172.21.15.69:0/1582919214 shutdown_connections 2024-04-24T09:08:52.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.252+0000 7fb65b200640 1 -- 172.21.15.69:0/1582919214 wait complete. 2024-04-24T09:08:52.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:08:52.271 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:08:52.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.504+0000 7efd00a00640 1 -- 172.21.15.69:0/1559846225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc103060 msgr2=0x7efcfc103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.504+0000 7efd00a00640 1 --2- 172.21.15.69:0/1559846225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc103060 0x7efcfc103440 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7efcec009900 tx=0x7efcec02a480 comp rx=0 tx=0).stop 2024-04-24T09:08:52.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.504+0000 7efd00a00640 1 -- 172.21.15.69:0/1559846225 shutdown_connections 2024-04-24T09:08:52.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.504+0000 7efd00a00640 1 --2- 172.21.15.69:0/1559846225 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcfc10c480 0x7efcfc10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.504+0000 7efd00a00640 1 --2- 172.21.15.69:0/1559846225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcfc103980 0x7efcfc10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.504+0000 7efd00a00640 1 --2- 172.21.15.69:0/1559846225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc103060 0x7efcfc103440 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.504+0000 7efd00a00640 1 -- 172.21.15.69:0/1559846225 >> 172.21.15.69:0/1559846225 conn(0x7efcfc0fc880 msgr2=0x7efcfc0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:52.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.504+0000 7efd00a00640 1 -- 172.21.15.69:0/1559846225 shutdown_connections 2024-04-24T09:08:52.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.504+0000 7efd00a00640 1 -- 172.21.15.69:0/1559846225 wait complete. 2024-04-24T09:08:52.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.506+0000 7efd00a00640 1 Processor -- start 2024-04-24T09:08:52.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.506+0000 7efd00a00640 1 -- start start 2024-04-24T09:08:52.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.506+0000 7efd00a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc103060 0x7efcfc19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.506+0000 7efd00a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcfc103980 0x7efcfc196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.506+0000 7efd00a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcfc10c480 0x7efcfc196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.506+0000 7efd00a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc10ff40 con 0x7efcfc103060 2024-04-24T09:08:52.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.506+0000 7efd00a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc10fdc0 con 0x7efcfc103980 2024-04-24T09:08:52.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.506+0000 7efd00a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1100c0 con 0x7efcfc10c480 2024-04-24T09:08:52.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.507+0000 7efcfb400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc103060 0x7efcfc19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.507+0000 7efcfb400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc103060 0x7efcfc19d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33662/0 (socket says 172.21.15.69:33662) 2024-04-24T09:08:52.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.507+0000 7efcfb400640 1 -- 172.21.15.69:0/271858633 learned_addr learned my addr 172.21.15.69:0/271858633 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:52.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.507+0000 7efcfbe00640 1 --2- 172.21.15.69:0/271858633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcfc10c480 0x7efcfc196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.507+0000 7efcfb400640 1 -- 172.21.15.69:0/271858633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcfc10c480 msgr2=0x7efcfc196890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.507+0000 7efcfb400640 1 --2- 172.21.15.69:0/271858633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcfc10c480 0x7efcfc196890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.507+0000 7efcfb400640 1 -- 172.21.15.69:0/271858633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcfc103980 msgr2=0x7efcfc196350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T09:08:52.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.507+0000 7efcfb400640 1 --2- 172.21.15.69:0/271858633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcfc103980 0x7efcfc196350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.507+0000 7efcfb400640 1 -- 172.21.15.69:0/271858633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc196fc0 con 0x7efcfc103060 2024-04-24T09:08:52.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.508+0000 7efcfb400640 1 --2- 172.21.15.69:0/271858633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc103060 0x7efcfc19d260 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7efcec005940 tx=0x7efcec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:52.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.508+0000 7efcf3e00640 1 -- 172.21.15.69:0/271858633 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec03f030 con 0x7efcfc103060 2024-04-24T09:08:52.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.508+0000 7efd00a00640 1 -- 172.21.15.69:0/271858633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcfc19adf0 con 0x7efcfc103060 2024-04-24T09:08:52.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.508+0000 7efcf3e00640 1 -- 172.21.15.69:0/271858633 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcec003aa0 con 0x7efcfc103060 2024-04-24T09:08:52.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.509+0000 7efcf3e00640 1 -- 172.21.15.69:0/271858633 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec035070 con 0x7efcfc103060 2024-04-24T09:08:52.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.509+0000 7efd00a00640 1 -- 172.21.15.69:0/271858633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcfc19b2e0 con 0x7efcfc103060 2024-04-24T09:08:52.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.513+0000 7efcf3e00640 1 -- 172.21.15.69:0/271858633 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efcec042030 con 0x7efcfc103060 2024-04-24T09:08:52.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.514+0000 7efd04e6c740 1 Processor -- start 2024-04-24T09:08:52.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.514+0000 7efd04e6c740 1 -- start start 2024-04-24T09:08:52.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.515+0000 7efd04e6c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 0x55e991965e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.515+0000 7efd04e6c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991966350 0x55e9919704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.515+0000 7efd04e6c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 0x55e991972e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.515+0000 7efd04e6c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e9918cb1f0 con 0x55e991970a20 2024-04-24T09:08:52.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.515+0000 7efd04e6c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e991871350 con 0x55e991965a30 2024-04-24T09:08:52.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.515+0000 7efd04e6c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55e9918c71b0 con 0x55e991966350 2024-04-24T09:08:52.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.515+0000 7efcfbe00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 0x55e991965e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.515+0000 7efcfbe00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 0x55e991965e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.69:40630/0 (socket says 172.21.15.69:40630) 2024-04-24T09:08:52.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.515+0000 7efcfbe00640 1 -- 172.21.15.69:0/609422552 learned_addr learned my addr 172.21.15.69:0/609422552 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:52.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.516+0000 7efcfb400640 1 --2- 172.21.15.69:0/609422552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991966350 0x55e9919704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.516+0000 7efcf3e00640 1 --2- 172.21.15.69:0/271858633 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7efce00a3da0 0x7efce00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.516+0000 7efcfbe00640 1 -- 172.21.15.69:0/609422552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991966350 msgr2=0x55e9919704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:52.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.516+0000 7efcfbe00640 1 --2- 172.21.15.69:0/609422552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991966350 0x55e9919704e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.516+0000 7efcfbe00640 1 -- 172.21.15.69:0/609422552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 msgr2=0x55e991972e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:08:52.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.516+0000 7efcfbe00640 1 --2- 172.21.15.69:0/609422552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 0x55e991972e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.516+0000 7efcfbe00640 1 -- 172.21.15.69:0/609422552 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9918c5f50 con 0x55e991965a30 2024-04-24T09:08:52.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.516+0000 7efcf3e00640 1 -- 172.21.15.69:0/271858633 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7efcec0f0eb0 con 0x7efcfc103060 2024-04-24T09:08:52.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.517+0000 7efcfbe00640 1 --2- 172.21.15.69:0/609422552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 0x55e991965e10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efcf40072e0 tx=0x7efcf400f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72fe2037c7f3b663 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:52.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.517+0000 7efce7e00640 1 -- 172.21.15.69:0/609422552 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4012920 con 0x55e991965a30 2024-04-24T09:08:52.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.517+0000 7efce7e00640 1 -- 172.21.15.69:0/609422552 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcf401a3f0 con 0x55e991965a30 2024-04-24T09:08:52.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.517+0000 7efce7e00640 1 -- 172.21.15.69:0/609422552 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4019600 con 0x55e991965a30 2024-04-24T09:08:52.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.518+0000 7efd04e6c740 1 -- 172.21.15.69:0/609422552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 msgr2=0x55e991965e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.518+0000 7efd04e6c740 1 --2- 172.21.15.69:0/609422552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 0x55e991965e10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efcf40072e0 tx=0x7efcf400f850 comp rx=0 tx=0).stop 2024-04-24T09:08:52.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.518+0000 7efcfbe00640 1 --2- 172.21.15.69:0/271858633 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7efce00a3da0 0x7efce00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.518+0000 7efd04e6c740 1 -- 172.21.15.69:0/609422552 shutdown_connections 2024-04-24T09:08:52.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.519+0000 7efd04e6c740 1 --2- 172.21.15.69:0/609422552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 0x55e991972e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.519+0000 7efd04e6c740 1 --2- 172.21.15.69:0/609422552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991966350 0x55e9919704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.519+0000 7efd04e6c740 1 --2- 172.21.15.69:0/609422552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 0x55e991965e10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.519+0000 7efd04e6c740 1 -- 172.21.15.69:0/609422552 >> 172.21.15.69:0/609422552 conn(0x55e991961d50 msgr2=0x55e991962130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:52.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.519+0000 7efd04e6c740 1 -- 172.21.15.69:0/609422552 shutdown_connections 2024-04-24T09:08:52.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.519+0000 7efd04e6c740 1 -- 172.21.15.69:0/609422552 wait complete. 2024-04-24T09:08:52.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.519+0000 7efcfbe00640 1 --2- 172.21.15.69:0/271858633 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7efce00a3da0 0x7efce00a6260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efcf4007ca0 tx=0x7efcf4014040 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:52.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.520+0000 7efd04e6c740 1 -- start start 2024-04-24T09:08:52.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.520+0000 7efd04e6c740 1 Processor -- start 2024-04-24T09:08:52.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.521+0000 7efd04e6c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 0x55e99196d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.521+0000 7efd04e6c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991966350 0x55e99196ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.521+0000 7efd04e6c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991970a20 0x55e99196e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.521+0000 7efd04e6c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e991973810 con 0x55e991966350 2024-04-24T09:08:52.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.521+0000 7efd04e6c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e991884690 con 0x55e991965a30 2024-04-24T09:08:52.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.521+0000 7efd04e6c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55e991973990 con 0x55e991970a20 2024-04-24T09:08:52.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.521+0000 7efd04e6c740 10 client.0 _ll_register_callbacks cb 0x55e9918c0650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:52.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.521+0000 7efcfaa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991966350 0x55e99196ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.521+0000 7efcfaa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991966350 0x55e99196ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33678/0 (socket says 172.21.15.69:33678) 2024-04-24T09:08:52.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.521+0000 7efcfaa00640 1 -- 172.21.15.69:0/2626903928 learned_addr learned my addr 172.21.15.69:0/2626903928 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:52.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.521+0000 7efcfbe00640 1 --2- 172.21.15.69:0/2626903928 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991970a20 0x55e99196e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.521+0000 7efcfb400640 1 --2- 172.21.15.69:0/2626903928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 0x55e99196d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.522+0000 7efcfaa00640 1 -- 172.21.15.69:0/2626903928 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991970a20 msgr2=0x55e99196e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:52.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.522+0000 7efcfaa00640 1 --2- 172.21.15.69:0/2626903928 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991970a20 0x55e99196e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.522+0000 7efcfaa00640 1 -- 172.21.15.69:0/2626903928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 msgr2=0x55e99196d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:52.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.522+0000 7efcfaa00640 1 --2- 172.21.15.69:0/2626903928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 0x55e99196d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.522+0000 7efcfaa00640 1 -- 172.21.15.69:0/2626903928 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e991883920 con 0x55e991966350 2024-04-24T09:08:52.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.522+0000 7efcfb400640 1 --2- 172.21.15.69:0/2626903928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 0x55e99196d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:52.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.522+0000 7efcfaa00640 1 --2- 172.21.15.69:0/2626903928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991966350 0x55e99196ddf0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7efce8005fd0 tx=0x7efce800d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d725ece699eab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:52.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.522+0000 7efce4c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:52.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.523+0000 7efce4c00640 1 -- 172.21.15.69:0/2626903928 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce8016070 con 0x55e991966350 2024-04-24T09:08:52.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.523+0000 7efd04e6c740 1 -- 172.21.15.69:0/2626903928 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf401a3f0 con 0x55e991966350 2024-04-24T09:08:52.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.523+0000 7efce4c00640 1 -- 172.21.15.69:0/2626903928 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7efce8005020 con 0x55e991966350 2024-04-24T09:08:52.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.523+0000 7efce4c00640 1 -- 172.21.15.69:0/2626903928 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efce8021050 con 0x55e991966350 2024-04-24T09:08:52.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.523+0000 7efd04e6c740 10 client.16182 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:52.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.523+0000 7efd04e6c740 1 -- 172.21.15.69:0/2626903928 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e9916a77d0 con 0x55e991966350 2024-04-24T09:08:52.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.523+0000 7efce4c00640 1 -- 172.21.15.69:0/2626903928 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcf401a3f0 con 0x55e991966350 2024-04-24T09:08:52.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.524+0000 7efce4c00640 1 -- 172.21.15.69:0/2626903928 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7efce8010460 con 0x55e991966350 2024-04-24T09:08:52.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.524+0000 7efd04e6c740 10 client.16182 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:52.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.524+0000 7efd04e6c740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:52.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.524+0000 7efd04e6c740 20 client.16182 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:52.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.524+0000 7efd04e6c740 1 --2- 172.21.15.69:0/2626903928 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e991980cf0 0x55e991983150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.524+0000 7efd04e6c740 4 client.16182 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:52.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.524+0000 7efd04e6c740 1 -- 172.21.15.69:0/2626903928 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9918d8c10 con 0x55e991980cf0 2024-04-24T09:08:52.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.525+0000 7efcfb400640 1 --2- 172.21.15.69:0/2626903928 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e991980cf0 0x55e991983150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.526+0000 7efcfb400640 1 --2- 172.21.15.69:0/2626903928 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e991980cf0 0x55e991983150 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b29299ae23073aa8 server_cookie=29dea0d386a765e2 in_seq=0 out_seq=0 2024-04-24T09:08:52.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.526+0000 7efce4c00640 10 client.16182 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:52.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.528+0000 7efce4c00640 1 -- 172.21.15.69:0/2626903928 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9918d8c10 con 0x55e991980cf0 2024-04-24T09:08:52.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.528+0000 7efce4c00640 10 client.16182 handle_command_reply: tid=0 2024-04-24T09:08:52.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.528+0000 7efd04e6c740 1 client.16182 shutdown 2024-04-24T09:08:52.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.528+0000 7efd04e6c740 1 -- 172.21.15.69:0/2626903928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991966350 msgr2=0x55e99196ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.528+0000 7efd04e6c740 1 --2- 172.21.15.69:0/2626903928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991966350 0x55e99196ddf0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7efce8005fd0 tx=0x7efce800d2e0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.528+0000 7efd04e6c740 1 -- 172.21.15.69:0/2626903928 shutdown_connections 2024-04-24T09:08:52.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.528+0000 7efd04e6c740 1 --2- 172.21.15.69:0/2626903928 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e991980cf0 0x55e991983150 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.528+0000 7efd04e6c740 1 --2- 172.21.15.69:0/2626903928 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991970a20 0x55e99196e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.528+0000 7efd04e6c740 1 --2- 172.21.15.69:0/2626903928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991966350 0x55e99196ddf0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.528+0000 7efd04e6c740 1 --2- 172.21.15.69:0/2626903928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 0x55e99196d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.528+0000 7efd04e6c740 1 -- 172.21.15.69:0/2626903928 >> 172.21.15.69:0/2626903928 conn(0x55e991961d50 msgr2=0x55e991967b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:52.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.528+0000 7efce4c00640 0 client.16182 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:52.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.528+0000 7efd04e6c740 1 -- 172.21.15.69:0/2626903928 shutdown_connections 2024-04-24T09:08:52.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.529+0000 7efd04e6c740 1 -- 172.21.15.69:0/2626903928 wait complete. 2024-04-24T09:08:52.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.529+0000 7efd04e6c740 20 client.16182 trim_cache size 0 max 16384 2024-04-24T09:08:52.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.557+0000 7efd04e6c740 1 Processor -- start 2024-04-24T09:08:52.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.558+0000 7efd04e6c740 1 -- start start 2024-04-24T09:08:52.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.558+0000 7efd04e6c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 0x55e99197c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.558+0000 7efd04e6c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991966350 0x55e991962130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.558+0000 7efd04e6c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 0x55e99196ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.558+0000 7efd04e6c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1c06b0 con 0x55e991970a20 2024-04-24T09:08:52.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.558+0000 7efd04e6c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1bdf80 con 0x55e991965a30 2024-04-24T09:08:52.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.558+0000 7efd04e6c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1bbdd0 con 0x55e991966350 2024-04-24T09:08:52.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.559+0000 7efcfbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 0x55e99196ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.559+0000 7efcfbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 0x55e99196ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33686/0 (socket says 172.21.15.69:33686) 2024-04-24T09:08:52.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.559+0000 7efcfbe00640 1 -- 172.21.15.69:0/3870094562 learned_addr learned my addr 172.21.15.69:0/3870094562 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:52.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.559+0000 7efcfaa00640 1 --2- 172.21.15.69:0/3870094562 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991966350 0x55e991962130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.559+0000 7efcfb400640 1 --2- 172.21.15.69:0/3870094562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 0x55e99197c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.559+0000 7efcfbe00640 1 -- 172.21.15.69:0/3870094562 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991966350 msgr2=0x55e991962130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:52.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.559+0000 7efcfbe00640 1 --2- 172.21.15.69:0/3870094562 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991966350 0x55e991962130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.559+0000 7efcfbe00640 1 -- 172.21.15.69:0/3870094562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 msgr2=0x55e99197c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:52.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.559+0000 7efcfbe00640 1 --2- 172.21.15.69:0/3870094562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 0x55e99197c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.559+0000 7efcfbe00640 1 -- 172.21.15.69:0/3870094562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9915dd3f0 con 0x55e991970a20 2024-04-24T09:08:52.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.559+0000 7efcfaa00640 1 --2- 172.21.15.69:0/3870094562 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991966350 0x55e991962130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:52.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.560+0000 7efcfbe00640 1 --2- 172.21.15.69:0/3870094562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 0x55e99196ca10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efcf400fd50 tx=0x7efcf4007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ab09d1d7779f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:52.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.560+0000 7efce7400640 1 -- 172.21.15.69:0/3870094562 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4034070 con 0x55e991970a20 2024-04-24T09:08:52.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.560+0000 7efce7400640 1 -- 172.21.15.69:0/3870094562 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcf401a8a0 con 0x55e991970a20 2024-04-24T09:08:52.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.560+0000 7efce7400640 1 -- 172.21.15.69:0/3870094562 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4012d00 con 0x55e991970a20 2024-04-24T09:08:52.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.561+0000 7efd04e6c740 1 -- 172.21.15.69:0/3870094562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 msgr2=0x55e99196ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.561+0000 7efd04e6c740 1 --2- 172.21.15.69:0/3870094562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 0x55e99196ca10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efcf400fd50 tx=0x7efcf4007520 comp rx=0 tx=0).stop 2024-04-24T09:08:52.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.561+0000 7efd04e6c740 1 -- 172.21.15.69:0/3870094562 shutdown_connections 2024-04-24T09:08:52.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.561+0000 7efd04e6c740 1 --2- 172.21.15.69:0/3870094562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 0x55e99196ca10 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.561+0000 7efd04e6c740 1 --2- 172.21.15.69:0/3870094562 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991966350 0x55e991962130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.561+0000 7efd04e6c740 1 --2- 172.21.15.69:0/3870094562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991965a30 0x55e99197c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.561+0000 7efd04e6c740 1 -- 172.21.15.69:0/3870094562 >> 172.21.15.69:0/3870094562 conn(0x55e991961d50 msgr2=0x55e991967290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:52.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.561+0000 7efd04e6c740 1 -- 172.21.15.69:0/3870094562 shutdown_connections 2024-04-24T09:08:52.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.561+0000 7efd04e6c740 1 -- 172.21.15.69:0/3870094562 wait complete. 2024-04-24T09:08:52.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.562+0000 7efd04e6c740 1 -- start start 2024-04-24T09:08:52.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.562+0000 7efd04e6c740 1 Processor -- start 2024-04-24T09:08:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.563+0000 7efd04e6c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991965a30 0x55e991978b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.563+0000 7efd04e6c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991966350 0x55e991979040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.563+0000 7efd04e6c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 0x55e991973ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.563+0000 7efd04e6c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1bc440 con 0x55e991970a20 2024-04-24T09:08:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.563+0000 7efd04e6c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1bd8b0 con 0x55e991966350 2024-04-24T09:08:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.563+0000 7efd04e6c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1bcb10 con 0x55e991965a30 2024-04-24T09:08:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.563+0000 7efd04e6c740 10 client.0 _ll_register_callbacks cb 0x7efcfc06a1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.563+0000 7efcfbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 0x55e991973ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.563+0000 7efcfbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 0x55e991973ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33700/0 (socket says 172.21.15.69:33700) 2024-04-24T09:08:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.563+0000 7efcfbe00640 1 -- 172.21.15.69:0/2434382005 learned_addr learned my addr 172.21.15.69:0/2434382005 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.563+0000 7efcfb400640 1 --2- 172.21.15.69:0/2434382005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991965a30 0x55e991978b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.563+0000 7efcfaa00640 1 --2- 172.21.15.69:0/2434382005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991966350 0x55e991979040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.564+0000 7efcfbe00640 1 -- 172.21.15.69:0/2434382005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991965a30 msgr2=0x55e991978b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.564+0000 7efcfbe00640 1 --2- 172.21.15.69:0/2434382005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991965a30 0x55e991978b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.564+0000 7efcfbe00640 1 -- 172.21.15.69:0/2434382005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991966350 msgr2=0x55e991979040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:52.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.564+0000 7efcfbe00640 1 --2- 172.21.15.69:0/2434382005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991966350 0x55e991979040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.564+0000 7efcfbe00640 1 -- 172.21.15.69:0/2434382005 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e99159d0f0 con 0x55e991970a20 2024-04-24T09:08:52.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.564+0000 7efcfbe00640 1 --2- 172.21.15.69:0/2434382005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 0x55e991973ff0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7efcf4007fd0 tx=0x7efcf401a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70923c185f07a44c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:52.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.564+0000 7efce4c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:52.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.564+0000 7efce4c00640 1 -- 172.21.15.69:0/2434382005 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4034070 con 0x55e991970a20 2024-04-24T09:08:52.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.564+0000 7efce4c00640 1 -- 172.21.15.69:0/2434382005 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7efcf4004dd0 con 0x55e991970a20 2024-04-24T09:08:52.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.565+0000 7efd04e6c740 1 -- 172.21.15.69:0/2434382005 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf401a8a0 con 0x55e991970a20 2024-04-24T09:08:52.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.565+0000 7efce4c00640 1 -- 172.21.15.69:0/2434382005 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efcf4034070 con 0x55e991970a20 2024-04-24T09:08:52.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.565+0000 7efd04e6c740 10 client.16188 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:52.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.565+0000 7efd04e6c740 1 -- 172.21.15.69:0/2434382005 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e991890810 con 0x55e991970a20 2024-04-24T09:08:52.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.565+0000 7efce4c00640 1 -- 172.21.15.69:0/2434382005 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcf401a8a0 con 0x55e991970a20 2024-04-24T09:08:52.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.566+0000 7efce4c00640 1 -- 172.21.15.69:0/2434382005 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7efcf402e070 con 0x55e991970a20 2024-04-24T09:08:52.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.566+0000 7efd04e6c740 10 client.16188 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:52.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.566+0000 7efd04e6c740 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:52.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.566+0000 7efd04e6c740 20 client.16188 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:52.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.566+0000 7efd04e6c740 1 --2- 172.21.15.69:0/2434382005 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e9919f7ea0 0x55e9919fa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.566+0000 7efd04e6c740 4 client.16188 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:08:52.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.566+0000 7efd04e6c740 1 -- 172.21.15.69:0/2434382005 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e9918d2a80 con 0x55e9919f7ea0 2024-04-24T09:08:52.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.566+0000 7efcfb400640 1 --2- 172.21.15.69:0/2434382005 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e9919f7ea0 0x55e9919fa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.567+0000 7efcfb400640 1 --2- 172.21.15.69:0/2434382005 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e9919f7ea0 0x55e9919fa290 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b411e3ed34daaf server_cookie=994fa78f93c746c8 in_seq=0 out_seq=0 2024-04-24T09:08:52.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.567+0000 7efce4c00640 10 client.16188 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:52.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.568+0000 7efce4c00640 1 -- 172.21.15.69:0/2434382005 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e9918d2a80 con 0x55e9919f7ea0 2024-04-24T09:08:52.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.568+0000 7efce4c00640 10 client.16188 handle_command_reply: tid=0 2024-04-24T09:08:52.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.568+0000 7efd04e6c740 1 client.16188 shutdown 2024-04-24T09:08:52.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.568+0000 7efd04e6c740 1 -- 172.21.15.69:0/2434382005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 msgr2=0x55e991973ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.568+0000 7efd04e6c740 1 --2- 172.21.15.69:0/2434382005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 0x55e991973ff0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7efcf4007fd0 tx=0x7efcf401a6a0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.568+0000 7efd04e6c740 1 -- 172.21.15.69:0/2434382005 shutdown_connections 2024-04-24T09:08:52.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.568+0000 7efd04e6c740 1 --2- 172.21.15.69:0/2434382005 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e9919f7ea0 0x55e9919fa290 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.568+0000 7efd04e6c740 1 --2- 172.21.15.69:0/2434382005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e991970a20 0x55e991973ff0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.568+0000 7efd04e6c740 1 --2- 172.21.15.69:0/2434382005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e991966350 0x55e991979040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.568+0000 7efd04e6c740 1 --2- 172.21.15.69:0/2434382005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e991965a30 0x55e991978b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.568+0000 7efd04e6c740 1 -- 172.21.15.69:0/2434382005 >> 172.21.15.69:0/2434382005 conn(0x55e991961d50 msgr2=0x55e991960fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:52.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.568+0000 7efce4c00640 0 client.16188 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:52.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.569+0000 7efd04e6c740 1 -- 172.21.15.69:0/2434382005 shutdown_connections 2024-04-24T09:08:52.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.569+0000 7efd04e6c740 1 -- 172.21.15.69:0/2434382005 wait complete. 2024-04-24T09:08:52.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.569+0000 7efd04e6c740 20 client.16188 trim_cache size 0 max 16384 2024-04-24T09:08:52.570 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:08:52.570 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-04-24T09:08:52.570 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-04-24T09:08:52.570 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:08:52.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.570+0000 7efd00a00640 1 -- 172.21.15.69:0/271858633 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7efce00a3da0 msgr2=0x7efce00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.570+0000 7efd00a00640 1 --2- 172.21.15.69:0/271858633 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7efce00a3da0 0x7efce00a6260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efcf4007ca0 tx=0x7efcf4014040 comp rx=0 tx=0).stop 2024-04-24T09:08:52.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.570+0000 7efd00a00640 1 -- 172.21.15.69:0/271858633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc103060 msgr2=0x7efcfc19d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.570+0000 7efd00a00640 1 --2- 172.21.15.69:0/271858633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc103060 0x7efcfc19d260 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7efcec005940 tx=0x7efcec004270 comp rx=0 tx=0).stop 2024-04-24T09:08:52.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.570+0000 7efd00a00640 1 -- 172.21.15.69:0/271858633 shutdown_connections 2024-04-24T09:08:52.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.570+0000 7efd00a00640 1 --2- 172.21.15.69:0/271858633 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7efce00a3da0 0x7efce00a6260 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.570+0000 7efd00a00640 1 --2- 172.21.15.69:0/271858633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcfc10c480 0x7efcfc196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.570+0000 7efd00a00640 1 --2- 172.21.15.69:0/271858633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcfc103980 0x7efcfc196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.570+0000 7efd00a00640 1 --2- 172.21.15.69:0/271858633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc103060 0x7efcfc19d260 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.570+0000 7efd00a00640 1 -- 172.21.15.69:0/271858633 >> 172.21.15.69:0/271858633 conn(0x7efcfc0fc880 msgr2=0x7efcfc0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:52.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.570+0000 7efd00a00640 1 -- 172.21.15.69:0/271858633 shutdown_connections 2024-04-24T09:08:52.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.570+0000 7efd00a00640 1 -- 172.21.15.69:0/271858633 wait complete. 2024-04-24T09:08:52.596 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:08:52.596 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:08:52.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.798+0000 7f261a000640 1 -- 172.21.15.69:0/24257756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2614104440 msgr2=0x7f2614104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.798+0000 7f261a000640 1 --2- 172.21.15.69:0/24257756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2614104440 0x7f2614104820 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2604009900 tx=0x7f260402a410 comp rx=0 tx=0).stop 2024-04-24T09:08:52.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.799+0000 7f261a000640 1 -- 172.21.15.69:0/24257756 shutdown_connections 2024-04-24T09:08:52.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.799+0000 7f261a000640 1 --2- 172.21.15.69:0/24257756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2614109510 0x7f26141110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.799+0000 7f261a000640 1 --2- 172.21.15.69:0/24257756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2614104df0 0x7f2614108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.799+0000 7f261a000640 1 --2- 172.21.15.69:0/24257756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2614104440 0x7f2614104820 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.799+0000 7f261a000640 1 -- 172.21.15.69:0/24257756 >> 172.21.15.69:0/24257756 conn(0x7f26140ffcc0 msgr2=0x7f26141020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:52.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.799+0000 7f261a000640 1 -- 172.21.15.69:0/24257756 shutdown_connections 2024-04-24T09:08:52.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.799+0000 7f261a000640 1 -- 172.21.15.69:0/24257756 wait complete. 2024-04-24T09:08:52.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.800+0000 7f261a000640 1 Processor -- start 2024-04-24T09:08:52.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.800+0000 7f261a000640 1 -- start start 2024-04-24T09:08:52.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.800+0000 7f261a000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2614104440 0x7f26141a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.800+0000 7f261a000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2614109510 0x7f26141a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.800+0000 7f261a000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f261419a8c0 0x7f261419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.800+0000 7f261a000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f26141144c0 con 0x7f2614104440 2024-04-24T09:08:52.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.800+0000 7f261a000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2614114340 con 0x7f2614109510 2024-04-24T09:08:52.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.800+0000 7f261a000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2614114640 con 0x7f261419a8c0 2024-04-24T09:08:52.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.801+0000 7f2613400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2614104440 0x7f26141a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.801+0000 7f2613400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2614104440 0x7f26141a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33712/0 (socket says 172.21.15.69:33712) 2024-04-24T09:08:52.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.801+0000 7f2613400640 1 -- 172.21.15.69:0/57762720 learned_addr learned my addr 172.21.15.69:0/57762720 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:52.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.801+0000 7f2613400640 1 -- 172.21.15.69:0/57762720 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f261419a8c0 msgr2=0x7f261419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.801+0000 7f2613e00640 1 --2- 172.21.15.69:0/57762720 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f261419a8c0 0x7f261419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.801+0000 7f2613400640 1 --2- 172.21.15.69:0/57762720 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f261419a8c0 0x7f261419ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.801+0000 7f2613400640 1 -- 172.21.15.69:0/57762720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2614109510 msgr2=0x7f26141a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:52.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.801+0000 7f2613400640 1 --2- 172.21.15.69:0/57762720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2614109510 0x7f26141a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:52.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.801+0000 7f2613400640 1 -- 172.21.15.69:0/57762720 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261419b3e0 con 0x7f2614104440 2024-04-24T09:08:52.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.801+0000 7f2613400640 1 --2- 172.21.15.69:0/57762720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2614104440 0x7f26141a06c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2604005ab0 tx=0x7f2604003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:52.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.802+0000 7f260be00640 1 -- 172.21.15.69:0/57762720 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260403f070 con 0x7f2614104440 2024-04-24T09:08:52.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.802+0000 7f260be00640 1 -- 172.21.15.69:0/57762720 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2604031d60 con 0x7f2614104440 2024-04-24T09:08:52.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.802+0000 7f261a000640 1 -- 172.21.15.69:0/57762720 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261419b670 con 0x7f2614104440 2024-04-24T09:08:52.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.802+0000 7f260be00640 1 -- 172.21.15.69:0/57762720 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604035030 con 0x7f2614104440 2024-04-24T09:08:52.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.802+0000 7f261a000640 1 -- 172.21.15.69:0/57762720 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26141a7810 con 0x7f2614104440 2024-04-24T09:08:52.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.803+0000 7f261a000640 1 -- 172.21.15.69:0/57762720 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2614069270 con 0x7f2614104440 2024-04-24T09:08:52.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.805+0000 7f260be00640 1 -- 172.21.15.69:0/57762720 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2604042070 con 0x7f2614104440 2024-04-24T09:08:52.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.806+0000 7f260be00640 1 --2- 172.21.15.69:0/57762720 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f25f80a3d20 0x7f25f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:52.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.807+0000 7f260be00640 1 -- 172.21.15.69:0/57762720 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f26040f11b0 con 0x7f2614104440 2024-04-24T09:08:52.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.807+0000 7f2612a00640 1 --2- 172.21.15.69:0/57762720 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f25f80a3d20 0x7f25f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:52.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.808+0000 7f2612a00640 1 --2- 172.21.15.69:0/57762720 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f25f80a3d20 0x7f25f80a61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2600005fd0 tx=0x7f2600005850 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:52.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:52.812+0000 7f260be00640 1 -- 172.21.15.69:0/57762720 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26040bd4b0 con 0x7f2614104440 2024-04-24T09:08:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.083+0000 7f261a000640 1 -- 172.21.15.69:0/57762720 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f261419bda0 con 0x7f2614104440 2024-04-24T09:08:53.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.085+0000 7f260be00640 1 -- 172.21.15.69:0/57762720 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11873 (secure 0 0 0) 0x7f26040bcdb0 con 0x7f2614104440 2024-04-24T09:08:53.085 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:08:53.085 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:04:01.991559+0000","modified":"2024-04-24T09:07:02.537848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34532,"mds_1":34496,"mds_2":34460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34496":{"gid":34496,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34532":{"gid":34532,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34532,"qdb_cluster":[34532,34460,34496]},"id":1}]} 2024-04-24T09:08:53.085 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 24 2024-04-24T09:08:53.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.088+0000 7f261a000640 1 -- 172.21.15.69:0/57762720 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f25f80a3d20 msgr2=0x7f25f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.088+0000 7f261a000640 1 --2- 172.21.15.69:0/57762720 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f25f80a3d20 0x7f25f80a61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2600005fd0 tx=0x7f2600005850 comp rx=0 tx=0).stop 2024-04-24T09:08:53.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.088+0000 7f261a000640 1 -- 172.21.15.69:0/57762720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2614104440 msgr2=0x7f26141a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.088+0000 7f261a000640 1 --2- 172.21.15.69:0/57762720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2614104440 0x7f26141a06c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2604005ab0 tx=0x7f2604003cd0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.088+0000 7f261a000640 1 -- 172.21.15.69:0/57762720 shutdown_connections 2024-04-24T09:08:53.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.088+0000 7f261a000640 1 --2- 172.21.15.69:0/57762720 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f25f80a3d20 0x7f25f80a61e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.088+0000 7f261a000640 1 --2- 172.21.15.69:0/57762720 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f261419a8c0 0x7f261419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.088+0000 7f261a000640 1 --2- 172.21.15.69:0/57762720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2614109510 0x7f26141a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.088+0000 7f261a000640 1 --2- 172.21.15.69:0/57762720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2614104440 0x7f26141a06c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.088+0000 7f261a000640 1 -- 172.21.15.69:0/57762720 >> 172.21.15.69:0/57762720 conn(0x7f26140ffcc0 msgr2=0x7f2614101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.088+0000 7f261a000640 1 -- 172.21.15.69:0/57762720 shutdown_connections 2024-04-24T09:08:53.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.088+0000 7f261a000640 1 -- 172.21.15.69:0/57762720 wait complete. 2024-04-24T09:08:53.109 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:08:53.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.320+0000 7fa327e00640 1 -- 172.21.15.69:0/80254641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328103980 msgr2=0x7fa32810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.320+0000 7fa327e00640 1 --2- 172.21.15.69:0/80254641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328103980 0x7fa32810be60 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa314009900 tx=0x7fa31402a480 comp rx=0 tx=0).stop 2024-04-24T09:08:53.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.320+0000 7fa327e00640 1 -- 172.21.15.69:0/80254641 shutdown_connections 2024-04-24T09:08:53.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.320+0000 7fa327e00640 1 --2- 172.21.15.69:0/80254641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa32810c480 0x7fa32810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.320+0000 7fa327e00640 1 --2- 172.21.15.69:0/80254641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328103980 0x7fa32810be60 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.320+0000 7fa327e00640 1 --2- 172.21.15.69:0/80254641 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa328103060 0x7fa328103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.320+0000 7fa327e00640 1 -- 172.21.15.69:0/80254641 >> 172.21.15.69:0/80254641 conn(0x7fa3280fc880 msgr2=0x7fa3280feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.320+0000 7fa327e00640 1 -- 172.21.15.69:0/80254641 shutdown_connections 2024-04-24T09:08:53.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.320+0000 7fa327e00640 1 -- 172.21.15.69:0/80254641 wait complete. 2024-04-24T09:08:53.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.321+0000 7fa327e00640 1 Processor -- start 2024-04-24T09:08:53.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.321+0000 7fa327e00640 1 -- start start 2024-04-24T09:08:53.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.321+0000 7fa327e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328103060 0x7fa328100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.321+0000 7fa327e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa328103980 0x7fa3281011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.321+0000 7fa327e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa32810c480 0x7fa328101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.321+0000 7fa327e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa328112160 con 0x7fa328103060 2024-04-24T09:08:53.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.321+0000 7fa327e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa328111fe0 con 0x7fa328103980 2024-04-24T09:08:53.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.321+0000 7fa327e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281122e0 con 0x7fa32810c480 2024-04-24T09:08:53.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.322+0000 7fa326a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328103060 0x7fa328100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.322+0000 7fa326a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328103060 0x7fa328100cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38220/0 (socket says 172.21.15.69:38220) 2024-04-24T09:08:53.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.322+0000 7fa326a00640 1 -- 172.21.15.69:0/2774499940 learned_addr learned my addr 172.21.15.69:0/2774499940 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:53.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.322+0000 7fa326000640 1 --2- 172.21.15.69:0/2774499940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa328103980 0x7fa3281011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.322+0000 7fa327400640 1 --2- 172.21.15.69:0/2774499940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa32810c480 0x7fa328101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.322+0000 7fa326a00640 1 -- 172.21.15.69:0/2774499940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa32810c480 msgr2=0x7fa328101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.322+0000 7fa326a00640 1 --2- 172.21.15.69:0/2774499940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa32810c480 0x7fa328101730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.322+0000 7fa326a00640 1 -- 172.21.15.69:0/2774499940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa328103980 msgr2=0x7fa3281011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.322+0000 7fa326a00640 1 --2- 172.21.15.69:0/2774499940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa328103980 0x7fa3281011f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.322+0000 7fa326a00640 1 -- 172.21.15.69:0/2774499940 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3281a6b60 con 0x7fa328103060 2024-04-24T09:08:53.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.323+0000 7fa326a00640 1 --2- 172.21.15.69:0/2774499940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328103060 0x7fa328100cb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa31800d770 tx=0x7fa31800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.323+0000 7fa31f400640 1 -- 172.21.15.69:0/2774499940 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318017070 con 0x7fa328103060 2024-04-24T09:08:53.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.323+0000 7fa31f400640 1 -- 172.21.15.69:0/2774499940 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3180054d0 con 0x7fa328103060 2024-04-24T09:08:53.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.323+0000 7fa327e00640 1 -- 172.21.15.69:0/2774499940 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3281a6e50 con 0x7fa328103060 2024-04-24T09:08:53.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.323+0000 7fa31f400640 1 -- 172.21.15.69:0/2774499940 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318004cf0 con 0x7fa328103060 2024-04-24T09:08:53.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.323+0000 7fa327e00640 1 -- 172.21.15.69:0/2774499940 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3281a72c0 con 0x7fa328103060 2024-04-24T09:08:53.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.327+0000 7fa3309b7740 1 Processor -- start 2024-04-24T09:08:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.327+0000 7fa3309b7740 1 -- start start 2024-04-24T09:08:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.327+0000 7fa31f400640 1 -- 172.21.15.69:0/2774499940 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa318004e90 con 0x7fa328103060 2024-04-24T09:08:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.327+0000 7fa3309b7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be175b40 0x5566be175f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.327+0000 7fa3309b7740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be176460 0x5566be1805f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.327+0000 7fa3309b7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be180b30 0x5566be182f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.327+0000 7fa3309b7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5566be0db1d0 con 0x5566be180b30 2024-04-24T09:08:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.327+0000 7fa3309b7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5566be081450 con 0x5566be175b40 2024-04-24T09:08:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.327+0000 7fa3309b7740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5566be0d7190 con 0x5566be176460 2024-04-24T09:08:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.328+0000 7fa326000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be180b30 0x5566be182f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.328+0000 7fa326000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be180b30 0x5566be182f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38224/0 (socket says 172.21.15.69:38224) 2024-04-24T09:08:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.328+0000 7fa326000640 1 -- 172.21.15.69:0/3542087396 learned_addr learned my addr 172.21.15.69:0/3542087396 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:53.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.328+0000 7fa327400640 1 --2- 172.21.15.69:0/3542087396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be175b40 0x5566be175f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.328+0000 7fa326a00640 1 --2- 172.21.15.69:0/3542087396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be176460 0x5566be1805f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.328+0000 7fa326000640 1 -- 172.21.15.69:0/3542087396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be176460 msgr2=0x5566be1805f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.328+0000 7fa326000640 1 --2- 172.21.15.69:0/3542087396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be176460 0x5566be1805f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.328+0000 7fa326000640 1 -- 172.21.15.69:0/3542087396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be175b40 msgr2=0x5566be175f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.328+0000 7fa326000640 1 --2- 172.21.15.69:0/3542087396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be175b40 0x5566be175f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.328+0000 7fa326000640 1 -- 172.21.15.69:0/3542087396 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566be13e310 con 0x5566be180b30 2024-04-24T09:08:53.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.328+0000 7fa327400640 1 --2- 172.21.15.69:0/3542087396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be175b40 0x5566be175f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:53.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.329+0000 7fa326000640 1 --2- 172.21.15.69:0/3542087396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be180b30 0x5566be182f20 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa314005ab0 tx=0x7fa3140040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8041a17b289ceda8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.329+0000 7fa31f400640 1 --2- 172.21.15.69:0/2774499940 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa30c0a3d50 0x7fa30c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.329+0000 7fa313e00640 1 -- 172.21.15.69:0/3542087396 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31403f070 con 0x5566be180b30 2024-04-24T09:08:53.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.329+0000 7fa313e00640 1 -- 172.21.15.69:0/3542087396 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3140044b0 con 0x5566be180b30 2024-04-24T09:08:53.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.329+0000 7fa313e00640 1 -- 172.21.15.69:0/3542087396 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314035030 con 0x5566be180b30 2024-04-24T09:08:53.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.329+0000 7fa31f400640 1 -- 172.21.15.69:0/2774499940 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa3180d0610 con 0x7fa328103060 2024-04-24T09:08:53.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.329+0000 7fa327400640 1 --2- 172.21.15.69:0/2774499940 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa30c0a3d50 0x7fa30c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.330+0000 7fa3309b7740 1 -- 172.21.15.69:0/3542087396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be180b30 msgr2=0x5566be182f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.330+0000 7fa3309b7740 1 --2- 172.21.15.69:0/3542087396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be180b30 0x5566be182f20 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa314005ab0 tx=0x7fa3140040c0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.330+0000 7fa3309b7740 1 -- 172.21.15.69:0/3542087396 shutdown_connections 2024-04-24T09:08:53.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.330+0000 7fa3309b7740 1 --2- 172.21.15.69:0/3542087396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be180b30 0x5566be182f20 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.330+0000 7fa3309b7740 1 --2- 172.21.15.69:0/3542087396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be176460 0x5566be1805f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.330+0000 7fa3309b7740 1 --2- 172.21.15.69:0/3542087396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be175b40 0x5566be175f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.330+0000 7fa3309b7740 1 -- 172.21.15.69:0/3542087396 >> 172.21.15.69:0/3542087396 conn(0x5566be171e60 msgr2=0x5566be172240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.330+0000 7fa3309b7740 1 -- 172.21.15.69:0/3542087396 shutdown_connections 2024-04-24T09:08:53.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.330+0000 7fa3309b7740 1 -- 172.21.15.69:0/3542087396 wait complete. 2024-04-24T09:08:53.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.330+0000 7fa327400640 1 --2- 172.21.15.69:0/2774499940 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa30c0a3d50 0x7fa30c0a6210 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa320005cc0 tx=0x7fa32000b420 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.331+0000 7fa3309b7740 1 -- start start 2024-04-24T09:08:53.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.331+0000 7fa3309b7740 1 Processor -- start 2024-04-24T09:08:53.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.332+0000 7fa3309b7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be175b40 0x5566be180540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.332+0000 7fa3309b7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be176460 0x5566be17d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.332+0000 7fa3309b7740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be180b30 0x5566be17de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.332+0000 7fa3309b7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5566be183960 con 0x5566be176460 2024-04-24T09:08:53.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.332+0000 7fa3309b7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5566be094810 con 0x5566be175b40 2024-04-24T09:08:53.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.332+0000 7fa3309b7740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5566be183ae0 con 0x5566be180b30 2024-04-24T09:08:53.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.332+0000 7fa3309b7740 10 client.0 _ll_register_callbacks cb 0x5566be062140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:53.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.332+0000 7fa326000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be176460 0x5566be17d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.332+0000 7fa326000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be176460 0x5566be17d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38232/0 (socket says 172.21.15.69:38232) 2024-04-24T09:08:53.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.332+0000 7fa326000640 1 -- 172.21.15.69:0/2164365222 learned_addr learned my addr 172.21.15.69:0/2164365222 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:53.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.332+0000 7fa326a00640 1 --2- 172.21.15.69:0/2164365222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be175b40 0x5566be180540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.332+0000 7fa327400640 1 --2- 172.21.15.69:0/2164365222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be180b30 0x5566be17de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.333+0000 7fa326000640 1 -- 172.21.15.69:0/2164365222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be180b30 msgr2=0x5566be17de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.333+0000 7fa326000640 1 --2- 172.21.15.69:0/2164365222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be180b30 0x5566be17de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.333+0000 7fa326000640 1 -- 172.21.15.69:0/2164365222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be175b40 msgr2=0x5566be180540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.333+0000 7fa326000640 1 --2- 172.21.15.69:0/2164365222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be175b40 0x5566be180540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.333+0000 7fa326000640 1 -- 172.21.15.69:0/2164365222 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566be0a53f0 con 0x5566be176460 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.333+0000 7fa326000640 1 --2- 172.21.15.69:0/2164365222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be176460 0x5566be17d830 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa3140049a0 tx=0x7fa3140049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d81d70bf8ab037 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.333+0000 7fa310c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.333+0000 7fa310c00640 1 -- 172.21.15.69:0/2164365222 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31404c020 con 0x5566be176460 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.333+0000 7fa310c00640 1 -- 172.21.15.69:0/2164365222 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa314003ac0 con 0x5566be176460 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.333+0000 7fa3309b7740 1 -- 172.21.15.69:0/2164365222 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3140044b0 con 0x5566be176460 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.334+0000 7fa310c00640 1 -- 172.21.15.69:0/2164365222 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa314057050 con 0x5566be176460 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.334+0000 7fa3309b7740 10 client.16206 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.334+0000 7fa3309b7740 1 -- 172.21.15.69:0/2164365222 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5566bdded3f0 con 0x5566be176460 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.334+0000 7fa310c00640 1 -- 172.21.15.69:0/2164365222 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3140044b0 con 0x5566be176460 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.334+0000 7fa310c00640 1 -- 172.21.15.69:0/2164365222 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fa3140314a0 con 0x5566be176460 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.334+0000 7fa3309b7740 10 client.16206 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.334+0000 7fa3309b7740 10 client.16206 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.334+0000 7fa3309b7740 20 client.16206 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.335+0000 7fa3309b7740 1 --2- 172.21.15.69:0/2164365222 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x5566be190b10 0x5566be192f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.335+0000 7fa3309b7740 4 client.16206 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:53.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.335+0000 7fa3309b7740 1 -- 172.21.15.69:0/2164365222 --> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566be0e8bf0 con 0x5566be190b10 2024-04-24T09:08:53.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.335+0000 7fa326a00640 1 --2- 172.21.15.69:0/2164365222 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x5566be190b10 0x5566be192f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.336+0000 7fa326a00640 1 --2- 172.21.15.69:0/2164365222 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x5566be190b10 0x5566be192f00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b01af784b47d019 server_cookie=9b03f3b854de4762 in_seq=0 out_seq=0 2024-04-24T09:08:53.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.336+0000 7fa310c00640 10 client.16206 ms_handle_connect on v2:172.21.15.81:6834/3260829578 2024-04-24T09:08:53.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.338+0000 7fa310c00640 1 -- 172.21.15.69:0/2164365222 <== mds.2 v2:172.21.15.81:6834/3260829578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5566be0e8bf0 con 0x5566be190b10 2024-04-24T09:08:53.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.338+0000 7fa310c00640 10 client.16206 handle_command_reply: tid=0 2024-04-24T09:08:53.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.338+0000 7fa3309b7740 1 client.16206 shutdown 2024-04-24T09:08:53.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.338+0000 7fa3309b7740 1 -- 172.21.15.69:0/2164365222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be176460 msgr2=0x5566be17d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.338+0000 7fa3309b7740 1 --2- 172.21.15.69:0/2164365222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be176460 0x5566be17d830 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa3140049a0 tx=0x7fa3140049d0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.339+0000 7fa3309b7740 1 -- 172.21.15.69:0/2164365222 shutdown_connections 2024-04-24T09:08:53.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.339+0000 7fa3309b7740 1 --2- 172.21.15.69:0/2164365222 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x5566be190b10 0x5566be192f00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.339+0000 7fa3309b7740 1 --2- 172.21.15.69:0/2164365222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be180b30 0x5566be17de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.339+0000 7fa3309b7740 1 --2- 172.21.15.69:0/2164365222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be176460 0x5566be17d830 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.339+0000 7fa3309b7740 1 --2- 172.21.15.69:0/2164365222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be175b40 0x5566be180540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.339+0000 7fa310c00640 0 client.16206 ms_handle_reset on v2:172.21.15.81:6834/3260829578 2024-04-24T09:08:53.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.339+0000 7fa3309b7740 1 -- 172.21.15.69:0/2164365222 >> 172.21.15.69:0/2164365222 conn(0x5566be171e60 msgr2=0x5566be177c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.339+0000 7fa3309b7740 1 -- 172.21.15.69:0/2164365222 shutdown_connections 2024-04-24T09:08:53.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.339+0000 7fa3309b7740 1 -- 172.21.15.69:0/2164365222 wait complete. 2024-04-24T09:08:53.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.339+0000 7fa3309b7740 20 client.16206 trim_cache size 0 max 16384 2024-04-24T09:08:53.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.359+0000 7fa3309b7740 1 Processor -- start 2024-04-24T09:08:53.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.359+0000 7fa3309b7740 1 -- start start 2024-04-24T09:08:53.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.359+0000 7fa3309b7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be175b40 0x5566be1782e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.359+0000 7fa3309b7740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be176460 0x5566be170dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.359+0000 7fa3309b7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be180b30 0x5566be180070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.359+0000 7fa3309b7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281b3a30 con 0x5566be180b30 2024-04-24T09:08:53.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.359+0000 7fa3309b7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281b1300 con 0x5566be175b40 2024-04-24T09:08:53.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.359+0000 7fa3309b7740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281af150 con 0x5566be176460 2024-04-24T09:08:53.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.360+0000 7fa327400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be180b30 0x5566be180070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.360+0000 7fa327400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be180b30 0x5566be180070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38238/0 (socket says 172.21.15.69:38238) 2024-04-24T09:08:53.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.360+0000 7fa327400640 1 -- 172.21.15.69:0/3495668232 learned_addr learned my addr 172.21.15.69:0/3495668232 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:53.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.360+0000 7fa326000640 1 --2- 172.21.15.69:0/3495668232 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be176460 0x5566be170dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.360+0000 7fa326a00640 1 --2- 172.21.15.69:0/3495668232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be175b40 0x5566be1782e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.360+0000 7fa327400640 1 -- 172.21.15.69:0/3495668232 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be176460 msgr2=0x5566be170dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.360+0000 7fa327400640 1 --2- 172.21.15.69:0/3495668232 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be176460 0x5566be170dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.360+0000 7fa327400640 1 -- 172.21.15.69:0/3495668232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be175b40 msgr2=0x5566be1782e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.360+0000 7fa327400640 1 --2- 172.21.15.69:0/3495668232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be175b40 0x5566be1782e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.360+0000 7fa327400640 1 -- 172.21.15.69:0/3495668232 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566be176290 con 0x5566be180b30 2024-04-24T09:08:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.360+0000 7fa327400640 1 --2- 172.21.15.69:0/3495668232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be180b30 0x5566be180070 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa320011760 tx=0x7fa320013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8f0a087f7cf581 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.361+0000 7fa313400640 1 -- 172.21.15.69:0/3495668232 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32001e940 con 0x5566be180b30 2024-04-24T09:08:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.361+0000 7fa313400640 1 -- 172.21.15.69:0/3495668232 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa32001f430 con 0x5566be180b30 2024-04-24T09:08:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.361+0000 7fa313400640 1 -- 172.21.15.69:0/3495668232 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa320027630 con 0x5566be180b30 2024-04-24T09:08:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.361+0000 7fa3309b7740 1 -- 172.21.15.69:0/3495668232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be180b30 msgr2=0x5566be180070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.361+0000 7fa3309b7740 1 --2- 172.21.15.69:0/3495668232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be180b30 0x5566be180070 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa320011760 tx=0x7fa320013000 comp rx=0 tx=0).stop 2024-04-24T09:08:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.362+0000 7fa3309b7740 1 -- 172.21.15.69:0/3495668232 shutdown_connections 2024-04-24T09:08:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.362+0000 7fa3309b7740 1 --2- 172.21.15.69:0/3495668232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be180b30 0x5566be180070 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.362+0000 7fa3309b7740 1 --2- 172.21.15.69:0/3495668232 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be176460 0x5566be170dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.362+0000 7fa3309b7740 1 --2- 172.21.15.69:0/3495668232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be175b40 0x5566be1782e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.362+0000 7fa3309b7740 1 -- 172.21.15.69:0/3495668232 >> 172.21.15.69:0/3495668232 conn(0x5566be171e60 msgr2=0x5566be1706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.362+0000 7fa3309b7740 1 -- 172.21.15.69:0/3495668232 shutdown_connections 2024-04-24T09:08:53.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.362+0000 7fa3309b7740 1 -- 172.21.15.69:0/3495668232 wait complete. 2024-04-24T09:08:53.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.362+0000 7fa3309b7740 1 -- start start 2024-04-24T09:08:53.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.362+0000 7fa3309b7740 1 Processor -- start 2024-04-24T09:08:53.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.363+0000 7fa3309b7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be175b40 0x5566be188d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.363+0000 7fa3309b7740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be176460 0x5566be189270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.363+0000 7fa3309b7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be180b30 0x5566be181d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.363+0000 7fa3309b7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281af7c0 con 0x5566be175b40 2024-04-24T09:08:53.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.363+0000 7fa3309b7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281b0c30 con 0x5566be180b30 2024-04-24T09:08:53.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.363+0000 7fa3309b7740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281afe90 con 0x5566be176460 2024-04-24T09:08:53.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.363+0000 7fa3309b7740 10 client.0 _ll_register_callbacks cb 0x7fa314009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:53.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.363+0000 7fa326a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be175b40 0x5566be188d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.363+0000 7fa326a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be175b40 0x5566be188d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38248/0 (socket says 172.21.15.69:38248) 2024-04-24T09:08:53.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.363+0000 7fa326a00640 1 -- 172.21.15.69:0/844438533 learned_addr learned my addr 172.21.15.69:0/844438533 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:53.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.363+0000 7fa326000640 1 --2- 172.21.15.69:0/844438533 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be176460 0x5566be189270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.364+0000 7fa327400640 1 --2- 172.21.15.69:0/844438533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be180b30 0x5566be181d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.364+0000 7fa326a00640 1 -- 172.21.15.69:0/844438533 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be176460 msgr2=0x5566be189270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.364+0000 7fa326a00640 1 --2- 172.21.15.69:0/844438533 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be176460 0x5566be189270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.364+0000 7fa326a00640 1 -- 172.21.15.69:0/844438533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be180b30 msgr2=0x5566be181d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.364+0000 7fa326a00640 1 --2- 172.21.15.69:0/844438533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be180b30 0x5566be181d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.364+0000 7fa326a00640 1 -- 172.21.15.69:0/844438533 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566bdeb77d0 con 0x5566be175b40 2024-04-24T09:08:53.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.364+0000 7fa326000640 1 --2- 172.21.15.69:0/844438533 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be176460 0x5566be189270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:53.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.364+0000 7fa327400640 1 --2- 172.21.15.69:0/844438533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be180b30 0x5566be181d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:53.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.364+0000 7fa326a00640 1 --2- 172.21.15.69:0/844438533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be175b40 0x5566be188d30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa318009610 tx=0x7fa31809bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafecc7bd04d0ac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.364+0000 7fa310c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:53.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.364+0000 7fa310c00640 1 -- 172.21.15.69:0/844438533 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3180cf040 con 0x5566be175b40 2024-04-24T09:08:53.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.364+0000 7fa310c00640 1 -- 172.21.15.69:0/844438533 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa3180cdca0 con 0x5566be175b40 2024-04-24T09:08:53.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.364+0000 7fa3309b7740 1 -- 172.21.15.69:0/844438533 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa32001f430 con 0x5566be175b40 2024-04-24T09:08:53.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.364+0000 7fa310c00640 1 -- 172.21.15.69:0/844438533 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa3180b5050 con 0x5566be175b40 2024-04-24T09:08:53.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.365+0000 7fa3309b7740 10 client.16212 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:53.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.365+0000 7fa3309b7740 1 -- 172.21.15.69:0/844438533 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5566be1f73a0 con 0x5566be175b40 2024-04-24T09:08:53.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.365+0000 7fa310c00640 1 -- 172.21.15.69:0/844438533 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa32001f430 con 0x5566be175b40 2024-04-24T09:08:53.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.365+0000 7fa310c00640 1 -- 172.21.15.69:0/844438533 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fa3180988e0 con 0x5566be175b40 2024-04-24T09:08:53.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.365+0000 7fa3309b7740 10 client.16212 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:53.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.365+0000 7fa3309b7740 10 client.16212 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:08:53.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.365+0000 7fa3309b7740 20 client.16212 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:53.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.365+0000 7fa3309b7740 1 --2- 172.21.15.69:0/844438533 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x5566be1f7a70 0x5566be1f9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.365+0000 7fa3309b7740 4 client.16212 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:53.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.365+0000 7fa3309b7740 1 -- 172.21.15.69:0/844438533 --> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566be0e2a60 con 0x5566be1f7a70 2024-04-24T09:08:53.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.366+0000 7fa326000640 1 --2- 172.21.15.69:0/844438533 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x5566be1f7a70 0x5566be1f9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.366+0000 7fa326000640 1 --2- 172.21.15.69:0/844438533 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x5566be1f7a70 0x5566be1f9e90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf06c20b2d42caf6 server_cookie=fea1521ca7aa6ab5 in_seq=0 out_seq=0 2024-04-24T09:08:53.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.366+0000 7fa310c00640 10 client.16212 ms_handle_connect on v2:172.21.15.81:6834/3260829578 2024-04-24T09:08:53.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.367+0000 7fa310c00640 1 -- 172.21.15.69:0/844438533 <== mds.2 v2:172.21.15.81:6834/3260829578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566be0e2a60 con 0x5566be1f7a70 2024-04-24T09:08:53.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.367+0000 7fa310c00640 10 client.16212 handle_command_reply: tid=0 2024-04-24T09:08:53.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.367+0000 7fa3309b7740 1 client.16212 shutdown 2024-04-24T09:08:53.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.367+0000 7fa3309b7740 1 -- 172.21.15.69:0/844438533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be175b40 msgr2=0x5566be188d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.367+0000 7fa3309b7740 1 --2- 172.21.15.69:0/844438533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be175b40 0x5566be188d30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa318009610 tx=0x7fa31809bea0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.367+0000 7fa3309b7740 1 -- 172.21.15.69:0/844438533 shutdown_connections 2024-04-24T09:08:53.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.367+0000 7fa3309b7740 1 --2- 172.21.15.69:0/844438533 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x5566be1f7a70 0x5566be1f9e90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.367+0000 7fa3309b7740 1 --2- 172.21.15.69:0/844438533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566be180b30 0x5566be181d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.367+0000 7fa3309b7740 1 --2- 172.21.15.69:0/844438533 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566be176460 0x5566be189270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.367+0000 7fa3309b7740 1 --2- 172.21.15.69:0/844438533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566be175b40 0x5566be188d30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.367+0000 7fa3309b7740 1 -- 172.21.15.69:0/844438533 >> 172.21.15.69:0/844438533 conn(0x5566be171e60 msgr2=0x5566be170f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.367+0000 7fa310c00640 0 client.16212 ms_handle_reset on v2:172.21.15.81:6834/3260829578 2024-04-24T09:08:53.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.367+0000 7fa3309b7740 1 -- 172.21.15.69:0/844438533 shutdown_connections 2024-04-24T09:08:53.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.367+0000 7fa3309b7740 1 -- 172.21.15.69:0/844438533 wait complete. 2024-04-24T09:08:53.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.368+0000 7fa3309b7740 20 client.16212 trim_cache size 0 max 16384 2024-04-24T09:08:53.371 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-04-24T09:08:53.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.368+0000 7fa327e00640 1 -- 172.21.15.69:0/2774499940 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa30c0a3d50 msgr2=0x7fa30c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.368+0000 7fa327e00640 1 --2- 172.21.15.69:0/2774499940 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa30c0a3d50 0x7fa30c0a6210 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa320005cc0 tx=0x7fa32000b420 comp rx=0 tx=0).stop 2024-04-24T09:08:53.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.368+0000 7fa327e00640 1 -- 172.21.15.69:0/2774499940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328103060 msgr2=0x7fa328100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.368+0000 7fa327e00640 1 --2- 172.21.15.69:0/2774499940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328103060 0x7fa328100cb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa31800d770 tx=0x7fa31800dc30 comp rx=0 tx=0).stop 2024-04-24T09:08:53.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.368+0000 7fa327e00640 1 -- 172.21.15.69:0/2774499940 shutdown_connections 2024-04-24T09:08:53.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.368+0000 7fa327e00640 1 --2- 172.21.15.69:0/2774499940 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa30c0a3d50 0x7fa30c0a6210 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.368+0000 7fa327e00640 1 --2- 172.21.15.69:0/2774499940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa32810c480 0x7fa328101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.368+0000 7fa327e00640 1 --2- 172.21.15.69:0/2774499940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa328103980 0x7fa3281011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.368+0000 7fa327e00640 1 --2- 172.21.15.69:0/2774499940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328103060 0x7fa328100cb0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.368+0000 7fa327e00640 1 -- 172.21.15.69:0/2774499940 >> 172.21.15.69:0/2774499940 conn(0x7fa3280fc880 msgr2=0x7fa3280fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.369+0000 7fa327e00640 1 -- 172.21.15.69:0/2774499940 shutdown_connections 2024-04-24T09:08:53.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.369+0000 7fa327e00640 1 -- 172.21.15.69:0/2774499940 wait complete. 2024-04-24T09:08:53.385 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:53 smithi069 ceph-mon[28170]: pgmap v57: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-04-24T09:08:53.385 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:53 smithi069 ceph-mon[28170]: scrub queued for path: / 2024-04-24T09:08:53.385 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:53 smithi069 ceph-mon[28170]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:53.385 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:53 smithi069 ceph-mon[28170]: scrub summary: active paths [/] 2024-04-24T09:08:53.385 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:53 smithi069 ceph-mon[28170]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:53.386 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:53 smithi069 ceph-mon[28170]: scrub summary: idle 2024-04-24T09:08:53.386 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:53 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/57762720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:08:53.388 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:08:53.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:53 smithi130 ceph-mon[31553]: pgmap v57: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-04-24T09:08:53.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:53 smithi130 ceph-mon[31553]: scrub queued for path: / 2024-04-24T09:08:53.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:53 smithi130 ceph-mon[31553]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:53.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:53 smithi130 ceph-mon[31553]: scrub summary: active paths [/] 2024-04-24T09:08:53.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:53 smithi130 ceph-mon[31553]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:53.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:53 smithi130 ceph-mon[31553]: scrub summary: idle 2024-04-24T09:08:53.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:53 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/57762720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:08:53.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:53 smithi081 ceph-mon[30961]: pgmap v57: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-04-24T09:08:53.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:53 smithi081 ceph-mon[30961]: scrub queued for path: / 2024-04-24T09:08:53.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:53 smithi081 ceph-mon[30961]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:53.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:53 smithi081 ceph-mon[30961]: scrub summary: active paths [/] 2024-04-24T09:08:53.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:53 smithi081 ceph-mon[30961]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:53.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:53 smithi081 ceph-mon[30961]: scrub summary: idle 2024-04-24T09:08:53.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:53 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/57762720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:08:53.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.601+0000 7f7f62400640 1 -- 172.21.15.69:0/1815071634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c102110 msgr2=0x7f7f5c1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.601+0000 7f7f62400640 1 --2- 172.21.15.69:0/1815071634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c102110 0x7f7f5c1024f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009900 tx=0x7f7f5002a410 comp rx=0 tx=0).stop 2024-04-24T09:08:53.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.601+0000 7f7f62400640 1 -- 172.21.15.69:0/1815071634 shutdown_connections 2024-04-24T09:08:53.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.601+0000 7f7f62400640 1 --2- 172.21.15.69:0/1815071634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f5c1071e0 0x7f7f5c10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.601+0000 7f7f62400640 1 --2- 172.21.15.69:0/1815071634 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f5c102ac0 0x7f7f5c106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.601+0000 7f7f62400640 1 --2- 172.21.15.69:0/1815071634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c102110 0x7f7f5c1024f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.601+0000 7f7f62400640 1 -- 172.21.15.69:0/1815071634 >> 172.21.15.69:0/1815071634 conn(0x7f7f5c0fd9d0 msgr2=0x7f7f5c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.601+0000 7f7f62400640 1 -- 172.21.15.69:0/1815071634 shutdown_connections 2024-04-24T09:08:53.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.602+0000 7f7f62400640 1 -- 172.21.15.69:0/1815071634 wait complete. 2024-04-24T09:08:53.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.603+0000 7f7f62400640 1 Processor -- start 2024-04-24T09:08:53.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.603+0000 7f7f62400640 1 -- start start 2024-04-24T09:08:53.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.603+0000 7f7f62400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f5c102110 0x7f7f5c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.603+0000 7f7f62400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f5c1071e0 0x7f7f5c1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.603+0000 7f7f62400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c19a780 0x7f7f5c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.603+0000 7f7f62400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1121c0 con 0x7f7f5c19a780 2024-04-24T09:08:53.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.603+0000 7f7f62400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c112040 con 0x7f7f5c102110 2024-04-24T09:08:53.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.603+0000 7f7f62400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c112340 con 0x7f7f5c1071e0 2024-04-24T09:08:53.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.604+0000 7f7f61a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c19a780 0x7f7f5c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.604+0000 7f7f61000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f5c102110 0x7f7f5c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.604+0000 7f7f61a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c19a780 0x7f7f5c19ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38266/0 (socket says 172.21.15.69:38266) 2024-04-24T09:08:53.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.604+0000 7f7f61a00640 1 -- 172.21.15.69:0/2704932197 learned_addr learned my addr 172.21.15.69:0/2704932197 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:53.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.604+0000 7f7f5be00640 1 --2- 172.21.15.69:0/2704932197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f5c1071e0 0x7f7f5c1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.605+0000 7f7f61a00640 1 -- 172.21.15.69:0/2704932197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f5c1071e0 msgr2=0x7f7f5c1a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.605+0000 7f7f61a00640 1 --2- 172.21.15.69:0/2704932197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f5c1071e0 0x7f7f5c1a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.605+0000 7f7f61a00640 1 -- 172.21.15.69:0/2704932197 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f5c102110 msgr2=0x7f7f5c1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.605+0000 7f7f61a00640 1 --2- 172.21.15.69:0/2704932197 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f5c102110 0x7f7f5c1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.605+0000 7f7f61a00640 1 -- 172.21.15.69:0/2704932197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5c19b310 con 0x7f7f5c19a780 2024-04-24T09:08:53.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.605+0000 7f7f61a00640 1 --2- 172.21.15.69:0/2704932197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c19a780 0x7f7f5c19ac30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400b970 tx=0x7f7f5400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.605+0000 7f7f59600640 1 -- 172.21.15.69:0/2704932197 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f54019070 con 0x7f7f5c19a780 2024-04-24T09:08:53.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.605+0000 7f7f59600640 1 -- 172.21.15.69:0/2704932197 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f54014440 con 0x7f7f5c19a780 2024-04-24T09:08:53.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.605+0000 7f7f62400640 1 -- 172.21.15.69:0/2704932197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5c19b600 con 0x7f7f5c19a780 2024-04-24T09:08:53.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.605+0000 7f7f59600640 1 -- 172.21.15.69:0/2704932197 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f540075d0 con 0x7f7f5c19a780 2024-04-24T09:08:53.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.605+0000 7f7f62400640 1 -- 172.21.15.69:0/2704932197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f5c1a77e0 con 0x7f7f5c19a780 2024-04-24T09:08:53.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.610+0000 7f7f667af740 1 Processor -- start 2024-04-24T09:08:53.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.610+0000 7f7f59600640 1 -- 172.21.15.69:0/2704932197 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7f5401c070 con 0x7f7f5c19a780 2024-04-24T09:08:53.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.610+0000 7f7f667af740 1 -- start start 2024-04-24T09:08:53.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.611+0000 7f7f667af740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008557b40 0x563008557f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.611+0000 7f7f667af740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008558460 0x5630085625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.611+0000 7f7f667af740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008562b30 0x563008564f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.611+0000 7f7f667af740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5630084bd1d0 con 0x563008562b30 2024-04-24T09:08:53.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.611+0000 7f7f667af740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563008463450 con 0x563008558460 2024-04-24T09:08:53.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.611+0000 7f7f667af740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5630084b9190 con 0x563008557b40 2024-04-24T09:08:53.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.611+0000 7f7f5be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008562b30 0x563008564f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.611+0000 7f7f5be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008562b30 0x563008564f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38274/0 (socket says 172.21.15.69:38274) 2024-04-24T09:08:53.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.611+0000 7f7f5be00640 1 -- 172.21.15.69:0/2995040590 learned_addr learned my addr 172.21.15.69:0/2995040590 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:53.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.611+0000 7f7f61000640 1 --2- 172.21.15.69:0/2995040590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008558460 0x5630085625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.612+0000 7f7f5be00640 1 --2- 172.21.15.69:0/2995040590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008557b40 0x563008557f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.612+0000 7f7f5be00640 1 -- 172.21.15.69:0/2995040590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008557b40 msgr2=0x563008557f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.612+0000 7f7f5be00640 1 --2- 172.21.15.69:0/2995040590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008557b40 0x563008557f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.612+0000 7f7f5be00640 1 -- 172.21.15.69:0/2995040590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008558460 msgr2=0x5630085625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.612+0000 7f7f5be00640 1 --2- 172.21.15.69:0/2995040590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008558460 0x5630085625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.612+0000 7f7f5be00640 1 -- 172.21.15.69:0/2995040590 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563008520310 con 0x563008562b30 2024-04-24T09:08:53.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.612+0000 7f7f59600640 1 --2- 172.21.15.69:0/2704932197 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7f440a3da0 0x7f7f440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.612+0000 7f7f5be00640 1 --2- 172.21.15.69:0/2995040590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008562b30 0x563008564f20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7f48011470 tx=0x7f7f4800a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c46a35e5210bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.612+0000 7f7f59600640 1 -- 172.21.15.69:0/2704932197 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7f540cffa0 con 0x7f7f5c19a780 2024-04-24T09:08:53.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.612+0000 7f7f61000640 1 --2- 172.21.15.69:0/2704932197 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7f440a3da0 0x7f7f440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.612+0000 7f7f4e000640 1 -- 172.21.15.69:0/2995040590 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48004220 con 0x563008562b30 2024-04-24T09:08:53.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.613+0000 7f7f4e000640 1 -- 172.21.15.69:0/2995040590 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f480043c0 con 0x563008562b30 2024-04-24T09:08:53.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.613+0000 7f7f4e000640 1 -- 172.21.15.69:0/2995040590 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48004e50 con 0x563008562b30 2024-04-24T09:08:53.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.613+0000 7f7f667af740 1 -- 172.21.15.69:0/2995040590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008562b30 msgr2=0x563008564f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.613+0000 7f7f61000640 1 --2- 172.21.15.69:0/2704932197 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7f440a3da0 0x7f7f440a6260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7f50031440 tx=0x7f7f50005e80 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.613+0000 7f7f667af740 1 --2- 172.21.15.69:0/2995040590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008562b30 0x563008564f20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7f48011470 tx=0x7f7f4800a6f0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.613+0000 7f7f667af740 1 -- 172.21.15.69:0/2995040590 shutdown_connections 2024-04-24T09:08:53.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.613+0000 7f7f667af740 1 --2- 172.21.15.69:0/2995040590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008562b30 0x563008564f20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.613+0000 7f7f667af740 1 --2- 172.21.15.69:0/2995040590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008558460 0x5630085625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.613+0000 7f7f667af740 1 --2- 172.21.15.69:0/2995040590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008557b40 0x563008557f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.613+0000 7f7f667af740 1 -- 172.21.15.69:0/2995040590 >> 172.21.15.69:0/2995040590 conn(0x563008553e60 msgr2=0x563008554240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.614+0000 7f7f667af740 1 -- 172.21.15.69:0/2995040590 shutdown_connections 2024-04-24T09:08:53.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.614+0000 7f7f667af740 1 -- 172.21.15.69:0/2995040590 wait complete. 2024-04-24T09:08:53.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.614+0000 7f7f667af740 1 -- start start 2024-04-24T09:08:53.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.614+0000 7f7f667af740 1 Processor -- start 2024-04-24T09:08:53.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.615+0000 7f7f667af740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008557b40 0x56300855f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.615+0000 7f7f667af740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008558460 0x56300855fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.615+0000 7f7f667af740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008562b30 0x563008560390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.615+0000 7f7f667af740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5630085659c0 con 0x563008558460 2024-04-24T09:08:53.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.615+0000 7f7f667af740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563008476810 con 0x563008562b30 2024-04-24T09:08:53.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.615+0000 7f7f667af740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563008565b40 con 0x563008557b40 2024-04-24T09:08:53.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.615+0000 7f7f667af740 10 client.0 _ll_register_callbacks cb 0x563008444140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:53.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.615+0000 7f7f5be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008558460 0x56300855fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.615+0000 7f7f5be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008558460 0x56300855fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38288/0 (socket says 172.21.15.69:38288) 2024-04-24T09:08:53.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.615+0000 7f7f5be00640 1 -- 172.21.15.69:0/892661506 learned_addr learned my addr 172.21.15.69:0/892661506 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:53.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.615+0000 7f7f61000640 1 --2- 172.21.15.69:0/892661506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008557b40 0x56300855f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.615+0000 7f7f5be00640 1 -- 172.21.15.69:0/892661506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008557b40 msgr2=0x56300855f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.615+0000 7f7f61a00640 1 --2- 172.21.15.69:0/892661506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008562b30 0x563008560390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.615+0000 7f7f5be00640 1 --2- 172.21.15.69:0/892661506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008557b40 0x56300855f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.615+0000 7f7f5be00640 1 -- 172.21.15.69:0/892661506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008562b30 msgr2=0x563008560390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.615+0000 7f7f5be00640 1 --2- 172.21.15.69:0/892661506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008562b30 0x563008560390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.616+0000 7f7f5be00640 1 -- 172.21.15.69:0/892661506 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630084873f0 con 0x563008558460 2024-04-24T09:08:53.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.616+0000 7f7f61000640 1 --2- 172.21.15.69:0/892661506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008557b40 0x56300855f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:53.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.616+0000 7f7f5be00640 1 --2- 172.21.15.69:0/892661506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008558460 0x56300855fe50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7f480058c0 tx=0x7f7f4800a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6799563469717b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.616+0000 7f7f42a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:53.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.616+0000 7f7f42a00640 1 -- 172.21.15.69:0/892661506 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4800f690 con 0x563008558460 2024-04-24T09:08:53.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.616+0000 7f7f667af740 1 -- 172.21.15.69:0/892661506 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f480043c0 con 0x563008558460 2024-04-24T09:08:53.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.616+0000 7f7f42a00640 1 -- 172.21.15.69:0/892661506 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7f4800f830 con 0x563008558460 2024-04-24T09:08:53.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.616+0000 7f7f42a00640 1 -- 172.21.15.69:0/892661506 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7f4800fa30 con 0x563008558460 2024-04-24T09:08:53.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.616+0000 7f7f667af740 10 client.16224 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:53.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.616+0000 7f7f667af740 1 -- 172.21.15.69:0/892661506 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5630081cf3f0 con 0x563008558460 2024-04-24T09:08:53.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.617+0000 7f7f42a00640 1 -- 172.21.15.69:0/892661506 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f480043c0 con 0x563008558460 2024-04-24T09:08:53.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.617+0000 7f7f42a00640 1 -- 172.21.15.69:0/892661506 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7f4801b030 con 0x563008558460 2024-04-24T09:08:53.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.617+0000 7f7f667af740 10 client.16224 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:53.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.617+0000 7f7f667af740 10 client.16224 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:08:53.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.617+0000 7f7f667af740 20 client.16224 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:53.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.617+0000 7f7f667af740 1 --2- 172.21.15.69:0/892661506 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x563008572c90 0x563008575080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.617+0000 7f7f667af740 4 client.16224 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:53.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.617+0000 7f7f667af740 1 -- 172.21.15.69:0/892661506 --> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630084cabf0 con 0x563008572c90 2024-04-24T09:08:53.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.618+0000 7f7f61000640 1 --2- 172.21.15.69:0/892661506 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x563008572c90 0x563008575080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.619+0000 7f7f61000640 1 --2- 172.21.15.69:0/892661506 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x563008572c90 0x563008575080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e38762398c698ca5 server_cookie=6ae8aab5f69e3084 in_seq=0 out_seq=0 2024-04-24T09:08:53.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.619+0000 7f7f42a00640 10 client.16224 ms_handle_connect on v2:172.21.15.81:6836/1973819593 2024-04-24T09:08:53.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.621+0000 7f7f42a00640 1 -- 172.21.15.69:0/892661506 <== mds.1 v2:172.21.15.81:6836/1973819593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630084cabf0 con 0x563008572c90 2024-04-24T09:08:53.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.621+0000 7f7f42a00640 10 client.16224 handle_command_reply: tid=0 2024-04-24T09:08:53.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.621+0000 7f7f667af740 1 client.16224 shutdown 2024-04-24T09:08:53.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.621+0000 7f7f667af740 1 -- 172.21.15.69:0/892661506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008558460 msgr2=0x56300855fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.621+0000 7f7f667af740 1 --2- 172.21.15.69:0/892661506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008558460 0x56300855fe50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7f480058c0 tx=0x7f7f4800a4c0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.621+0000 7f7f667af740 1 -- 172.21.15.69:0/892661506 shutdown_connections 2024-04-24T09:08:53.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.621+0000 7f7f667af740 1 --2- 172.21.15.69:0/892661506 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x563008572c90 0x563008575080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.621+0000 7f7f667af740 1 --2- 172.21.15.69:0/892661506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008562b30 0x563008560390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.621+0000 7f7f667af740 1 --2- 172.21.15.69:0/892661506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008558460 0x56300855fe50 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.621+0000 7f7f667af740 1 --2- 172.21.15.69:0/892661506 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008557b40 0x56300855f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.621+0000 7f7f42a00640 0 client.16224 ms_handle_reset on v2:172.21.15.81:6836/1973819593 2024-04-24T09:08:53.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.621+0000 7f7f667af740 1 -- 172.21.15.69:0/892661506 >> 172.21.15.69:0/892661506 conn(0x563008553e60 msgr2=0x563008559c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.621+0000 7f7f667af740 1 -- 172.21.15.69:0/892661506 shutdown_connections 2024-04-24T09:08:53.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.621+0000 7f7f667af740 1 -- 172.21.15.69:0/892661506 wait complete. 2024-04-24T09:08:53.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.622+0000 7f7f667af740 20 client.16224 trim_cache size 0 max 16384 2024-04-24T09:08:53.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.636+0000 7f7f667af740 1 Processor -- start 2024-04-24T09:08:53.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.636+0000 7f7f667af740 1 -- start start 2024-04-24T09:08:53.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.637+0000 7f7f667af740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008557b40 0x56300856e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.637+0000 7f7f667af740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008558460 0x56300855e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.637+0000 7f7f667af740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008562b30 0x56300857a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.637+0000 7f7f667af740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1b3b60 con 0x563008557b40 2024-04-24T09:08:53.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.637+0000 7f7f667af740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1b1430 con 0x563008558460 2024-04-24T09:08:53.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.637+0000 7f7f667af740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1af280 con 0x563008562b30 2024-04-24T09:08:53.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.637+0000 7f7f61000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008557b40 0x56300856e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.637+0000 7f7f61000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008557b40 0x56300856e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38294/0 (socket says 172.21.15.69:38294) 2024-04-24T09:08:53.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.637+0000 7f7f61000640 1 -- 172.21.15.69:0/1160876073 learned_addr learned my addr 172.21.15.69:0/1160876073 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:53.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.637+0000 7f7f5be00640 1 --2- 172.21.15.69:0/1160876073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008558460 0x56300855e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.637+0000 7f7f61000640 1 -- 172.21.15.69:0/1160876073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008562b30 msgr2=0x56300857a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.637+0000 7f7f61a00640 1 --2- 172.21.15.69:0/1160876073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008562b30 0x56300857a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.637+0000 7f7f61000640 1 --2- 172.21.15.69:0/1160876073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008562b30 0x56300857a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.637+0000 7f7f61000640 1 -- 172.21.15.69:0/1160876073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008558460 msgr2=0x56300855e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.637+0000 7f7f61000640 1 --2- 172.21.15.69:0/1160876073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008558460 0x56300855e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.637+0000 7f7f61000640 1 -- 172.21.15.69:0/1160876073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630082997d0 con 0x563008557b40 2024-04-24T09:08:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.638+0000 7f7f5be00640 1 --2- 172.21.15.69:0/1160876073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008558460 0x56300855e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.638+0000 7f7f61a00640 1 --2- 172.21.15.69:0/1160876073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008562b30 0x56300857a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.638+0000 7f7f61000640 1 --2- 172.21.15.69:0/1160876073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008557b40 0x56300856e880 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7f50005c00 tx=0x7f7f5003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eba33f14a695222 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.638+0000 7f7f4d600640 1 -- 172.21.15.69:0/1160876073 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50057070 con 0x563008557b40 2024-04-24T09:08:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.638+0000 7f7f4d600640 1 -- 172.21.15.69:0/1160876073 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f50051cd0 con 0x563008557b40 2024-04-24T09:08:53.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.638+0000 7f7f4d600640 1 -- 172.21.15.69:0/1160876073 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50035040 con 0x563008557b40 2024-04-24T09:08:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.639+0000 7f7f667af740 1 -- 172.21.15.69:0/1160876073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008557b40 msgr2=0x56300856e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.639+0000 7f7f667af740 1 --2- 172.21.15.69:0/1160876073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008557b40 0x56300856e880 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7f50005c00 tx=0x7f7f5003f000 comp rx=0 tx=0).stop 2024-04-24T09:08:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.639+0000 7f7f667af740 1 -- 172.21.15.69:0/1160876073 shutdown_connections 2024-04-24T09:08:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.639+0000 7f7f667af740 1 --2- 172.21.15.69:0/1160876073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008562b30 0x56300857a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.639+0000 7f7f667af740 1 --2- 172.21.15.69:0/1160876073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008558460 0x56300855e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.639+0000 7f7f667af740 1 --2- 172.21.15.69:0/1160876073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008557b40 0x56300856e880 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.639+0000 7f7f667af740 1 -- 172.21.15.69:0/1160876073 >> 172.21.15.69:0/1160876073 conn(0x563008553e60 msgr2=0x56300855e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.639+0000 7f7f667af740 1 -- 172.21.15.69:0/1160876073 shutdown_connections 2024-04-24T09:08:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.639+0000 7f7f667af740 1 -- 172.21.15.69:0/1160876073 wait complete. 2024-04-24T09:08:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.640+0000 7f7f667af740 1 -- start start 2024-04-24T09:08:53.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.640+0000 7f7f667af740 1 Processor -- start 2024-04-24T09:08:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.640+0000 7f7f667af740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008557b40 0x5630085647a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.640+0000 7f7f667af740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008558460 0x563008562f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.640+0000 7f7f667af740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008562b30 0x563008563450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.640+0000 7f7f667af740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1af8f0 con 0x563008558460 2024-04-24T09:08:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.640+0000 7f7f667af740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1b0d60 con 0x563008557b40 2024-04-24T09:08:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.640+0000 7f7f667af740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1affc0 con 0x563008562b30 2024-04-24T09:08:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.640+0000 7f7f667af740 10 client.0 _ll_register_callbacks cb 0x7f7f50009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.641+0000 7f7f5be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008558460 0x563008562f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.641+0000 7f7f5be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008558460 0x563008562f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38304/0 (socket says 172.21.15.69:38304) 2024-04-24T09:08:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.641+0000 7f7f5be00640 1 -- 172.21.15.69:0/1313436847 learned_addr learned my addr 172.21.15.69:0/1313436847 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.641+0000 7f7f61000640 1 --2- 172.21.15.69:0/1313436847 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008557b40 0x5630085647a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.641+0000 7f7f61a00640 1 --2- 172.21.15.69:0/1313436847 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008562b30 0x563008563450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.641+0000 7f7f5be00640 1 -- 172.21.15.69:0/1313436847 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008562b30 msgr2=0x563008563450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.641+0000 7f7f5be00640 1 --2- 172.21.15.69:0/1313436847 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008562b30 0x563008563450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.641+0000 7f7f5be00640 1 -- 172.21.15.69:0/1313436847 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008557b40 msgr2=0x5630085647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.641+0000 7f7f5be00640 1 --2- 172.21.15.69:0/1313436847 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008557b40 0x5630085647a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.641+0000 7f7f5be00640 1 -- 172.21.15.69:0/1313436847 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563008563c30 con 0x563008558460 2024-04-24T09:08:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.641+0000 7f7f61a00640 1 --2- 172.21.15.69:0/1313436847 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008562b30 0x563008563450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.641+0000 7f7f5be00640 1 --2- 172.21.15.69:0/1313436847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008558460 0x563008562f10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x563008560ee0 tx=0x7f7f48003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232f48165d8ebfbf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.641+0000 7f7f42a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.642+0000 7f7f42a00640 1 -- 172.21.15.69:0/1313436847 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4800a640 con 0x563008558460 2024-04-24T09:08:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.642+0000 7f7f42a00640 1 -- 172.21.15.69:0/1313436847 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7f4800fbd0 con 0x563008558460 2024-04-24T09:08:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.642+0000 7f7f667af740 1 -- 172.21.15.69:0/1313436847 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f50051cd0 con 0x563008558460 2024-04-24T09:08:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.642+0000 7f7f42a00640 1 -- 172.21.15.69:0/1313436847 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7f48039050 con 0x563008558460 2024-04-24T09:08:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.642+0000 7f7f667af740 10 client.16230 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.642+0000 7f7f667af740 1 -- 172.21.15.69:0/1313436847 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56300856ad10 con 0x563008558460 2024-04-24T09:08:53.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.642+0000 7f7f42a00640 1 -- 172.21.15.69:0/1313436847 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f50051cd0 con 0x563008558460 2024-04-24T09:08:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.643+0000 7f7f42a00640 1 -- 172.21.15.69:0/1313436847 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7f4801b040 con 0x563008558460 2024-04-24T09:08:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.643+0000 7f7f667af740 10 client.16230 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.643+0000 7f7f667af740 10 client.16230 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:08:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.643+0000 7f7f667af740 20 client.16230 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.643+0000 7f7f667af740 1 --2- 172.21.15.69:0/1313436847 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5630085d90a0 0x5630085db490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.643+0000 7f7f667af740 4 client.16230 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.643+0000 7f7f667af740 1 -- 172.21.15.69:0/1313436847 --> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630084c4a60 con 0x5630085d90a0 2024-04-24T09:08:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.643+0000 7f7f61000640 1 --2- 172.21.15.69:0/1313436847 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5630085d90a0 0x5630085db490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.644+0000 7f7f42a00640 10 client.16230 ms_handle_connect on v2:172.21.15.81:6836/1973819593 2024-04-24T09:08:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.644+0000 7f7f61000640 1 --2- 172.21.15.69:0/1313436847 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5630085d90a0 0x5630085db490 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=edd484807f3fd4bc server_cookie=fc981569d6ef43e0 in_seq=0 out_seq=0 2024-04-24T09:08:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.645+0000 7f7f42a00640 1 -- 172.21.15.69:0/1313436847 <== mds.1 v2:172.21.15.81:6836/1973819593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630084c4a60 con 0x5630085d90a0 2024-04-24T09:08:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.645+0000 7f7f42a00640 10 client.16230 handle_command_reply: tid=0 2024-04-24T09:08:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.645+0000 7f7f667af740 1 client.16230 shutdown 2024-04-24T09:08:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.645+0000 7f7f667af740 1 -- 172.21.15.69:0/1313436847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008558460 msgr2=0x563008562f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.645+0000 7f7f667af740 1 --2- 172.21.15.69:0/1313436847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008558460 0x563008562f10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x563008560ee0 tx=0x7f7f48003820 comp rx=0 tx=0).stop 2024-04-24T09:08:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.645+0000 7f7f667af740 1 -- 172.21.15.69:0/1313436847 shutdown_connections 2024-04-24T09:08:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.645+0000 7f7f667af740 1 --2- 172.21.15.69:0/1313436847 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5630085d90a0 0x5630085db490 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.645+0000 7f7f667af740 1 --2- 172.21.15.69:0/1313436847 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563008562b30 0x563008563450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.645+0000 7f7f42a00640 0 client.16230 ms_handle_reset on v2:172.21.15.81:6836/1973819593 2024-04-24T09:08:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.645+0000 7f7f667af740 1 --2- 172.21.15.69:0/1313436847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563008558460 0x563008562f10 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.645+0000 7f7f667af740 1 --2- 172.21.15.69:0/1313436847 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563008557b40 0x5630085647a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.645+0000 7f7f667af740 1 -- 172.21.15.69:0/1313436847 >> 172.21.15.69:0/1313436847 conn(0x563008553e60 msgr2=0x5630085526c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.646+0000 7f7f667af740 1 -- 172.21.15.69:0/1313436847 shutdown_connections 2024-04-24T09:08:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.646+0000 7f7f667af740 1 -- 172.21.15.69:0/1313436847 wait complete. 2024-04-24T09:08:53.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.646+0000 7f7f667af740 20 client.16230 trim_cache size 0 max 16384 2024-04-24T09:08:53.648 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-04-24T09:08:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.646+0000 7f7f62400640 1 -- 172.21.15.69:0/2704932197 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7f440a3da0 msgr2=0x7f7f440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.646+0000 7f7f62400640 1 --2- 172.21.15.69:0/2704932197 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7f440a3da0 0x7f7f440a6260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7f50031440 tx=0x7f7f50005e80 comp rx=0 tx=0).stop 2024-04-24T09:08:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.646+0000 7f7f62400640 1 -- 172.21.15.69:0/2704932197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c19a780 msgr2=0x7f7f5c19ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.646+0000 7f7f62400640 1 --2- 172.21.15.69:0/2704932197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c19a780 0x7f7f5c19ac30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400b970 tx=0x7f7f5400be30 comp rx=0 tx=0).stop 2024-04-24T09:08:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.647+0000 7f7f62400640 1 -- 172.21.15.69:0/2704932197 shutdown_connections 2024-04-24T09:08:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.647+0000 7f7f62400640 1 --2- 172.21.15.69:0/2704932197 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7f440a3da0 0x7f7f440a6260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.647+0000 7f7f62400640 1 --2- 172.21.15.69:0/2704932197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f5c19a780 0x7f7f5c19ac30 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.647+0000 7f7f62400640 1 --2- 172.21.15.69:0/2704932197 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7f5c1071e0 0x7f7f5c1a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.647+0000 7f7f62400640 1 --2- 172.21.15.69:0/2704932197 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f5c102110 0x7f7f5c1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.647+0000 7f7f62400640 1 -- 172.21.15.69:0/2704932197 >> 172.21.15.69:0/2704932197 conn(0x7f7f5c0fd9d0 msgr2=0x7f7f5c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.647+0000 7f7f62400640 1 -- 172.21.15.69:0/2704932197 shutdown_connections 2024-04-24T09:08:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.647+0000 7f7f62400640 1 -- 172.21.15.69:0/2704932197 wait complete. 2024-04-24T09:08:53.665 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:08:53.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.873+0000 7f6c34400640 1 -- 172.21.15.69:0/427186501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c2c1044b0 msgr2=0x7f6c2c104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.873+0000 7f6c34400640 1 --2- 172.21.15.69:0/427186501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c2c1044b0 0x7f6c2c104890 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009900 tx=0x7f6c2002a410 comp rx=0 tx=0).stop 2024-04-24T09:08:53.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.873+0000 7f6c34400640 1 -- 172.21.15.69:0/427186501 shutdown_connections 2024-04-24T09:08:53.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.873+0000 7f6c34400640 1 --2- 172.21.15.69:0/427186501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c2c109580 0x7f6c2c111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.873+0000 7f6c34400640 1 --2- 172.21.15.69:0/427186501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c2c104e60 0x7f6c2c108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.873+0000 7f6c34400640 1 --2- 172.21.15.69:0/427186501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c2c1044b0 0x7f6c2c104890 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.873+0000 7f6c34400640 1 -- 172.21.15.69:0/427186501 >> 172.21.15.69:0/427186501 conn(0x7f6c2c0ffd90 msgr2=0x7f6c2c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.873+0000 7f6c34400640 1 -- 172.21.15.69:0/427186501 shutdown_connections 2024-04-24T09:08:53.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.873+0000 7f6c34400640 1 -- 172.21.15.69:0/427186501 wait complete. 2024-04-24T09:08:53.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.874+0000 7f6c34400640 1 Processor -- start 2024-04-24T09:08:53.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.874+0000 7f6c34400640 1 -- start start 2024-04-24T09:08:53.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c34400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c2c1044b0 0x7f6c2c1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c34400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c2c109580 0x7f6c2c1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c34400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c2c19a930 0x7f6c2c19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c34400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c114530 con 0x7f6c2c1044b0 2024-04-24T09:08:53.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c34400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1143b0 con 0x7f6c2c109580 2024-04-24T09:08:53.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c34400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1146b0 con 0x7f6c2c19a930 2024-04-24T09:08:53.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c31a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c2c1044b0 0x7f6c2c1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c31a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c2c1044b0 0x7f6c2c1a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38316/0 (socket says 172.21.15.69:38316) 2024-04-24T09:08:53.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c31a00640 1 -- 172.21.15.69:0/460126812 learned_addr learned my addr 172.21.15.69:0/460126812 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:53.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c31000640 1 --2- 172.21.15.69:0/460126812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c2c109580 0x7f6c2c1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c32400640 1 --2- 172.21.15.69:0/460126812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c2c19a930 0x7f6c2c19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c31a00640 1 -- 172.21.15.69:0/460126812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c2c19a930 msgr2=0x7f6c2c19ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c31a00640 1 --2- 172.21.15.69:0/460126812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c2c19a930 0x7f6c2c19ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c31a00640 1 -- 172.21.15.69:0/460126812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c2c109580 msgr2=0x7f6c2c1a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c31a00640 1 --2- 172.21.15.69:0/460126812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c2c109580 0x7f6c2c1a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c31a00640 1 -- 172.21.15.69:0/460126812 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c19b450 con 0x7f6c2c1044b0 2024-04-24T09:08:53.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.875+0000 7f6c32400640 1 --2- 172.21.15.69:0/460126812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c2c19a930 0x7f6c2c19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:53.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.876+0000 7f6c31a00640 1 --2- 172.21.15.69:0/460126812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c2c1044b0 0x7f6c2c1a0780 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6c20005ab0 tx=0x7f6c20003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.876+0000 7f6c2a000640 1 -- 172.21.15.69:0/460126812 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2003f070 con 0x7f6c2c1044b0 2024-04-24T09:08:53.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.876+0000 7f6c2a000640 1 -- 172.21.15.69:0/460126812 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c20031d60 con 0x7f6c2c1044b0 2024-04-24T09:08:53.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.876+0000 7f6c34400640 1 -- 172.21.15.69:0/460126812 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2c19b6e0 con 0x7f6c2c1044b0 2024-04-24T09:08:53.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.876+0000 7f6c2a000640 1 -- 172.21.15.69:0/460126812 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20035030 con 0x7f6c2c1044b0 2024-04-24T09:08:53.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.876+0000 7f6c34400640 1 -- 172.21.15.69:0/460126812 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2c1a7930 con 0x7f6c2c1044b0 2024-04-24T09:08:53.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.880+0000 7f6c2a000640 1 -- 172.21.15.69:0/460126812 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6c20042070 con 0x7f6c2c1044b0 2024-04-24T09:08:53.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.880+0000 7f6c373be740 1 Processor -- start 2024-04-24T09:08:53.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.880+0000 7f6c373be740 1 -- start start 2024-04-24T09:08:53.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.881+0000 7f6c373be740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c7260950 0x5606c7260d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.881+0000 7f6c373be740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7261270 0x5606c726b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.881+0000 7f6c373be740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c726b940 0x5606c726dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.881+0000 7f6c373be740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5606c71c61d0 con 0x5606c7261270 2024-04-24T09:08:53.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.881+0000 7f6c373be740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5606c716c450 con 0x5606c7260950 2024-04-24T09:08:53.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.881+0000 7f6c373be740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5606c71c2190 con 0x5606c726b940 2024-04-24T09:08:53.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.881+0000 7f6c31a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7261270 0x5606c726b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.881+0000 7f6c31a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7261270 0x5606c726b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38324/0 (socket says 172.21.15.69:38324) 2024-04-24T09:08:53.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.881+0000 7f6c31a00640 1 -- 172.21.15.69:0/1678968831 learned_addr learned my addr 172.21.15.69:0/1678968831 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:53.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.881+0000 7f6c32400640 1 --2- 172.21.15.69:0/1678968831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c7260950 0x5606c7260d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.882+0000 7f6c31000640 1 --2- 172.21.15.69:0/1678968831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c726b940 0x5606c726dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.882+0000 7f6c31a00640 1 -- 172.21.15.69:0/1678968831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c726b940 msgr2=0x5606c726dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.882+0000 7f6c31a00640 1 --2- 172.21.15.69:0/1678968831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c726b940 0x5606c726dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.882+0000 7f6c31a00640 1 -- 172.21.15.69:0/1678968831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c7260950 msgr2=0x5606c7260d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.882+0000 7f6c31a00640 1 --2- 172.21.15.69:0/1678968831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c7260950 0x5606c7260d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.882+0000 7f6c2a000640 1 --2- 172.21.15.69:0/460126812 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f6c140a3da0 0x7f6c140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.882+0000 7f6c31a00640 1 -- 172.21.15.69:0/1678968831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606c7229310 con 0x5606c7261270 2024-04-24T09:08:53.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.882+0000 7f6c31000640 1 --2- 172.21.15.69:0/1678968831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c726b940 0x5606c726dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:53.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.882+0000 7f6c2a000640 1 -- 172.21.15.69:0/460126812 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f6c200f1340 con 0x7f6c2c1044b0 2024-04-24T09:08:53.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.882+0000 7f6c32400640 1 --2- 172.21.15.69:0/460126812 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f6c140a3da0 0x7f6c140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.882+0000 7f6c31a00640 1 --2- 172.21.15.69:0/1678968831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7261270 0x5606c726b400 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6c200ee6c0 tx=0x7f6c200b9300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc4097e618a4f7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.882+0000 7f6c1ea00640 1 -- 172.21.15.69:0/1678968831 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20049030 con 0x5606c7261270 2024-04-24T09:08:53.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.882+0000 7f6c1ea00640 1 -- 172.21.15.69:0/1678968831 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c200bd6a0 con 0x5606c7261270 2024-04-24T09:08:53.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.883+0000 7f6c1ea00640 1 -- 172.21.15.69:0/1678968831 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c200bcd70 con 0x5606c7261270 2024-04-24T09:08:53.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.883+0000 7f6c373be740 1 -- 172.21.15.69:0/1678968831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7261270 msgr2=0x5606c726b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.883+0000 7f6c32400640 1 --2- 172.21.15.69:0/460126812 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f6c140a3da0 0x7f6c140a6260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6c24005fd0 tx=0x7f6c24009530 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.883+0000 7f6c373be740 1 --2- 172.21.15.69:0/1678968831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7261270 0x5606c726b400 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6c200ee6c0 tx=0x7f6c200b9300 comp rx=0 tx=0).stop 2024-04-24T09:08:53.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.884+0000 7f6c373be740 1 -- 172.21.15.69:0/1678968831 shutdown_connections 2024-04-24T09:08:53.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.884+0000 7f6c373be740 1 --2- 172.21.15.69:0/1678968831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c726b940 0x5606c726dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.884+0000 7f6c373be740 1 --2- 172.21.15.69:0/1678968831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7261270 0x5606c726b400 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.884+0000 7f6c373be740 1 --2- 172.21.15.69:0/1678968831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c7260950 0x5606c7260d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.884+0000 7f6c373be740 1 -- 172.21.15.69:0/1678968831 >> 172.21.15.69:0/1678968831 conn(0x5606c725cc70 msgr2=0x5606c725d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.884+0000 7f6c373be740 1 -- 172.21.15.69:0/1678968831 shutdown_connections 2024-04-24T09:08:53.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.884+0000 7f6c373be740 1 -- 172.21.15.69:0/1678968831 wait complete. 2024-04-24T09:08:53.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.884+0000 7f6c373be740 1 -- start start 2024-04-24T09:08:53.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.884+0000 7f6c373be740 1 Processor -- start 2024-04-24T09:08:53.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.885+0000 7f6c373be740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c7260950 0x5606c726b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.885+0000 7f6c373be740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7261270 0x5606c72686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.885+0000 7f6c373be740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c726b940 0x5606c7268e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.885+0000 7f6c373be740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5606c726e4f0 con 0x5606c7261270 2024-04-24T09:08:53.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.885+0000 7f6c373be740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5606c717f810 con 0x5606c7260950 2024-04-24T09:08:53.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.885+0000 7f6c373be740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5606c726e670 con 0x5606c726b940 2024-04-24T09:08:53.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.885+0000 7f6c373be740 10 client.0 _ll_register_callbacks cb 0x5606c714d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.885+0000 7f6c31000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7261270 0x5606c72686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.885+0000 7f6c31000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7261270 0x5606c72686e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38338/0 (socket says 172.21.15.69:38338) 2024-04-24T09:08:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.885+0000 7f6c31000640 1 -- 172.21.15.69:0/507573496 learned_addr learned my addr 172.21.15.69:0/507573496 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.886+0000 7f6c31a00640 1 --2- 172.21.15.69:0/507573496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c7260950 0x5606c726b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.886+0000 7f6c32400640 1 --2- 172.21.15.69:0/507573496 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c726b940 0x5606c7268e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.886+0000 7f6c31000640 1 -- 172.21.15.69:0/507573496 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c726b940 msgr2=0x5606c7268e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.886+0000 7f6c31000640 1 --2- 172.21.15.69:0/507573496 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c726b940 0x5606c7268e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.886+0000 7f6c31000640 1 -- 172.21.15.69:0/507573496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c7260950 msgr2=0x5606c726b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.886+0000 7f6c31000640 1 --2- 172.21.15.69:0/507573496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c7260950 0x5606c726b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.886+0000 7f6c31000640 1 -- 172.21.15.69:0/507573496 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606c71903f0 con 0x5606c7261270 2024-04-24T09:08:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.886+0000 7f6c31000640 1 --2- 172.21.15.69:0/507573496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7261270 0x5606c72686e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6c18012700 tx=0x7f6c18012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854431866b10789f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.886+0000 7f6c13400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.886+0000 7f6c13400640 1 -- 172.21.15.69:0/507573496 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18010dc0 con 0x5606c7261270 2024-04-24T09:08:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.886+0000 7f6c13400640 1 -- 172.21.15.69:0/507573496 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f6c18004030 con 0x5606c7261270 2024-04-24T09:08:53.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.886+0000 7f6c373be740 1 -- 172.21.15.69:0/507573496 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c200bd6a0 con 0x5606c7261270 2024-04-24T09:08:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.887+0000 7f6c13400640 1 -- 172.21.15.69:0/507573496 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6c18025050 con 0x5606c7261270 2024-04-24T09:08:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.887+0000 7f6c373be740 10 client.16242 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.887+0000 7f6c373be740 1 -- 172.21.15.69:0/507573496 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5606c6ed83f0 con 0x5606c7261270 2024-04-24T09:08:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.887+0000 7f6c13400640 1 -- 172.21.15.69:0/507573496 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c200bd6a0 con 0x5606c7261270 2024-04-24T09:08:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.887+0000 7f6c13400640 1 -- 172.21.15.69:0/507573496 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f6c1801a420 con 0x5606c7261270 2024-04-24T09:08:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.887+0000 7f6c373be740 10 client.16242 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.887+0000 7f6c373be740 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.887+0000 7f6c373be740 20 client.16242 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.888+0000 7f6c373be740 1 --2- 172.21.15.69:0/507573496 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5606c727b900 0x5606c727dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.888+0000 7f6c373be740 4 client.16242 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:53.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.888+0000 7f6c373be740 1 -- 172.21.15.69:0/507573496 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606c71d3bf0 con 0x5606c727b900 2024-04-24T09:08:53.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.888+0000 7f6c31a00640 1 --2- 172.21.15.69:0/507573496 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5606c727b900 0x5606c727dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.889+0000 7f6c31a00640 1 --2- 172.21.15.69:0/507573496 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5606c727b900 0x5606c727dd40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1627698e1268856c server_cookie=fc0e0aef0bab4ac4 in_seq=0 out_seq=0 2024-04-24T09:08:53.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.889+0000 7f6c13400640 10 client.16242 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:53.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.891+0000 7f6c13400640 1 -- 172.21.15.69:0/507573496 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5606c71d3bf0 con 0x5606c727b900 2024-04-24T09:08:53.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.891+0000 7f6c13400640 10 client.16242 handle_command_reply: tid=0 2024-04-24T09:08:53.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.891+0000 7f6c373be740 1 client.16242 shutdown 2024-04-24T09:08:53.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.892+0000 7f6c373be740 1 -- 172.21.15.69:0/507573496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7261270 msgr2=0x5606c72686e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.892+0000 7f6c373be740 1 --2- 172.21.15.69:0/507573496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7261270 0x5606c72686e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6c18012700 tx=0x7f6c18012bc0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.892+0000 7f6c373be740 1 -- 172.21.15.69:0/507573496 shutdown_connections 2024-04-24T09:08:53.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.892+0000 7f6c373be740 1 --2- 172.21.15.69:0/507573496 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5606c727b900 0x5606c727dd40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.892+0000 7f6c373be740 1 --2- 172.21.15.69:0/507573496 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c726b940 0x5606c7268e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.892+0000 7f6c373be740 1 --2- 172.21.15.69:0/507573496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7261270 0x5606c72686e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.892+0000 7f6c373be740 1 --2- 172.21.15.69:0/507573496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c7260950 0x5606c726b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.892+0000 7f6c373be740 1 -- 172.21.15.69:0/507573496 >> 172.21.15.69:0/507573496 conn(0x5606c725cc70 msgr2=0x5606c7262970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.892+0000 7f6c13400640 0 client.16242 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:53.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.892+0000 7f6c373be740 1 -- 172.21.15.69:0/507573496 shutdown_connections 2024-04-24T09:08:53.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.892+0000 7f6c373be740 1 -- 172.21.15.69:0/507573496 wait complete. 2024-04-24T09:08:53.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.892+0000 7f6c373be740 20 client.16242 trim_cache size 0 max 16384 2024-04-24T09:08:53.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.909+0000 7f6c373be740 1 Processor -- start 2024-04-24T09:08:53.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.910+0000 7f6c373be740 1 -- start start 2024-04-24T09:08:53.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.910+0000 7f6c373be740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7260950 0x5606c72630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.910+0000 7f6c373be740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c7261270 0x5606c725c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.910+0000 7f6c373be740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c726b940 0x5606c726af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.910+0000 7f6c373be740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1b3df0 con 0x5606c7260950 2024-04-24T09:08:53.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.910+0000 7f6c373be740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1b16c0 con 0x5606c726b940 2024-04-24T09:08:53.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.910+0000 7f6c373be740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1af510 con 0x5606c7261270 2024-04-24T09:08:53.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.910+0000 7f6c31a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7260950 0x5606c72630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.910+0000 7f6c31a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7260950 0x5606c72630f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38350/0 (socket says 172.21.15.69:38350) 2024-04-24T09:08:53.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.910+0000 7f6c31a00640 1 -- 172.21.15.69:0/964450969 learned_addr learned my addr 172.21.15.69:0/964450969 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:53.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.910+0000 7f6c31000640 1 --2- 172.21.15.69:0/964450969 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c7261270 0x5606c725c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.911+0000 7f6c32400640 1 --2- 172.21.15.69:0/964450969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c726b940 0x5606c726af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.911+0000 7f6c31a00640 1 -- 172.21.15.69:0/964450969 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c7261270 msgr2=0x5606c725c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.911+0000 7f6c31a00640 1 --2- 172.21.15.69:0/964450969 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c7261270 0x5606c725c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.911+0000 7f6c31a00640 1 -- 172.21.15.69:0/964450969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c726b940 msgr2=0x5606c726af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.911+0000 7f6c31a00640 1 --2- 172.21.15.69:0/964450969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c726b940 0x5606c726af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.911+0000 7f6c31a00640 1 -- 172.21.15.69:0/964450969 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606c7258450 con 0x5606c7260950 2024-04-24T09:08:53.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.911+0000 7f6c32400640 1 --2- 172.21.15.69:0/964450969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c726b940 0x5606c726af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:53.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.911+0000 7f6c31a00640 1 --2- 172.21.15.69:0/964450969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7260950 0x5606c72630f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6c200ee800 tx=0x7f6c20009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32de9f209fc8585f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.911+0000 7f6c1e000640 1 -- 172.21.15.69:0/964450969 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c200c9070 con 0x5606c7260950 2024-04-24T09:08:53.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.911+0000 7f6c1e000640 1 -- 172.21.15.69:0/964450969 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c200bd6a0 con 0x5606c7260950 2024-04-24T09:08:53.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.911+0000 7f6c1e000640 1 -- 172.21.15.69:0/964450969 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c200c5e60 con 0x5606c7260950 2024-04-24T09:08:53.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.912+0000 7f6c373be740 1 -- 172.21.15.69:0/964450969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7260950 msgr2=0x5606c72630f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.912+0000 7f6c373be740 1 --2- 172.21.15.69:0/964450969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7260950 0x5606c72630f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6c200ee800 tx=0x7f6c20009260 comp rx=0 tx=0).stop 2024-04-24T09:08:53.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.912+0000 7f6c373be740 1 -- 172.21.15.69:0/964450969 shutdown_connections 2024-04-24T09:08:53.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.912+0000 7f6c373be740 1 --2- 172.21.15.69:0/964450969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c726b940 0x5606c726af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.912+0000 7f6c373be740 1 --2- 172.21.15.69:0/964450969 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c7261270 0x5606c725c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.912+0000 7f6c373be740 1 --2- 172.21.15.69:0/964450969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c7260950 0x5606c72630f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.912+0000 7f6c373be740 1 -- 172.21.15.69:0/964450969 >> 172.21.15.69:0/964450969 conn(0x5606c725cc70 msgr2=0x5606c7263980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.912+0000 7f6c373be740 1 -- 172.21.15.69:0/964450969 shutdown_connections 2024-04-24T09:08:53.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.912+0000 7f6c373be740 1 -- 172.21.15.69:0/964450969 wait complete. 2024-04-24T09:08:53.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.913+0000 7f6c373be740 1 -- start start 2024-04-24T09:08:53.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.913+0000 7f6c373be740 1 Processor -- start 2024-04-24T09:08:53.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.913+0000 7f6c373be740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c7260950 0x5606c7273d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.913+0000 7f6c373be740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c7261270 0x5606c7265960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.913+0000 7f6c373be740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c726b940 0x5606c7265ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.913+0000 7f6c373be740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1afb80 con 0x5606c726b940 2024-04-24T09:08:53.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.913+0000 7f6c373be740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1b0ff0 con 0x5606c7261270 2024-04-24T09:08:53.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.913+0000 7f6c373be740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c1b0250 con 0x5606c7260950 2024-04-24T09:08:53.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.913+0000 7f6c373be740 10 client.0 _ll_register_callbacks cb 0x7f6c18009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:53.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.914+0000 7f6c32400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c726b940 0x5606c7265ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.914+0000 7f6c32400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c726b940 0x5606c7265ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38366/0 (socket says 172.21.15.69:38366) 2024-04-24T09:08:53.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.914+0000 7f6c32400640 1 -- 172.21.15.69:0/2946581876 learned_addr learned my addr 172.21.15.69:0/2946581876 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:53.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.914+0000 7f6c31a00640 1 --2- 172.21.15.69:0/2946581876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c7260950 0x5606c7273d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.914+0000 7f6c31000640 1 --2- 172.21.15.69:0/2946581876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c7261270 0x5606c7265960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.914+0000 7f6c32400640 1 -- 172.21.15.69:0/2946581876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c7260950 msgr2=0x5606c7273d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.914+0000 7f6c32400640 1 --2- 172.21.15.69:0/2946581876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c7260950 0x5606c7273d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.914+0000 7f6c32400640 1 -- 172.21.15.69:0/2946581876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c7261270 msgr2=0x5606c7265960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:53.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.914+0000 7f6c32400640 1 --2- 172.21.15.69:0/2946581876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c7261270 0x5606c7265960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.914+0000 7f6c32400640 1 -- 172.21.15.69:0/2946581876 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606c6fa27d0 con 0x5606c726b940 2024-04-24T09:08:53.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.914+0000 7f6c31000640 1 --2- 172.21.15.69:0/2946581876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c7261270 0x5606c7265960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:53.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.914+0000 7f6c32400640 1 --2- 172.21.15.69:0/2946581876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c726b940 0x5606c7265ec0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6c24012a70 tx=0x7f6c24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4327bba1be812caa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:53.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.914+0000 7f6c13400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:53.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.915+0000 7f6c13400640 1 -- 172.21.15.69:0/2946581876 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2400fe20 con 0x5606c726b940 2024-04-24T09:08:53.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.915+0000 7f6c13400640 1 -- 172.21.15.69:0/2946581876 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f6c24021440 con 0x5606c726b940 2024-04-24T09:08:53.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.915+0000 7f6c373be740 1 -- 172.21.15.69:0/2946581876 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c200bd6a0 con 0x5606c726b940 2024-04-24T09:08:53.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.915+0000 7f6c13400640 1 -- 172.21.15.69:0/2946581876 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6c24021640 con 0x5606c726b940 2024-04-24T09:08:53.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.915+0000 7f6c373be740 10 client.16248 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:53.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.915+0000 7f6c373be740 1 -- 172.21.15.69:0/2946581876 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5606c72e1e80 con 0x5606c726b940 2024-04-24T09:08:53.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.915+0000 7f6c13400640 1 -- 172.21.15.69:0/2946581876 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c200bd6a0 con 0x5606c726b940 2024-04-24T09:08:53.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.915+0000 7f6c13400640 1 -- 172.21.15.69:0/2946581876 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f6c24029a10 con 0x5606c726b940 2024-04-24T09:08:53.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.916+0000 7f6c373be740 10 client.16248 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:53.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.916+0000 7f6c373be740 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:53.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.916+0000 7f6c373be740 20 client.16248 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:53.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.916+0000 7f6c373be740 1 --2- 172.21.15.69:0/2946581876 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5606c72e24c0 0x5606c72e48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:53.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.916+0000 7f6c373be740 4 client.16248 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:53.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.916+0000 7f6c373be740 1 -- 172.21.15.69:0/2946581876 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606c71cda60 con 0x5606c72e24c0 2024-04-24T09:08:53.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.916+0000 7f6c31a00640 1 --2- 172.21.15.69:0/2946581876 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5606c72e24c0 0x5606c72e48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:53.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.917+0000 7f6c31a00640 1 --2- 172.21.15.69:0/2946581876 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5606c72e24c0 0x5606c72e48e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f26fe2fd5585b12 server_cookie=1a7f47ea4a98261e in_seq=0 out_seq=0 2024-04-24T09:08:53.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.917+0000 7f6c13400640 10 client.16248 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:53.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.922+0000 7f6c13400640 1 -- 172.21.15.69:0/2946581876 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606c71cda60 con 0x5606c72e24c0 2024-04-24T09:08:53.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.922+0000 7f6c13400640 10 client.16248 handle_command_reply: tid=0 2024-04-24T09:08:53.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.922+0000 7f6c373be740 1 client.16248 shutdown 2024-04-24T09:08:53.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.922+0000 7f6c373be740 1 -- 172.21.15.69:0/2946581876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c726b940 msgr2=0x5606c7265ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.922+0000 7f6c373be740 1 --2- 172.21.15.69:0/2946581876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c726b940 0x5606c7265ec0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6c24012a70 tx=0x7f6c24004a40 comp rx=0 tx=0).stop 2024-04-24T09:08:53.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.922+0000 7f6c373be740 1 -- 172.21.15.69:0/2946581876 shutdown_connections 2024-04-24T09:08:53.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.922+0000 7f6c373be740 1 --2- 172.21.15.69:0/2946581876 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5606c72e24c0 0x5606c72e48e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.922+0000 7f6c373be740 1 --2- 172.21.15.69:0/2946581876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5606c726b940 0x5606c7265ec0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.922+0000 7f6c373be740 1 --2- 172.21.15.69:0/2946581876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606c7261270 0x5606c7265960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.922+0000 7f6c373be740 1 --2- 172.21.15.69:0/2946581876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606c7260950 0x5606c7273d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.922+0000 7f6c373be740 1 -- 172.21.15.69:0/2946581876 >> 172.21.15.69:0/2946581876 conn(0x5606c725cc70 msgr2=0x5606c72620a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.922+0000 7f6c13400640 0 client.16248 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:53.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.922+0000 7f6c373be740 1 -- 172.21.15.69:0/2946581876 shutdown_connections 2024-04-24T09:08:53.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.922+0000 7f6c373be740 1 -- 172.21.15.69:0/2946581876 wait complete. 2024-04-24T09:08:53.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.923+0000 7f6c373be740 20 client.16248 trim_cache size 0 max 16384 2024-04-24T09:08:53.923 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-04-24T09:08:53.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.923+0000 7f6c34400640 1 -- 172.21.15.69:0/460126812 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f6c140a3da0 msgr2=0x7f6c140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.923+0000 7f6c34400640 1 --2- 172.21.15.69:0/460126812 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f6c140a3da0 0x7f6c140a6260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6c24005fd0 tx=0x7f6c24009530 comp rx=0 tx=0).stop 2024-04-24T09:08:53.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.923+0000 7f6c34400640 1 -- 172.21.15.69:0/460126812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c2c1044b0 msgr2=0x7f6c2c1a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:53.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.923+0000 7f6c34400640 1 --2- 172.21.15.69:0/460126812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c2c1044b0 0x7f6c2c1a0780 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6c20005ab0 tx=0x7f6c20003c80 comp rx=0 tx=0).stop 2024-04-24T09:08:53.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.923+0000 7f6c34400640 1 -- 172.21.15.69:0/460126812 shutdown_connections 2024-04-24T09:08:53.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.923+0000 7f6c34400640 1 --2- 172.21.15.69:0/460126812 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f6c140a3da0 0x7f6c140a6260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.923+0000 7f6c34400640 1 --2- 172.21.15.69:0/460126812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c2c19a930 0x7f6c2c19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.923+0000 7f6c34400640 1 --2- 172.21.15.69:0/460126812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c2c109580 0x7f6c2c1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.923+0000 7f6c34400640 1 --2- 172.21.15.69:0/460126812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c2c1044b0 0x7f6c2c1a0780 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:53.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.923+0000 7f6c34400640 1 -- 172.21.15.69:0/460126812 >> 172.21.15.69:0/460126812 conn(0x7f6c2c0ffd90 msgr2=0x7f6c2c1011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:53.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.924+0000 7f6c34400640 1 -- 172.21.15.69:0/460126812 shutdown_connections 2024-04-24T09:08:53.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:53.924+0000 7f6c34400640 1 -- 172.21.15.69:0/460126812 wait complete. 2024-04-24T09:08:54.942 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:08:54.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:08:54.943 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:08:55.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.182+0000 7f2bda400640 1 -- 172.21.15.69:0/3746211881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd41060d0 msgr2=0x7f2bd410dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.182+0000 7f2bda400640 1 --2- 172.21.15.69:0/3746211881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd41060d0 0x7f2bd410dca0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009900 tx=0x7f2bcc02a410 comp rx=0 tx=0).stop 2024-04-24T09:08:55.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.182+0000 7f2bda400640 1 -- 172.21.15.69:0/3746211881 shutdown_connections 2024-04-24T09:08:55.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.182+0000 7f2bda400640 1 --2- 172.21.15.69:0/3746211881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd41060d0 0x7f2bd410dca0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.182+0000 7f2bda400640 1 --2- 172.21.15.69:0/3746211881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bd41019b0 0x7f2bd41059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.182+0000 7f2bda400640 1 --2- 172.21.15.69:0/3746211881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bd4101000 0x7f2bd41013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.182+0000 7f2bda400640 1 -- 172.21.15.69:0/3746211881 >> 172.21.15.69:0/3746211881 conn(0x7f2bd40fc880 msgr2=0x7f2bd40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:55.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.183+0000 7f2bda400640 1 -- 172.21.15.69:0/3746211881 shutdown_connections 2024-04-24T09:08:55.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.183+0000 7f2bda400640 1 -- 172.21.15.69:0/3746211881 wait complete. 2024-04-24T09:08:55.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.184+0000 7f2bda400640 1 Processor -- start 2024-04-24T09:08:55.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.184+0000 7f2bda400640 1 -- start start 2024-04-24T09:08:55.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.184+0000 7f2bda400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd4101000 0x7f2bd41a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.184+0000 7f2bda400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bd41060d0 0x7f2bd41a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.184+0000 7f2bda400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bd419a670 0x7f2bd419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.184+0000 7f2bda400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd4112190 con 0x7f2bd4101000 2024-04-24T09:08:55.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.184+0000 7f2bda400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd4112010 con 0x7f2bd419a670 2024-04-24T09:08:55.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.184+0000 7f2bda400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd4112310 con 0x7f2bd41060d0 2024-04-24T09:08:55.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.184+0000 7f2bd9000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd4101000 0x7f2bd41a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.184+0000 7f2bd9000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd4101000 0x7f2bd41a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38384/0 (socket says 172.21.15.69:38384) 2024-04-24T09:08:55.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.184+0000 7f2bd9000640 1 -- 172.21.15.69:0/3297421534 learned_addr learned my addr 172.21.15.69:0/3297421534 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:55.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.185+0000 7f2bd3e00640 1 --2- 172.21.15.69:0/3297421534 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bd41060d0 0x7f2bd41a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.185+0000 7f2bd9000640 1 -- 172.21.15.69:0/3297421534 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bd41060d0 msgr2=0x7f2bd41a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.185+0000 7f2bd9000640 1 --2- 172.21.15.69:0/3297421534 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bd41060d0 0x7f2bd41a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.185+0000 7f2bd9000640 1 -- 172.21.15.69:0/3297421534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bd419a670 msgr2=0x7f2bd419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.185+0000 7f2bd9000640 1 --2- 172.21.15.69:0/3297421534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bd419a670 0x7f2bd419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.185+0000 7f2bd9000640 1 -- 172.21.15.69:0/3297421534 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd419b270 con 0x7f2bd4101000 2024-04-24T09:08:55.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.185+0000 7f2bd9000640 1 --2- 172.21.15.69:0/3297421534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd4101000 0x7f2bd41a05b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800ea80 tx=0x7f2bc800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:55.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.185+0000 7f2bd1600640 1 -- 172.21.15.69:0/3297421534 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800ce40 con 0x7f2bd4101000 2024-04-24T09:08:55.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.185+0000 7f2bd1600640 1 -- 172.21.15.69:0/3297421534 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bc8005020 con 0x7f2bd4101000 2024-04-24T09:08:55.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.185+0000 7f2bda400640 1 -- 172.21.15.69:0/3297421534 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd41a7350 con 0x7f2bd4101000 2024-04-24T09:08:55.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.185+0000 7f2bd1600640 1 -- 172.21.15.69:0/3297421534 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80116d0 con 0x7f2bd4101000 2024-04-24T09:08:55.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.186+0000 7f2bda400640 1 -- 172.21.15.69:0/3297421534 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd41a7750 con 0x7f2bd4101000 2024-04-24T09:08:55.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.189+0000 7f2bde774740 1 Processor -- start 2024-04-24T09:08:55.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.189+0000 7f2bde774740 1 -- start start 2024-04-24T09:08:55.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.190+0000 7f2bd1600640 1 -- 172.21.15.69:0/3297421534 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2bc80051d0 con 0x7f2bd4101000 2024-04-24T09:08:55.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.190+0000 7f2bde774740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4571f270 0x557c4571f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.190+0000 7f2bde774740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4571fb90 0x557c45729d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.190+0000 7f2bde774740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4572a260 0x557c4572c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.190+0000 7f2bde774740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557c45684e70 con 0x557c4572a260 2024-04-24T09:08:55.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.190+0000 7f2bde774740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x557c4562b380 con 0x557c4571f270 2024-04-24T09:08:55.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.190+0000 7f2bde774740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557c45680e30 con 0x557c4571fb90 2024-04-24T09:08:55.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.190+0000 7f2bd3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4572a260 0x557c4572c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.190+0000 7f2bd3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4572a260 0x557c4572c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38396/0 (socket says 172.21.15.69:38396) 2024-04-24T09:08:55.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.190+0000 7f2bd3e00640 1 -- 172.21.15.69:0/2000894503 learned_addr learned my addr 172.21.15.69:0/2000894503 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:55.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.190+0000 7f2bd9a00640 1 --2- 172.21.15.69:0/2000894503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4571f270 0x557c4571f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.191+0000 7f2bd9000640 1 --2- 172.21.15.69:0/2000894503 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4571fb90 0x557c45729d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.191+0000 7f2bd3e00640 1 -- 172.21.15.69:0/2000894503 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4571fb90 msgr2=0x557c45729d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:55.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.191+0000 7f2bd3e00640 1 --2- 172.21.15.69:0/2000894503 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4571fb90 0x557c45729d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.191+0000 7f2bd3e00640 1 -- 172.21.15.69:0/2000894503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4571f270 msgr2=0x557c4571f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:55.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.191+0000 7f2bd3e00640 1 --2- 172.21.15.69:0/2000894503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4571f270 0x557c4571f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.191+0000 7f2bd3e00640 1 -- 172.21.15.69:0/2000894503 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c4545f7e0 con 0x557c4572a260 2024-04-24T09:08:55.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.192+0000 7f2bd3e00640 1 --2- 172.21.15.69:0/2000894503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4572a260 0x557c4572c650 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009a90 tx=0x7f2bc0009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3268ed00c39cbc4c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:55.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.192+0000 7f2bd1600640 1 --2- 172.21.15.69:0/3297421534 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f2bbc0a3da0 0x7f2bbc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.192+0000 7f2bc6000640 1 -- 172.21.15.69:0/2000894503 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0016070 con 0x557c4572a260 2024-04-24T09:08:55.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.192+0000 7f2bc6000640 1 -- 172.21.15.69:0/2000894503 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bc00054d0 con 0x557c4572a260 2024-04-24T09:08:55.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.192+0000 7f2bc6000640 1 -- 172.21.15.69:0/2000894503 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0004dd0 con 0x557c4572a260 2024-04-24T09:08:55.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.192+0000 7f2bd1600640 1 -- 172.21.15.69:0/3297421534 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f2bc80d0e40 con 0x7f2bd4101000 2024-04-24T09:08:55.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.192+0000 7f2bd9a00640 1 --2- 172.21.15.69:0/3297421534 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f2bbc0a3da0 0x7f2bbc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.193+0000 7f2bde774740 1 -- 172.21.15.69:0/2000894503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4572a260 msgr2=0x557c4572c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.193+0000 7f2bde774740 1 --2- 172.21.15.69:0/2000894503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4572a260 0x557c4572c650 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009a90 tx=0x7f2bc0009f50 comp rx=0 tx=0).stop 2024-04-24T09:08:55.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.193+0000 7f2bd9a00640 1 --2- 172.21.15.69:0/3297421534 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f2bbc0a3da0 0x7f2bbc0a6260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0057c0 tx=0x7f2bcc031400 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:55.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.193+0000 7f2bde774740 1 -- 172.21.15.69:0/2000894503 shutdown_connections 2024-04-24T09:08:55.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.193+0000 7f2bde774740 1 --2- 172.21.15.69:0/2000894503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4572a260 0x557c4572c650 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.193+0000 7f2bde774740 1 --2- 172.21.15.69:0/2000894503 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4571fb90 0x557c45729d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.193+0000 7f2bde774740 1 --2- 172.21.15.69:0/2000894503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4571f270 0x557c4571f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.193+0000 7f2bde774740 1 -- 172.21.15.69:0/2000894503 >> 172.21.15.69:0/2000894503 conn(0x557c4571b590 msgr2=0x557c4571b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:55.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.193+0000 7f2bde774740 1 -- 172.21.15.69:0/2000894503 shutdown_connections 2024-04-24T09:08:55.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.193+0000 7f2bde774740 1 -- 172.21.15.69:0/2000894503 wait complete. 2024-04-24T09:08:55.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.194+0000 7f2bde774740 1 -- start start 2024-04-24T09:08:55.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.194+0000 7f2bde774740 1 Processor -- start 2024-04-24T09:08:55.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bde774740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4571f270 0x557c45727100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bde774740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4571fb90 0x557c45727640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bde774740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4572a260 0x557c45727b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bde774740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557c4572d050 con 0x557c4572a260 2024-04-24T09:08:55.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bde774740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x557c4563e660 con 0x557c4571fb90 2024-04-24T09:08:55.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bde774740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557c4572d1d0 con 0x557c4571f270 2024-04-24T09:08:55.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bde774740 10 client.0 _ll_register_callbacks cb 0x557c4562caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:55.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bd9a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4572a260 0x557c45727b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bd9a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4572a260 0x557c45727b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38400/0 (socket says 172.21.15.69:38400) 2024-04-24T09:08:55.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bd9a00640 1 -- 172.21.15.69:0/2852606953 learned_addr learned my addr 172.21.15.69:0/2852606953 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:55.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bd9000640 1 --2- 172.21.15.69:0/2852606953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4571f270 0x557c45727100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bd3e00640 1 --2- 172.21.15.69:0/2852606953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4571fb90 0x557c45727640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bd9a00640 1 -- 172.21.15.69:0/2852606953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4571f270 msgr2=0x557c45727100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:55.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bd9a00640 1 --2- 172.21.15.69:0/2852606953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4571f270 0x557c45727100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bd9a00640 1 -- 172.21.15.69:0/2852606953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4571fb90 msgr2=0x557c45727640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:55.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bd9a00640 1 --2- 172.21.15.69:0/2852606953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4571fb90 0x557c45727640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bd9a00640 1 -- 172.21.15.69:0/2852606953 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c454bff70 con 0x557c4572a260 2024-04-24T09:08:55.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bd9000640 1 --2- 172.21.15.69:0/2852606953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4571f270 0x557c45727100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:55.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.195+0000 7f2bd3e00640 1 --2- 172.21.15.69:0/2852606953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4571fb90 0x557c45727640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:55.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.196+0000 7f2bd9a00640 1 --2- 172.21.15.69:0/2852606953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4572a260 0x557c45727b80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc033000 tx=0x7f2bcc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6846db1a669d225 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.196+0000 7f2bbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.196+0000 7f2bbaa00640 1 -- 172.21.15.69:0/2852606953 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc04d020 con 0x557c4572a260 2024-04-24T09:08:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.196+0000 7f2bbaa00640 1 -- 172.21.15.69:0/2852606953 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f2bcc003c00 con 0x557c4572a260 2024-04-24T09:08:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.196+0000 7f2bde774740 1 -- 172.21.15.69:0/2852606953 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc00054d0 con 0x557c4572a260 2024-04-24T09:08:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.197+0000 7f2bbaa00640 1 -- 172.21.15.69:0/2852606953 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2bcc058050 con 0x557c4572a260 2024-04-24T09:08:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.197+0000 7f2bde774740 10 client.16260 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.197+0000 7f2bde774740 1 -- 172.21.15.69:0/2852606953 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557c45357170 con 0x557c4572a260 2024-04-24T09:08:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.197+0000 7f2bbaa00640 1 -- 172.21.15.69:0/2852606953 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bc00054d0 con 0x557c4572a260 2024-04-24T09:08:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.197+0000 7f2bbaa00640 1 -- 172.21.15.69:0/2852606953 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2bcc049480 con 0x557c4572a260 2024-04-24T09:08:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.197+0000 7f2bde774740 10 client.16260 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.197+0000 7f2bde774740 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.197+0000 7f2bde774740 20 client.16260 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.197+0000 7f2bde774740 1 --2- 172.21.15.69:0/2852606953 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x557c4573a5f0 0x557c4573ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.197+0000 7f2bde774740 4 client.16260 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.197+0000 7f2bde774740 1 -- 172.21.15.69:0/2852606953 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c45692890 con 0x557c4573a5f0 2024-04-24T09:08:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.198+0000 7f2bd9000640 1 --2- 172.21.15.69:0/2852606953 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x557c4573a5f0 0x557c4573ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.199+0000 7f2bd9000640 1 --2- 172.21.15.69:0/2852606953 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x557c4573a5f0 0x557c4573ca90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=931cc95a415eecc server_cookie=973855df8a3c97c3 in_seq=0 out_seq=0 2024-04-24T09:08:55.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.199+0000 7f2bbaa00640 10 client.16260 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:55.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.203+0000 7f2bbaa00640 1 -- 172.21.15.69:0/2852606953 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c45692890 con 0x557c4573a5f0 2024-04-24T09:08:55.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.203+0000 7f2bbaa00640 10 client.16260 handle_command_reply: tid=0 2024-04-24T09:08:55.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.203+0000 7f2bde774740 1 client.16260 shutdown 2024-04-24T09:08:55.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.204+0000 7f2bde774740 1 -- 172.21.15.69:0/2852606953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4572a260 msgr2=0x557c45727b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.204+0000 7f2bde774740 1 --2- 172.21.15.69:0/2852606953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4572a260 0x557c45727b80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc033000 tx=0x7f2bcc005fd0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.204+0000 7f2bde774740 1 -- 172.21.15.69:0/2852606953 shutdown_connections 2024-04-24T09:08:55.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.204+0000 7f2bde774740 1 --2- 172.21.15.69:0/2852606953 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x557c4573a5f0 0x557c4573ca90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.204+0000 7f2bde774740 1 --2- 172.21.15.69:0/2852606953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4572a260 0x557c45727b80 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.204+0000 7f2bde774740 1 --2- 172.21.15.69:0/2852606953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4571fb90 0x557c45727640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.204+0000 7f2bde774740 1 --2- 172.21.15.69:0/2852606953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4571f270 0x557c45727100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.204+0000 7f2bde774740 1 -- 172.21.15.69:0/2852606953 >> 172.21.15.69:0/2852606953 conn(0x557c4571b590 msgr2=0x557c45721390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:55.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.204+0000 7f2bbaa00640 0 client.16260 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:55.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.204+0000 7f2bde774740 1 -- 172.21.15.69:0/2852606953 shutdown_connections 2024-04-24T09:08:55.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.204+0000 7f2bde774740 1 -- 172.21.15.69:0/2852606953 wait complete. 2024-04-24T09:08:55.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.204+0000 7f2bde774740 20 client.16260 trim_cache size 0 max 16384 2024-04-24T09:08:55.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.227+0000 7f2bde774740 1 Processor -- start 2024-04-24T09:08:55.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.228+0000 7f2bde774740 1 -- start start 2024-04-24T09:08:55.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.228+0000 7f2bde774740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4571f270 0x557c45735f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.228+0000 7f2bde774740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4571fb90 0x557c45721a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.228+0000 7f2bde774740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4572a260 0x557c45742360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.228+0000 7f2bde774740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41b3ae0 con 0x557c4571f270 2024-04-24T09:08:55.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.228+0000 7f2bde774740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41b13b0 con 0x557c4571fb90 2024-04-24T09:08:55.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.228+0000 7f2bde774740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41af200 con 0x557c4572a260 2024-04-24T09:08:55.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.228+0000 7f2bd9000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4571f270 0x557c45735f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.228+0000 7f2bd9000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4571f270 0x557c45735f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38416/0 (socket says 172.21.15.69:38416) 2024-04-24T09:08:55.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.228+0000 7f2bd9000640 1 -- 172.21.15.69:0/4162473436 learned_addr learned my addr 172.21.15.69:0/4162473436 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:55.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.229+0000 7f2bd9000640 1 -- 172.21.15.69:0/4162473436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4572a260 msgr2=0x557c45742360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:55.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.229+0000 7f2bd9000640 1 --2- 172.21.15.69:0/4162473436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4572a260 0x557c45742360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.229+0000 7f2bd9000640 1 -- 172.21.15.69:0/4162473436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4571fb90 msgr2=0x557c45721a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:55.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.229+0000 7f2bd3e00640 1 --2- 172.21.15.69:0/4162473436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4571fb90 0x557c45721a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.229+0000 7f2bd9000640 1 --2- 172.21.15.69:0/4162473436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4571fb90 0x557c45721a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.229+0000 7f2bd9000640 1 -- 172.21.15.69:0/4162473436 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c45715a60 con 0x557c4571f270 2024-04-24T09:08:55.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.229+0000 7f2bd9000640 1 --2- 172.21.15.69:0/4162473436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4571f270 0x557c45735f10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8005bc0 tx=0x7f2bc80bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6f3fd171d4cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:55.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.229+0000 7f2bc5600640 1 -- 172.21.15.69:0/4162473436 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc801f040 con 0x557c4571f270 2024-04-24T09:08:55.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.229+0000 7f2bc5600640 1 -- 172.21.15.69:0/4162473436 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bc80a0e50 con 0x557c4571f270 2024-04-24T09:08:55.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.229+0000 7f2bc5600640 1 -- 172.21.15.69:0/4162473436 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80ce4d0 con 0x557c4571f270 2024-04-24T09:08:55.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.230+0000 7f2bde774740 1 -- 172.21.15.69:0/4162473436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4571f270 msgr2=0x557c45735f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.230+0000 7f2bde774740 1 --2- 172.21.15.69:0/4162473436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4571f270 0x557c45735f10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8005bc0 tx=0x7f2bc80bd040 comp rx=0 tx=0).stop 2024-04-24T09:08:55.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.230+0000 7f2bde774740 1 -- 172.21.15.69:0/4162473436 shutdown_connections 2024-04-24T09:08:55.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.230+0000 7f2bde774740 1 --2- 172.21.15.69:0/4162473436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4572a260 0x557c45742360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.230+0000 7f2bde774740 1 --2- 172.21.15.69:0/4162473436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4571fb90 0x557c45721a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.230+0000 7f2bde774740 1 --2- 172.21.15.69:0/4162473436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4571f270 0x557c45735f10 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.230+0000 7f2bde774740 1 -- 172.21.15.69:0/4162473436 >> 172.21.15.69:0/4162473436 conn(0x557c4571b590 msgr2=0x557c4571a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:55.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.230+0000 7f2bde774740 1 -- 172.21.15.69:0/4162473436 shutdown_connections 2024-04-24T09:08:55.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.230+0000 7f2bde774740 1 -- 172.21.15.69:0/4162473436 wait complete. 2024-04-24T09:08:55.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.231+0000 7f2bde774740 1 -- start start 2024-04-24T09:08:55.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.231+0000 7f2bde774740 1 Processor -- start 2024-04-24T09:08:55.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.231+0000 7f2bde774740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4571f270 0x557c457326f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.231+0000 7f2bde774740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4571fb90 0x557c4572f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.231+0000 7f2bde774740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4572a260 0x557c4572f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.231+0000 7f2bde774740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41af870 con 0x557c4571f270 2024-04-24T09:08:55.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.231+0000 7f2bde774740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41b0ce0 con 0x557c4572a260 2024-04-24T09:08:55.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.231+0000 7f2bde774740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41aff40 con 0x557c4571fb90 2024-04-24T09:08:55.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.231+0000 7f2bde774740 10 client.0 _ll_register_callbacks cb 0x7f2bcc040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:55.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.231+0000 7f2bd9000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4571f270 0x557c457326f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.232+0000 7f2bd9000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4571f270 0x557c457326f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38426/0 (socket says 172.21.15.69:38426) 2024-04-24T09:08:55.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.232+0000 7f2bd9000640 1 -- 172.21.15.69:0/3012849105 learned_addr learned my addr 172.21.15.69:0/3012849105 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:55.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.232+0000 7f2bd9a00640 1 --2- 172.21.15.69:0/3012849105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4572a260 0x557c4572f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.232+0000 7f2bd9000640 1 -- 172.21.15.69:0/3012849105 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4571fb90 msgr2=0x557c4572f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:55.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.232+0000 7f2bd9000640 1 --2- 172.21.15.69:0/3012849105 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4571fb90 0x557c4572f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.232+0000 7f2bd9000640 1 -- 172.21.15.69:0/3012849105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4572a260 msgr2=0x557c4572f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:55.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.232+0000 7f2bd9000640 1 --2- 172.21.15.69:0/3012849105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4572a260 0x557c4572f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.232+0000 7f2bd9000640 1 -- 172.21.15.69:0/3012849105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c4572fff0 con 0x557c4571f270 2024-04-24T09:08:55.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.232+0000 7f2bd9a00640 1 --2- 172.21.15.69:0/3012849105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4572a260 0x557c4572f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:55.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.232+0000 7f2bd9000640 1 --2- 172.21.15.69:0/3012849105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4571f270 0x557c457326f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80a51e0 tx=0x7f2bc80d4430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42eaafdcefd5f679 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:55.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.232+0000 7f2bbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:55.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.232+0000 7f2bbaa00640 1 -- 172.21.15.69:0/3012849105 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc801f040 con 0x557c4571f270 2024-04-24T09:08:55.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.233+0000 7f2bbaa00640 1 -- 172.21.15.69:0/3012849105 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f2bc80ce4d0 con 0x557c4571f270 2024-04-24T09:08:55.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.233+0000 7f2bde774740 1 -- 172.21.15.69:0/3012849105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc80a0e50 con 0x557c4571f270 2024-04-24T09:08:55.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.233+0000 7f2bbaa00640 1 -- 172.21.15.69:0/3012849105 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2bc80b6020 con 0x557c4571f270 2024-04-24T09:08:55.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.233+0000 7f2bde774740 10 client.16266 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:55.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.233+0000 7f2bde774740 1 -- 172.21.15.69:0/3012849105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557c457a1b10 con 0x557c4571f270 2024-04-24T09:08:55.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.233+0000 7f2bbaa00640 1 -- 172.21.15.69:0/3012849105 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bc80a0e50 con 0x557c4571f270 2024-04-24T09:08:55.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.233+0000 7f2bbaa00640 1 -- 172.21.15.69:0/3012849105 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2bc80d2040 con 0x557c4571f270 2024-04-24T09:08:55.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.233+0000 7f2bde774740 10 client.16266 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:55.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.233+0000 7f2bde774740 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:55.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.233+0000 7f2bde774740 20 client.16266 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:55.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.233+0000 7f2bde774740 1 --2- 172.21.15.69:0/3012849105 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x557c457a2200 0x557c457a4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.233+0000 7f2bde774740 4 client.16266 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:08:55.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.233+0000 7f2bde774740 1 -- 172.21.15.69:0/3012849105 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557c4568c700 con 0x557c457a2200 2024-04-24T09:08:55.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.234+0000 7f2bd3e00640 1 --2- 172.21.15.69:0/3012849105 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x557c457a2200 0x557c457a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.235+0000 7f2bd3e00640 1 --2- 172.21.15.69:0/3012849105 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x557c457a2200 0x557c457a4620 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb3468868ea8ba3 server_cookie=fffd908d40480a1f in_seq=0 out_seq=0 2024-04-24T09:08:55.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.235+0000 7f2bbaa00640 10 client.16266 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:55.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.238+0000 7f2bbaa00640 1 -- 172.21.15.69:0/3012849105 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557c4568c700 con 0x557c457a2200 2024-04-24T09:08:55.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.238+0000 7f2bbaa00640 10 client.16266 handle_command_reply: tid=0 2024-04-24T09:08:55.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.238+0000 7f2bde774740 1 client.16266 shutdown 2024-04-24T09:08:55.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.238+0000 7f2bde774740 1 -- 172.21.15.69:0/3012849105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4571f270 msgr2=0x557c457326f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.238+0000 7f2bde774740 1 --2- 172.21.15.69:0/3012849105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4571f270 0x557c457326f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80a51e0 tx=0x7f2bc80d4430 comp rx=0 tx=0).stop 2024-04-24T09:08:55.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.238+0000 7f2bde774740 1 -- 172.21.15.69:0/3012849105 shutdown_connections 2024-04-24T09:08:55.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.239+0000 7f2bde774740 1 --2- 172.21.15.69:0/3012849105 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x557c457a2200 0x557c457a4620 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.239+0000 7f2bde774740 1 --2- 172.21.15.69:0/3012849105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c4572a260 0x557c4572f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.239+0000 7f2bde774740 1 --2- 172.21.15.69:0/3012849105 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c4571fb90 0x557c4572f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.239+0000 7f2bbaa00640 0 client.16266 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:55.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.239+0000 7f2bde774740 1 --2- 172.21.15.69:0/3012849105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c4571f270 0x557c457326f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.239+0000 7f2bde774740 1 -- 172.21.15.69:0/3012849105 >> 172.21.15.69:0/3012849105 conn(0x557c4571b590 msgr2=0x557c457270c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:55.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.239+0000 7f2bde774740 1 -- 172.21.15.69:0/3012849105 shutdown_connections 2024-04-24T09:08:55.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.239+0000 7f2bde774740 1 -- 172.21.15.69:0/3012849105 wait complete. 2024-04-24T09:08:55.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.239+0000 7f2bde774740 20 client.16266 trim_cache size 0 max 16384 2024-04-24T09:08:55.240 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:08:55.240 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-04-24T09:08:55.240 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "6ee0eed2-afa2-43a2-9f3b-5a58af4afe23", 2024-04-24T09:08:55.240 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-04-24T09:08:55.240 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:08:55.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.240+0000 7f2bda400640 1 -- 172.21.15.69:0/3297421534 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f2bbc0a3da0 msgr2=0x7f2bbc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.240+0000 7f2bda400640 1 --2- 172.21.15.69:0/3297421534 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f2bbc0a3da0 0x7f2bbc0a6260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0057c0 tx=0x7f2bcc031400 comp rx=0 tx=0).stop 2024-04-24T09:08:55.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.240+0000 7f2bda400640 1 -- 172.21.15.69:0/3297421534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd4101000 msgr2=0x7f2bd41a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.240+0000 7f2bda400640 1 --2- 172.21.15.69:0/3297421534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd4101000 0x7f2bd41a05b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800ea80 tx=0x7f2bc800ef40 comp rx=0 tx=0).stop 2024-04-24T09:08:55.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.240+0000 7f2bda400640 1 -- 172.21.15.69:0/3297421534 shutdown_connections 2024-04-24T09:08:55.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.240+0000 7f2bda400640 1 --2- 172.21.15.69:0/3297421534 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f2bbc0a3da0 0x7f2bbc0a6260 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.240+0000 7f2bda400640 1 --2- 172.21.15.69:0/3297421534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bd419a670 0x7f2bd419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.240+0000 7f2bda400640 1 --2- 172.21.15.69:0/3297421534 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bd41060d0 0x7f2bd41a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.240+0000 7f2bda400640 1 --2- 172.21.15.69:0/3297421534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd4101000 0x7f2bd41a05b0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.240+0000 7f2bda400640 1 -- 172.21.15.69:0/3297421534 >> 172.21.15.69:0/3297421534 conn(0x7f2bd40fc880 msgr2=0x7f2bd40fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:55.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.240+0000 7f2bda400640 1 -- 172.21.15.69:0/3297421534 shutdown_connections 2024-04-24T09:08:55.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.240+0000 7f2bda400640 1 -- 172.21.15.69:0/3297421534 wait complete. 2024-04-24T09:08:55.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:08:55.261 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:08:55.325 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:55 smithi069 ceph-mon[28170]: pgmap v58: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-04-24T09:08:55.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:55 smithi130 ceph-mon[31553]: pgmap v58: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-04-24T09:08:55.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.474+0000 7fd887e00640 1 -- 172.21.15.69:0/2871837878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd888104cd0 msgr2=0x7fd888108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.474+0000 7fd887e00640 1 --2- 172.21.15.69:0/2871837878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd888104cd0 0x7fd888108d10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd878009900 tx=0x7fd87802a410 comp rx=0 tx=0).stop 2024-04-24T09:08:55.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.475+0000 7fd887e00640 1 -- 172.21.15.69:0/2871837878 shutdown_connections 2024-04-24T09:08:55.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.475+0000 7fd887e00640 1 --2- 172.21.15.69:0/2871837878 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8881093f0 0x7fd888110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.475+0000 7fd887e00640 1 --2- 172.21.15.69:0/2871837878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd888104cd0 0x7fd888108d10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.475+0000 7fd887e00640 1 --2- 172.21.15.69:0/2871837878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd888104320 0x7fd888104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.475+0000 7fd887e00640 1 -- 172.21.15.69:0/2871837878 >> 172.21.15.69:0/2871837878 conn(0x7fd8880ffb40 msgr2=0x7fd888101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:55.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.475+0000 7fd887e00640 1 -- 172.21.15.69:0/2871837878 shutdown_connections 2024-04-24T09:08:55.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.475+0000 7fd887e00640 1 -- 172.21.15.69:0/2871837878 wait complete. 2024-04-24T09:08:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.476+0000 7fd887e00640 1 Processor -- start 2024-04-24T09:08:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.476+0000 7fd887e00640 1 -- start start 2024-04-24T09:08:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.476+0000 7fd887e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd888104320 0x7fd8881a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.476+0000 7fd887e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8881093f0 0x7fd8881a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.476+0000 7fd887e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd88819a720 0x7fd88819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.476+0000 7fd887e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd888114380 con 0x7fd888104320 2024-04-24T09:08:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.476+0000 7fd887e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd888114200 con 0x7fd88819a720 2024-04-24T09:08:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.476+0000 7fd887e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd888114500 con 0x7fd8881093f0 2024-04-24T09:08:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.476+0000 7fd886a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd888104320 0x7fd8881a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.477+0000 7fd886a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd888104320 0x7fd8881a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38446/0 (socket says 172.21.15.69:38446) 2024-04-24T09:08:55.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.477+0000 7fd886a00640 1 -- 172.21.15.69:0/3644809446 learned_addr learned my addr 172.21.15.69:0/3644809446 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:55.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.477+0000 7fd887400640 1 --2- 172.21.15.69:0/3644809446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd88819a720 0x7fd88819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.477+0000 7fd886000640 1 --2- 172.21.15.69:0/3644809446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8881093f0 0x7fd8881a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.477+0000 7fd886a00640 1 -- 172.21.15.69:0/3644809446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8881093f0 msgr2=0x7fd8881a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.477+0000 7fd886a00640 1 --2- 172.21.15.69:0/3644809446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8881093f0 0x7fd8881a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.477+0000 7fd886a00640 1 -- 172.21.15.69:0/3644809446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd88819a720 msgr2=0x7fd88819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.477+0000 7fd886a00640 1 --2- 172.21.15.69:0/3644809446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd88819a720 0x7fd88819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.477+0000 7fd886a00640 1 -- 172.21.15.69:0/3644809446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88819b320 con 0x7fd888104320 2024-04-24T09:08:55.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.478+0000 7fd886a00640 1 --2- 172.21.15.69:0/3644809446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd888104320 0x7fd8881a0660 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd87000e9f0 tx=0x7fd87000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:55.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.478+0000 7fd87f400640 1 -- 172.21.15.69:0/3644809446 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87000cd80 con 0x7fd888104320 2024-04-24T09:08:55.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.478+0000 7fd87f400640 1 -- 172.21.15.69:0/3644809446 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8700108e0 con 0x7fd888104320 2024-04-24T09:08:55.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.478+0000 7fd887e00640 1 -- 172.21.15.69:0/3644809446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8881a7400 con 0x7fd888104320 2024-04-24T09:08:55.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.478+0000 7fd87f400640 1 -- 172.21.15.69:0/3644809446 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870018920 con 0x7fd888104320 2024-04-24T09:08:55.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.478+0000 7fd887e00640 1 -- 172.21.15.69:0/3644809446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8881a7800 con 0x7fd888104320 2024-04-24T09:08:55.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.482+0000 7fd87f400640 1 -- 172.21.15.69:0/3644809446 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd870010a90 con 0x7fd888104320 2024-04-24T09:08:55.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.482+0000 7fd890640740 1 Processor -- start 2024-04-24T09:08:55.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.482+0000 7fd890640740 1 -- start start 2024-04-24T09:08:55.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.482+0000 7fd890640740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c5293fa30 0x557c5293fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.482+0000 7fd890640740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c52940350 0x557c5294a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.482+0000 7fd890640740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5294aa20 0x557c5294ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.482+0000 7fd890640740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557c528a51f0 con 0x557c5294aa20 2024-04-24T09:08:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.482+0000 7fd890640740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x557c5284b350 con 0x557c5293fa30 2024-04-24T09:08:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.482+0000 7fd890640740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557c528a11b0 con 0x557c52940350 2024-04-24T09:08:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.483+0000 7fd887400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c5293fa30 0x557c5293fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.483+0000 7fd887400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c5293fa30 0x557c5293fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.69:56614/0 (socket says 172.21.15.69:56614) 2024-04-24T09:08:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.483+0000 7fd887400640 1 -- 172.21.15.69:0/2156421660 learned_addr learned my addr 172.21.15.69:0/2156421660 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.483+0000 7fd886a00640 1 --2- 172.21.15.69:0/2156421660 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c52940350 0x557c5294a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.483+0000 7fd886000640 1 --2- 172.21.15.69:0/2156421660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5294aa20 0x557c5294ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.483+0000 7fd887400640 1 -- 172.21.15.69:0/2156421660 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c52940350 msgr2=0x557c5294a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:55.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.484+0000 7fd887400640 1 --2- 172.21.15.69:0/2156421660 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c52940350 0x557c5294a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.484+0000 7fd887400640 1 -- 172.21.15.69:0/2156421660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5294aa20 msgr2=0x557c5294ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:55.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.484+0000 7fd887400640 1 --2- 172.21.15.69:0/2156421660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5294aa20 0x557c5294ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.484+0000 7fd887400640 1 -- 172.21.15.69:0/2156421660 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c5289ff50 con 0x557c5293fa30 2024-04-24T09:08:55.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.484+0000 7fd886000640 1 --2- 172.21.15.69:0/2156421660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5294aa20 0x557c5294ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:55.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.484+0000 7fd87f400640 1 --2- 172.21.15.69:0/3644809446 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd86c0a3da0 0x7fd86c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.484+0000 7fd87f400640 1 -- 172.21.15.69:0/3644809446 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd8700d0e20 con 0x7fd888104320 2024-04-24T09:08:55.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.484+0000 7fd886a00640 1 --2- 172.21.15.69:0/3644809446 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd86c0a3da0 0x7fd86c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.484+0000 7fd887400640 1 --2- 172.21.15.69:0/2156421660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c5293fa30 0x557c5293fe10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd880009f50 tx=0x7fd8800118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce2ce1cc90884e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:55.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.484+0000 7fd877e00640 1 -- 172.21.15.69:0/2156421660 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880004d20 con 0x557c5293fa30 2024-04-24T09:08:55.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.484+0000 7fd877e00640 1 -- 172.21.15.69:0/2156421660 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd880004050 con 0x557c5293fa30 2024-04-24T09:08:55.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.484+0000 7fd877e00640 1 -- 172.21.15.69:0/2156421660 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880005280 con 0x557c5293fa30 2024-04-24T09:08:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.486+0000 7fd890640740 1 -- 172.21.15.69:0/2156421660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c5293fa30 msgr2=0x557c5293fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.486+0000 7fd886a00640 1 --2- 172.21.15.69:0/3644809446 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd86c0a3da0 0x7fd86c0a6260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd8700ce2e0 tx=0x7fd870000bc0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.486+0000 7fd890640740 1 --2- 172.21.15.69:0/2156421660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c5293fa30 0x557c5293fe10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd880009f50 tx=0x7fd8800118d0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.486+0000 7fd890640740 1 -- 172.21.15.69:0/2156421660 shutdown_connections 2024-04-24T09:08:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.486+0000 7fd890640740 1 --2- 172.21.15.69:0/2156421660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5294aa20 0x557c5294ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.486+0000 7fd890640740 1 --2- 172.21.15.69:0/2156421660 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c52940350 0x557c5294a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.486+0000 7fd890640740 1 --2- 172.21.15.69:0/2156421660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c5293fa30 0x557c5293fe10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.486+0000 7fd890640740 1 -- 172.21.15.69:0/2156421660 >> 172.21.15.69:0/2156421660 conn(0x557c5293bd50 msgr2=0x557c5293c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.486+0000 7fd890640740 1 -- 172.21.15.69:0/2156421660 shutdown_connections 2024-04-24T09:08:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.486+0000 7fd890640740 1 -- 172.21.15.69:0/2156421660 wait complete. 2024-04-24T09:08:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.487+0000 7fd890640740 1 -- start start 2024-04-24T09:08:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.487+0000 7fd890640740 1 Processor -- start 2024-04-24T09:08:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.488+0000 7fd890640740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5293fa30 0x557c52947910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.488+0000 7fd890640740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c52940350 0x557c52947e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.488+0000 7fd890640740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c5294aa20 0x557c52948390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.488+0000 7fd890640740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557c52956020 con 0x557c5293fa30 2024-04-24T09:08:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.488+0000 7fd890640740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x557c5285e690 con 0x557c52940350 2024-04-24T09:08:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.488+0000 7fd890640740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557c529561a0 con 0x557c5294aa20 2024-04-24T09:08:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.488+0000 7fd887400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5293fa30 0x557c52947910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.488+0000 7fd887400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5293fa30 0x557c52947910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38468/0 (socket says 172.21.15.69:38468) 2024-04-24T09:08:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.488+0000 7fd887400640 1 -- 172.21.15.69:0/1463376437 learned_addr learned my addr 172.21.15.69:0/1463376437 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.488+0000 7fd890640740 10 client.0 _ll_register_callbacks cb 0x557c5289a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.489+0000 7fd887400640 1 --2- 172.21.15.69:0/1463376437 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c5294aa20 0x557c52948390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.489+0000 7fd886000640 1 --2- 172.21.15.69:0/1463376437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c52940350 0x557c52947e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.489+0000 7fd887400640 1 -- 172.21.15.69:0/1463376437 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c5294aa20 msgr2=0x557c52948390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.489+0000 7fd887400640 1 --2- 172.21.15.69:0/1463376437 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c5294aa20 0x557c52948390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.489+0000 7fd887400640 1 -- 172.21.15.69:0/1463376437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c52940350 msgr2=0x557c52947e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:55.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.489+0000 7fd887400640 1 --2- 172.21.15.69:0/1463376437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c52940350 0x557c52947e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.489+0000 7fd887400640 1 -- 172.21.15.69:0/1463376437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c5285d920 con 0x557c5293fa30 2024-04-24T09:08:55.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.489+0000 7fd887400640 1 --2- 172.21.15.69:0/1463376437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5293fa30 0x557c52947910 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd880009f20 tx=0x7fd880003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896555a492204787 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:55.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.489+0000 7fd874c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:55.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.490+0000 7fd874c00640 1 -- 172.21.15.69:0/1463376437 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88002b030 con 0x557c5293fa30 2024-04-24T09:08:55.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.490+0000 7fd874c00640 1 -- 172.21.15.69:0/1463376437 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd880025670 con 0x557c5293fa30 2024-04-24T09:08:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.490+0000 7fd890640740 1 -- 172.21.15.69:0/1463376437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd880004050 con 0x557c5293fa30 2024-04-24T09:08:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.490+0000 7fd874c00640 1 -- 172.21.15.69:0/1463376437 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd880025870 con 0x557c5293fa30 2024-04-24T09:08:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.490+0000 7fd890640740 10 client.16278 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.490+0000 7fd890640740 1 -- 172.21.15.69:0/1463376437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557c526817d0 con 0x557c5293fa30 2024-04-24T09:08:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.491+0000 7fd874c00640 1 -- 172.21.15.69:0/1463376437 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd880004050 con 0x557c5293fa30 2024-04-24T09:08:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.491+0000 7fd874c00640 1 -- 172.21.15.69:0/1463376437 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fd880027b00 con 0x557c5293fa30 2024-04-24T09:08:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.491+0000 7fd890640740 10 client.16278 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.491+0000 7fd890640740 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.491+0000 7fd890640740 20 client.16278 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.491+0000 7fd890640740 1 --2- 172.21.15.69:0/1463376437 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x557c52963560 0x557c52965950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.491+0000 7fd890640740 4 client.16278 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.491+0000 7fd890640740 1 -- 172.21.15.69:0/1463376437 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c528b2c10 con 0x557c52963560 2024-04-24T09:08:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.492+0000 7fd886000640 1 --2- 172.21.15.69:0/1463376437 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x557c52963560 0x557c52965950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.493+0000 7fd886000640 1 --2- 172.21.15.69:0/1463376437 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x557c52963560 0x557c52965950 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93efc6f676251de server_cookie=9594d339651de019 in_seq=0 out_seq=0 2024-04-24T09:08:55.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.493+0000 7fd874c00640 10 client.16278 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:55.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.494+0000 7fd874c00640 1 -- 172.21.15.69:0/1463376437 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c528b2c10 con 0x557c52963560 2024-04-24T09:08:55.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.494+0000 7fd874c00640 10 client.16278 handle_command_reply: tid=0 2024-04-24T09:08:55.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.494+0000 7fd890640740 1 client.16278 shutdown 2024-04-24T09:08:55.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.494+0000 7fd890640740 1 -- 172.21.15.69:0/1463376437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5293fa30 msgr2=0x557c52947910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.494+0000 7fd890640740 1 --2- 172.21.15.69:0/1463376437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5293fa30 0x557c52947910 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd880009f20 tx=0x7fd880003f50 comp rx=0 tx=0).stop 2024-04-24T09:08:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.494+0000 7fd890640740 1 -- 172.21.15.69:0/1463376437 shutdown_connections 2024-04-24T09:08:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.494+0000 7fd890640740 1 --2- 172.21.15.69:0/1463376437 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x557c52963560 0x557c52965950 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.494+0000 7fd890640740 1 --2- 172.21.15.69:0/1463376437 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c5294aa20 0x557c52948390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.494+0000 7fd890640740 1 --2- 172.21.15.69:0/1463376437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c52940350 0x557c52947e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.494+0000 7fd890640740 1 --2- 172.21.15.69:0/1463376437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5293fa30 0x557c52947910 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.494+0000 7fd890640740 1 -- 172.21.15.69:0/1463376437 >> 172.21.15.69:0/1463376437 conn(0x557c5293bd50 msgr2=0x557c52941b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.494+0000 7fd874c00640 0 client.16278 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.495+0000 7fd890640740 1 -- 172.21.15.69:0/1463376437 shutdown_connections 2024-04-24T09:08:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.495+0000 7fd890640740 1 -- 172.21.15.69:0/1463376437 wait complete. 2024-04-24T09:08:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.495+0000 7fd890640740 20 client.16278 trim_cache size 0 max 16384 2024-04-24T09:08:55.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.519+0000 7fd890640740 1 Processor -- start 2024-04-24T09:08:55.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.519+0000 7fd890640740 1 -- start start 2024-04-24T09:08:55.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.519+0000 7fd890640740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5293fa30 0x557c5295eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.519+0000 7fd890640740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c52940350 0x557c529471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.519+0000 7fd890640740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c5294aa20 0x557c52947fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.519+0000 7fd890640740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881af6c0 con 0x557c5293fa30 2024-04-24T09:08:55.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.519+0000 7fd890640740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881acf90 con 0x557c52940350 2024-04-24T09:08:55.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.519+0000 7fd890640740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881aae00 con 0x557c5294aa20 2024-04-24T09:08:55.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.520+0000 7fd887400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5293fa30 0x557c5295eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.520+0000 7fd887400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5293fa30 0x557c5295eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38478/0 (socket says 172.21.15.69:38478) 2024-04-24T09:08:55.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.520+0000 7fd887400640 1 -- 172.21.15.69:0/3470229472 learned_addr learned my addr 172.21.15.69:0/3470229472 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:55.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.520+0000 7fd886000640 1 --2- 172.21.15.69:0/3470229472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c52940350 0x557c529471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.520+0000 7fd887400640 1 -- 172.21.15.69:0/3470229472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c5294aa20 msgr2=0x557c52947fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:55.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.520+0000 7fd887400640 1 --2- 172.21.15.69:0/3470229472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c5294aa20 0x557c52947fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.520+0000 7fd887400640 1 -- 172.21.15.69:0/3470229472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c52940350 msgr2=0x557c529471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:55.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.520+0000 7fd887400640 1 --2- 172.21.15.69:0/3470229472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c52940350 0x557c529471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.520+0000 7fd887400640 1 -- 172.21.15.69:0/3470229472 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c525b73f0 con 0x557c5293fa30 2024-04-24T09:08:55.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.521+0000 7fd887400640 1 --2- 172.21.15.69:0/3470229472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5293fa30 0x557c5295eee0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd880009600 tx=0x7fd880025720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bd4421ba9723a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:55.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.521+0000 7fd877400640 1 -- 172.21.15.69:0/3470229472 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88002b040 con 0x557c5293fa30 2024-04-24T09:08:55.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.521+0000 7fd877400640 1 -- 172.21.15.69:0/3470229472 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8800046b0 con 0x557c5293fa30 2024-04-24T09:08:55.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.521+0000 7fd877400640 1 -- 172.21.15.69:0/3470229472 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8800276b0 con 0x557c5293fa30 2024-04-24T09:08:55.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.522+0000 7fd890640740 1 -- 172.21.15.69:0/3470229472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5293fa30 msgr2=0x557c5295eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.522+0000 7fd890640740 1 --2- 172.21.15.69:0/3470229472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5293fa30 0x557c5295eee0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd880009600 tx=0x7fd880025720 comp rx=0 tx=0).stop 2024-04-24T09:08:55.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.522+0000 7fd890640740 1 -- 172.21.15.69:0/3470229472 shutdown_connections 2024-04-24T09:08:55.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.522+0000 7fd890640740 1 --2- 172.21.15.69:0/3470229472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c5294aa20 0x557c52947fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.522+0000 7fd890640740 1 --2- 172.21.15.69:0/3470229472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c52940350 0x557c529471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.522+0000 7fd890640740 1 --2- 172.21.15.69:0/3470229472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5293fa30 0x557c5295eee0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.522+0000 7fd890640740 1 -- 172.21.15.69:0/3470229472 >> 172.21.15.69:0/3470229472 conn(0x557c5293bd50 msgr2=0x557c529478d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:55.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.522+0000 7fd890640740 1 -- 172.21.15.69:0/3470229472 shutdown_connections 2024-04-24T09:08:55.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.522+0000 7fd890640740 1 -- 172.21.15.69:0/3470229472 wait complete. 2024-04-24T09:08:55.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.522+0000 7fd890640740 1 -- start start 2024-04-24T09:08:55.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.523+0000 7fd890640740 1 Processor -- start 2024-04-24T09:08:55.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.523+0000 7fd890640740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c5293fa30 0x557c5295b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.523+0000 7fd890640740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c52940350 0x557c529569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.523+0000 7fd890640740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5294aa20 0x557c52956f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.523+0000 7fd890640740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881ab450 con 0x557c5294aa20 2024-04-24T09:08:55.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.523+0000 7fd890640740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881ac8c0 con 0x557c52940350 2024-04-24T09:08:55.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.523+0000 7fd890640740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881abb20 con 0x557c5293fa30 2024-04-24T09:08:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.523+0000 7fd890640740 10 client.0 _ll_register_callbacks cb 0x7fd880009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.524+0000 7fd887400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5294aa20 0x557c52956f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.524+0000 7fd887400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5294aa20 0x557c52956f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38490/0 (socket says 172.21.15.69:38490) 2024-04-24T09:08:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.524+0000 7fd887400640 1 -- 172.21.15.69:0/1605519142 learned_addr learned my addr 172.21.15.69:0/1605519142 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:08:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.524+0000 7fd886000640 1 --2- 172.21.15.69:0/1605519142 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c52940350 0x557c529569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.524+0000 7fd887400640 1 --2- 172.21.15.69:0/1605519142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c5293fa30 0x557c5295b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.524+0000 7fd887400640 1 -- 172.21.15.69:0/1605519142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c5293fa30 msgr2=0x557c5295b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.524+0000 7fd887400640 1 --2- 172.21.15.69:0/1605519142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c5293fa30 0x557c5295b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.524+0000 7fd887400640 1 -- 172.21.15.69:0/1605519142 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c52940350 msgr2=0x557c529569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.524+0000 7fd887400640 1 --2- 172.21.15.69:0/1605519142 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c52940350 0x557c529569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.524+0000 7fd887400640 1 -- 172.21.15.69:0/1605519142 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c525770f0 con 0x557c5294aa20 2024-04-24T09:08:55.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.524+0000 7fd886000640 1 --2- 172.21.15.69:0/1605519142 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c52940350 0x557c529569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.525+0000 7fd887400640 1 --2- 172.21.15.69:0/1605519142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5294aa20 0x557c52956f10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd880005d70 tx=0x7fd880004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835c5a7f3b0377ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.525+0000 7fd874c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:08:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.525+0000 7fd874c00640 1 -- 172.21.15.69:0/1605519142 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880004a70 con 0x557c5294aa20 2024-04-24T09:08:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.525+0000 7fd874c00640 1 -- 172.21.15.69:0/1605519142 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd880004c10 con 0x557c5294aa20 2024-04-24T09:08:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.525+0000 7fd890640740 1 -- 172.21.15.69:0/1605519142 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8800046b0 con 0x557c5294aa20 2024-04-24T09:08:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.525+0000 7fd874c00640 1 -- 172.21.15.69:0/1605519142 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd880027470 con 0x557c5294aa20 2024-04-24T09:08:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.525+0000 7fd890640740 10 client.16284 fetch_fsmap learned FSMap version 24 2024-04-24T09:08:55.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.525+0000 7fd890640740 1 -- 172.21.15.69:0/1605519142 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557c5286a810 con 0x557c5294aa20 2024-04-24T09:08:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.526+0000 7fd874c00640 1 -- 172.21.15.69:0/1605519142 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8800046b0 con 0x557c5294aa20 2024-04-24T09:08:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.526+0000 7fd874c00640 1 -- 172.21.15.69:0/1605519142 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fd88000f720 con 0x557c5294aa20 2024-04-24T09:08:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.526+0000 7fd890640740 10 client.16284 fetch_fsmap finished waiting for FSMap version 24 2024-04-24T09:08:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.526+0000 7fd890640740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.526+0000 7fd890640740 20 client.16284 populate_metadata read hostname 'smithi069' 2024-04-24T09:08:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.526+0000 7fd890640740 1 --2- 172.21.15.69:0/1605519142 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x557c529c9830 0x557c529cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.526+0000 7fd890640740 4 client.16284 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:08:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.526+0000 7fd890640740 1 -- 172.21.15.69:0/1605519142 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c528aca80 con 0x557c529c9830 2024-04-24T09:08:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.527+0000 7fd886000640 1 --2- 172.21.15.69:0/1605519142 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x557c529c9830 0x557c529cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.528+0000 7fd886000640 1 --2- 172.21.15.69:0/1605519142 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x557c529c9830 0x557c529cbc50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=230bbccc23019e37 server_cookie=1ab54de67f45d52d in_seq=0 out_seq=0 2024-04-24T09:08:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.528+0000 7fd874c00640 10 client.16284 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.528+0000 7fd874c00640 1 -- 172.21.15.69:0/1605519142 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557c528aca80 con 0x557c529c9830 2024-04-24T09:08:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.528+0000 7fd874c00640 10 client.16284 handle_command_reply: tid=0 2024-04-24T09:08:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.528+0000 7fd890640740 1 client.16284 shutdown 2024-04-24T09:08:55.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.528+0000 7fd890640740 1 -- 172.21.15.69:0/1605519142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5294aa20 msgr2=0x557c52956f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.528+0000 7fd890640740 1 --2- 172.21.15.69:0/1605519142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5294aa20 0x557c52956f10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd880005d70 tx=0x7fd880004f50 comp rx=0 tx=0).stop 2024-04-24T09:08:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.528+0000 7fd890640740 1 -- 172.21.15.69:0/1605519142 shutdown_connections 2024-04-24T09:08:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.528+0000 7fd890640740 1 --2- 172.21.15.69:0/1605519142 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x557c529c9830 0x557c529cbc50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.528+0000 7fd890640740 1 --2- 172.21.15.69:0/1605519142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c5294aa20 0x557c52956f10 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.528+0000 7fd890640740 1 --2- 172.21.15.69:0/1605519142 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c52940350 0x557c529569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.528+0000 7fd890640740 1 --2- 172.21.15.69:0/1605519142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557c5293fa30 0x557c5295b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.528+0000 7fd874c00640 0 client.16284 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:08:55.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.528+0000 7fd890640740 1 -- 172.21.15.69:0/1605519142 >> 172.21.15.69:0/1605519142 conn(0x557c5293bd50 msgr2=0x557c52948160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:55.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.529+0000 7fd890640740 1 -- 172.21.15.69:0/1605519142 shutdown_connections 2024-04-24T09:08:55.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.529+0000 7fd890640740 1 -- 172.21.15.69:0/1605519142 wait complete. 2024-04-24T09:08:55.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.529+0000 7fd890640740 20 client.16284 trim_cache size 0 max 16384 2024-04-24T09:08:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:08:55.535 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:08:55.535 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-04-24T09:08:55.535 INFO:teuthology.orchestra.run.smithi069.stdout: "6ee0eed2-afa2-43a2-9f3b-5a58af4afe23": { 2024-04-24T09:08:55.535 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-04-24T09:08:55.535 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "6ee0eed2-afa2-43a2-9f3b-5a58af4afe23", 2024-04-24T09:08:55.535 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-04-24T09:08:55.535 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:08:55.535 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:08:55.535 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:08:55.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.530+0000 7fd887e00640 1 -- 172.21.15.69:0/3644809446 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd86c0a3da0 msgr2=0x7fd86c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.530+0000 7fd887e00640 1 --2- 172.21.15.69:0/3644809446 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd86c0a3da0 0x7fd86c0a6260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd8700ce2e0 tx=0x7fd870000bc0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.530+0000 7fd887e00640 1 -- 172.21.15.69:0/3644809446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd888104320 msgr2=0x7fd8881a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:55.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.530+0000 7fd887e00640 1 --2- 172.21.15.69:0/3644809446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd888104320 0x7fd8881a0660 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd87000e9f0 tx=0x7fd87000eeb0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.530+0000 7fd887e00640 1 -- 172.21.15.69:0/3644809446 shutdown_connections 2024-04-24T09:08:55.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.530+0000 7fd887e00640 1 --2- 172.21.15.69:0/3644809446 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd86c0a3da0 0x7fd86c0a6260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.530+0000 7fd887e00640 1 --2- 172.21.15.69:0/3644809446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd88819a720 0x7fd88819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.530+0000 7fd887e00640 1 --2- 172.21.15.69:0/3644809446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8881093f0 0x7fd8881a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.530+0000 7fd887e00640 1 --2- 172.21.15.69:0/3644809446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd888104320 0x7fd8881a0660 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.530+0000 7fd887e00640 1 -- 172.21.15.69:0/3644809446 >> 172.21.15.69:0/3644809446 conn(0x7fd8880ffb40 msgr2=0x7fd888101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.530+0000 7fd887e00640 1 -- 172.21.15.69:0/3644809446 shutdown_connections 2024-04-24T09:08:55.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:08:55.530+0000 7fd887e00640 1 -- 172.21.15.69:0/3644809446 wait complete. 2024-04-24T09:08:55.550 INFO:tasks.cephfs.filesystem:scrub status for tag:6ee0eed2-afa2-43a2-9f3b-5a58af4afe23 - {'path': '/', 'tag': '6ee0eed2-afa2-43a2-9f3b-5a58af4afe23', 'options': 'recursive,force'} 2024-04-24T09:08:55.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:55 smithi081 ceph-mon[30961]: pgmap v58: 129 pgs: 129 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-04-24T09:08:56.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:56 smithi130 ceph-mon[31553]: scrub queued for path: / 2024-04-24T09:08:56.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:56 smithi130 ceph-mon[31553]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:56.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:56 smithi130 ceph-mon[31553]: scrub summary: active paths [/] 2024-04-24T09:08:56.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:56 smithi130 ceph-mon[31553]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:56.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:56 smithi081 ceph-mon[30961]: scrub queued for path: / 2024-04-24T09:08:56.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:56 smithi081 ceph-mon[30961]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:56.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:56 smithi081 ceph-mon[30961]: scrub summary: active paths [/] 2024-04-24T09:08:56.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:56 smithi081 ceph-mon[30961]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:56.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:56 smithi069 ceph-mon[28170]: scrub queued for path: / 2024-04-24T09:08:56.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:56 smithi069 ceph-mon[28170]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:56.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:56 smithi069 ceph-mon[28170]: scrub summary: active paths [/] 2024-04-24T09:08:56.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:56 smithi069 ceph-mon[28170]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:57.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:57 smithi130 ceph-mon[31553]: pgmap v59: 129 pgs: 129 active+clean; 15 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.2 MiB/s wr, 41 op/s 2024-04-24T09:08:57.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:57 smithi081 ceph-mon[30961]: pgmap v59: 129 pgs: 129 active+clean; 15 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.2 MiB/s wr, 41 op/s 2024-04-24T09:08:57.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:57 smithi069 ceph-mon[28170]: pgmap v59: 129 pgs: 129 active+clean; 15 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.2 MiB/s wr, 41 op/s 2024-04-24T09:08:58.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:58 smithi130 ceph-mon[31553]: scrub summary: idle 2024-04-24T09:08:58.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:58 smithi081 ceph-mon[30961]: scrub summary: idle 2024-04-24T09:08:58.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:58 smithi069 ceph-mon[28170]: scrub summary: idle 2024-04-24T09:08:59.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:59 smithi130 ceph-mon[31553]: pgmap v60: 129 pgs: 129 active+clean; 15 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 1.2 MiB/s wr, 38 op/s 2024-04-24T09:08:59.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:08:59 smithi130 ceph-mon[31553]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:08:59.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:59 smithi081 ceph-mon[30961]: pgmap v60: 129 pgs: 129 active+clean; 15 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 1.2 MiB/s wr, 38 op/s 2024-04-24T09:08:59.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:08:59 smithi081 ceph-mon[30961]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:08:59.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:59 smithi069 ceph-mon[28170]: pgmap v60: 129 pgs: 129 active+clean; 15 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 1.2 MiB/s wr, 38 op/s 2024-04-24T09:08:59.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:08:59 smithi069 ceph-mon[28170]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:09:00.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:00 smithi130 ceph-mon[31553]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:09:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.491+0000 7f9512a00640 1 -- 172.21.15.69:0/2940200036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f950c06b650 msgr2=0x7f950c06ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.491+0000 7f9512a00640 1 --2- 172.21.15.69:0/2940200036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f950c06b650 0x7f950c06ba30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9500009900 tx=0x7f950002a410 comp rx=0 tx=0).stop 2024-04-24T09:09:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.491+0000 7f9512a00640 1 -- 172.21.15.69:0/2940200036 shutdown_connections 2024-04-24T09:09:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.491+0000 7f9512a00640 1 --2- 172.21.15.69:0/2940200036 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f950c116f90 0x7f950c119380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.491+0000 7f9512a00640 1 --2- 172.21.15.69:0/2940200036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f950c06bf70 0x7f950c116a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.491+0000 7f9512a00640 1 --2- 172.21.15.69:0/2940200036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f950c06b650 0x7f950c06ba30 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.491+0000 7f9512a00640 1 -- 172.21.15.69:0/2940200036 >> 172.21.15.69:0/2940200036 conn(0x7f950c06ed50 msgr2=0x7f950c071170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.491+0000 7f9512a00640 1 -- 172.21.15.69:0/2940200036 shutdown_connections 2024-04-24T09:09:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.491+0000 7f9512a00640 1 -- 172.21.15.69:0/2940200036 wait complete. 2024-04-24T09:09:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.492+0000 7f9512a00640 1 Processor -- start 2024-04-24T09:09:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.493+0000 7f9512a00640 1 -- start start 2024-04-24T09:09:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.493+0000 7f9512a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f950c06b650 0x7f950c1a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.493+0000 7f9512a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f950c06bf70 0x7f950c1a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.493+0000 7f9512a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f950c116f90 0x7f950c1a7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.493+0000 7f9512a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f950c11caf0 con 0x7f950c06bf70 2024-04-24T09:09:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.493+0000 7f9512a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f950c11c970 con 0x7f950c06b650 2024-04-24T09:09:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.493+0000 7f9512a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f950c11cc70 con 0x7f950c116f90 2024-04-24T09:09:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.493+0000 7f9510c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f950c06bf70 0x7f950c1a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.493+0000 7f9510c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f950c06bf70 0x7f950c1a35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38502/0 (socket says 172.21.15.69:38502) 2024-04-24T09:09:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.493+0000 7f9510c00640 1 -- 172.21.15.69:0/2690393987 learned_addr learned my addr 172.21.15.69:0/2690393987 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.494+0000 7f9512000640 1 --2- 172.21.15.69:0/2690393987 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f950c116f90 0x7f950c1a7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.494+0000 7f9510c00640 1 -- 172.21.15.69:0/2690393987 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f950c116f90 msgr2=0x7f950c1a7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.494+0000 7f9510c00640 1 --2- 172.21.15.69:0/2690393987 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f950c116f90 0x7f950c1a7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.494+0000 7f9510c00640 1 -- 172.21.15.69:0/2690393987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f950c06b650 msgr2=0x7f950c1a3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.494+0000 7f9510c00640 1 --2- 172.21.15.69:0/2690393987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f950c06b650 0x7f950c1a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.494+0000 7f9510c00640 1 -- 172.21.15.69:0/2690393987 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f950c1a8060 con 0x7f950c06bf70 2024-04-24T09:09:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.494+0000 7f9512000640 1 --2- 172.21.15.69:0/2690393987 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f950c116f90 0x7f950c1a7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.494+0000 7f9510c00640 1 --2- 172.21.15.69:0/2690393987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f950c06bf70 0x7f950c1a35b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f94f800c890 tx=0x7f94f800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.494+0000 7f950a000640 1 -- 172.21.15.69:0/2690393987 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8004220 con 0x7f950c06bf70 2024-04-24T09:09:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.495+0000 7f950a000640 1 -- 172.21.15.69:0/2690393987 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94f80043c0 con 0x7f950c06bf70 2024-04-24T09:09:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.495+0000 7f9512a00640 1 -- 172.21.15.69:0/2690393987 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f950c1a8350 con 0x7f950c06bf70 2024-04-24T09:09:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.495+0000 7f950a000640 1 -- 172.21.15.69:0/2690393987 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8004a00 con 0x7f950c06bf70 2024-04-24T09:09:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.495+0000 7f9512a00640 1 -- 172.21.15.69:0/2690393987 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f950c1afbe0 con 0x7f950c06bf70 2024-04-24T09:09:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.499+0000 7f9516d6a740 1 Processor -- start 2024-04-24T09:09:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.499+0000 7f9516d6a740 1 -- start start 2024-04-24T09:09:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.499+0000 7f9516d6a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 0x55558e12b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.499+0000 7f9516d6a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e12bce0 0x55558e136060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.500+0000 7f9516d6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e1365a0 0x55558e138990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.500+0000 7f9516d6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55558e091900 con 0x55558e1365a0 2024-04-24T09:09:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.500+0000 7f9516d6a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55558e037f20 con 0x55558e12b3c0 2024-04-24T09:09:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.500+0000 7f9516d6a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55558e08d8c0 con 0x55558e12bce0 2024-04-24T09:09:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.500+0000 7f950a000640 1 -- 172.21.15.69:0/2690393987 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f94f8004ba0 con 0x7f950c06bf70 2024-04-24T09:09:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.500+0000 7f9510c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e1365a0 0x55558e138990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.500+0000 7f9510c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e1365a0 0x55558e138990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38504/0 (socket says 172.21.15.69:38504) 2024-04-24T09:09:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.500+0000 7f9510c00640 1 -- 172.21.15.69:0/396560401 learned_addr learned my addr 172.21.15.69:0/396560401 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.500+0000 7f9511600640 1 --2- 172.21.15.69:0/396560401 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e12bce0 0x55558e136060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.500+0000 7f9512000640 1 --2- 172.21.15.69:0/396560401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 0x55558e12b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.501+0000 7f9510c00640 1 -- 172.21.15.69:0/396560401 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e12bce0 msgr2=0x55558e136060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.501+0000 7f9510c00640 1 --2- 172.21.15.69:0/396560401 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e12bce0 0x55558e136060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.501+0000 7f9510c00640 1 -- 172.21.15.69:0/396560401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 msgr2=0x55558e12b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.501+0000 7f9510c00640 1 --2- 172.21.15.69:0/396560401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 0x55558e12b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.501+0000 7f9510c00640 1 -- 172.21.15.69:0/396560401 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55558e05ce10 con 0x55558e1365a0 2024-04-24T09:09:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.501+0000 7f9510c00640 1 --2- 172.21.15.69:0/396560401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e1365a0 0x55558e138990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f94f80cdb60 tx=0x7f94f80d8750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc1d0de1de8ab61 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.502+0000 7f950a000640 1 --2- 172.21.15.69:0/2690393987 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f94f40a3d90 0x7f94f40a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.502+0000 7f94fe600640 1 -- 172.21.15.69:0/396560401 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8098710 con 0x55558e1365a0 2024-04-24T09:09:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.502+0000 7f94fe600640 1 -- 172.21.15.69:0/396560401 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94f80dd070 con 0x55558e1365a0 2024-04-24T09:09:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.502+0000 7f94fe600640 1 -- 172.21.15.69:0/396560401 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f809bbf0 con 0x55558e1365a0 2024-04-24T09:09:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.502+0000 7f950a000640 1 -- 172.21.15.69:0/2690393987 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f94f80d0870 con 0x7f950c06bf70 2024-04-24T09:09:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.502+0000 7f9512000640 1 --2- 172.21.15.69:0/2690393987 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f94f40a3d90 0x7f94f40a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.503+0000 7f9516d6a740 1 -- 172.21.15.69:0/396560401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e1365a0 msgr2=0x55558e138990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.503+0000 7f9512000640 1 --2- 172.21.15.69:0/2690393987 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f94f40a3d90 0x7f94f40a6250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9504005fd0 tx=0x7f95040085f0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.503+0000 7f9516d6a740 1 --2- 172.21.15.69:0/396560401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e1365a0 0x55558e138990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f94f80cdb60 tx=0x7f94f80d8750 comp rx=0 tx=0).stop 2024-04-24T09:09:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.503+0000 7f9516d6a740 1 -- 172.21.15.69:0/396560401 shutdown_connections 2024-04-24T09:09:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.503+0000 7f9516d6a740 1 --2- 172.21.15.69:0/396560401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e1365a0 0x55558e138990 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.503+0000 7f9516d6a740 1 --2- 172.21.15.69:0/396560401 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e12bce0 0x55558e136060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.503+0000 7f9516d6a740 1 --2- 172.21.15.69:0/396560401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 0x55558e12b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.503+0000 7f9516d6a740 1 -- 172.21.15.69:0/396560401 >> 172.21.15.69:0/396560401 conn(0x55558e1278d0 msgr2=0x55558e127cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.503+0000 7f9516d6a740 1 -- 172.21.15.69:0/396560401 shutdown_connections 2024-04-24T09:09:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.503+0000 7f9516d6a740 1 -- 172.21.15.69:0/396560401 wait complete. 2024-04-24T09:09:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.504+0000 7f9516d6a740 1 -- start start 2024-04-24T09:09:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.504+0000 7f9516d6a740 1 Processor -- start 2024-04-24T09:09:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.505+0000 7f9516d6a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 0x55558e133550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.505+0000 7f9516d6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e133a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.505+0000 7f9516d6a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e1365a0 0x55558e133fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.505+0000 7f9516d6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55558e1390e0 con 0x55558e12bce0 2024-04-24T09:09:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.505+0000 7f9516d6a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55558e04b2c0 con 0x55558e12b3c0 2024-04-24T09:09:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.505+0000 7f9516d6a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55558e139260 con 0x55558e1365a0 2024-04-24T09:09:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.505+0000 7f9516d6a740 10 client.0 _ll_register_callbacks cb 0x55558dda3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.505+0000 7f9510c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e133a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.505+0000 7f9510c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e133a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38510/0 (socket says 172.21.15.69:38510) 2024-04-24T09:09:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.505+0000 7f9510c00640 1 -- 172.21.15.69:0/3970892065 learned_addr learned my addr 172.21.15.69:0/3970892065 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.505+0000 7f9511600640 1 --2- 172.21.15.69:0/3970892065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 0x55558e133550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.505+0000 7f9510c00640 1 -- 172.21.15.69:0/3970892065 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e1365a0 msgr2=0x55558e133fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.505+0000 7f9510c00640 1 --2- 172.21.15.69:0/3970892065 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e1365a0 0x55558e133fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.505+0000 7f9510c00640 1 -- 172.21.15.69:0/3970892065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 msgr2=0x55558e133550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.506+0000 7f9510c00640 1 --2- 172.21.15.69:0/3970892065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 0x55558e133550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.506+0000 7f9510c00640 1 -- 172.21.15.69:0/3970892065 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55558e05bfd0 con 0x55558e12bce0 2024-04-24T09:09:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.506+0000 7f9510c00640 1 --2- 172.21.15.69:0/3970892065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e133a90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f94f8005fd0 tx=0x7f94f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564fdf7bd2746cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.506+0000 7f94f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.507+0000 7f94f2a00640 1 -- 172.21.15.69:0/3970892065 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f801d070 con 0x55558e12bce0 2024-04-24T09:09:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.507+0000 7f9516d6a740 1 -- 172.21.15.69:0/3970892065 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94f80dd070 con 0x55558e12bce0 2024-04-24T09:09:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.507+0000 7f94f2a00640 1 -- 172.21.15.69:0/3970892065 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f94f8004e90 con 0x55558e12bce0 2024-04-24T09:09:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.507+0000 7f94f2a00640 1 -- 172.21.15.69:0/3970892065 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f94f801d070 con 0x55558e12bce0 2024-04-24T09:09:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.507+0000 7f9516d6a740 10 client.16296 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.507+0000 7f9516d6a740 1 -- 172.21.15.69:0/3970892065 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55558dda4210 con 0x55558e12bce0 2024-04-24T09:09:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.508+0000 7f94f2a00640 1 -- 172.21.15.69:0/3970892065 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94f80dd070 con 0x55558e12bce0 2024-04-24T09:09:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.508+0000 7f94f2a00640 1 -- 172.21.15.69:0/3970892065 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7f94f80cdea0 con 0x55558e12bce0 2024-04-24T09:09:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.508+0000 7f9516d6a740 10 client.16296 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.508+0000 7f9516d6a740 10 client.16296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.508+0000 7f9516d6a740 20 client.16296 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.508+0000 7f9516d6a740 1 --2- 172.21.15.69:0/3970892065 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55558e1465c0 0x55558e1489b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.508+0000 7f9516d6a740 4 client.16296 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.508+0000 7f9516d6a740 1 -- 172.21.15.69:0/3970892065 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55558e09f320 con 0x55558e1465c0 2024-04-24T09:09:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.509+0000 7f9511600640 1 --2- 172.21.15.69:0/3970892065 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55558e1465c0 0x55558e1489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.510+0000 7f9511600640 1 --2- 172.21.15.69:0/3970892065 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55558e1465c0 0x55558e1489b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d96bd65107dbdc9 server_cookie=18558a81ce9aea31 in_seq=0 out_seq=0 2024-04-24T09:09:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.510+0000 7f94f2a00640 10 client.16296 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.512+0000 7f94f2a00640 1 -- 172.21.15.69:0/3970892065 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55558e09f320 con 0x55558e1465c0 2024-04-24T09:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.512+0000 7f94f2a00640 10 client.16296 handle_command_reply: tid=0 2024-04-24T09:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.512+0000 7f9516d6a740 1 client.16296 shutdown 2024-04-24T09:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.512+0000 7f9516d6a740 1 -- 172.21.15.69:0/3970892065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 msgr2=0x55558e133a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.512+0000 7f9516d6a740 1 --2- 172.21.15.69:0/3970892065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e133a90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f94f8005fd0 tx=0x7f94f80046c0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.512+0000 7f9516d6a740 1 -- 172.21.15.69:0/3970892065 shutdown_connections 2024-04-24T09:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.512+0000 7f9516d6a740 1 --2- 172.21.15.69:0/3970892065 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55558e1465c0 0x55558e1489b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.512+0000 7f9516d6a740 1 --2- 172.21.15.69:0/3970892065 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e1365a0 0x55558e133fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.512+0000 7f9516d6a740 1 --2- 172.21.15.69:0/3970892065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e133a90 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.512+0000 7f94f2a00640 0 client.16296 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.512+0000 7f9516d6a740 1 --2- 172.21.15.69:0/3970892065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 0x55558e133550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.512+0000 7f9516d6a740 1 -- 172.21.15.69:0/3970892065 >> 172.21.15.69:0/3970892065 conn(0x55558e1278d0 msgr2=0x55558e126a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.513+0000 7f9516d6a740 1 -- 172.21.15.69:0/3970892065 shutdown_connections 2024-04-24T09:09:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.513+0000 7f9516d6a740 1 -- 172.21.15.69:0/3970892065 wait complete. 2024-04-24T09:09:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.513+0000 7f9516d6a740 20 client.16296 trim_cache size 0 max 16384 2024-04-24T09:09:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.534+0000 7f9516d6a740 1 Processor -- start 2024-04-24T09:09:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.534+0000 7f9516d6a740 1 -- start start 2024-04-24T09:09:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.534+0000 7f9516d6a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 0x55558e135c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.534+0000 7f9516d6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e141d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.534+0000 7f9516d6a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e1365a0 0x55558e1307d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.535+0000 7f9516d6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f950c1bc500 con 0x55558e12bce0 2024-04-24T09:09:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.535+0000 7f9516d6a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f950c1b9da0 con 0x55558e12b3c0 2024-04-24T09:09:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.535+0000 7f9516d6a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f950c1b7bf0 con 0x55558e1365a0 2024-04-24T09:09:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.535+0000 7f9510c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e141d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.535+0000 7f9510c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e141d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38512/0 (socket says 172.21.15.69:38512) 2024-04-24T09:09:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.535+0000 7f9510c00640 1 -- 172.21.15.69:0/1808065255 learned_addr learned my addr 172.21.15.69:0/1808065255 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.535+0000 7f9512000640 1 --2- 172.21.15.69:0/1808065255 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e1365a0 0x55558e1307d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.536+0000 7f9510c00640 1 -- 172.21.15.69:0/1808065255 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e1365a0 msgr2=0x55558e1307d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.536+0000 7f9510c00640 1 --2- 172.21.15.69:0/1808065255 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e1365a0 0x55558e1307d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.536+0000 7f9510c00640 1 -- 172.21.15.69:0/1808065255 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 msgr2=0x55558e135c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.536+0000 7f9510c00640 1 --2- 172.21.15.69:0/1808065255 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 0x55558e135c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.536+0000 7f9510c00640 1 -- 172.21.15.69:0/1808065255 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55558de70620 con 0x55558e12bce0 2024-04-24T09:09:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.536+0000 7f9510c00640 1 --2- 172.21.15.69:0/1808065255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e141d00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f94f8005840 tx=0x7f94f809cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a91fe94c37bee1b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.536+0000 7f94fdc00640 1 -- 172.21.15.69:0/1808065255 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f809bdd0 con 0x55558e12bce0 2024-04-24T09:09:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.537+0000 7f94fdc00640 1 -- 172.21.15.69:0/1808065255 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94f80dd040 con 0x55558e12bce0 2024-04-24T09:09:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.537+0000 7f94fdc00640 1 -- 172.21.15.69:0/1808065255 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f809bfb0 con 0x55558e12bce0 2024-04-24T09:09:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.538+0000 7f9516d6a740 1 -- 172.21.15.69:0/1808065255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 msgr2=0x55558e141d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.538+0000 7f9516d6a740 1 --2- 172.21.15.69:0/1808065255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e141d00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f94f8005840 tx=0x7f94f809cce0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.538+0000 7f9516d6a740 1 -- 172.21.15.69:0/1808065255 shutdown_connections 2024-04-24T09:09:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.538+0000 7f9516d6a740 1 --2- 172.21.15.69:0/1808065255 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e1365a0 0x55558e1307d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.538+0000 7f9516d6a740 1 --2- 172.21.15.69:0/1808065255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e141d00 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.538+0000 7f9516d6a740 1 --2- 172.21.15.69:0/1808065255 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 0x55558e135c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.538+0000 7f9516d6a740 1 -- 172.21.15.69:0/1808065255 >> 172.21.15.69:0/1808065255 conn(0x55558e1278d0 msgr2=0x55558e1415f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.538+0000 7f9516d6a740 1 -- 172.21.15.69:0/1808065255 shutdown_connections 2024-04-24T09:09:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.538+0000 7f9516d6a740 1 -- 172.21.15.69:0/1808065255 wait complete. 2024-04-24T09:09:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.539+0000 7f9516d6a740 1 -- start start 2024-04-24T09:09:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.539+0000 7f9516d6a740 1 Processor -- start 2024-04-24T09:09:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.539+0000 7f9516d6a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 0x55558e149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.539+0000 7f9516d6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e14a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.539+0000 7f9516d6a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e1365a0 0x55558e1a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.539+0000 7f9516d6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f950c1b8260 con 0x55558e12bce0 2024-04-24T09:09:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.539+0000 7f9516d6a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f950c1b96d0 con 0x55558e12b3c0 2024-04-24T09:09:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.539+0000 7f9516d6a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f950c1b8930 con 0x55558e1365a0 2024-04-24T09:09:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.540+0000 7f9516d6a740 10 client.0 _ll_register_callbacks cb 0x7f9504008490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.540+0000 7f9510c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e14a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.540+0000 7f9510c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e14a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38524/0 (socket says 172.21.15.69:38524) 2024-04-24T09:09:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.540+0000 7f9510c00640 1 -- 172.21.15.69:0/568401689 learned_addr learned my addr 172.21.15.69:0/568401689 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.540+0000 7f9511600640 1 --2- 172.21.15.69:0/568401689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 0x55558e149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.540+0000 7f9512000640 1 --2- 172.21.15.69:0/568401689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e1365a0 0x55558e1a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.540+0000 7f9510c00640 1 -- 172.21.15.69:0/568401689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e1365a0 msgr2=0x55558e1a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.540+0000 7f9510c00640 1 --2- 172.21.15.69:0/568401689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e1365a0 0x55558e1a57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.540+0000 7f9510c00640 1 -- 172.21.15.69:0/568401689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 msgr2=0x55558e149cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.540+0000 7f9510c00640 1 --2- 172.21.15.69:0/568401689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 0x55558e149cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.540+0000 7f9510c00640 1 -- 172.21.15.69:0/568401689 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55558e1a5da0 con 0x55558e12bce0 2024-04-24T09:09:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.541+0000 7f9510c00640 1 --2- 172.21.15.69:0/568401689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e14a230 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f94f8098df0 tx=0x7f94f80ce1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac3b21e40b9308d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.541+0000 7f94f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.541+0000 7f94f2a00640 1 -- 172.21.15.69:0/568401689 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f809c240 con 0x55558e12bce0 2024-04-24T09:09:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.541+0000 7f94f2a00640 1 -- 172.21.15.69:0/568401689 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f94f809c6d0 con 0x55558e12bce0 2024-04-24T09:09:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.541+0000 7f9516d6a740 1 -- 172.21.15.69:0/568401689 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94f80dd040 con 0x55558e12bce0 2024-04-24T09:09:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.541+0000 7f94f2a00640 1 -- 172.21.15.69:0/568401689 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f94f8097fe0 con 0x55558e12bce0 2024-04-24T09:09:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.541+0000 7f9516d6a740 10 client.16302 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.541+0000 7f9516d6a740 1 -- 172.21.15.69:0/568401689 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55558e1a5da0 con 0x55558e12bce0 2024-04-24T09:09:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.541+0000 7f94f2a00640 1 -- 172.21.15.69:0/568401689 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94f80dd040 con 0x55558e12bce0 2024-04-24T09:09:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.542+0000 7f94f2a00640 1 -- 172.21.15.69:0/568401689 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7f94f80a4c50 con 0x55558e12bce0 2024-04-24T09:09:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.542+0000 7f9516d6a740 10 client.16302 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.542+0000 7f9516d6a740 10 client.16302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.542+0000 7f9516d6a740 20 client.16302 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.542+0000 7f9516d6a740 1 --2- 172.21.15.69:0/568401689 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55558e1ad430 0x55558e1af850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.542+0000 7f9516d6a740 4 client.16302 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:09:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.542+0000 7f9516d6a740 1 -- 172.21.15.69:0/568401689 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55558e099190 con 0x55558e1ad430 2024-04-24T09:09:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.544+0000 7f9511600640 1 --2- 172.21.15.69:0/568401689 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55558e1ad430 0x55558e1af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.545+0000 7f9511600640 1 --2- 172.21.15.69:0/568401689 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55558e1ad430 0x55558e1af850 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d9caedcbce8a18 server_cookie=3b6504fadcc0a1c7 in_seq=0 out_seq=0 2024-04-24T09:09:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:00.545+0000 7f94f2a00640 10 client.16302 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:00.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:00 smithi081 ceph-mon[30961]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:09:00.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:00 smithi069 ceph-mon[28170]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:09:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.194+0000 7f94f2a00640 1 -- 172.21.15.69:0/568401689 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55558e099190 con 0x55558e1ad430 2024-04-24T09:09:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.194+0000 7f94f2a00640 10 client.16302 handle_command_reply: tid=0 2024-04-24T09:09:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.194+0000 7f9516d6a740 1 client.16302 shutdown 2024-04-24T09:09:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.194+0000 7f9516d6a740 1 -- 172.21.15.69:0/568401689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 msgr2=0x55558e14a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.194+0000 7f9516d6a740 1 --2- 172.21.15.69:0/568401689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e14a230 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f94f8098df0 tx=0x7f94f80ce1d0 comp rx=0 tx=0).stop 2024-04-24T09:09:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.194+0000 7f9516d6a740 1 -- 172.21.15.69:0/568401689 shutdown_connections 2024-04-24T09:09:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.194+0000 7f9516d6a740 1 --2- 172.21.15.69:0/568401689 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55558e1ad430 0x55558e1af850 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.194+0000 7f9516d6a740 1 --2- 172.21.15.69:0/568401689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55558e1365a0 0x55558e1a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.194+0000 7f94f2a00640 0 client.16302 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.194+0000 7f9516d6a740 1 --2- 172.21.15.69:0/568401689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55558e12bce0 0x55558e14a230 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.194+0000 7f9516d6a740 1 --2- 172.21.15.69:0/568401689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55558e12b3c0 0x55558e149cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.194+0000 7f9516d6a740 1 -- 172.21.15.69:0/568401689 >> 172.21.15.69:0/568401689 conn(0x55558e1278d0 msgr2=0x55558e1334f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.194+0000 7f9516d6a740 1 -- 172.21.15.69:0/568401689 shutdown_connections 2024-04-24T09:09:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.194+0000 7f9516d6a740 1 -- 172.21.15.69:0/568401689 wait complete. 2024-04-24T09:09:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.195+0000 7f9516d6a740 20 client.16302 trim_cache size 0 max 16384 2024-04-24T09:09:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:09:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:09:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:09:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:09:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.195+0000 7f9512a00640 1 -- 172.21.15.69:0/2690393987 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f94f40a3d90 msgr2=0x7f94f40a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.195+0000 7f9512a00640 1 --2- 172.21.15.69:0/2690393987 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f94f40a3d90 0x7f94f40a6250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9504005fd0 tx=0x7f95040085f0 comp rx=0 tx=0).stop 2024-04-24T09:09:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.195+0000 7f9512a00640 1 -- 172.21.15.69:0/2690393987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f950c06bf70 msgr2=0x7f950c1a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.195+0000 7f9512a00640 1 --2- 172.21.15.69:0/2690393987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f950c06bf70 0x7f950c1a35b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f94f800c890 tx=0x7f94f800cd50 comp rx=0 tx=0).stop 2024-04-24T09:09:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.195+0000 7f9512a00640 1 -- 172.21.15.69:0/2690393987 shutdown_connections 2024-04-24T09:09:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.195+0000 7f9512a00640 1 --2- 172.21.15.69:0/2690393987 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f94f40a3d90 0x7f94f40a6250 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.195+0000 7f9512a00640 1 --2- 172.21.15.69:0/2690393987 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f950c116f90 0x7f950c1a7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.195+0000 7f9512a00640 1 --2- 172.21.15.69:0/2690393987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f950c06bf70 0x7f950c1a35b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.195+0000 7f9512a00640 1 --2- 172.21.15.69:0/2690393987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f950c06b650 0x7f950c1a3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.196+0000 7f9512a00640 1 -- 172.21.15.69:0/2690393987 >> 172.21.15.69:0/2690393987 conn(0x7f950c06ed50 msgr2=0x7f950c070720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.196+0000 7f9512a00640 1 -- 172.21.15.69:0/2690393987 shutdown_connections 2024-04-24T09:09:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:01.196+0000 7f9512a00640 1 -- 172.21.15.69:0/2690393987 wait complete. 2024-04-24T09:09:01.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:01 smithi130 ceph-mon[31553]: pgmap v61: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.5 MiB/s wr, 88 op/s 2024-04-24T09:09:01.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:01 smithi081 ceph-mon[30961]: pgmap v61: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.5 MiB/s wr, 88 op/s 2024-04-24T09:09:01.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:01 smithi069 ceph-mon[28170]: pgmap v61: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.5 MiB/s wr, 88 op/s 2024-04-24T09:09:03.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:03 smithi130 ceph-mon[31553]: pgmap v62: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-04-24T09:09:03.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:03 smithi081 ceph-mon[30961]: pgmap v62: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-04-24T09:09:03.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:03 smithi069 ceph-mon[28170]: pgmap v62: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-04-24T09:09:05.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:05 smithi130 ceph-mon[31553]: pgmap v63: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-04-24T09:09:05.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:05 smithi081 ceph-mon[30961]: pgmap v63: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-04-24T09:09:05.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:05 smithi069 ceph-mon[28170]: pgmap v63: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-04-24T09:09:07.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:07 smithi130 ceph-mon[31553]: pgmap v64: 129 pgs: 129 active+clean; 22 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.8 MiB/s wr, 265 op/s 2024-04-24T09:09:07.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:07 smithi081 ceph-mon[30961]: pgmap v64: 129 pgs: 129 active+clean; 22 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.8 MiB/s wr, 265 op/s 2024-04-24T09:09:07.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:07 smithi069 ceph-mon[28170]: pgmap v64: 129 pgs: 129 active+clean; 22 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.8 MiB/s wr, 265 op/s 2024-04-24T09:09:09.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:09 smithi081 ceph-mon[30961]: pgmap v65: 129 pgs: 129 active+clean; 22 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 652 KiB/s wr, 249 op/s 2024-04-24T09:09:09.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:09 smithi069 ceph-mon[28170]: pgmap v65: 129 pgs: 129 active+clean; 22 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 652 KiB/s wr, 249 op/s 2024-04-24T09:09:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:09 smithi130 ceph-mon[31553]: pgmap v65: 129 pgs: 129 active+clean; 22 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 652 KiB/s wr, 249 op/s 2024-04-24T09:09:11.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:11 smithi081 ceph-mon[30961]: pgmap v66: 129 pgs: 129 active+clean; 24 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 858 KiB/s wr, 275 op/s 2024-04-24T09:09:11.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:11 smithi069 ceph-mon[28170]: pgmap v66: 129 pgs: 129 active+clean; 24 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 858 KiB/s wr, 275 op/s 2024-04-24T09:09:11.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:11 smithi130 ceph-mon[31553]: pgmap v66: 129 pgs: 129 active+clean; 24 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 858 KiB/s wr, 275 op/s 2024-04-24T09:09:13.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:13 smithi081 ceph-mon[30961]: pgmap v67: 129 pgs: 129 active+clean; 24 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 507 KiB/s wr, 225 op/s 2024-04-24T09:09:13.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:13 smithi069 ceph-mon[28170]: pgmap v67: 129 pgs: 129 active+clean; 24 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 507 KiB/s wr, 225 op/s 2024-04-24T09:09:13.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:13 smithi130 ceph-mon[31553]: pgmap v67: 129 pgs: 129 active+clean; 24 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 507 KiB/s wr, 225 op/s 2024-04-24T09:09:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.351+0000 7f273fe00640 1 -- 172.21.15.69:0/1598371439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2740074cf0 msgr2=0x7f274010e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.351+0000 7f273fe00640 1 --2- 172.21.15.69:0/1598371439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2740074cf0 0x7f274010e180 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2728009900 tx=0x7f272802a480 comp rx=0 tx=0).stop 2024-04-24T09:09:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.351+0000 7f273fe00640 1 -- 172.21.15.69:0/1598371439 shutdown_connections 2024-04-24T09:09:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.351+0000 7f273fe00640 1 --2- 172.21.15.69:0/1598371439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f274010e6c0 0x7f2740110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.351+0000 7f273fe00640 1 --2- 172.21.15.69:0/1598371439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2740074cf0 0x7f274010e180 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.351+0000 7f273fe00640 1 --2- 172.21.15.69:0/1598371439 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27400743d0 0x7f27400747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.351+0000 7f273fe00640 1 -- 172.21.15.69:0/1598371439 >> 172.21.15.69:0/1598371439 conn(0x7f27400fd950 msgr2=0x7f27400ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.351+0000 7f273fe00640 1 -- 172.21.15.69:0/1598371439 shutdown_connections 2024-04-24T09:09:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.351+0000 7f273fe00640 1 -- 172.21.15.69:0/1598371439 wait complete. 2024-04-24T09:09:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.352+0000 7f273fe00640 1 Processor -- start 2024-04-24T09:09:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.352+0000 7f273fe00640 1 -- start start 2024-04-24T09:09:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.353+0000 7f273fe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27400743d0 0x7f27401a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.353+0000 7f273fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2740074cf0 0x7f27401a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.353+0000 7f273fe00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f274010e6c0 0x7f274019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.353+0000 7f273fe00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2740114340 con 0x7f2740074cf0 2024-04-24T09:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.353+0000 7f273fe00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f27401141c0 con 0x7f274010e6c0 2024-04-24T09:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.353+0000 7f273fe00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f27401144c0 con 0x7f27400743d0 2024-04-24T09:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.353+0000 7f2736000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2740074cf0 0x7f27401a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.353+0000 7f2736000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2740074cf0 0x7f27401a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54102/0 (socket says 172.21.15.69:54102) 2024-04-24T09:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.353+0000 7f2736000640 1 -- 172.21.15.69:0/1281791501 learned_addr learned my addr 172.21.15.69:0/1281791501 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.353+0000 7f273ea00640 1 --2- 172.21.15.69:0/1281791501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27400743d0 0x7f27401a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.354+0000 7f273f400640 1 --2- 172.21.15.69:0/1281791501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f274010e6c0 0x7f274019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.354+0000 7f2736000640 1 -- 172.21.15.69:0/1281791501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27400743d0 msgr2=0x7f27401a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.354+0000 7f2736000640 1 --2- 172.21.15.69:0/1281791501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27400743d0 0x7f27401a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.354+0000 7f2736000640 1 -- 172.21.15.69:0/1281791501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f274010e6c0 msgr2=0x7f274019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.354+0000 7f2736000640 1 --2- 172.21.15.69:0/1281791501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f274010e6c0 0x7f274019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.354+0000 7f2736000640 1 -- 172.21.15.69:0/1281791501 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274019afa0 con 0x7f2740074cf0 2024-04-24T09:09:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.354+0000 7f2736000640 1 --2- 172.21.15.69:0/1281791501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2740074cf0 0x7f27401a0b60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2728005ab0 tx=0x7f2728003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.354+0000 7f2737e00640 1 -- 172.21.15.69:0/1281791501 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272803f070 con 0x7f2740074cf0 2024-04-24T09:09:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.354+0000 7f273fe00640 1 -- 172.21.15.69:0/1281791501 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274019b230 con 0x7f2740074cf0 2024-04-24T09:09:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.354+0000 7f2737e00640 1 -- 172.21.15.69:0/1281791501 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2728031d60 con 0x7f2740074cf0 2024-04-24T09:09:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.354+0000 7f2737e00640 1 -- 172.21.15.69:0/1281791501 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728035030 con 0x7f2740074cf0 2024-04-24T09:09:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.355+0000 7f273fe00640 1 -- 172.21.15.69:0/1281791501 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27401a75a0 con 0x7f2740074cf0 2024-04-24T09:09:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.359+0000 7f2748679740 1 Processor -- start 2024-04-24T09:09:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.359+0000 7f2737e00640 1 -- 172.21.15.69:0/1281791501 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2728042070 con 0x7f2740074cf0 2024-04-24T09:09:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.359+0000 7f2748679740 1 -- start start 2024-04-24T09:09:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.359+0000 7f2748679740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca383c0 0x55c69ca387a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.359+0000 7f2748679740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca38ce0 0x55c69ca43060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.359+0000 7f2748679740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca435a0 0x55c69ca45990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.359+0000 7f2748679740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c69c99e900 con 0x55c69ca435a0 2024-04-24T09:09:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.359+0000 7f2748679740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55c69c944f20 con 0x55c69ca383c0 2024-04-24T09:09:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.359+0000 7f2748679740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c69c99a8c0 con 0x55c69ca38ce0 2024-04-24T09:09:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.360+0000 7f2736000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca435a0 0x55c69ca45990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.360+0000 7f2736000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca435a0 0x55c69ca45990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54112/0 (socket says 172.21.15.69:54112) 2024-04-24T09:09:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.360+0000 7f2736000640 1 -- 172.21.15.69:0/1735276431 learned_addr learned my addr 172.21.15.69:0/1735276431 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.360+0000 7f273ea00640 1 --2- 172.21.15.69:0/1735276431 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca38ce0 0x55c69ca43060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.360+0000 7f2736000640 1 -- 172.21.15.69:0/1735276431 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca38ce0 msgr2=0x55c69ca43060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.360+0000 7f2736000640 1 --2- 172.21.15.69:0/1735276431 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca38ce0 0x55c69ca43060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.360+0000 7f2736000640 1 -- 172.21.15.69:0/1735276431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca383c0 msgr2=0x55c69ca387a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.360+0000 7f2736000640 1 --2- 172.21.15.69:0/1735276431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca383c0 0x55c69ca387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.360+0000 7f2736000640 1 -- 172.21.15.69:0/1735276431 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c69c969e10 con 0x55c69ca435a0 2024-04-24T09:09:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.360+0000 7f2737e00640 1 --2- 172.21.15.69:0/1281791501 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f27240a3d10 0x7f27240a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.360+0000 7f2736000640 1 --2- 172.21.15.69:0/1735276431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca435a0 0x55c69ca45990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f27280ee4c0 tx=0x7f27280f9540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de38f452e1808d6d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.361+0000 7f273f400640 1 --2- 172.21.15.69:0/1281791501 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f27240a3d10 0x7f27240a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.361+0000 7f2737e00640 1 -- 172.21.15.69:0/1281791501 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f27280f1140 con 0x7f2740074cf0 2024-04-24T09:09:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.361+0000 7f272fe00640 1 -- 172.21.15.69:0/1735276431 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27280b9040 con 0x55c69ca435a0 2024-04-24T09:09:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.361+0000 7f272fe00640 1 -- 172.21.15.69:0/1735276431 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27280bd4a0 con 0x55c69ca435a0 2024-04-24T09:09:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.361+0000 7f272fe00640 1 -- 172.21.15.69:0/1735276431 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27280bc700 con 0x55c69ca435a0 2024-04-24T09:09:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.362+0000 7f2748679740 1 -- 172.21.15.69:0/1735276431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca435a0 msgr2=0x55c69ca45990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.362+0000 7f2748679740 1 --2- 172.21.15.69:0/1735276431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca435a0 0x55c69ca45990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f27280ee4c0 tx=0x7f27280f9540 comp rx=0 tx=0).stop 2024-04-24T09:09:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.362+0000 7f273f400640 1 --2- 172.21.15.69:0/1281791501 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f27240a3d10 0x7f27240a61d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2738005fd0 tx=0x7f2738005cf0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.362+0000 7f2748679740 1 -- 172.21.15.69:0/1735276431 shutdown_connections 2024-04-24T09:09:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.362+0000 7f2748679740 1 --2- 172.21.15.69:0/1735276431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca435a0 0x55c69ca45990 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.362+0000 7f2748679740 1 --2- 172.21.15.69:0/1735276431 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca38ce0 0x55c69ca43060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.362+0000 7f2748679740 1 --2- 172.21.15.69:0/1735276431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca383c0 0x55c69ca387a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.362+0000 7f2748679740 1 -- 172.21.15.69:0/1735276431 >> 172.21.15.69:0/1735276431 conn(0x55c69ca348d0 msgr2=0x55c69ca34cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.362+0000 7f2748679740 1 -- 172.21.15.69:0/1735276431 shutdown_connections 2024-04-24T09:09:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.362+0000 7f2748679740 1 -- 172.21.15.69:0/1735276431 wait complete. 2024-04-24T09:09:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.363+0000 7f2748679740 1 -- start start 2024-04-24T09:09:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.363+0000 7f2748679740 1 Processor -- start 2024-04-24T09:09:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.363+0000 7f2748679740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca383c0 0x55c69ca40540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.363+0000 7f2748679740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca38ce0 0x55c69ca40a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.363+0000 7f2748679740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca435a0 0x55c69ca40fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.363+0000 7f2748679740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c69ca45f90 con 0x55c69ca38ce0 2024-04-24T09:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.363+0000 7f2748679740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55c69c9582c0 con 0x55c69ca435a0 2024-04-24T09:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.363+0000 7f2748679740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c69ca46110 con 0x55c69ca383c0 2024-04-24T09:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.364+0000 7f2748679740 10 client.0 _ll_register_callbacks cb 0x55c69c6b0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.364+0000 7f2736000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca38ce0 0x55c69ca40a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.364+0000 7f2736000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca38ce0 0x55c69ca40a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54120/0 (socket says 172.21.15.69:54120) 2024-04-24T09:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.364+0000 7f2736000640 1 -- 172.21.15.69:0/3352110352 learned_addr learned my addr 172.21.15.69:0/3352110352 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.364+0000 7f273f400640 1 --2- 172.21.15.69:0/3352110352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca435a0 0x55c69ca40fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.364+0000 7f273ea00640 1 --2- 172.21.15.69:0/3352110352 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca383c0 0x55c69ca40540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.364+0000 7f2736000640 1 -- 172.21.15.69:0/3352110352 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca383c0 msgr2=0x55c69ca40540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.364+0000 7f2736000640 1 --2- 172.21.15.69:0/3352110352 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca383c0 0x55c69ca40540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.364+0000 7f2736000640 1 -- 172.21.15.69:0/3352110352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca435a0 msgr2=0x55c69ca40fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.364+0000 7f2736000640 1 --2- 172.21.15.69:0/3352110352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca435a0 0x55c69ca40fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.364+0000 7f2736000640 1 -- 172.21.15.69:0/3352110352 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c69c968fd0 con 0x55c69ca38ce0 2024-04-24T09:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.364+0000 7f2736000640 1 --2- 172.21.15.69:0/3352110352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca38ce0 0x55c69ca40a80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2728005ec0 tx=0x7f2728031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03acf1a68fde3a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.365+0000 7f272cc00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.365+0000 7f272cc00640 1 -- 172.21.15.69:0/3352110352 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728049070 con 0x55c69ca38ce0 2024-04-24T09:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.365+0000 7f272cc00640 1 -- 172.21.15.69:0/3352110352 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f27280f9ce0 con 0x55c69ca38ce0 2024-04-24T09:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.365+0000 7f2748679740 1 -- 172.21.15.69:0/3352110352 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27280bd4a0 con 0x55c69ca38ce0 2024-04-24T09:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.365+0000 7f272cc00640 1 -- 172.21.15.69:0/3352110352 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f27280bc8e0 con 0x55c69ca38ce0 2024-04-24T09:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.365+0000 7f2748679740 10 client.16314 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.365+0000 7f2748679740 1 -- 172.21.15.69:0/3352110352 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c69c6b1210 con 0x55c69ca38ce0 2024-04-24T09:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.365+0000 7f272cc00640 1 -- 172.21.15.69:0/3352110352 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27280bd4a0 con 0x55c69ca38ce0 2024-04-24T09:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.366+0000 7f272cc00640 1 -- 172.21.15.69:0/3352110352 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7f27280d3ce0 con 0x55c69ca38ce0 2024-04-24T09:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.366+0000 7f2748679740 10 client.16314 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.366+0000 7f2748679740 10 client.16314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.366+0000 7f2748679740 20 client.16314 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.366+0000 7f2748679740 1 --2- 172.21.15.69:0/3352110352 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c69ca532c0 0x55c69ca556b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.366+0000 7f2748679740 4 client.16314 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:14.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.366+0000 7f2748679740 1 -- 172.21.15.69:0/3352110352 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c69c9ac320 con 0x55c69ca532c0 2024-04-24T09:09:14.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.367+0000 7f273ea00640 1 --2- 172.21.15.69:0/3352110352 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c69ca532c0 0x55c69ca556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.368+0000 7f272cc00640 10 client.16314 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.368+0000 7f273ea00640 1 --2- 172.21.15.69:0/3352110352 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c69ca532c0 0x55c69ca556b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddcab61ad7cbe8cc server_cookie=e9b74d0edb6eb535 in_seq=0 out_seq=0 2024-04-24T09:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.369+0000 7f272cc00640 1 -- 172.21.15.69:0/3352110352 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c69c9ac320 con 0x55c69ca532c0 2024-04-24T09:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.369+0000 7f272cc00640 10 client.16314 handle_command_reply: tid=0 2024-04-24T09:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.369+0000 7f2748679740 1 client.16314 shutdown 2024-04-24T09:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.370+0000 7f2748679740 1 -- 172.21.15.69:0/3352110352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca38ce0 msgr2=0x55c69ca40a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.370+0000 7f2748679740 1 --2- 172.21.15.69:0/3352110352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca38ce0 0x55c69ca40a80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2728005ec0 tx=0x7f2728031f10 comp rx=0 tx=0).stop 2024-04-24T09:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.370+0000 7f2748679740 1 -- 172.21.15.69:0/3352110352 shutdown_connections 2024-04-24T09:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.370+0000 7f2748679740 1 --2- 172.21.15.69:0/3352110352 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c69ca532c0 0x55c69ca556b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.370+0000 7f2748679740 1 --2- 172.21.15.69:0/3352110352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca435a0 0x55c69ca40fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.370+0000 7f2748679740 1 --2- 172.21.15.69:0/3352110352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca38ce0 0x55c69ca40a80 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.370+0000 7f2748679740 1 --2- 172.21.15.69:0/3352110352 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca383c0 0x55c69ca40540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.370+0000 7f272cc00640 0 client.16314 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.370+0000 7f2748679740 1 -- 172.21.15.69:0/3352110352 >> 172.21.15.69:0/3352110352 conn(0x55c69ca348d0 msgr2=0x55c69ca33a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.370+0000 7f2748679740 1 -- 172.21.15.69:0/3352110352 shutdown_connections 2024-04-24T09:09:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.370+0000 7f2748679740 1 -- 172.21.15.69:0/3352110352 wait complete. 2024-04-24T09:09:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.371+0000 7f2748679740 20 client.16314 trim_cache size 0 max 16384 2024-04-24T09:09:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.392+0000 7f2748679740 1 Processor -- start 2024-04-24T09:09:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.392+0000 7f2748679740 1 -- start start 2024-04-24T09:09:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.392+0000 7f2748679740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca383c0 0x55c69ca4eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.392+0000 7f2748679740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca38ce0 0x55c69ca33910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.392+0000 7f2748679740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca435a0 0x55c69ca40de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.392+0000 7f2748679740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f27401b3c80 con 0x55c69ca38ce0 2024-04-24T09:09:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.392+0000 7f2748679740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f27401b1520 con 0x55c69ca435a0 2024-04-24T09:09:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.392+0000 7f2748679740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f27401af370 con 0x55c69ca383c0 2024-04-24T09:09:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.392+0000 7f2736000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca38ce0 0x55c69ca33910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.392+0000 7f2736000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca38ce0 0x55c69ca33910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54132/0 (socket says 172.21.15.69:54132) 2024-04-24T09:09:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.392+0000 7f2736000640 1 -- 172.21.15.69:0/2720924039 learned_addr learned my addr 172.21.15.69:0/2720924039 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.393+0000 7f273ea00640 1 --2- 172.21.15.69:0/2720924039 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca383c0 0x55c69ca4eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.393+0000 7f273f400640 1 --2- 172.21.15.69:0/2720924039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca435a0 0x55c69ca40de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.393+0000 7f2736000640 1 -- 172.21.15.69:0/2720924039 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca383c0 msgr2=0x55c69ca4eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.393+0000 7f2736000640 1 --2- 172.21.15.69:0/2720924039 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca383c0 0x55c69ca4eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.393+0000 7f2736000640 1 -- 172.21.15.69:0/2720924039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca435a0 msgr2=0x55c69ca40de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.393+0000 7f2736000640 1 --2- 172.21.15.69:0/2720924039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca435a0 0x55c69ca40de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.393+0000 7f2736000640 1 -- 172.21.15.69:0/2720924039 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c69ca38b10 con 0x55c69ca38ce0 2024-04-24T09:09:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.393+0000 7f2736000640 1 --2- 172.21.15.69:0/2720924039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca38ce0 0x55c69ca33910 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f27280bb050 tx=0x7f27280f9540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802882eb46b667ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.394+0000 7f272f400640 1 -- 172.21.15.69:0/2720924039 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728049070 con 0x55c69ca38ce0 2024-04-24T09:09:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.394+0000 7f272f400640 1 -- 172.21.15.69:0/2720924039 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f272803be50 con 0x55c69ca38ce0 2024-04-24T09:09:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.394+0000 7f272f400640 1 -- 172.21.15.69:0/2720924039 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27280bcb70 con 0x55c69ca38ce0 2024-04-24T09:09:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.394+0000 7f2748679740 1 -- 172.21.15.69:0/2720924039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca38ce0 msgr2=0x55c69ca33910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.394+0000 7f2748679740 1 --2- 172.21.15.69:0/2720924039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca38ce0 0x55c69ca33910 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f27280bb050 tx=0x7f27280f9540 comp rx=0 tx=0).stop 2024-04-24T09:09:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.395+0000 7f2748679740 1 -- 172.21.15.69:0/2720924039 shutdown_connections 2024-04-24T09:09:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.395+0000 7f2748679740 1 --2- 172.21.15.69:0/2720924039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca435a0 0x55c69ca40de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.395+0000 7f2748679740 1 --2- 172.21.15.69:0/2720924039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca38ce0 0x55c69ca33910 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.395+0000 7f2748679740 1 --2- 172.21.15.69:0/2720924039 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca383c0 0x55c69ca4eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.395+0000 7f2748679740 1 -- 172.21.15.69:0/2720924039 >> 172.21.15.69:0/2720924039 conn(0x55c69ca348d0 msgr2=0x55c69ca33200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.395+0000 7f2748679740 1 -- 172.21.15.69:0/2720924039 shutdown_connections 2024-04-24T09:09:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.395+0000 7f2748679740 1 -- 172.21.15.69:0/2720924039 wait complete. 2024-04-24T09:09:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.395+0000 7f2748679740 1 -- start start 2024-04-24T09:09:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.395+0000 7f2748679740 1 Processor -- start 2024-04-24T09:09:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.396+0000 7f2748679740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca383c0 0x55c69ca4b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.396+0000 7f2748679740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca38ce0 0x55c69ca3cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.396+0000 7f2748679740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca435a0 0x55c69ca3d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.396+0000 7f2748679740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f27401af9e0 con 0x55c69ca435a0 2024-04-24T09:09:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.396+0000 7f2748679740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f27401b0e50 con 0x55c69ca38ce0 2024-04-24T09:09:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.396+0000 7f2748679740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f27401b00b0 con 0x55c69ca383c0 2024-04-24T09:09:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.396+0000 7f2748679740 10 client.0 _ll_register_callbacks cb 0x7f27280f9300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.396+0000 7f273f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca435a0 0x55c69ca3d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.397+0000 7f273f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca435a0 0x55c69ca3d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54136/0 (socket says 172.21.15.69:54136) 2024-04-24T09:09:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.397+0000 7f273f400640 1 -- 172.21.15.69:0/1338256458 learned_addr learned my addr 172.21.15.69:0/1338256458 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.397+0000 7f273ea00640 1 --2- 172.21.15.69:0/1338256458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca383c0 0x55c69ca4b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.397+0000 7f2736000640 1 --2- 172.21.15.69:0/1338256458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca38ce0 0x55c69ca3cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.397+0000 7f273f400640 1 -- 172.21.15.69:0/1338256458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca383c0 msgr2=0x55c69ca4b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.397+0000 7f273f400640 1 --2- 172.21.15.69:0/1338256458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca383c0 0x55c69ca4b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.397+0000 7f273f400640 1 -- 172.21.15.69:0/1338256458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca38ce0 msgr2=0x55c69ca3cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.397+0000 7f273f400640 1 --2- 172.21.15.69:0/1338256458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca38ce0 0x55c69ca3cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.397+0000 7f273f400640 1 -- 172.21.15.69:0/1338256458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c69c77d620 con 0x55c69ca435a0 2024-04-24T09:09:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.397+0000 7f273f400640 1 --2- 172.21.15.69:0/1338256458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca435a0 0x55c69ca3d520 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2738005d80 tx=0x7f2738004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa057512707f0a70 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.397+0000 7f272cc00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.398+0000 7f272cc00640 1 -- 172.21.15.69:0/1338256458 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738020850 con 0x55c69ca435a0 2024-04-24T09:09:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.398+0000 7f272cc00640 1 -- 172.21.15.69:0/1338256458 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f27380209f0 con 0x55c69ca435a0 2024-04-24T09:09:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.398+0000 7f2748679740 1 -- 172.21.15.69:0/1338256458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f272803be50 con 0x55c69ca435a0 2024-04-24T09:09:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.398+0000 7f272cc00640 1 -- 172.21.15.69:0/1338256458 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2738032050 con 0x55c69ca435a0 2024-04-24T09:09:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.398+0000 7f2748679740 10 client.16320 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.398+0000 7f2748679740 1 -- 172.21.15.69:0/1338256458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c69cac1df0 con 0x55c69ca435a0 2024-04-24T09:09:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.399+0000 7f272cc00640 1 -- 172.21.15.69:0/1338256458 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f272803be50 con 0x55c69ca435a0 2024-04-24T09:09:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.399+0000 7f272cc00640 1 -- 172.21.15.69:0/1338256458 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7f273801fb50 con 0x55c69ca435a0 2024-04-24T09:09:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.399+0000 7f2748679740 10 client.16320 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.399+0000 7f2748679740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.399+0000 7f2748679740 20 client.16320 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.399+0000 7f2748679740 1 --2- 172.21.15.69:0/1338256458 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c69cac2600 0x55c69cac4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.399+0000 7f2748679740 4 client.16320 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:09:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.399+0000 7f2748679740 1 -- 172.21.15.69:0/1338256458 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c69c9a6190 con 0x55c69cac2600 2024-04-24T09:09:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.400+0000 7f273ea00640 1 --2- 172.21.15.69:0/1338256458 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c69cac2600 0x55c69cac4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.400+0000 7f273ea00640 1 --2- 172.21.15.69:0/1338256458 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c69cac2600 0x55c69cac4a20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ba82d076da92edf server_cookie=be6a75d68515a403 in_seq=0 out_seq=0 2024-04-24T09:09:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.400+0000 7f272cc00640 10 client.16320 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.455+0000 7f272cc00640 1 -- 172.21.15.69:0/1338256458 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c69c9a6190 con 0x55c69cac2600 2024-04-24T09:09:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.455+0000 7f272cc00640 10 client.16320 handle_command_reply: tid=0 2024-04-24T09:09:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.455+0000 7f2748679740 1 client.16320 shutdown 2024-04-24T09:09:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.455+0000 7f2748679740 1 -- 172.21.15.69:0/1338256458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca435a0 msgr2=0x55c69ca3d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.455+0000 7f2748679740 1 --2- 172.21.15.69:0/1338256458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca435a0 0x55c69ca3d520 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2738005d80 tx=0x7f2738004730 comp rx=0 tx=0).stop 2024-04-24T09:09:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.455+0000 7f2748679740 1 -- 172.21.15.69:0/1338256458 shutdown_connections 2024-04-24T09:09:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.455+0000 7f2748679740 1 --2- 172.21.15.69:0/1338256458 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c69cac2600 0x55c69cac4a20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.455+0000 7f2748679740 1 --2- 172.21.15.69:0/1338256458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c69ca435a0 0x55c69ca3d520 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.455+0000 7f2748679740 1 --2- 172.21.15.69:0/1338256458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c69ca38ce0 0x55c69ca3cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.455+0000 7f2748679740 1 --2- 172.21.15.69:0/1338256458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c69ca383c0 0x55c69ca4b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.455+0000 7f272cc00640 0 client.16320 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.455+0000 7f2748679740 1 -- 172.21.15.69:0/1338256458 >> 172.21.15.69:0/1338256458 conn(0x55c69ca348d0 msgr2=0x55c69ca33a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.456+0000 7f2748679740 1 -- 172.21.15.69:0/1338256458 shutdown_connections 2024-04-24T09:09:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.456+0000 7f2748679740 1 -- 172.21.15.69:0/1338256458 wait complete. 2024-04-24T09:09:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.456+0000 7f2748679740 20 client.16320 trim_cache size 0 max 16384 2024-04-24T09:09:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:09:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:09:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:09:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:09:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.457+0000 7f273fe00640 1 -- 172.21.15.69:0/1281791501 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f27240a3d10 msgr2=0x7f27240a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.457+0000 7f273fe00640 1 --2- 172.21.15.69:0/1281791501 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f27240a3d10 0x7f27240a61d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2738005fd0 tx=0x7f2738005cf0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.457+0000 7f273fe00640 1 -- 172.21.15.69:0/1281791501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2740074cf0 msgr2=0x7f27401a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.457+0000 7f273fe00640 1 --2- 172.21.15.69:0/1281791501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2740074cf0 0x7f27401a0b60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2728005ab0 tx=0x7f2728003cd0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.457+0000 7f273fe00640 1 -- 172.21.15.69:0/1281791501 shutdown_connections 2024-04-24T09:09:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.457+0000 7f273fe00640 1 --2- 172.21.15.69:0/1281791501 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f27240a3d10 0x7f27240a61d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.457+0000 7f273fe00640 1 --2- 172.21.15.69:0/1281791501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f274010e6c0 0x7f274019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.457+0000 7f273fe00640 1 --2- 172.21.15.69:0/1281791501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2740074cf0 0x7f27401a0b60 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.457+0000 7f273fe00640 1 --2- 172.21.15.69:0/1281791501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27400743d0 0x7f27401a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.457+0000 7f273fe00640 1 -- 172.21.15.69:0/1281791501 >> 172.21.15.69:0/1281791501 conn(0x7f27400fd950 msgr2=0x7f27400fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:14.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.457+0000 7f273fe00640 1 -- 172.21.15.69:0/1281791501 shutdown_connections 2024-04-24T09:09:14.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:14.457+0000 7f273fe00640 1 -- 172.21.15.69:0/1281791501 wait complete. 2024-04-24T09:09:15.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:15 smithi081 ceph-mon[30961]: pgmap v68: 129 pgs: 129 active+clean; 24 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 507 KiB/s wr, 225 op/s 2024-04-24T09:09:15.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:15 smithi069 ceph-mon[28170]: pgmap v68: 129 pgs: 129 active+clean; 24 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 507 KiB/s wr, 225 op/s 2024-04-24T09:09:15.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:15 smithi130 ceph-mon[31553]: pgmap v68: 129 pgs: 129 active+clean; 24 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 507 KiB/s wr, 225 op/s 2024-04-24T09:09:16.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:16 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:09:16.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:16 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:09:16.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:16 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:09:17.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:17 smithi081 ceph-mon[30961]: pgmap v69: 129 pgs: 129 active+clean; 26 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 732 KiB/s wr, 279 op/s 2024-04-24T09:09:17.590 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:17 smithi069 ceph-mon[28170]: pgmap v69: 129 pgs: 129 active+clean; 26 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 732 KiB/s wr, 279 op/s 2024-04-24T09:09:17.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:17 smithi130 ceph-mon[31553]: pgmap v69: 129 pgs: 129 active+clean; 26 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 732 KiB/s wr, 279 op/s 2024-04-24T09:09:18.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:18 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:09:18.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:18 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:09:18.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:18 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:09:18.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:18 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:09:18.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:18 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:09:18.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:18 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:09:18.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:18 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:09:18.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:18 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:09:18.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:18 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:09:19.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:19 smithi081 ceph-mon[30961]: pgmap v70: 129 pgs: 129 active+clean; 26 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 431 KiB/s wr, 80 op/s 2024-04-24T09:09:19.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:19 smithi069 ceph-mon[28170]: pgmap v70: 129 pgs: 129 active+clean; 26 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 431 KiB/s wr, 80 op/s 2024-04-24T09:09:19.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:19 smithi130 ceph-mon[31553]: pgmap v70: 129 pgs: 129 active+clean; 26 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 431 KiB/s wr, 80 op/s 2024-04-24T09:09:21.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:21 smithi081 ceph-mon[30961]: pgmap v71: 129 pgs: 129 active+clean; 28 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 620 KiB/s wr, 127 op/s 2024-04-24T09:09:21.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:21 smithi069 ceph-mon[28170]: pgmap v71: 129 pgs: 129 active+clean; 28 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 620 KiB/s wr, 127 op/s 2024-04-24T09:09:21.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:21 smithi130 ceph-mon[31553]: pgmap v71: 129 pgs: 129 active+clean; 28 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 620 KiB/s wr, 127 op/s 2024-04-24T09:09:23.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:23 smithi081 ceph-mon[30961]: pgmap v72: 129 pgs: 129 active+clean; 28 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 414 KiB/s wr, 101 op/s 2024-04-24T09:09:23.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:23 smithi069 ceph-mon[28170]: pgmap v72: 129 pgs: 129 active+clean; 28 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 414 KiB/s wr, 101 op/s 2024-04-24T09:09:23.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:23 smithi130 ceph-mon[31553]: pgmap v72: 129 pgs: 129 active+clean; 28 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 414 KiB/s wr, 101 op/s 2024-04-24T09:09:25.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:09:25.550 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:09:25.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:25 smithi081 ceph-mon[30961]: pgmap v73: 129 pgs: 129 active+clean; 28 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 414 KiB/s wr, 101 op/s 2024-04-24T09:09:25.590 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:25 smithi069 ceph-mon[28170]: pgmap v73: 129 pgs: 129 active+clean; 28 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 414 KiB/s wr, 101 op/s 2024-04-24T09:09:25.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:25 smithi130 ceph-mon[31553]: pgmap v73: 129 pgs: 129 active+clean; 28 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 414 KiB/s wr, 101 op/s 2024-04-24T09:09:25.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.773+0000 7f9d8b200640 1 -- 172.21.15.69:0/370070329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d8c102120 msgr2=0x7f9d8c102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:25.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.773+0000 7f9d8b200640 1 --2- 172.21.15.69:0/370070329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d8c102120 0x7f9d8c102500 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009900 tx=0x7f9d7c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.773+0000 7f9d8b200640 1 -- 172.21.15.69:0/370070329 shutdown_connections 2024-04-24T09:09:25.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.773+0000 7f9d8b200640 1 --2- 172.21.15.69:0/370070329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d8c1071f0 0x7f9d8c10edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.773+0000 7f9d8b200640 1 --2- 172.21.15.69:0/370070329 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d8c102ad0 0x7f9d8c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.773+0000 7f9d8b200640 1 --2- 172.21.15.69:0/370070329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d8c102120 0x7f9d8c102500 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.773+0000 7f9d8b200640 1 -- 172.21.15.69:0/370070329 >> 172.21.15.69:0/370070329 conn(0x7f9d8c0fd9a0 msgr2=0x7f9d8c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:25.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.773+0000 7f9d8b200640 1 -- 172.21.15.69:0/370070329 shutdown_connections 2024-04-24T09:09:25.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.773+0000 7f9d8b200640 1 -- 172.21.15.69:0/370070329 wait complete. 2024-04-24T09:09:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.774+0000 7f9d8b200640 1 Processor -- start 2024-04-24T09:09:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.774+0000 7f9d8b200640 1 -- start start 2024-04-24T09:09:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.775+0000 7f9d8b200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d8c102120 0x7f9d8c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.775+0000 7f9d8b200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d8c1071f0 0x7f9d8c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.775+0000 7f9d8b200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d8c19a770 0x7f9d8c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.775+0000 7f9d8b200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1121a0 con 0x7f9d8c19a770 2024-04-24T09:09:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.775+0000 7f9d8b200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c112020 con 0x7f9d8c1071f0 2024-04-24T09:09:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.775+0000 7f9d8b200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c112320 con 0x7f9d8c102120 2024-04-24T09:09:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.775+0000 7f9d8a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d8c19a770 0x7f9d8c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.775+0000 7f9d8a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d8c19a770 0x7f9d8c19abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51550/0 (socket says 172.21.15.69:51550) 2024-04-24T09:09:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.775+0000 7f9d8a800640 1 -- 172.21.15.69:0/4106098224 learned_addr learned my addr 172.21.15.69:0/4106098224 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.775+0000 7f9d89400640 1 --2- 172.21.15.69:0/4106098224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d8c1071f0 0x7f9d8c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.775+0000 7f9d89e00640 1 --2- 172.21.15.69:0/4106098224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d8c102120 0x7f9d8c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.776+0000 7f9d8a800640 1 -- 172.21.15.69:0/4106098224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d8c102120 msgr2=0x7f9d8c1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:25.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.776+0000 7f9d8a800640 1 --2- 172.21.15.69:0/4106098224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d8c102120 0x7f9d8c1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.776+0000 7f9d8a800640 1 -- 172.21.15.69:0/4106098224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d8c1071f0 msgr2=0x7f9d8c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:25.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.776+0000 7f9d8a800640 1 --2- 172.21.15.69:0/4106098224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d8c1071f0 0x7f9d8c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.776+0000 7f9d8a800640 1 -- 172.21.15.69:0/4106098224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c19b370 con 0x7f9d8c19a770 2024-04-24T09:09:25.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.776+0000 7f9d8a800640 1 --2- 172.21.15.69:0/4106098224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d8c19a770 0x7f9d8c19abf0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400eaa0 tx=0x7f9d8400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:25.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.776+0000 7f9d82a00640 1 -- 172.21.15.69:0/4106098224 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8400cdb0 con 0x7f9d8c19a770 2024-04-24T09:09:25.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.776+0000 7f9d82a00640 1 -- 172.21.15.69:0/4106098224 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d84005020 con 0x7f9d8c19a770 2024-04-24T09:09:25.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.776+0000 7f9d82a00640 1 -- 172.21.15.69:0/4106098224 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840116d0 con 0x7f9d8c19a770 2024-04-24T09:09:25.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.776+0000 7f9d8b200640 1 -- 172.21.15.69:0/4106098224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8c1a7450 con 0x7f9d8c19a770 2024-04-24T09:09:25.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.777+0000 7f9d8b200640 1 -- 172.21.15.69:0/4106098224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8c1a7800 con 0x7f9d8c19a770 2024-04-24T09:09:25.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.780+0000 7f9d93772740 1 Processor -- start 2024-04-24T09:09:25.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.781+0000 7f9d82a00640 1 -- 172.21.15.69:0/4106098224 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9d840051d0 con 0x7f9d8c19a770 2024-04-24T09:09:25.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.781+0000 7f9d93772740 1 -- start start 2024-04-24T09:09:25.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.781+0000 7f9d93772740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 0x55f09f3b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.782+0000 7f9d93772740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.782+0000 7f9d93772740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3c1a20 0x55f09f3c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.782+0000 7f9d93772740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f09f31c1f0 con 0x55f09f3b7350 2024-04-24T09:09:25.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.782+0000 7f9d93772740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f09f2c2350 con 0x55f09f3b6a30 2024-04-24T09:09:25.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.782+0000 7f9d93772740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f09f3181b0 con 0x55f09f3c1a20 2024-04-24T09:09:25.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.782+0000 7f9d89e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.782+0000 7f9d89e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3c14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51554/0 (socket says 172.21.15.69:51554) 2024-04-24T09:09:25.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.782+0000 7f9d89e00640 1 -- 172.21.15.69:0/2162562450 learned_addr learned my addr 172.21.15.69:0/2162562450 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:25.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.782+0000 7f9d89400640 1 --2- 172.21.15.69:0/2162562450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 0x55f09f3b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.782+0000 7f9d89400640 1 --2- 172.21.15.69:0/2162562450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3c1a20 0x55f09f3c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.783+0000 7f9d89e00640 1 -- 172.21.15.69:0/2162562450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3c1a20 msgr2=0x55f09f3c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.783+0000 7f9d89e00640 1 --2- 172.21.15.69:0/2162562450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3c1a20 0x55f09f3c3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.783+0000 7f9d89e00640 1 -- 172.21.15.69:0/2162562450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 msgr2=0x55f09f3b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.783+0000 7f9d89e00640 1 --2- 172.21.15.69:0/2162562450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 0x55f09f3b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.783+0000 7f9d89e00640 1 -- 172.21.15.69:0/2162562450 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f09f316f50 con 0x55f09f3b7350 2024-04-24T09:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.783+0000 7f9d82a00640 1 --2- 172.21.15.69:0/4106098224 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9d700a3d20 0x7f9d700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.783+0000 7f9d89e00640 1 --2- 172.21.15.69:0/2162562450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3c14e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009900 tx=0x7f9d7c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957351e5b640c354 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.783+0000 7f9d82a00640 1 -- 172.21.15.69:0/4106098224 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9d840d0df0 con 0x7f9d8c19a770 2024-04-24T09:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.783+0000 7f9d89400640 1 --2- 172.21.15.69:0/4106098224 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9d700a3d20 0x7f9d700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.783+0000 7f9d77000640 1 -- 172.21.15.69:0/2162562450 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c035030 con 0x55f09f3b7350 2024-04-24T09:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.784+0000 7f9d77000640 1 -- 172.21.15.69:0/2162562450 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d7c0044b0 con 0x55f09f3b7350 2024-04-24T09:09:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.784+0000 7f9d77000640 1 -- 172.21.15.69:0/2162562450 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0035f0 con 0x55f09f3b7350 2024-04-24T09:09:25.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.784+0000 7f9d93772740 1 -- 172.21.15.69:0/2162562450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 msgr2=0x55f09f3c14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:25.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.784+0000 7f9d93772740 1 --2- 172.21.15.69:0/2162562450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3c14e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009900 tx=0x7f9d7c0040c0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.784+0000 7f9d89400640 1 --2- 172.21.15.69:0/4106098224 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9d700a3d20 0x7f9d700a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55f09f3b3ee0 tx=0x7f9d78009640 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.785+0000 7f9d93772740 1 -- 172.21.15.69:0/2162562450 shutdown_connections 2024-04-24T09:09:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.785+0000 7f9d93772740 1 --2- 172.21.15.69:0/2162562450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3c1a20 0x55f09f3c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.785+0000 7f9d93772740 1 --2- 172.21.15.69:0/2162562450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3c14e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.785+0000 7f9d93772740 1 --2- 172.21.15.69:0/2162562450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 0x55f09f3b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.785+0000 7f9d93772740 1 -- 172.21.15.69:0/2162562450 >> 172.21.15.69:0/2162562450 conn(0x55f09f3b2d50 msgr2=0x55f09f3b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.785+0000 7f9d93772740 1 -- 172.21.15.69:0/2162562450 shutdown_connections 2024-04-24T09:09:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.785+0000 7f9d93772740 1 -- 172.21.15.69:0/2162562450 wait complete. 2024-04-24T09:09:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.785+0000 7f9d93772740 1 -- start start 2024-04-24T09:09:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.785+0000 7f9d93772740 1 Processor -- start 2024-04-24T09:09:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.786+0000 7f9d93772740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 0x55f09f3be9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.786+0000 7f9d93772740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3beee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.786+0000 7f9d93772740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3c1a20 0x55f09f3bf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.786+0000 7f9d93772740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f09f3c47b0 con 0x55f09f3b7350 2024-04-24T09:09:25.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.786+0000 7f9d93772740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f09f2d5690 con 0x55f09f3b6a30 2024-04-24T09:09:25.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.786+0000 7f9d93772740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f09f3c4930 con 0x55f09f3c1a20 2024-04-24T09:09:25.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.786+0000 7f9d93772740 10 client.0 _ll_register_callbacks cb 0x55f09f311650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:25.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.787+0000 7f9d89e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 0x55f09f3be9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.787+0000 7f9d89400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3beee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.787+0000 7f9d89e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 0x55f09f3be9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.69:39330/0 (socket says 172.21.15.69:39330) 2024-04-24T09:09:25.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.787+0000 7f9d89e00640 1 -- 172.21.15.69:0/2669487986 learned_addr learned my addr 172.21.15.69:0/2669487986 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:25.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.787+0000 7f9d8a800640 1 --2- 172.21.15.69:0/2669487986 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3c1a20 0x55f09f3bf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.788+0000 7f9d89400640 1 -- 172.21.15.69:0/2669487986 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3c1a20 msgr2=0x55f09f3bf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:25.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.788+0000 7f9d89400640 1 --2- 172.21.15.69:0/2669487986 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3c1a20 0x55f09f3bf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.788+0000 7f9d89400640 1 -- 172.21.15.69:0/2669487986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 msgr2=0x55f09f3be9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:25.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.788+0000 7f9d89400640 1 --2- 172.21.15.69:0/2669487986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 0x55f09f3be9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.788+0000 7f9d89400640 1 -- 172.21.15.69:0/2669487986 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f09f2d4920 con 0x55f09f3b7350 2024-04-24T09:09:25.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.788+0000 7f9d8a800640 1 --2- 172.21.15.69:0/2669487986 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3c1a20 0x55f09f3bf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:25.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.788+0000 7f9d89e00640 1 --2- 172.21.15.69:0/2669487986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 0x55f09f3be9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:25.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.788+0000 7f9d89400640 1 --2- 172.21.15.69:0/2669487986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3beee0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c19bfc0 tx=0x7f9d78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ce2e6a7b5aa2b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:25.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.788+0000 7f9d6b400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:25.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.788+0000 7f9d6b400640 1 -- 172.21.15.69:0/2669487986 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78005220 con 0x55f09f3b7350 2024-04-24T09:09:25.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.788+0000 7f9d6b400640 1 -- 172.21.15.69:0/2669487986 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9d780053c0 con 0x55f09f3b7350 2024-04-24T09:09:25.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.788+0000 7f9d93772740 1 -- 172.21.15.69:0/2669487986 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c0044b0 con 0x55f09f3b7350 2024-04-24T09:09:25.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.789+0000 7f9d6b400640 1 -- 172.21.15.69:0/2669487986 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d78032050 con 0x55f09f3b7350 2024-04-24T09:09:25.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.789+0000 7f9d93772740 10 client.16332 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:25.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.789+0000 7f9d93772740 1 -- 172.21.15.69:0/2669487986 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f09f0f87d0 con 0x55f09f3b7350 2024-04-24T09:09:25.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.789+0000 7f9d6b400640 1 -- 172.21.15.69:0/2669487986 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d7c0044b0 con 0x55f09f3b7350 2024-04-24T09:09:25.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.790+0000 7f9d6b400640 1 -- 172.21.15.69:0/2669487986 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7f9d78004ae0 con 0x55f09f3b7350 2024-04-24T09:09:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.790+0000 7f9d93772740 10 client.16332 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.790+0000 7f9d93772740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.790+0000 7f9d93772740 20 client.16332 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.790+0000 7f9d93772740 1 --2- 172.21.15.69:0/2669487986 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f09f3d1d50 0x55f09f3d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.790+0000 7f9d93772740 4 client.16332 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.790+0000 7f9d93772740 1 -- 172.21.15.69:0/2669487986 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f09f329c10 con 0x55f09f3d1d50 2024-04-24T09:09:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.790+0000 7f9d89e00640 1 --2- 172.21.15.69:0/2669487986 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f09f3d1d50 0x55f09f3d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.791+0000 7f9d89e00640 1 --2- 172.21.15.69:0/2669487986 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f09f3d1d50 0x55f09f3d41f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a0e22a7640eacc server_cookie=ee7f49ea93a2240f in_seq=0 out_seq=0 2024-04-24T09:09:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.791+0000 7f9d6b400640 10 client.16332 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:25.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.793+0000 7f9d6b400640 1 -- 172.21.15.69:0/2669487986 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f09f329c10 con 0x55f09f3d1d50 2024-04-24T09:09:25.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.793+0000 7f9d6b400640 10 client.16332 handle_command_reply: tid=0 2024-04-24T09:09:25.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.793+0000 7f9d93772740 1 client.16332 shutdown 2024-04-24T09:09:25.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.794+0000 7f9d93772740 1 -- 172.21.15.69:0/2669487986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 msgr2=0x55f09f3beee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:25.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.794+0000 7f9d93772740 1 --2- 172.21.15.69:0/2669487986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3beee0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c19bfc0 tx=0x7f9d78004730 comp rx=0 tx=0).stop 2024-04-24T09:09:25.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.794+0000 7f9d93772740 1 -- 172.21.15.69:0/2669487986 shutdown_connections 2024-04-24T09:09:25.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.794+0000 7f9d93772740 1 --2- 172.21.15.69:0/2669487986 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f09f3d1d50 0x55f09f3d41f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.794+0000 7f9d93772740 1 --2- 172.21.15.69:0/2669487986 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3c1a20 0x55f09f3bf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.794+0000 7f9d93772740 1 --2- 172.21.15.69:0/2669487986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3beee0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.794+0000 7f9d93772740 1 --2- 172.21.15.69:0/2669487986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 0x55f09f3be9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.794+0000 7f9d6b400640 0 client.16332 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:25.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.794+0000 7f9d93772740 1 -- 172.21.15.69:0/2669487986 >> 172.21.15.69:0/2669487986 conn(0x55f09f3b2d50 msgr2=0x55f09f3b1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:25.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.794+0000 7f9d93772740 1 -- 172.21.15.69:0/2669487986 shutdown_connections 2024-04-24T09:09:25.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.794+0000 7f9d93772740 1 -- 172.21.15.69:0/2669487986 wait complete. 2024-04-24T09:09:25.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.794+0000 7f9d93772740 20 client.16332 trim_cache size 0 max 16384 2024-04-24T09:09:25.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.816+0000 7f9d93772740 1 Processor -- start 2024-04-24T09:09:25.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.816+0000 7f9d93772740 1 -- start start 2024-04-24T09:09:25.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.816+0000 7f9d93772740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 0x55f09f3cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.816+0000 7f9d93772740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.816+0000 7f9d93772740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3c1a20 0x55f09f3b8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.817+0000 7f9d93772740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1b3b70 con 0x55f09f3b7350 2024-04-24T09:09:25.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.817+0000 7f9d93772740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1b1410 con 0x55f09f3b6a30 2024-04-24T09:09:25.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.817+0000 7f9d93772740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1af260 con 0x55f09f3c1a20 2024-04-24T09:09:25.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.817+0000 7f9d89400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.817+0000 7f9d89400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3bdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51574/0 (socket says 172.21.15.69:51574) 2024-04-24T09:09:25.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.817+0000 7f9d89400640 1 -- 172.21.15.69:0/2675957267 learned_addr learned my addr 172.21.15.69:0/2675957267 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:25.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.817+0000 7f9d89e00640 1 --2- 172.21.15.69:0/2675957267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 0x55f09f3cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.817+0000 7f9d8a800640 1 --2- 172.21.15.69:0/2675957267 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3c1a20 0x55f09f3b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.817+0000 7f9d89400640 1 -- 172.21.15.69:0/2675957267 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3c1a20 msgr2=0x55f09f3b8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:25.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.817+0000 7f9d89400640 1 --2- 172.21.15.69:0/2675957267 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3c1a20 0x55f09f3b8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.817+0000 7f9d89400640 1 -- 172.21.15.69:0/2675957267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 msgr2=0x55f09f3cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:25.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.817+0000 7f9d89400640 1 --2- 172.21.15.69:0/2675957267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 0x55f09f3cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.817+0000 7f9d89400640 1 -- 172.21.15.69:0/2675957267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f09f3b7180 con 0x55f09f3b7350 2024-04-24T09:09:25.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.818+0000 7f9d89400640 1 --2- 172.21.15.69:0/2675957267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3bdb00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9d7801a040 tx=0x7f9d78005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c889042c14c6bb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:25.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.818+0000 7f9d76600640 1 -- 172.21.15.69:0/2675957267 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7803b070 con 0x55f09f3b7350 2024-04-24T09:09:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.818+0000 7f9d76600640 1 -- 172.21.15.69:0/2675957267 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d780216b0 con 0x55f09f3b7350 2024-04-24T09:09:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.818+0000 7f9d76600640 1 -- 172.21.15.69:0/2675957267 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78004fd0 con 0x55f09f3b7350 2024-04-24T09:09:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.819+0000 7f9d93772740 1 -- 172.21.15.69:0/2675957267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 msgr2=0x55f09f3bdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.819+0000 7f9d93772740 1 --2- 172.21.15.69:0/2675957267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3bdb00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9d7801a040 tx=0x7f9d78005f90 comp rx=0 tx=0).stop 2024-04-24T09:09:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.819+0000 7f9d93772740 1 -- 172.21.15.69:0/2675957267 shutdown_connections 2024-04-24T09:09:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.819+0000 7f9d93772740 1 --2- 172.21.15.69:0/2675957267 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3c1a20 0x55f09f3b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.819+0000 7f9d93772740 1 --2- 172.21.15.69:0/2675957267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b7350 0x55f09f3bdb00 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.819+0000 7f9d93772740 1 --2- 172.21.15.69:0/2675957267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3b6a30 0x55f09f3cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.819+0000 7f9d93772740 1 -- 172.21.15.69:0/2675957267 >> 172.21.15.69:0/2675957267 conn(0x55f09f3b2d50 msgr2=0x55f09f3b8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.819+0000 7f9d93772740 1 -- 172.21.15.69:0/2675957267 shutdown_connections 2024-04-24T09:09:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.819+0000 7f9d93772740 1 -- 172.21.15.69:0/2675957267 wait complete. 2024-04-24T09:09:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.820+0000 7f9d93772740 1 -- start start 2024-04-24T09:09:25.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.820+0000 7f9d93772740 1 Processor -- start 2024-04-24T09:09:25.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.820+0000 7f9d93772740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b6a30 0x55f09f3c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.820+0000 7f9d93772740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3b7350 0x55f09f3c5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.820+0000 7f9d93772740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3c1a20 0x55f09f3c5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.820+0000 7f9d93772740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1af8d0 con 0x55f09f3b6a30 2024-04-24T09:09:25.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.820+0000 7f9d93772740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1b0d40 con 0x55f09f3c1a20 2024-04-24T09:09:25.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.820+0000 7f9d93772740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c1affa0 con 0x55f09f3b7350 2024-04-24T09:09:25.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.820+0000 7f9d93772740 10 client.0 _ll_register_callbacks cb 0x7f9d78009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:25.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.821+0000 7f9d89e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b6a30 0x55f09f3c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.821+0000 7f9d89e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b6a30 0x55f09f3c9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51578/0 (socket says 172.21.15.69:51578) 2024-04-24T09:09:25.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.821+0000 7f9d89e00640 1 -- 172.21.15.69:0/4057142647 learned_addr learned my addr 172.21.15.69:0/4057142647 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:25.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.821+0000 7f9d89400640 1 --2- 172.21.15.69:0/4057142647 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3b7350 0x55f09f3c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.821+0000 7f9d89e00640 1 -- 172.21.15.69:0/4057142647 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3b7350 msgr2=0x55f09f3c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:25.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.821+0000 7f9d89e00640 1 --2- 172.21.15.69:0/4057142647 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3b7350 0x55f09f3c5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.821+0000 7f9d89e00640 1 -- 172.21.15.69:0/4057142647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3c1a20 msgr2=0x55f09f3c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:25.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.821+0000 7f9d8a800640 1 --2- 172.21.15.69:0/4057142647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3c1a20 0x55f09f3c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.821+0000 7f9d89e00640 1 --2- 172.21.15.69:0/4057142647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3c1a20 0x55f09f3c5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.821+0000 7f9d89e00640 1 -- 172.21.15.69:0/4057142647 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f09f02e3f0 con 0x55f09f3b6a30 2024-04-24T09:09:25.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.821+0000 7f9d89e00640 1 --2- 172.21.15.69:0/4057142647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b6a30 0x55f09f3c9ec0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c006010 tx=0x7f9d7c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108dc539d4e80366 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:25.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.822+0000 7f9d6b400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:25.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.822+0000 7f9d6b400640 1 -- 172.21.15.69:0/4057142647 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c035040 con 0x55f09f3b6a30 2024-04-24T09:09:25.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.822+0000 7f9d6b400640 1 -- 172.21.15.69:0/4057142647 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9d7c003810 con 0x55f09f3b6a30 2024-04-24T09:09:25.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.822+0000 7f9d93772740 1 -- 172.21.15.69:0/4057142647 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d780216b0 con 0x55f09f3b6a30 2024-04-24T09:09:25.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.822+0000 7f9d6b400640 1 -- 172.21.15.69:0/4057142647 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d7c060050 con 0x55f09f3b6a30 2024-04-24T09:09:25.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.822+0000 7f9d93772740 10 client.16338 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:25.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.822+0000 7f9d93772740 1 -- 172.21.15.69:0/4057142647 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f09efee0f0 con 0x55f09f3b6a30 2024-04-24T09:09:25.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.822+0000 7f9d6b400640 1 -- 172.21.15.69:0/4057142647 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d780216b0 con 0x55f09f3b6a30 2024-04-24T09:09:25.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.822+0000 7f9d6b400640 1 -- 172.21.15.69:0/4057142647 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7f9d7c0517a0 con 0x55f09f3b6a30 2024-04-24T09:09:25.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.823+0000 7f9d93772740 10 client.16338 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:25.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.823+0000 7f9d93772740 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:25.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.823+0000 7f9d93772740 20 client.16338 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:25.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.823+0000 7f9d93772740 1 --2- 172.21.15.69:0/4057142647 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f09f440ae0 0x55f09f442ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:25.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.823+0000 7f9d93772740 4 client.16338 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:09:25.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.823+0000 7f9d93772740 1 -- 172.21.15.69:0/4057142647 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f09f323a80 con 0x55f09f440ae0 2024-04-24T09:09:25.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.823+0000 7f9d89400640 1 --2- 172.21.15.69:0/4057142647 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f09f440ae0 0x55f09f442ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:25.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.824+0000 7f9d89400640 1 --2- 172.21.15.69:0/4057142647 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f09f440ae0 0x55f09f442ed0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f1edb87c943757b server_cookie=3e589feedf764b56 in_seq=0 out_seq=0 2024-04-24T09:09:25.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.824+0000 7f9d6b400640 10 client.16338 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:25.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.825+0000 7f9d6b400640 1 -- 172.21.15.69:0/4057142647 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f09f323a80 con 0x55f09f440ae0 2024-04-24T09:09:25.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.825+0000 7f9d6b400640 10 client.16338 handle_command_reply: tid=0 2024-04-24T09:09:25.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.825+0000 7f9d93772740 1 client.16338 shutdown 2024-04-24T09:09:25.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.825+0000 7f9d93772740 1 -- 172.21.15.69:0/4057142647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b6a30 msgr2=0x55f09f3c9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:25.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.825+0000 7f9d93772740 1 --2- 172.21.15.69:0/4057142647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b6a30 0x55f09f3c9ec0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c006010 tx=0x7f9d7c005b50 comp rx=0 tx=0).stop 2024-04-24T09:09:25.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.825+0000 7f9d93772740 1 -- 172.21.15.69:0/4057142647 shutdown_connections 2024-04-24T09:09:25.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.825+0000 7f9d93772740 1 --2- 172.21.15.69:0/4057142647 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f09f440ae0 0x55f09f442ed0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.825+0000 7f9d93772740 1 --2- 172.21.15.69:0/4057142647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f09f3c1a20 0x55f09f3c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.825+0000 7f9d93772740 1 --2- 172.21.15.69:0/4057142647 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f09f3b7350 0x55f09f3c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.825+0000 7f9d93772740 1 --2- 172.21.15.69:0/4057142647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f09f3b6a30 0x55f09f3c9ec0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.825+0000 7f9d6b400640 0 client.16338 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:25.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.825+0000 7f9d93772740 1 -- 172.21.15.69:0/4057142647 >> 172.21.15.69:0/4057142647 conn(0x55f09f3b2d50 msgr2=0x55f09f3be960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:25.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.826+0000 7f9d93772740 1 -- 172.21.15.69:0/4057142647 shutdown_connections 2024-04-24T09:09:25.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.826+0000 7f9d93772740 1 -- 172.21.15.69:0/4057142647 wait complete. 2024-04-24T09:09:25.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.826+0000 7f9d93772740 20 client.16338 trim_cache size 0 max 16384 2024-04-24T09:09:25.828 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:09:25.828 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-04-24T09:09:25.828 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-04-24T09:09:25.828 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:09:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.827+0000 7f9d8b200640 1 -- 172.21.15.69:0/4106098224 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9d700a3d20 msgr2=0x7f9d700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.827+0000 7f9d8b200640 1 --2- 172.21.15.69:0/4106098224 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9d700a3d20 0x7f9d700a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55f09f3b3ee0 tx=0x7f9d78009640 comp rx=0 tx=0).stop 2024-04-24T09:09:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.827+0000 7f9d8b200640 1 -- 172.21.15.69:0/4106098224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d8c19a770 msgr2=0x7f9d8c19abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.827+0000 7f9d8b200640 1 --2- 172.21.15.69:0/4106098224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d8c19a770 0x7f9d8c19abf0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400eaa0 tx=0x7f9d8400ef60 comp rx=0 tx=0).stop 2024-04-24T09:09:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.827+0000 7f9d8b200640 1 -- 172.21.15.69:0/4106098224 shutdown_connections 2024-04-24T09:09:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.827+0000 7f9d8b200640 1 --2- 172.21.15.69:0/4106098224 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9d700a3d20 0x7f9d700a61e0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55f09f3b3ee0 tx=0x7f9d78009640 comp rx=0 tx=0).stop 2024-04-24T09:09:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.827+0000 7f9d8b200640 1 --2- 172.21.15.69:0/4106098224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d8c19a770 0x7f9d8c19abf0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.827+0000 7f9d8b200640 1 --2- 172.21.15.69:0/4106098224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d8c1071f0 0x7f9d8c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.827+0000 7f9d8b200640 1 --2- 172.21.15.69:0/4106098224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d8c102120 0x7f9d8c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.827+0000 7f9d8b200640 1 -- 172.21.15.69:0/4106098224 >> 172.21.15.69:0/4106098224 conn(0x7f9d8c0fd9a0 msgr2=0x7f9d8c0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.827+0000 7f9d8b200640 1 -- 172.21.15.69:0/4106098224 shutdown_connections 2024-04-24T09:09:25.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:25.827+0000 7f9d8b200640 1 -- 172.21.15.69:0/4106098224 wait complete. 2024-04-24T09:09:25.846 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:09:25.846 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:09:26.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.112+0000 7fe60b000640 1 -- 172.21.15.69:0/318972551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe604076530 msgr2=0x7fe604076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.112+0000 7fe60b000640 1 --2- 172.21.15.69:0/318972551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe604076530 0x7fe604076910 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009ca0 tx=0x7fe5f802a8b0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.112+0000 7fe60b000640 1 -- 172.21.15.69:0/318972551 shutdown_connections 2024-04-24T09:09:26.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.112+0000 7fe60b000640 1 --2- 172.21.15.69:0/318972551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe604112cb0 0x7fe6041150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.112+0000 7fe60b000640 1 --2- 172.21.15.69:0/318972551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe604076e50 0x7fe604112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.112+0000 7fe60b000640 1 --2- 172.21.15.69:0/318972551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe604076530 0x7fe604076910 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.112+0000 7fe60b000640 1 -- 172.21.15.69:0/318972551 >> 172.21.15.69:0/318972551 conn(0x7fe6040ffaa0 msgr2=0x7fe604101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:26.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.112+0000 7fe60b000640 1 -- 172.21.15.69:0/318972551 shutdown_connections 2024-04-24T09:09:26.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.112+0000 7fe60b000640 1 -- 172.21.15.69:0/318972551 wait complete. 2024-04-24T09:09:26.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.113+0000 7fe60b000640 1 Processor -- start 2024-04-24T09:09:26.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.113+0000 7fe60b000640 1 -- start start 2024-04-24T09:09:26.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.114+0000 7fe60b000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe604076530 0x7fe6041a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.114+0000 7fe60b000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe604076e50 0x7fe6041a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.114+0000 7fe60b000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe604112cb0 0x7fe60419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.114+0000 7fe60b000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe6041186b0 con 0x7fe604076530 2024-04-24T09:09:26.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.114+0000 7fe60b000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe604118530 con 0x7fe604112cb0 2024-04-24T09:09:26.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.114+0000 7fe60b000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe604118830 con 0x7fe604076e50 2024-04-24T09:09:26.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.114+0000 7fe609c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe604076530 0x7fe6041a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.114+0000 7fe609c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe604076530 0x7fe6041a4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51594/0 (socket says 172.21.15.69:51594) 2024-04-24T09:09:26.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.114+0000 7fe609c00640 1 -- 172.21.15.69:0/889318824 learned_addr learned my addr 172.21.15.69:0/889318824 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:26.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.114+0000 7fe60a600640 1 --2- 172.21.15.69:0/889318824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe604112cb0 0x7fe60419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.115+0000 7fe609200640 1 --2- 172.21.15.69:0/889318824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe604076e50 0x7fe6041a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.115+0000 7fe609c00640 1 -- 172.21.15.69:0/889318824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe604076e50 msgr2=0x7fe6041a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.115+0000 7fe609c00640 1 --2- 172.21.15.69:0/889318824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe604076e50 0x7fe6041a4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.115+0000 7fe609c00640 1 -- 172.21.15.69:0/889318824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe604112cb0 msgr2=0x7fe60419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.115+0000 7fe609c00640 1 --2- 172.21.15.69:0/889318824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe604112cb0 0x7fe60419e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.115+0000 7fe609c00640 1 -- 172.21.15.69:0/889318824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60419f130 con 0x7fe604076530 2024-04-24T09:09:26.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.115+0000 7fe609200640 1 --2- 172.21.15.69:0/889318824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe604076e50 0x7fe6041a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:26.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.115+0000 7fe609c00640 1 --2- 172.21.15.69:0/889318824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe604076530 0x7fe6041a4800 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe5f802add0 tx=0x7fe5f8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:26.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.115+0000 7fe602000640 1 -- 172.21.15.69:0/889318824 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8004450 con 0x7fe604076530 2024-04-24T09:09:26.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.115+0000 7fe60b000640 1 -- 172.21.15.69:0/889318824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60419f3c0 con 0x7fe604076530 2024-04-24T09:09:26.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.115+0000 7fe602000640 1 -- 172.21.15.69:0/889318824 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5f8040070 con 0x7fe604076530 2024-04-24T09:09:26.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.116+0000 7fe60b000640 1 -- 172.21.15.69:0/889318824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6041ab6c0 con 0x7fe604076530 2024-04-24T09:09:26.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.116+0000 7fe602000640 1 -- 172.21.15.69:0/889318824 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8045dc0 con 0x7fe604076530 2024-04-24T09:09:26.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.117+0000 7fe60b000640 1 -- 172.21.15.69:0/889318824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe604068d40 con 0x7fe604076530 2024-04-24T09:09:26.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.119+0000 7fe602000640 1 -- 172.21.15.69:0/889318824 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe5f8045470 con 0x7fe604076530 2024-04-24T09:09:26.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.120+0000 7fe602000640 1 --2- 172.21.15.69:0/889318824 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe5ec0a3d20 0x7fe5ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.120+0000 7fe602000640 1 -- 172.21.15.69:0/889318824 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe5f80f21a0 con 0x7fe604076530 2024-04-24T09:09:26.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.120+0000 7fe609200640 1 --2- 172.21.15.69:0/889318824 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe5ec0a3d20 0x7fe5ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.121+0000 7fe609200640 1 --2- 172.21.15.69:0/889318824 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe5ec0a3d20 0x7fe5ec0a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe604079550 tx=0x7fe5f0005d90 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:26.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.125+0000 7fe602000640 1 -- 172.21.15.69:0/889318824 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5f80be5b0 con 0x7fe604076530 2024-04-24T09:09:26.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.417+0000 7fe60b000640 1 -- 172.21.15.69:0/889318824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe60410de70 con 0x7fe604076530 2024-04-24T09:09:26.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.418+0000 7fe602000640 1 -- 172.21.15.69:0/889318824 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11877 (secure 0 0 0) 0x7fe5f80bdeb0 con 0x7fe604076530 2024-04-24T09:09:26.419 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:09:26.419 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:04:01.991559+0000","modified":"2024-04-24T09:08:58.646121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34532,"mds_1":34496,"mds_2":34460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34496":{"gid":34496,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34532":{"gid":34532,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34532,"qdb_cluster":[34460,34496,34532]},"id":1}]} 2024-04-24T09:09:26.419 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-04-24T09:09:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.421+0000 7fe60b000640 1 -- 172.21.15.69:0/889318824 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe5ec0a3d20 msgr2=0x7fe5ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.421+0000 7fe60b000640 1 --2- 172.21.15.69:0/889318824 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe5ec0a3d20 0x7fe5ec0a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe604079550 tx=0x7fe5f0005d90 comp rx=0 tx=0).stop 2024-04-24T09:09:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.421+0000 7fe60b000640 1 -- 172.21.15.69:0/889318824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe604076530 msgr2=0x7fe6041a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.421+0000 7fe60b000640 1 --2- 172.21.15.69:0/889318824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe604076530 0x7fe6041a4800 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe5f802add0 tx=0x7fe5f8003ba0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.421+0000 7fe60b000640 1 -- 172.21.15.69:0/889318824 shutdown_connections 2024-04-24T09:09:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.421+0000 7fe60b000640 1 --2- 172.21.15.69:0/889318824 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe5ec0a3d20 0x7fe5ec0a61e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.421+0000 7fe60b000640 1 --2- 172.21.15.69:0/889318824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe604112cb0 0x7fe60419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.421+0000 7fe60b000640 1 --2- 172.21.15.69:0/889318824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe604076e50 0x7fe6041a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.421+0000 7fe60b000640 1 --2- 172.21.15.69:0/889318824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe604076530 0x7fe6041a4800 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.421+0000 7fe60b000640 1 -- 172.21.15.69:0/889318824 >> 172.21.15.69:0/889318824 conn(0x7fe6040ffaa0 msgr2=0x7fe604113920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:26.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.421+0000 7fe60b000640 1 -- 172.21.15.69:0/889318824 shutdown_connections 2024-04-24T09:09:26.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.421+0000 7fe60b000640 1 -- 172.21.15.69:0/889318824 wait complete. 2024-04-24T09:09:26.442 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:09:26.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.653+0000 7fe8b7000640 1 -- 172.21.15.69:0/571746363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b0102ac0 msgr2=0x7fe8b0106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.653+0000 7fe8b7000640 1 --2- 172.21.15.69:0/571746363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b0102ac0 0x7fe8b0106b00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009900 tx=0x7fe89c02a410 comp rx=0 tx=0).stop 2024-04-24T09:09:26.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.653+0000 7fe8b7000640 1 -- 172.21.15.69:0/571746363 shutdown_connections 2024-04-24T09:09:26.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.653+0000 7fe8b7000640 1 --2- 172.21.15.69:0/571746363 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8b01071e0 0x7fe8b010edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.654+0000 7fe8b7000640 1 --2- 172.21.15.69:0/571746363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b0102ac0 0x7fe8b0106b00 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.654+0000 7fe8b7000640 1 --2- 172.21.15.69:0/571746363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8b0102110 0x7fe8b01024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.654+0000 7fe8b7000640 1 -- 172.21.15.69:0/571746363 >> 172.21.15.69:0/571746363 conn(0x7fe8b00fd9d0 msgr2=0x7fe8b00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:26.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.654+0000 7fe8b7000640 1 -- 172.21.15.69:0/571746363 shutdown_connections 2024-04-24T09:09:26.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.654+0000 7fe8b7000640 1 -- 172.21.15.69:0/571746363 wait complete. 2024-04-24T09:09:26.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.655+0000 7fe8b7000640 1 Processor -- start 2024-04-24T09:09:26.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.655+0000 7fe8b7000640 1 -- start start 2024-04-24T09:09:26.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.656+0000 7fe8b7000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8b0102110 0x7fe8b01a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.656+0000 7fe8b7000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b01071e0 0x7fe8b01a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.656+0000 7fe8b7000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8b019a7b0 0x7fe8b019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.656+0000 7fe8b7000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0112170 con 0x7fe8b01071e0 2024-04-24T09:09:26.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.656+0000 7fe8b7000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0111ff0 con 0x7fe8b0102110 2024-04-24T09:09:26.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.656+0000 7fe8b7000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b01122f0 con 0x7fe8b019a7b0 2024-04-24T09:09:26.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.656+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b01071e0 0x7fe8b01a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.656+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b01071e0 0x7fe8b01a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51618/0 (socket says 172.21.15.69:51618) 2024-04-24T09:09:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.656+0000 7fe8b5200640 1 -- 172.21.15.69:0/584974983 learned_addr learned my addr 172.21.15.69:0/584974983 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.656+0000 7fe8b5c00640 1 --2- 172.21.15.69:0/584974983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8b0102110 0x7fe8b01a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.656+0000 7fe8b6600640 1 --2- 172.21.15.69:0/584974983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8b019a7b0 0x7fe8b019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.657+0000 7fe8b5200640 1 -- 172.21.15.69:0/584974983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8b019a7b0 msgr2=0x7fe8b019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.657+0000 7fe8b5200640 1 --2- 172.21.15.69:0/584974983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8b019a7b0 0x7fe8b019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.657+0000 7fe8b5200640 1 -- 172.21.15.69:0/584974983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8b0102110 msgr2=0x7fe8b01a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.657+0000 7fe8b5200640 1 --2- 172.21.15.69:0/584974983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8b0102110 0x7fe8b01a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.657+0000 7fe8b5200640 1 -- 172.21.15.69:0/584974983 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b019b340 con 0x7fe8b01071e0 2024-04-24T09:09:26.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.657+0000 7fe8b5200640 1 --2- 172.21.15.69:0/584974983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b01071e0 0x7fe8b01a0b80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe89c005ab0 tx=0x7fe89c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:26.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.658+0000 7fe8ae000640 1 -- 172.21.15.69:0/584974983 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c03f070 con 0x7fe8b01071e0 2024-04-24T09:09:26.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.658+0000 7fe8b7000640 1 -- 172.21.15.69:0/584974983 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b019b5d0 con 0x7fe8b01071e0 2024-04-24T09:09:26.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.658+0000 7fe8ae000640 1 -- 172.21.15.69:0/584974983 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe89c031d60 con 0x7fe8b01071e0 2024-04-24T09:09:26.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.658+0000 7fe8ae000640 1 -- 172.21.15.69:0/584974983 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c035030 con 0x7fe8b01071e0 2024-04-24T09:09:26.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.658+0000 7fe8b7000640 1 -- 172.21.15.69:0/584974983 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b01a7780 con 0x7fe8b01071e0 2024-04-24T09:09:26.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.662+0000 7fe8ae000640 1 -- 172.21.15.69:0/584974983 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe89c042070 con 0x7fe8b01071e0 2024-04-24T09:09:26.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.662+0000 7fe8bb3e0740 1 Processor -- start 2024-04-24T09:09:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.662+0000 7fe8bb3e0740 1 -- start start 2024-04-24T09:09:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.662+0000 7fe8bb3e0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae038f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.662+0000 7fe8bb3e0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae039460 0x55bbae0435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.662+0000 7fe8bb3e0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae043b30 0x55bbae045f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.662+0000 7fe8bb3e0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bbadf9e1d0 con 0x55bbae038b40 2024-04-24T09:09:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.662+0000 7fe8bb3e0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55bbadf44450 con 0x55bbae039460 2024-04-24T09:09:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.662+0000 7fe8bb3e0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55bbadf9a190 con 0x55bbae043b30 2024-04-24T09:09:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.663+0000 7fe8b6600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae038f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.663+0000 7fe8b6600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae038f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51632/0 (socket says 172.21.15.69:51632) 2024-04-24T09:09:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.663+0000 7fe8b6600640 1 -- 172.21.15.69:0/4086967004 learned_addr learned my addr 172.21.15.69:0/4086967004 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.663+0000 7fe8b5200640 1 --2- 172.21.15.69:0/4086967004 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae043b30 0x55bbae045f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.663+0000 7fe8b5c00640 1 --2- 172.21.15.69:0/4086967004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae039460 0x55bbae0435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.663+0000 7fe8b6600640 1 -- 172.21.15.69:0/4086967004 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae043b30 msgr2=0x55bbae045f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:26.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.663+0000 7fe8b6600640 1 --2- 172.21.15.69:0/4086967004 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae043b30 0x55bbae045f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.663+0000 7fe8b6600640 1 -- 172.21.15.69:0/4086967004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae039460 msgr2=0x55bbae0435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:26.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.663+0000 7fe8b6600640 1 --2- 172.21.15.69:0/4086967004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae039460 0x55bbae0435f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.663+0000 7fe8b6600640 1 -- 172.21.15.69:0/4086967004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbae001310 con 0x55bbae038b40 2024-04-24T09:09:26.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.664+0000 7fe8b6600640 1 --2- 172.21.15.69:0/4086967004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae038f20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009f70 tx=0x7fe8a8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9514ce06bd73877d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:26.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.664+0000 7fe8ae000640 1 --2- 172.21.15.69:0/584974983 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe8980a3d20 0x7fe8980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.664+0000 7fe8ae000640 1 -- 172.21.15.69:0/584974983 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe89c0f1290 con 0x7fe8b01071e0 2024-04-24T09:09:26.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.664+0000 7fe8a2a00640 1 -- 172.21.15.69:0/4086967004 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8016070 con 0x55bbae038b40 2024-04-24T09:09:26.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.664+0000 7fe8a2a00640 1 -- 172.21.15.69:0/4086967004 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8a8004d20 con 0x55bbae038b40 2024-04-24T09:09:26.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.664+0000 7fe8a2a00640 1 -- 172.21.15.69:0/4086967004 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80127d0 con 0x55bbae038b40 2024-04-24T09:09:26.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.665+0000 7fe8b5c00640 1 --2- 172.21.15.69:0/584974983 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe8980a3d20 0x7fe8980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.665+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/4086967004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 msgr2=0x55bbae038f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.665+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/4086967004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae038f20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009f70 tx=0x7fe8a8010800 comp rx=0 tx=0).stop 2024-04-24T09:09:26.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.665+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/4086967004 shutdown_connections 2024-04-24T09:09:26.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.665+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/4086967004 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae043b30 0x55bbae045f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.665+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/4086967004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae039460 0x55bbae0435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.665+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/4086967004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae038f20 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.665+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/4086967004 >> 172.21.15.69:0/4086967004 conn(0x55bbae034e60 msgr2=0x55bbae035240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:26.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.666+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/4086967004 shutdown_connections 2024-04-24T09:09:26.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.666+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/4086967004 wait complete. 2024-04-24T09:09:26.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.666+0000 7fe8b5c00640 1 --2- 172.21.15.69:0/584974983 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe8980a3d20 0x7fe8980a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55bbae03a100 tx=0x7fe8a400b390 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:26.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.666+0000 7fe8bb3e0740 1 -- start start 2024-04-24T09:09:26.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.666+0000 7fe8bb3e0740 1 Processor -- start 2024-04-24T09:09:26.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.667+0000 7fe8bb3e0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae040900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.667+0000 7fe8bb3e0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae039460 0x55bbae040e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.667+0000 7fe8bb3e0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae043b30 0x55bbae041380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.667+0000 7fe8bb3e0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bbae04f170 con 0x55bbae038b40 2024-04-24T09:09:26.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.667+0000 7fe8bb3e0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55bbadf57810 con 0x55bbae039460 2024-04-24T09:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.667+0000 7fe8bb3e0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55bbae04f2f0 con 0x55bbae043b30 2024-04-24T09:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.667+0000 7fe8bb3e0740 10 client.0 _ll_register_callbacks cb 0x55bbadf25140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.667+0000 7fe8b5c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae040900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.667+0000 7fe8b5c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae040900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51636/0 (socket says 172.21.15.69:51636) 2024-04-24T09:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.667+0000 7fe8b5c00640 1 -- 172.21.15.69:0/3272356678 learned_addr learned my addr 172.21.15.69:0/3272356678 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.668+0000 7fe8b6600640 1 --2- 172.21.15.69:0/3272356678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae043b30 0x55bbae041380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.668+0000 7fe8b5200640 1 --2- 172.21.15.69:0/3272356678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae039460 0x55bbae040e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.668+0000 7fe8b5c00640 1 -- 172.21.15.69:0/3272356678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae043b30 msgr2=0x55bbae041380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.668+0000 7fe8b5c00640 1 --2- 172.21.15.69:0/3272356678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae043b30 0x55bbae041380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.668+0000 7fe8b5c00640 1 -- 172.21.15.69:0/3272356678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae039460 msgr2=0x55bbae040e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.668+0000 7fe8b5c00640 1 --2- 172.21.15.69:0/3272356678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae039460 0x55bbae040e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.668+0000 7fe8b5c00640 1 -- 172.21.15.69:0/3272356678 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbadf683f0 con 0x55bbae038b40 2024-04-24T09:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.668+0000 7fe8b5c00640 1 --2- 172.21.15.69:0/3272356678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae040900 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4003e00 tx=0x7fe8a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf7adf03d9bcf64 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.668+0000 7fe897400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:26.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.669+0000 7fe897400640 1 -- 172.21.15.69:0/3272356678 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a401b660 con 0x55bbae038b40 2024-04-24T09:09:26.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.669+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3272356678 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8a8004d20 con 0x55bbae038b40 2024-04-24T09:09:26.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.669+0000 7fe897400640 1 -- 172.21.15.69:0/3272356678 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe8a401b800 con 0x55bbae038b40 2024-04-24T09:09:26.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.669+0000 7fe897400640 1 -- 172.21.15.69:0/3272356678 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe8a401ba00 con 0x55bbae038b40 2024-04-24T09:09:26.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.669+0000 7fe8bb3e0740 10 client.16356 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:26.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.669+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3272356678 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bbadcb03f0 con 0x55bbae038b40 2024-04-24T09:09:26.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.670+0000 7fe897400640 1 -- 172.21.15.69:0/3272356678 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8a8004d20 con 0x55bbae038b40 2024-04-24T09:09:26.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.670+0000 7fe897400640 1 -- 172.21.15.69:0/3272356678 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7fe8a4024070 con 0x55bbae038b40 2024-04-24T09:09:26.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.670+0000 7fe8bb3e0740 10 client.16356 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:26.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.670+0000 7fe8bb3e0740 10 client.16356 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:09:26.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.670+0000 7fe8bb3e0740 20 client.16356 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:26.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.670+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/3272356678 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55bbae05c560 0x55bbae05ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.670+0000 7fe8bb3e0740 4 client.16356 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:26.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.670+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3272356678 --> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbadfabbf0 con 0x55bbae05c560 2024-04-24T09:09:26.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.671+0000 7fe8b5200640 1 --2- 172.21.15.69:0/3272356678 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55bbae05c560 0x55bbae05ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.672+0000 7fe8b5200640 1 --2- 172.21.15.69:0/3272356678 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55bbae05c560 0x55bbae05ea20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d224081b2e9ed96 server_cookie=c9e57970c97d0b9a in_seq=0 out_seq=0 2024-04-24T09:09:26.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.672+0000 7fe897400640 10 client.16356 ms_handle_connect on v2:172.21.15.81:6834/3260829578 2024-04-24T09:09:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.675+0000 7fe897400640 1 -- 172.21.15.69:0/3272356678 <== mds.2 v2:172.21.15.81:6834/3260829578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bbadfabbf0 con 0x55bbae05c560 2024-04-24T09:09:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.675+0000 7fe897400640 10 client.16356 handle_command_reply: tid=0 2024-04-24T09:09:26.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.675+0000 7fe8bb3e0740 1 client.16356 shutdown 2024-04-24T09:09:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.675+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3272356678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 msgr2=0x55bbae040900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.675+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/3272356678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae040900 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4003e00 tx=0x7fe8a40046c0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.675+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3272356678 shutdown_connections 2024-04-24T09:09:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.675+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/3272356678 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55bbae05c560 0x55bbae05ea20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.675+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/3272356678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae043b30 0x55bbae041380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.675+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/3272356678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae039460 0x55bbae040e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.675+0000 7fe897400640 0 client.16356 ms_handle_reset on v2:172.21.15.81:6834/3260829578 2024-04-24T09:09:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.675+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/3272356678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae040900 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.675+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3272356678 >> 172.21.15.69:0/3272356678 conn(0x55bbae034e60 msgr2=0x55bbae03ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.676+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3272356678 shutdown_connections 2024-04-24T09:09:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.676+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3272356678 wait complete. 2024-04-24T09:09:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.676+0000 7fe8bb3e0740 20 client.16356 trim_cache size 0 max 16384 2024-04-24T09:09:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.692+0000 7fe8bb3e0740 1 Processor -- start 2024-04-24T09:09:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.692+0000 7fe8bb3e0740 1 -- start start 2024-04-24T09:09:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.692+0000 7fe8bb3e0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae038b40 0x55bbae058030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.692+0000 7fe8bb3e0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae039460 0x55bbae0642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.692+0000 7fe8bb3e0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae043b30 0x55bbae064830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.692+0000 7fe8bb3e0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b01af780 con 0x55bbae043b30 2024-04-24T09:09:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.692+0000 7fe8bb3e0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b01ad020 con 0x55bbae039460 2024-04-24T09:09:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.692+0000 7fe8bb3e0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b01aae90 con 0x55bbae038b40 2024-04-24T09:09:26.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.692+0000 7fe8b6600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae043b30 0x55bbae064830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.692+0000 7fe8b6600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae043b30 0x55bbae064830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51640/0 (socket says 172.21.15.69:51640) 2024-04-24T09:09:26.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.693+0000 7fe8b6600640 1 -- 172.21.15.69:0/3137914668 learned_addr learned my addr 172.21.15.69:0/3137914668 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:26.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.693+0000 7fe8b5c00640 1 --2- 172.21.15.69:0/3137914668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae038b40 0x55bbae058030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.693+0000 7fe8b6600640 1 -- 172.21.15.69:0/3137914668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae038b40 msgr2=0x55bbae058030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:26.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.693+0000 7fe8b6600640 1 --2- 172.21.15.69:0/3137914668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae038b40 0x55bbae058030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.693+0000 7fe8b6600640 1 -- 172.21.15.69:0/3137914668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae039460 msgr2=0x55bbae0642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:26.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.693+0000 7fe8b6600640 1 --2- 172.21.15.69:0/3137914668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae039460 0x55bbae0642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.693+0000 7fe8b6600640 1 -- 172.21.15.69:0/3137914668 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbadd7a7d0 con 0x55bbae043b30 2024-04-24T09:09:26.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.693+0000 7fe8b6600640 1 --2- 172.21.15.69:0/3137914668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae043b30 0x55bbae064830 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8006010 tx=0x7fe8a8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a99720c15b9752e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:26.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.694+0000 7fe8a2000640 1 -- 172.21.15.69:0/3137914668 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8016050 con 0x55bbae043b30 2024-04-24T09:09:26.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.694+0000 7fe8a2000640 1 -- 172.21.15.69:0/3137914668 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8a8005110 con 0x55bbae043b30 2024-04-24T09:09:26.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.694+0000 7fe8a2000640 1 -- 172.21.15.69:0/3137914668 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80126f0 con 0x55bbae043b30 2024-04-24T09:09:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.694+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3137914668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae043b30 msgr2=0x55bbae064830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.694+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/3137914668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae043b30 0x55bbae064830 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8006010 tx=0x7fe8a8009c20 comp rx=0 tx=0).stop 2024-04-24T09:09:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.694+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3137914668 shutdown_connections 2024-04-24T09:09:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.694+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/3137914668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae043b30 0x55bbae064830 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.694+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/3137914668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae039460 0x55bbae0642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.694+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/3137914668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae038b40 0x55bbae058030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.695+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3137914668 >> 172.21.15.69:0/3137914668 conn(0x55bbae034e60 msgr2=0x55bbae03f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.695+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3137914668 shutdown_connections 2024-04-24T09:09:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.695+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3137914668 wait complete. 2024-04-24T09:09:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.695+0000 7fe8bb3e0740 1 -- start start 2024-04-24T09:09:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.695+0000 7fe8bb3e0740 1 Processor -- start 2024-04-24T09:09:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.696+0000 7fe8bb3e0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae044840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.696+0000 7fe8bb3e0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae039460 0x55bbae044d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.696+0000 7fe8bb3e0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae043b30 0x55bbae0452c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.696+0000 7fe8bb3e0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b01ab4e0 con 0x55bbae038b40 2024-04-24T09:09:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.696+0000 7fe8bb3e0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b01ac950 con 0x55bbae043b30 2024-04-24T09:09:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.696+0000 7fe8bb3e0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b01abbb0 con 0x55bbae039460 2024-04-24T09:09:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.696+0000 7fe8bb3e0740 10 client.0 _ll_register_callbacks cb 0x55bbae043a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.696+0000 7fe8b5c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae044840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.696+0000 7fe8b5c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae044840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51642/0 (socket says 172.21.15.69:51642) 2024-04-24T09:09:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.696+0000 7fe8b5c00640 1 -- 172.21.15.69:0/3007724554 learned_addr learned my addr 172.21.15.69:0/3007724554 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:26.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.696+0000 7fe8b5200640 1 --2- 172.21.15.69:0/3007724554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae039460 0x55bbae044d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.697+0000 7fe8b6600640 1 --2- 172.21.15.69:0/3007724554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae043b30 0x55bbae0452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.697+0000 7fe8b5c00640 1 -- 172.21.15.69:0/3007724554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae039460 msgr2=0x55bbae044d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:26.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.697+0000 7fe8b5c00640 1 --2- 172.21.15.69:0/3007724554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae039460 0x55bbae044d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.697+0000 7fe8b5c00640 1 -- 172.21.15.69:0/3007724554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae043b30 msgr2=0x55bbae0452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.697+0000 7fe8b5c00640 1 --2- 172.21.15.69:0/3007724554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae043b30 0x55bbae0452c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.697+0000 7fe8b5c00640 1 -- 172.21.15.69:0/3007724554 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbae052680 con 0x55bbae038b40 2024-04-24T09:09:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.697+0000 7fe8b5c00640 1 --2- 172.21.15.69:0/3007724554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae044840 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400bfd0 tx=0x7fe8a401b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2b6a59e5fa94a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.697+0000 7fe897400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.697+0000 7fe897400640 1 -- 172.21.15.69:0/3007724554 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4038020 con 0x55bbae038b40 2024-04-24T09:09:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.697+0000 7fe897400640 1 -- 172.21.15.69:0/3007724554 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe8a4005450 con 0x55bbae038b40 2024-04-24T09:09:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.697+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3007724554 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8a8005110 con 0x55bbae038b40 2024-04-24T09:09:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.697+0000 7fe897400640 1 -- 172.21.15.69:0/3007724554 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe8a4043050 con 0x55bbae038b40 2024-04-24T09:09:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.698+0000 7fe8bb3e0740 10 client.16362 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.698+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3007724554 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bbae0c2d70 con 0x55bbae038b40 2024-04-24T09:09:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.698+0000 7fe897400640 1 -- 172.21.15.69:0/3007724554 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8a8005110 con 0x55bbae038b40 2024-04-24T09:09:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.698+0000 7fe897400640 1 -- 172.21.15.69:0/3007724554 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7fe8a4024040 con 0x55bbae038b40 2024-04-24T09:09:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.698+0000 7fe8bb3e0740 10 client.16362 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.698+0000 7fe8bb3e0740 10 client.16362 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:09:26.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.698+0000 7fe8bb3e0740 20 client.16362 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:26.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.698+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/3007724554 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55bbae0c3510 0x55bbae0c5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.698+0000 7fe8bb3e0740 4 client.16362 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:09:26.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.698+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3007724554 --> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbadfa5a60 con 0x55bbae0c3510 2024-04-24T09:09:26.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.699+0000 7fe8b5200640 1 --2- 172.21.15.69:0/3007724554 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55bbae0c3510 0x55bbae0c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.700+0000 7fe897400640 10 client.16362 ms_handle_connect on v2:172.21.15.81:6834/3260829578 2024-04-24T09:09:26.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.700+0000 7fe8b5200640 1 --2- 172.21.15.69:0/3007724554 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55bbae0c3510 0x55bbae0c5930 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d27588cefb00ee7 server_cookie=cab3868c75c165a in_seq=0 out_seq=0 2024-04-24T09:09:26.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.701+0000 7fe897400640 1 -- 172.21.15.69:0/3007724554 <== mds.2 v2:172.21.15.81:6834/3260829578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbadfa5a60 con 0x55bbae0c3510 2024-04-24T09:09:26.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.701+0000 7fe897400640 10 client.16362 handle_command_reply: tid=0 2024-04-24T09:09:26.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.701+0000 7fe8bb3e0740 1 client.16362 shutdown 2024-04-24T09:09:26.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.701+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3007724554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 msgr2=0x55bbae044840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.701+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/3007724554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae044840 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400bfd0 tx=0x7fe8a401b710 comp rx=0 tx=0).stop 2024-04-24T09:09:26.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.701+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3007724554 shutdown_connections 2024-04-24T09:09:26.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.701+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/3007724554 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x55bbae0c3510 0x55bbae0c5930 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.701+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/3007724554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbae043b30 0x55bbae0452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.701+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/3007724554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bbae039460 0x55bbae044d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.701+0000 7fe8bb3e0740 1 --2- 172.21.15.69:0/3007724554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbae038b40 0x55bbae044840 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.701+0000 7fe897400640 0 client.16362 ms_handle_reset on v2:172.21.15.81:6834/3260829578 2024-04-24T09:09:26.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.701+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3007724554 >> 172.21.15.69:0/3007724554 conn(0x55bbae034e60 msgr2=0x55bbae040800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:26.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.701+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3007724554 shutdown_connections 2024-04-24T09:09:26.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.701+0000 7fe8bb3e0740 1 -- 172.21.15.69:0/3007724554 wait complete. 2024-04-24T09:09:26.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.701+0000 7fe8bb3e0740 20 client.16362 trim_cache size 0 max 16384 2024-04-24T09:09:26.703 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-04-24T09:09:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.702+0000 7fe8b7000640 1 -- 172.21.15.69:0/584974983 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe8980a3d20 msgr2=0x7fe8980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.702+0000 7fe8b7000640 1 --2- 172.21.15.69:0/584974983 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe8980a3d20 0x7fe8980a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55bbae03a100 tx=0x7fe8a400b390 comp rx=0 tx=0).stop 2024-04-24T09:09:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.702+0000 7fe8b7000640 1 -- 172.21.15.69:0/584974983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b01071e0 msgr2=0x7fe8b01a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.702+0000 7fe8b7000640 1 --2- 172.21.15.69:0/584974983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b01071e0 0x7fe8b01a0b80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe89c005ab0 tx=0x7fe89c003cd0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.702+0000 7fe8b7000640 1 -- 172.21.15.69:0/584974983 shutdown_connections 2024-04-24T09:09:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.702+0000 7fe8b7000640 1 --2- 172.21.15.69:0/584974983 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe8980a3d20 0x7fe8980a61e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.702+0000 7fe8b7000640 1 --2- 172.21.15.69:0/584974983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8b019a7b0 0x7fe8b019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.702+0000 7fe8b7000640 1 --2- 172.21.15.69:0/584974983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b01071e0 0x7fe8b01a0b80 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.702+0000 7fe8b7000640 1 --2- 172.21.15.69:0/584974983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8b0102110 0x7fe8b01a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.702+0000 7fe8b7000640 1 -- 172.21.15.69:0/584974983 >> 172.21.15.69:0/584974983 conn(0x7fe8b00fd9d0 msgr2=0x7fe8b00ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.703+0000 7fe8b7000640 1 -- 172.21.15.69:0/584974983 shutdown_connections 2024-04-24T09:09:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.703+0000 7fe8b7000640 1 -- 172.21.15.69:0/584974983 wait complete. 2024-04-24T09:09:26.719 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:09:26.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.926+0000 7fafd7200640 1 -- 172.21.15.69:0/3783718399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafd0106220 msgr2=0x7fafd010ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.926+0000 7fafd7200640 1 --2- 172.21.15.69:0/3783718399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafd0106220 0x7fafd010ddf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009ca0 tx=0x7fafc802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.926+0000 7fafd7200640 1 -- 172.21.15.69:0/3783718399 shutdown_connections 2024-04-24T09:09:26.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.926+0000 7fafd7200640 1 --2- 172.21.15.69:0/3783718399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafd0106220 0x7fafd010ddf0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.926+0000 7fafd7200640 1 --2- 172.21.15.69:0/3783718399 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fafd0101b00 0x7fafd0105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.926+0000 7fafd7200640 1 --2- 172.21.15.69:0/3783718399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fafd0101150 0x7fafd0101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.926+0000 7fafd7200640 1 -- 172.21.15.69:0/3783718399 >> 172.21.15.69:0/3783718399 conn(0x7fafd0077f60 msgr2=0x7fafd0078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:26.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.926+0000 7fafd7200640 1 -- 172.21.15.69:0/3783718399 shutdown_connections 2024-04-24T09:09:26.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.926+0000 7fafd7200640 1 -- 172.21.15.69:0/3783718399 wait complete. 2024-04-24T09:09:26.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.927+0000 7fafd7200640 1 Processor -- start 2024-04-24T09:09:26.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.928+0000 7fafd7200640 1 -- start start 2024-04-24T09:09:26.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.928+0000 7fafd7200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fafd0101150 0x7fafd01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.928+0000 7fafd7200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fafd0106220 0x7fafd01a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.928+0000 7fafd7200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafd019a820 0x7fafd019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.928+0000 7fafd7200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fafd01122a0 con 0x7fafd019a820 2024-04-24T09:09:26.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.928+0000 7fafd7200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fafd0112120 con 0x7fafd0106220 2024-04-24T09:09:26.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.928+0000 7fafd7200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fafd0112420 con 0x7fafd0101150 2024-04-24T09:09:26.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.928+0000 7fafd5200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafd019a820 0x7fafd019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.928+0000 7fafd5200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafd019a820 0x7fafd019acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51662/0 (socket says 172.21.15.69:51662) 2024-04-24T09:09:26.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.928+0000 7fafd5200640 1 -- 172.21.15.69:0/3612997436 learned_addr learned my addr 172.21.15.69:0/3612997436 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:26.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.929+0000 7fafcf400640 1 --2- 172.21.15.69:0/3612997436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fafd0106220 0x7fafd01a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.929+0000 7fafcfe00640 1 --2- 172.21.15.69:0/3612997436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fafd0101150 0x7fafd01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.929+0000 7fafd5200640 1 -- 172.21.15.69:0/3612997436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fafd0101150 msgr2=0x7fafd01a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.929+0000 7fafd5200640 1 --2- 172.21.15.69:0/3612997436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fafd0101150 0x7fafd01a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.929+0000 7fafd5200640 1 -- 172.21.15.69:0/3612997436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fafd0106220 msgr2=0x7fafd01a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.929+0000 7fafd5200640 1 --2- 172.21.15.69:0/3612997436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fafd0106220 0x7fafd01a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.929+0000 7fafd5200640 1 -- 172.21.15.69:0/3612997436 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd019b3b0 con 0x7fafd019a820 2024-04-24T09:09:26.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.930+0000 7fafd5200640 1 --2- 172.21.15.69:0/3612997436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafd019a820 0x7fafd019acd0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009dc0 tx=0x7fafc8032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:26.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.930+0000 7fafccc00640 1 -- 172.21.15.69:0/3612997436 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8003cf0 con 0x7fafd019a820 2024-04-24T09:09:26.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.930+0000 7fafd7200640 1 -- 172.21.15.69:0/3612997436 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd019b640 con 0x7fafd019a820 2024-04-24T09:09:26.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.930+0000 7fafccc00640 1 -- 172.21.15.69:0/3612997436 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafc80044f0 con 0x7fafd019a820 2024-04-24T09:09:26.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.930+0000 7fafccc00640 1 -- 172.21.15.69:0/3612997436 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8030bf0 con 0x7fafd019a820 2024-04-24T09:09:26.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.930+0000 7fafd7200640 1 -- 172.21.15.69:0/3612997436 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd01a77d0 con 0x7fafd019a820 2024-04-24T09:09:26.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.934+0000 7fafccc00640 1 -- 172.21.15.69:0/3612997436 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fafc8004040 con 0x7fafd019a820 2024-04-24T09:09:26.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.935+0000 7fafda164740 1 Processor -- start 2024-04-24T09:09:26.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.935+0000 7fafda164740 1 -- start start 2024-04-24T09:09:26.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.935+0000 7fafda164740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d0950 0x5633972d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.935+0000 7fafda164740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972d1270 0x5633972db400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.935+0000 7fafda164740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972db940 0x5633972ddd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.935+0000 7fafda164740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5633972361d0 con 0x5633972d1270 2024-04-24T09:09:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.935+0000 7fafda164740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5633971dc450 con 0x5633972db940 2024-04-24T09:09:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.935+0000 7fafda164740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563397232190 con 0x5633972d0950 2024-04-24T09:09:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.935+0000 7fafcfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972d1270 0x5633972db400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.935+0000 7fafcfe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972d1270 0x5633972db400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51668/0 (socket says 172.21.15.69:51668) 2024-04-24T09:09:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.935+0000 7fafcfe00640 1 -- 172.21.15.69:0/3903742917 learned_addr learned my addr 172.21.15.69:0/3903742917 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.936+0000 7fafcf400640 1 --2- 172.21.15.69:0/3903742917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d0950 0x5633972d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.936+0000 7fafcfe00640 1 -- 172.21.15.69:0/3903742917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d0950 msgr2=0x5633972d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.936+0000 7fafccc00640 1 --2- 172.21.15.69:0/3612997436 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fafb40a3da0 0x7fafb40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.936+0000 7fafcfe00640 1 --2- 172.21.15.69:0/3903742917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d0950 0x5633972d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.936+0000 7fafcfe00640 1 -- 172.21.15.69:0/3903742917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972db940 msgr2=0x5633972ddd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.936+0000 7fafcfe00640 1 --2- 172.21.15.69:0/3903742917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972db940 0x5633972ddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.936+0000 7fafcfe00640 1 -- 172.21.15.69:0/3903742917 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563397299310 con 0x5633972d1270 2024-04-24T09:09:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.936+0000 7fafcf400640 1 --2- 172.21.15.69:0/3903742917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d0950 0x5633972d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.936+0000 7fafccc00640 1 -- 172.21.15.69:0/3612997436 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fafc80f2030 con 0x7fafd019a820 2024-04-24T09:09:26.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.936+0000 7fafd5200640 1 --2- 172.21.15.69:0/3612997436 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fafb40a3da0 0x7fafb40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.936+0000 7fafcfe00640 1 --2- 172.21.15.69:0/3903742917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972d1270 0x5633972db400 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fafc000bd10 tx=0x7fafc00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4744109fee2ffdf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:26.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.937+0000 7fafc5600640 1 -- 172.21.15.69:0/3903742917 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc0004030 con 0x5633972d1270 2024-04-24T09:09:26.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.937+0000 7fafc5600640 1 -- 172.21.15.69:0/3903742917 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafc00035f0 con 0x5633972d1270 2024-04-24T09:09:26.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.937+0000 7fafc5600640 1 -- 172.21.15.69:0/3903742917 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc00117b0 con 0x5633972d1270 2024-04-24T09:09:26.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.938+0000 7fafda164740 1 -- 172.21.15.69:0/3903742917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972d1270 msgr2=0x5633972db400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.938+0000 7fafd5200640 1 --2- 172.21.15.69:0/3612997436 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fafb40a3da0 0x7fafb40a6260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fafc8039040 tx=0x7fafc803e040 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:26.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.938+0000 7fafda164740 1 --2- 172.21.15.69:0/3903742917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972d1270 0x5633972db400 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fafc000bd10 tx=0x7fafc00107a0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.938+0000 7fafda164740 1 -- 172.21.15.69:0/3903742917 shutdown_connections 2024-04-24T09:09:26.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.938+0000 7fafda164740 1 --2- 172.21.15.69:0/3903742917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972db940 0x5633972ddd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.938+0000 7fafda164740 1 --2- 172.21.15.69:0/3903742917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972d1270 0x5633972db400 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.938+0000 7fafda164740 1 --2- 172.21.15.69:0/3903742917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d0950 0x5633972d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.938+0000 7fafda164740 1 -- 172.21.15.69:0/3903742917 >> 172.21.15.69:0/3903742917 conn(0x5633972ccc70 msgr2=0x5633972cd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:26.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.938+0000 7fafda164740 1 -- 172.21.15.69:0/3903742917 shutdown_connections 2024-04-24T09:09:26.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.938+0000 7fafda164740 1 -- 172.21.15.69:0/3903742917 wait complete. 2024-04-24T09:09:26.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.938+0000 7fafda164740 1 -- start start 2024-04-24T09:09:26.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.939+0000 7fafda164740 1 Processor -- start 2024-04-24T09:09:26.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.939+0000 7fafda164740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d0950 0x5633972db3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.939+0000 7fafda164740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972d1270 0x5633972d8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.939+0000 7fafda164740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972db940 0x5633972d8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.939+0000 7fafda164740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5633972de7d0 con 0x5633972db940 2024-04-24T09:09:26.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.939+0000 7fafda164740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5633971ef810 con 0x5633972d1270 2024-04-24T09:09:26.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.939+0000 7fafda164740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5633972de950 con 0x5633972d0950 2024-04-24T09:09:26.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.939+0000 7fafda164740 10 client.0 _ll_register_callbacks cb 0x5633971bd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:26.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.940+0000 7fafcf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972db940 0x5633972d8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.940+0000 7fafcf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972db940 0x5633972d8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51676/0 (socket says 172.21.15.69:51676) 2024-04-24T09:09:26.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.940+0000 7fafcf400640 1 -- 172.21.15.69:0/643274194 learned_addr learned my addr 172.21.15.69:0/643274194 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:26.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.940+0000 7fafcfe00640 1 --2- 172.21.15.69:0/643274194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972d1270 0x5633972d8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.940+0000 7fafcf400640 1 --2- 172.21.15.69:0/643274194 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d0950 0x5633972db3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.940+0000 7fafcf400640 1 -- 172.21.15.69:0/643274194 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d0950 msgr2=0x5633972db3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:26.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.940+0000 7fafcf400640 1 --2- 172.21.15.69:0/643274194 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d0950 0x5633972db3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.940+0000 7fafcf400640 1 -- 172.21.15.69:0/643274194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972d1270 msgr2=0x5633972d8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:26.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.940+0000 7fafcf400640 1 --2- 172.21.15.69:0/643274194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972d1270 0x5633972d8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.940+0000 7fafcf400640 1 -- 172.21.15.69:0/643274194 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633972003f0 con 0x5633972db940 2024-04-24T09:09:26.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.940+0000 7fafcf400640 1 --2- 172.21.15.69:0/643274194 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972db940 0x5633972d8e90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fafd006a4d0 tx=0x7fafbc00a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3394d538125cfd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:26.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.940+0000 7fafba000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:26.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.941+0000 7fafba000640 1 -- 172.21.15.69:0/643274194 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc00ee00 con 0x5633972db940 2024-04-24T09:09:26.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.941+0000 7fafba000640 1 -- 172.21.15.69:0/643274194 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fafbc004020 con 0x5633972db940 2024-04-24T09:09:26.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.941+0000 7fafda164740 1 -- 172.21.15.69:0/643274194 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafc00035f0 con 0x5633972db940 2024-04-24T09:09:26.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.941+0000 7fafba000640 1 -- 172.21.15.69:0/643274194 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fafbc024050 con 0x5633972db940 2024-04-24T09:09:26.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.941+0000 7fafda164740 10 client.16374 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:26.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.941+0000 7fafda164740 1 -- 172.21.15.69:0/643274194 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563396f483f0 con 0x5633972db940 2024-04-24T09:09:26.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.941+0000 7fafba000640 1 -- 172.21.15.69:0/643274194 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafc00035f0 con 0x5633972db940 2024-04-24T09:09:26.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.942+0000 7fafba000640 1 -- 172.21.15.69:0/643274194 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7fafbc004830 con 0x5633972db940 2024-04-24T09:09:26.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.942+0000 7fafda164740 10 client.16374 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:26.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.942+0000 7fafda164740 10 client.16374 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:09:26.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.942+0000 7fafda164740 20 client.16374 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:26.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.942+0000 7fafda164740 1 --2- 172.21.15.69:0/643274194 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5633972ebb00 0x5633972edf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.942+0000 7fafda164740 4 client.16374 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:26.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.942+0000 7fafda164740 1 -- 172.21.15.69:0/643274194 --> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563397243bf0 con 0x5633972ebb00 2024-04-24T09:09:26.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.943+0000 7fafcfe00640 1 --2- 172.21.15.69:0/643274194 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5633972ebb00 0x5633972edf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.944+0000 7fafcfe00640 1 --2- 172.21.15.69:0/643274194 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5633972ebb00 0x5633972edf40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d8641547c3ee8ed server_cookie=2ce21c1eadc06da7 in_seq=0 out_seq=0 2024-04-24T09:09:26.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.944+0000 7fafba000640 10 client.16374 ms_handle_connect on v2:172.21.15.81:6836/1973819593 2024-04-24T09:09:26.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.945+0000 7fafba000640 1 -- 172.21.15.69:0/643274194 <== mds.1 v2:172.21.15.81:6836/1973819593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563397243bf0 con 0x5633972ebb00 2024-04-24T09:09:26.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.945+0000 7fafba000640 10 client.16374 handle_command_reply: tid=0 2024-04-24T09:09:26.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.945+0000 7fafda164740 1 client.16374 shutdown 2024-04-24T09:09:26.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.946+0000 7fafda164740 1 -- 172.21.15.69:0/643274194 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972db940 msgr2=0x5633972d8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.946+0000 7fafda164740 1 --2- 172.21.15.69:0/643274194 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972db940 0x5633972d8e90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fafd006a4d0 tx=0x7fafbc00a730 comp rx=0 tx=0).stop 2024-04-24T09:09:26.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.946+0000 7fafda164740 1 -- 172.21.15.69:0/643274194 shutdown_connections 2024-04-24T09:09:26.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.946+0000 7fafda164740 1 --2- 172.21.15.69:0/643274194 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x5633972ebb00 0x5633972edf40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.946+0000 7fafda164740 1 --2- 172.21.15.69:0/643274194 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972db940 0x5633972d8e90 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.946+0000 7fafda164740 1 --2- 172.21.15.69:0/643274194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972d1270 0x5633972d8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.946+0000 7fafda164740 1 --2- 172.21.15.69:0/643274194 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d0950 0x5633972db3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.946+0000 7fafda164740 1 -- 172.21.15.69:0/643274194 >> 172.21.15.69:0/643274194 conn(0x5633972ccc70 msgr2=0x5633972d2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:26.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.946+0000 7fafba000640 0 client.16374 ms_handle_reset on v2:172.21.15.81:6836/1973819593 2024-04-24T09:09:26.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.946+0000 7fafda164740 1 -- 172.21.15.69:0/643274194 shutdown_connections 2024-04-24T09:09:26.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.946+0000 7fafda164740 1 -- 172.21.15.69:0/643274194 wait complete. 2024-04-24T09:09:26.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.946+0000 7fafda164740 20 client.16374 trim_cache size 0 max 16384 2024-04-24T09:09:26.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.963+0000 7fafda164740 1 Processor -- start 2024-04-24T09:09:26.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.963+0000 7fafda164740 1 -- start start 2024-04-24T09:09:26.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.963+0000 7fafda164740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972d0950 0x5633972e7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.963+0000 7fafda164740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972d1270 0x5633972d8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.963+0000 7fafda164740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972da690 0x5633972f3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.963+0000 7fafda164740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fafd01b3db0 con 0x5633972d0950 2024-04-24T09:09:26.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.963+0000 7fafda164740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fafd01b1650 con 0x5633972d1270 2024-04-24T09:09:26.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.963+0000 7fafda164740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fafd01af4a0 con 0x5633972da690 2024-04-24T09:09:26.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.964+0000 7fafcf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972d0950 0x5633972e7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.964+0000 7fafcf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972d0950 0x5633972e7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51692/0 (socket says 172.21.15.69:51692) 2024-04-24T09:09:26.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.964+0000 7fafcf400640 1 -- 172.21.15.69:0/179289934 learned_addr learned my addr 172.21.15.69:0/179289934 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:26.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.964+0000 7fafcfe00640 1 --2- 172.21.15.69:0/179289934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972d1270 0x5633972d8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.964+0000 7fafcf400640 1 -- 172.21.15.69:0/179289934 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972da690 msgr2=0x5633972f3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:26.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.964+0000 7fafcf400640 1 --2- 172.21.15.69:0/179289934 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972da690 0x5633972f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.964+0000 7fafcf400640 1 -- 172.21.15.69:0/179289934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972d1270 msgr2=0x5633972d8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:26.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.964+0000 7fafcf400640 1 --2- 172.21.15.69:0/179289934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972d1270 0x5633972d8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.964+0000 7fafcf400640 1 -- 172.21.15.69:0/179289934 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633972c8450 con 0x5633972d0950 2024-04-24T09:09:26.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.964+0000 7fafcfe00640 1 --2- 172.21.15.69:0/179289934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972d1270 0x5633972d8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:26.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.964+0000 7fafcf400640 1 --2- 172.21.15.69:0/179289934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972d0950 0x5633972e7690 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00a3b0 tx=0x7fafbc010870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd816b57419477b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:26.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.965+0000 7fafc4c00640 1 -- 172.21.15.69:0/179289934 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc02d030 con 0x5633972d0950 2024-04-24T09:09:26.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.965+0000 7fafc4c00640 1 -- 172.21.15.69:0/179289934 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafbc005ce0 con 0x5633972d0950 2024-04-24T09:09:26.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.965+0000 7fafc4c00640 1 -- 172.21.15.69:0/179289934 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc0050e0 con 0x5633972d0950 2024-04-24T09:09:26.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.965+0000 7fafda164740 1 -- 172.21.15.69:0/179289934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972d0950 msgr2=0x5633972e7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.965+0000 7fafda164740 1 --2- 172.21.15.69:0/179289934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972d0950 0x5633972e7690 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00a3b0 tx=0x7fafbc010870 comp rx=0 tx=0).stop 2024-04-24T09:09:26.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.966+0000 7fafda164740 1 -- 172.21.15.69:0/179289934 shutdown_connections 2024-04-24T09:09:26.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.966+0000 7fafda164740 1 --2- 172.21.15.69:0/179289934 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972da690 0x5633972f3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.966+0000 7fafda164740 1 --2- 172.21.15.69:0/179289934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972d1270 0x5633972d8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.966+0000 7fafda164740 1 --2- 172.21.15.69:0/179289934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972d0950 0x5633972e7690 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.966+0000 7fafda164740 1 -- 172.21.15.69:0/179289934 >> 172.21.15.69:0/179289934 conn(0x5633972ccc70 msgr2=0x5633972d9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:26.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.966+0000 7fafda164740 1 -- 172.21.15.69:0/179289934 shutdown_connections 2024-04-24T09:09:26.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.966+0000 7fafda164740 1 -- 172.21.15.69:0/179289934 wait complete. 2024-04-24T09:09:26.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.966+0000 7fafda164740 1 -- start start 2024-04-24T09:09:26.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.967+0000 7fafda164740 1 Processor -- start 2024-04-24T09:09:26.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.967+0000 7fafda164740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972d0950 0x5633972d5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.967+0000 7fafda164740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d1270 0x5633972d6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.967+0000 7fafda164740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972da690 0x5633972e22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.967+0000 7fafda164740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fafd01afb10 con 0x5633972da690 2024-04-24T09:09:26.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.967+0000 7fafda164740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fafd01b0f80 con 0x5633972d0950 2024-04-24T09:09:26.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.967+0000 7fafda164740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fafd01b01e0 con 0x5633972d1270 2024-04-24T09:09:26.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.967+0000 7fafda164740 10 client.0 _ll_register_callbacks cb 0x7fafbc00a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:26.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.968+0000 7fafcf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972da690 0x5633972e22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.968+0000 7fafcf400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972da690 0x5633972e22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51708/0 (socket says 172.21.15.69:51708) 2024-04-24T09:09:26.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.968+0000 7fafcf400640 1 -- 172.21.15.69:0/1697248718 learned_addr learned my addr 172.21.15.69:0/1697248718 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:26.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.968+0000 7fafcf400640 1 --2- 172.21.15.69:0/1697248718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972d0950 0x5633972d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.968+0000 7fafcfe00640 1 --2- 172.21.15.69:0/1697248718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d1270 0x5633972d6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.968+0000 7fafcf400640 1 -- 172.21.15.69:0/1697248718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d1270 msgr2=0x5633972d6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:26.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.968+0000 7fafcf400640 1 --2- 172.21.15.69:0/1697248718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d1270 0x5633972d6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.968+0000 7fafcf400640 1 -- 172.21.15.69:0/1697248718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972d0950 msgr2=0x5633972d5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:26.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.968+0000 7fafcf400640 1 --2- 172.21.15.69:0/1697248718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972d0950 0x5633972d5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.968+0000 7fafcf400640 1 -- 172.21.15.69:0/1697248718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633972d10a0 con 0x5633972da690 2024-04-24T09:09:26.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.968+0000 7fafcfe00640 1 --2- 172.21.15.69:0/1697248718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d1270 0x5633972d6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:26.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.968+0000 7fafcf400640 1 --2- 172.21.15.69:0/1697248718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972da690 0x5633972e22b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00afe0 tx=0x7fafbc010b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6380f793c221baa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:26.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.968+0000 7fafba000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:26.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.969+0000 7fafba000640 1 -- 172.21.15.69:0/1697248718 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc02d070 con 0x5633972da690 2024-04-24T09:09:26.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.969+0000 7fafba000640 1 -- 172.21.15.69:0/1697248718 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fafbc010030 con 0x5633972da690 2024-04-24T09:09:26.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.969+0000 7fafda164740 1 -- 172.21.15.69:0/1697248718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafbc005ce0 con 0x5633972da690 2024-04-24T09:09:26.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.969+0000 7fafba000640 1 -- 172.21.15.69:0/1697248718 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fafbc010230 con 0x5633972da690 2024-04-24T09:09:26.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.969+0000 7fafda164740 10 client.16380 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:26.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.969+0000 7fafda164740 1 -- 172.21.15.69:0/1697248718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5633970127d0 con 0x5633972da690 2024-04-24T09:09:26.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.969+0000 7fafba000640 1 -- 172.21.15.69:0/1697248718 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafbc005ce0 con 0x5633972da690 2024-04-24T09:09:26.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.970+0000 7fafba000640 1 -- 172.21.15.69:0/1697248718 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7fafbc019070 con 0x5633972da690 2024-04-24T09:09:26.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.970+0000 7fafda164740 10 client.16380 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:26.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.970+0000 7fafda164740 10 client.16380 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:09:26.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.970+0000 7fafda164740 20 client.16380 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:26.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.970+0000 7fafda164740 1 --2- 172.21.15.69:0/1697248718 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x563397352af0 0x563397354f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:26.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.970+0000 7fafda164740 4 client.16380 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:09:26.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.970+0000 7fafda164740 1 -- 172.21.15.69:0/1697248718 --> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56339723da60 con 0x563397352af0 2024-04-24T09:09:26.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.970+0000 7fafcfe00640 1 --2- 172.21.15.69:0/1697248718 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x563397352af0 0x563397354f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:26.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.971+0000 7fafcfe00640 1 --2- 172.21.15.69:0/1697248718 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x563397352af0 0x563397354f10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1769733e3a74a53c server_cookie=44ddda54eedbe8cf in_seq=0 out_seq=0 2024-04-24T09:09:26.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.971+0000 7fafba000640 10 client.16380 ms_handle_connect on v2:172.21.15.81:6836/1973819593 2024-04-24T09:09:26.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.972+0000 7fafba000640 1 -- 172.21.15.69:0/1697248718 <== mds.1 v2:172.21.15.81:6836/1973819593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56339723da60 con 0x563397352af0 2024-04-24T09:09:26.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.972+0000 7fafba000640 10 client.16380 handle_command_reply: tid=0 2024-04-24T09:09:26.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.972+0000 7fafda164740 1 client.16380 shutdown 2024-04-24T09:09:26.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.972+0000 7fafda164740 1 -- 172.21.15.69:0/1697248718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972da690 msgr2=0x5633972e22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.972+0000 7fafda164740 1 --2- 172.21.15.69:0/1697248718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972da690 0x5633972e22b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00afe0 tx=0x7fafbc010b10 comp rx=0 tx=0).stop 2024-04-24T09:09:26.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.972+0000 7fafda164740 1 -- 172.21.15.69:0/1697248718 shutdown_connections 2024-04-24T09:09:26.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.972+0000 7fafda164740 1 --2- 172.21.15.69:0/1697248718 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x563397352af0 0x563397354f10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.972+0000 7fafda164740 1 --2- 172.21.15.69:0/1697248718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633972da690 0x5633972e22b0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.972+0000 7fafda164740 1 --2- 172.21.15.69:0/1697248718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5633972d1270 0x5633972d6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.972+0000 7fafba000640 0 client.16380 ms_handle_reset on v2:172.21.15.81:6836/1973819593 2024-04-24T09:09:26.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.972+0000 7fafda164740 1 --2- 172.21.15.69:0/1697248718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633972d0950 0x5633972d5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.972+0000 7fafda164740 1 -- 172.21.15.69:0/1697248718 >> 172.21.15.69:0/1697248718 conn(0x5633972ccc70 msgr2=0x5633972e6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:26.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.972+0000 7fafda164740 1 -- 172.21.15.69:0/1697248718 shutdown_connections 2024-04-24T09:09:26.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.972+0000 7fafda164740 1 -- 172.21.15.69:0/1697248718 wait complete. 2024-04-24T09:09:26.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.972+0000 7fafda164740 20 client.16380 trim_cache size 0 max 16384 2024-04-24T09:09:26.975 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-04-24T09:09:26.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.973+0000 7fafd7200640 1 -- 172.21.15.69:0/3612997436 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fafb40a3da0 msgr2=0x7fafb40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.973+0000 7fafd7200640 1 --2- 172.21.15.69:0/3612997436 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fafb40a3da0 0x7fafb40a6260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fafc8039040 tx=0x7fafc803e040 comp rx=0 tx=0).stop 2024-04-24T09:09:26.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.973+0000 7fafd7200640 1 -- 172.21.15.69:0/3612997436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafd019a820 msgr2=0x7fafd019acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:26.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.973+0000 7fafd7200640 1 --2- 172.21.15.69:0/3612997436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafd019a820 0x7fafd019acd0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009dc0 tx=0x7fafc8032ff0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.973+0000 7fafd7200640 1 -- 172.21.15.69:0/3612997436 shutdown_connections 2024-04-24T09:09:26.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.973+0000 7fafd7200640 1 --2- 172.21.15.69:0/3612997436 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fafb40a3da0 0x7fafb40a6260 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.973+0000 7fafd7200640 1 --2- 172.21.15.69:0/3612997436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafd019a820 0x7fafd019acd0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.973+0000 7fafd7200640 1 --2- 172.21.15.69:0/3612997436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fafd0106220 0x7fafd01a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.973+0000 7fafd7200640 1 --2- 172.21.15.69:0/3612997436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fafd0101150 0x7fafd01a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:26.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.973+0000 7fafd7200640 1 -- 172.21.15.69:0/3612997436 >> 172.21.15.69:0/3612997436 conn(0x7fafd0077f60 msgr2=0x7fafd010dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:26.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.973+0000 7fafd7200640 1 -- 172.21.15.69:0/3612997436 shutdown_connections 2024-04-24T09:09:26.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:26.973+0000 7fafd7200640 1 -- 172.21.15.69:0/3612997436 wait complete. 2024-04-24T09:09:26.992 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:09:27.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.201+0000 7f1d3e800640 1 -- 172.21.15.69:0/3043351678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d3810d500 msgr2=0x7f1d3810d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.201+0000 7f1d3e800640 1 --2- 172.21.15.69:0/3043351678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d3810d500 0x7f1d3810d8e0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009930 tx=0x7f1d2c02a420 comp rx=0 tx=0).stop 2024-04-24T09:09:27.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.201+0000 7f1d3e800640 1 -- 172.21.15.69:0/3043351678 shutdown_connections 2024-04-24T09:09:27.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.201+0000 7f1d3e800640 1 --2- 172.21.15.69:0/3043351678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d38108f40 0x7f1d381153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.201+0000 7f1d3e800640 1 --2- 172.21.15.69:0/3043351678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d38108580 0x7f1d38108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.201+0000 7f1d3e800640 1 --2- 172.21.15.69:0/3043351678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d3810d500 0x7f1d3810d8e0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.201+0000 7f1d3e800640 1 -- 172.21.15.69:0/3043351678 >> 172.21.15.69:0/3043351678 conn(0x7f1d38077e20 msgr2=0x7f1d38106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:27.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.201+0000 7f1d3e800640 1 -- 172.21.15.69:0/3043351678 shutdown_connections 2024-04-24T09:09:27.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.201+0000 7f1d3e800640 1 -- 172.21.15.69:0/3043351678 wait complete. 2024-04-24T09:09:27.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.202+0000 7f1d3e800640 1 Processor -- start 2024-04-24T09:09:27.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.203+0000 7f1d3e800640 1 -- start start 2024-04-24T09:09:27.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.203+0000 7f1d3e800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d38108580 0x7f1d381a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.203+0000 7f1d3e800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d38108f40 0x7f1d381a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.203+0000 7f1d3e800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d3810d500 0x7f1d3819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.203+0000 7f1d3e800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1d38118770 con 0x7f1d38108f40 2024-04-24T09:09:27.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.203+0000 7f1d3e800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1d381185f0 con 0x7f1d38108580 2024-04-24T09:09:27.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.203+0000 7f1d3e800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1d381188f0 con 0x7f1d3810d500 2024-04-24T09:09:27.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.203+0000 7f1d3ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d38108f40 0x7f1d381a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.203+0000 7f1d3ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d38108f40 0x7f1d381a4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51724/0 (socket says 172.21.15.69:51724) 2024-04-24T09:09:27.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.203+0000 7f1d3ca00640 1 -- 172.21.15.69:0/398495078 learned_addr learned my addr 172.21.15.69:0/398495078 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:27.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.204+0000 7f1d3d400640 1 --2- 172.21.15.69:0/398495078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d38108580 0x7f1d381a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.204+0000 7f1d3de00640 1 --2- 172.21.15.69:0/398495078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d3810d500 0x7f1d3819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.204+0000 7f1d3ca00640 1 -- 172.21.15.69:0/398495078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d3810d500 msgr2=0x7f1d3819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.204+0000 7f1d3ca00640 1 --2- 172.21.15.69:0/398495078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d3810d500 0x7f1d3819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.204+0000 7f1d3ca00640 1 -- 172.21.15.69:0/398495078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d38108580 msgr2=0x7f1d381a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.204+0000 7f1d3ca00640 1 --2- 172.21.15.69:0/398495078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d38108580 0x7f1d381a4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.204+0000 7f1d3ca00640 1 -- 172.21.15.69:0/398495078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3819f230 con 0x7f1d38108f40 2024-04-24T09:09:27.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.204+0000 7f1d3d400640 1 --2- 172.21.15.69:0/398495078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d38108580 0x7f1d381a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:27.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.204+0000 7f1d3ca00640 1 --2- 172.21.15.69:0/398495078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d38108f40 0x7f1d381a4e40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1d2400d770 tx=0x7f1d2400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:27.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.204+0000 7f1d36000640 1 -- 172.21.15.69:0/398495078 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d24017070 con 0x7f1d38108f40 2024-04-24T09:09:27.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.204+0000 7f1d36000640 1 -- 172.21.15.69:0/398495078 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d240054d0 con 0x7f1d38108f40 2024-04-24T09:09:27.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.204+0000 7f1d36000640 1 -- 172.21.15.69:0/398495078 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d24004cf0 con 0x7f1d38108f40 2024-04-24T09:09:27.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.205+0000 7f1d3e800640 1 -- 172.21.15.69:0/398495078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3819f520 con 0x7f1d38108f40 2024-04-24T09:09:27.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.205+0000 7f1d3e800640 1 -- 172.21.15.69:0/398495078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d381ab7c0 con 0x7f1d38108f40 2024-04-24T09:09:27.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.209+0000 7f1d42b7f740 1 Processor -- start 2024-04-24T09:09:27.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.209+0000 7f1d42b7f740 1 -- start start 2024-04-24T09:09:27.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.210+0000 7f1d36000640 1 -- 172.21.15.69:0/398495078 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1d24004e90 con 0x7f1d38108f40 2024-04-24T09:09:27.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.210+0000 7f1d42b7f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b553f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.210+0000 7f1d42b7f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b554460 0x55aa0b55e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.210+0000 7f1d42b7f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b55eb30 0x55aa0b560f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.210+0000 7f1d42b7f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55aa0b4b91d0 con 0x55aa0b553b40 2024-04-24T09:09:27.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.210+0000 7f1d42b7f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55aa0b45f450 con 0x55aa0b554460 2024-04-24T09:09:27.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.210+0000 7f1d42b7f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55aa0b4b5190 con 0x55aa0b55eb30 2024-04-24T09:09:27.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.210+0000 7f1d3de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b553f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.210+0000 7f1d3de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b553f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51726/0 (socket says 172.21.15.69:51726) 2024-04-24T09:09:27.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.210+0000 7f1d3de00640 1 -- 172.21.15.69:0/1501661364 learned_addr learned my addr 172.21.15.69:0/1501661364 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:27.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.211+0000 7f1d3d400640 1 --2- 172.21.15.69:0/1501661364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b554460 0x55aa0b55e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.211+0000 7f1d3de00640 1 -- 172.21.15.69:0/1501661364 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b55eb30 msgr2=0x55aa0b560f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:09:27.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.211+0000 7f1d3de00640 1 --2- 172.21.15.69:0/1501661364 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b55eb30 0x55aa0b560f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.211+0000 7f1d3de00640 1 -- 172.21.15.69:0/1501661364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b554460 msgr2=0x55aa0b55e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:27.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.211+0000 7f1d3de00640 1 --2- 172.21.15.69:0/1501661364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b554460 0x55aa0b55e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.211+0000 7f1d3de00640 1 -- 172.21.15.69:0/1501661364 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa0b51c310 con 0x55aa0b553b40 2024-04-24T09:09:27.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.211+0000 7f1d3d400640 1 --2- 172.21.15.69:0/1501661364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b554460 0x55aa0b55e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:27.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.211+0000 7f1d3de00640 1 --2- 172.21.15.69:0/1501661364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b553f20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000ae30 tx=0x7f1d3000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3cc1359e6fe3e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:27.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.211+0000 7f1d36000640 1 --2- 172.21.15.69:0/398495078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1d200a3da0 0x7f1d200a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.212+0000 7f1d2a600640 1 -- 172.21.15.69:0/1501661364 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30003b70 con 0x55aa0b553b40 2024-04-24T09:09:27.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.212+0000 7f1d2a600640 1 -- 172.21.15.69:0/1501661364 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d30003d10 con 0x55aa0b553b40 2024-04-24T09:09:27.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.212+0000 7f1d36000640 1 -- 172.21.15.69:0/398495078 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f1d240d09f0 con 0x7f1d38108f40 2024-04-24T09:09:27.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.212+0000 7f1d2a600640 1 -- 172.21.15.69:0/1501661364 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d300109e0 con 0x55aa0b553b40 2024-04-24T09:09:27.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.212+0000 7f1d3d400640 1 --2- 172.21.15.69:0/398495078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1d200a3da0 0x7f1d200a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.213+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1501661364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 msgr2=0x55aa0b553f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.213+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/1501661364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b553f20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000ae30 tx=0x7f1d3000e720 comp rx=0 tx=0).stop 2024-04-24T09:09:27.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.213+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1501661364 shutdown_connections 2024-04-24T09:09:27.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.213+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/1501661364 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b55eb30 0x55aa0b560f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.213+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/1501661364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b554460 0x55aa0b55e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.213+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/1501661364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b553f20 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.213+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1501661364 >> 172.21.15.69:0/1501661364 conn(0x55aa0b54fe60 msgr2=0x55aa0b550240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:27.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.213+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1501661364 shutdown_connections 2024-04-24T09:09:27.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.213+0000 7f1d3d400640 1 --2- 172.21.15.69:0/398495078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1d200a3da0 0x7f1d200a6260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55aa0b555100 tx=0x7f1d2c032da0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:27.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.213+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1501661364 wait complete. 2024-04-24T09:09:27.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.213+0000 7f1d42b7f740 1 -- start start 2024-04-24T09:09:27.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.214+0000 7f1d42b7f740 1 Processor -- start 2024-04-24T09:09:27.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.214+0000 7f1d42b7f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b553b40 0x55aa0b55e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.214+0000 7f1d42b7f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b554460 0x55aa0b55b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.214+0000 7f1d42b7f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b55eb30 0x55aa0b55be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.214+0000 7f1d42b7f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55aa0b561a10 con 0x55aa0b554460 2024-04-24T09:09:27.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.214+0000 7f1d42b7f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55aa0b472810 con 0x55aa0b553b40 2024-04-24T09:09:27.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.214+0000 7f1d42b7f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55aa0b561b90 con 0x55aa0b55eb30 2024-04-24T09:09:27.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.214+0000 7f1d42b7f740 10 client.0 _ll_register_callbacks cb 0x55aa0b440140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:27.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.215+0000 7f1d3ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b554460 0x55aa0b55b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.215+0000 7f1d3ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b554460 0x55aa0b55b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51736/0 (socket says 172.21.15.69:51736) 2024-04-24T09:09:27.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.215+0000 7f1d3ca00640 1 -- 172.21.15.69:0/2641977058 learned_addr learned my addr 172.21.15.69:0/2641977058 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:27.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.215+0000 7f1d3d400640 1 --2- 172.21.15.69:0/2641977058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b553b40 0x55aa0b55e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.215+0000 7f1d3de00640 1 --2- 172.21.15.69:0/2641977058 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b55eb30 0x55aa0b55be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.215+0000 7f1d3ca00640 1 -- 172.21.15.69:0/2641977058 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b55eb30 msgr2=0x55aa0b55be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:27.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.215+0000 7f1d3ca00640 1 --2- 172.21.15.69:0/2641977058 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b55eb30 0x55aa0b55be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.215+0000 7f1d3ca00640 1 -- 172.21.15.69:0/2641977058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b553b40 msgr2=0x55aa0b55e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:27.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.215+0000 7f1d3ca00640 1 --2- 172.21.15.69:0/2641977058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b553b40 0x55aa0b55e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.215+0000 7f1d3ca00640 1 -- 172.21.15.69:0/2641977058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa0b4833f0 con 0x55aa0b554460 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.215+0000 7f1d3d400640 1 --2- 172.21.15.69:0/2641977058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b553b40 0x55aa0b55e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.215+0000 7f1d3ca00640 1 --2- 172.21.15.69:0/2641977058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b554460 0x55aa0b55b870 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1d2409c580 tx=0x7f1d2409c130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37a8225f2058437 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.215+0000 7f1d1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.216+0000 7f1d1ea00640 1 -- 172.21.15.69:0/2641977058 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d240ce220 con 0x55aa0b554460 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.216+0000 7f1d1ea00640 1 -- 172.21.15.69:0/2641977058 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f1d24022b90 con 0x55aa0b554460 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.216+0000 7f1d42b7f740 1 -- 172.21.15.69:0/2641977058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d30003d10 con 0x55aa0b554460 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.216+0000 7f1d1ea00640 1 -- 172.21.15.69:0/2641977058 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d24009c30 con 0x55aa0b554460 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.216+0000 7f1d42b7f740 10 client.16392 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.216+0000 7f1d42b7f740 1 -- 172.21.15.69:0/2641977058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa0b1cb3f0 con 0x55aa0b554460 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.216+0000 7f1d1ea00640 1 -- 172.21.15.69:0/2641977058 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d30003d10 con 0x55aa0b554460 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.217+0000 7f1d1ea00640 1 -- 172.21.15.69:0/2641977058 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7f1d240bb380 con 0x55aa0b554460 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.217+0000 7f1d42b7f740 10 client.16392 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.217+0000 7f1d42b7f740 10 client.16392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.217+0000 7f1d42b7f740 20 client.16392 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.217+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/2641977058 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55aa0b56ed80 0x55aa0b5711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.217+0000 7f1d42b7f740 4 client.16392 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:27.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.217+0000 7f1d42b7f740 1 -- 172.21.15.69:0/2641977058 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa0b4c6bf0 con 0x55aa0b56ed80 2024-04-24T09:09:27.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.217+0000 7f1d3d400640 1 --2- 172.21.15.69:0/2641977058 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55aa0b56ed80 0x55aa0b5711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.219+0000 7f1d3d400640 1 --2- 172.21.15.69:0/2641977058 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55aa0b56ed80 0x55aa0b5711a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=995767598d291534 server_cookie=2840c148c5a0a50b in_seq=0 out_seq=0 2024-04-24T09:09:27.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.219+0000 7f1d1ea00640 10 client.16392 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:27.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.221+0000 7f1d1ea00640 1 -- 172.21.15.69:0/2641977058 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa0b4c6bf0 con 0x55aa0b56ed80 2024-04-24T09:09:27.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.221+0000 7f1d1ea00640 10 client.16392 handle_command_reply: tid=0 2024-04-24T09:09:27.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.221+0000 7f1d42b7f740 1 client.16392 shutdown 2024-04-24T09:09:27.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.222+0000 7f1d42b7f740 1 -- 172.21.15.69:0/2641977058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b554460 msgr2=0x55aa0b55b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.222+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/2641977058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b554460 0x55aa0b55b870 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1d2409c580 tx=0x7f1d2409c130 comp rx=0 tx=0).stop 2024-04-24T09:09:27.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.222+0000 7f1d42b7f740 1 -- 172.21.15.69:0/2641977058 shutdown_connections 2024-04-24T09:09:27.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.222+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/2641977058 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55aa0b56ed80 0x55aa0b5711a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.222+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/2641977058 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b55eb30 0x55aa0b55be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.222+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/2641977058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b554460 0x55aa0b55b870 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.222+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/2641977058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b553b40 0x55aa0b55e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.222+0000 7f1d42b7f740 1 -- 172.21.15.69:0/2641977058 >> 172.21.15.69:0/2641977058 conn(0x55aa0b54fe60 msgr2=0x55aa0b555b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:27.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.222+0000 7f1d1ea00640 0 client.16392 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:27.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.222+0000 7f1d42b7f740 1 -- 172.21.15.69:0/2641977058 shutdown_connections 2024-04-24T09:09:27.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.222+0000 7f1d42b7f740 1 -- 172.21.15.69:0/2641977058 wait complete. 2024-04-24T09:09:27.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.222+0000 7f1d42b7f740 20 client.16392 trim_cache size 0 max 16384 2024-04-24T09:09:27.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.240+0000 7f1d42b7f740 1 Processor -- start 2024-04-24T09:09:27.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.240+0000 7f1d42b7f740 1 -- start start 2024-04-24T09:09:27.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.241+0000 7f1d42b7f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b56a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.241+0000 7f1d42b7f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b554460 0x55aa0b570d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.241+0000 7f1d42b7f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b55eb30 0x55aa0b576a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.241+0000 7f1d42b7f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1d381b7f80 con 0x55aa0b553b40 2024-04-24T09:09:27.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.241+0000 7f1d42b7f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1d381b5820 con 0x55aa0b554460 2024-04-24T09:09:27.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.241+0000 7f1d42b7f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1d381b3670 con 0x55aa0b55eb30 2024-04-24T09:09:27.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.241+0000 7f1d3d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b56a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.241+0000 7f1d3d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b56a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51744/0 (socket says 172.21.15.69:51744) 2024-04-24T09:09:27.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.241+0000 7f1d3d400640 1 -- 172.21.15.69:0/1389417467 learned_addr learned my addr 172.21.15.69:0/1389417467 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:27.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.241+0000 7f1d3ca00640 1 --2- 172.21.15.69:0/1389417467 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b554460 0x55aa0b570d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.241+0000 7f1d3de00640 1 --2- 172.21.15.69:0/1389417467 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b55eb30 0x55aa0b576a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.242+0000 7f1d3d400640 1 -- 172.21.15.69:0/1389417467 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b55eb30 msgr2=0x55aa0b576a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:27.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.242+0000 7f1d3d400640 1 --2- 172.21.15.69:0/1389417467 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b55eb30 0x55aa0b576a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.242+0000 7f1d3d400640 1 -- 172.21.15.69:0/1389417467 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b554460 msgr2=0x55aa0b570d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:27.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.242+0000 7f1d3d400640 1 --2- 172.21.15.69:0/1389417467 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b554460 0x55aa0b570d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.242+0000 7f1d3d400640 1 -- 172.21.15.69:0/1389417467 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa0b2957d0 con 0x55aa0b553b40 2024-04-24T09:09:27.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.242+0000 7f1d3d400640 1 --2- 172.21.15.69:0/1389417467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b56a900 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c031440 tx=0x7f1d2c03b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a4ef0e1402a071 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:27.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.242+0000 7f1d29c00640 1 -- 172.21.15.69:0/1389417467 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c051070 con 0x55aa0b553b40 2024-04-24T09:09:27.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.242+0000 7f1d29c00640 1 -- 172.21.15.69:0/1389417467 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d2c041040 con 0x55aa0b553b40 2024-04-24T09:09:27.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.243+0000 7f1d29c00640 1 -- 172.21.15.69:0/1389417467 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c035050 con 0x55aa0b553b40 2024-04-24T09:09:27.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.243+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1389417467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 msgr2=0x55aa0b56a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.243+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/1389417467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b56a900 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c031440 tx=0x7f1d2c03b000 comp rx=0 tx=0).stop 2024-04-24T09:09:27.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.243+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1389417467 shutdown_connections 2024-04-24T09:09:27.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.243+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/1389417467 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b55eb30 0x55aa0b576a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.243+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/1389417467 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b554460 0x55aa0b570d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.243+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/1389417467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b56a900 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.243+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1389417467 >> 172.21.15.69:0/1389417467 conn(0x55aa0b54fe60 msgr2=0x55aa0b556b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:27.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.243+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1389417467 shutdown_connections 2024-04-24T09:09:27.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.244+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1389417467 wait complete. 2024-04-24T09:09:27.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.244+0000 7f1d42b7f740 1 -- start start 2024-04-24T09:09:27.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.244+0000 7f1d42b7f740 1 Processor -- start 2024-04-24T09:09:27.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.245+0000 7f1d42b7f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b5672a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.245+0000 7f1d42b7f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b554460 0x55aa0b558e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.245+0000 7f1d42b7f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b55eb30 0x55aa0b559470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.245+0000 7f1d42b7f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1d381b3ce0 con 0x55aa0b553b40 2024-04-24T09:09:27.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.245+0000 7f1d42b7f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1d381b5150 con 0x55aa0b55eb30 2024-04-24T09:09:27.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.245+0000 7f1d42b7f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1d381b43b0 con 0x55aa0b554460 2024-04-24T09:09:27.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.245+0000 7f1d42b7f740 10 client.0 _ll_register_callbacks cb 0x7f1d24009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:27.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.245+0000 7f1d3d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b5672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.245+0000 7f1d3d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b5672a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51748/0 (socket says 172.21.15.69:51748) 2024-04-24T09:09:27.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.245+0000 7f1d3d400640 1 -- 172.21.15.69:0/1104440604 learned_addr learned my addr 172.21.15.69:0/1104440604 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:27.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.245+0000 7f1d3de00640 1 --2- 172.21.15.69:0/1104440604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b55eb30 0x55aa0b559470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.246+0000 7f1d3ca00640 1 --2- 172.21.15.69:0/1104440604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b554460 0x55aa0b558e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.247+0000 7f1d3de00640 1 -- 172.21.15.69:0/1104440604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b554460 msgr2=0x55aa0b558e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:27.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.247+0000 7f1d3de00640 1 --2- 172.21.15.69:0/1104440604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b554460 0x55aa0b558e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.247+0000 7f1d3de00640 1 -- 172.21.15.69:0/1104440604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 msgr2=0x55aa0b5672a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:27.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.247+0000 7f1d3de00640 1 --2- 172.21.15.69:0/1104440604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b5672a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.247+0000 7f1d3de00640 1 -- 172.21.15.69:0/1104440604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa0b55fba0 con 0x55aa0b55eb30 2024-04-24T09:09:27.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.247+0000 7f1d3d400640 1 --2- 172.21.15.69:0/1104440604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b5672a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:27.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.247+0000 7f1d3de00640 1 --2- 172.21.15.69:0/1104440604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b55eb30 0x55aa0b559470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000ec20 tx=0x7f1d3000a7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8d1e97c772b91f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:27.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.247+0000 7f1d1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-04-24T09:09:27.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.248+0000 7f1d1ea00640 1 -- 172.21.15.69:0/1104440604 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3002b060 con 0x55aa0b55eb30 2024-04-24T09:09:27.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.248+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1104440604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d2c041040 con 0x55aa0b55eb30 2024-04-24T09:09:27.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.248+0000 7f1d1ea00640 1 -- 172.21.15.69:0/1104440604 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f1d30010a50 con 0x55aa0b55eb30 2024-04-24T09:09:27.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.248+0000 7f1d1ea00640 1 -- 172.21.15.69:0/1104440604 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d30032050 con 0x55aa0b55eb30 2024-04-24T09:09:27.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.248+0000 7f1d42b7f740 10 client.35618 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:27.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.248+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1104440604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa0b5e6780 con 0x55aa0b55eb30 2024-04-24T09:09:27.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.249+0000 7f1d1ea00640 1 -- 172.21.15.69:0/1104440604 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d2c041040 con 0x55aa0b55eb30 2024-04-24T09:09:27.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.249+0000 7f1d1ea00640 1 -- 172.21.15.69:0/1104440604 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7f1d3001a460 con 0x55aa0b55eb30 2024-04-24T09:09:27.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.249+0000 7f1d42b7f740 10 client.35618 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:27.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.249+0000 7f1d42b7f740 10 client.35618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:27.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.249+0000 7f1d42b7f740 20 client.35618 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:27.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.250+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/1104440604 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55aa0b5e6c70 0x55aa0b5e9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.250+0000 7f1d42b7f740 4 client.35618 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:09:27.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.250+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1104440604 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa0b4c0a60 con 0x55aa0b5e6c70 2024-04-24T09:09:27.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.250+0000 7f1d3d400640 1 --2- 172.21.15.69:0/1104440604 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55aa0b5e6c70 0x55aa0b5e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.251+0000 7f1d3d400640 1 --2- 172.21.15.69:0/1104440604 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55aa0b5e6c70 0x55aa0b5e9090 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc253e14f18b7f40 server_cookie=3ab60d32a161a944 in_seq=0 out_seq=0 2024-04-24T09:09:27.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.251+0000 7f1d1ea00640 10 client.35618 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:27.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.252+0000 7f1d1ea00640 1 -- 172.21.15.69:0/1104440604 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa0b4c0a60 con 0x55aa0b5e6c70 2024-04-24T09:09:27.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.252+0000 7f1d1ea00640 10 client.35618 handle_command_reply: tid=0 2024-04-24T09:09:27.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.252+0000 7f1d42b7f740 1 client.35618 shutdown 2024-04-24T09:09:27.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.252+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1104440604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b55eb30 msgr2=0x55aa0b559470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.252+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/1104440604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b55eb30 0x55aa0b559470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000ec20 tx=0x7f1d3000a7d0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.252+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1104440604 shutdown_connections 2024-04-24T09:09:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.252+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/1104440604 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55aa0b5e6c70 0x55aa0b5e9090 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.252+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/1104440604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa0b55eb30 0x55aa0b559470 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.252+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/1104440604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aa0b554460 0x55aa0b558e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.252+0000 7f1d1ea00640 0 client.35618 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.252+0000 7f1d42b7f740 1 --2- 172.21.15.69:0/1104440604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa0b553b40 0x55aa0b5672a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.252+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1104440604 >> 172.21.15.69:0/1104440604 conn(0x55aa0b54fe60 msgr2=0x55aa0b55a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.252+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1104440604 shutdown_connections 2024-04-24T09:09:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.252+0000 7f1d42b7f740 1 -- 172.21.15.69:0/1104440604 wait complete. 2024-04-24T09:09:27.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.253+0000 7f1d42b7f740 20 client.35618 trim_cache size 0 max 16384 2024-04-24T09:09:27.253 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-04-24T09:09:27.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.253+0000 7f1d3e800640 1 -- 172.21.15.69:0/398495078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1d200a3da0 msgr2=0x7f1d200a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.253+0000 7f1d3e800640 1 --2- 172.21.15.69:0/398495078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1d200a3da0 0x7f1d200a6260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55aa0b555100 tx=0x7f1d2c032da0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.253+0000 7f1d3e800640 1 -- 172.21.15.69:0/398495078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d38108f40 msgr2=0x7f1d381a4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.253+0000 7f1d3e800640 1 --2- 172.21.15.69:0/398495078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d38108f40 0x7f1d381a4e40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1d2400d770 tx=0x7f1d2400dc30 comp rx=0 tx=0).stop 2024-04-24T09:09:27.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.253+0000 7f1d3e800640 1 -- 172.21.15.69:0/398495078 shutdown_connections 2024-04-24T09:09:27.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.253+0000 7f1d3e800640 1 --2- 172.21.15.69:0/398495078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f1d200a3da0 0x7f1d200a6260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.253+0000 7f1d3e800640 1 --2- 172.21.15.69:0/398495078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d3810d500 0x7f1d3819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.253+0000 7f1d3e800640 1 --2- 172.21.15.69:0/398495078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d38108f40 0x7f1d381a4e40 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.253+0000 7f1d3e800640 1 --2- 172.21.15.69:0/398495078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d38108580 0x7f1d381a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.253+0000 7f1d3e800640 1 -- 172.21.15.69:0/398495078 >> 172.21.15.69:0/398495078 conn(0x7f1d38077e20 msgr2=0x7f1d38078400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:27.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.254+0000 7f1d3e800640 1 -- 172.21.15.69:0/398495078 shutdown_connections 2024-04-24T09:09:27.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:27.254+0000 7f1d3e800640 1 -- 172.21.15.69:0/398495078 wait complete. 2024-04-24T09:09:27.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:27 smithi081 ceph-mon[30961]: pgmap v74: 129 pgs: 129 active+clean; 30 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 704 KiB/s wr, 142 op/s 2024-04-24T09:09:27.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:27 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/889318824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:09:27.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:27 smithi069 ceph-mon[28170]: pgmap v74: 129 pgs: 129 active+clean; 30 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 704 KiB/s wr, 142 op/s 2024-04-24T09:09:27.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:27 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/889318824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:09:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.608+0000 7f373a800640 1 -- 172.21.15.69:0/3330066723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3734102c30 msgr2=0x7f3734106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.608+0000 7f373a800640 1 --2- 172.21.15.69:0/3330066723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3734102c30 0x7f3734106c70 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f371c009900 tx=0x7f371c02a410 comp rx=0 tx=0).stop 2024-04-24T09:09:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.608+0000 7f373a800640 1 -- 172.21.15.69:0/3330066723 shutdown_connections 2024-04-24T09:09:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.608+0000 7f373a800640 1 --2- 172.21.15.69:0/3330066723 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3734107350 0x7f373410ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.608+0000 7f373a800640 1 --2- 172.21.15.69:0/3330066723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3734102c30 0x7f3734106c70 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.608+0000 7f373a800640 1 --2- 172.21.15.69:0/3330066723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3734102280 0x7f3734102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.608+0000 7f373a800640 1 -- 172.21.15.69:0/3330066723 >> 172.21.15.69:0/3330066723 conn(0x7f3734079490 msgr2=0x7f37340798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.608+0000 7f373a800640 1 -- 172.21.15.69:0/3330066723 shutdown_connections 2024-04-24T09:09:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.608+0000 7f373a800640 1 -- 172.21.15.69:0/3330066723 wait complete. 2024-04-24T09:09:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.609+0000 7f373a800640 1 Processor -- start 2024-04-24T09:09:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.609+0000 7f373a800640 1 -- start start 2024-04-24T09:09:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.610+0000 7f373a800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3734102280 0x7f37341a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.610+0000 7f373a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3734107350 0x7f37341a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.610+0000 7f373a800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f373419a7b0 0x7f373419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.610+0000 7f373a800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3734112270 con 0x7f3734107350 2024-04-24T09:09:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.610+0000 7f373a800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f37341120f0 con 0x7f373419a7b0 2024-04-24T09:09:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.610+0000 7f373a800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f37341123f0 con 0x7f3734102280 2024-04-24T09:09:27.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.610+0000 7f372aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3734107350 0x7f37341a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.610+0000 7f372aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3734107350 0x7f37341a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51764/0 (socket says 172.21.15.69:51764) 2024-04-24T09:09:27.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.610+0000 7f372aa00640 1 -- 172.21.15.69:0/4024676879 learned_addr learned my addr 172.21.15.69:0/4024676879 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.611+0000 7f3733400640 1 --2- 172.21.15.69:0/4024676879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3734102280 0x7f37341a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.611+0000 7f3733e00640 1 --2- 172.21.15.69:0/4024676879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f373419a7b0 0x7f373419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.611+0000 7f372aa00640 1 -- 172.21.15.69:0/4024676879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3734102280 msgr2=0x7f37341a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.611+0000 7f372aa00640 1 --2- 172.21.15.69:0/4024676879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3734102280 0x7f37341a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.611+0000 7f372aa00640 1 -- 172.21.15.69:0/4024676879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f373419a7b0 msgr2=0x7f373419ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.611+0000 7f372aa00640 1 --2- 172.21.15.69:0/4024676879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f373419a7b0 0x7f373419ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.611+0000 7f372aa00640 1 -- 172.21.15.69:0/4024676879 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373419b340 con 0x7f3734107350 2024-04-24T09:09:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.611+0000 7f372aa00640 1 --2- 172.21.15.69:0/4024676879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3734107350 0x7f37341a0b80 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f371c004270 tx=0x7f371c0042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:27.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.611+0000 7f3730c00640 1 -- 172.21.15.69:0/4024676879 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c03f030 con 0x7f3734107350 2024-04-24T09:09:27.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.611+0000 7f3730c00640 1 -- 172.21.15.69:0/4024676879 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f371c0044c0 con 0x7f3734107350 2024-04-24T09:09:27.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.611+0000 7f373a800640 1 -- 172.21.15.69:0/4024676879 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373419b5d0 con 0x7f3734107350 2024-04-24T09:09:27.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.611+0000 7f3730c00640 1 -- 172.21.15.69:0/4024676879 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c035070 con 0x7f3734107350 2024-04-24T09:09:27.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.612+0000 7f373a800640 1 -- 172.21.15.69:0/4024676879 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37341a7780 con 0x7f3734107350 2024-04-24T09:09:27.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.616+0000 7f373d76f740 1 Processor -- start 2024-04-24T09:09:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.616+0000 7f373d76f740 1 -- start start 2024-04-24T09:09:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.616+0000 7f373d76f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a621b0 0x55d693a62590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f3730c00640 1 -- 172.21.15.69:0/4024676879 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f371c042030 con 0x7f3734107350 2024-04-24T09:09:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f373d76f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a62ad0 0x55d693a6ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f373d76f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a6d390 0x55d693a6f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f373d76f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d6939c8900 con 0x55d693a62ad0 2024-04-24T09:09:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f373d76f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55d69396ef20 con 0x55d693a621b0 2024-04-24T09:09:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f373d76f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d6939c48c0 con 0x55d693a6d390 2024-04-24T09:09:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f3733400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a62ad0 0x55d693a6ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f3733400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a62ad0 0x55d693a6ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51770/0 (socket says 172.21.15.69:51770) 2024-04-24T09:09:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f3733400640 1 -- 172.21.15.69:0/3549007907 learned_addr learned my addr 172.21.15.69:0/3549007907 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f3733e00640 1 --2- 172.21.15.69:0/3549007907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a621b0 0x55d693a62590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f372aa00640 1 --2- 172.21.15.69:0/3549007907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a6d390 0x55d693a6f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f3733400640 1 -- 172.21.15.69:0/3549007907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a6d390 msgr2=0x55d693a6f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f3733400640 1 --2- 172.21.15.69:0/3549007907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a6d390 0x55d693a6f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f3733400640 1 -- 172.21.15.69:0/3549007907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a621b0 msgr2=0x55d693a62590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f3733400640 1 --2- 172.21.15.69:0/3549007907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a621b0 0x55d693a62590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f3733400640 1 -- 172.21.15.69:0/3549007907 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d693993e10 con 0x55d693a62ad0 2024-04-24T09:09:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.617+0000 7f3733e00640 1 --2- 172.21.15.69:0/3549007907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a621b0 0x55d693a62590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.618+0000 7f3733400640 1 --2- 172.21.15.69:0/3549007907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a62ad0 0x55d693a6ce50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f372400ae90 tx=0x7f372400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e92515d95efb4b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.618+0000 7f3730c00640 1 --2- 172.21.15.69:0/4024676879 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37180a3dc0 0x7f37180a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.618+0000 7f3728c00640 1 -- 172.21.15.69:0/3549007907 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724011850 con 0x55d693a62ad0 2024-04-24T09:09:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.618+0000 7f3728c00640 1 -- 172.21.15.69:0/3549007907 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f372401a480 con 0x55d693a62ad0 2024-04-24T09:09:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.618+0000 7f3728c00640 1 -- 172.21.15.69:0/3549007907 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724010610 con 0x55d693a62ad0 2024-04-24T09:09:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.618+0000 7f3730c00640 1 -- 172.21.15.69:0/4024676879 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f371c0f11d0 con 0x7f3734107350 2024-04-24T09:09:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.618+0000 7f3733e00640 1 --2- 172.21.15.69:0/4024676879 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37180a3dc0 0x7f37180a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.619+0000 7f373d76f740 1 -- 172.21.15.69:0/3549007907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a62ad0 msgr2=0x55d693a6ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.619+0000 7f373d76f740 1 --2- 172.21.15.69:0/3549007907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a62ad0 0x55d693a6ce50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f372400ae90 tx=0x7f372400f730 comp rx=0 tx=0).stop 2024-04-24T09:09:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.619+0000 7f3733e00640 1 --2- 172.21.15.69:0/4024676879 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37180a3dc0 0x7f37180a6280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f372c005fd0 tx=0x7f372c005f60 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.619+0000 7f373d76f740 1 -- 172.21.15.69:0/3549007907 shutdown_connections 2024-04-24T09:09:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.619+0000 7f373d76f740 1 --2- 172.21.15.69:0/3549007907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a6d390 0x55d693a6f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.619+0000 7f373d76f740 1 --2- 172.21.15.69:0/3549007907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a62ad0 0x55d693a6ce50 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.619+0000 7f373d76f740 1 --2- 172.21.15.69:0/3549007907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a621b0 0x55d693a62590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.619+0000 7f373d76f740 1 -- 172.21.15.69:0/3549007907 >> 172.21.15.69:0/3549007907 conn(0x55d693a5e6c0 msgr2=0x55d693a5eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.619+0000 7f373d76f740 1 -- 172.21.15.69:0/3549007907 shutdown_connections 2024-04-24T09:09:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.619+0000 7f373d76f740 1 -- 172.21.15.69:0/3549007907 wait complete. 2024-04-24T09:09:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.620+0000 7f373d76f740 1 -- start start 2024-04-24T09:09:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.620+0000 7f373d76f740 1 Processor -- start 2024-04-24T09:09:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f373d76f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a621b0 0x55d693a6a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f373d76f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 0x55d693a6a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f373d76f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a6d390 0x55d693a6ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f373d76f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d693a6ffc0 con 0x55d693a6d390 2024-04-24T09:09:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f373d76f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55d6939822c0 con 0x55d693a62ad0 2024-04-24T09:09:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f373d76f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d693a70140 con 0x55d693a621b0 2024-04-24T09:09:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f373d76f740 10 client.0 _ll_register_callbacks cb 0x55d6936daa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f3733e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a6d390 0x55d693a6ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f3733e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a6d390 0x55d693a6ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51780/0 (socket says 172.21.15.69:51780) 2024-04-24T09:09:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f3733e00640 1 -- 172.21.15.69:0/3811009434 learned_addr learned my addr 172.21.15.69:0/3811009434 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f372aa00640 1 --2- 172.21.15.69:0/3811009434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 0x55d693a6a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f3733400640 1 --2- 172.21.15.69:0/3811009434 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a621b0 0x55d693a6a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f3733e00640 1 -- 172.21.15.69:0/3811009434 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a621b0 msgr2=0x55d693a6a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f3733e00640 1 --2- 172.21.15.69:0/3811009434 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a621b0 0x55d693a6a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f3733e00640 1 -- 172.21.15.69:0/3811009434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 msgr2=0x55d693a6a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f3733e00640 1 --2- 172.21.15.69:0/3811009434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 0x55d693a6a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.621+0000 7f3733e00640 1 -- 172.21.15.69:0/3811009434 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d693992fd0 con 0x55d693a6d390 2024-04-24T09:09:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.622+0000 7f3733e00640 1 --2- 172.21.15.69:0/3811009434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a6d390 0x55d693a6ad90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f372c011b30 tx=0x7f372c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfa8d5760c5e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.622+0000 7f3721600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.622+0000 7f3721600640 1 -- 172.21.15.69:0/3811009434 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c00ee10 con 0x55d693a6d390 2024-04-24T09:09:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.622+0000 7f373d76f740 1 -- 172.21.15.69:0/3811009434 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f372401a480 con 0x55d693a6d390 2024-04-24T09:09:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.622+0000 7f3721600640 1 -- 172.21.15.69:0/3811009434 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f372c01e420 con 0x55d693a6d390 2024-04-24T09:09:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.622+0000 7f3721600640 1 -- 172.21.15.69:0/3811009434 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f372c02f050 con 0x55d693a6d390 2024-04-24T09:09:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.622+0000 7f373d76f740 10 client.16410 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.622+0000 7f373d76f740 1 -- 172.21.15.69:0/3811009434 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d6936db210 con 0x55d693a6d390 2024-04-24T09:09:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.623+0000 7f3721600640 1 -- 172.21.15.69:0/3811009434 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f372401a480 con 0x55d693a6d390 2024-04-24T09:09:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.623+0000 7f3721600640 1 -- 172.21.15.69:0/3811009434 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7f372c026730 con 0x55d693a6d390 2024-04-24T09:09:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.623+0000 7f373d76f740 10 client.16410 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.623+0000 7f373d76f740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.623+0000 7f373d76f740 20 client.16410 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.623+0000 7f373d76f740 1 --2- 172.21.15.69:0/3811009434 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55d693a7d560 0x55d693a7f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.623+0000 7f373d76f740 4 client.16410 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.623+0000 7f373d76f740 1 -- 172.21.15.69:0/3811009434 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6939d6320 con 0x55d693a7d560 2024-04-24T09:09:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.624+0000 7f3733400640 1 --2- 172.21.15.69:0/3811009434 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55d693a7d560 0x55d693a7f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.625+0000 7f3721600640 10 client.16410 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.625+0000 7f3733400640 1 --2- 172.21.15.69:0/3811009434 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55d693a7d560 0x55d693a7f990 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f562040a16904c6 server_cookie=71b8ea86ff789ceb in_seq=0 out_seq=0 2024-04-24T09:09:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.627+0000 7f3721600640 1 -- 172.21.15.69:0/3811009434 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d6939d6320 con 0x55d693a7d560 2024-04-24T09:09:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.627+0000 7f3721600640 10 client.16410 handle_command_reply: tid=0 2024-04-24T09:09:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.627+0000 7f373d76f740 1 client.16410 shutdown 2024-04-24T09:09:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.628+0000 7f373d76f740 1 -- 172.21.15.69:0/3811009434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a6d390 msgr2=0x55d693a6ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.628+0000 7f373d76f740 1 --2- 172.21.15.69:0/3811009434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a6d390 0x55d693a6ad90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f372c011b30 tx=0x7f372c0046c0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.628+0000 7f373d76f740 1 -- 172.21.15.69:0/3811009434 shutdown_connections 2024-04-24T09:09:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.628+0000 7f373d76f740 1 --2- 172.21.15.69:0/3811009434 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55d693a7d560 0x55d693a7f990 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.628+0000 7f373d76f740 1 --2- 172.21.15.69:0/3811009434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a6d390 0x55d693a6ad90 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.628+0000 7f373d76f740 1 --2- 172.21.15.69:0/3811009434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 0x55d693a6a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.628+0000 7f3721600640 0 client.16410 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.628+0000 7f373d76f740 1 --2- 172.21.15.69:0/3811009434 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a621b0 0x55d693a6a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.628+0000 7f373d76f740 1 -- 172.21.15.69:0/3811009434 >> 172.21.15.69:0/3811009434 conn(0x55d693a5e6c0 msgr2=0x55d693a64440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.628+0000 7f373d76f740 1 -- 172.21.15.69:0/3811009434 shutdown_connections 2024-04-24T09:09:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.628+0000 7f373d76f740 1 -- 172.21.15.69:0/3811009434 wait complete. 2024-04-24T09:09:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.628+0000 7f373d76f740 20 client.16410 trim_cache size 0 max 16384 2024-04-24T09:09:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.644+0000 7f373d76f740 1 Processor -- start 2024-04-24T09:09:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.644+0000 7f373d76f740 1 -- start start 2024-04-24T09:09:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.645+0000 7f373d76f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a621b0 0x55d693a78e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.645+0000 7f373d76f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 0x55d693a64b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.645+0000 7f373d76f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a6d390 0x55d693a6a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.645+0000 7f373d76f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37341b3cd0 con 0x55d693a6d390 2024-04-24T09:09:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.645+0000 7f373d76f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f37341b1570 con 0x55d693a62ad0 2024-04-24T09:09:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.645+0000 7f373d76f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f37341af3c0 con 0x55d693a621b0 2024-04-24T09:09:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.645+0000 7f3733e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a6d390 0x55d693a6a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.645+0000 7f3733e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a6d390 0x55d693a6a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51792/0 (socket says 172.21.15.69:51792) 2024-04-24T09:09:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.645+0000 7f3733e00640 1 -- 172.21.15.69:0/2001028249 learned_addr learned my addr 172.21.15.69:0/2001028249 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.645+0000 7f372aa00640 1 --2- 172.21.15.69:0/2001028249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 0x55d693a64b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.645+0000 7f3733400640 1 --2- 172.21.15.69:0/2001028249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a621b0 0x55d693a78e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.646+0000 7f3733e00640 1 -- 172.21.15.69:0/2001028249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a621b0 msgr2=0x55d693a78e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.646+0000 7f3733e00640 1 --2- 172.21.15.69:0/2001028249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a621b0 0x55d693a78e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.646+0000 7f3733e00640 1 -- 172.21.15.69:0/2001028249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 msgr2=0x55d693a64b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.646+0000 7f3733e00640 1 --2- 172.21.15.69:0/2001028249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 0x55d693a64b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.646+0000 7f3733e00640 1 -- 172.21.15.69:0/2001028249 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d693a62900 con 0x55d693a6d390 2024-04-24T09:09:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.646+0000 7f372aa00640 1 --2- 172.21.15.69:0/2001028249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 0x55d693a64b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.646+0000 7f3733e00640 1 --2- 172.21.15.69:0/2001028249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a6d390 0x55d693a6a9e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f372c0114c0 tx=0x7f372c010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8fce4bf825442a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.646+0000 7f3723e00640 1 -- 172.21.15.69:0/2001028249 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c00ee10 con 0x55d693a6d390 2024-04-24T09:09:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.646+0000 7f3723e00640 1 -- 172.21.15.69:0/2001028249 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f372c00e790 con 0x55d693a6d390 2024-04-24T09:09:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.646+0000 7f3723e00640 1 -- 172.21.15.69:0/2001028249 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c01cce0 con 0x55d693a6d390 2024-04-24T09:09:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.647+0000 7f373d76f740 1 -- 172.21.15.69:0/2001028249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a6d390 msgr2=0x55d693a6a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.647+0000 7f373d76f740 1 --2- 172.21.15.69:0/2001028249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a6d390 0x55d693a6a9e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f372c0114c0 tx=0x7f372c010e00 comp rx=0 tx=0).stop 2024-04-24T09:09:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.647+0000 7f373d76f740 1 -- 172.21.15.69:0/2001028249 shutdown_connections 2024-04-24T09:09:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.647+0000 7f373d76f740 1 --2- 172.21.15.69:0/2001028249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a6d390 0x55d693a6a9e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.647+0000 7f373d76f740 1 --2- 172.21.15.69:0/2001028249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 0x55d693a64b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.647+0000 7f373d76f740 1 --2- 172.21.15.69:0/2001028249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a621b0 0x55d693a78e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.647+0000 7f373d76f740 1 -- 172.21.15.69:0/2001028249 >> 172.21.15.69:0/2001028249 conn(0x55d693a5e6c0 msgr2=0x55d693a6a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.647+0000 7f373d76f740 1 -- 172.21.15.69:0/2001028249 shutdown_connections 2024-04-24T09:09:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.647+0000 7f373d76f740 1 -- 172.21.15.69:0/2001028249 wait complete. 2024-04-24T09:09:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.648+0000 7f373d76f740 1 -- start start 2024-04-24T09:09:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.648+0000 7f373d76f740 1 Processor -- start 2024-04-24T09:09:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.648+0000 7f373d76f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a621b0 0x55d693a756e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.648+0000 7f373d76f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 0x55d693a70a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.648+0000 7f373d76f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a6d390 0x55d693a70f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.648+0000 7f373d76f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37341afa30 con 0x55d693a621b0 2024-04-24T09:09:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.648+0000 7f373d76f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f37341b0ea0 con 0x55d693a62ad0 2024-04-24T09:09:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.648+0000 7f373d76f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f37341b0100 con 0x55d693a6d390 2024-04-24T09:09:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.648+0000 7f373d76f740 10 client.0 _ll_register_callbacks cb 0x7f373406b880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.649+0000 7f3733400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a621b0 0x55d693a756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.649+0000 7f3733400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a621b0 0x55d693a756e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51798/0 (socket says 172.21.15.69:51798) 2024-04-24T09:09:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.649+0000 7f3733400640 1 -- 172.21.15.69:0/3163089530 learned_addr learned my addr 172.21.15.69:0/3163089530 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.649+0000 7f372aa00640 1 --2- 172.21.15.69:0/3163089530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 0x55d693a70a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.649+0000 7f3733e00640 1 --2- 172.21.15.69:0/3163089530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a6d390 0x55d693a70f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.649+0000 7f3733400640 1 -- 172.21.15.69:0/3163089530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a6d390 msgr2=0x55d693a70f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.649+0000 7f3733400640 1 --2- 172.21.15.69:0/3163089530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a6d390 0x55d693a70f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.649+0000 7f3733400640 1 -- 172.21.15.69:0/3163089530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 msgr2=0x55d693a70a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.649+0000 7f3733400640 1 --2- 172.21.15.69:0/3163089530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 0x55d693a70a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.649+0000 7f3733400640 1 -- 172.21.15.69:0/3163089530 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6937a7620 con 0x55d693a621b0 2024-04-24T09:09:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.649+0000 7f372aa00640 1 --2- 172.21.15.69:0/3163089530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 0x55d693a70a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.649+0000 7f3733e00640 1 --2- 172.21.15.69:0/3163089530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a6d390 0x55d693a70f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.649+0000 7f3733400640 1 --2- 172.21.15.69:0/3163089530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a621b0 0x55d693a756e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f372400ae90 tx=0x7f372400ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f541e9ed1fc5b337 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.649+0000 7f3721600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.649+0000 7f3721600640 1 -- 172.21.15.69:0/3163089530 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372402a070 con 0x55d693a621b0 2024-04-24T09:09:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.650+0000 7f3721600640 1 -- 172.21.15.69:0/3163089530 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f3724004be0 con 0x55d693a621b0 2024-04-24T09:09:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.650+0000 7f373d76f740 1 -- 172.21.15.69:0/3163089530 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f372c00e790 con 0x55d693a621b0 2024-04-24T09:09:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.650+0000 7f3721600640 1 -- 172.21.15.69:0/3163089530 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3724035050 con 0x55d693a621b0 2024-04-24T09:09:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.650+0000 7f373d76f740 10 client.16416 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.650+0000 7f373d76f740 1 -- 172.21.15.69:0/3163089530 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d693ae3920 con 0x55d693a621b0 2024-04-24T09:09:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.650+0000 7f3721600640 1 -- 172.21.15.69:0/3163089530 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f372c00e790 con 0x55d693a621b0 2024-04-24T09:09:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.650+0000 7f3721600640 1 -- 172.21.15.69:0/3163089530 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7f3724011650 con 0x55d693a621b0 2024-04-24T09:09:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.650+0000 7f373d76f740 10 client.16416 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.650+0000 7f373d76f740 10 client.16416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.650+0000 7f373d76f740 20 client.16416 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.650+0000 7f373d76f740 1 --2- 172.21.15.69:0/3163089530 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55d693ae3e10 0x55d693ae6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.650+0000 7f373d76f740 4 client.16416 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:09:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.650+0000 7f373d76f740 1 -- 172.21.15.69:0/3163089530 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d6939d0190 con 0x55d693ae3e10 2024-04-24T09:09:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.651+0000 7f372aa00640 1 --2- 172.21.15.69:0/3163089530 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55d693ae3e10 0x55d693ae6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.652+0000 7f372aa00640 1 --2- 172.21.15.69:0/3163089530 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55d693ae3e10 0x55d693ae6230 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b71d3474604b9032 server_cookie=f18b905e3d3be0b8 in_seq=0 out_seq=0 2024-04-24T09:09:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.652+0000 7f3721600640 10 client.16416 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.680+0000 7f3721600640 1 -- 172.21.15.69:0/3163089530 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d6939d0190 con 0x55d693ae3e10 2024-04-24T09:09:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.680+0000 7f3721600640 10 client.16416 handle_command_reply: tid=0 2024-04-24T09:09:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.680+0000 7f373d76f740 1 client.16416 shutdown 2024-04-24T09:09:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.680+0000 7f373d76f740 1 -- 172.21.15.69:0/3163089530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a621b0 msgr2=0x55d693a756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.680+0000 7f373d76f740 1 --2- 172.21.15.69:0/3163089530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a621b0 0x55d693a756e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f372400ae90 tx=0x7f372400ffd0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.680+0000 7f373d76f740 1 -- 172.21.15.69:0/3163089530 shutdown_connections 2024-04-24T09:09:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.680+0000 7f373d76f740 1 --2- 172.21.15.69:0/3163089530 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55d693ae3e10 0x55d693ae6230 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.680+0000 7f373d76f740 1 --2- 172.21.15.69:0/3163089530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d693a6d390 0x55d693a70f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.680+0000 7f373d76f740 1 --2- 172.21.15.69:0/3163089530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d693a62ad0 0x55d693a70a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.680+0000 7f3721600640 0 client.16416 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.680+0000 7f373d76f740 1 --2- 172.21.15.69:0/3163089530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d693a621b0 0x55d693a756e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.680+0000 7f373d76f740 1 -- 172.21.15.69:0/3163089530 >> 172.21.15.69:0/3163089530 conn(0x55d693a5e6c0 msgr2=0x55d693a5d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.681+0000 7f373d76f740 1 -- 172.21.15.69:0/3163089530 shutdown_connections 2024-04-24T09:09:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.681+0000 7f373d76f740 1 -- 172.21.15.69:0/3163089530 wait complete. 2024-04-24T09:09:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.681+0000 7f373d76f740 20 client.16416 trim_cache size 0 max 16384 2024-04-24T09:09:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:09:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:09:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:09:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:09:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.682+0000 7f373a800640 1 -- 172.21.15.69:0/4024676879 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37180a3dc0 msgr2=0x7f37180a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.682+0000 7f373a800640 1 --2- 172.21.15.69:0/4024676879 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37180a3dc0 0x7f37180a6280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f372c005fd0 tx=0x7f372c005f60 comp rx=0 tx=0).stop 2024-04-24T09:09:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.682+0000 7f373a800640 1 -- 172.21.15.69:0/4024676879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3734107350 msgr2=0x7f37341a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.682+0000 7f373a800640 1 --2- 172.21.15.69:0/4024676879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3734107350 0x7f37341a0b80 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f371c004270 tx=0x7f371c0042a0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.682+0000 7f373a800640 1 -- 172.21.15.69:0/4024676879 shutdown_connections 2024-04-24T09:09:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.682+0000 7f373a800640 1 --2- 172.21.15.69:0/4024676879 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f37180a3dc0 0x7f37180a6280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.682+0000 7f373a800640 1 --2- 172.21.15.69:0/4024676879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f373419a7b0 0x7f373419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.682+0000 7f373a800640 1 --2- 172.21.15.69:0/4024676879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3734107350 0x7f37341a0b80 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.682+0000 7f373a800640 1 --2- 172.21.15.69:0/4024676879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3734102280 0x7f37341a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.682+0000 7f373a800640 1 -- 172.21.15.69:0/4024676879 >> 172.21.15.69:0/4024676879 conn(0x7f3734079490 msgr2=0x7f37340ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.682+0000 7f373a800640 1 -- 172.21.15.69:0/4024676879 shutdown_connections 2024-04-24T09:09:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:27.682+0000 7f373a800640 1 -- 172.21.15.69:0/4024676879 wait complete. 2024-04-24T09:09:27.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:27 smithi130 ceph-mon[31553]: pgmap v74: 129 pgs: 129 active+clean; 30 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 704 KiB/s wr, 142 op/s 2024-04-24T09:09:27.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:27 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/889318824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:09:28.273 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:09:28.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:09:28.274 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:09:28.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.500+0000 7fb05f200640 1 -- 172.21.15.69:0/4187480321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0601071e0 msgr2=0x7fb06010edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.500+0000 7fb05f200640 1 --2- 172.21.15.69:0/4187480321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0601071e0 0x7fb06010edb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb050009900 tx=0x7fb05002a410 comp rx=0 tx=0).stop 2024-04-24T09:09:28.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.500+0000 7fb05f200640 1 -- 172.21.15.69:0/4187480321 shutdown_connections 2024-04-24T09:09:28.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.500+0000 7fb05f200640 1 --2- 172.21.15.69:0/4187480321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0601071e0 0x7fb06010edb0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.500+0000 7fb05f200640 1 --2- 172.21.15.69:0/4187480321 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb060102ac0 0x7fb060106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.500+0000 7fb05f200640 1 --2- 172.21.15.69:0/4187480321 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb060102110 0x7fb0601024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.500+0000 7fb05f200640 1 -- 172.21.15.69:0/4187480321 >> 172.21.15.69:0/4187480321 conn(0x7fb0600fd9d0 msgr2=0x7fb0600ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:28.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.500+0000 7fb05f200640 1 -- 172.21.15.69:0/4187480321 shutdown_connections 2024-04-24T09:09:28.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.500+0000 7fb05f200640 1 -- 172.21.15.69:0/4187480321 wait complete. 2024-04-24T09:09:28.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.501+0000 7fb05f200640 1 Processor -- start 2024-04-24T09:09:28.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.501+0000 7fb05f200640 1 -- start start 2024-04-24T09:09:28.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.502+0000 7fb05f200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb060102110 0x7fb0601a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.502+0000 7fb05f200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0601071e0 0x7fb0601a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.502+0000 7fb05f200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb06019a760 0x7fb06019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.502+0000 7fb05f200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb060112130 con 0x7fb0601071e0 2024-04-24T09:09:28.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.502+0000 7fb05f200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb060111fb0 con 0x7fb060102110 2024-04-24T09:09:28.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.502+0000 7fb05f200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601122b0 con 0x7fb06019a760 2024-04-24T09:09:28.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.502+0000 7fb05d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0601071e0 0x7fb0601a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.502+0000 7fb05d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0601071e0 0x7fb0601a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51814/0 (socket says 172.21.15.69:51814) 2024-04-24T09:09:28.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.502+0000 7fb05d400640 1 -- 172.21.15.69:0/1719031310 learned_addr learned my addr 172.21.15.69:0/1719031310 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:28.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.502+0000 7fb05de00640 1 --2- 172.21.15.69:0/1719031310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb060102110 0x7fb0601a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.503+0000 7fb05e800640 1 --2- 172.21.15.69:0/1719031310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb06019a760 0x7fb06019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.503+0000 7fb05d400640 1 -- 172.21.15.69:0/1719031310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb06019a760 msgr2=0x7fb06019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.503+0000 7fb05d400640 1 --2- 172.21.15.69:0/1719031310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb06019a760 0x7fb06019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.503+0000 7fb05d400640 1 -- 172.21.15.69:0/1719031310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb060102110 msgr2=0x7fb0601a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.503+0000 7fb05d400640 1 --2- 172.21.15.69:0/1719031310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb060102110 0x7fb0601a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.503+0000 7fb05d400640 1 -- 172.21.15.69:0/1719031310 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06019b360 con 0x7fb0601071e0 2024-04-24T09:09:28.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.503+0000 7fb05e800640 1 --2- 172.21.15.69:0/1719031310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb06019a760 0x7fb06019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:28.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.503+0000 7fb05d400640 1 --2- 172.21.15.69:0/1719031310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0601071e0 0x7fb0601a0b40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb04800d770 tx=0x7fb04800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:28.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.503+0000 7fb056a00640 1 -- 172.21.15.69:0/1719031310 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb048017070 con 0x7fb0601071e0 2024-04-24T09:09:28.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.503+0000 7fb056a00640 1 -- 172.21.15.69:0/1719031310 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb048009c30 con 0x7fb0601071e0 2024-04-24T09:09:28.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.503+0000 7fb05f200640 1 -- 172.21.15.69:0/1719031310 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0601a73a0 con 0x7fb0601071e0 2024-04-24T09:09:28.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.503+0000 7fb056a00640 1 -- 172.21.15.69:0/1719031310 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04800f620 con 0x7fb0601071e0 2024-04-24T09:09:28.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.503+0000 7fb05f200640 1 -- 172.21.15.69:0/1719031310 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0601a7800 con 0x7fb0601071e0 2024-04-24T09:09:28.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.507+0000 7fb06786c740 1 Processor -- start 2024-04-24T09:09:28.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.507+0000 7fb06786c740 1 -- start start 2024-04-24T09:09:28.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.508+0000 7fb06786c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7270 0x562c5b4d7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.508+0000 7fb06786c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4d7b90 0x562c5b4e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.508+0000 7fb056a00640 1 -- 172.21.15.69:0/1719031310 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb04801a070 con 0x7fb0601071e0 2024-04-24T09:09:28.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.508+0000 7fb06786c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4e2260 0x562c5b4e4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.508+0000 7fb06786c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562c5b43ce70 con 0x562c5b4d7270 2024-04-24T09:09:28.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.508+0000 7fb06786c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562c5b3e3380 con 0x562c5b4d7b90 2024-04-24T09:09:28.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.508+0000 7fb06786c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562c5b438e30 con 0x562c5b4e2260 2024-04-24T09:09:28.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.508+0000 7fb05e800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7270 0x562c5b4d7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.508+0000 7fb05e800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7270 0x562c5b4d7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51816/0 (socket says 172.21.15.69:51816) 2024-04-24T09:09:28.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.508+0000 7fb05e800640 1 -- 172.21.15.69:0/593810220 learned_addr learned my addr 172.21.15.69:0/593810220 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:28.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.508+0000 7fb05de00640 1 --2- 172.21.15.69:0/593810220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4d7b90 0x562c5b4e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.508+0000 7fb05e800640 1 -- 172.21.15.69:0/593810220 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4e2260 msgr2=0x562c5b4e4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:28.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.508+0000 7fb05e800640 1 --2- 172.21.15.69:0/593810220 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4e2260 0x562c5b4e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.508+0000 7fb05e800640 1 -- 172.21.15.69:0/593810220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4d7b90 msgr2=0x562c5b4e1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:28.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.508+0000 7fb05e800640 1 --2- 172.21.15.69:0/593810220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4d7b90 0x562c5b4e1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.508+0000 7fb05e800640 1 -- 172.21.15.69:0/593810220 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c5b2177e0 con 0x562c5b4d7270 2024-04-24T09:09:28.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.509+0000 7fb05e800640 1 --2- 172.21.15.69:0/593810220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7270 0x562c5b4d7650 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb050031440 tx=0x7fb050032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ff65300a9d8a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:28.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.509+0000 7fb04f000640 1 -- 172.21.15.69:0/593810220 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05003f070 con 0x562c5b4d7270 2024-04-24T09:09:28.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.509+0000 7fb04f000640 1 -- 172.21.15.69:0/593810220 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0500316a0 con 0x562c5b4d7270 2024-04-24T09:09:28.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.509+0000 7fb04f000640 1 -- 172.21.15.69:0/593810220 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050035070 con 0x562c5b4d7270 2024-04-24T09:09:28.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.510+0000 7fb056a00640 1 --2- 172.21.15.69:0/1719031310 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb0440a3da0 0x7fb0440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.510+0000 7fb056a00640 1 -- 172.21.15.69:0/1719031310 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb0480d0650 con 0x7fb0601071e0 2024-04-24T09:09:28.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.510+0000 7fb05de00640 1 --2- 172.21.15.69:0/1719031310 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb0440a3da0 0x7fb0440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.510+0000 7fb06786c740 1 -- 172.21.15.69:0/593810220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7270 msgr2=0x562c5b4d7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.510+0000 7fb06786c740 1 --2- 172.21.15.69:0/593810220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7270 0x562c5b4d7650 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb050031440 tx=0x7fb050032c60 comp rx=0 tx=0).stop 2024-04-24T09:09:28.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.510+0000 7fb06786c740 1 -- 172.21.15.69:0/593810220 shutdown_connections 2024-04-24T09:09:28.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.510+0000 7fb06786c740 1 --2- 172.21.15.69:0/593810220 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4e2260 0x562c5b4e4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.510+0000 7fb06786c740 1 --2- 172.21.15.69:0/593810220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4d7b90 0x562c5b4e1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.510+0000 7fb06786c740 1 --2- 172.21.15.69:0/593810220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7270 0x562c5b4d7650 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.510+0000 7fb06786c740 1 -- 172.21.15.69:0/593810220 >> 172.21.15.69:0/593810220 conn(0x562c5b4d3590 msgr2=0x562c5b4d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:28.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.510+0000 7fb06786c740 1 -- 172.21.15.69:0/593810220 shutdown_connections 2024-04-24T09:09:28.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.510+0000 7fb06786c740 1 -- 172.21.15.69:0/593810220 wait complete. 2024-04-24T09:09:28.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.511+0000 7fb05de00640 1 --2- 172.21.15.69:0/1719031310 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb0440a3da0 0x7fb0440a6260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x562c5b4d8830 tx=0x7fb05800a3b0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:28.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.511+0000 7fb06786c740 1 -- start start 2024-04-24T09:09:28.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.511+0000 7fb06786c740 1 Processor -- start 2024-04-24T09:09:28.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.512+0000 7fb06786c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4d7270 0x562c5b4df240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.512+0000 7fb06786c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7b90 0x562c5b4df780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.512+0000 7fb06786c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4e2260 0x562c5b4dfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.512+0000 7fb06786c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562c5b4e4ff0 con 0x562c5b4d7b90 2024-04-24T09:09:28.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.512+0000 7fb06786c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562c5b3f6660 con 0x562c5b4e2260 2024-04-24T09:09:28.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.512+0000 7fb06786c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562c5b4e5170 con 0x562c5b4d7270 2024-04-24T09:09:28.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.512+0000 7fb06786c740 10 client.0 _ll_register_callbacks cb 0x562c5b3e4aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:28.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.512+0000 7fb05d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7b90 0x562c5b4df780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.512+0000 7fb05d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7b90 0x562c5b4df780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51820/0 (socket says 172.21.15.69:51820) 2024-04-24T09:09:28.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.512+0000 7fb05d400640 1 -- 172.21.15.69:0/2875453974 learned_addr learned my addr 172.21.15.69:0/2875453974 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:28.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.512+0000 7fb05e800640 1 --2- 172.21.15.69:0/2875453974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4e2260 0x562c5b4dfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.512+0000 7fb05de00640 1 --2- 172.21.15.69:0/2875453974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4d7270 0x562c5b4df240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.513+0000 7fb05d400640 1 -- 172.21.15.69:0/2875453974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4d7270 msgr2=0x562c5b4df240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:28.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.513+0000 7fb05d400640 1 --2- 172.21.15.69:0/2875453974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4d7270 0x562c5b4df240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.513+0000 7fb05d400640 1 -- 172.21.15.69:0/2875453974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4e2260 msgr2=0x562c5b4dfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:28.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.513+0000 7fb05d400640 1 --2- 172.21.15.69:0/2875453974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4e2260 0x562c5b4dfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.513+0000 7fb05d400640 1 -- 172.21.15.69:0/2875453974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c5b277f70 con 0x562c5b4d7b90 2024-04-24T09:09:28.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.513+0000 7fb05de00640 1 --2- 172.21.15.69:0/2875453974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4d7270 0x562c5b4df240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:28.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.513+0000 7fb05d400640 1 --2- 172.21.15.69:0/2875453974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7b90 0x562c5b4df780 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb0480986b0 tx=0x7fb04809be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ff121161fd998e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:28.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.514+0000 7fb043400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:28.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.514+0000 7fb043400640 1 -- 172.21.15.69:0/2875453974 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb048021070 con 0x562c5b4d7b90 2024-04-24T09:09:28.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.514+0000 7fb043400640 1 -- 172.21.15.69:0/2875453974 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb0480cdc80 con 0x562c5b4d7b90 2024-04-24T09:09:28.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.514+0000 7fb06786c740 1 -- 172.21.15.69:0/2875453974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0500316a0 con 0x562c5b4d7b90 2024-04-24T09:09:28.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.514+0000 7fb043400640 1 -- 172.21.15.69:0/2875453974 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb0480d0450 con 0x562c5b4d7b90 2024-04-24T09:09:28.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.514+0000 7fb06786c740 10 client.16428 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:28.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.514+0000 7fb06786c740 1 -- 172.21.15.69:0/2875453974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562c5b10f170 con 0x562c5b4d7b90 2024-04-24T09:09:28.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.515+0000 7fb043400640 1 -- 172.21.15.69:0/2875453974 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0500316a0 con 0x562c5b4d7b90 2024-04-24T09:09:28.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.515+0000 7fb043400640 1 -- 172.21.15.69:0/2875453974 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7fb0480a0120 con 0x562c5b4d7b90 2024-04-24T09:09:28.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.515+0000 7fb06786c740 10 client.16428 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:28.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.515+0000 7fb06786c740 10 client.16428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:28.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.515+0000 7fb06786c740 20 client.16428 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:28.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.515+0000 7fb06786c740 1 --2- 172.21.15.69:0/2875453974 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562c5b4f2650 0x562c5b4f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.515+0000 7fb06786c740 4 client.16428 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:28.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.515+0000 7fb06786c740 1 -- 172.21.15.69:0/2875453974 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c5b44a890 con 0x562c5b4f2650 2024-04-24T09:09:28.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.516+0000 7fb05de00640 1 --2- 172.21.15.69:0/2875453974 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562c5b4f2650 0x562c5b4f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.517+0000 7fb05de00640 1 --2- 172.21.15.69:0/2875453974 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562c5b4f2650 0x562c5b4f4a60 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51bfc985c5912d30 server_cookie=724fdcc2690f1539 in_seq=0 out_seq=0 2024-04-24T09:09:28.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.517+0000 7fb043400640 10 client.16428 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:28.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.519+0000 7fb043400640 1 -- 172.21.15.69:0/2875453974 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c5b44a890 con 0x562c5b4f2650 2024-04-24T09:09:28.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.519+0000 7fb043400640 10 client.16428 handle_command_reply: tid=0 2024-04-24T09:09:28.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.519+0000 7fb06786c740 1 client.16428 shutdown 2024-04-24T09:09:28.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.520+0000 7fb06786c740 1 -- 172.21.15.69:0/2875453974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7b90 msgr2=0x562c5b4df780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.520+0000 7fb06786c740 1 --2- 172.21.15.69:0/2875453974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7b90 0x562c5b4df780 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb0480986b0 tx=0x7fb04809be80 comp rx=0 tx=0).stop 2024-04-24T09:09:28.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.520+0000 7fb06786c740 1 -- 172.21.15.69:0/2875453974 shutdown_connections 2024-04-24T09:09:28.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.520+0000 7fb06786c740 1 --2- 172.21.15.69:0/2875453974 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562c5b4f2650 0x562c5b4f4a60 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.520+0000 7fb06786c740 1 --2- 172.21.15.69:0/2875453974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4e2260 0x562c5b4dfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.520+0000 7fb06786c740 1 --2- 172.21.15.69:0/2875453974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7b90 0x562c5b4df780 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.520+0000 7fb06786c740 1 --2- 172.21.15.69:0/2875453974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4d7270 0x562c5b4df240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.520+0000 7fb043400640 0 client.16428 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:28.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.520+0000 7fb06786c740 1 -- 172.21.15.69:0/2875453974 >> 172.21.15.69:0/2875453974 conn(0x562c5b4d3590 msgr2=0x562c5b4d9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:28.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.520+0000 7fb06786c740 1 -- 172.21.15.69:0/2875453974 shutdown_connections 2024-04-24T09:09:28.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.520+0000 7fb06786c740 1 -- 172.21.15.69:0/2875453974 wait complete. 2024-04-24T09:09:28.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.520+0000 7fb06786c740 20 client.16428 trim_cache size 0 max 16384 2024-04-24T09:09:28.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.540+0000 7fb06786c740 1 Processor -- start 2024-04-24T09:09:28.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.541+0000 7fb06786c740 1 -- start start 2024-04-24T09:09:28.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.541+0000 7fb06786c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7270 0x562c5b4edee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.541+0000 7fb06786c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4d7b90 0x562c5b4d99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.541+0000 7fb06786c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4e2260 0x562c5b4fa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.541+0000 7fb06786c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601b3c00 con 0x562c5b4d7270 2024-04-24T09:09:28.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.541+0000 7fb06786c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601b14a0 con 0x562c5b4d7b90 2024-04-24T09:09:28.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.541+0000 7fb06786c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601af2f0 con 0x562c5b4e2260 2024-04-24T09:09:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.541+0000 7fb05de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7270 0x562c5b4edee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.541+0000 7fb05de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7270 0x562c5b4edee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51822/0 (socket says 172.21.15.69:51822) 2024-04-24T09:09:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.541+0000 7fb05de00640 1 -- 172.21.15.69:0/4182496627 learned_addr learned my addr 172.21.15.69:0/4182496627 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.542+0000 7fb05e800640 1 --2- 172.21.15.69:0/4182496627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4e2260 0x562c5b4fa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.542+0000 7fb05d400640 1 --2- 172.21.15.69:0/4182496627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4d7b90 0x562c5b4d99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.542+0000 7fb05de00640 1 -- 172.21.15.69:0/4182496627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4e2260 msgr2=0x562c5b4fa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.542+0000 7fb05de00640 1 --2- 172.21.15.69:0/4182496627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4e2260 0x562c5b4fa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.542+0000 7fb05de00640 1 -- 172.21.15.69:0/4182496627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4d7b90 msgr2=0x562c5b4d99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.542+0000 7fb05de00640 1 --2- 172.21.15.69:0/4182496627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4d7b90 0x562c5b4d99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.542+0000 7fb05de00640 1 -- 172.21.15.69:0/4182496627 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c5b4f50e0 con 0x562c5b4d7270 2024-04-24T09:09:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.542+0000 7fb05de00640 1 --2- 172.21.15.69:0/4182496627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7270 0x562c5b4edee0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x562c5b4f52b0 tx=0x7fb058013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755ecbbaefa955b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.542+0000 7fb04e600640 1 -- 172.21.15.69:0/4182496627 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058011cd0 con 0x562c5b4d7270 2024-04-24T09:09:28.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.543+0000 7fb04e600640 1 -- 172.21.15.69:0/4182496627 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0580273a0 con 0x562c5b4d7270 2024-04-24T09:09:28.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.543+0000 7fb04e600640 1 -- 172.21.15.69:0/4182496627 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0580053c0 con 0x562c5b4d7270 2024-04-24T09:09:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.543+0000 7fb06786c740 1 -- 172.21.15.69:0/4182496627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7270 msgr2=0x562c5b4edee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.543+0000 7fb06786c740 1 --2- 172.21.15.69:0/4182496627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7270 0x562c5b4edee0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x562c5b4f52b0 tx=0x7fb058013000 comp rx=0 tx=0).stop 2024-04-24T09:09:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.544+0000 7fb06786c740 1 -- 172.21.15.69:0/4182496627 shutdown_connections 2024-04-24T09:09:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.544+0000 7fb06786c740 1 --2- 172.21.15.69:0/4182496627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4e2260 0x562c5b4fa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.544+0000 7fb06786c740 1 --2- 172.21.15.69:0/4182496627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4d7b90 0x562c5b4d99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.544+0000 7fb06786c740 1 --2- 172.21.15.69:0/4182496627 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4d7270 0x562c5b4edee0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.544+0000 7fb06786c740 1 -- 172.21.15.69:0/4182496627 >> 172.21.15.69:0/4182496627 conn(0x562c5b4d3590 msgr2=0x562c5b4d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.544+0000 7fb06786c740 1 -- 172.21.15.69:0/4182496627 shutdown_connections 2024-04-24T09:09:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.544+0000 7fb06786c740 1 -- 172.21.15.69:0/4182496627 wait complete. 2024-04-24T09:09:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.544+0000 7fb06786c740 1 -- start start 2024-04-24T09:09:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.544+0000 7fb06786c740 1 Processor -- start 2024-04-24T09:09:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.545+0000 7fb06786c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4d7270 0x562c5b4ea720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.545+0000 7fb06786c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4d7b90 0x562c5b4e6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.545+0000 7fb06786c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4e2260 0x562c5b4e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.545+0000 7fb06786c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601af960 con 0x562c5b4e2260 2024-04-24T09:09:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.545+0000 7fb06786c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601b0dd0 con 0x562c5b4d7b90 2024-04-24T09:09:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.545+0000 7fb06786c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601b0030 con 0x562c5b4d7270 2024-04-24T09:09:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.545+0000 7fb06786c740 10 client.0 _ll_register_callbacks cb 0x7fb04800de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.545+0000 7fb05e800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4e2260 0x562c5b4e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.545+0000 7fb05e800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4e2260 0x562c5b4e6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51824/0 (socket says 172.21.15.69:51824) 2024-04-24T09:09:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.545+0000 7fb05e800640 1 -- 172.21.15.69:0/2484979487 learned_addr learned my addr 172.21.15.69:0/2484979487 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.545+0000 7fb05d400640 1 --2- 172.21.15.69:0/2484979487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4d7b90 0x562c5b4e6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.546+0000 7fb05de00640 1 --2- 172.21.15.69:0/2484979487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4d7270 0x562c5b4ea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.546+0000 7fb05e800640 1 -- 172.21.15.69:0/2484979487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4d7270 msgr2=0x562c5b4ea720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.546+0000 7fb05e800640 1 --2- 172.21.15.69:0/2484979487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4d7270 0x562c5b4ea720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.546+0000 7fb05e800640 1 -- 172.21.15.69:0/2484979487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4d7b90 msgr2=0x562c5b4e6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.546+0000 7fb05e800640 1 --2- 172.21.15.69:0/2484979487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4d7b90 0x562c5b4e6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.546+0000 7fb05e800640 1 -- 172.21.15.69:0/2484979487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c5b4e77e0 con 0x562c5b4e2260 2024-04-24T09:09:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.546+0000 7fb05de00640 1 --2- 172.21.15.69:0/2484979487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4d7270 0x562c5b4ea720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.546+0000 7fb05e800640 1 --2- 172.21.15.69:0/2484979487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4e2260 0x562c5b4e6fe0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb05002a930 tx=0x7fb0500318f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ee439017760813 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.546+0000 7fb043400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.546+0000 7fb043400640 1 -- 172.21.15.69:0/2484979487 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05003f050 con 0x562c5b4e2260 2024-04-24T09:09:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.546+0000 7fb06786c740 1 -- 172.21.15.69:0/2484979487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0580273a0 con 0x562c5b4e2260 2024-04-24T09:09:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.546+0000 7fb043400640 1 -- 172.21.15.69:0/2484979487 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb050004450 con 0x562c5b4e2260 2024-04-24T09:09:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.547+0000 7fb043400640 1 -- 172.21.15.69:0/2484979487 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb050057050 con 0x562c5b4e2260 2024-04-24T09:09:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.547+0000 7fb06786c740 10 client.16434 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.547+0000 7fb06786c740 1 -- 172.21.15.69:0/2484979487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562c5b559bf0 con 0x562c5b4e2260 2024-04-24T09:09:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.547+0000 7fb043400640 1 -- 172.21.15.69:0/2484979487 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0580273a0 con 0x562c5b4e2260 2024-04-24T09:09:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.547+0000 7fb043400640 1 -- 172.21.15.69:0/2484979487 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7fb050004b10 con 0x562c5b4e2260 2024-04-24T09:09:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.547+0000 7fb06786c740 10 client.16434 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.547+0000 7fb06786c740 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.547+0000 7fb06786c740 20 client.16434 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.547+0000 7fb06786c740 1 --2- 172.21.15.69:0/2484979487 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562c5b559f30 0x562c5b55c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.547+0000 7fb06786c740 4 client.16434 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:09:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.547+0000 7fb06786c740 1 -- 172.21.15.69:0/2484979487 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562c5b444700 con 0x562c5b559f30 2024-04-24T09:09:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.548+0000 7fb05de00640 1 --2- 172.21.15.69:0/2484979487 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562c5b559f30 0x562c5b55c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.549+0000 7fb05de00640 1 --2- 172.21.15.69:0/2484979487 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562c5b559f30 0x562c5b55c320 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb640a6e5c456371 server_cookie=95396d17856ef3e7 in_seq=0 out_seq=0 2024-04-24T09:09:28.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.549+0000 7fb043400640 10 client.16434 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:28.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.551+0000 7fb043400640 1 -- 172.21.15.69:0/2484979487 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562c5b444700 con 0x562c5b559f30 2024-04-24T09:09:28.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.551+0000 7fb043400640 10 client.16434 handle_command_reply: tid=0 2024-04-24T09:09:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.551+0000 7fb06786c740 1 client.16434 shutdown 2024-04-24T09:09:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.552+0000 7fb06786c740 1 -- 172.21.15.69:0/2484979487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4e2260 msgr2=0x562c5b4e6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.552+0000 7fb06786c740 1 --2- 172.21.15.69:0/2484979487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4e2260 0x562c5b4e6fe0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb05002a930 tx=0x7fb0500318f0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.552+0000 7fb06786c740 1 -- 172.21.15.69:0/2484979487 shutdown_connections 2024-04-24T09:09:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.552+0000 7fb06786c740 1 --2- 172.21.15.69:0/2484979487 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x562c5b559f30 0x562c5b55c320 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.552+0000 7fb06786c740 1 --2- 172.21.15.69:0/2484979487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c5b4e2260 0x562c5b4e6fe0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.552+0000 7fb06786c740 1 --2- 172.21.15.69:0/2484979487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c5b4d7b90 0x562c5b4e6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.552+0000 7fb06786c740 1 --2- 172.21.15.69:0/2484979487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c5b4d7270 0x562c5b4ea720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.552+0000 7fb06786c740 1 -- 172.21.15.69:0/2484979487 >> 172.21.15.69:0/2484979487 conn(0x562c5b4d3590 msgr2=0x562c5b4d1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.552+0000 7fb043400640 0 client.16434 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.552+0000 7fb06786c740 1 -- 172.21.15.69:0/2484979487 shutdown_connections 2024-04-24T09:09:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.552+0000 7fb06786c740 1 -- 172.21.15.69:0/2484979487 wait complete. 2024-04-24T09:09:28.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.552+0000 7fb06786c740 20 client.16434 trim_cache size 0 max 16384 2024-04-24T09:09:28.554 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:09:28.554 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-04-24T09:09:28.554 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "c4b7a266-0dd2-49e4-bef9-0def57b81ee0", 2024-04-24T09:09:28.554 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-04-24T09:09:28.554 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:09:28.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.553+0000 7fb05f200640 1 -- 172.21.15.69:0/1719031310 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb0440a3da0 msgr2=0x7fb0440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.553+0000 7fb05f200640 1 --2- 172.21.15.69:0/1719031310 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb0440a3da0 0x7fb0440a6260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x562c5b4d8830 tx=0x7fb05800a3b0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.553+0000 7fb05f200640 1 -- 172.21.15.69:0/1719031310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0601071e0 msgr2=0x7fb0601a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.553+0000 7fb05f200640 1 --2- 172.21.15.69:0/1719031310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0601071e0 0x7fb0601a0b40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb04800d770 tx=0x7fb04800dc30 comp rx=0 tx=0).stop 2024-04-24T09:09:28.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.553+0000 7fb05f200640 1 -- 172.21.15.69:0/1719031310 shutdown_connections 2024-04-24T09:09:28.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.553+0000 7fb05f200640 1 --2- 172.21.15.69:0/1719031310 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb0440a3da0 0x7fb0440a6260 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.553+0000 7fb05f200640 1 --2- 172.21.15.69:0/1719031310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb06019a760 0x7fb06019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.553+0000 7fb05f200640 1 --2- 172.21.15.69:0/1719031310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0601071e0 0x7fb0601a0b40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.553+0000 7fb05f200640 1 --2- 172.21.15.69:0/1719031310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb060102110 0x7fb0601a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.553+0000 7fb05f200640 1 -- 172.21.15.69:0/1719031310 >> 172.21.15.69:0/1719031310 conn(0x7fb0600fd9d0 msgr2=0x7fb0600fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:28.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.554+0000 7fb05f200640 1 -- 172.21.15.69:0/1719031310 shutdown_connections 2024-04-24T09:09:28.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.554+0000 7fb05f200640 1 -- 172.21.15.69:0/1719031310 wait complete. 2024-04-24T09:09:28.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:09:28.572 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:09:28.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.785+0000 7f3ca9000640 1 -- 172.21.15.69:0/433819291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ca41071e0 msgr2=0x7f3ca410edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.785+0000 7f3ca9000640 1 --2- 172.21.15.69:0/433819291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ca41071e0 0x7f3ca410edb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009ca0 tx=0x7f3c9c02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.785+0000 7f3ca9000640 1 -- 172.21.15.69:0/433819291 shutdown_connections 2024-04-24T09:09:28.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.785+0000 7f3ca9000640 1 --2- 172.21.15.69:0/433819291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ca41071e0 0x7f3ca410edb0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.785+0000 7f3ca9000640 1 --2- 172.21.15.69:0/433819291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ca4102ac0 0x7f3ca4106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.785+0000 7f3ca9000640 1 --2- 172.21.15.69:0/433819291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ca4102110 0x7f3ca41024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.785+0000 7f3ca9000640 1 -- 172.21.15.69:0/433819291 >> 172.21.15.69:0/433819291 conn(0x7f3ca40fd9d0 msgr2=0x7f3ca40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:28.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.785+0000 7f3ca9000640 1 -- 172.21.15.69:0/433819291 shutdown_connections 2024-04-24T09:09:28.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.785+0000 7f3ca9000640 1 -- 172.21.15.69:0/433819291 wait complete. 2024-04-24T09:09:28.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.786+0000 7f3ca9000640 1 Processor -- start 2024-04-24T09:09:28.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.787+0000 7f3ca9000640 1 -- start start 2024-04-24T09:09:28.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.787+0000 7f3ca9000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ca4102110 0x7f3ca4074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.787+0000 7f3ca9000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ca41071e0 0x7f3ca4074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.787+0000 7f3ca9000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ca4074d80 0x7f3ca41a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.787+0000 7f3ca9000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41121a0 con 0x7f3ca4102110 2024-04-24T09:09:28.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.787+0000 7f3ca9000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4112020 con 0x7f3ca4074d80 2024-04-24T09:09:28.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.787+0000 7f3ca9000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4112320 con 0x7f3ca41071e0 2024-04-24T09:09:28.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.787+0000 7f3ca3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ca4102110 0x7f3ca4074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.787+0000 7f3ca3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ca4102110 0x7f3ca4074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51830/0 (socket says 172.21.15.69:51830) 2024-04-24T09:09:28.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.787+0000 7f3ca3400640 1 -- 172.21.15.69:0/2899163018 learned_addr learned my addr 172.21.15.69:0/2899163018 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:28.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.788+0000 7f3ca3e00640 1 --2- 172.21.15.69:0/2899163018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ca4074d80 0x7f3ca41a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.788+0000 7f3c9be00640 1 --2- 172.21.15.69:0/2899163018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ca41071e0 0x7f3ca4074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.788+0000 7f3ca3400640 1 -- 172.21.15.69:0/2899163018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ca41071e0 msgr2=0x7f3ca4074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.788+0000 7f3ca3400640 1 --2- 172.21.15.69:0/2899163018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ca41071e0 0x7f3ca4074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.788+0000 7f3ca3400640 1 -- 172.21.15.69:0/2899163018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ca4074d80 msgr2=0x7f3ca41a6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.788+0000 7f3ca3400640 1 --2- 172.21.15.69:0/2899163018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ca4074d80 0x7f3ca41a6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.788+0000 7f3ca3400640 1 -- 172.21.15.69:0/2899163018 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca41a6f70 con 0x7f3ca4102110 2024-04-24T09:09:28.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.789+0000 7f3ca3400640 1 --2- 172.21.15.69:0/2899163018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ca4102110 0x7f3ca4074300 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400d950 tx=0x7f3c9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:28.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.789+0000 7f3ca0c00640 1 -- 172.21.15.69:0/2899163018 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94017070 con 0x7f3ca4102110 2024-04-24T09:09:28.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.789+0000 7f3ca0c00640 1 -- 172.21.15.69:0/2899163018 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c94009c30 con 0x7f3ca4102110 2024-04-24T09:09:28.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.789+0000 7f3ca9000640 1 -- 172.21.15.69:0/2899163018 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca41a7260 con 0x7f3ca4102110 2024-04-24T09:09:28.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.789+0000 7f3ca0c00640 1 -- 172.21.15.69:0/2899163018 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9400f620 con 0x7f3ca4102110 2024-04-24T09:09:28.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.789+0000 7f3ca9000640 1 -- 172.21.15.69:0/2899163018 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca41a7800 con 0x7f3ca4102110 2024-04-24T09:09:28.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.793+0000 7f3ca0c00640 1 -- 172.21.15.69:0/2899163018 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3c940109a0 con 0x7f3ca4102110 2024-04-24T09:09:28.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.793+0000 7f3cad42f740 1 Processor -- start 2024-04-24T09:09:28.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.794+0000 7f3cad42f740 1 -- start start 2024-04-24T09:09:28.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.794+0000 7f3cad42f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4a9a30 0x55646e4a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.794+0000 7f3cad42f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4aa350 0x55646e4b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.794+0000 7f3cad42f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 0x55646e4b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.794+0000 7f3cad42f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55646e40f1f0 con 0x55646e4a9a30 2024-04-24T09:09:28.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.794+0000 7f3cad42f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55646e3b5350 con 0x55646e4b4a20 2024-04-24T09:09:28.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.794+0000 7f3cad42f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55646e40b1b0 con 0x55646e4aa350 2024-04-24T09:09:28.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.795+0000 7f3ca3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4a9a30 0x55646e4a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.795+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 0x55646e4b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.795+0000 7f3ca3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4a9a30 0x55646e4a9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51836/0 (socket says 172.21.15.69:51836) 2024-04-24T09:09:28.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.795+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 0x55646e4b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.69:39682/0 (socket says 172.21.15.69:39682) 2024-04-24T09:09:28.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.795+0000 7f3ca3e00640 1 -- 172.21.15.69:0/3523269560 learned_addr learned my addr 172.21.15.69:0/3523269560 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:28.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.795+0000 7f3ca3400640 1 --2- 172.21.15.69:0/3523269560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4aa350 0x55646e4b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.795+0000 7f3ca0c00640 1 --2- 172.21.15.69:0/2899163018 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f3c880a3dd0 0x7f3c880a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.795+0000 7f3c9be00640 1 -- 172.21.15.69:0/3523269560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4aa350 msgr2=0x55646e4b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:28.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.795+0000 7f3c9be00640 1 --2- 172.21.15.69:0/3523269560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4aa350 0x55646e4b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.795+0000 7f3c9be00640 1 -- 172.21.15.69:0/3523269560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4a9a30 msgr2=0x55646e4a9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:28.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.795+0000 7f3c9be00640 1 --2- 172.21.15.69:0/3523269560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4a9a30 0x55646e4a9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.795+0000 7f3c9be00640 1 -- 172.21.15.69:0/3523269560 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55646e409f50 con 0x55646e4b4a20 2024-04-24T09:09:28.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.795+0000 7f3ca0c00640 1 -- 172.21.15.69:0/2899163018 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f3c940d0700 con 0x7f3ca4102110 2024-04-24T09:09:28.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.796+0000 7f3ca3e00640 1 --2- 172.21.15.69:0/2899163018 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f3c880a3dd0 0x7f3c880a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.796+0000 7f3c9be00640 1 --2- 172.21.15.69:0/3523269560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 0x55646e4b6e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4075d90 tx=0x7f3c8c00cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4944ec6872e72176 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:28.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.796+0000 7f3c98c00640 1 -- 172.21.15.69:0/3523269560 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c00ee90 con 0x55646e4b4a20 2024-04-24T09:09:28.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.796+0000 7f3c98c00640 1 -- 172.21.15.69:0/3523269560 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c8c0108e0 con 0x55646e4b4a20 2024-04-24T09:09:28.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.796+0000 7f3c98c00640 1 -- 172.21.15.69:0/3523269560 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c01a980 con 0x55646e4b4a20 2024-04-24T09:09:28.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.797+0000 7f3cad42f740 1 -- 172.21.15.69:0/3523269560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 msgr2=0x55646e4b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.797+0000 7f3ca3e00640 1 --2- 172.21.15.69:0/2899163018 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f3c880a3dd0 0x7f3c880a6290 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c02aeb0 tx=0x7f3c9c0095c0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:28.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.797+0000 7f3cad42f740 1 --2- 172.21.15.69:0/3523269560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 0x55646e4b6e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4075d90 tx=0x7f3c8c00cf30 comp rx=0 tx=0).stop 2024-04-24T09:09:28.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.797+0000 7f3cad42f740 1 -- 172.21.15.69:0/3523269560 shutdown_connections 2024-04-24T09:09:28.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.797+0000 7f3cad42f740 1 --2- 172.21.15.69:0/3523269560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 0x55646e4b6e10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.797+0000 7f3cad42f740 1 --2- 172.21.15.69:0/3523269560 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4aa350 0x55646e4b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.797+0000 7f3cad42f740 1 --2- 172.21.15.69:0/3523269560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4a9a30 0x55646e4a9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.797+0000 7f3cad42f740 1 -- 172.21.15.69:0/3523269560 >> 172.21.15.69:0/3523269560 conn(0x55646e4a5d50 msgr2=0x55646e4a6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:28.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.797+0000 7f3cad42f740 1 -- 172.21.15.69:0/3523269560 shutdown_connections 2024-04-24T09:09:28.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.797+0000 7f3cad42f740 1 -- 172.21.15.69:0/3523269560 wait complete. 2024-04-24T09:09:28.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.798+0000 7f3cad42f740 1 -- start start 2024-04-24T09:09:28.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.798+0000 7f3cad42f740 1 Processor -- start 2024-04-24T09:09:28.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.799+0000 7f3cad42f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4a9a30 0x55646e4b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.799+0000 7f3cad42f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4aa350 0x55646e4b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.799+0000 7f3cad42f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 0x55646e4b2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.799+0000 7f3cad42f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55646e4b7810 con 0x55646e4aa350 2024-04-24T09:09:28.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.799+0000 7f3cad42f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55646e3c8690 con 0x55646e4b4a20 2024-04-24T09:09:28.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.799+0000 7f3cad42f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55646e4b7990 con 0x55646e4a9a30 2024-04-24T09:09:28.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.799+0000 7f3cad42f740 10 client.0 _ll_register_callbacks cb 0x55646e404650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:28.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.799+0000 7f3ca3e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 0x55646e4b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.799+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4aa350 0x55646e4b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.799+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4aa350 0x55646e4b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51850/0 (socket says 172.21.15.69:51850) 2024-04-24T09:09:28.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.799+0000 7f3c9be00640 1 -- 172.21.15.69:0/1663259795 learned_addr learned my addr 172.21.15.69:0/1663259795 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:28.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.799+0000 7f3ca3400640 1 --2- 172.21.15.69:0/1663259795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4a9a30 0x55646e4b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.799+0000 7f3ca3e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 0x55646e4b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.69:39686/0 (socket says 172.21.15.69:39686) 2024-04-24T09:09:28.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.800+0000 7f3c9be00640 1 -- 172.21.15.69:0/1663259795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4a9a30 msgr2=0x55646e4b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:28.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.800+0000 7f3c9be00640 1 --2- 172.21.15.69:0/1663259795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4a9a30 0x55646e4b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.800+0000 7f3c9be00640 1 -- 172.21.15.69:0/1663259795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 msgr2=0x55646e4b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:28.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.800+0000 7f3c9be00640 1 --2- 172.21.15.69:0/1663259795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 0x55646e4b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.800+0000 7f3c9be00640 1 -- 172.21.15.69:0/1663259795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55646e3c7920 con 0x55646e4aa350 2024-04-24T09:09:28.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.800+0000 7f3c9be00640 1 --2- 172.21.15.69:0/1663259795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4aa350 0x55646e4b1ee0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00f010 tx=0x7f3c8c010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb5645b1ccd5b52 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:28.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.800+0000 7f3c91600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:28.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.801+0000 7f3c91600640 1 -- 172.21.15.69:0/1663259795 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c02a060 con 0x55646e4aa350 2024-04-24T09:09:28.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.801+0000 7f3cad42f740 1 -- 172.21.15.69:0/1663259795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c8c0108e0 con 0x55646e4aa350 2024-04-24T09:09:28.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.801+0000 7f3c91600640 1 -- 172.21.15.69:0/1663259795 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f3c8c010a90 con 0x55646e4aa350 2024-04-24T09:09:28.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.801+0000 7f3c91600640 1 -- 172.21.15.69:0/1663259795 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c8c031050 con 0x55646e4aa350 2024-04-24T09:09:28.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.801+0000 7f3cad42f740 10 client.16446 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:28.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.801+0000 7f3cad42f740 1 -- 172.21.15.69:0/1663259795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55646e1eb7d0 con 0x55646e4aa350 2024-04-24T09:09:28.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.801+0000 7f3c91600640 1 -- 172.21.15.69:0/1663259795 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c8c0108e0 con 0x55646e4aa350 2024-04-24T09:09:28.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.802+0000 7f3c91600640 1 -- 172.21.15.69:0/1663259795 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7f3c8c004880 con 0x55646e4aa350 2024-04-24T09:09:28.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.802+0000 7f3cad42f740 10 client.16446 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:28.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.802+0000 7f3cad42f740 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:28.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.802+0000 7f3cad42f740 20 client.16446 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:28.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.802+0000 7f3cad42f740 1 --2- 172.21.15.69:0/1663259795 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55646e4c4d20 0x55646e4c7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.802+0000 7f3cad42f740 4 client.16446 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:28.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.802+0000 7f3cad42f740 1 -- 172.21.15.69:0/1663259795 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55646e41cc10 con 0x55646e4c4d20 2024-04-24T09:09:28.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.802+0000 7f3ca3400640 1 --2- 172.21.15.69:0/1663259795 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55646e4c4d20 0x55646e4c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.803+0000 7f3ca3400640 1 --2- 172.21.15.69:0/1663259795 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55646e4c4d20 0x55646e4c7110 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b10c756c901a6d server_cookie=f1d32c90f6dcdf1a in_seq=0 out_seq=0 2024-04-24T09:09:28.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.803+0000 7f3c91600640 10 client.16446 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:28.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.804+0000 7f3c91600640 1 -- 172.21.15.69:0/1663259795 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55646e41cc10 con 0x55646e4c4d20 2024-04-24T09:09:28.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.804+0000 7f3c91600640 10 client.16446 handle_command_reply: tid=0 2024-04-24T09:09:28.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.804+0000 7f3cad42f740 1 client.16446 shutdown 2024-04-24T09:09:28.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.804+0000 7f3cad42f740 1 -- 172.21.15.69:0/1663259795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4aa350 msgr2=0x55646e4b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.804+0000 7f3cad42f740 1 --2- 172.21.15.69:0/1663259795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4aa350 0x55646e4b1ee0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00f010 tx=0x7f3c8c010660 comp rx=0 tx=0).stop 2024-04-24T09:09:28.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.804+0000 7f3cad42f740 1 -- 172.21.15.69:0/1663259795 shutdown_connections 2024-04-24T09:09:28.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.805+0000 7f3cad42f740 1 --2- 172.21.15.69:0/1663259795 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55646e4c4d20 0x55646e4c7110 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.805+0000 7f3cad42f740 1 --2- 172.21.15.69:0/1663259795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 0x55646e4b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.805+0000 7f3cad42f740 1 --2- 172.21.15.69:0/1663259795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4aa350 0x55646e4b1ee0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.805+0000 7f3c91600640 0 client.16446 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:28.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.805+0000 7f3cad42f740 1 --2- 172.21.15.69:0/1663259795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4a9a30 0x55646e4b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.805+0000 7f3cad42f740 1 -- 172.21.15.69:0/1663259795 >> 172.21.15.69:0/1663259795 conn(0x55646e4a5d50 msgr2=0x55646e4abb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:28.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.805+0000 7f3cad42f740 1 -- 172.21.15.69:0/1663259795 shutdown_connections 2024-04-24T09:09:28.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.805+0000 7f3cad42f740 1 -- 172.21.15.69:0/1663259795 wait complete. 2024-04-24T09:09:28.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.805+0000 7f3cad42f740 20 client.16446 trim_cache size 0 max 16384 2024-04-24T09:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.829+0000 7f3cad42f740 1 Processor -- start 2024-04-24T09:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.829+0000 7f3cad42f740 1 -- start start 2024-04-24T09:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.829+0000 7f3cad42f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4a9a30 0x55646e4c06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.830+0000 7f3cad42f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4aa350 0x55646e4ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.830+0000 7f3cad42f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 0x55646e4cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.830+0000 7f3cad42f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41b3bd0 con 0x55646e4aa350 2024-04-24T09:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.830+0000 7f3cad42f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41b1470 con 0x55646e4b4a20 2024-04-24T09:09:28.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.830+0000 7f3cad42f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41af2c0 con 0x55646e4a9a30 2024-04-24T09:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.830+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4aa350 0x55646e4ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.830+0000 7f3ca3e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 0x55646e4cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.830+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4aa350 0x55646e4ac2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51852/0 (socket says 172.21.15.69:51852) 2024-04-24T09:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.830+0000 7f3c9be00640 1 -- 172.21.15.69:0/3305867673 learned_addr learned my addr 172.21.15.69:0/3305867673 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.830+0000 7f3ca3400640 1 --2- 172.21.15.69:0/3305867673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4a9a30 0x55646e4c06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.830+0000 7f3c9be00640 1 -- 172.21.15.69:0/3305867673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4a9a30 msgr2=0x55646e4c06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.830+0000 7f3c9be00640 1 --2- 172.21.15.69:0/3305867673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4a9a30 0x55646e4c06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.830+0000 7f3c9be00640 1 -- 172.21.15.69:0/3305867673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 msgr2=0x55646e4cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.830+0000 7f3c9be00640 1 --2- 172.21.15.69:0/3305867673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 0x55646e4cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.830+0000 7f3c9be00640 1 -- 172.21.15.69:0/3305867673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55646e1213f0 con 0x55646e4aa350 2024-04-24T09:09:28.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.831+0000 7f3c9be00640 1 --2- 172.21.15.69:0/3305867673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4aa350 0x55646e4ac2a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c004c80 tx=0x7f3c8c004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8e48e884726a7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:28.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.831+0000 7f3c93e00640 1 -- 172.21.15.69:0/3305867673 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c02a040 con 0x55646e4aa350 2024-04-24T09:09:28.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.831+0000 7f3c93e00640 1 -- 172.21.15.69:0/3305867673 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c8c01a4a0 con 0x55646e4aa350 2024-04-24T09:09:28.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.831+0000 7f3c93e00640 1 -- 172.21.15.69:0/3305867673 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c005000 con 0x55646e4aa350 2024-04-24T09:09:28.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.832+0000 7f3cad42f740 1 -- 172.21.15.69:0/3305867673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4aa350 msgr2=0x55646e4ac2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.832+0000 7f3cad42f740 1 --2- 172.21.15.69:0/3305867673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4aa350 0x55646e4ac2a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c004c80 tx=0x7f3c8c004e70 comp rx=0 tx=0).stop 2024-04-24T09:09:28.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.832+0000 7f3cad42f740 1 -- 172.21.15.69:0/3305867673 shutdown_connections 2024-04-24T09:09:28.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.832+0000 7f3cad42f740 1 --2- 172.21.15.69:0/3305867673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4b4a20 0x55646e4cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.832+0000 7f3cad42f740 1 --2- 172.21.15.69:0/3305867673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4aa350 0x55646e4ac2a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.832+0000 7f3cad42f740 1 --2- 172.21.15.69:0/3305867673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4a9a30 0x55646e4c06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.832+0000 7f3cad42f740 1 -- 172.21.15.69:0/3305867673 >> 172.21.15.69:0/3305867673 conn(0x55646e4a5d50 msgr2=0x55646e4b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:28.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.832+0000 7f3cad42f740 1 -- 172.21.15.69:0/3305867673 shutdown_connections 2024-04-24T09:09:28.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.832+0000 7f3cad42f740 1 -- 172.21.15.69:0/3305867673 wait complete. 2024-04-24T09:09:28.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.833+0000 7f3cad42f740 1 -- start start 2024-04-24T09:09:28.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.833+0000 7f3cad42f740 1 Processor -- start 2024-04-24T09:09:28.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.833+0000 7f3cad42f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4a9a30 0x55646e4bcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.833+0000 7f3cad42f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4aa350 0x55646e4ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.833+0000 7f3cad42f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4b4a20 0x55646e4ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.833+0000 7f3cad42f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41af930 con 0x55646e4a9a30 2024-04-24T09:09:28.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.833+0000 7f3cad42f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41b0da0 con 0x55646e4aa350 2024-04-24T09:09:28.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.833+0000 7f3cad42f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41b0000 con 0x55646e4b4a20 2024-04-24T09:09:28.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.833+0000 7f3cad42f740 10 client.0 _ll_register_callbacks cb 0x7f3c8c014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:28.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.834+0000 7f3ca3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4a9a30 0x55646e4bcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.834+0000 7f3ca3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4a9a30 0x55646e4bcdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51858/0 (socket says 172.21.15.69:51858) 2024-04-24T09:09:28.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.834+0000 7f3ca3400640 1 -- 172.21.15.69:0/314535861 learned_addr learned my addr 172.21.15.69:0/314535861 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:28.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.834+0000 7f3c9be00640 1 --2- 172.21.15.69:0/314535861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4aa350 0x55646e4ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.834+0000 7f3ca3e00640 1 --2- 172.21.15.69:0/314535861 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4b4a20 0x55646e4ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.834+0000 7f3ca3400640 1 -- 172.21.15.69:0/314535861 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4b4a20 msgr2=0x55646e4ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:28.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.834+0000 7f3ca3400640 1 --2- 172.21.15.69:0/314535861 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4b4a20 0x55646e4ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.834+0000 7f3ca3400640 1 -- 172.21.15.69:0/314535861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4aa350 msgr2=0x55646e4ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:28.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.834+0000 7f3ca3400640 1 --2- 172.21.15.69:0/314535861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4aa350 0x55646e4ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.834+0000 7f3ca3400640 1 -- 172.21.15.69:0/314535861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55646e0e10f0 con 0x55646e4a9a30 2024-04-24T09:09:28.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.834+0000 7f3c9be00640 1 --2- 172.21.15.69:0/314535861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4aa350 0x55646e4ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:28.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.834+0000 7f3ca3400640 1 --2- 172.21.15.69:0/314535861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4a9a30 0x55646e4bcdd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3c940cdb10 tx=0x7f3c9400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57542d9751d66b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:28.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.834+0000 7f3c91600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:28.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.835+0000 7f3c91600640 1 -- 172.21.15.69:0/314535861 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94020040 con 0x55646e4a9a30 2024-04-24T09:09:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.835+0000 7f3c91600640 1 -- 172.21.15.69:0/314535861 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f3c94098970 con 0x55646e4a9a30 2024-04-24T09:09:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.835+0000 7f3cad42f740 1 -- 172.21.15.69:0/314535861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c8c01a4a0 con 0x55646e4a9a30 2024-04-24T09:09:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.835+0000 7f3c91600640 1 -- 172.21.15.69:0/314535861 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c940a4f70 con 0x55646e4a9a30 2024-04-24T09:09:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.835+0000 7f3cad42f740 10 client.16452 fetch_fsmap learned FSMap version 26 2024-04-24T09:09:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.835+0000 7f3cad42f740 1 -- 172.21.15.69:0/314535861 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55646e3d4810 con 0x55646e4a9a30 2024-04-24T09:09:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.835+0000 7f3c91600640 1 -- 172.21.15.69:0/314535861 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c8c01a4a0 con 0x55646e4a9a30 2024-04-24T09:09:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.836+0000 7f3c91600640 1 -- 172.21.15.69:0/314535861 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7f3c940d3ce0 con 0x55646e4a9a30 2024-04-24T09:09:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.836+0000 7f3cad42f740 10 client.16452 fetch_fsmap finished waiting for FSMap version 26 2024-04-24T09:09:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.836+0000 7f3cad42f740 10 client.16452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.836+0000 7f3cad42f740 20 client.16452 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.836+0000 7f3cad42f740 1 --2- 172.21.15.69:0/314535861 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55646e52b160 0x55646e52d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.836+0000 7f3cad42f740 4 client.16452 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:09:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.836+0000 7f3cad42f740 1 -- 172.21.15.69:0/314535861 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55646e416a80 con 0x55646e52b160 2024-04-24T09:09:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.836+0000 7f3c9be00640 1 --2- 172.21.15.69:0/314535861 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55646e52b160 0x55646e52d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.837+0000 7f3c9be00640 1 --2- 172.21.15.69:0/314535861 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55646e52b160 0x55646e52d5a0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b80b16badfe9f9e server_cookie=6444cd793109ff12 in_seq=0 out_seq=0 2024-04-24T09:09:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.837+0000 7f3c91600640 10 client.16452 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.837+0000 7f3c91600640 1 -- 172.21.15.69:0/314535861 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55646e416a80 con 0x55646e52b160 2024-04-24T09:09:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.837+0000 7f3c91600640 10 client.16452 handle_command_reply: tid=0 2024-04-24T09:09:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.838+0000 7f3cad42f740 1 client.16452 shutdown 2024-04-24T09:09:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.838+0000 7f3cad42f740 1 -- 172.21.15.69:0/314535861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4a9a30 msgr2=0x55646e4bcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.838+0000 7f3cad42f740 1 --2- 172.21.15.69:0/314535861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4a9a30 0x55646e4bcdd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3c940cdb10 tx=0x7f3c9400bf60 comp rx=0 tx=0).stop 2024-04-24T09:09:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.838+0000 7f3cad42f740 1 -- 172.21.15.69:0/314535861 shutdown_connections 2024-04-24T09:09:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.838+0000 7f3cad42f740 1 --2- 172.21.15.69:0/314535861 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55646e52b160 0x55646e52d5a0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.838+0000 7f3cad42f740 1 --2- 172.21.15.69:0/314535861 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55646e4b4a20 0x55646e4ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.838+0000 7f3cad42f740 1 --2- 172.21.15.69:0/314535861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55646e4aa350 0x55646e4ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.838+0000 7f3cad42f740 1 --2- 172.21.15.69:0/314535861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55646e4a9a30 0x55646e4bcdd0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.838+0000 7f3cad42f740 1 -- 172.21.15.69:0/314535861 >> 172.21.15.69:0/314535861 conn(0x55646e4a5d50 msgr2=0x55646e4a4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.838+0000 7f3c91600640 0 client.16452 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.839+0000 7f3cad42f740 1 -- 172.21.15.69:0/314535861 shutdown_connections 2024-04-24T09:09:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.839+0000 7f3cad42f740 1 -- 172.21.15.69:0/314535861 wait complete. 2024-04-24T09:09:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.839+0000 7f3cad42f740 20 client.16452 trim_cache size 0 max 16384 2024-04-24T09:09:28.841 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:09:28.841 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:09:28.841 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-04-24T09:09:28.842 INFO:teuthology.orchestra.run.smithi069.stdout: "c4b7a266-0dd2-49e4-bef9-0def57b81ee0": { 2024-04-24T09:09:28.842 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-04-24T09:09:28.842 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "c4b7a266-0dd2-49e4-bef9-0def57b81ee0", 2024-04-24T09:09:28.842 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-04-24T09:09:28.842 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:09:28.842 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:09:28.842 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:09:28.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.840+0000 7f3ca9000640 1 -- 172.21.15.69:0/2899163018 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f3c880a3dd0 msgr2=0x7f3c880a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.840+0000 7f3ca9000640 1 --2- 172.21.15.69:0/2899163018 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f3c880a3dd0 0x7f3c880a6290 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c02aeb0 tx=0x7f3c9c0095c0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.840+0000 7f3ca9000640 1 -- 172.21.15.69:0/2899163018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ca4102110 msgr2=0x7f3ca4074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:28.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.840+0000 7f3ca9000640 1 --2- 172.21.15.69:0/2899163018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ca4102110 0x7f3ca4074300 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400d950 tx=0x7f3c9400de10 comp rx=0 tx=0).stop 2024-04-24T09:09:28.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.840+0000 7f3ca9000640 1 -- 172.21.15.69:0/2899163018 shutdown_connections 2024-04-24T09:09:28.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.840+0000 7f3ca9000640 1 --2- 172.21.15.69:0/2899163018 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f3c880a3dd0 0x7f3c880a6290 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.840+0000 7f3ca9000640 1 --2- 172.21.15.69:0/2899163018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ca4074d80 0x7f3ca41a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.840+0000 7f3ca9000640 1 --2- 172.21.15.69:0/2899163018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3ca41071e0 0x7f3ca4074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.840+0000 7f3ca9000640 1 --2- 172.21.15.69:0/2899163018 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ca4102110 0x7f3ca4074300 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:28.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.840+0000 7f3ca9000640 1 -- 172.21.15.69:0/2899163018 >> 172.21.15.69:0/2899163018 conn(0x7f3ca40fd9d0 msgr2=0x7f3ca40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:28.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.840+0000 7f3ca9000640 1 -- 172.21.15.69:0/2899163018 shutdown_connections 2024-04-24T09:09:28.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:28.840+0000 7f3ca9000640 1 -- 172.21.15.69:0/2899163018 wait complete. 2024-04-24T09:09:28.858 INFO:tasks.cephfs.filesystem:scrub status for tag:c4b7a266-0dd2-49e4-bef9-0def57b81ee0 - {'path': '/', 'tag': 'c4b7a266-0dd2-49e4-bef9-0def57b81ee0', 'options': 'recursive,force'} 2024-04-24T09:09:29.067 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.573+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000334 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/md5.c auth v438 ap=1 s=8743 n(v0 rc2024-04-24T09:08:56.121480+0000 b8743 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589e9ed1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000334:[<0x100000002d8/md5.c v184>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000334:[<0x100000002d8/md5.c v438>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.068 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.573+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000030f [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fiemap-fault.c auth v430 ap=1 s=1542 n(v0 rc2024-04-24T09:08:55.770195+0000 b1542 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589e9ed1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000030f:[<0x100000002d8/fiemap-fault.c v110>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000030f:[<0x100000002d8/fiemap-fault.c v430>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.068 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.574+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002db [...3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/ auth v402 ap=1 f(v0 m2024-04-24T09:08:55.526413+0000 19=19+0) n(v0 rc2024-04-24T09:08:55.529265+0000 b77976 20=19+1) (inest lock) (ifile excl) caps={15489=pAsLsXsFsx/-@1},l=15489 | importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589e9ed2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000002db:[<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[]","memoryvalue":"(2)0x100000002db:[<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:08:55.526413+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:08:55.529265+0000 b77976 20=19+1)","memory_value.dirstat":"f(v0 m2024-04-24T09:08:55.526413+0000 19=19+0)","memory_value.rstat":"n(v0 rc2024-04-24T09:08:55.529265+0000 b77976 20=19+1)","error_str":""},"return_code":0} 2024-04-24T09:09:29.068 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.575+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000340 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/parse-extent-tree.awk auth v440 ap=1 s=4368 n(v0 rc2024-04-24T09:08:56.234384+0000 b4368 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589e9ed0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000340:[<0x100000002d8/parse-extent-tree.awk v208>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000340:[<0x100000002d8/parse-extent-tree.awk v440>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.068 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.575+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000360 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/statx.h auth v452 ap=1 s=6142 n(v0 rc2024-04-24T09:08:56.557824+0000 b6142 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0f4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000360:[<0x100000002d8/statx.h v272>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000360:[<0x100000002d8/statx.h v452>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.068 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.576+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000353 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/renameat2.c auth v444 ap=1 s=2226 n(v0 rc2024-04-24T09:08:56.425076+0000 b2226 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589e9ed3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000353:[<0x100000002d8/renameat2.c v246>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000353:[<0x100000002d8/renameat2.c v444>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.068 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.577+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000367 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_offset2.c auth v453 ap=1 s=4908 n(v0 rc2024-04-24T09:08:56.624000+0000 b4908 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0f5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000367:[<0x100000002d8/t_dir_offset2.c v286>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000367:[<0x100000002d8/t_dir_offset2.c v453>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.068 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.577+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000037b [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_mtab.c auth v454 ap=1 s=7461 n(v0 rc2024-04-24T09:08:56.809895+0000 b7461 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0f6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000037b:[<0x100000002d8/t_mtab.c v326>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000037b:[<0x100000002d8/t_mtab.c v454>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.068 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:28.647+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a3 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/Makefile auth v1187 ap=1 s=455 n(v0 rc2024-04-24T09:08:57.960881+0000 b455 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe09180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a3:[<0x100000003a2/Makefile v2>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003a3:[<0x100000003a2/Makefile v1187>,<0x10000000007/tests v1645>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.069 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:28.648+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a4 [...4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/ auth{0=1} fragtree_t(*^3 001*^3 010*^3 011*^3 100*^3) v3934 ap=1 f(v0 m2024-04-24T09:09:09.160637+0000 639=639+0) n(v0 rc2024-04-24T09:09:09.167321+0000 b1232532 640=639+1) old_inodes=1 (inest mix) (ifile excl) caps={15489=pAsLsXsFsx/-@312},l=15489 | lock=0 importingcaps=0 dirfrag=36 caps=1 dirtyrstat=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55a84fe09700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000003a4:[<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[]","memoryvalue":"(2)0x100000003a4:[<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1645>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:09:09.160637+0000 639=639+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:09:09.167321+0000 b1232532 640=639+1)","memory_value.dirstat":"f(v0 m2024-04-24T09:09:09.160637+0000 639=639+0)","memory_value.rstat":"n(v0 rc2024-04-24T09:09:09.167321+0000 b1232532 640=639+1)","error_str":""},"return_code":0} 2024-04-24T09:09:29.070 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.578+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000323 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/helpers.h auth v436 ap=1 s=899 n(v0 rc2024-04-24T09:08:55.963593+0000 b899 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0f5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000323:[<0x100000002d8/helpers.h v150>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000323:[<0x100000002d8/helpers.h v436>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.070 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.579+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000030c [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fake-dump-rootino.c auth v428 ap=1 s=7697 n(v0 rc2024-04-24T09:08:55.743795+0000 b7697 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0f7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000030c:[<0x100000002d8/fake-dump-rootino.c v104>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000030c:[<0x100000002d8/fake-dump-rootino.c v428>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.070 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.581+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000321 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/global.h auth v434 ap=1 s=2827 n(v0 rc2024-04-24T09:08:55.947062+0000 b2827 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0fc000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000321:[<0x100000002d8/global.h v146>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000321:[<0x100000002d8/global.h v434>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.582+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000034b [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/popdir.pl auth v442 ap=1 s=1996 n(v0 rc2024-04-24T09:08:56.342252+0000 b1996 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589e9ed2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000034b:[<0x100000002d8/popdir.pl v230>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000034b:[<0x100000002d8/popdir.pl v442>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.582+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002f4 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/attr_replace_test.c auth v464 ap=1 s=1978 n(v0 rc2024-04-24T09:08:55.559277+0000 b1978 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0b6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002f4:[<0x100000002d8/attr_replace_test.c v56>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002f4:[<0x100000002d8/attr_replace_test.c v464>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.582+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000311 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fiexchange.h auth v432 ap=1 s=3455 n(v0 rc2024-04-24T09:08:55.788515+0000 b3455 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0fcb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000311:[<0x100000002d8/fiexchange.h v114>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000311:[<0x100000002d8/fiexchange.h v432>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.583+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000035f [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/stat_test.c auth v504 ap=1 s=21149 n(v0 rc2024-04-24T09:08:56.546503+0000 b21149 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0b7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000035f:[<0x100000002d8/stat_test.c v270>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000035f:[<0x100000002d8/stat_test.c v504>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.583+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002ff [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/detached_mounts_propagation.c auth v468 ap=1 s=4905 n(v0 rc2024-04-24T09:08:55.631868+0000 b4905 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0b8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002ff:[<0x100000002d8/detached_mounts_propagation.c v78>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002ff:[<0x100000002d8/detached_mounts_propagation.c v468>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.585+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000351 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/readdir-while-renames.c auth v492 ap=1 s=3763 n(v0 rc2024-04-24T09:08:56.405668+0000 b3763 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0beb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000351:[<0x100000002d8/readdir-while-renames.c v242>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000351:[<0x100000002d8/readdir-while-renames.c v492>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.586+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000033a [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mmapcat.c auth v486 ap=1 s=559 n(v0 rc2024-04-24T09:08:56.179532+0000 b559 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0be000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000033a:[<0x100000002d8/mmapcat.c v196>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000033a:[<0x100000002d8/mmapcat.c v486>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.588+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000033b [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/multi_open_unlink.c auth v488 ap=1 s=2751 n(v0 rc2024-04-24T09:08:56.188418+0000 b2751 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0b9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000033b:[<0x100000002d8/multi_open_unlink.c v198>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000033b:[<0x100000002d8/multi_open_unlink.c v488>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.589+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000325 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/holetest.c auth v476 ap=1 s=9160 n(v0 rc2024-04-24T09:08:55.978266+0000 b9160 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0b7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000325:[<0x100000002d8/holetest.c v154>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000325:[<0x100000002d8/holetest.c v476>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.591+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000349 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/permname.c auth v490 ap=1 s=4501 n(v0 rc2024-04-24T09:08:56.323811+0000 b4501 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0bf600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000349:[<0x100000002d8/permname.c v226>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000349:[<0x100000002d8/permname.c v490>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.592+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000326 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/iopat.c auth v478 ap=1 s=870 n(v0 rc2024-04-24T09:08:55.985924+0000 b870 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0c0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000326:[<0x100000002d8/iopat.c v156>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000326:[<0x100000002d8/iopat.c v478>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.593+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000308 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/dumpfile auth v470 ap=1 s=27200 n(v0 rc2024-04-24T09:08:55.708816+0000 b27200 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0c0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000308:[<0x100000002d8/dumpfile v96>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000308:[<0x100000002d8/dumpfile v470>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.071 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.594+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000390 [...4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/ auth v535 ap=1 f(v0 m2024-04-24T09:08:57.904106+0000 13=13+0) n(v0 rc2024-04-24T09:08:57.909396+0000 b462889 14=13+1) old_inodes=1 (inest lock) (ifile excl) caps={15489=pAsLsXsFsx/-@1},l=15489 | lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0c1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000390:[<0x100000002d8/vfs v392>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[]","memoryvalue":"(2)0x10000000390:[<0x100000002d8/vfs v535>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:08:57.904106+0000 13=13+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:08:57.909396+0000 b462889 14=13+1)","memory_value.dirstat":"f(v0 m2024-04-24T09:08:57.904106+0000 13=13+0)","memory_value.rstat":"n(v0 rc2024-04-24T09:08:57.909396+0000 b462889 14=13+1)","error_str":""},"return_code":0} 2024-04-24T09:09:29.072 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.595+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a1 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/xfsfind.c auth v537 ap=1 s=5428 n(v0 rc2024-04-24T09:08:57.945702+0000 b5428 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0c1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a1:[<0x100000002d8/xfsfind.c v394>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003a1:[<0x100000002d8/xfsfind.c v537>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.072 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.596+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000313 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fill2.c auth v474 ap=1 s=8932 n(v0 rc2024-04-24T09:08:55.805342+0000 b8932 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0c6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000313:[<0x100000002d8/fill2.c v118>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000313:[<0x100000002d8/fill2.c v474>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.072 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.597+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000312 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fill.c auth v472 ap=1 s=1481 n(v0 rc2024-04-24T09:08:55.796143+0000 b1481 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0c7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000312:[<0x100000002d8/fill.c v116>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000312:[<0x100000002d8/fill.c v472>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.072 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.598+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000329 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/locktest.c auth v480 ap=1 s=46475 n(v0 rc2024-04-24T09:08:56.012940+0000 b46475 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0c7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000329:[<0x100000002d8/locktest.c v162>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000329:[<0x100000002d8/locktest.c v480>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.072 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.598+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000337 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mkswap.c auth v482 ap=1 s=1460 n(v0 rc2024-04-24T09:08:56.151387+0000 b1460 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0c8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000337:[<0x100000002d8/mkswap.c v190>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000337:[<0x100000002d8/mkswap.c v482>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.072 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.598+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002fe [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/deduperace.c auth v466 ap=1 s=7666 n(v0 rc2024-04-24T09:08:55.623282+0000 b7666 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0c9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002fe:[<0x100000002d8/deduperace.c v76>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002fe:[<0x100000002d8/deduperace.c v466>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.072 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.599+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000370 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_holes.c auth v507 ap=1 s=650 n(v0 rc2024-04-24T09:08:56.705371+0000 b650 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0ceb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000370:[<0x100000002d8/t_holes.c v304>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000370:[<0x100000002d8/t_holes.c v507>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.072 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.599+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000368 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_type.c auth v506 ap=1 s=1906 n(v0 rc2024-04-24T09:08:56.633256+0000 b1906 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0cf600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000368:[<0x100000002d8/t_dir_type.c v288>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000368:[<0x100000002d8/t_dir_type.c v506>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.072 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.599+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000037a [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_mmap_writev_overlap.c auth v508 ap=1 s=3243 n(v0 rc2024-04-24T09:08:56.800753+0000 b3243 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0ce000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000037a:[<0x100000002d8/t_mmap_writev_overlap.c v324>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000037a:[<0x100000002d8/t_mmap_writev_overlap.c v508>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.072 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.600+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000366 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_offset.c auth v505 ap=1 s=1104 n(v0 rc2024-04-24T09:08:56.615609+0000 b1104 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0d0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000366:[<0x100000002d8/t_dir_offset.c v284>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000366:[<0x100000002d8/t_dir_offset.c v505>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.072 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.600+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000338 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mmap-rw-fault.c auth v484 ap=1 s=4652 n(v0 rc2024-04-24T09:08:56.160616+0000 b4652 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0d0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000338:[<0x100000002d8/mmap-rw-fault.c v192>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000338:[<0x100000002d8/mmap-rw-fault.c v484>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.072 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.601+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e0 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-cow-race.c auth v48 ap=1 s=2311 n(v0 rc2024-04-24T09:08:55.455288+0000 b2311 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea0fd080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e0:[<0x100000002db/aio-dio-cow-race.c v48>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002e0:[<0x100000002db/aio-dio-cow-race.c v48>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.073 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.601+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002ed [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aiocp.c auth v74 ap=1 s=11153 n(v0 rc2024-04-24T09:08:55.524277+0000 b11153 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0fdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002ed:[<0x100000002db/aiocp.c v36>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002ed:[<0x100000002db/aiocp.c v74>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.076 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.602+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e2 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-eof-race.c auth v52 ap=1 s=4816 n(v0 rc2024-04-24T09:08:55.468606+0000 b4816 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea0fe680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e2:[<0x100000002db/aio-dio-eof-race.c v52>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002e2:[<0x100000002db/aio-dio-eof-race.c v52>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.076 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.602+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e5 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-hole-filling-race.c auth v58 ap=1 s=2278 n(v0 rc2024-04-24T09:08:55.482541+0000 b2278 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea0fd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e5:[<0x100000002db/aio-dio-hole-filling-race.c v58>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002e5:[<0x100000002db/aio-dio-hole-filling-race.c v58>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.076 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.602+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002ee [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aiodio_sparse2.c auth v76 ap=1 s=9491 n(v0 rc2024-04-24T09:08:55.529265+0000 b9491 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0fe100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002ee:[<0x100000002db/aiodio_sparse2.c v38>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002ee:[<0x100000002db/aiodio_sparse2.c v76>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.077 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.603+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002dc [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/Makefile auth v40 ap=1 s=514 n(v0 rc2024-04-24T09:08:55.421761+0000 b514 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea0fec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002dc:[<0x100000002db/Makefile v40>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002dc:[<0x100000002db/Makefile v40>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.077 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.603+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002ea [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-free-ring-with-bogus-nr-pages.c auth v68 ap=1 s=1982 n(v0 rc2024-04-24T09:08:55.508271+0000 b1982 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0ff180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002ea:[<0x100000002db/aio-free-ring-with-bogus-nr-pages.c v30>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002ea:[<0x100000002db/aio-free-ring-with-bogus-nr-pages.c v68>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.077 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.603+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002ec [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-last-ref-held-by-io.c auth v72 ap=1 s=5255 n(v0 rc2024-04-24T09:08:55.518468+0000 b5255 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0ff700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002ec:[<0x100000002db/aio-last-ref-held-by-io.c v34>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002ec:[<0x100000002db/aio-last-ref-held-by-io.c v72>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.077 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.604+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e8 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-subblock-eof-read.c auth v64 ap=1 s=2715 n(v0 rc2024-04-24T09:08:55.498039+0000 b2715 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea106000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e8:[<0x100000002db/aio-dio-subblock-eof-read.c v26>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002e8:[<0x100000002db/aio-dio-subblock-eof-read.c v64>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.077 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.604+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e9 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-write-verify.c auth v66 ap=1 s=7738 n(v0 rc2024-04-24T09:08:55.502900+0000 b7738 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea107600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e9:[<0x100000002db/aio-dio-write-verify.c v28>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002e9:[<0x100000002db/aio-dio-write-verify.c v66>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.077 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.604+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e4 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-fcntl-race.c auth v56 ap=1 s=3150 n(v0 rc2024-04-24T09:08:55.478835+0000 b3150 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea106580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e4:[<0x100000002db/aio-dio-fcntl-race.c v56>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002e4:[<0x100000002db/aio-dio-fcntl-race.c v56>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.077 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.605+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002eb [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-io-setup-with-nonwritable-context-pointer.c auth v70 ap=1 s=849 n(v0 rc2024-04-24T09:08:55.513694+0000 b849 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea106b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002eb:[<0x100000002db/aio-io-setup-with-nonwritable-context-pointer.c v32>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002eb:[<0x100000002db/aio-io-setup-with-nonwritable-context-pointer.c v70>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.077 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.605+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002df [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-append-write-read-race.c auth v46 ap=1 s=4485 n(v0 rc2024-04-24T09:08:55.446160+0000 b4485 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea107080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002df:[<0x100000002db/aio-dio-append-write-read-race.c v46>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002df:[<0x100000002db/aio-dio-append-write-read-race.c v46>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.077 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.605+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e1 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-cycle-write.c auth v50 ap=1 s=5257 n(v0 rc2024-04-24T09:08:55.463234+0000 b5257 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea107b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e1:[<0x100000002db/aio-dio-cycle-write.c v50>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002e1:[<0x100000002db/aio-dio-cycle-write.c v50>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.077 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.605+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e3 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-extend-stat.c auth v54 ap=1 s=3901 n(v0 rc2024-04-24T09:08:55.473769+0000 b3901 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea108100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e3:[<0x100000002db/aio-dio-extend-stat.c v54>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002e3:[<0x100000002db/aio-dio-extend-stat.c v54>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.077 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.606+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e6 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-invalidate-failure.c auth v60 ap=1 s=3929 n(v0 rc2024-04-24T09:08:55.486933+0000 b3929 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea108680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e6:[<0x100000002db/aio-dio-invalidate-failure.c v60>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002e6:[<0x100000002db/aio-dio-invalidate-failure.c v60>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.077 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.606+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002dd [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/SOURCE auth v42 ap=1 s=282 n(v0 rc2024-04-24T09:08:55.430745+0000 b282 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea108c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002dd:[<0x100000002db/SOURCE v42>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002dd:[<0x100000002db/SOURCE v42>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.078 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.607+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002e7 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-invalidate-readahead.c auth v62 ap=1 s=4134 n(v0 rc2024-04-24T09:08:55.492498+0000 b4134 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea109180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002e7:[<0x100000002db/aio-dio-invalidate-readahead.c v62>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002e7:[<0x100000002db/aio-dio-invalidate-readahead.c v62>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.078 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.607+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002de [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-append-write-fallocate-race.c auth v44 ap=1 s=3736 n(v0 rc2024-04-24T09:08:55.437581+0000 b3736 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea109700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002de:[<0x100000002db/aio-dio-append-write-fallocate-race.c v44>,<0x100000002d8/aio-dio-regress v388>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000002de:[<0x100000002db/aio-dio-append-write-fallocate-race.c v44>,<0x100000002d8/aio-dio-regress v402>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.078 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.608+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023f [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/index.lock auth v250 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15489=0-4194304@2} caps={15489=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15489 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea116580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000023f:[<0x10000000008/index.lock v250>,<0x10000000007/.git v818>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000023f:[<0x10000000008/index.lock v250>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.078 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.608+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000236 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/HEAD auth v263 ap=1 s=23 n(v0 rc2024-04-24T09:08:53.715417+0000 b23 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea116b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000236:[<0x10000000008/HEAD v263>,<0x10000000007/.git v818>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000236:[<0x10000000008/HEAD v263>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.078 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.608+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000009 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/description auth v147 ap=1 s=73 n(v0 rc2024-04-24T09:08:45.048616+0000 b73 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea117080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000009:[<0x10000000008/description v2>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[3]","memoryvalue":"(4)0x10000000009:[<0x10000000008/description v147>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.078 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.610+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021f [...3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects/ auth{0=1,1=1} v269 ap=1 f(v0 m2024-04-24T09:08:45.654807+0000 2=0+2) n(v1 rc2024-04-24T09:08:53.974287+0000 b13437509 6=3+3) (inest mix) caps={15489=pAsLsXs/-@1},l=15489 | lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 dirwaiter=0 replicated=1 dirty=1 authpin=1 discoverbase=0 scrubqueue=0 0x5589e9f30680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000021f:[<0x10000000008/objects v105>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[]","memoryvalue":"(2)0x1000000021f:[<0x10000000008/objects v269>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:08:45.654807+0000 2=0+2)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:08:53.974287+0000 b13437509 6=3+3)","memory_value.dirstat":"f(v0 m2024-04-24T09:08:45.654807+0000 2=0+2)","memory_value.rstat":"n(v1 rc2024-04-24T09:08:53.974287+0000 b13437509 6=3+3)","error_str":""},"return_code":0} 2024-04-24T09:09:29.078 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.610+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000231 [...4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/ auth v275 ap=1 f(v0 m2024-04-24T09:08:53.830338+0000 2=1+1) n(v2 rc2024-04-24T09:08:53.867683+0000 b588 8=3+5) old_inodes=1 (inest lock) (ifile excl) caps={15489=pAsLsXsFsx/-@1},l=15489 | importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea117600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000231:[<0x10000000008/logs v261>,<0x10000000007/.git v818>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[]","memoryvalue":"(2)0x10000000231:[<0x10000000008/logs v275>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:08:53.830338+0000 2=1+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:08:53.867683+0000 b588 8=3+5)","memory_value.dirstat":"f(v0 m2024-04-24T09:08:53.830338+0000 2=1+1)","memory_value.rstat":"n(v2 rc2024-04-24T09:08:53.867683+0000 b588 8=3+5)","error_str":""},"return_code":0} 2024-04-24T09:09:29.078 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.611+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023e [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/config auth v265 ap=1 s=262 n(v0 rc2024-04-24T09:08:53.963523+0000 b262 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea118100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000023e:[<0x10000000008/config v265>,<0x10000000007/.git v818>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000023e:[<0x10000000008/config v265>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.078 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.611+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000203 [...3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/info/ auth{0=1,1=1} v149 ap=1 f(v0 m2024-04-24T09:08:45.066792+0000 1=1+0) n(v0 rc2024-04-24T09:08:45.073803+0000 b240 2=1+1) (inest mix) caps={15489=pAsLsXs/-@1},l=15489 | lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 dirwaiter=0 replicated=1 authpin=1 discoverbase=0 scrubqueue=0 0x5589e9fc8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000203:[<0x10000000008/info v6>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[]","memoryvalue":"(2)0x10000000203:[<0x10000000008/info v149>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:08:45.066792+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:08:45.073803+0000 b240 2=1+1)","memory_value.dirstat":"f(v0 m2024-04-24T09:08:45.066792+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-24T09:08:45.073803+0000 b240 2=1+1)","error_str":""},"return_code":0} 2024-04-24T09:09:29.078 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.611+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [...4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs/ auth{0=1,1=1} v277 ap=1 f(v0 m2024-04-24T09:08:53.629419+0000 3=0+3) n(v1 rc2024-04-24T09:08:53.872855+0000 b73 7=2+5) old_inodes=1 (inest mix) caps={15489=pAsLsXs/-@1},l=15489 | dirtyscattered=0 lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 dirtyrstat=0 dirwaiter=0 replicated=1 dirty=1 authpin=1 discoverbase=0 scrubqueue=0 0x5589e9f30c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000215:[<0x10000000008/refs v44>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[]","memoryvalue":"(2)0x10000000215:[<0x10000000008/refs v277>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:08:53.629419+0000 3=0+3)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:08:53.872855+0000 b73 7=2+5)","memory_value.dirstat":"f(v0 m2024-04-24T09:08:53.629419+0000 3=0+3)","memory_value.rstat":"n(v1 rc2024-04-24T09:08:53.872855+0000 b73 7=2+5)","error_str":""},"return_code":0} 2024-04-24T09:09:29.078 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.612+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022d [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/packed-refs auth v255 ap=1 s=6242 n(v0 rc2024-04-24T09:08:53.606182+0000 b6242 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea117b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022d:[<0x10000000008/packed-refs v255>,<0x10000000007/.git v818>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000022d:[<0x10000000008/packed-refs v255>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.078 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.612+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000022b [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/shallow auth v259 ap=1 s=41 n(v0 rc2024-04-24T09:08:52.897593+0000 b41 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea118680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000022b:[<0x10000000008/shallow v259>,<0x10000000007/.git v818>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000022b:[<0x10000000008/shallow v259>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.079 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.613+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000205 [...3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/branches/ auth v8 ap=1 f() n(v0 1=0+1) (inest mix) caps={15489=pAsLsXs/-@1},l=15489 | request=0 lock=0 importingcaps=0 importing=0 dirfrag=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589e9ffcb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000205:[<0x10000000008/branches v8>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[]","memoryvalue":"(2)0x10000000205:[<0x10000000008/branches v8>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":0} 2024-04-24T09:09:29.079 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.613+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000039c [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/vfstest.c auth v76 ap=1 s=66253 n(v0 rc2024-04-24T09:08:57.899163+0000 b66253 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0d1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000039c:[<0x10000000390/vfstest.c v24>,<0x100000002d8/vfs v392>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000039c:[<0x10000000390/vfstest.c v76>,<0x100000002d8/vfs v535>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.079 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.613+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000397 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/mount-idmapped.c auth v66 ap=1 s=5163 n(v0 rc2024-04-24T09:08:57.841084+0000 b5163 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0da000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000397:[<0x10000000390/mount-idmapped.c v14>,<0x100000002d8/vfs v392>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000397:[<0x10000000390/mount-idmapped.c v66>,<0x100000002d8/vfs v535>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.079 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.614+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000391 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/Makefile auth v54 ap=1 s=1052 n(v0 rc2024-04-24T09:08:57.751238+0000 b1052 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0dab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000391:[<0x10000000390/Makefile v2>,<0x100000002d8/vfs v392>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000391:[<0x10000000390/Makefile v54>,<0x100000002d8/vfs v535>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.079 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.614+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000398 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/tmpfs-idmapped-mounts.c auth v68 ap=1 s=13890 n(v0 rc2024-04-24T09:08:57.851751+0000 b13890 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0d1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000398:[<0x10000000390/tmpfs-idmapped-mounts.c v16>,<0x100000002d8/vfs v392>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000398:[<0x10000000390/tmpfs-idmapped-mounts.c v68>,<0x100000002d8/vfs v535>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.079 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.614+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000039d [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/vfstest.h auth v78 ap=1 s=198 n(v0 rc2024-04-24T09:08:57.909396+0000 b198 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0da580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000039d:[<0x10000000390/vfstest.h v26>,<0x100000002d8/vfs v392>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000039d:[<0x10000000390/vfstest.h v78>,<0x100000002d8/vfs v535>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.079 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.615+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000394 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/idmapped-mounts.c auth v60 ap=1 s=240071 n(v0 rc2024-04-24T09:08:57.807960+0000 b240071 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0db080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000394:[<0x10000000390/idmapped-mounts.c v8>,<0x100000002d8/vfs v392>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000394:[<0x10000000390/idmapped-mounts.c v60>,<0x100000002d8/vfs v535>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.079 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.615+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000395 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/idmapped-mounts.h auth v62 ap=1 s=3211 n(v0 rc2024-04-24T09:08:57.819233+0000 b3211 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0dbb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000395:[<0x10000000390/idmapped-mounts.h v10>,<0x100000002d8/vfs v392>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000395:[<0x10000000390/idmapped-mounts.h v62>,<0x100000002d8/vfs v535>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.079 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.615+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000396 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/missing.h auth v64 ap=1 s=3677 n(v0 rc2024-04-24T09:08:57.830048+0000 b3677 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0db600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000396:[<0x10000000390/missing.h v12>,<0x100000002d8/vfs v392>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000396:[<0x10000000390/missing.h v64>,<0x100000002d8/vfs v535>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.079 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.615+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000399 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/tmpfs-idmapped-mounts.h auth v70 ap=1 s=273 n(v0 rc2024-04-24T09:08:57.862118+0000 b273 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0dc100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000399:[<0x10000000390/tmpfs-idmapped-mounts.h v18>,<0x100000002d8/vfs v392>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000399:[<0x10000000390/tmpfs-idmapped-mounts.h v70>,<0x100000002d8/vfs v535>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.079 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.616+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000393 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/btrfs-idmapped-mounts.h auth v58 ap=1 s=274 n(v0 rc2024-04-24T09:08:57.781876+0000 b274 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0dd180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000393:[<0x10000000390/btrfs-idmapped-mounts.h v6>,<0x100000002d8/vfs v392>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000393:[<0x10000000390/btrfs-idmapped-mounts.h v58>,<0x100000002d8/vfs v535>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.079 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.616+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000392 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/btrfs-idmapped-mounts.c auth v56 ap=1 s=96274 n(v0 rc2024-04-24T09:08:57.771358+0000 b96274 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0dcc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000392:[<0x10000000390/btrfs-idmapped-mounts.c v4>,<0x100000002d8/vfs v392>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000392:[<0x10000000390/btrfs-idmapped-mounts.c v56>,<0x100000002d8/vfs v535>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.079 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.616+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000039a [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/utils.c auth v72 ap=1 s=22310 n(v0 rc2024-04-24T09:08:57.873047+0000 b22310 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0dc680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000039a:[<0x10000000390/utils.c v20>,<0x100000002d8/vfs v392>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000039a:[<0x10000000390/utils.c v72>,<0x100000002d8/vfs v535>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.080 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.616+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000039b [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/utils.h auth v74 ap=1 s=10243 n(v0 rc2024-04-24T09:08:57.883210+0000 b10243 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea0dd700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000039b:[<0x10000000390/utils.h v22>,<0x100000002d8/vfs v392>,<0x10000000007/src v1096>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000039b:[<0x10000000390/utils.h v74>,<0x100000002d8/vfs v535>,<0x10000000007/src v1629>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.080 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.617+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000280 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/repair auth v180 ap=1 s=4565 n(v0 rc2024-04-24T09:08:54.716410+0000 b4565 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589e9ffd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000280:[<0x10000000257/repair v180>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000280:[<0x10000000257/repair v180>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.080 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.618+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000026e [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filter.btrfs auth v144 ap=1 s=5430 n(v0 rc2024-04-24T09:08:54.538833+0000 b5430 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589e9ffdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000026e:[<0x10000000257/filter.btrfs v144>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000026e:[<0x10000000257/filter.btrfs v144>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.080 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.618+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000265 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmlogwrites auth v126 ap=1 s=5995 n(v0 rc2024-04-24T09:08:54.449474+0000 b5995 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589e9ffe680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000265:[<0x10000000257/dmlogwrites v126>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000265:[<0x10000000257/dmlogwrites v126>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.080 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.618+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000281 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/report auth v182 ap=1 s=8402 n(v0 rc2024-04-24T09:08:54.725814+0000 b8402 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589e9ffe100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000281:[<0x10000000257/report v182>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000281:[<0x10000000257/report v182>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.080 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.618+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000025b [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/casefold auth v106 ap=1 s=2067 n(v0 rc2024-04-24T09:08:54.352027+0000 b2067 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589e9ffec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000025b:[<0x10000000257/casefold v106>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000025b:[<0x10000000257/casefold v106>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.080 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.619+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000278 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/perf auth v164 ap=1 s=937 n(v0 rc2024-04-24T09:08:54.631061+0000 b937 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589e9fff180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000278:[<0x10000000257/perf v164>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000278:[<0x10000000257/perf v164>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.080 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.619+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000026b [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/fail_make_request auth v138 ap=1 s=1907 n(v0 rc2024-04-24T09:08:54.508084+0000 b1907 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589e9fff700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000026b:[<0x10000000257/fail_make_request v138>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000026b:[<0x10000000257/fail_make_request v138>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.080 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.619+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000262 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmerror auth v120 ap=1 s=10761 n(v0 rc2024-04-24T09:08:54.419639+0000 b10761 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea05c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000262:[<0x10000000257/dmerror v120>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000262:[<0x10000000257/dmerror v120>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.080 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.620+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000270 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/gcov auth v148 ap=1 s=2881 n(v0 rc2024-04-24T09:08:54.559170+0000 b2881 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea05cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000270:[<0x10000000257/gcov v148>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000270:[<0x10000000257/gcov v148>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.080 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.620+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000282 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/scsi_debug auth v184 ap=1 s=2176 n(v0 rc2024-04-24T09:08:54.734070+0000 b2176 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea05c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000282:[<0x10000000257/scsi_debug v184>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000282:[<0x10000000257/scsi_debug v184>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.081 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.620+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000287 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/xfs auth v194 ap=1 s=53397 n(v0 rc2024-04-24T09:08:54.789445+0000 b53397 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea05db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000287:[<0x10000000257/xfs v194>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000287:[<0x10000000257/xfs v194>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.081 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.621+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000286 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/verity auth v192 ap=1 s=12253 n(v0 rc2024-04-24T09:08:54.775498+0000 b12253 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea05d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000286:[<0x10000000257/verity v192>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000286:[<0x10000000257/verity v192>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.081 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.622+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000026c [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filestreams auth v140 ap=1 s=4995 n(v0 rc2024-04-24T09:08:54.517931+0000 b4995 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea05e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000026c:[<0x10000000257/filestreams v140>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000026c:[<0x10000000257/filestreams v140>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.081 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.622+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000269 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ext4 auth v134 ap=1 s=5862 n(v0 rc2024-04-24T09:08:54.490527+0000 b5862 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea05ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000269:[<0x10000000257/ext4 v134>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000269:[<0x10000000257/ext4 v134>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.082 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.622+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000268 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/encrypt auth v132 ap=1 s=31115 n(v0 rc2024-04-24T09:08:54.481569+0000 b31115 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea05f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000268:[<0x10000000257/encrypt v132>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000268:[<0x10000000257/encrypt v132>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.082 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.623+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000027a [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/preamble auth v168 ap=1 s=1327 n(v0 rc2024-04-24T09:08:54.650512+0000 b1327 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea05f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000027a:[<0x10000000257/preamble v168>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000027a:[<0x10000000257/preamble v168>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.082 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.624+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000264 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmhugedisk auth v124 ap=1 s=1098 n(v0 rc2024-04-24T09:08:54.439737+0000 b1098 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea05d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000264:[<0x10000000257/dmhugedisk v124>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000264:[<0x10000000257/dmhugedisk v124>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.082 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.624+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000025e [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/config auth v112 ap=1 s=29481 n(v0 rc2024-04-24T09:08:54.383324+0000 b29481 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea064000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000025e:[<0x10000000257/config v112>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000025e:[<0x10000000257/config v112>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.082 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.624+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000272 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/locktest auth v152 ap=1 s=1856 n(v0 rc2024-04-24T09:08:54.576490+0000 b1856 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea05e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000272:[<0x10000000257/locktest v152>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000272:[<0x10000000257/locktest v152>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.082 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.624+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000027d [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/rc auth v174 ap=1 s=133528 n(v0 rc2024-04-24T09:08:54.687923+0000 b133528 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea064580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000027d:[<0x10000000257/rc v174>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000027d:[<0x10000000257/rc v174>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.083 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.625+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000025d [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/cgroup2 auth v110 ap=1 s=493 n(v0 rc2024-04-24T09:08:54.371733+0000 b493 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea065080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000025d:[<0x10000000257/cgroup2 v110>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000025d:[<0x10000000257/cgroup2 v110>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.083 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.625+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000267 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dump auth v130 ap=1 s=36169 n(v0 rc2024-04-24T09:08:54.471178+0000 b36169 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea064b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000267:[<0x10000000257/dump v130>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000267:[<0x10000000257/dump v130>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.083 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.625+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000288 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/zoned auth v196 ap=1 s=1090 n(v0 rc2024-04-24T09:08:54.798807+0000 b1090 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea065600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000288:[<0x10000000257/zoned v196>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000288:[<0x10000000257/zoned v196>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.083 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.626+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000027f [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/renameat2 auth v178 ap=1 s=2520 n(v0 rc2024-04-24T09:08:54.706905+0000 b2520 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea065b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000027f:[<0x10000000257/renameat2 v178>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000027f:[<0x10000000257/renameat2 v178>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.083 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.626+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000025c [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ceph auth v108 ap=1 s=986 n(v0 rc2024-04-24T09:08:54.361854+0000 b986 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea066c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000025c:[<0x10000000257/ceph v108>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000025c:[<0x10000000257/ceph v108>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.083 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.626+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000258 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/Makefile auth v100 ap=1 s=305 n(v0 rc2024-04-24T09:08:54.320841+0000 b305 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea066100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000258:[<0x10000000257/Makefile v100>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000258:[<0x10000000257/Makefile v100>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.083 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.626+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000274 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/metadump auth v156 ap=1 s=4321 n(v0 rc2024-04-24T09:08:54.595939+0000 b4321 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea066680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000274:[<0x10000000257/metadump v156>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000274:[<0x10000000257/metadump v156>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.083 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.627+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000027b [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/punch auth v170 ap=1 s=15181 n(v0 rc2024-04-24T09:08:54.658903+0000 b15181 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea067180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000027b:[<0x10000000257/punch v170>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000027b:[<0x10000000257/punch v170>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.086 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.627+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000263 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmflakey auth v122 ap=1 s=6034 n(v0 rc2024-04-24T09:08:54.429218+0000 b6034 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea07e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000263:[<0x10000000257/dmflakey v122>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000263:[<0x10000000257/dmflakey v122>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.086 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.628+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000276 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/nfs auth v160 ap=1 s=794 n(v0 rc2024-04-24T09:08:54.613594+0000 b794 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea067700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000276:[<0x10000000257/nfs v160>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000276:[<0x10000000257/nfs v160>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.086 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.628+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000285 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ubifs auth v190 ap=1 s=145 n(v0 rc2024-04-24T09:08:54.764663+0000 b145 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea07e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000285:[<0x10000000257/ubifs v190>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000285:[<0x10000000257/ubifs v190>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.086 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.628+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000026d [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filter auth v142 ap=1 s=18600 n(v0 rc2024-04-24T09:08:54.528719+0000 b18600 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea07eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000026d:[<0x10000000257/filter v142>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000026d:[<0x10000000257/filter v142>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.086 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.629+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000260 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmdelay auth v116 ap=1 s=1776 n(v0 rc2024-04-24T09:08:54.400675+0000 b1776 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea07f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000260:[<0x10000000257/dmdelay v116>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000260:[<0x10000000257/dmdelay v116>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.086 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.629+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000275 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/module auth v158 ap=1 s=6439 n(v0 rc2024-04-24T09:08:54.604809+0000 b6439 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea07f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000275:[<0x10000000257/module v158>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000275:[<0x10000000257/module v158>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.087 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.629+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000025a [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/btrfs auth v104 ap=1 s=23632 n(v0 rc2024-04-24T09:08:54.341749+0000 b23632 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea07fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000025a:[<0x10000000257/btrfs v104>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000025a:[<0x10000000257/btrfs v104>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.087 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.630+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000027e [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/reflink auth v176 ap=1 s=14649 n(v0 rc2024-04-24T09:08:54.698355+0000 b14649 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea080100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000027e:[<0x10000000257/reflink v176>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000027e:[<0x10000000257/reflink v176>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.087 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.630+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000026f [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/fuzzy auth v146 ap=1 s=42616 n(v0 rc2024-04-24T09:08:54.550190+0000 b42616 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea080680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000026f:[<0x10000000257/fuzzy v146>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000026f:[<0x10000000257/fuzzy v146>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.087 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.630+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000283 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/test_names auth v186 ap=1 s=312 n(v0 rc2024-04-24T09:08:54.744211+0000 b312 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea080c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000283:[<0x10000000257/test_names v186>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000283:[<0x10000000257/test_names v186>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.087 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.631+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000261 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmdust auth v118 ap=1 s=839 n(v0 rc2024-04-24T09:08:54.409548+0000 b839 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea081180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000261:[<0x10000000257/dmdust v118>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000261:[<0x10000000257/dmdust v118>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.087 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.631+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000259 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/attr auth v102 ap=1 s=6641 n(v0 rc2024-04-24T09:08:54.330274+0000 b6641 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea081700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000259:[<0x10000000257/attr v102>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000259:[<0x10000000257/attr v102>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.087 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.631+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000027c [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/quota auth v172 ap=1 s=13055 n(v0 rc2024-04-24T09:08:54.668557+0000 b13055 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea086580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000027c:[<0x10000000257/quota v172>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000027c:[<0x10000000257/quota v172>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.087 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.632+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000266 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmthin auth v128 ap=1 s=7964 n(v0 rc2024-04-24T09:08:54.458928+0000 b7964 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea086b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000266:[<0x10000000257/dmthin v128>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000266:[<0x10000000257/dmthin v128>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.087 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.632+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000271 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/inject auth v150 ap=1 s=2962 n(v0 rc2024-04-24T09:08:54.567892+0000 b2962 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea087b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000271:[<0x10000000257/inject v150>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000271:[<0x10000000257/inject v150>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.087 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.632+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000273 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/log auth v154 ap=1 s=16161 n(v0 rc2024-04-24T09:08:54.586267+0000 b16161 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea087600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000273:[<0x10000000257/log v154>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000273:[<0x10000000257/log v154>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.088 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.633+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000279 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/populate auth v166 ap=1 s=32678 n(v0 rc2024-04-24T09:08:54.641842+0000 b32678 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea088100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000279:[<0x10000000257/populate v166>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000279:[<0x10000000257/populate v166>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.088 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.633+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000277 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/overlay auth v162 ap=1 s=13439 n(v0 rc2024-04-24T09:08:54.622919+0000 b13439 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea087080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000277:[<0x10000000257/overlay v162>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000277:[<0x10000000257/overlay v162>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.088 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.633+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000026a [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/f2fs auth v136 ap=1 s=812 n(v0 rc2024-04-24T09:08:54.498868+0000 b812 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea088680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000026a:[<0x10000000257/f2fs v136>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000026a:[<0x10000000257/f2fs v136>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.088 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.634+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000221 [...3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects/info/ auth v4 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={15489=pAsLsXsFsx/AsLsXsFsx@2},l=15489 | importingcaps=0 dirfrag=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589e9fc8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000221:[<0x1000000021f/info v4>,<0x10000000008/objects v105>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[]","memoryvalue":"(2)0x10000000221:[<0x1000000021f/info v4>,<0x10000000008/objects v269>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":0} 2024-04-24T09:09:29.088 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.634+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000025f [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/defrag auth v114 ap=1 s=4131 n(v0 rc2024-04-24T09:08:54.391967+0000 b4131 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea089180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000025f:[<0x10000000257/defrag v114>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000025f:[<0x10000000257/defrag v114>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.088 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.634+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000284 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/tracing auth v188 ap=1 s=1830 n(v0 rc2024-04-24T09:08:54.754360+0000 b1830 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea088c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000284:[<0x10000000257/tracing v188>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000284:[<0x10000000257/tracing v188>,<0x10000000007/common v948>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.088 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.635+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000220 [...3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects/pack/ auth{1=1} v70 ap=1 f(v0 m2024-04-24T09:08:53.974287+0000 3=3+0) n(v0 rc2024-04-24T09:08:53.974287+0000 b13437509 4=3+1) (inest mix) (ifile excl) caps={15489=pAsLsXsFsx/-@1},l=15489 | lock=0 importingcaps=0 dirfrag=1 caps=1 dirwaiter=0 replicated=1 dirty=1 authpin=1 discoverbase=0 scrubqueue=0 0x5589e9fc8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000220:[<0x1000000021f/pack v2>,<0x10000000008/objects v105>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[]","memoryvalue":"(2)0x10000000220:[<0x1000000021f/pack v70>,<0x10000000008/objects v269>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:08:53.974287+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:08:53.974287+0000 b13437509 4=3+1)","memory_value.dirstat":"f(v0 m2024-04-24T09:08:53.974287+0000 3=3+0)","memory_value.rstat":"n(v0 rc2024-04-24T09:08:53.974287+0000 b13437509 4=3+1)","error_str":""},"return_code":0} 2024-04-24T09:09:29.088 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.635+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000239 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/HEAD auth v12 ap=1 s=196 n(v0 rc2024-04-24T09:08:53.834731+0000 b196 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea118c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000239:[<0x10000000231/HEAD v12>,<0x10000000008/logs v261>,<0x10000000007/.git v818>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000239:[<0x10000000231/HEAD v12>,<0x10000000008/logs v275>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.088 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.635+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000232 [...3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/ auth v18 ap=1 f(v0 m2024-04-24T09:08:53.856336+0000 2=0+2) n(v1 rc2024-04-24T09:08:53.867683+0000 b392 6=2+4) (inest lock) (ifile excl) caps={15489=pAsLsXsFsx/-@1},l=15489 | importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea119180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000232:[<0x10000000231/refs v10>,<0x10000000008/logs v261>,<0x10000000007/.git v818>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[]","memoryvalue":"(2)0x10000000232:[<0x10000000231/refs v18>,<0x10000000008/logs v275>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:08:53.856336+0000 2=0+2)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:08:53.867683+0000 b392 6=2+4)","memory_value.dirstat":"f(v0 m2024-04-24T09:08:53.856336+0000 2=0+2)","memory_value.rstat":"n(v1 rc2024-04-24T09:08:53.867683+0000 b392 6=2+4)","error_str":""},"return_code":0} 2024-04-24T09:09:29.089 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.636+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000204 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/info/exclude auth v4 ap=1 s=240 n(v0 rc2024-04-24T09:08:45.073803+0000 b240 1=1+0) caps={15489=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589e9ffc580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000204:[<0x10000000203/exclude v2>,<0x10000000008/info v6>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[3]","memoryvalue":"(4)0x10000000204:[<0x10000000203/exclude v4>,<0x10000000008/info v149>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.089 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.636+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000216 [...3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs/heads/ auth v18 ap=1 f(v0 m2024-04-24T09:08:53.872855+0000 1=1+0) n(v0 rc2024-04-24T09:08:53.872855+0000 b41 2=1+1) (inest lock) (ifile excl) caps={15489=pAsLsXsFsx/-@1},l=15489 | lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589e9fca680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000216:[<0x10000000215/heads v2>,<0x10000000008/refs v44>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[]","memoryvalue":"(2)0x10000000216:[<0x10000000215/heads v18>,<0x10000000008/refs v277>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:08:53.872855+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:08:53.872855+0000 b41 2=1+1)","memory_value.dirstat":"f(v0 m2024-04-24T09:08:53.872855+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-24T09:08:53.872855+0000 b41 2=1+1)","error_str":""},"return_code":0} 2024-04-24T09:09:29.089 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.637+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000217 [...3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs/tags/ auth v4 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) caps={15489=pAsLsXsFsx/AsLsXsFsx@2},l=15489 | importingcaps=0 dirfrag=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589e9fca100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000217:[<0x10000000215/tags v4>,<0x10000000008/refs v44>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[]","memoryvalue":"(2)0x10000000217:[<0x10000000215/tags v4>,<0x10000000008/refs v277>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":0} 2024-04-24T09:09:29.089 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.638+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023a [...3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/heads/ auth v12 ap=1 f(v0 m2024-04-24T09:08:53.863551+0000 1=1+0) n(v0 rc2024-04-24T09:08:53.867683+0000 b196 2=1+1) (inest lock) (ifile excl) caps={15489=pAsLsXsFsx/AsLsXsFsx@1},l=15489 | importingcaps=0 dirfrag=1 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea119700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000023a:[<0x10000000232/heads v12>,<0x10000000231/refs v10>,<0x10000000008/logs v261>,<0x10000000007/.git v818>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[]","memoryvalue":"(2)0x1000000023a:[<0x10000000232/heads v12>,<0x10000000231/refs v18>,<0x10000000008/logs v275>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:08:53.863551+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:08:53.867683+0000 b196 2=1+1)","memory_value.dirstat":"f(v0 m2024-04-24T09:08:53.863551+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-24T09:08:53.867683+0000 b196 2=1+1)","error_str":""},"return_code":0} 2024-04-24T09:09:29.089 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.638+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000233 [...3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/ auth v14 ap=1 f(v0 m2024-04-24T09:08:53.676370+0000 1=0+1) n(v1 rc2024-04-24T09:08:53.686342+0000 b196 3=1+2) (inest lock) (ifile excl) caps={15489=pAsLsXsFsx/AsLsXsFsx@1},l=15489 | importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5589ea086000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000233:[<0x10000000232/remotes v10>,<0x10000000231/refs v10>,<0x10000000008/logs v261>,<0x10000000007/.git v818>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[]","memoryvalue":"(2)0x10000000233:[<0x10000000232/remotes v14>,<0x10000000231/refs v18>,<0x10000000008/logs v275>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:08:53.676370+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:08:53.686342+0000 b196 3=1+2)","memory_value.dirstat":"f(v0 m2024-04-24T09:08:53.676370+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-04-24T09:08:53.686342+0000 b196 3=1+2)","error_str":""},"return_code":0} 2024-04-24T09:09:29.089 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.639+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000234 [...3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin/ auth v6 ap=1 f(v0 m2024-04-24T09:08:53.681819+0000 1=1+0) n(v0 rc2024-04-24T09:08:53.686342+0000 b196 2=1+1) (inest lock) (ifile excl) caps={15489=pAsLsXsFsx/AsLsXsFsx@1},l=15489 | importingcaps=0 dirfrag=1 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea128580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000234:[<0x10000000233/origin v6>,<0x10000000232/remotes v10>,<0x10000000231/refs v10>,<0x10000000008/logs v261>,<0x10000000007/.git v818>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[]","memoryvalue":"(2)0x10000000234:[<0x10000000233/origin v6>,<0x10000000232/remotes v14>,<0x10000000231/refs v18>,<0x10000000008/logs v275>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:08:53.681819+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:08:53.686342+0000 b196 2=1+1)","memory_value.dirstat":"f(v0 m2024-04-24T09:08:53.681819+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-24T09:08:53.686342+0000 b196 2=1+1)","error_str":""},"return_code":0} 2024-04-24T09:09:29.089 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.639+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023b [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/heads/master auth v4 ap=1 s=196 n(v0 rc2024-04-24T09:08:53.867683+0000 b196 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea128000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000023b:[<0x1000000023a/master v4>,<0x10000000232/heads v12>,<0x10000000231/refs v10>,<0x10000000008/logs v261>,<0x10000000007/.git v818>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000023b:[<0x1000000023a/master v4>,<0x10000000232/heads v12>,<0x10000000231/refs v18>,<0x10000000008/logs v275>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.090 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.640+0000 7f2593000700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000235 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin/HEAD auth v4 ap=1 s=196 n(v0 rc2024-04-24T09:08:53.686342+0000 b196 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5589ea128b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000235:[<0x10000000234/HEAD v4>,<0x10000000233/origin v6>,<0x10000000232/remotes v10>,<0x10000000231/refs v10>,<0x10000000008/logs v261>,<0x10000000007/.git v818>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000235:[<0x10000000234/HEAD v4>,<0x10000000233/origin v6>,<0x10000000232/remotes v14>,<0x10000000231/refs v18>,<0x10000000008/logs v275>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.090 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:09:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:09:28.653+0000 7f259b200700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000206 [...3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/ auth{0=1,1=1} v177 ap=2 f(v0 m2024-04-24T09:08:45.277998+0000 14=14+0) n(v0 rc2024-04-24T09:08:45.284285+0000 b25873 15=14+1) (inest sync->mix g=0,1) caps={15489=pAsLsXs/-@1} | lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirwaiter=0 replicated=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x5589ea116000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000206:[<0x10000000008/hooks v38>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[]","memoryvalue":"(2)0x10000000206:[<0x10000000008/hooks v177>,<0x10000000007/.git v1625>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:08:45.277998+0000 14=14+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:08:45.284285+0000 b25873 15=14+1)","memory_value.dirstat":"f(v0 m2024-04-24T09:08:45.277998+0000 14=14+0)","memory_value.rstat":"n(v0 rc2024-04-24T09:08:45.284285+0000 b25873 15=14+1)","error_str":""},"return_code":0} 2024-04-24T09:09:29.568 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.118+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a9 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/003 auth v406 ap=1 s=4721 n(v0 rc2024-04-24T09:08:58.024056+0000 b4721 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe4c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a9:[<0x100000003a4/003 v10>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003a9:[<0x100000003a4/003 v406>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.568 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.119+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000043f [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/078 auth v450 ap=1 s=1505 n(v0 rc2024-04-24T09:08:59.845590+0000 b1505 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe4c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000043f:[<0x100000003a4/078 v304>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000043f:[<0x100000003a4/078 v450>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: pgmap v75: 129 pgs: 129 active+clean; 30 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 479 KiB/s wr, 88 op/s 2024-04-24T09:09:29.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: scrub summary: active paths [/] 2024-04-24T09:09:29.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000334 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/md5.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000030f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fiemap-fault.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002db (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000340 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/parse-extent-tree.awk) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000360 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/statx.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000353 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/renameat2.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000367 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_offset2.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000037b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_mtab.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000323 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/helpers.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000030c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fake-dump-rootino.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000321 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/global.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000034b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/popdir.pl) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002f4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/attr_replace_test.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000311 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fiexchange.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000035f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/stat_test.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002ff (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/detached_mounts_propagation.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.570 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.120+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ee [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/037.out auth v428 ap=1 s=125 n(v0 rc2024-04-24T09:08:58.774197+0000 b125 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe4d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ee:[<0x100000003a4/037.out v148>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003ee:[<0x100000003a4/037.out v428>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.570 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.120+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000042a [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/067.out auth v440 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.444760+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe4db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000042a:[<0x100000003a4/067.out v268>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000042a:[<0x100000003a4/067.out v440>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.570 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.122+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e1 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/031 auth v420 ap=1 s=3221 n(v0 rc2024-04-24T09:08:58.628820+0000 b3221 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe4e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e1:[<0x100000003a4/031 v122>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003e1:[<0x100000003a4/031 v420>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000351 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/readdir-while-renames.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000033a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mmapcat.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000033b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/multi_open_unlink.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000325 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/holetest.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000349 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/permname.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000326 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/iopat.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000308 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/dumpfile) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000390 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003a1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/xfsfind.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000313 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fill2.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000312 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fill.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000329 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/locktest.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000337 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mkswap.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002fe (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/deduperace.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000370 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_holes.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000368 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_type.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000037a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_mmap_writev_overlap.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000366 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_offset.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000338 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mmap-rw-fault.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.572 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.123+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f6 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/041.out auth v434 ap=1 s=1845 n(v0 rc2024-04-24T09:08:58.857936+0000 b1845 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe56000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003f6:[<0x100000003a4/041.out v164>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003f6:[<0x100000003a4/041.out v434>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.572 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.123+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003db [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/028 auth v418 ap=1 s=1596 n(v0 rc2024-04-24T09:08:58.563073+0000 b1596 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe4ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003db:[<0x100000003a4/028 v110>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003db:[<0x100000003a4/028 v418>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.572 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.123+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e8 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/034.out auth v424 ap=1 s=93239 n(v0 rc2024-04-24T09:08:58.709189+0000 b93239 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe56580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e8:[<0x100000003a4/034.out v136>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003e8:[<0x100000003a4/034.out v424>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.572 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.124+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000400 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/046.out auth v438 ap=1 s=10836 n(v0 rc2024-04-24T09:08:58.971808+0000 b10836 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe56b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000400:[<0x100000003a4/046.out v184>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000400:[<0x100000003a4/046.out v438>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.572 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.124+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003fa [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/043.out auth v436 ap=1 s=25 n(v0 rc2024-04-24T09:08:58.905320+0000 b25 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe57600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003fa:[<0x100000003a4/043.out v172>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003fa:[<0x100000003a4/043.out v436>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002e0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-cow-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002ed (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aiocp.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002e2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-eof-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002e5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-hole-filling-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002ee (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aiodio_sparse2.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002dc (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/Makefile) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002ea (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-free-ring-with-bogus-nr-pages.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002ec (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-last-ref-held-by-io.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002e8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-subblock-eof-read.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002e9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-write-verify.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002e4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-fcntl-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002eb (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-io-setup-with-nonwritable-context-pointer.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002df (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-append-write-read-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002e1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-cycle-write.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002e3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-extend-stat.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002e6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-invalidate-failure.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.574 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.125+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c8 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/018.out auth v412 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.351246+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe57080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c8:[<0x100000003a4/018.out v72>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003c8:[<0x100000003a4/018.out v412>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.574 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.125+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ea [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/035.out auth v426 ap=1 s=418 n(v0 rc2024-04-24T09:08:58.730766+0000 b418 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe58680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ea:[<0x100000003a4/035.out v140>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003ea:[<0x100000003a4/035.out v426>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.575 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.125+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e7 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/034 auth v422 ap=1 s=2026 n(v0 rc2024-04-24T09:08:58.693822+0000 b2026 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe58100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e7:[<0x100000003a4/034 v134>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003e7:[<0x100000003a4/034 v422>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.575 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.125+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003cb [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/020 auth v414 ap=1 s=1451 n(v0 rc2024-04-24T09:08:58.384599+0000 b1451 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe57b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003cb:[<0x100000003a4/020 v78>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003cb:[<0x100000003a4/020 v414>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.575 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.126+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003bb [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/012 auth v408 ap=1 s=3419 n(v0 rc2024-04-24T09:08:58.223016+0000 b3419 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe58c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003bb:[<0x100000003a4/012 v46>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003bb:[<0x100000003a4/012 v408>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.575 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.126+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000451 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/087 auth v454 ap=1 s=7758 n(v0 rc2024-04-24T09:09:00.041567+0000 b7758 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe59180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000451:[<0x100000003a4/087 v308>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000451:[<0x100000003a4/087 v454>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.575 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.128+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000478 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/106.out auth v460 ap=1 s=196 n(v0 rc2024-04-24T09:09:00.474970+0000 b196 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe5c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000478:[<0x100000003a4/106.out v314>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000478:[<0x100000003a4/106.out v460>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.575 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.128+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f3 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/040 auth v430 ap=1 s=2826 n(v0 rc2024-04-24T09:08:58.826560+0000 b2826 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe5d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003f3:[<0x100000003a4/040 v158>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003f3:[<0x100000003a4/040 v430>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002dd (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/SOURCE) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002e7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-invalidate-readahead.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000002de (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-append-write-fallocate-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000023f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/index.lock) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000236 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/HEAD) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000009 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/description) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000021f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000231 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/config) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000203 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/info) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000022d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/packed-refs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000022b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/shallow) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000205 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/branches) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000039c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/vfstest.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000397 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/mount-idmapped.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000391 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/Makefile) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000398 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/tmpfs-idmapped-mounts.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.577 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.131+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000044f [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/086 auth v452 ap=1 s=1497 n(v0 rc2024-04-24T09:09:00.020904+0000 b1497 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe5d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000044f:[<0x100000003a4/086 v306>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000044f:[<0x100000003a4/086 v452>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.577 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.131+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000475 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/105 auth v456 ap=1 s=3144 n(v0 rc2024-04-24T09:09:00.443252+0000 b3144 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe5db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000475:[<0x100000003a4/105 v310>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000475:[<0x100000003a4/105 v456>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.577 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.132+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000480 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/110.out auth v464 ap=1 s=696 n(v0 rc2024-04-24T09:09:00.530145+0000 b696 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe5e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000480:[<0x100000003a4/110.out v318>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000480:[<0x100000003a4/110.out v464>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.577 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.134+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000047d [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/109 auth v462 ap=1 s=2824 n(v0 rc2024-04-24T09:09:00.512563+0000 b2824 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe5ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000047d:[<0x100000003a4/109 v316>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000047d:[<0x100000003a4/109 v462>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.577 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.135+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f4 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/040.out auth v432 ap=1 s=25 n(v0 rc2024-04-24T09:08:58.836038+0000 b25 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe64000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003f4:[<0x100000003a4/040.out v160>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003f4:[<0x100000003a4/040.out v432>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.577 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000039d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/vfstest.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.578 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000394 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/idmapped-mounts.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.578 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000395 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/idmapped-mounts.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.578 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000396 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/missing.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.578 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000399 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/tmpfs-idmapped-mounts.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.578 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000393 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/btrfs-idmapped-mounts.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.578 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000392 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/btrfs-idmapped-mounts.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.578 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000039a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/utils.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.578 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000039b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/utils.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.578 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000280 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/repair) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.578 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000026e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filter.btrfs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.578 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000265 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmlogwrites) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.578 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000281 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/report) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000025b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/casefold) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000278 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/perf) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000026b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/fail_make_request) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000262 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmerror) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000270 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/gcov) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.579 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.136+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c3 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/016 auth v410 ap=1 s=2227 n(v0 rc2024-04-24T09:08:58.309351+0000 b2227 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe65080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c3:[<0x100000003a4/016 v62>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003c3:[<0x100000003a4/016 v410>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.580 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.137+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000477 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/106 auth v458 ap=1 s=2423 n(v0 rc2024-04-24T09:09:00.463714+0000 b2423 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe65600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000477:[<0x100000003a4/106 v312>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000477:[<0x100000003a4/106 v458>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.580 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.137+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ac [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/004.out auth v400 ap=1 s=59 n(v0 rc2024-04-24T09:08:58.056400+0000 b59 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe65b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ac:[<0x100000003a4/004.out v16>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003ac:[<0x100000003a4/004.out v400>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000282 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/scsi_debug) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000287 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/xfs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000286 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/verity) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000026c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filestreams) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000269 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ext4) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000268 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/encrypt) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000027a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/preamble) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000264 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmhugedisk) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000025e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/config) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000272 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/locktest) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000027d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/rc) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000025d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/cgroup2) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000267 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dump) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000288 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/zoned) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000027f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/renameat2) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000025c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ceph) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000258 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/Makefile) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000274 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/metadump) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.582 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.138+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b4 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/008.out auth v406 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.146235+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe66100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b4:[<0x100000003a4/008.out v32>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003b4:[<0x100000003a4/008.out v406>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.582 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.138+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ba [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/011.out auth v408 ap=1 s=57 n(v0 rc2024-04-24T09:08:58.212384+0000 b57 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe66680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ba:[<0x100000003a4/011.out v44>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003ba:[<0x100000003a4/011.out v408>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.582 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.138+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a7 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/002 auth v396 ap=1 s=3463 n(v0 rc2024-04-24T09:08:58.003611+0000 b3463 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe66c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a7:[<0x100000003a4/002 v6>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003a7:[<0x100000003a4/002 v396>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.582 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.139+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000466 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/097.out auth v468 ap=1 s=453 n(v0 rc2024-04-24T09:09:00.291606+0000 b453 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84feba000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000466:[<0x100000003a4/097.out v312>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000466:[<0x100000003a4/097.out v468>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.582 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.139+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ae [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/005.out auth v402 ap=1 s=564 n(v0 rc2024-04-24T09:08:58.077478+0000 b564 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe67700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ae:[<0x100000003a4/005.out v20>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003ae:[<0x100000003a4/005.out v402>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000027b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/punch) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000263 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmflakey) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000276 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/nfs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000285 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ubifs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000026d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filter) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000260 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmdelay) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000275 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/module) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000025a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/btrfs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000027e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/reflink) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000026f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/fuzzy) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000283 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/test_names) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000261 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmdust) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000259 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/attr) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000027c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/quota) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000266 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmthin) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.584 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000271 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/inject) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.584 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000273 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/log) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.584 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000279 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/populate) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.584 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.139+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000043e [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/077.out auth v460 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.834479+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84febab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000043e:[<0x100000003a4/077.out v304>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000043e:[<0x100000003a4/077.out v460>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.584 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.140+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ca [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/019.out auth v414 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.371775+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84feba580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ca:[<0x100000003a4/019.out v76>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003ca:[<0x100000003a4/019.out v414>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.584 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.141+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000436 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/073.out auth v458 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.634304+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84febb600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000436:[<0x100000003a4/073.out v292>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000436:[<0x100000003a4/073.out v458>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.584 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.142+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000407 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/050 auth v436 ap=1 s=2444 n(v0 rc2024-04-24T09:08:59.049344+0000 b2444 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84febc100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000407:[<0x100000003a4/050 v198>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000407:[<0x100000003a4/050 v436>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.585 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.142+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003df [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/030 auth v426 ap=1 s=5993 n(v0 rc2024-04-24T09:08:58.607022+0000 b5993 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84febd180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003df:[<0x100000003a4/030 v118>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003df:[<0x100000003a4/030 v426>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000277 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/overlay) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000026a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/f2fs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects/info) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000025f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/defrag) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000284 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/tracing) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000220 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects/pack) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000239 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/HEAD) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000232 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000204 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/info/exclude) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs/heads) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs/tags) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000023a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/heads) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000233 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000234 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000023b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/heads/master) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000235 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin/HEAD) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000206 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-mon[30961]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:29.590 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.143+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003fc [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/044.out auth v434 ap=1 s=25 n(v0 rc2024-04-24T09:08:58.927889+0000 b25 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84febcc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003fc:[<0x100000003a4/044.out v176>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003fc:[<0x100000003a4/044.out v434>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.590 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.144+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f7 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/042 auth v432 ap=1 s=1589 n(v0 rc2024-04-24T09:08:58.870060+0000 b1589 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff08000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003f7:[<0x100000003a4/042 v166>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003f7:[<0x100000003a4/042 v432>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.590 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.146+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000045f [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/094 auth v464 ap=1 s=4105 n(v0 rc2024-04-24T09:09:00.212374+0000 b4105 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff08580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000045f:[<0x100000003a4/094 v308>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000045f:[<0x100000003a4/094 v464>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.590 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.146+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000045a [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/091.out auth v462 ap=1 s=173 n(v0 rc2024-04-24T09:09:00.150936+0000 b173 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff08b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000045a:[<0x100000003a4/091.out v306>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000045a:[<0x100000003a4/091.out v462>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.590 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.147+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003bc [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/012.out auth v410 ap=1 s=25 n(v0 rc2024-04-24T09:08:58.234054+0000 b25 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff09080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003bc:[<0x100000003a4/012.out v48>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003bc:[<0x100000003a4/012.out v410>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.591 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.147+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003aa [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/003.out auth v398 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.034398+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff09600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003aa:[<0x100000003a4/003.out v12>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003aa:[<0x100000003a4/003.out v398>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.591 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.149+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000431 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/071 auth v440 ap=1 s=2595 n(v0 rc2024-04-24T09:08:59.520963+0000 b2595 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff0ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000431:[<0x100000003a4/071 v282>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000431:[<0x100000003a4/071 v440>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.591 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.149+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000424 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/064.out auth v438 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.379405+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff0b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000424:[<0x100000003a4/064.out v256>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000424:[<0x100000003a4/064.out v438>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.591 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.150+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c4 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/016.out auth v412 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.317482+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff0b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c4:[<0x100000003a4/016.out v64>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003c4:[<0x100000003a4/016.out v412>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.591 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.150+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b1 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/007 auth v404 ap=1 s=2678 n(v0 rc2024-04-24T09:08:58.109577+0000 b2678 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff14580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b1:[<0x100000003a4/007 v26>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003b1:[<0x100000003a4/007 v404>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.591 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.151+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d1 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/023 auth v416 ap=1 s=1418 n(v0 rc2024-04-24T09:08:58.448974+0000 b1418 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff14000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d1:[<0x100000003a4/023 v90>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003d1:[<0x100000003a4/023 v416>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.591 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.153+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e3 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/032 auth v428 ap=1 s=591 n(v0 rc2024-04-24T09:08:58.651442+0000 b591 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff15600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e3:[<0x100000003a4/032 v126>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003e3:[<0x100000003a4/032 v428>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.591 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.155+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e5 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/033 auth v430 ap=1 s=940 n(v0 rc2024-04-24T09:08:58.672328+0000 b940 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff16100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e5:[<0x100000003a4/033 v130>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003e5:[<0x100000003a4/033 v430>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.591 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.155+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000464 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/096.out auth v466 ap=1 s=181 n(v0 rc2024-04-24T09:09:00.268495+0000 b181 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff15b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000464:[<0x100000003a4/096.out v310>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000464:[<0x100000003a4/096.out v466>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.591 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.156+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000468 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/098.out auth v530 ap=1 s=558 n(v0 rc2024-04-24T09:09:00.313390+0000 b558 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff16680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000468:[<0x100000003a4/098.out v316>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000468:[<0x100000003a4/098.out v530>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.591 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.157+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ff [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/046 auth v474 ap=1 s=11673 n(v0 rc2024-04-24T09:08:58.960646+0000 b11673 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff16c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ff:[<0x100000003a4/046 v182>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003ff:[<0x100000003a4/046 v474>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.591 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.158+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b5 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/009 auth v438 ap=1 s=886 n(v0 rc2024-04-24T09:08:58.158676+0000 b886 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff17700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b5:[<0x100000003a4/009 v34>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003b5:[<0x100000003a4/009 v438>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1647>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.591 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.159+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000432 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/071.out auth v514 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.590778+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe34000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000432:[<0x100000003a4/071.out v284>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000432:[<0x100000003a4/071.out v514>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.592 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.159+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003fe [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/045.out auth v472 ap=1 s=25 n(v0 rc2024-04-24T09:08:58.949654+0000 b25 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe34580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003fe:[<0x100000003a4/045.out v180>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003fe:[<0x100000003a4/045.out v472>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.592 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.160+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000404 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/048.out auth v476 ap=1 s=1835 n(v0 rc2024-04-24T09:08:59.017380+0000 b1835 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe34b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000404:[<0x100000003a4/048.out v192>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000404:[<0x100000003a4/048.out v476>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.592 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.160+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b8 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/010.out auth v440 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.189621+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe35600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b8:[<0x100000003a4/010.out v40>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003b8:[<0x100000003a4/010.out v440>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.592 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.160+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000044a [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/083.out auth v520 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.963280+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe35b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000044a:[<0x100000003a4/083.out v306>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000044a:[<0x100000003a4/083.out v520>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.592 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.161+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000043b [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/076 auth v516 ap=1 s=1668 n(v0 rc2024-04-24T09:08:59.688673+0000 b1668 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe36100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000043b:[<0x100000003a4/076 v302>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000043b:[<0x100000003a4/076 v516>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.592 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.161+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000045b [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/092 auth v528 ap=1 s=7161 n(v0 rc2024-04-24T09:09:00.163727+0000 b7161 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe36680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000045b:[<0x100000003a4/092 v314>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000045b:[<0x100000003a4/092 v528>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.592 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.162+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d8 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/026.out auth v456 ap=1 s=742 n(v0 rc2024-04-24T09:08:58.525912+0000 b742 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe37700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d8:[<0x100000003a4/026.out v104>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003d8:[<0x100000003a4/026.out v456>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.592 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.163+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003bd [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/013 auth v442 ap=1 s=1583 n(v0 rc2024-04-24T09:08:58.246537+0000 b1583 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe3f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003bd:[<0x100000003a4/013 v50>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003bd:[<0x100000003a4/013 v442>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.592 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.163+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d6 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/025.out auth v452 ap=1 s=307 n(v0 rc2024-04-24T09:08:58.502059+0000 b307 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe3e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d6:[<0x100000003a4/025.out v100>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003d6:[<0x100000003a4/025.out v452>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.592 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.165+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000041c [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/060.out auth v484 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.288955+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe40100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000041c:[<0x100000003a4/060.out v240>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000041c:[<0x100000003a4/060.out v484>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.592 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.166+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c1 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/015 auth v446 ap=1 s=666 n(v0 rc2024-04-24T09:08:58.289065+0000 b666 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe40680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c1:[<0x100000003a4/015 v58>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003c1:[<0x100000003a4/015 v446>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.592 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.167+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000047f [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/110 auth v538 ap=1 s=3027 n(v0 rc2024-04-24T09:09:00.523939+0000 b3027 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe41700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000047f:[<0x100000003a4/110 v324>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000047f:[<0x100000003a4/110 v538>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.593 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.167+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000046a [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/099.out auth v532 ap=1 s=633 n(v0 rc2024-04-24T09:09:00.336038+0000 b633 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff62000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000046a:[<0x100000003a4/099.out v318>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000046a:[<0x100000003a4/099.out v532>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.593 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.168+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000423 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/064 auth v486 ap=1 s=2854 n(v0 rc2024-04-24T09:08:59.368850+0000 b2854 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff62b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000423:[<0x100000003a4/064 v254>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000423:[<0x100000003a4/064 v486>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.593 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.168+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000043c [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/076.out auth v518 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.811727+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff64100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000043c:[<0x100000003a4/076.out v304>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000043c:[<0x100000003a4/076.out v518>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.593 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.168+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d7 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/026 auth v454 ap=1 s=3125 n(v0 rc2024-04-24T09:08:58.511405+0000 b3125 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff63080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d7:[<0x100000003a4/026 v102>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003d7:[<0x100000003a4/026 v454>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.593 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.169+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000457 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/090 auth v526 ap=1 s=993 n(v0 rc2024-04-24T09:09:00.116407+0000 b993 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff63600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000457:[<0x100000003a4/090 v312>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000457:[<0x100000003a4/090 v526>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.593 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.169+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000453 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/088 auth v522 ap=1 s=3551 n(v0 rc2024-04-24T09:09:00.066708+0000 b3551 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff64c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000453:[<0x100000003a4/088 v308>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000453:[<0x100000003a4/088 v522>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.593 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.170+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000413 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/056 auth v482 ap=1 s=4107 n(v0 rc2024-04-24T09:08:59.180906+0000 b4107 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff64680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000413:[<0x100000003a4/056 v222>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000413:[<0x100000003a4/056 v482>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.593 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.170+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000040c [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/052.out auth v480 ap=1 s=15846 n(v0 rc2024-04-24T09:08:59.102644+0000 b15846 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff65180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000040c:[<0x100000003a4/052.out v208>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000040c:[<0x100000003a4/052.out v480>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.593 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.171+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f8 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/042.out auth v470 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.882092+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff65700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003f8:[<0x100000003a4/042.out v168>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003f8:[<0x100000003a4/042.out v470>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.593 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.172+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000472 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/103.out auth v534 ap=1 s=2294 n(v0 rc2024-04-24T09:09:00.413007+0000 b2294 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff6c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000472:[<0x100000003a4/103.out v320>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000472:[<0x100000003a4/103.out v534>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.593 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.173+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000454 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/088.out auth v524 ap=1 s=346 n(v0 rc2024-04-24T09:09:00.078498+0000 b346 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff6d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000454:[<0x100000003a4/088.out v310>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000454:[<0x100000003a4/088.out v524>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.593 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.174+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d4 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/024.out auth v450 ap=1 s=280 n(v0 rc2024-04-24T09:08:58.481566+0000 b280 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff6ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d4:[<0x100000003a4/024.out v96>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003d4:[<0x100000003a4/024.out v450>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.593 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.174+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c0 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/014.out auth v444 ap=1 s=68 n(v0 rc2024-04-24T09:08:58.279962+0000 b68 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff6e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c0:[<0x100000003a4/014.out v56>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003c0:[<0x100000003a4/014.out v444>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.594 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.175+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000047c [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/108.out auth v536 ap=1 s=617 n(v0 rc2024-04-24T09:09:00.505653+0000 b617 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fd8a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000047c:[<0x100000003a4/108.out v322>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000047c:[<0x100000003a4/108.out v536>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.594 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.176+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000409 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/051 auth v478 ap=1 s=1716 n(v0 rc2024-04-24T09:08:59.070445+0000 b1716 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fd8a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000409:[<0x100000003a4/051 v202>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000409:[<0x100000003a4/051 v478>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.594 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.176+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000042b [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/068 auth v488 ap=1 s=2382 n(v0 rc2024-04-24T09:08:59.455882+0000 b2382 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fd8ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000042b:[<0x100000003a4/068 v270>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000042b:[<0x100000003a4/068 v488>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.594 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.176+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d2 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/023.out auth v448 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.459798+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff6f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d2:[<0x100000003a4/023.out v92>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003d2:[<0x100000003a4/023.out v448>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.594 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.178+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000430 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/070.out auth v490 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.510427+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fe64b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000430:[<0x100000003a4/070.out v280>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000430:[<0x100000003a4/070.out v490>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.594 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.178+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a5 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/001 auth v436 ap=1 s=2165 n(v0 rc2024-04-24T09:08:57.980663+0000 b2165 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fd8b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a5:[<0x100000003a4/001 v2>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003a5:[<0x100000003a4/001 v436>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.594 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.178+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000455 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/089 auth v532 ap=1 s=1424 n(v0 rc2024-04-24T09:09:00.092488+0000 b1424 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fd8bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000455:[<0x100000003a4/089 v318>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000455:[<0x100000003a4/089 v532>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.594 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.179+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000448 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/082.out auth v524 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.942288+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fd8d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000448:[<0x100000003a4/082.out v310>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000448:[<0x100000003a4/082.out v524>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.594 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.179+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b3 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/008 auth v456 ap=1 s=1991 n(v0 rc2024-04-24T09:08:58.137884+0000 b1991 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fd8c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b3:[<0x100000003a4/008 v30>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003b3:[<0x100000003a4/008 v456>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.594 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.179+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b2 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/007.out auth v454 ap=1 s=58 n(v0 rc2024-04-24T09:08:58.122663+0000 b58 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fd8cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b2:[<0x100000003a4/007.out v28>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003b2:[<0x100000003a4/007.out v454>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.594 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.180+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000042c [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/068.out auth v502 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.467522+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fd9c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000042c:[<0x100000003a4/068.out v272>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000042c:[<0x100000003a4/068.out v502>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.595 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.180+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000467 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/098 auth v540 ap=1 s=3977 n(v0 rc2024-04-24T09:09:00.302504+0000 b3977 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fd8d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000467:[<0x100000003a4/098 v326>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000467:[<0x100000003a4/098 v540>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.595 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.180+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d3 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/024 auth v462 ap=1 s=1014 n(v0 rc2024-04-24T09:08:58.470352+0000 b1014 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fd9cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d3:[<0x100000003a4/024 v94>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003d3:[<0x100000003a4/024 v462>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.595 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.181+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000420 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/062.out auth v496 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.337483+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fd9d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000420:[<0x100000003a4/062.out v248>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000420:[<0x100000003a4/062.out v496>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.595 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.181+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000421 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/063 auth v498 ap=1 s=2136 n(v0 rc2024-04-24T09:08:59.347918+0000 b2136 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fd9d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000421:[<0x100000003a4/063 v250>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000421:[<0x100000003a4/063 v498>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.595 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.181+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000411 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/055 auth v488 ap=1 s=5906 n(v0 rc2024-04-24T09:08:59.155980+0000 b5906 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fd9db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000411:[<0x100000003a4/055 v218>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000411:[<0x100000003a4/055 v488>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.595 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.182+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000045d [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/093 auth v536 ap=1 s=3124 n(v0 rc2024-04-24T09:09:00.189826+0000 b3124 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fd9e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000045d:[<0x100000003a4/093 v322>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000045d:[<0x100000003a4/093 v536>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.595 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.184+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000428 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/066.out auth v500 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.422815+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fd9f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000428:[<0x100000003a4/066.out v264>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000428:[<0x100000003a4/066.out v500>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.595 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.184+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000459 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/091 auth v534 ap=1 s=1585 n(v0 rc2024-04-24T09:09:00.140914+0000 b1585 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff1a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000459:[<0x100000003a4/091 v320>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000459:[<0x100000003a4/091 v534>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.595 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.184+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000406 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/049.out auth v480 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.038290+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff1a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000406:[<0x100000003a4/049.out v196>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000406:[<0x100000003a4/049.out v480>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.595 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.185+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000443 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/080 auth v520 ap=1 s=4036 n(v0 rc2024-04-24T09:08:59.889213+0000 b4036 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff1ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000443:[<0x100000003a4/080 v306>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000443:[<0x100000003a4/080 v520>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.595 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.185+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000473 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/104 auth v546 ap=1 s=3862 n(v0 rc2024-04-24T09:09:00.421905+0000 b3862 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff1b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000473:[<0x100000003a4/104 v332>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000473:[<0x100000003a4/104 v546>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.595 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.186+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000044e [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/085.out auth v528 ap=1 s=152 n(v0 rc2024-04-24T09:09:00.008968+0000 b152 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff1b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000044e:[<0x100000003a4/085.out v314>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000044e:[<0x100000003a4/085.out v528>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.595 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.186+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000465 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/097 auth v538 ap=1 s=3530 n(v0 rc2024-04-24T09:09:00.278573+0000 b3530 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff1c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000465:[<0x100000003a4/097 v324>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000465:[<0x100000003a4/097 v538>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.596 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.186+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000410 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/054.out auth v486 ap=1 s=25 n(v0 rc2024-04-24T09:08:59.143830+0000 b25 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff1c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000410:[<0x100000003a4/054.out v216>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000410:[<0x100000003a4/054.out v486>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.596 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.187+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000040e [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/053.out auth v482 ap=1 s=25 n(v0 rc2024-04-24T09:08:59.122992+0000 b25 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff1d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000040e:[<0x100000003a4/053.out v212>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000040e:[<0x100000003a4/053.out v482>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.596 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.190+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003fd [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/045 auth v478 ap=1 s=12842 n(v0 rc2024-04-24T09:08:58.938893+0000 b12842 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff24580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003fd:[<0x100000003a4/045 v178>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003fd:[<0x100000003a4/045 v478>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.596 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.190+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000442 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/079.out auth v518 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.877779+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff24b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000442:[<0x100000003a4/079.out v304>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000442:[<0x100000003a4/079.out v518>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.596 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.190+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000041e [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/061.out auth v494 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.317432+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff25080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000041e:[<0x100000003a4/061.out v244>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000041e:[<0x100000003a4/061.out v494>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.596 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.191+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000044c [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/084.out auth v526 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.988591+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff25600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000044c:[<0x100000003a4/084.out v312>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000044c:[<0x100000003a4/084.out v526>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.596 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.192+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b7 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/010 auth v458 ap=1 s=1324 n(v0 rc2024-04-24T09:08:58.179225+0000 b1324 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff25b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b7:[<0x100000003a4/010 v38>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003b7:[<0x100000003a4/010 v458>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.596 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.192+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b9 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/011 auth v460 ap=1 s=8315 n(v0 rc2024-04-24T09:08:58.200985+0000 b8315 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff26100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b9:[<0x100000003a4/011 v42>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003b9:[<0x100000003a4/011 v460>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.596 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.193+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000444 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/080.out auth v522 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.899953+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff26680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000444:[<0x100000003a4/080.out v308>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000444:[<0x100000003a4/080.out v522>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.596 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.193+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000041b [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/060 auth v492 ap=1 s=2320 n(v0 rc2024-04-24T09:08:59.276862+0000 b2320 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff27180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000041b:[<0x100000003a4/060 v238>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000041b:[<0x100000003a4/060 v492>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.596 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.193+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000046b [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/100 auth v542 ap=1 s=1521 n(v0 rc2024-04-24T09:09:00.346980+0000 b1521 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff27700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000046b:[<0x100000003a4/100 v328>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000046b:[<0x100000003a4/100 v542>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.596 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.194+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d5 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/025 auth v464 ap=1 s=2158 n(v0 rc2024-04-24T09:08:58.491227+0000 b2158 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff26c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d5:[<0x100000003a4/025 v98>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003d5:[<0x100000003a4/025 v464>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.597 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.195+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000452 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/087.out auth v530 ap=1 s=43 n(v0 rc2024-04-24T09:09:00.053971+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff2f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000452:[<0x100000003a4/087.out v316>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000452:[<0x100000003a4/087.out v530>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.597 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.196+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000433 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/072 auth v516 ap=1 s=2174 n(v0 rc2024-04-24T09:08:59.601740+0000 b2174 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff2fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000433:[<0x100000003a4/072 v286>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000433:[<0x100000003a4/072 v516>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.597 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.196+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a8 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/002.out auth v452 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.013062+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff30100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a8:[<0x100000003a4/002.out v8>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003a8:[<0x100000003a4/002.out v452>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.597 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.197+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000046e [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/101.out auth v544 ap=1 s=370 n(v0 rc2024-04-24T09:09:00.373343+0000 b370 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff31180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000046e:[<0x100000003a4/101.out v330>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000046e:[<0x100000003a4/101.out v544>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: pgmap v75: 129 pgs: 129 active+clean; 30 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 479 KiB/s wr, 88 op/s 2024-04-24T09:09:29.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: scrub summary: active paths [/] 2024-04-24T09:09:29.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000334 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/md5.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000030f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fiemap-fault.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002db (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000340 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/parse-extent-tree.awk) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000360 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/statx.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000353 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/renameat2.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000367 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_offset2.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000037b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_mtab.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000323 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/helpers.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000030c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fake-dump-rootino.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000321 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/global.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000034b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/popdir.pl) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002f4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/attr_replace_test.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000311 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fiexchange.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000035f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/stat_test.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002ff (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/detached_mounts_propagation.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000351 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/readdir-while-renames.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000033a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mmapcat.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000033b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/multi_open_unlink.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000325 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/holetest.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000349 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/permname.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000326 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/iopat.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000308 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/dumpfile) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000390 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003a1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/xfsfind.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000313 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fill2.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000312 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fill.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000329 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/locktest.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000337 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mkswap.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002fe (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/deduperace.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000370 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_holes.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000368 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_type.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000037a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_mmap_writev_overlap.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000366 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_offset.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000338 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mmap-rw-fault.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002e0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-cow-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002ed (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aiocp.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002e2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-eof-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002e5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-hole-filling-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002ee (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aiodio_sparse2.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002dc (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/Makefile) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002ea (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-free-ring-with-bogus-nr-pages.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002ec (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-last-ref-held-by-io.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002e8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-subblock-eof-read.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002e9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-write-verify.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002e4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-fcntl-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002eb (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-io-setup-with-nonwritable-context-pointer.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002df (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-append-write-read-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002e1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-cycle-write.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002e3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-extend-stat.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002e6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-invalidate-failure.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002dd (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/SOURCE) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002e7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-invalidate-readahead.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000002de (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-append-write-fallocate-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000023f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/index.lock) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000236 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/HEAD) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000009 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/description) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000021f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000231 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/config) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000203 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/info) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000022d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/packed-refs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000022b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/shallow) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000205 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/branches) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000039c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/vfstest.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000397 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/mount-idmapped.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000391 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/Makefile) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000398 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/tmpfs-idmapped-mounts.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000039d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/vfstest.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000394 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/idmapped-mounts.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000395 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/idmapped-mounts.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000396 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/missing.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000399 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/tmpfs-idmapped-mounts.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000393 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/btrfs-idmapped-mounts.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000392 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/btrfs-idmapped-mounts.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000039a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/utils.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000039b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/utils.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000280 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/repair) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000026e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filter.btrfs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000265 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmlogwrites) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000281 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/report) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000025b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/casefold) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000278 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/perf) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000026b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/fail_make_request) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000262 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmerror) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000270 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/gcov) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000282 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/scsi_debug) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000287 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/xfs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000286 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/verity) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000026c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filestreams) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000269 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ext4) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000268 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/encrypt) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000027a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/preamble) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000264 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmhugedisk) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000025e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/config) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000272 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/locktest) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000027d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/rc) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000025d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/cgroup2) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000267 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dump) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000288 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/zoned) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000027f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/renameat2) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000025c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ceph) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000258 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/Makefile) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000274 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/metadump) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000027b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/punch) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000263 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmflakey) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000276 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/nfs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000285 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ubifs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000026d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filter) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000260 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmdelay) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000275 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/module) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000025a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/btrfs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000027e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/reflink) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000026f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/fuzzy) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000283 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/test_names) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000261 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmdust) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000259 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/attr) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000027c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/quota) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000266 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmthin) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000271 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/inject) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000273 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/log) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000279 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/populate) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000277 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/overlay) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000026a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/f2fs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects/info) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000025f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/defrag) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000284 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/tracing) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000220 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects/pack) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000239 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/HEAD) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000232 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000204 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/info/exclude) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs/heads) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs/tags) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000023a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/heads) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000233 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000234 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000023b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/heads/master) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000235 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin/HEAD) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000206 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:29 smithi069 ceph-mon[28170]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:29.614 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.198+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ec [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/036.out auth v476 ap=1 s=54 n(v0 rc2024-04-24T09:08:58.752425+0000 b54 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff3c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ec:[<0x100000003a4/036.out v144>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003ec:[<0x100000003a4/036.out v476>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.614 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.199+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000040f [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/054 auth v484 ap=1 s=2543 n(v0 rc2024-04-24T09:08:59.133547+0000 b2543 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff3cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000040f:[<0x100000003a4/054 v214>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000040f:[<0x100000003a4/054 v484>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.614 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.200+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000040b [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/052 auth v480 ap=1 s=5497 n(v0 rc2024-04-24T09:08:59.091673+0000 b5497 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff3d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000040b:[<0x100000003a4/052 v206>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000040b:[<0x100000003a4/052 v480>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.614 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.200+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000417 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/058 auth v490 ap=1 s=1556 n(v0 rc2024-04-24T09:08:59.226864+0000 b1556 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff3d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000417:[<0x100000003a4/058 v230>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000417:[<0x100000003a4/058 v490>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.614 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.201+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d0 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/022.out auth v450 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.437931+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff3e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d0:[<0x100000003a4/022.out v88>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003d0:[<0x100000003a4/022.out v450>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.614 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.202+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f5 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/041 auth v472 ap=1 s=2537 n(v0 rc2024-04-24T09:08:58.846559+0000 b2537 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff48000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003f5:[<0x100000003a4/041 v162>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003f5:[<0x100000003a4/041 v472>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.615 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.202+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000476 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/105.out auth v516 ap=1 s=538 n(v0 rc2024-04-24T09:09:00.452985+0000 b538 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff48580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000476:[<0x100000003a4/105.out v318>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000476:[<0x100000003a4/105.out v516>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.615 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.203+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b6 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/009.out auth v436 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.167506+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff48b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b6:[<0x100000003a4/009.out v36>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003b6:[<0x100000003a4/009.out v436>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.615 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.203+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003b0 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/006.out auth v434 ap=1 s=1347 n(v0 rc2024-04-24T09:08:58.098180+0000 b1347 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff49600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003b0:[<0x100000003a4/006.out v24>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003b0:[<0x100000003a4/006.out v434>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.615 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.205+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000043d [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/077 auth v502 ap=1 s=4434 n(v0 rc2024-04-24T09:08:59.822608+0000 b4434 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff4a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000043d:[<0x100000003a4/077 v304>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000043d:[<0x100000003a4/077 v502>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.615 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.206+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000458 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/090.out auth v510 ap=1 s=43 n(v0 rc2024-04-24T09:09:00.128342+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000458:[<0x100000003a4/090.out v312>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000458:[<0x100000003a4/090.out v510>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.615 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.207+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ad [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/005 auth v432 ap=1 s=3067 n(v0 rc2024-04-24T09:08:58.066591+0000 b3067 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff52580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ad:[<0x100000003a4/005 v18>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003ad:[<0x100000003a4/005 v432>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.615 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.207+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000481 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/111 auth v520 ap=1 s=3987 n(v0 rc2024-04-24T09:09:00.541510+0000 b3987 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff52b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000481:[<0x100000003a4/111 v322>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000481:[<0x100000003a4/111 v520>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.615 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.210+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003cd [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/021 auth v446 ap=1 s=1471 n(v0 rc2024-04-24T09:08:58.406188+0000 b1471 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff53080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003cd:[<0x100000003a4/021 v82>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003cd:[<0x100000003a4/021 v446>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.615 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.210+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000471 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/103 auth v514 ap=1 s=2811 n(v0 rc2024-04-24T09:09:00.403653+0000 b2811 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff53600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000471:[<0x100000003a4/103 v316>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000471:[<0x100000003a4/103 v514>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.615 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.210+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000044d [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/085 auth v508 ap=1 s=2760 n(v0 rc2024-04-24T09:08:59.999616+0000 b2760 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff54100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000044d:[<0x100000003a4/085 v310>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000044d:[<0x100000003a4/085 v508>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.615 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.211+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000040d [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/053 auth v482 ap=1 s=2647 n(v0 rc2024-04-24T09:08:59.112165+0000 b2647 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff53b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000040d:[<0x100000003a4/053 v210>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000040d:[<0x100000003a4/053 v482>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.616 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.211+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003de [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/029.out auth v466 ap=1 s=536 n(v0 rc2024-04-24T09:08:58.596340+0000 b536 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff54680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003de:[<0x100000003a4/029.out v116>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003de:[<0x100000003a4/029.out v466>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.616 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.212+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000441 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/079 auth v506 ap=1 s=2955 n(v0 rc2024-04-24T09:08:59.866366+0000 b2955 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff54c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000441:[<0x100000003a4/079 v308>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000441:[<0x100000003a4/079 v506>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.616 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.212+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000440 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/078.out auth v504 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.855999+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff55180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000440:[<0x100000003a4/078.out v306>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000440:[<0x100000003a4/078.out v504>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.616 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.212+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c6 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/017.out auth v442 ap=1 s=74 n(v0 rc2024-04-24T09:08:58.334313+0000 b74 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84ff55700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c6:[<0x100000003a4/017.out v68>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003c6:[<0x100000003a4/017.out v442>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.616 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.214+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000402 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/047.out auth v478 ap=1 s=87 n(v0 rc2024-04-24T09:08:58.995230+0000 b87 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fec7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000402:[<0x100000003a4/047.out v188>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000402:[<0x100000003a4/047.out v478>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1649>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.616 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.214+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000401 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/047 auth v476 ap=1 s=1047 n(v0 rc2024-04-24T09:08:58.985324+0000 b1047 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fec7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000401:[<0x100000003a4/047 v186>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000401:[<0x100000003a4/047 v476>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.616 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.215+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000418 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/058.out auth v484 ap=1 s=25 n(v0 rc2024-04-24T09:08:59.238928+0000 b25 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fec7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000418:[<0x100000003a4/058.out v232>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000418:[<0x100000003a4/058.out v484>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.616 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.215+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e9 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/035 auth v470 ap=1 s=1887 n(v0 rc2024-04-24T09:08:58.719543+0000 b1887 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fec8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e9:[<0x100000003a4/035 v138>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003e9:[<0x100000003a4/035 v470>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.616 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.215+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c7 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/018 auth v444 ap=1 s=750 n(v0 rc2024-04-24T09:08:58.342699+0000 b750 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fec8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c7:[<0x100000003a4/018 v70>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003c7:[<0x100000003a4/018 v444>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.616 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.215+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003fb [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/044 auth v474 ap=1 s=3196 n(v0 rc2024-04-24T09:08:58.916832+0000 b3196 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fec9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003fb:[<0x100000003a4/044 v174>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003fb:[<0x100000003a4/044 v474>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.616 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.215+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003bf [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/014 auth v438 ap=1 s=793 n(v0 rc2024-04-24T09:08:58.270959+0000 b793 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fec8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003bf:[<0x100000003a4/014 v54>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003bf:[<0x100000003a4/014 v438>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.616 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.216+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000041f [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/062 auth v486 ap=1 s=2198 n(v0 rc2024-04-24T09:08:59.328079+0000 b2198 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fec9700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000041f:[<0x100000003a4/062 v246>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000041f:[<0x100000003a4/062 v486>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.617 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.216+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e2 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/031.out auth v468 ap=1 s=1304 n(v0 rc2024-04-24T09:08:58.639354+0000 b1304 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fed4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e2:[<0x100000003a4/031.out v124>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003e2:[<0x100000003a4/031.out v468>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.617 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.217+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000460 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/094.out auth v512 ap=1 s=1329 n(v0 rc2024-04-24T09:09:00.223065+0000 b1329 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fed4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000460:[<0x100000003a4/094.out v314>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000460:[<0x100000003a4/094.out v512>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.617 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.218+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000479 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/107 auth v518 ap=1 s=1354 n(v0 rc2024-04-24T09:09:00.484377+0000 b1354 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fed5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000479:[<0x100000003a4/107 v320>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000479:[<0x100000003a4/107 v518>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.617 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.220+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c2 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/015.out auth v440 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.300512+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fed6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c2:[<0x100000003a4/015.out v60>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003c2:[<0x100000003a4/015.out v440>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.617 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.220+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ce [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/021.out auth v448 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.416746+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fed5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ce:[<0x100000003a4/021.out v84>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003ce:[<0x100000003a4/021.out v448>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.617 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.220+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000426 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/065.out auth v488 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.400431+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x55a84fed6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000426:[<0x100000003a4/065.out v260>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000426:[<0x100000003a4/065.out v488>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.617 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.221+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000046c [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/100.out auth v464 ap=1 s=371 n(v0 rc2024-04-24T09:09:00.357105+0000 b371 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fed6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000046c:[<0x100000003a4/100.out v312>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000046c:[<0x100000003a4/100.out v464>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.617 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.222+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f2 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/039.out auth v434 ap=1 s=25 n(v0 rc2024-04-24T09:08:58.816028+0000 b25 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fed7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003f2:[<0x100000003a4/039.out v156>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003f2:[<0x100000003a4/039.out v434>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.617 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.223+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000419 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/059 auth v440 ap=1 s=1855 n(v0 rc2024-04-24T09:08:59.250406+0000 b1855 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fede000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000419:[<0x100000003a4/059 v234>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000419:[<0x100000003a4/059 v440>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.617 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.223+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f0 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/038.out auth v432 ap=1 s=424 n(v0 rc2024-04-24T09:08:58.795018+0000 b424 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fed7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003f0:[<0x100000003a4/038.out v152>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003f0:[<0x100000003a4/038.out v432>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.617 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.223+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000405 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/049 auth v436 ap=1 s=3031 n(v0 rc2024-04-24T09:08:59.027909+0000 b3031 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fede580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000405:[<0x100000003a4/049 v194>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000405:[<0x100000003a4/049 v436>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.617 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.224+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e0 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/030.out auth v430 ap=1 s=25 n(v0 rc2024-04-24T09:08:58.618812+0000 b25 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fedf080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e0:[<0x100000003a4/030.out v120>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003e0:[<0x100000003a4/030.out v430>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.617 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.226+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000046d [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/101 auth v466 ap=1 s=1540 n(v0 rc2024-04-24T09:09:00.365496+0000 b1540 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fee1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000046d:[<0x100000003a4/101 v314>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000046d:[<0x100000003a4/101 v466>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.617 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.226+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/017 auth v406 ap=1 s=1799 n(v0 rc2024-04-24T09:08:58.325900+0000 b1799 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fee8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c5:[<0x100000003a4/017 v66>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003c5:[<0x100000003a4/017 v406>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.618 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.226+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003dd [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/029 auth v428 ap=1 s=2411 n(v0 rc2024-04-24T09:08:58.586093+0000 b2411 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fee1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003dd:[<0x100000003a4/029 v114>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003dd:[<0x100000003a4/029 v428>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.618 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.227+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000425 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/065 auth v442 ap=1 s=2757 n(v0 rc2024-04-24T09:08:59.390261+0000 b2757 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fee8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000425:[<0x100000003a4/065 v258>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000425:[<0x100000003a4/065 v442>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.618 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.227+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000470 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/102.out auth v468 ap=1 s=43 n(v0 rc2024-04-24T09:09:00.393348+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fee8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000470:[<0x100000003a4/102.out v316>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000470:[<0x100000003a4/102.out v468>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.618 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.228+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000047e [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/109.out auth v472 ap=1 s=616 n(v0 rc2024-04-24T09:09:00.517722+0000 b616 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fee9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000047e:[<0x100000003a4/109.out v320>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000047e:[<0x100000003a4/109.out v472>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.618 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.228+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000042f [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/070 auth v448 ap=1 s=2641 n(v0 rc2024-04-24T09:08:59.499968+0000 b2641 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fee9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000042f:[<0x100000003a4/070 v278>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000042f:[<0x100000003a4/070 v448>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.618 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.228+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000456 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/089.out auth v460 ap=1 s=43 n(v0 rc2024-04-24T09:09:00.103993+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84feea100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000456:[<0x100000003a4/089.out v308>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000456:[<0x100000003a4/089.out v460>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.618 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.229+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000047a [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/107.out auth v470 ap=1 s=120 n(v0 rc2024-04-24T09:09:00.492135+0000 b120 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84feea680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000047a:[<0x100000003a4/107.out v318>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000047a:[<0x100000003a4/107.out v470>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.618 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.229+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000044b [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/084 auth v458 ap=1 s=3726 n(v0 rc2024-04-24T09:08:59.975237+0000 b3726 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84feeb180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000044b:[<0x100000003a4/084 v306>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000044b:[<0x100000003a4/084 v458>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.618 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.231+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003cc [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/020.out auth v408 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.395825+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fef4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003cc:[<0x100000003a4/020.out v80>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003cc:[<0x100000003a4/020.out v408>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.618 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.231+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000042e [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/069.out auth v446 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.489080+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fef4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000042e:[<0x100000003a4/069.out v276>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000042e:[<0x100000003a4/069.out v446>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.618 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.232+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000463 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/096 auth v462 ap=1 s=1846 n(v0 rc2024-04-24T09:09:00.256052+0000 b1846 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fef5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000463:[<0x100000003a4/096 v310>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000463:[<0x100000003a4/096 v462>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.619 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.232+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003af [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/006 auth v404 ap=1 s=2198 n(v0 rc2024-04-24T09:08:58.086885+0000 b2198 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fef5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003af:[<0x100000003a4/006 v22>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003af:[<0x100000003a4/006 v404>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.619 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.234+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000446 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/081.out auth v456 ap=1 s=10189 n(v0 rc2024-04-24T09:08:59.920282+0000 b10189 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fef6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000446:[<0x100000003a4/081.out v304>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000446:[<0x100000003a4/081.out v456>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.619 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.234+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000042d [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/069 auth v444 ap=1 s=2512 n(v0 rc2024-04-24T09:08:59.478440+0000 b2512 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fef6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000042d:[<0x100000003a4/069 v274>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000042d:[<0x100000003a4/069 v444>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.619 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.234+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000438 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/074.out auth v454 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.655772+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fef6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000438:[<0x100000003a4/074.out v296>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000438:[<0x100000003a4/074.out v454>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.619 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.235+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000040a [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/051.out auth v438 ap=1 s=25 n(v0 rc2024-04-24T09:08:59.081092+0000 b25 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fef7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000040a:[<0x100000003a4/051.out v204>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000040a:[<0x100000003a4/051.out v438>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.619 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.235+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003da [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/027.out auth v386 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.551281+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fef7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003da:[<0x100000003a4/027.out v108>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003da:[<0x100000003a4/027.out v386>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.619 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.236+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000414 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/056.out auth v410 ap=1 s=2419 n(v0 rc2024-04-24T09:08:59.191735+0000 b2419 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84ff02000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000414:[<0x100000003a4/056.out v224>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000414:[<0x100000003a4/056.out v410>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.619 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.236+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000422 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/063.out auth v418 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.358095+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84ff02580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000422:[<0x100000003a4/063.out v252>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000422:[<0x100000003a4/063.out v418>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.619 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.237+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ef [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/038 auth v404 ap=1 s=2917 n(v0 rc2024-04-24T09:08:58.784286+0000 b2917 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84ff03080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ef:[<0x100000003a4/038 v150>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003ef:[<0x100000003a4/038 v404>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.619 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.239+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c9 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/019 auth v382 ap=1 s=2496 n(v0 rc2024-04-24T09:08:58.361214+0000 b2496 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84ff03600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003c9:[<0x100000003a4/019 v74>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003c9:[<0x100000003a4/019 v382>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.619 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.239+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000045e [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/093.out auth v440 ap=1 s=1027 n(v0 rc2024-04-24T09:09:00.202390+0000 b1027 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84ff04680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000045e:[<0x100000003a4/093.out v306>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000045e:[<0x100000003a4/093.out v440>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.619 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.240+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000437 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/074 auth v436 ap=1 s=2248 n(v0 rc2024-04-24T09:08:59.644391+0000 b2248 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84ff05700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000437:[<0x100000003a4/074 v294>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000437:[<0x100000003a4/074 v436>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.620 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.240+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003cf [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/022 auth v384 ap=1 s=1078 n(v0 rc2024-04-24T09:08:58.427831+0000 b1078 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84ff05180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003cf:[<0x100000003a4/022 v86>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003cf:[<0x100000003a4/022 v384>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.620 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.241+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000403 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/048 auth v406 ap=1 s=8635 n(v0 rc2024-04-24T09:08:59.006285+0000 b8635 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502c2b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000403:[<0x100000003a4/048 v190>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000403:[<0x100000003a4/048 v406>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.620 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.241+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000469 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/099 auth v442 ap=1 s=1623 n(v0 rc2024-04-24T09:09:00.325236+0000 b1623 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502c2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000469:[<0x100000003a4/099 v308>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000469:[<0x100000003a4/099 v442>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.620 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.243+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003be [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/013.out auth v380 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.255926+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502c3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003be:[<0x100000003a4/013.out v52>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003be:[<0x100000003a4/013.out v380>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.620 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.243+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003eb [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/036 auth v402 ap=1 s=1543 n(v0 rc2024-04-24T09:08:58.740764+0000 b1543 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502c3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003eb:[<0x100000003a4/036 v142>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003eb:[<0x100000003a4/036 v402>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.620 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.244+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000434 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/072.out auth v434 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.613088+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502c3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000434:[<0x100000003a4/072.out v288>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000434:[<0x100000003a4/072.out v434>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.620 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.245+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000041d [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/061 auth v416 ap=1 s=2069 n(v0 rc2024-04-24T09:08:59.304069+0000 b2069 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502c5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000041d:[<0x100000003a4/061 v242>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000041d:[<0x100000003a4/061 v416>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.620 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.245+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000429 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/067 auth v420 ap=1 s=2449 n(v0 rc2024-04-24T09:08:59.434396+0000 b2449 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502ce000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000429:[<0x100000003a4/067 v266>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000429:[<0x100000003a4/067 v420>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.620 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.247+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000415 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/057 auth v412 ap=1 s=989 n(v0 rc2024-04-24T09:08:59.202220+0000 b989 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502cf600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000415:[<0x100000003a4/057 v226>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000415:[<0x100000003a4/057 v412>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.620 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.247+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000412 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/055.out auth v408 ap=1 s=9130 n(v0 rc2024-04-24T09:08:59.169648+0000 b9130 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502ceb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000412:[<0x100000003a4/055.out v220>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000412:[<0x100000003a4/055.out v408>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.620 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.249+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000449 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/083 auth v438 ap=1 s=4448 n(v0 rc2024-04-24T09:08:59.952778+0000 b4448 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502d0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000449:[<0x100000003a4/083 v304>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000449:[<0x100000003a4/083 v438>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.620 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.251+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000416 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/057.out auth v414 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.214529+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502d1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000416:[<0x100000003a4/057.out v228>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000416:[<0x100000003a4/057.out v414>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.620 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.253+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ab [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/004 auth v402 ap=1 s=6457 n(v0 rc2024-04-24T09:08:58.046746+0000 b6457 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502d6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ab:[<0x100000003a4/004 v14>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003ab:[<0x100000003a4/004 v402>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.620 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.254+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000408 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/050.out auth v438 ap=1 s=25 n(v0 rc2024-04-24T09:08:59.059925+0000 b25 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502d6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000408:[<0x100000003a4/050.out v200>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000408:[<0x100000003a4/050.out v438>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.621 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.255+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e4 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/032.out auth v428 ap=1 s=126 n(v0 rc2024-04-24T09:08:58.661860+0000 b126 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502d7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e4:[<0x100000003a4/032.out v128>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003e4:[<0x100000003a4/032.out v428>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.621 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.255+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003a6 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/001.out auth v400 ap=1 s=695 n(v0 rc2024-04-24T09:08:57.993037+0000 b695 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502d7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003a6:[<0x100000003a4/001.out v4>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003a6:[<0x100000003a4/001.out v400>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.621 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.256+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000450 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/086.out auth v468 ap=1 s=64 n(v0 rc2024-04-24T09:09:00.030827+0000 b64 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502d8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000450:[<0x100000003a4/086.out v308>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000450:[<0x100000003a4/086.out v468>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.621 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.256+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000474 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/104.out auth v478 ap=1 s=25 n(v0 rc2024-04-24T09:09:00.432713+0000 b25 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502d8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000474:[<0x100000003a4/104.out v318>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000474:[<0x100000003a4/104.out v478>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.621 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.259+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003d9 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/027 auth v404 ap=1 s=2430 n(v0 rc2024-04-24T09:08:58.538795+0000 b2430 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502d8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003d9:[<0x100000003a4/027 v106>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003d9:[<0x100000003a4/027 v404>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.621 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.260+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003e6 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/033.out auth v430 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.683696+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502d9700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003e6:[<0x100000003a4/033.out v132>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003e6:[<0x100000003a4/033.out v430>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.621 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.260+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f9 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/043 auth v436 ap=1 s=3518 n(v0 rc2024-04-24T09:08:58.894649+0000 b3518 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502d9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003f9:[<0x100000003a4/043 v170>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003f9:[<0x100000003a4/043 v436>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.621 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.260+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000439 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/075 auth v460 ap=1 s=1136 n(v0 rc2024-04-24T09:08:59.666859+0000 b1136 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502e4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000439:[<0x100000003a4/075 v298>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000439:[<0x100000003a4/075 v460>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.621 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.261+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000047b [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/108 auth v480 ap=1 s=2323 n(v0 rc2024-04-24T09:09:00.499428+0000 b2323 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502e4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000047b:[<0x100000003a4/108 v320>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000047b:[<0x100000003a4/108 v480>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1651>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.621 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.263+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000447 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/082 auth v466 ap=1 s=914 n(v0 rc2024-04-24T09:08:59.931622+0000 b914 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502e5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000447:[<0x100000003a4/082 v306>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000447:[<0x100000003a4/082 v466>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1653>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.621 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.264+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f1 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/039 auth v434 ap=1 s=3022 n(v0 rc2024-04-24T09:08:58.805768+0000 b3022 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502e6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003f1:[<0x100000003a4/039 v154>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003f1:[<0x100000003a4/039 v434>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1653>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.621 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.264+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003dc [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/028.out auth v426 ap=1 s=43 n(v0 rc2024-04-24T09:08:58.575505+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502e7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003dc:[<0x100000003a4/028.out v112>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003dc:[<0x100000003a4/028.out v426>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1653>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.622 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.264+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000043a [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/075.out auth v462 ap=1 s=43 n(v0 rc2024-04-24T09:08:59.677874+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502e6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000043a:[<0x100000003a4/075.out v300>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000043a:[<0x100000003a4/075.out v462>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1653>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.622 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.264+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000435 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/073 auth v458 ap=1 s=2117 n(v0 rc2024-04-24T09:08:59.624763+0000 b2117 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502e6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000435:[<0x100000003a4/073 v290>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000435:[<0x100000003a4/073 v458>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1653>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.622 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.266+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ed [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/037 auth v432 ap=1 s=3087 n(v0 rc2024-04-24T09:08:58.763593+0000 b3087 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502f2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000003ed:[<0x100000003a4/037 v146>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x100000003ed:[<0x100000003a4/037 v432>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1653>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.622 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.266+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000041a [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/059.out auth v440 ap=1 s=340 n(v0 rc2024-04-24T09:08:59.263603+0000 b340 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502f3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000041a:[<0x100000003a4/059.out v236>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000041a:[<0x100000003a4/059.out v440>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1653>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.622 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.266+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000462 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/095.out auth v474 ap=1 s=919 n(v0 rc2024-04-24T09:09:00.244377+0000 b919 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502f2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000462:[<0x100000003a4/095.out v314>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000462:[<0x100000003a4/095.out v474>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1653>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.622 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.267+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000046f [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/102 auth v476 ap=1 s=2074 n(v0 rc2024-04-24T09:09:00.383322+0000 b2074 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502f3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000046f:[<0x100000003a4/102 v316>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000046f:[<0x100000003a4/102 v476>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1653>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.622 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.268+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000461 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/095 auth v472 ap=1 s=5851 n(v0 rc2024-04-24T09:09:00.233505+0000 b5851 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502f4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000461:[<0x100000003a4/095 v312>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000461:[<0x100000003a4/095 v472>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1653>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.622 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.268+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000427 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/066 auth v442 ap=1 s=2290 n(v0 rc2024-04-24T09:08:59.411942+0000 b2290 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502f4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000427:[<0x100000003a4/066 v262>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000427:[<0x100000003a4/066 v442>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1653>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.622 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.268+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000045c [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/092.out auth v470 ap=1 s=43 n(v0 rc2024-04-24T09:09:00.177368+0000 b43 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502f5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000045c:[<0x100000003a4/092.out v310>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x1000000045c:[<0x100000003a4/092.out v470>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1653>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.622 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.269+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000445 [4,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/081 auth v464 ap=1 s=3290 n(v0 rc2024-04-24T09:08:59.910469+0000 b3290 1=1+0) caps={15489=pAsLsXsFsc/-@1},l=15489(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a8502f5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000445:[<0x100000003a4/081 v304>,<0x100000003a2/btrfs v448>,<0x10000000007/tests v892>,<0x10000000006/xfstests-dev v136>,<0x10000000005/tmp v30>,<0x10000000003/client.0 v31>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v42>,<0x10000000001/sv_0 v66>,<0x10000000000/qa v55>,<0x1/volumes v53>]//[3]","memoryvalue":"(4)0x10000000445:[<0x100000003a4/081 v464>,<0x100000003a2/btrfs v3934>,<0x10000000007/tests v1653>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.622 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.269+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-commit.sample auth v36 ap=1 s=1647 n(v0 rc2024-04-24T09:08:45.143443+0000 b1647 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fdf2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000020a:[<0x10000000206/pre-commit.sample v8>,<0x10000000008/hooks v38>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[3]","memoryvalue":"(4)0x1000000020a:[<0x10000000206/pre-commit.sample v36>,<0x10000000008/hooks v177>,<0x10000000007/.git v1641>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.622 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.269+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000212 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/commit-msg.sample auth v52 ap=1 s=900 n(v0 rc2024-04-24T09:08:45.254460+0000 b900 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fdf2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000212:[<0x10000000206/commit-msg.sample v24>,<0x10000000008/hooks v38>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[3]","memoryvalue":"(4)0x10000000212:[<0x10000000206/commit-msg.sample v52>,<0x10000000008/hooks v177>,<0x10000000007/.git v1641>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.623 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.270+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020b [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-merge-commit.sample auth v38 ap=1 s=420 n(v0 rc2024-04-24T09:08:45.155289+0000 b420 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fdf1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000020b:[<0x10000000206/pre-merge-commit.sample v10>,<0x10000000008/hooks v38>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[3]","memoryvalue":"(4)0x1000000020b:[<0x10000000206/pre-merge-commit.sample v38>,<0x10000000008/hooks v177>,<0x10000000007/.git v1641>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.623 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.270+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020f [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/fsmonitor-watchman.sample auth v46 ap=1 s=4726 n(v0 rc2024-04-24T09:08:45.210530+0000 b4726 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fdf2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000020f:[<0x10000000206/fsmonitor-watchman.sample v18>,<0x10000000008/hooks v38>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[3]","memoryvalue":"(4)0x1000000020f:[<0x10000000206/fsmonitor-watchman.sample v46>,<0x10000000008/hooks v177>,<0x10000000007/.git v1641>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.623 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.270+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000214 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-rebase.sample auth v56 ap=1 s=4902 n(v0 rc2024-04-24T09:08:45.284285+0000 b4902 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fdf3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000214:[<0x10000000206/pre-rebase.sample v28>,<0x10000000008/hooks v38>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[3]","memoryvalue":"(4)0x10000000214:[<0x10000000206/pre-rebase.sample v56>,<0x10000000008/hooks v177>,<0x10000000007/.git v1641>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.623 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.271+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/update.sample auth v40 ap=1 s=3654 n(v0 rc2024-04-24T09:08:45.166206+0000 b3654 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe06580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000020c:[<0x10000000206/update.sample v12>,<0x10000000008/hooks v38>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[3]","memoryvalue":"(4)0x1000000020c:[<0x10000000206/update.sample v40>,<0x10000000008/hooks v177>,<0x10000000007/.git v1641>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.623 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.271+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/prepare-commit-msg.sample auth v42 ap=1 s=1496 n(v0 rc2024-04-24T09:08:45.178754+0000 b1496 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe06000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000020d:[<0x10000000206/prepare-commit-msg.sample v14>,<0x10000000008/hooks v38>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[3]","memoryvalue":"(4)0x1000000020d:[<0x10000000206/prepare-commit-msg.sample v42>,<0x10000000008/hooks v177>,<0x10000000007/.git v1641>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.623 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.271+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000209 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/sendemail-validate.sample auth v34 ap=1 s=2312 n(v0 rc2024-04-24T09:08:45.129596+0000 b2312 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe07080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000209:[<0x10000000206/sendemail-validate.sample v6>,<0x10000000008/hooks v38>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[3]","memoryvalue":"(4)0x10000000209:[<0x10000000206/sendemail-validate.sample v34>,<0x10000000008/hooks v177>,<0x10000000007/.git v1641>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.623 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.271+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020e [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-receive.sample auth v44 ap=1 s=548 n(v0 rc2024-04-24T09:08:45.196270+0000 b548 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fdf3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000020e:[<0x10000000206/pre-receive.sample v16>,<0x10000000008/hooks v38>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[3]","memoryvalue":"(4)0x1000000020e:[<0x10000000206/pre-receive.sample v44>,<0x10000000008/hooks v177>,<0x10000000007/.git v1641>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.623 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.272+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000210 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-push.sample auth v48 ap=1 s=1378 n(v0 rc2024-04-24T09:08:45.225046+0000 b1378 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe06b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000210:[<0x10000000206/pre-push.sample v20>,<0x10000000008/hooks v38>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[3]","memoryvalue":"(4)0x10000000210:[<0x10000000206/pre-push.sample v48>,<0x10000000008/hooks v177>,<0x10000000007/.git v1641>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.623 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.272+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000207 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/push-to-checkout.sample auth v30 ap=1 s=2787 n(v0 rc2024-04-24T09:08:45.100552+0000 b2787 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe07b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000207:[<0x10000000206/push-to-checkout.sample v2>,<0x10000000008/hooks v38>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[3]","memoryvalue":"(4)0x10000000207:[<0x10000000206/push-to-checkout.sample v30>,<0x10000000008/hooks v177>,<0x10000000007/.git v1641>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.623 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.272+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000211 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-applypatch.sample auth v50 ap=1 s=428 n(v0 rc2024-04-24T09:08:45.239987+0000 b428 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe07600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000211:[<0x10000000206/pre-applypatch.sample v22>,<0x10000000008/hooks v38>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[3]","memoryvalue":"(4)0x10000000211:[<0x10000000206/pre-applypatch.sample v50>,<0x10000000008/hooks v177>,<0x10000000007/.git v1641>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.623 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.272+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000213 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/applypatch-msg.sample auth v54 ap=1 s=482 n(v0 rc2024-04-24T09:08:45.268239+0000 b482 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe08680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000213:[<0x10000000206/applypatch-msg.sample v26>,<0x10000000008/hooks v38>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[3]","memoryvalue":"(4)0x10000000213:[<0x10000000206/applypatch-msg.sample v54>,<0x10000000008/hooks v177>,<0x10000000007/.git v1641>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.623 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:09:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:09:29.273+0000 7f84a5800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [3,head] /volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/post-update.sample auth v32 ap=1 s=193 n(v0 rc2024-04-24T09:08:45.115288+0000 b193 1=1+0) caps={15489=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55a84fe08100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000208:[<0x10000000206/post-update.sample v4>,<0x10000000008/hooks v38>,<0x10000000007/.git v108>,<0x10000000006/xfstests-dev v6>,<0x10000000005/tmp v6>,<0x10000000003/client.0 v8>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v24>,<0x10000000001/sv_0 v48>,<0x10000000000/qa v37>,<0x1/volumes v35>]//[3]","memoryvalue":"(4)0x10000000208:[<0x10000000206/post-update.sample v32>,<0x10000000008/hooks v177>,<0x10000000007/.git v1641>,<0x10000000006/xfstests-dev v168>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v33>,<0x10000000002/65ebee0b-37c1-4c5d-bbf7-ad639a509f68 v62>,<0x10000000001/sv_0 v34>,<0x10000000000/qa v71>,<0x1/volumes v25>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-24T09:09:29.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: pgmap v75: 129 pgs: 129 active+clean; 30 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 479 KiB/s wr, 88 op/s 2024-04-24T09:09:29.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: scrub summary: active paths [/] 2024-04-24T09:09:29.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000334 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/md5.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000030f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fiemap-fault.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002db (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000340 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/parse-extent-tree.awk) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000360 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/statx.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000353 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/renameat2.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000367 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_offset2.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000037b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_mtab.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000323 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/helpers.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000030c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fake-dump-rootino.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000321 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/global.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000034b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/popdir.pl) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002f4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/attr_replace_test.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000311 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fiexchange.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000035f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/stat_test.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002ff (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/detached_mounts_propagation.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000351 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/readdir-while-renames.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000033a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mmapcat.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000033b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/multi_open_unlink.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000325 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/holetest.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000349 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/permname.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000326 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/iopat.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000308 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/dumpfile) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000390 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003a1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/xfsfind.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000313 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fill2.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000312 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fill.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000329 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/locktest.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000337 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mkswap.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002fe (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/deduperace.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000370 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_holes.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000368 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_type.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000037a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_mmap_writev_overlap.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000366 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_offset.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000338 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mmap-rw-fault.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002e0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-cow-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002ed (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aiocp.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002e2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-eof-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002e5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-hole-filling-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002ee (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aiodio_sparse2.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002dc (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/Makefile) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002ea (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-free-ring-with-bogus-nr-pages.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002ec (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-last-ref-held-by-io.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002e8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-subblock-eof-read.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002e9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-write-verify.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002e4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-fcntl-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002eb (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-io-setup-with-nonwritable-context-pointer.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002df (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-append-write-read-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002e1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-cycle-write.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002e3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-extend-stat.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002e6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-invalidate-failure.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002dd (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/SOURCE) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002e7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-invalidate-readahead.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000002de (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-append-write-fallocate-race.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000023f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/index.lock) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000236 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/HEAD) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000009 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/description) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000021f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000231 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/config) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000203 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/info) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000022d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/packed-refs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000022b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/shallow) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000205 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/branches) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000039c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/vfstest.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000397 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/mount-idmapped.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000391 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/Makefile) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000398 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/tmpfs-idmapped-mounts.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000039d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/vfstest.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000394 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/idmapped-mounts.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000395 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/idmapped-mounts.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000396 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/missing.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000399 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/tmpfs-idmapped-mounts.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000393 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/btrfs-idmapped-mounts.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000392 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/btrfs-idmapped-mounts.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000039a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/utils.c) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000039b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/utils.h) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000280 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/repair) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000026e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filter.btrfs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000265 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmlogwrites) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000281 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/report) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000025b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/casefold) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000278 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/perf) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000026b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/fail_make_request) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000262 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmerror) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000270 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/gcov) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000282 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/scsi_debug) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000287 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/xfs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000286 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/verity) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000026c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filestreams) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000269 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ext4) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000268 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/encrypt) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000027a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/preamble) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000264 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmhugedisk) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000025e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/config) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000272 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/locktest) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000027d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/rc) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000025d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/cgroup2) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000267 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dump) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000288 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/zoned) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000027f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/renameat2) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000025c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ceph) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000258 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/Makefile) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000274 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/metadump) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000027b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/punch) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000263 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmflakey) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000276 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/nfs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000285 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ubifs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000026d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filter) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000260 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmdelay) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000275 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/module) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000025a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/btrfs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000027e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/reflink) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000026f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/fuzzy) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000283 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/test_names) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000261 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmdust) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000259 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/attr) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000027c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/quota) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000266 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmthin) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000271 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/inject) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000273 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/log) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000279 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/populate) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000277 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/overlay) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000026a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/f2fs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects/info) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000025f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/defrag) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000284 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/tracing) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000220 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects/pack) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000239 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/HEAD) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000232 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000204 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/info/exclude) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000216 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs/heads) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000217 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs/tags) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000023a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/heads) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000233 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000234 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000023b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/heads/master) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000235 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin/HEAD) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000206 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks) see mds.e log and `damage ls` output for details 2024-04-24T09:09:29.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:29 smithi130 ceph-mon[31553]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:30.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: scrub queued for path: / 2024-04-24T09:09:30.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:30.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: scrub summary: active paths [/] 2024-04-24T09:09:30.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: scrub summary: active paths [/] 2024-04-24T09:09:30.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003a3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/Makefile) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003a4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:30.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003a9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/003) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000043f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/078) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003ee (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/037.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000042a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/067.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003e1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/031) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003f6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/041.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.569 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003db (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/028) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003e8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/034.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000400 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/046.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003fa (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/043.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003c8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/018.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003ea (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/035.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003e7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/034) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003cb (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/020) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003bb (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/012) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000451 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/087) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000478 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/106.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/040) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000044f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/086) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000475 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/105) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000480 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/110.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000047d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/109) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003f4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/040.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003c3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/016) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000477 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/106) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/004.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003b4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/008.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003ba (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/011.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003a7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/002) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000466 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/097.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003ae (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/005.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.571 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000043e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/077.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.572 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003ca (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/019.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.572 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000436 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/073.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.572 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000407 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/050) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.572 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003df (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/030) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.572 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003fc (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/044.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.572 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003f7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/042) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000045f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/094) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000045a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/091.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003bc (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/012.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003aa (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/003.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000431 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/071) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000424 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/064.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003c4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/016.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003b1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/007) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003d1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/023) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003e3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/032) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003e5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/033) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000464 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/096.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.573 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000468 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/098.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003ff (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/046) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003b5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/009) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000432 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/071.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003fe (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/045.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000404 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/048.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003b8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/010.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000044a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/083.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000043b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/076) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000045b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/092) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003d8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/026.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003bd (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/013) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003d6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/025.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000041c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/060.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003c1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/015) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.574 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000047f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/110) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000046a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/099.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000423 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/064) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000043c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/076.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003d7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/026) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000457 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/090) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000453 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/088) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000413 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/056) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000040c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/052.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003f8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/042.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000472 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/103.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000454 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/088.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003d4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/024.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003c0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/014.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.575 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000047c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/108.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000409 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/051) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000042b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/068) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003d2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/023.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000430 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/070.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003a5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/001) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000455 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/089) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000448 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/082.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003b3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/008) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003b2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/007.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000042c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/068.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000467 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/098) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003d3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/024) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000420 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/062.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.576 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000421 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/063) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.577 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000411 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/055) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.577 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000045d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/093) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.577 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000428 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/066.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.577 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000459 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/091) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.577 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000406 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/049.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.577 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000443 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/080) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.577 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000473 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/104) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.577 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000044e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/085.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.577 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000465 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/097) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.577 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000410 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/054.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000040e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/053.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003fd (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/045) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000442 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/079.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000041e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/061.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000044c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/084.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003b7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/010) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003b9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/011) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000444 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/080.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000041b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/060) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000046b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/100) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003d5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/025) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000452 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/087.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000433 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/072) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.579 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003a8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/002.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000046e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/101.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003ec (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/036.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000040f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/054) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000040b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/052) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000417 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/058) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003d0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/022.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003f5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/041) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000476 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/105.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003b6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/009.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003b0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/006.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000043d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/077) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000458 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/090.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003ad (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/005) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.580 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000481 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/111) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003cd (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/021) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000471 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/103) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000044d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/085) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000040d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/053) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003de (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/029.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000441 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/079) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000440 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/078.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003c6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/017.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000402 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/047.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000401 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/047) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000418 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/058.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003e9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/035) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/018) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003fb (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/044) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.581 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003bf (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/014) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000041f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/062) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003e2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/031.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000460 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/094.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000479 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/107) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003c2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/015.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003ce (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/021.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000426 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/065.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000046c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/100.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003f2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/039.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000419 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/059) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003f0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/038.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000405 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/049) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003e0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/030.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.582 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000046d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/101) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003c5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/017) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003dd (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/029) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000425 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/065) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000470 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/102.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000047e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/109.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000042f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/070) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000456 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/089.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000047a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/107.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000044b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/084) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003cc (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/020.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000042e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/069.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000463 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/096) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003af (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/006) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000446 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/081.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.583 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000042d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/069) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.584 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000438 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/074.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.584 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000040a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/051.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.584 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003da (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/027.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.584 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000414 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/056.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.584 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000422 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/063.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.584 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003ef (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/038) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.584 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003c9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/019) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.584 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000045e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/093.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.584 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000437 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/074) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.584 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003cf (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/022) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.584 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000403 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/048) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.584 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000469 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/099) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.584 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003be (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/013.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/036) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000434 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/072.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000041d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/061) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000429 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/067) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000415 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/057) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000412 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/055.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000449 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/083) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000416 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/057.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003ab (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/004) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000408 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/050.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003e4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/032.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003a6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/001.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000450 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/086.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000474 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/104.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.585 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003d9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/027) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003e6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/033.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003f9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/043) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000439 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/075) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000047b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/108) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000447 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/082) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003f1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/039) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003dc (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/028.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000043a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/075.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000435 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/073) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x100000003ed (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/037) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000041a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/059.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000462 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/095.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.586 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000046f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/102) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.587 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000461 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/095) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.587 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000427 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/066) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.587 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000045c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/092.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.587 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000445 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/081) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.587 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-commit.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.587 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/commit-msg.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.587 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-merge-commit.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.587 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/fsmonitor-watchman.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.587 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-rebase.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.587 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/update.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.587 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/prepare-commit-msg.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.587 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/sendemail-validate.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.587 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-receive.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.587 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-push.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.587 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000207 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/push-to-checkout.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.588 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-applypatch.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.588 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/applypatch-msg.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.588 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/post-update.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.588 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:30 smithi081 ceph-mon[30961]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:30.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a[28166]: 2024-04-24T09:09:30.290+0000 7fe78aa00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T09:09:30.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: scrub queued for path: / 2024-04-24T09:09:30.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:30.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: scrub summary: active paths [/] 2024-04-24T09:09:30.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: scrub summary: active paths [/] 2024-04-24T09:09:30.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003a3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/Makefile) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003a4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:30.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003a9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/003) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000043f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/078) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003ee (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/037.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000042a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/067.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003e1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/031) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003f6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/041.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003db (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/028) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003e8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/034.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000400 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/046.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003fa (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/043.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003c8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/018.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003ea (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/035.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.593 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003e7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/034) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003cb (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/020) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003bb (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/012) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000451 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/087) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000478 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/106.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/040) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000044f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/086) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000475 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/105) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000480 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/110.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000047d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/109) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003f4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/040.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003c3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/016) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.594 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000477 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/106) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/004.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003b4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/008.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003ba (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/011.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003a7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/002) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000466 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/097.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003ae (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/005.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000043e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/077.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003ca (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/019.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000436 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/073.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000407 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/050) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003df (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/030) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003fc (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/044.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003f7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/042) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000045f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/094) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000045a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/091.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003bc (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/012.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003aa (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/003.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000431 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/071) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000424 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/064.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003c4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/016.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003b1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/007) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003d1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/023) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003e3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/032) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.596 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003e5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/033) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000464 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/096.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000468 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/098.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003ff (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/046) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003b5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/009) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000432 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/071.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003fe (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/045.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000404 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/048.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003b8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/010.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000044a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/083.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000043b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/076) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000045b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/092) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003d8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/026.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003bd (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/013) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003d6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/025.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000041c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/060.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003c1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/015) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000047f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/110) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000046a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/099.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000423 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/064) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000043c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/076.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003d7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/026) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000457 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/090) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000453 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/088) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000413 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/056) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000040c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/052.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003f8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/042.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000472 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/103.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000454 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/088.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003d4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/024.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003c0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/014.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000047c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/108.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000409 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/051) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000042b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/068) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003d2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/023.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000430 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/070.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003a5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/001) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000455 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/089) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000448 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/082.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003b3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/008) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003b2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/007.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000042c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/068.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000467 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/098) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003d3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/024) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000420 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/062.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000421 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/063) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000411 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/055) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000045d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/093) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000428 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/066.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000459 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/091) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000406 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/049.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000443 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/080) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000473 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/104) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000044e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/085.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.600 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000465 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/097) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000410 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/054.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000040e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/053.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003fd (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/045) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.601 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000442 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/079.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000041e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/061.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000044c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/084.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003b7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/010) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003b9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/011) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000444 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/080.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000041b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/060) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000046b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/100) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003d5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/025) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000452 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/087.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000433 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/072) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003a8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/002.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000046e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/101.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003ec (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/036.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.602 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000040f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/054) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000040b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/052) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000417 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/058) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003d0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/022.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003f5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/041) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000476 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/105.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003b6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/009.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003b0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/006.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000043d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/077) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000458 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/090.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003ad (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/005) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000481 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/111) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003cd (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/021) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000471 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/103) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000044d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/085) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.603 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000040d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/053) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003de (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/029.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000441 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/079) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000440 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/078.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003c6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/017.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000402 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/047.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000401 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/047) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000418 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/058.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003e9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/035) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/018) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003fb (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/044) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003bf (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/014) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000041f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/062) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003e2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/031.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000460 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/094.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000479 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/107) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003c2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/015.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003ce (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/021.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000426 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/065.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000046c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/100.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003f2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/039.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000419 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/059) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003f0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/038.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000405 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/049) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003e0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/030.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000046d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/101) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003c5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/017) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003dd (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/029) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000425 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/065) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000470 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/102.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000047e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/109.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000042f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/070) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000456 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/089.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000047a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/107.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000044b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/084) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003cc (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/020.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000042e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/069.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000463 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/096) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003af (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/006) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.606 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000446 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/081.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000042d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/069) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000438 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/074.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000040a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/051.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003da (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/027.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000414 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/056.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000422 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/063.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.607 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003ef (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/038) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003c9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/019) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000045e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/093.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000437 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/074) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003cf (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/022) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000403 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/048) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000469 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/099) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003be (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/013.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/036) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000434 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/072.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000041d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/061) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000429 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/067) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000415 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/057) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000412 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/055.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000449 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/083) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000416 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/057.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003ab (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/004) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000408 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/050.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003e4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/032.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003a6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/001.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000450 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/086.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000474 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/104.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003d9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/027) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003e6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/033.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003f9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/043) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000439 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/075) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000047b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/108) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000447 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/082) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003f1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/039) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003dc (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/028.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000043a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/075.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000435 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/073) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x100000003ed (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/037) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000041a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/059.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000462 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/095.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000046f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/102) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000461 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/095) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000427 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/066) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000045c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/092.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000445 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/081) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-commit.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/commit-msg.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-merge-commit.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/fsmonitor-watchman.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-rebase.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/update.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/prepare-commit-msg.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/sendemail-validate.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-receive.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-push.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000207 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/push-to-checkout.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-applypatch.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/applypatch-msg.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/post-update.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:30 smithi069 ceph-mon[28170]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:30.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: scrub queued for path: / 2024-04-24T09:09:30.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:30.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: scrub summary: active paths [/] 2024-04-24T09:09:30.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: scrub summary: active paths [/] 2024-04-24T09:09:30.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003a3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/Makefile) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003a4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:30.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003a9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/003) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000043f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/078) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003ee (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/037.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000042a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/067.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003e1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/031) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003f6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/041.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003db (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/028) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003e8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/034.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000400 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/046.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003fa (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/043.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003c8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/018.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003ea (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/035.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003e7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/034) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003cb (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/020) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003bb (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/012) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000451 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/087) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000478 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/106.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/040) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000044f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/086) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000475 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/105) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000480 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/110.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000047d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/109) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003f4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/040.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003c3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/016) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000477 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/106) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.707 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/004.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003b4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/008.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003ba (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/011.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003a7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/002) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000466 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/097.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003ae (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/005.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000043e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/077.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.708 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003ca (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/019.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000436 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/073.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000407 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/050) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003df (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/030) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003fc (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/044.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003f7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/042) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000045f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/094) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000045a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/091.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003bc (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/012.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003aa (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/003.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000431 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/071) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000424 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/064.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003c4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/016.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.709 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003b1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/007) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003d1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/023) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003e3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/032) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003e5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/033) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000464 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/096.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000468 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/098.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003ff (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/046) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003b5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/009) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000432 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/071.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003fe (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/045.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000404 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/048.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003b8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/010.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000044a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/083.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000043b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/076) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.710 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000045b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/092) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003d8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/026.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003bd (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/013) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003d6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/025.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000041c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/060.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003c1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/015) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000047f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/110) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000046a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/099.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.711 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000423 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/064) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000043c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/076.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003d7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/026) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000457 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/090) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000453 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/088) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000413 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/056) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000040c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/052.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003f8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/042.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000472 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/103.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.712 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000454 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/088.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003d4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/024.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003c0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/014.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000047c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/108.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000409 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/051) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000042b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/068) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003d2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/023.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000430 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/070.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003a5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/001) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000455 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/089) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000448 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/082.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003b3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/008) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003b2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/007.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000042c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/068.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.713 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000467 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/098) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003d3 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/024) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000420 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/062.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000421 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/063) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000411 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/055) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000045d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/093) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000428 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/066.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000459 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/091) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000406 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/049.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000443 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/080) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000473 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/104) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000044e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/085.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000465 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/097) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000410 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/054.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000040e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/053.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003fd (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/045) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000442 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/079.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000041e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/061.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000044c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/084.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003b7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/010) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003b9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/011) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000444 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/080.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000041b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/060) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000046b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/100) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003d5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/025) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000452 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/087.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000433 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/072) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003a8 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/002.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000046e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/101.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003ec (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/036.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000040f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/054) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000040b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/052) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000417 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/058) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003d0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/022.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003f5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/041) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000476 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/105.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003b6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/009.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003b0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/006.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000043d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/077) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000458 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/090.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003ad (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/005) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000481 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/111) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003cd (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/021) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000471 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/103) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000044d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/085) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000040d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/053) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003de (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/029.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000441 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/079) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000440 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/078.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003c6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/017.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000402 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/047.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000401 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/047) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000418 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/058.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003e9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/035) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003c7 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/018) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003fb (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/044) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003bf (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/014) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000041f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/062) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003e2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/031.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000460 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/094.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000479 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/107) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003c2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/015.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003ce (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/021.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000426 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/065.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000046c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/100.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003f2 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/039.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000419 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/059) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003f0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/038.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.719 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000405 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/049) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.719 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003e0 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/030.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.719 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000046d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/101) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.719 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003c5 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/017) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.719 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003dd (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/029) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000425 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/065) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000470 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/102.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000047e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/109.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000042f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/070) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000456 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/089.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000047a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/107.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000044b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/084) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003cc (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/020.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000042e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/069.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000463 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/096) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003af (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/006) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000446 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/081.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.720 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000042d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/069) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000438 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/074.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000040a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/051.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003da (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/027.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000414 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/056.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000422 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/063.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003ef (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/038) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003c9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/019) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000045e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/093.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000437 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/074) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003cf (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/022) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000403 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/048) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000469 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/099) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003be (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/013.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/036) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.721 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000434 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/072.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000041d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/061) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000429 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/067) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000415 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/057) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000412 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/055.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000449 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/083) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000416 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/057.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003ab (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/004) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000408 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/050.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003e4 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/032.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003a6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/001.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000450 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/086.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000474 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/104.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.722 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003d9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/027) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003e6 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/033.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003f9 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/043) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000439 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/075) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000047b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/108) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000447 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/082) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003f1 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/039) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003dc (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/028.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000043a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/075.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000435 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/073) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x100000003ed (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/037) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000041a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/059.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000462 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/095.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000046f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/102) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.723 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000461 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/095) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000427 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/066) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000045c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/092.out) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000445 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/081) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-commit.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/commit-msg.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-merge-commit.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/fsmonitor-watchman.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-rebase.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/update.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/prepare-commit-msg.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/sendemail-validate.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-receive.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-push.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.724 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000207 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/push-to-checkout.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.725 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-applypatch.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.725 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/applypatch-msg.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.725 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/post-update.sample) see mds.h log and `damage ls` output for details 2024-04-24T09:09:30.725 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:30 smithi130 ceph-mon[31553]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:31.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:31 smithi081 ceph-mon[30961]: pgmap v76: 129 pgs: 129 active+clean; 33 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 721 KiB/s wr, 194 op/s 2024-04-24T09:09:31.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:31 smithi081 ceph-mon[30961]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T09:09:31.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:31 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] up:active 2024-04-24T09:09:31.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:31 smithi081 ceph-mon[30961]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:09:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:31 smithi069 ceph-mon[28170]: pgmap v76: 129 pgs: 129 active+clean; 33 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 721 KiB/s wr, 194 op/s 2024-04-24T09:09:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:31 smithi069 ceph-mon[28170]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T09:09:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:31 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] up:active 2024-04-24T09:09:31.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:31 smithi069 ceph-mon[28170]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:09:31.605 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:31 smithi130 ceph-mon[31553]: pgmap v76: 129 pgs: 129 active+clean; 33 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 721 KiB/s wr, 194 op/s 2024-04-24T09:09:31.606 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:31 smithi130 ceph-mon[31553]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T09:09:31.606 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:31 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] up:active 2024-04-24T09:09:31.606 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:31 smithi130 ceph-mon[31553]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:09:32.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:32 smithi130 ceph-mon[31553]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:09:32.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:32 smithi081 ceph-mon[30961]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:09:32.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:32 smithi069 ceph-mon[28170]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:09:33.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:33 smithi130 ceph-mon[31553]: pgmap v77: 129 pgs: 129 active+clean; 33 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 533 KiB/s wr, 147 op/s 2024-04-24T09:09:33.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:33 smithi081 ceph-mon[30961]: pgmap v77: 129 pgs: 129 active+clean; 33 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 533 KiB/s wr, 147 op/s 2024-04-24T09:09:33.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:33 smithi069 ceph-mon[28170]: pgmap v77: 129 pgs: 129 active+clean; 33 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 533 KiB/s wr, 147 op/s 2024-04-24T09:09:34.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:34 smithi130 ceph-mon[31553]: mds.? [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] up:active 2024-04-24T09:09:34.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:34 smithi130 ceph-mon[31553]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:09:34.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:34 smithi081 ceph-mon[30961]: mds.? [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] up:active 2024-04-24T09:09:34.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:34 smithi081 ceph-mon[30961]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:09:34.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:34 smithi069 ceph-mon[28170]: mds.? [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] up:active 2024-04-24T09:09:34.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:34 smithi069 ceph-mon[28170]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:09:35.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:35 smithi130 ceph-mon[31553]: pgmap v78: 129 pgs: 129 active+clean; 33 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 533 KiB/s wr, 147 op/s 2024-04-24T09:09:35.718 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:35 smithi081 ceph-mon[30961]: pgmap v78: 129 pgs: 129 active+clean; 33 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 533 KiB/s wr, 147 op/s 2024-04-24T09:09:35.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:35 smithi069 ceph-mon[28170]: pgmap v78: 129 pgs: 129 active+clean; 33 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 533 KiB/s wr, 147 op/s 2024-04-24T09:09:37.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:37 smithi130 ceph-mon[31553]: pgmap v79: 129 pgs: 129 active+clean; 36 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 843 KiB/s wr, 196 op/s 2024-04-24T09:09:37.749 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:37 smithi069 ceph-mon[28170]: pgmap v79: 129 pgs: 129 active+clean; 36 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 843 KiB/s wr, 196 op/s 2024-04-24T09:09:37.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:37 smithi081 ceph-mon[30961]: pgmap v79: 129 pgs: 129 active+clean; 36 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 843 KiB/s wr, 196 op/s 2024-04-24T09:09:38.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:37 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a[28166]: 2024-04-24T09:09:37.748+0000 7fe78aa00700 -1 log_channel(cluster) log [ERR] : Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T09:09:38.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:38 smithi130 ceph-mon[31553]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T09:09:38.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:38 smithi081 ceph-mon[30961]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T09:09:38.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:38 smithi069 ceph-mon[28170]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-24T09:09:39.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:39 smithi130 ceph-mon[31553]: pgmap v80: 129 pgs: 129 active+clean; 36 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 553 KiB/s wr, 155 op/s 2024-04-24T09:09:39.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:39 smithi081 ceph-mon[30961]: pgmap v80: 129 pgs: 129 active+clean; 36 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 553 KiB/s wr, 155 op/s 2024-04-24T09:09:39.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:39 smithi069 ceph-mon[28170]: pgmap v80: 129 pgs: 129 active+clean; 36 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 553 KiB/s wr, 155 op/s 2024-04-24T09:09:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.838+0000 7fb096c00640 1 -- 172.21.15.69:0/1391236463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb090104e10 msgr2=0x7fb090108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.838+0000 7fb096c00640 1 --2- 172.21.15.69:0/1391236463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb090104e10 0x7fb090108e50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb078009900 tx=0x7fb07802a410 comp rx=0 tx=0).stop 2024-04-24T09:09:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.838+0000 7fb096c00640 1 -- 172.21.15.69:0/1391236463 shutdown_connections 2024-04-24T09:09:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.838+0000 7fb096c00640 1 --2- 172.21.15.69:0/1391236463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb090109530 0x7fb090111100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.838+0000 7fb096c00640 1 --2- 172.21.15.69:0/1391236463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb090104e10 0x7fb090108e50 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.838+0000 7fb096c00640 1 --2- 172.21.15.69:0/1391236463 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb090104460 0x7fb090104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.838+0000 7fb096c00640 1 -- 172.21.15.69:0/1391236463 >> 172.21.15.69:0/1391236463 conn(0x7fb0900ffd00 msgr2=0x7fb090102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.838+0000 7fb096c00640 1 -- 172.21.15.69:0/1391236463 shutdown_connections 2024-04-24T09:09:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.838+0000 7fb096c00640 1 -- 172.21.15.69:0/1391236463 wait complete. 2024-04-24T09:09:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.839+0000 7fb096c00640 1 Processor -- start 2024-04-24T09:09:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.839+0000 7fb096c00640 1 -- start start 2024-04-24T09:09:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.840+0000 7fb096c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb090104460 0x7fb0901a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.840+0000 7fb096c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb090109530 0x7fb0901a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.840+0000 7fb096c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb09019a880 0x7fb09019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.840+0000 7fb096c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb090114520 con 0x7fb090109530 2024-04-24T09:09:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.840+0000 7fb096c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb0901143a0 con 0x7fb09019a880 2024-04-24T09:09:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.840+0000 7fb096c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb0901146a0 con 0x7fb090104460 2024-04-24T09:09:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.840+0000 7fb08f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb090109530 0x7fb0901a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.840+0000 7fb08f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb090109530 0x7fb0901a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41266/0 (socket says 172.21.15.69:41266) 2024-04-24T09:09:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.840+0000 7fb08f400640 1 -- 172.21.15.69:0/780358585 learned_addr learned my addr 172.21.15.69:0/780358585 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.840+0000 7fb08fe00640 1 --2- 172.21.15.69:0/780358585 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb090104460 0x7fb0901a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.840+0000 7fb094c00640 1 --2- 172.21.15.69:0/780358585 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb09019a880 0x7fb09019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.840+0000 7fb08f400640 1 -- 172.21.15.69:0/780358585 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb090104460 msgr2=0x7fb0901a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.840+0000 7fb08f400640 1 --2- 172.21.15.69:0/780358585 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb090104460 0x7fb0901a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.840+0000 7fb08f400640 1 -- 172.21.15.69:0/780358585 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb09019a880 msgr2=0x7fb09019ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.840+0000 7fb08f400640 1 --2- 172.21.15.69:0/780358585 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb09019a880 0x7fb09019ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.840+0000 7fb08f400640 1 -- 172.21.15.69:0/780358585 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09019b3a0 con 0x7fb090109530 2024-04-24T09:09:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.841+0000 7fb08f400640 1 --2- 172.21.15.69:0/780358585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb090109530 0x7fb0901a0c10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb078005ab0 tx=0x7fb078003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.841+0000 7fb08cc00640 1 -- 172.21.15.69:0/780358585 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07803f070 con 0x7fb090109530 2024-04-24T09:09:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.841+0000 7fb08cc00640 1 -- 172.21.15.69:0/780358585 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb078031d60 con 0x7fb090109530 2024-04-24T09:09:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.841+0000 7fb08cc00640 1 -- 172.21.15.69:0/780358585 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078035030 con 0x7fb090109530 2024-04-24T09:09:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.841+0000 7fb096c00640 1 -- 172.21.15.69:0/780358585 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09019b630 con 0x7fb090109530 2024-04-24T09:09:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.841+0000 7fb096c00640 1 -- 172.21.15.69:0/780358585 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0901a7820 con 0x7fb090109530 2024-04-24T09:09:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.845+0000 7fb08cc00640 1 -- 172.21.15.69:0/780358585 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb078042070 con 0x7fb090109530 2024-04-24T09:09:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.845+0000 7fb099995740 1 Processor -- start 2024-04-24T09:09:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.845+0000 7fb099995740 1 -- start start 2024-04-24T09:09:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.846+0000 7fb099995740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a0651b0 0x55dd0a065590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.846+0000 7fb099995740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a065ad0 0x55dd0a06fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.846+0000 7fb099995740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a070390 0x55dd0a072780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.846+0000 7fb099995740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55dd09fcb900 con 0x55dd0a0651b0 2024-04-24T09:09:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.846+0000 7fb099995740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55dd09f71f20 con 0x55dd0a065ad0 2024-04-24T09:09:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.846+0000 7fb099995740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55dd09fc78c0 con 0x55dd0a070390 2024-04-24T09:09:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.846+0000 7fb094c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a0651b0 0x55dd0a065590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.846+0000 7fb094c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a0651b0 0x55dd0a065590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41280/0 (socket says 172.21.15.69:41280) 2024-04-24T09:09:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.846+0000 7fb094c00640 1 -- 172.21.15.69:0/3330557935 learned_addr learned my addr 172.21.15.69:0/3330557935 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.846+0000 7fb08fe00640 1 --2- 172.21.15.69:0/3330557935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a065ad0 0x55dd0a06fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.846+0000 7fb094c00640 1 -- 172.21.15.69:0/3330557935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a070390 msgr2=0x55dd0a072780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.846+0000 7fb08f400640 1 --2- 172.21.15.69:0/3330557935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a070390 0x55dd0a072780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.847+0000 7fb094c00640 1 --2- 172.21.15.69:0/3330557935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a070390 0x55dd0a072780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.847+0000 7fb094c00640 1 -- 172.21.15.69:0/3330557935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a065ad0 msgr2=0x55dd0a06fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.847+0000 7fb094c00640 1 --2- 172.21.15.69:0/3330557935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a065ad0 0x55dd0a06fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.847+0000 7fb094c00640 1 -- 172.21.15.69:0/3330557935 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd09f96e10 con 0x55dd0a0651b0 2024-04-24T09:09:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.847+0000 7fb08fe00640 1 --2- 172.21.15.69:0/3330557935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a065ad0 0x55dd0a06fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.847+0000 7fb08f400640 1 --2- 172.21.15.69:0/3330557935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a070390 0x55dd0a072780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.847+0000 7fb08cc00640 1 --2- 172.21.15.69:0/780358585 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb0740a3f20 0x7fb0740a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.847+0000 7fb08cc00640 1 -- 172.21.15.69:0/780358585 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb0780f1350 con 0x7fb090109530 2024-04-24T09:09:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.847+0000 7fb094c00640 1 --2- 172.21.15.69:0/3330557935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a0651b0 0x55dd0a065590 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb08800bc70 tx=0x7fb08800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3054db6d6795b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.847+0000 7fb08fe00640 1 --2- 172.21.15.69:0/780358585 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb0740a3f20 0x7fb0740a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.847+0000 7fb085200640 1 -- 172.21.15.69:0/3330557935 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088015070 con 0x55dd0a0651b0 2024-04-24T09:09:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.847+0000 7fb085200640 1 -- 172.21.15.69:0/3330557935 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0880118f0 con 0x55dd0a0651b0 2024-04-24T09:09:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.847+0000 7fb085200640 1 -- 172.21.15.69:0/3330557935 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880107b0 con 0x55dd0a0651b0 2024-04-24T09:09:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.848+0000 7fb099995740 1 -- 172.21.15.69:0/3330557935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a0651b0 msgr2=0x55dd0a065590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.848+0000 7fb08fe00640 1 --2- 172.21.15.69:0/780358585 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb0740a3f20 0x7fb0740a63e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55dd0a066750 tx=0x7fb0800095c0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.848+0000 7fb099995740 1 --2- 172.21.15.69:0/3330557935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a0651b0 0x55dd0a065590 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb08800bc70 tx=0x7fb08800f730 comp rx=0 tx=0).stop 2024-04-24T09:09:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.848+0000 7fb099995740 1 -- 172.21.15.69:0/3330557935 shutdown_connections 2024-04-24T09:09:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.848+0000 7fb099995740 1 --2- 172.21.15.69:0/3330557935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a070390 0x55dd0a072780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.848+0000 7fb099995740 1 --2- 172.21.15.69:0/3330557935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a065ad0 0x55dd0a06fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.848+0000 7fb099995740 1 --2- 172.21.15.69:0/3330557935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a0651b0 0x55dd0a065590 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.848+0000 7fb099995740 1 -- 172.21.15.69:0/3330557935 >> 172.21.15.69:0/3330557935 conn(0x55dd0a0616c0 msgr2=0x55dd0a061aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.848+0000 7fb099995740 1 -- 172.21.15.69:0/3330557935 shutdown_connections 2024-04-24T09:09:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.848+0000 7fb099995740 1 -- 172.21.15.69:0/3330557935 wait complete. 2024-04-24T09:09:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.849+0000 7fb099995740 1 -- start start 2024-04-24T09:09:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.849+0000 7fb099995740 1 Processor -- start 2024-04-24T09:09:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb099995740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a0651b0 0x55dd0a06d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb099995740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a065ad0 0x55dd0a06d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb099995740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a070390 0x55dd0a06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb099995740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55dd0a072fc0 con 0x55dd0a065ad0 2024-04-24T09:09:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb099995740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55dd09f852c0 con 0x55dd0a070390 2024-04-24T09:09:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb099995740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55dd0a073140 con 0x55dd0a0651b0 2024-04-24T09:09:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb099995740 10 client.0 _ll_register_callbacks cb 0x55dd09cdda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb08f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a065ad0 0x55dd0a06d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb08f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a065ad0 0x55dd0a06d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41284/0 (socket says 172.21.15.69:41284) 2024-04-24T09:09:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb08f400640 1 -- 172.21.15.69:0/2214630320 learned_addr learned my addr 172.21.15.69:0/2214630320 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb094c00640 1 --2- 172.21.15.69:0/2214630320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a070390 0x55dd0a06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb08fe00640 1 --2- 172.21.15.69:0/2214630320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a0651b0 0x55dd0a06d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb08f400640 1 -- 172.21.15.69:0/2214630320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a0651b0 msgr2=0x55dd0a06d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb08f400640 1 --2- 172.21.15.69:0/2214630320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a0651b0 0x55dd0a06d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb08f400640 1 -- 172.21.15.69:0/2214630320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a070390 msgr2=0x55dd0a06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb08f400640 1 --2- 172.21.15.69:0/2214630320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a070390 0x55dd0a06dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb08f400640 1 -- 172.21.15.69:0/2214630320 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd09f95fd0 con 0x55dd0a065ad0 2024-04-24T09:09:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb094c00640 1 --2- 172.21.15.69:0/2214630320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a070390 0x55dd0a06dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.850+0000 7fb08fe00640 1 --2- 172.21.15.69:0/2214630320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a0651b0 0x55dd0a06d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.851+0000 7fb08f400640 1 --2- 172.21.15.69:0/2214630320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a065ad0 0x55dd0a06d7d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb0780ee760 tx=0x7fb078031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4505d78870fc745b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.851+0000 7fb07d600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.851+0000 7fb07d600640 1 -- 172.21.15.69:0/2214630320 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078049040 con 0x55dd0a065ad0 2024-04-24T09:09:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.851+0000 7fb07d600640 1 -- 172.21.15.69:0/2214630320 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb0780ee980 con 0x55dd0a065ad0 2024-04-24T09:09:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.851+0000 7fb099995740 1 -- 172.21.15.69:0/2214630320 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0880118f0 con 0x55dd0a065ad0 2024-04-24T09:09:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.851+0000 7fb07d600640 1 -- 172.21.15.69:0/2214630320 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb0780eeb80 con 0x55dd0a065ad0 2024-04-24T09:09:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.851+0000 7fb099995740 10 client.16464 fetch_fsmap learned FSMap version 29 2024-04-24T09:09:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.851+0000 7fb099995740 1 -- 172.21.15.69:0/2214630320 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dd09cde210 con 0x55dd0a065ad0 2024-04-24T09:09:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.851+0000 7fb07d600640 1 -- 172.21.15.69:0/2214630320 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0880118f0 con 0x55dd0a065ad0 2024-04-24T09:09:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.852+0000 7fb07d600640 1 -- 172.21.15.69:0/2214630320 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fb0780bd1b0 con 0x55dd0a065ad0 2024-04-24T09:09:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.852+0000 7fb099995740 10 client.16464 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:09:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.852+0000 7fb099995740 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.852+0000 7fb099995740 20 client.16464 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.852+0000 7fb099995740 1 --2- 172.21.15.69:0/2214630320 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55dd0a080340 0x55dd0a082800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.852+0000 7fb099995740 4 client.16464 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.852+0000 7fb099995740 1 -- 172.21.15.69:0/2214630320 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd09fd9320 con 0x55dd0a080340 2024-04-24T09:09:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.852+0000 7fb08fe00640 1 --2- 172.21.15.69:0/2214630320 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55dd0a080340 0x55dd0a082800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.854+0000 7fb07d600640 10 client.16464 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.854+0000 7fb08fe00640 1 --2- 172.21.15.69:0/2214630320 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55dd0a080340 0x55dd0a082800 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d252d2302d82ba1a server_cookie=376d238f5ebf7222 in_seq=0 out_seq=0 2024-04-24T09:09:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.855+0000 7fb07d600640 1 -- 172.21.15.69:0/2214630320 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd09fd9320 con 0x55dd0a080340 2024-04-24T09:09:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.855+0000 7fb07d600640 10 client.16464 handle_command_reply: tid=0 2024-04-24T09:09:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.855+0000 7fb099995740 1 client.16464 shutdown 2024-04-24T09:09:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.856+0000 7fb099995740 1 -- 172.21.15.69:0/2214630320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a065ad0 msgr2=0x55dd0a06d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.856+0000 7fb099995740 1 --2- 172.21.15.69:0/2214630320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a065ad0 0x55dd0a06d7d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb0780ee760 tx=0x7fb078031f10 comp rx=0 tx=0).stop 2024-04-24T09:09:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.856+0000 7fb099995740 1 -- 172.21.15.69:0/2214630320 shutdown_connections 2024-04-24T09:09:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.856+0000 7fb099995740 1 --2- 172.21.15.69:0/2214630320 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55dd0a080340 0x55dd0a082800 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.856+0000 7fb099995740 1 --2- 172.21.15.69:0/2214630320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a070390 0x55dd0a06dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.856+0000 7fb099995740 1 --2- 172.21.15.69:0/2214630320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a065ad0 0x55dd0a06d7d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.856+0000 7fb099995740 1 --2- 172.21.15.69:0/2214630320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a0651b0 0x55dd0a06d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.856+0000 7fb07d600640 0 client.16464 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.856+0000 7fb099995740 1 -- 172.21.15.69:0/2214630320 >> 172.21.15.69:0/2214630320 conn(0x55dd0a0616c0 msgr2=0x55dd0a067440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.856+0000 7fb099995740 1 -- 172.21.15.69:0/2214630320 shutdown_connections 2024-04-24T09:09:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.856+0000 7fb099995740 1 -- 172.21.15.69:0/2214630320 wait complete. 2024-04-24T09:09:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.856+0000 7fb099995740 20 client.16464 trim_cache size 0 max 16384 2024-04-24T09:09:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.873+0000 7fb099995740 1 Processor -- start 2024-04-24T09:09:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.873+0000 7fb099995740 1 -- start start 2024-04-24T09:09:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.873+0000 7fb099995740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a0651b0 0x55dd0a06f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.873+0000 7fb099995740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a065ad0 0x55dd0a088110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.873+0000 7fb099995740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a070390 0x55dd0a088650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.873+0000 7fb099995740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0901b3c00 con 0x55dd0a0651b0 2024-04-24T09:09:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.873+0000 7fb099995740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb0901b1440 con 0x55dd0a070390 2024-04-24T09:09:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.873+0000 7fb099995740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb0901af290 con 0x55dd0a065ad0 2024-04-24T09:09:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.874+0000 7fb08f400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a065ad0 0x55dd0a088110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.874+0000 7fb08f400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a065ad0 0x55dd0a088110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.69:47470/0 (socket says 172.21.15.69:47470) 2024-04-24T09:09:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.874+0000 7fb08f400640 1 -- 172.21.15.69:0/2474781011 learned_addr learned my addr 172.21.15.69:0/2474781011 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.874+0000 7fb08fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a0651b0 0x55dd0a06f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.874+0000 7fb094c00640 1 --2- 172.21.15.69:0/2474781011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a070390 0x55dd0a088650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.875+0000 7fb08f400640 1 -- 172.21.15.69:0/2474781011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a070390 msgr2=0x55dd0a088650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.875+0000 7fb08f400640 1 --2- 172.21.15.69:0/2474781011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a070390 0x55dd0a088650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.875+0000 7fb08f400640 1 -- 172.21.15.69:0/2474781011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a0651b0 msgr2=0x55dd0a06f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.875+0000 7fb08f400640 1 --2- 172.21.15.69:0/2474781011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a0651b0 0x55dd0a06f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.875+0000 7fb08f400640 1 -- 172.21.15.69:0/2474781011 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd09daa620 con 0x55dd0a065ad0 2024-04-24T09:09:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.875+0000 7fb08fe00640 1 --2- 172.21.15.69:0/2474781011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a0651b0 0x55dd0a06f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.875+0000 7fb08f400640 1 --2- 172.21.15.69:0/2474781011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a065ad0 0x55dd0a088110 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb078006010 tx=0x7fb07802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8784e9342ffb015 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.876+0000 7fb07fe00640 1 -- 172.21.15.69:0/2474781011 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0780bca90 con 0x55dd0a065ad0 2024-04-24T09:09:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.876+0000 7fb07fe00640 1 -- 172.21.15.69:0/2474781011 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0780eee50 con 0x55dd0a065ad0 2024-04-24T09:09:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.876+0000 7fb07fe00640 1 -- 172.21.15.69:0/2474781011 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0780ee980 con 0x55dd0a065ad0 2024-04-24T09:09:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.877+0000 7fb099995740 1 -- 172.21.15.69:0/2474781011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a065ad0 msgr2=0x55dd0a088110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.877+0000 7fb099995740 1 --2- 172.21.15.69:0/2474781011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a065ad0 0x55dd0a088110 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb078006010 tx=0x7fb07802a410 comp rx=0 tx=0).stop 2024-04-24T09:09:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.877+0000 7fb099995740 1 -- 172.21.15.69:0/2474781011 shutdown_connections 2024-04-24T09:09:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.877+0000 7fb099995740 1 --2- 172.21.15.69:0/2474781011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a070390 0x55dd0a088650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.877+0000 7fb099995740 1 --2- 172.21.15.69:0/2474781011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a065ad0 0x55dd0a088110 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.877+0000 7fb099995740 1 --2- 172.21.15.69:0/2474781011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a0651b0 0x55dd0a06f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.877+0000 7fb099995740 1 -- 172.21.15.69:0/2474781011 >> 172.21.15.69:0/2474781011 conn(0x55dd0a0616c0 msgr2=0x55dd0a06eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.877+0000 7fb099995740 1 -- 172.21.15.69:0/2474781011 shutdown_connections 2024-04-24T09:09:40.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.877+0000 7fb099995740 1 -- 172.21.15.69:0/2474781011 wait complete. 2024-04-24T09:09:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.877+0000 7fb099995740 1 -- start start 2024-04-24T09:09:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.878+0000 7fb099995740 1 Processor -- start 2024-04-24T09:09:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.878+0000 7fb099995740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a0651b0 0x55dd0a06a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.878+0000 7fb099995740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a065ad0 0x55dd0a06a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.878+0000 7fb099995740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a070390 0x55dd0a0e5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.878+0000 7fb099995740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0901af900 con 0x55dd0a065ad0 2024-04-24T09:09:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.878+0000 7fb099995740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb0901b0d70 con 0x55dd0a0651b0 2024-04-24T09:09:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.878+0000 7fb099995740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb0901affd0 con 0x55dd0a070390 2024-04-24T09:09:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.878+0000 7fb099995740 10 client.0 _ll_register_callbacks cb 0x7fb078005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.878+0000 7fb08f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a065ad0 0x55dd0a06a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.878+0000 7fb08f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a065ad0 0x55dd0a06a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41304/0 (socket says 172.21.15.69:41304) 2024-04-24T09:09:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.878+0000 7fb08f400640 1 -- 172.21.15.69:0/4234459463 learned_addr learned my addr 172.21.15.69:0/4234459463 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.879+0000 7fb094c00640 1 --2- 172.21.15.69:0/4234459463 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a070390 0x55dd0a0e5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.879+0000 7fb08f400640 1 -- 172.21.15.69:0/4234459463 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a070390 msgr2=0x55dd0a0e5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.879+0000 7fb08f400640 1 --2- 172.21.15.69:0/4234459463 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a070390 0x55dd0a0e5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.879+0000 7fb08f400640 1 -- 172.21.15.69:0/4234459463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a0651b0 msgr2=0x55dd0a06a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.879+0000 7fb08f400640 1 --2- 172.21.15.69:0/4234459463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a0651b0 0x55dd0a06a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.879+0000 7fb08f400640 1 -- 172.21.15.69:0/4234459463 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd0a06b0b0 con 0x55dd0a065ad0 2024-04-24T09:09:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.879+0000 7fb08f400640 1 --2- 172.21.15.69:0/4234459463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a065ad0 0x55dd0a06a7e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb0780c5690 tx=0x7fb07803b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ca6009dba88e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.879+0000 7fb07d600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.879+0000 7fb07d600640 1 -- 172.21.15.69:0/4234459463 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0780bca90 con 0x55dd0a065ad0 2024-04-24T09:09:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.879+0000 7fb07d600640 1 -- 172.21.15.69:0/4234459463 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb0780d6460 con 0x55dd0a065ad0 2024-04-24T09:09:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.879+0000 7fb099995740 1 -- 172.21.15.69:0/4234459463 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0780eee50 con 0x55dd0a065ad0 2024-04-24T09:09:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.880+0000 7fb07d600640 1 -- 172.21.15.69:0/4234459463 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb0780eeb90 con 0x55dd0a065ad0 2024-04-24T09:09:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.880+0000 7fb099995740 10 client.16470 fetch_fsmap learned FSMap version 29 2024-04-24T09:09:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.880+0000 7fb099995740 1 -- 172.21.15.69:0/4234459463 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dd0a06b0b0 con 0x55dd0a065ad0 2024-04-24T09:09:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.880+0000 7fb07d600640 1 -- 172.21.15.69:0/4234459463 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0780eee50 con 0x55dd0a065ad0 2024-04-24T09:09:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.880+0000 7fb07d600640 1 -- 172.21.15.69:0/4234459463 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fb0780b8ca0 con 0x55dd0a065ad0 2024-04-24T09:09:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.880+0000 7fb099995740 10 client.16470 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:09:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.880+0000 7fb099995740 10 client.16470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.880+0000 7fb099995740 20 client.16470 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.880+0000 7fb099995740 1 --2- 172.21.15.69:0/4234459463 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55dd0a0e7460 0x55dd0a0e9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.880+0000 7fb099995740 4 client.16470 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:09:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.880+0000 7fb099995740 1 -- 172.21.15.69:0/4234459463 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd09fd3190 con 0x55dd0a0e7460 2024-04-24T09:09:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.881+0000 7fb08fe00640 1 --2- 172.21.15.69:0/4234459463 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55dd0a0e7460 0x55dd0a0e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.882+0000 7fb08fe00640 1 --2- 172.21.15.69:0/4234459463 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55dd0a0e7460 0x55dd0a0e9880 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f03eb6fb09c71e1f server_cookie=d5951f040bfd6eba in_seq=0 out_seq=0 2024-04-24T09:09:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:40.882+0000 7fb07d600640 10 client.16470 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.165+0000 7fb07d600640 1 -- 172.21.15.69:0/4234459463 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd09fd3190 con 0x55dd0a0e7460 2024-04-24T09:09:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.165+0000 7fb07d600640 10 client.16470 handle_command_reply: tid=0 2024-04-24T09:09:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.165+0000 7fb099995740 1 client.16470 shutdown 2024-04-24T09:09:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.165+0000 7fb099995740 1 -- 172.21.15.69:0/4234459463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a065ad0 msgr2=0x55dd0a06a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.165+0000 7fb099995740 1 --2- 172.21.15.69:0/4234459463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a065ad0 0x55dd0a06a7e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb0780c5690 tx=0x7fb07803b790 comp rx=0 tx=0).stop 2024-04-24T09:09:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.165+0000 7fb099995740 1 -- 172.21.15.69:0/4234459463 shutdown_connections 2024-04-24T09:09:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.165+0000 7fb099995740 1 --2- 172.21.15.69:0/4234459463 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55dd0a0e7460 0x55dd0a0e9880 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.165+0000 7fb099995740 1 --2- 172.21.15.69:0/4234459463 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd0a070390 0x55dd0a0e5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.165+0000 7fb099995740 1 --2- 172.21.15.69:0/4234459463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dd0a065ad0 0x55dd0a06a7e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.165+0000 7fb07d600640 0 client.16470 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.165+0000 7fb099995740 1 --2- 172.21.15.69:0/4234459463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd0a0651b0 0x55dd0a06a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.166+0000 7fb099995740 1 -- 172.21.15.69:0/4234459463 >> 172.21.15.69:0/4234459463 conn(0x55dd0a0616c0 msgr2=0x55dd0a067440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.166+0000 7fb099995740 1 -- 172.21.15.69:0/4234459463 shutdown_connections 2024-04-24T09:09:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.166+0000 7fb099995740 1 -- 172.21.15.69:0/4234459463 wait complete. 2024-04-24T09:09:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.166+0000 7fb099995740 20 client.16470 trim_cache size 0 max 16384 2024-04-24T09:09:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:09:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:09:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:09:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:09:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.167+0000 7fb096c00640 1 -- 172.21.15.69:0/780358585 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb0740a3f20 msgr2=0x7fb0740a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.167+0000 7fb096c00640 1 --2- 172.21.15.69:0/780358585 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb0740a3f20 0x7fb0740a63e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55dd0a066750 tx=0x7fb0800095c0 comp rx=0 tx=0).stop 2024-04-24T09:09:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.167+0000 7fb096c00640 1 -- 172.21.15.69:0/780358585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb090109530 msgr2=0x7fb0901a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.167+0000 7fb096c00640 1 --2- 172.21.15.69:0/780358585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb090109530 0x7fb0901a0c10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb078005ab0 tx=0x7fb078003cd0 comp rx=0 tx=0).stop 2024-04-24T09:09:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.167+0000 7fb096c00640 1 -- 172.21.15.69:0/780358585 shutdown_connections 2024-04-24T09:09:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.167+0000 7fb096c00640 1 --2- 172.21.15.69:0/780358585 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fb0740a3f20 0x7fb0740a63e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.167+0000 7fb096c00640 1 --2- 172.21.15.69:0/780358585 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb09019a880 0x7fb09019ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.167+0000 7fb096c00640 1 --2- 172.21.15.69:0/780358585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb090109530 0x7fb0901a0c10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.167+0000 7fb096c00640 1 --2- 172.21.15.69:0/780358585 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb090104460 0x7fb0901a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.167+0000 7fb096c00640 1 -- 172.21.15.69:0/780358585 >> 172.21.15.69:0/780358585 conn(0x7fb0900ffd00 msgr2=0x7fb090102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.167+0000 7fb096c00640 1 -- 172.21.15.69:0/780358585 shutdown_connections 2024-04-24T09:09:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:41.167+0000 7fb096c00640 1 -- 172.21.15.69:0/780358585 wait complete. 2024-04-24T09:09:41.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:41 smithi130 ceph-mon[31553]: pgmap v81: 129 pgs: 129 active+clean; 38 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 775 KiB/s wr, 192 op/s 2024-04-24T09:09:41.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:41 smithi081 ceph-mon[30961]: pgmap v81: 129 pgs: 129 active+clean; 38 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 775 KiB/s wr, 192 op/s 2024-04-24T09:09:41.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:41 smithi069 ceph-mon[28170]: pgmap v81: 129 pgs: 129 active+clean; 38 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 775 KiB/s wr, 192 op/s 2024-04-24T09:09:43.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:43 smithi130 ceph-mon[31553]: pgmap v82: 129 pgs: 129 active+clean; 38 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 532 KiB/s wr, 86 op/s 2024-04-24T09:09:43.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:43 smithi130 ceph-mon[31553]: scrub summary: idle 2024-04-24T09:09:43.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:43 smithi130 ceph-mon[31553]: scrub summary: idle 2024-04-24T09:09:43.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:43 smithi130 ceph-mon[31553]: scrub summary: idle 2024-04-24T09:09:43.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:43 smithi081 ceph-mon[30961]: pgmap v82: 129 pgs: 129 active+clean; 38 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 532 KiB/s wr, 86 op/s 2024-04-24T09:09:43.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:43 smithi081 ceph-mon[30961]: scrub summary: idle 2024-04-24T09:09:43.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:43 smithi081 ceph-mon[30961]: scrub summary: idle 2024-04-24T09:09:43.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:43 smithi081 ceph-mon[30961]: scrub summary: idle 2024-04-24T09:09:43.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:43 smithi069 ceph-mon[28170]: pgmap v82: 129 pgs: 129 active+clean; 38 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 532 KiB/s wr, 86 op/s 2024-04-24T09:09:43.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:43 smithi069 ceph-mon[28170]: scrub summary: idle 2024-04-24T09:09:43.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:43 smithi069 ceph-mon[28170]: scrub summary: idle 2024-04-24T09:09:43.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:43 smithi069 ceph-mon[28170]: scrub summary: idle 2024-04-24T09:09:45.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:45 smithi130 ceph-mon[31553]: pgmap v83: 129 pgs: 129 active+clean; 38 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 532 KiB/s wr, 86 op/s 2024-04-24T09:09:45.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:45 smithi081 ceph-mon[30961]: pgmap v83: 129 pgs: 129 active+clean; 38 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 532 KiB/s wr, 86 op/s 2024-04-24T09:09:45.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:45 smithi069 ceph-mon[28170]: pgmap v83: 129 pgs: 129 active+clean; 38 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 532 KiB/s wr, 86 op/s 2024-04-24T09:09:47.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:47 smithi130 ceph-mon[31553]: pgmap v84: 129 pgs: 129 active+clean; 35 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 659 KiB/s wr, 252 op/s 2024-04-24T09:09:47.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:47 smithi081 ceph-mon[30961]: pgmap v84: 129 pgs: 129 active+clean; 35 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 659 KiB/s wr, 252 op/s 2024-04-24T09:09:47.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:47 smithi069 ceph-mon[28170]: pgmap v84: 129 pgs: 129 active+clean; 35 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 659 KiB/s wr, 252 op/s 2024-04-24T09:09:49.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:49 smithi081 ceph-mon[30961]: pgmap v85: 129 pgs: 129 active+clean; 35 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 349 KiB/s wr, 202 op/s 2024-04-24T09:09:49.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:49 smithi069 ceph-mon[28170]: pgmap v85: 129 pgs: 129 active+clean; 35 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 349 KiB/s wr, 202 op/s 2024-04-24T09:09:49.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:49 smithi130 ceph-mon[31553]: pgmap v85: 129 pgs: 129 active+clean; 35 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 349 KiB/s wr, 202 op/s 2024-04-24T09:09:51.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:51 smithi081 ceph-mon[30961]: pgmap v86: 129 pgs: 129 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 426 KiB/s wr, 235 op/s 2024-04-24T09:09:51.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:51 smithi069 ceph-mon[28170]: pgmap v86: 129 pgs: 129 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 426 KiB/s wr, 235 op/s 2024-04-24T09:09:51.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:51 smithi130 ceph-mon[31553]: pgmap v86: 129 pgs: 129 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 426 KiB/s wr, 235 op/s 2024-04-24T09:09:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:53 smithi081 ceph-mon[30961]: pgmap v87: 129 pgs: 129 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 204 KiB/s wr, 197 op/s 2024-04-24T09:09:53.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:53 smithi069 ceph-mon[28170]: pgmap v87: 129 pgs: 129 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 204 KiB/s wr, 197 op/s 2024-04-24T09:09:53.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:53 smithi130 ceph-mon[31553]: pgmap v87: 129 pgs: 129 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 204 KiB/s wr, 197 op/s 2024-04-24T09:09:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.315+0000 7f18e3400640 1 -- 172.21.15.69:0/1855087231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18dc102c30 msgr2=0x7f18dc106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.315+0000 7f18e3400640 1 --2- 172.21.15.69:0/1855087231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18dc102c30 0x7f18dc106c70 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009900 tx=0x7f18c402a410 comp rx=0 tx=0).stop 2024-04-24T09:09:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.315+0000 7f18e3400640 1 -- 172.21.15.69:0/1855087231 shutdown_connections 2024-04-24T09:09:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.315+0000 7f18e3400640 1 --2- 172.21.15.69:0/1855087231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18dc107350 0x7f18dc10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.315+0000 7f18e3400640 1 --2- 172.21.15.69:0/1855087231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18dc102c30 0x7f18dc106c70 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.315+0000 7f18e3400640 1 --2- 172.21.15.69:0/1855087231 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18dc102280 0x7f18dc102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.315+0000 7f18e3400640 1 -- 172.21.15.69:0/1855087231 >> 172.21.15.69:0/1855087231 conn(0x7f18dc079490 msgr2=0x7f18dc0798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.315+0000 7f18e3400640 1 -- 172.21.15.69:0/1855087231 shutdown_connections 2024-04-24T09:09:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.316+0000 7f18e3400640 1 -- 172.21.15.69:0/1855087231 wait complete. 2024-04-24T09:09:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.316+0000 7f18e3400640 1 Processor -- start 2024-04-24T09:09:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.317+0000 7f18e3400640 1 -- start start 2024-04-24T09:09:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.317+0000 7f18e3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18dc102280 0x7f18dc074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.317+0000 7f18e3400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18dc107350 0x7f18dc074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.317+0000 7f18e3400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18dc074d80 0x7f18dc1a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.317+0000 7f18e3400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1122c0 con 0x7f18dc102280 2024-04-24T09:09:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.317+0000 7f18e3400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc112140 con 0x7f18dc074d80 2024-04-24T09:09:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.317+0000 7f18e3400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc112440 con 0x7f18dc107350 2024-04-24T09:09:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.317+0000 7f18e0a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18dc102280 0x7f18dc074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.317+0000 7f18e0a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18dc102280 0x7f18dc074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39292/0 (socket says 172.21.15.69:39292) 2024-04-24T09:09:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.317+0000 7f18e0a00640 1 -- 172.21.15.69:0/328406373 learned_addr learned my addr 172.21.15.69:0/328406373 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.318+0000 7f18e0a00640 1 -- 172.21.15.69:0/328406373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18dc107350 msgr2=0x7f18dc074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.318+0000 7f18e0a00640 1 --2- 172.21.15.69:0/328406373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18dc107350 0x7f18dc074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.318+0000 7f18e0a00640 1 -- 172.21.15.69:0/328406373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18dc074d80 msgr2=0x7f18dc1a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.318+0000 7f18e0a00640 1 --2- 172.21.15.69:0/328406373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18dc074d80 0x7f18dc1a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.318+0000 7f18e0a00640 1 -- 172.21.15.69:0/328406373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18dc1a6f90 con 0x7f18dc102280 2024-04-24T09:09:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.318+0000 7f18e0a00640 1 --2- 172.21.15.69:0/328406373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18dc102280 0x7f18dc074300 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00c920 tx=0x7f18cc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.318+0000 7f18da000640 1 -- 172.21.15.69:0/328406373 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc007c70 con 0x7f18dc102280 2024-04-24T09:09:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.318+0000 7f18da000640 1 -- 172.21.15.69:0/328406373 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18cc007e10 con 0x7f18dc102280 2024-04-24T09:09:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.318+0000 7f18da000640 1 -- 172.21.15.69:0/328406373 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc018820 con 0x7f18dc102280 2024-04-24T09:09:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.318+0000 7f18e3400640 1 -- 172.21.15.69:0/328406373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18dc1a7280 con 0x7f18dc102280 2024-04-24T09:09:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.319+0000 7f18e3400640 1 -- 172.21.15.69:0/328406373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18dc1a7820 con 0x7f18dc102280 2024-04-24T09:09:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.323+0000 7f18e621f740 1 Processor -- start 2024-04-24T09:09:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.323+0000 7f18e621f740 1 -- start start 2024-04-24T09:09:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.323+0000 7f18da000640 1 -- 172.21.15.69:0/328406373 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f18cc0189c0 con 0x7f18dc102280 2024-04-24T09:09:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.323+0000 7f18e621f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc281b0 0x560fabc28590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.323+0000 7f18e621f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc28ad0 0x560fabc32e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.323+0000 7f18e621f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc35780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.323+0000 7f18e621f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560fabb8e900 con 0x560fabc33390 2024-04-24T09:09:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.323+0000 7f18e621f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x560fabb34f20 con 0x560fabc28ad0 2024-04-24T09:09:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.323+0000 7f18e621f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x560fabb8a8c0 con 0x560fabc281b0 2024-04-24T09:09:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.324+0000 7f18d3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc35780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.324+0000 7f18d3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc35780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39294/0 (socket says 172.21.15.69:39294) 2024-04-24T09:09:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.324+0000 7f18d3e00640 1 -- 172.21.15.69:0/384712260 learned_addr learned my addr 172.21.15.69:0/384712260 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.324+0000 7f18e1400640 1 --2- 172.21.15.69:0/384712260 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc281b0 0x560fabc28590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.324+0000 7f18e0a00640 1 --2- 172.21.15.69:0/384712260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc28ad0 0x560fabc32e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.324+0000 7f18d3e00640 1 -- 172.21.15.69:0/384712260 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc281b0 msgr2=0x560fabc28590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.324+0000 7f18d3e00640 1 --2- 172.21.15.69:0/384712260 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc281b0 0x560fabc28590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.324+0000 7f18d3e00640 1 -- 172.21.15.69:0/384712260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc28ad0 msgr2=0x560fabc32e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.324+0000 7f18d3e00640 1 --2- 172.21.15.69:0/384712260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc28ad0 0x560fabc32e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.324+0000 7f18d3e00640 1 -- 172.21.15.69:0/384712260 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fabb59e10 con 0x560fabc33390 2024-04-24T09:09:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.325+0000 7f18d3e00640 1 --2- 172.21.15.69:0/384712260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc35780 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f18c402a930 tx=0x7f18c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa279c23145cddd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.325+0000 7f18da000640 1 --2- 172.21.15.69:0/328406373 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f18c00a3e70 0x7f18c00a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.325+0000 7f18d1c00640 1 -- 172.21.15.69:0/384712260 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c403e030 con 0x560fabc33390 2024-04-24T09:09:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.325+0000 7f18d1c00640 1 -- 172.21.15.69:0/384712260 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18c40041f0 con 0x560fabc33390 2024-04-24T09:09:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.325+0000 7f18d1c00640 1 -- 172.21.15.69:0/384712260 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4035040 con 0x560fabc33390 2024-04-24T09:09:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.325+0000 7f18da000640 1 -- 172.21.15.69:0/328406373 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f18cc0d0cb0 con 0x7f18dc102280 2024-04-24T09:09:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.325+0000 7f18e1400640 1 --2- 172.21.15.69:0/328406373 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f18c00a3e70 0x7f18c00a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.326+0000 7f18e621f740 1 -- 172.21.15.69:0/384712260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 msgr2=0x560fabc35780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.326+0000 7f18e621f740 1 --2- 172.21.15.69:0/384712260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc35780 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f18c402a930 tx=0x7f18c4000bc0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.326+0000 7f18e1400640 1 --2- 172.21.15.69:0/328406373 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f18c00a3e70 0x7f18c00a6330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f18d4005fd0 tx=0x7f18d40095f0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.326+0000 7f18e621f740 1 -- 172.21.15.69:0/384712260 shutdown_connections 2024-04-24T09:09:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.326+0000 7f18e621f740 1 --2- 172.21.15.69:0/384712260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc35780 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.326+0000 7f18e621f740 1 --2- 172.21.15.69:0/384712260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc28ad0 0x560fabc32e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.326+0000 7f18e621f740 1 --2- 172.21.15.69:0/384712260 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc281b0 0x560fabc28590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.326+0000 7f18e621f740 1 -- 172.21.15.69:0/384712260 >> 172.21.15.69:0/384712260 conn(0x560fabc246c0 msgr2=0x560fabc24aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.326+0000 7f18e621f740 1 -- 172.21.15.69:0/384712260 shutdown_connections 2024-04-24T09:09:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.326+0000 7f18e621f740 1 -- 172.21.15.69:0/384712260 wait complete. 2024-04-24T09:09:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.327+0000 7f18e621f740 1 -- start start 2024-04-24T09:09:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.327+0000 7f18e621f740 1 Processor -- start 2024-04-24T09:09:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.327+0000 7f18e621f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc281b0 0x560fabc302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.327+0000 7f18e621f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc28ad0 0x560fabc30830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.327+0000 7f18e621f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc33390 0x560fabc30d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.327+0000 7f18e621f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560fabc35f60 con 0x560fabc28ad0 2024-04-24T09:09:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.327+0000 7f18e621f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x560fabb482c0 con 0x560fabc281b0 2024-04-24T09:09:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.327+0000 7f18e621f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x560fabc360e0 con 0x560fabc33390 2024-04-24T09:09:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.327+0000 7f18e621f740 10 client.0 _ll_register_callbacks cb 0x560fab8a0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.328+0000 7f18d3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc28ad0 0x560fabc30830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.328+0000 7f18d3e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc28ad0 0x560fabc30830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39310/0 (socket says 172.21.15.69:39310) 2024-04-24T09:09:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.328+0000 7f18d3e00640 1 -- 172.21.15.69:0/3286977926 learned_addr learned my addr 172.21.15.69:0/3286977926 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.328+0000 7f18e0a00640 1 --2- 172.21.15.69:0/3286977926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc281b0 0x560fabc302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.328+0000 7f18e1400640 1 --2- 172.21.15.69:0/3286977926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc33390 0x560fabc30d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.328+0000 7f18d3e00640 1 -- 172.21.15.69:0/3286977926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc33390 msgr2=0x560fabc30d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.328+0000 7f18d3e00640 1 --2- 172.21.15.69:0/3286977926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc33390 0x560fabc30d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.328+0000 7f18d3e00640 1 -- 172.21.15.69:0/3286977926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc281b0 msgr2=0x560fabc302f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.328+0000 7f18d3e00640 1 --2- 172.21.15.69:0/3286977926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc281b0 0x560fabc302f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.328+0000 7f18d3e00640 1 -- 172.21.15.69:0/3286977926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fabb58fd0 con 0x560fabc28ad0 2024-04-24T09:09:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.328+0000 7f18d3e00640 1 --2- 172.21.15.69:0/3286977926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc28ad0 0x560fabc30830 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f18c4003820 tx=0x7f18c4003850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a58c394c9cb180 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.328+0000 7f18ca000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.329+0000 7f18ca000640 1 -- 172.21.15.69:0/3286977926 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4047070 con 0x560fabc28ad0 2024-04-24T09:09:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.329+0000 7f18ca000640 1 -- 172.21.15.69:0/3286977926 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f18c40314a0 con 0x560fabc28ad0 2024-04-24T09:09:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.329+0000 7f18e621f740 1 -- 172.21.15.69:0/3286977926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18c40041f0 con 0x560fabc28ad0 2024-04-24T09:09:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.329+0000 7f18ca000640 1 -- 172.21.15.69:0/3286977926 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f18c4055050 con 0x560fabc28ad0 2024-04-24T09:09:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.329+0000 7f18e621f740 10 client.16482 fetch_fsmap learned FSMap version 29 2024-04-24T09:09:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.329+0000 7f18e621f740 1 -- 172.21.15.69:0/3286977926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560fab8a1210 con 0x560fabc28ad0 2024-04-24T09:09:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.329+0000 7f18ca000640 1 -- 172.21.15.69:0/3286977926 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18c40041f0 con 0x560fabc28ad0 2024-04-24T09:09:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.329+0000 7f18ca000640 1 -- 172.21.15.69:0/3286977926 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f18c404c660 con 0x560fabc28ad0 2024-04-24T09:09:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.329+0000 7f18e621f740 10 client.16482 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:09:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.330+0000 7f18e621f740 10 client.16482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.330+0000 7f18e621f740 20 client.16482 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.330+0000 7f18e621f740 1 --2- 172.21.15.69:0/3286977926 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x560fabc432c0 0x560fabc456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.330+0000 7f18e621f740 4 client.16482 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.330+0000 7f18e621f740 1 -- 172.21.15.69:0/3286977926 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fabb9c320 con 0x560fabc432c0 2024-04-24T09:09:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.330+0000 7f18e0a00640 1 --2- 172.21.15.69:0/3286977926 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x560fabc432c0 0x560fabc456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.331+0000 7f18ca000640 10 client.16482 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.331+0000 7f18e0a00640 1 --2- 172.21.15.69:0/3286977926 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x560fabc432c0 0x560fabc456b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8494c5ca746220c server_cookie=956c5f55261e7262 in_seq=0 out_seq=0 2024-04-24T09:09:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.334+0000 7f18ca000640 1 -- 172.21.15.69:0/3286977926 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560fabb9c320 con 0x560fabc432c0 2024-04-24T09:09:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.334+0000 7f18ca000640 10 client.16482 handle_command_reply: tid=0 2024-04-24T09:09:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.334+0000 7f18e621f740 1 client.16482 shutdown 2024-04-24T09:09:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.334+0000 7f18e621f740 1 -- 172.21.15.69:0/3286977926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc28ad0 msgr2=0x560fabc30830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.334+0000 7f18e621f740 1 --2- 172.21.15.69:0/3286977926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc28ad0 0x560fabc30830 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f18c4003820 tx=0x7f18c4003850 comp rx=0 tx=0).stop 2024-04-24T09:09:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.334+0000 7f18e621f740 1 -- 172.21.15.69:0/3286977926 shutdown_connections 2024-04-24T09:09:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.334+0000 7f18e621f740 1 --2- 172.21.15.69:0/3286977926 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x560fabc432c0 0x560fabc456b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.334+0000 7f18e621f740 1 --2- 172.21.15.69:0/3286977926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc33390 0x560fabc30d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.334+0000 7f18ca000640 0 client.16482 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.334+0000 7f18e621f740 1 --2- 172.21.15.69:0/3286977926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc28ad0 0x560fabc30830 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.334+0000 7f18e621f740 1 --2- 172.21.15.69:0/3286977926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc281b0 0x560fabc302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.334+0000 7f18e621f740 1 -- 172.21.15.69:0/3286977926 >> 172.21.15.69:0/3286977926 conn(0x560fabc246c0 msgr2=0x560fabc2a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.335+0000 7f18e621f740 1 -- 172.21.15.69:0/3286977926 shutdown_connections 2024-04-24T09:09:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.335+0000 7f18e621f740 1 -- 172.21.15.69:0/3286977926 wait complete. 2024-04-24T09:09:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.335+0000 7f18e621f740 20 client.16482 trim_cache size 0 max 16384 2024-04-24T09:09:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.352+0000 7f18e621f740 1 Processor -- start 2024-04-24T09:09:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.352+0000 7f18e621f740 1 -- start start 2024-04-24T09:09:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.352+0000 7f18e621f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc281b0 0x560fabc3ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.352+0000 7f18e621f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc28ad0 0x560fabc23fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.352+0000 7f18e621f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc2a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.352+0000 7f18e621f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1b3b60 con 0x560fabc33390 2024-04-24T09:09:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.352+0000 7f18e621f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1b13a0 con 0x560fabc281b0 2024-04-24T09:09:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.352+0000 7f18e621f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1af1f0 con 0x560fabc28ad0 2024-04-24T09:09:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.353+0000 7f18e1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc2a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.353+0000 7f18e1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc2a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39312/0 (socket says 172.21.15.69:39312) 2024-04-24T09:09:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.353+0000 7f18e1400640 1 -- 172.21.15.69:0/731069864 learned_addr learned my addr 172.21.15.69:0/731069864 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.353+0000 7f18d3e00640 1 --2- 172.21.15.69:0/731069864 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc28ad0 0x560fabc23fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.353+0000 7f18e0a00640 1 --2- 172.21.15.69:0/731069864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc281b0 0x560fabc3ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.353+0000 7f18e1400640 1 -- 172.21.15.69:0/731069864 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc28ad0 msgr2=0x560fabc23fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.353+0000 7f18e1400640 1 --2- 172.21.15.69:0/731069864 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc28ad0 0x560fabc23fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.353+0000 7f18e1400640 1 -- 172.21.15.69:0/731069864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc281b0 msgr2=0x560fabc3ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.353+0000 7f18e1400640 1 --2- 172.21.15.69:0/731069864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc281b0 0x560fabc3ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.353+0000 7f18e1400640 1 -- 172.21.15.69:0/731069864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fab96d620 con 0x560fabc33390 2024-04-24T09:09:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.354+0000 7f18e1400640 1 --2- 172.21.15.69:0/731069864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc2a270 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f18d40098f0 tx=0x7f18d4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917f53563678cf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.354+0000 7f18d1200640 1 -- 172.21.15.69:0/731069864 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d401f620 con 0x560fabc33390 2024-04-24T09:09:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.354+0000 7f18d1200640 1 -- 172.21.15.69:0/731069864 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18d401fc70 con 0x560fabc33390 2024-04-24T09:09:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.354+0000 7f18d1200640 1 -- 172.21.15.69:0/731069864 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4027c50 con 0x560fabc33390 2024-04-24T09:09:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.355+0000 7f18e621f740 1 -- 172.21.15.69:0/731069864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 msgr2=0x560fabc2a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.355+0000 7f18e621f740 1 --2- 172.21.15.69:0/731069864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc2a270 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f18d40098f0 tx=0x7f18d4013000 comp rx=0 tx=0).stop 2024-04-24T09:09:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.355+0000 7f18e621f740 1 -- 172.21.15.69:0/731069864 shutdown_connections 2024-04-24T09:09:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.355+0000 7f18e621f740 1 --2- 172.21.15.69:0/731069864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc2a270 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.355+0000 7f18e621f740 1 --2- 172.21.15.69:0/731069864 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc28ad0 0x560fabc23fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.355+0000 7f18e621f740 1 --2- 172.21.15.69:0/731069864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc281b0 0x560fabc3ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.355+0000 7f18e621f740 1 -- 172.21.15.69:0/731069864 >> 172.21.15.69:0/731069864 conn(0x560fabc246c0 msgr2=0x560fabc29b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.355+0000 7f18e621f740 1 -- 172.21.15.69:0/731069864 shutdown_connections 2024-04-24T09:09:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.356+0000 7f18e621f740 1 -- 172.21.15.69:0/731069864 wait complete. 2024-04-24T09:09:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.356+0000 7f18e621f740 1 -- start start 2024-04-24T09:09:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.356+0000 7f18e621f740 1 Processor -- start 2024-04-24T09:09:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.357+0000 7f18e621f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc281b0 0x560fabc37030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.357+0000 7f18e621f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc28ad0 0x560fabc37570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.357+0000 7f18e621f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc37af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.357+0000 7f18e621f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1af860 con 0x560fabc33390 2024-04-24T09:09:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.357+0000 7f18e621f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1b0cd0 con 0x560fabc28ad0 2024-04-24T09:09:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.357+0000 7f18e621f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1aff30 con 0x560fabc281b0 2024-04-24T09:09:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.357+0000 7f18e621f740 10 client.0 _ll_register_callbacks cb 0x7f18c4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.357+0000 7f18e1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc37af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.357+0000 7f18e1400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc37af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39328/0 (socket says 172.21.15.69:39328) 2024-04-24T09:09:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.357+0000 7f18e1400640 1 -- 172.21.15.69:0/2865497438 learned_addr learned my addr 172.21.15.69:0/2865497438 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.357+0000 7f18e0a00640 1 --2- 172.21.15.69:0/2865497438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc281b0 0x560fabc37030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.357+0000 7f18e1400640 1 -- 172.21.15.69:0/2865497438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc281b0 msgr2=0x560fabc37030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.357+0000 7f18e1400640 1 --2- 172.21.15.69:0/2865497438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc281b0 0x560fabc37030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.357+0000 7f18e1400640 1 -- 172.21.15.69:0/2865497438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc28ad0 msgr2=0x560fabc37570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.358+0000 7f18e1400640 1 --2- 172.21.15.69:0/2865497438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc28ad0 0x560fabc37570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.358+0000 7f18e1400640 1 -- 172.21.15.69:0/2865497438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fabca88a0 con 0x560fabc33390 2024-04-24T09:09:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.358+0000 7f18e1400640 1 --2- 172.21.15.69:0/2865497438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc37af0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f18d4012cb0 tx=0x7f18d4028b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac9487cde479a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.358+0000 7f18ca000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.358+0000 7f18ca000640 1 -- 172.21.15.69:0/2865497438 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4038060 con 0x560fabc33390 2024-04-24T09:09:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.358+0000 7f18ca000640 1 -- 172.21.15.69:0/2865497438 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f18d4028cb0 con 0x560fabc33390 2024-04-24T09:09:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.358+0000 7f18e621f740 1 -- 172.21.15.69:0/2865497438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18d401fc70 con 0x560fabc33390 2024-04-24T09:09:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.358+0000 7f18ca000640 1 -- 172.21.15.69:0/2865497438 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f18d403f050 con 0x560fabc33390 2024-04-24T09:09:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.358+0000 7f18e621f740 10 client.16488 fetch_fsmap learned FSMap version 29 2024-04-24T09:09:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.358+0000 7f18e621f740 1 -- 172.21.15.69:0/2865497438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560fabca9bb0 con 0x560fabc33390 2024-04-24T09:09:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.359+0000 7f18ca000640 1 -- 172.21.15.69:0/2865497438 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18d401fc70 con 0x560fabc33390 2024-04-24T09:09:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.359+0000 7f18ca000640 1 -- 172.21.15.69:0/2865497438 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f18d400f040 con 0x560fabc33390 2024-04-24T09:09:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.359+0000 7f18e621f740 10 client.16488 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:09:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.359+0000 7f18e621f740 10 client.16488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.359+0000 7f18e621f740 20 client.16488 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.359+0000 7f18e621f740 1 --2- 172.21.15.69:0/2865497438 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x560fabcaa250 0x560fabcac670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.359+0000 7f18e621f740 4 client.16488 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:09:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.359+0000 7f18e621f740 1 -- 172.21.15.69:0/2865497438 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560fabb96190 con 0x560fabcaa250 2024-04-24T09:09:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.360+0000 7f18e0a00640 1 --2- 172.21.15.69:0/2865497438 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x560fabcaa250 0x560fabcac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.360+0000 7f18ca000640 10 client.16488 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.360+0000 7f18e0a00640 1 --2- 172.21.15.69:0/2865497438 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x560fabcaa250 0x560fabcac670 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af0336a5bbe73d0c server_cookie=2177a3e9e57955d in_seq=0 out_seq=0 2024-04-24T09:09:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.573+0000 7f18ca000640 1 -- 172.21.15.69:0/2865497438 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560fabb96190 con 0x560fabcaa250 2024-04-24T09:09:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.574+0000 7f18ca000640 10 client.16488 handle_command_reply: tid=0 2024-04-24T09:09:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.574+0000 7f18e621f740 1 client.16488 shutdown 2024-04-24T09:09:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.574+0000 7f18e621f740 1 -- 172.21.15.69:0/2865497438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 msgr2=0x560fabc37af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.574+0000 7f18e621f740 1 --2- 172.21.15.69:0/2865497438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc37af0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f18d4012cb0 tx=0x7f18d4028b10 comp rx=0 tx=0).stop 2024-04-24T09:09:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.574+0000 7f18e621f740 1 -- 172.21.15.69:0/2865497438 shutdown_connections 2024-04-24T09:09:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.574+0000 7f18e621f740 1 --2- 172.21.15.69:0/2865497438 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x560fabcaa250 0x560fabcac670 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.574+0000 7f18e621f740 1 --2- 172.21.15.69:0/2865497438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560fabc33390 0x560fabc37af0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.574+0000 7f18e621f740 1 --2- 172.21.15.69:0/2865497438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fabc28ad0 0x560fabc37570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.574+0000 7f18e621f740 1 --2- 172.21.15.69:0/2865497438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560fabc281b0 0x560fabc37030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.574+0000 7f18e621f740 1 -- 172.21.15.69:0/2865497438 >> 172.21.15.69:0/2865497438 conn(0x560fabc246c0 msgr2=0x560fabc2a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.574+0000 7f18ca000640 0 client.16488 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.575+0000 7f18e621f740 1 -- 172.21.15.69:0/2865497438 shutdown_connections 2024-04-24T09:09:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.575+0000 7f18e621f740 1 -- 172.21.15.69:0/2865497438 wait complete. 2024-04-24T09:09:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.575+0000 7f18e621f740 20 client.16488 trim_cache size 0 max 16384 2024-04-24T09:09:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:09:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:09:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:09:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:09:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.575+0000 7f18e3400640 1 -- 172.21.15.69:0/328406373 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f18c00a3e70 msgr2=0x7f18c00a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.575+0000 7f18e3400640 1 --2- 172.21.15.69:0/328406373 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f18c00a3e70 0x7f18c00a6330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f18d4005fd0 tx=0x7f18d40095f0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.575+0000 7f18e3400640 1 -- 172.21.15.69:0/328406373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18dc102280 msgr2=0x7f18dc074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.575+0000 7f18e3400640 1 --2- 172.21.15.69:0/328406373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18dc102280 0x7f18dc074300 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00c920 tx=0x7f18cc00cde0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.576+0000 7f18e3400640 1 -- 172.21.15.69:0/328406373 shutdown_connections 2024-04-24T09:09:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.576+0000 7f18e3400640 1 --2- 172.21.15.69:0/328406373 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f18c00a3e70 0x7f18c00a6330 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.576+0000 7f18e3400640 1 --2- 172.21.15.69:0/328406373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18dc074d80 0x7f18dc1a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.576+0000 7f18e3400640 1 --2- 172.21.15.69:0/328406373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18dc107350 0x7f18dc074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.576+0000 7f18e3400640 1 --2- 172.21.15.69:0/328406373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18dc102280 0x7f18dc074300 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.576+0000 7f18e3400640 1 -- 172.21.15.69:0/328406373 >> 172.21.15.69:0/328406373 conn(0x7f18dc079490 msgr2=0x7f18dc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.576+0000 7f18e3400640 1 -- 172.21.15.69:0/328406373 shutdown_connections 2024-04-24T09:09:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:09:54.576+0000 7f18e3400640 1 -- 172.21.15.69:0/328406373 wait complete. 2024-04-24T09:09:55.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:55 smithi081 ceph-mon[30961]: pgmap v88: 129 pgs: 129 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 204 KiB/s wr, 197 op/s 2024-04-24T09:09:55.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:55 smithi069 ceph-mon[28170]: pgmap v88: 129 pgs: 129 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 204 KiB/s wr, 197 op/s 2024-04-24T09:09:55.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:55 smithi130 ceph-mon[31553]: pgmap v88: 129 pgs: 129 active+clean; 36 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 204 KiB/s wr, 197 op/s 2024-04-24T09:09:57.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:57 smithi081 ceph-mon[30961]: pgmap v89: 129 pgs: 129 active+clean; 37 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 331 KiB/s wr, 322 op/s 2024-04-24T09:09:57.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:57 smithi069 ceph-mon[28170]: pgmap v89: 129 pgs: 129 active+clean; 37 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 331 KiB/s wr, 322 op/s 2024-04-24T09:09:57.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:57 smithi130 ceph-mon[31553]: pgmap v89: 129 pgs: 129 active+clean; 37 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 331 KiB/s wr, 322 op/s 2024-04-24T09:09:58.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:09:58.860 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:09:59.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.077+0000 7f69e3000640 1 -- 172.21.15.69:0/2712757983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69dc100c40 msgr2=0x7f69dc101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.077+0000 7f69e3000640 1 --2- 172.21.15.69:0/2712757983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69dc100c40 0x7f69dc101020 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009900 tx=0x7f69d002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.077+0000 7f69e3000640 1 -- 172.21.15.69:0/2712757983 shutdown_connections 2024-04-24T09:09:59.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.078+0000 7f69e3000640 1 --2- 172.21.15.69:0/2712757983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69dc10e7f0 0x7f69dc110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.078+0000 7f69e3000640 1 --2- 172.21.15.69:0/2712757983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f69dc1015f0 0x7f69dc10e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.078+0000 7f69e3000640 1 --2- 172.21.15.69:0/2712757983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69dc100c40 0x7f69dc101020 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.078+0000 7f69e3000640 1 -- 172.21.15.69:0/2712757983 >> 172.21.15.69:0/2712757983 conn(0x7f69dc0fc880 msgr2=0x7f69dc0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:59.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.078+0000 7f69e3000640 1 -- 172.21.15.69:0/2712757983 shutdown_connections 2024-04-24T09:09:59.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.078+0000 7f69e3000640 1 -- 172.21.15.69:0/2712757983 wait complete. 2024-04-24T09:09:59.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.079+0000 7f69e3000640 1 Processor -- start 2024-04-24T09:09:59.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.080+0000 7f69e3000640 1 -- start start 2024-04-24T09:09:59.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.080+0000 7f69e3000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69dc100c40 0x7f69dc1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.080+0000 7f69e3000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f69dc1015f0 0x7f69dc1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.080+0000 7f69e3000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69dc10e7f0 0x7f69dc19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.080+0000 7f69e3000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc114460 con 0x7f69dc10e7f0 2024-04-24T09:09:59.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.080+0000 7f69e3000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc1142e0 con 0x7f69dc100c40 2024-04-24T09:09:59.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.080+0000 7f69e3000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc1145e0 con 0x7f69dc1015f0 2024-04-24T09:09:59.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.081+0000 7f69e2600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69dc10e7f0 0x7f69dc19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.081+0000 7f69e2600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69dc10e7f0 0x7f69dc19a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39352/0 (socket says 172.21.15.69:39352) 2024-04-24T09:09:59.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.081+0000 7f69e2600640 1 -- 172.21.15.69:0/3699195887 learned_addr learned my addr 172.21.15.69:0/3699195887 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:59.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.081+0000 7f69e1200640 1 --2- 172.21.15.69:0/3699195887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f69dc1015f0 0x7f69dc1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.081+0000 7f69e1c00640 1 --2- 172.21.15.69:0/3699195887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69dc100c40 0x7f69dc1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.081+0000 7f69e2600640 1 -- 172.21.15.69:0/3699195887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f69dc1015f0 msgr2=0x7f69dc1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.081+0000 7f69e2600640 1 --2- 172.21.15.69:0/3699195887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f69dc1015f0 0x7f69dc1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.081+0000 7f69e2600640 1 -- 172.21.15.69:0/3699195887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69dc100c40 msgr2=0x7f69dc1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.081+0000 7f69e2600640 1 --2- 172.21.15.69:0/3699195887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69dc100c40 0x7f69dc1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.081+0000 7f69e2600640 1 -- 172.21.15.69:0/3699195887 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69dc19afe0 con 0x7f69dc10e7f0 2024-04-24T09:09:59.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.081+0000 7f69e2600640 1 --2- 172.21.15.69:0/3699195887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69dc10e7f0 0x7f69dc19a860 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f69d400b9e0 tx=0x7f69d400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:59.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.082+0000 7f69da000640 1 -- 172.21.15.69:0/3699195887 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4019070 con 0x7f69dc10e7f0 2024-04-24T09:09:59.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.082+0000 7f69da000640 1 -- 172.21.15.69:0/3699195887 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69d4014440 con 0x7f69dc10e7f0 2024-04-24T09:09:59.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.082+0000 7f69e3000640 1 -- 172.21.15.69:0/3699195887 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69dc19b2d0 con 0x7f69dc10e7f0 2024-04-24T09:09:59.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.082+0000 7f69da000640 1 -- 172.21.15.69:0/3699195887 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d40125a0 con 0x7f69dc10e7f0 2024-04-24T09:09:59.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.082+0000 7f69e3000640 1 -- 172.21.15.69:0/3699195887 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69dc1a7570 con 0x7f69dc10e7f0 2024-04-24T09:09:59.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.085+0000 7f69e73d2740 1 Processor -- start 2024-04-24T09:09:59.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.086+0000 7f69da000640 1 -- 172.21.15.69:0/3699195887 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f69d401c070 con 0x7f69dc10e7f0 2024-04-24T09:09:59.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.086+0000 7f69e73d2740 1 -- start start 2024-04-24T09:09:59.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.086+0000 7f69e73d2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c87a30 0x5641f2c87e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.086+0000 7f69e73d2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c88350 0x5641f2c924e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.086+0000 7f69e73d2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c92a20 0x5641f2c94e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.086+0000 7f69e73d2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5641f2bed1f0 con 0x5641f2c92a20 2024-04-24T09:09:59.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.086+0000 7f69e73d2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5641f2b93350 con 0x5641f2c87a30 2024-04-24T09:09:59.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.086+0000 7f69e73d2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5641f2be91b0 con 0x5641f2c88350 2024-04-24T09:09:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.087+0000 7f69e1200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c92a20 0x5641f2c94e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.087+0000 7f69e1200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c92a20 0x5641f2c94e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39368/0 (socket says 172.21.15.69:39368) 2024-04-24T09:09:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.087+0000 7f69e1200640 1 -- 172.21.15.69:0/1676903875 learned_addr learned my addr 172.21.15.69:0/1676903875 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.087+0000 7f69e1c00640 1 --2- 172.21.15.69:0/1676903875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c88350 0x5641f2c924e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.087+0000 7f69e1200640 1 --2- 172.21.15.69:0/1676903875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c87a30 0x5641f2c87e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.087+0000 7f69e1200640 1 -- 172.21.15.69:0/1676903875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c88350 msgr2=0x5641f2c924e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.087+0000 7f69e1200640 1 --2- 172.21.15.69:0/1676903875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c88350 0x5641f2c924e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.087+0000 7f69e1200640 1 -- 172.21.15.69:0/1676903875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c87a30 msgr2=0x5641f2c87e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.087+0000 7f69e1200640 1 --2- 172.21.15.69:0/1676903875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c87a30 0x5641f2c87e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.087+0000 7f69e1200640 1 -- 172.21.15.69:0/1676903875 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f2be7f50 con 0x5641f2c92a20 2024-04-24T09:09:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.087+0000 7f69da000640 1 --2- 172.21.15.69:0/3699195887 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69c40a3d60 0x7f69c40a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.087+0000 7f69e1200640 1 --2- 172.21.15.69:0/1676903875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c92a20 0x5641f2c94e10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f69c800add0 tx=0x7f69c800a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497beff2a99dc592 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.087+0000 7f69da000640 1 -- 172.21.15.69:0/3699195887 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f69d40d00a0 con 0x7f69dc10e7f0 2024-04-24T09:09:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.088+0000 7f69e1c00640 1 --2- 172.21.15.69:0/3699195887 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69c40a3d60 0x7f69c40a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.088+0000 7f69cea00640 1 -- 172.21.15.69:0/1676903875 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c801c070 con 0x5641f2c92a20 2024-04-24T09:09:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.088+0000 7f69cea00640 1 -- 172.21.15.69:0/1676903875 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69c800fd70 con 0x5641f2c92a20 2024-04-24T09:09:59.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.088+0000 7f69cea00640 1 -- 172.21.15.69:0/1676903875 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8014650 con 0x5641f2c92a20 2024-04-24T09:09:59.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.089+0000 7f69e73d2740 1 -- 172.21.15.69:0/1676903875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c92a20 msgr2=0x5641f2c94e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.089+0000 7f69e1c00640 1 --2- 172.21.15.69:0/3699195887 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69c40a3d60 0x7f69c40a6220 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f69d0006010 tx=0x7f69d0031400 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:59.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.089+0000 7f69e73d2740 1 --2- 172.21.15.69:0/1676903875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c92a20 0x5641f2c94e10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f69c800add0 tx=0x7f69c800a470 comp rx=0 tx=0).stop 2024-04-24T09:09:59.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.090+0000 7f69e73d2740 1 -- 172.21.15.69:0/1676903875 shutdown_connections 2024-04-24T09:09:59.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.090+0000 7f69e73d2740 1 --2- 172.21.15.69:0/1676903875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c92a20 0x5641f2c94e10 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.090+0000 7f69e73d2740 1 --2- 172.21.15.69:0/1676903875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c88350 0x5641f2c924e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.090+0000 7f69e73d2740 1 --2- 172.21.15.69:0/1676903875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c87a30 0x5641f2c87e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.090+0000 7f69e73d2740 1 -- 172.21.15.69:0/1676903875 >> 172.21.15.69:0/1676903875 conn(0x5641f2c83d50 msgr2=0x5641f2c84130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:59.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.090+0000 7f69e73d2740 1 -- 172.21.15.69:0/1676903875 shutdown_connections 2024-04-24T09:09:59.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.090+0000 7f69e73d2740 1 -- 172.21.15.69:0/1676903875 wait complete. 2024-04-24T09:09:59.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.090+0000 7f69e73d2740 1 -- start start 2024-04-24T09:09:59.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.091+0000 7f69e73d2740 1 Processor -- start 2024-04-24T09:09:59.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.091+0000 7f69e73d2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c87a30 0x5641f2c8fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.091+0000 7f69e73d2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c88350 0x5641f2c8ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.091+0000 7f69e73d2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c92a20 0x5641f2c904a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.091+0000 7f69e73d2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5641f2c95810 con 0x5641f2c88350 2024-04-24T09:09:59.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.091+0000 7f69e73d2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5641f2ba6690 con 0x5641f2c87a30 2024-04-24T09:09:59.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.091+0000 7f69e73d2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5641f2c95990 con 0x5641f2c92a20 2024-04-24T09:09:59.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.091+0000 7f69e73d2740 10 client.0 _ll_register_callbacks cb 0x5641f2be2650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:59.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.091+0000 7f69e1200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c88350 0x5641f2c8ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.092+0000 7f69e1200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c88350 0x5641f2c8ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39382/0 (socket says 172.21.15.69:39382) 2024-04-24T09:09:59.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.092+0000 7f69e1200640 1 -- 172.21.15.69:0/533143062 learned_addr learned my addr 172.21.15.69:0/533143062 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:59.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.092+0000 7f69e1c00640 1 --2- 172.21.15.69:0/533143062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c87a30 0x5641f2c8fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.092+0000 7f69e2600640 1 --2- 172.21.15.69:0/533143062 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c92a20 0x5641f2c904a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.092+0000 7f69e1200640 1 -- 172.21.15.69:0/533143062 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c92a20 msgr2=0x5641f2c904a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:59.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.092+0000 7f69e1200640 1 --2- 172.21.15.69:0/533143062 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c92a20 0x5641f2c904a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.092+0000 7f69e1200640 1 -- 172.21.15.69:0/533143062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c87a30 msgr2=0x5641f2c8fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.092+0000 7f69e1200640 1 --2- 172.21.15.69:0/533143062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c87a30 0x5641f2c8fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.092+0000 7f69e1200640 1 -- 172.21.15.69:0/533143062 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f2ba5920 con 0x5641f2c88350 2024-04-24T09:09:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.092+0000 7f69e1200640 1 --2- 172.21.15.69:0/533143062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c88350 0x5641f2c8ff60 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f69c80178a0 tx=0x7f69c80178d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fbb7e3d7e04b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.092+0000 7f69c3400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.092+0000 7f69c3400640 1 -- 172.21.15.69:0/533143062 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8024020 con 0x5641f2c88350 2024-04-24T09:09:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.092+0000 7f69c3400640 1 -- 172.21.15.69:0/533143062 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f69c8020400 con 0x5641f2c88350 2024-04-24T09:09:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.093+0000 7f69e73d2740 1 -- 172.21.15.69:0/533143062 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69c800fd70 con 0x5641f2c88350 2024-04-24T09:09:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.093+0000 7f69c3400640 1 -- 172.21.15.69:0/533143062 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69c802f050 con 0x5641f2c88350 2024-04-24T09:09:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.093+0000 7f69e73d2740 10 client.16500 fetch_fsmap learned FSMap version 29 2024-04-24T09:09:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.093+0000 7f69e73d2740 1 -- 172.21.15.69:0/533143062 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5641f29c97d0 con 0x5641f2c88350 2024-04-24T09:09:59.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.093+0000 7f69c3400640 1 -- 172.21.15.69:0/533143062 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69c800fd70 con 0x5641f2c88350 2024-04-24T09:09:59.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.093+0000 7f69c3400640 1 -- 172.21.15.69:0/533143062 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f69c8015040 con 0x5641f2c88350 2024-04-24T09:09:59.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.093+0000 7f69e73d2740 10 client.16500 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:09:59.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.093+0000 7f69e73d2740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:59.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.093+0000 7f69e73d2740 20 client.16500 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:59.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.093+0000 7f69e73d2740 1 --2- 172.21.15.69:0/533143062 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5641f2ca2d40 0x5641f2ca5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.093+0000 7f69e73d2740 4 client.16500 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:59.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.093+0000 7f69e73d2740 1 -- 172.21.15.69:0/533143062 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641f2bfac10 con 0x5641f2ca2d40 2024-04-24T09:09:59.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.094+0000 7f69e1c00640 1 --2- 172.21.15.69:0/533143062 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5641f2ca2d40 0x5641f2ca5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.095+0000 7f69e1c00640 1 --2- 172.21.15.69:0/533143062 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5641f2ca2d40 0x5641f2ca5130 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46d354ed1d8d22c1 server_cookie=d0a68f738b90f866 in_seq=0 out_seq=0 2024-04-24T09:09:59.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.095+0000 7f69c3400640 10 client.16500 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:59.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.098+0000 7f69c3400640 1 -- 172.21.15.69:0/533143062 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641f2bfac10 con 0x5641f2ca2d40 2024-04-24T09:09:59.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.098+0000 7f69c3400640 10 client.16500 handle_command_reply: tid=0 2024-04-24T09:09:59.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.098+0000 7f69e73d2740 1 client.16500 shutdown 2024-04-24T09:09:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.098+0000 7f69e73d2740 1 -- 172.21.15.69:0/533143062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c88350 msgr2=0x5641f2c8ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.098+0000 7f69e73d2740 1 --2- 172.21.15.69:0/533143062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c88350 0x5641f2c8ff60 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f69c80178a0 tx=0x7f69c80178d0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.098+0000 7f69e73d2740 1 -- 172.21.15.69:0/533143062 shutdown_connections 2024-04-24T09:09:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.098+0000 7f69e73d2740 1 --2- 172.21.15.69:0/533143062 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5641f2ca2d40 0x5641f2ca5130 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.098+0000 7f69e73d2740 1 --2- 172.21.15.69:0/533143062 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c92a20 0x5641f2c904a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.098+0000 7f69c3400640 0 client.16500 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.098+0000 7f69e73d2740 1 --2- 172.21.15.69:0/533143062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c88350 0x5641f2c8ff60 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.098+0000 7f69e73d2740 1 --2- 172.21.15.69:0/533143062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c87a30 0x5641f2c8fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.098+0000 7f69e73d2740 1 -- 172.21.15.69:0/533143062 >> 172.21.15.69:0/533143062 conn(0x5641f2c83d50 msgr2=0x5641f2c89b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.098+0000 7f69e73d2740 1 -- 172.21.15.69:0/533143062 shutdown_connections 2024-04-24T09:09:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.098+0000 7f69e73d2740 1 -- 172.21.15.69:0/533143062 wait complete. 2024-04-24T09:09:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.098+0000 7f69e73d2740 20 client.16500 trim_cache size 0 max 16384 2024-04-24T09:09:59.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.119+0000 7f69e73d2740 1 Processor -- start 2024-04-24T09:09:59.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.119+0000 7f69e73d2740 1 -- start start 2024-04-24T09:09:59.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.119+0000 7f69e73d2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c87a30 0x5641f2c9e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.119+0000 7f69e73d2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c88350 0x5641f2c8a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.119+0000 7f69e73d2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c92a20 0x5641f2caa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.119+0000 7f69e73d2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc1b3c90 con 0x5641f2c92a20 2024-04-24T09:09:59.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.119+0000 7f69e73d2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc1b14d0 con 0x5641f2c88350 2024-04-24T09:09:59.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.119+0000 7f69e73d2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc1af320 con 0x5641f2c87a30 2024-04-24T09:09:59.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.120+0000 7f69e2600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c92a20 0x5641f2caa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.120+0000 7f69e2600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c92a20 0x5641f2caa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39398/0 (socket says 172.21.15.69:39398) 2024-04-24T09:09:59.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.120+0000 7f69e2600640 1 -- 172.21.15.69:0/1250916440 learned_addr learned my addr 172.21.15.69:0/1250916440 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:59.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.120+0000 7f69e1c00640 1 --2- 172.21.15.69:0/1250916440 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c87a30 0x5641f2c9e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.120+0000 7f69e1200640 1 --2- 172.21.15.69:0/1250916440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c88350 0x5641f2c8a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.120+0000 7f69e2600640 1 -- 172.21.15.69:0/1250916440 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c87a30 msgr2=0x5641f2c9e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:59.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.120+0000 7f69e2600640 1 --2- 172.21.15.69:0/1250916440 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c87a30 0x5641f2c9e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.120+0000 7f69e2600640 1 -- 172.21.15.69:0/1250916440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c88350 msgr2=0x5641f2c8a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:59.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.120+0000 7f69e2600640 1 --2- 172.21.15.69:0/1250916440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c88350 0x5641f2c8a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.120+0000 7f69e2600640 1 -- 172.21.15.69:0/1250916440 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f28ff3f0 con 0x5641f2c92a20 2024-04-24T09:09:59.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.120+0000 7f69e2600640 1 --2- 172.21.15.69:0/1250916440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c92a20 0x5641f2caa760 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f69d400b9b0 tx=0x7f69d409c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276ac6a688854b5c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:59.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.121+0000 7f69ce000640 1 -- 172.21.15.69:0/1250916440 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d40b5070 con 0x5641f2c92a20 2024-04-24T09:09:59.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.121+0000 7f69ce000640 1 -- 172.21.15.69:0/1250916440 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69d40a4a20 con 0x5641f2c92a20 2024-04-24T09:09:59.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.121+0000 7f69ce000640 1 -- 172.21.15.69:0/1250916440 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d40145f0 con 0x5641f2c92a20 2024-04-24T09:09:59.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.121+0000 7f69e73d2740 1 -- 172.21.15.69:0/1250916440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c92a20 msgr2=0x5641f2caa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.121+0000 7f69e73d2740 1 --2- 172.21.15.69:0/1250916440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c92a20 0x5641f2caa760 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f69d400b9b0 tx=0x7f69d409c5f0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.122+0000 7f69e73d2740 1 -- 172.21.15.69:0/1250916440 shutdown_connections 2024-04-24T09:09:59.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.122+0000 7f69e73d2740 1 --2- 172.21.15.69:0/1250916440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c92a20 0x5641f2caa760 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.122+0000 7f69e73d2740 1 --2- 172.21.15.69:0/1250916440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c88350 0x5641f2c8a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.122+0000 7f69e73d2740 1 --2- 172.21.15.69:0/1250916440 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c87a30 0x5641f2c9e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.122+0000 7f69e73d2740 1 -- 172.21.15.69:0/1250916440 >> 172.21.15.69:0/1250916440 conn(0x5641f2c83d50 msgr2=0x5641f2c90270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:59.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.122+0000 7f69e73d2740 1 -- 172.21.15.69:0/1250916440 shutdown_connections 2024-04-24T09:09:59.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.122+0000 7f69e73d2740 1 -- 172.21.15.69:0/1250916440 wait complete. 2024-04-24T09:09:59.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.122+0000 7f69e73d2740 1 -- start start 2024-04-24T09:09:59.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.122+0000 7f69e73d2740 1 Processor -- start 2024-04-24T09:09:59.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.123+0000 7f69e73d2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c87a30 0x5641f2c9ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.123+0000 7f69e73d2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c88350 0x5641f2c961a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.123+0000 7f69e73d2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c92a20 0x5641f2c966e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.123+0000 7f69e73d2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc1af990 con 0x5641f2c87a30 2024-04-24T09:09:59.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.123+0000 7f69e73d2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc1b0e00 con 0x5641f2c88350 2024-04-24T09:09:59.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.123+0000 7f69e73d2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc1b0060 con 0x5641f2c92a20 2024-04-24T09:09:59.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.123+0000 7f69e73d2740 10 client.0 _ll_register_callbacks cb 0x7f69c801a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:59.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.123+0000 7f69e1c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c87a30 0x5641f2c9ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.123+0000 7f69e1c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c87a30 0x5641f2c9ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39408/0 (socket says 172.21.15.69:39408) 2024-04-24T09:09:59.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.123+0000 7f69e1c00640 1 -- 172.21.15.69:0/1851978658 learned_addr learned my addr 172.21.15.69:0/1851978658 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:59.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.123+0000 7f69e1200640 1 --2- 172.21.15.69:0/1851978658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c88350 0x5641f2c961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.124+0000 7f69e1c00640 1 -- 172.21.15.69:0/1851978658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c92a20 msgr2=0x5641f2c966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:59.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.124+0000 7f69e1c00640 1 --2- 172.21.15.69:0/1851978658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c92a20 0x5641f2c966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.124+0000 7f69e1c00640 1 -- 172.21.15.69:0/1851978658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c88350 msgr2=0x5641f2c961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:59.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.124+0000 7f69e1c00640 1 --2- 172.21.15.69:0/1851978658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c88350 0x5641f2c961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.124+0000 7f69e1c00640 1 -- 172.21.15.69:0/1851978658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f28bf0f0 con 0x5641f2c87a30 2024-04-24T09:09:59.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.124+0000 7f69e1200640 1 --2- 172.21.15.69:0/1851978658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c88350 0x5641f2c961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:59.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.124+0000 7f69e1c00640 1 --2- 172.21.15.69:0/1851978658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c87a30 0x5641f2c9ad30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009230 tx=0x7f69d0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc89e63b10b0d4e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:59.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.124+0000 7f69c3400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:59.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.124+0000 7f69c3400640 1 -- 172.21.15.69:0/1851978658 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0035050 con 0x5641f2c87a30 2024-04-24T09:09:59.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.124+0000 7f69e73d2740 1 -- 172.21.15.69:0/1851978658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69d40a4a20 con 0x5641f2c87a30 2024-04-24T09:09:59.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.124+0000 7f69c3400640 1 -- 172.21.15.69:0/1851978658 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f69d0050250 con 0x5641f2c87a30 2024-04-24T09:09:59.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.125+0000 7f69c3400640 1 -- 172.21.15.69:0/1851978658 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69d0060050 con 0x5641f2c87a30 2024-04-24T09:09:59.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.125+0000 7f69e73d2740 10 client.16506 fetch_fsmap learned FSMap version 29 2024-04-24T09:09:59.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.125+0000 7f69e73d2740 1 -- 172.21.15.69:0/1851978658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5641f2bb2810 con 0x5641f2c87a30 2024-04-24T09:09:59.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.125+0000 7f69c3400640 1 -- 172.21.15.69:0/1851978658 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69d40a4a20 con 0x5641f2c87a30 2024-04-24T09:09:59.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.125+0000 7f69c3400640 1 -- 172.21.15.69:0/1851978658 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f69d0031930 con 0x5641f2c87a30 2024-04-24T09:09:59.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.125+0000 7f69e73d2740 10 client.16506 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:09:59.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.125+0000 7f69e73d2740 10 client.16506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:59.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.125+0000 7f69e73d2740 20 client.16506 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:59.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.125+0000 7f69e73d2740 1 --2- 172.21.15.69:0/1851978658 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5641f2d091c0 0x5641f2d0b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.125+0000 7f69e73d2740 4 client.16506 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:09:59.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.125+0000 7f69e73d2740 1 -- 172.21.15.69:0/1851978658 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5641f2bf4a80 con 0x5641f2d091c0 2024-04-24T09:09:59.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.126+0000 7f69e1200640 1 --2- 172.21.15.69:0/1851978658 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5641f2d091c0 0x5641f2d0b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.127+0000 7f69e1200640 1 --2- 172.21.15.69:0/1851978658 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5641f2d091c0 0x5641f2d0b600 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc10fe7b8f6298a server_cookie=ec5e05c7702686c2 in_seq=0 out_seq=0 2024-04-24T09:09:59.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.127+0000 7f69c3400640 10 client.16506 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:59.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.128+0000 7f69c3400640 1 -- 172.21.15.69:0/1851978658 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5641f2bf4a80 con 0x5641f2d091c0 2024-04-24T09:09:59.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.128+0000 7f69c3400640 10 client.16506 handle_command_reply: tid=0 2024-04-24T09:09:59.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.128+0000 7f69e73d2740 1 client.16506 shutdown 2024-04-24T09:09:59.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.128+0000 7f69e73d2740 1 -- 172.21.15.69:0/1851978658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c87a30 msgr2=0x5641f2c9ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.128+0000 7f69e73d2740 1 --2- 172.21.15.69:0/1851978658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c87a30 0x5641f2c9ad30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009230 tx=0x7f69d0009260 comp rx=0 tx=0).stop 2024-04-24T09:09:59.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.128+0000 7f69e73d2740 1 -- 172.21.15.69:0/1851978658 shutdown_connections 2024-04-24T09:09:59.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.128+0000 7f69e73d2740 1 --2- 172.21.15.69:0/1851978658 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5641f2d091c0 0x5641f2d0b600 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.128+0000 7f69e73d2740 1 --2- 172.21.15.69:0/1851978658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5641f2c92a20 0x5641f2c966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.128+0000 7f69e73d2740 1 --2- 172.21.15.69:0/1851978658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641f2c88350 0x5641f2c961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.128+0000 7f69e73d2740 1 --2- 172.21.15.69:0/1851978658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5641f2c87a30 0x5641f2c9ad30 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.128+0000 7f69c3400640 0 client.16506 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:09:59.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.128+0000 7f69e73d2740 1 -- 172.21.15.69:0/1851978658 >> 172.21.15.69:0/1851978658 conn(0x5641f2c83d50 msgr2=0x5641f2c82740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:59.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.128+0000 7f69e73d2740 1 -- 172.21.15.69:0/1851978658 shutdown_connections 2024-04-24T09:09:59.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.129+0000 7f69e73d2740 1 -- 172.21.15.69:0/1851978658 wait complete. 2024-04-24T09:09:59.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.129+0000 7f69e73d2740 20 client.16506 trim_cache size 0 max 16384 2024-04-24T09:09:59.131 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:09:59.131 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-04-24T09:09:59.131 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-04-24T09:09:59.131 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:09:59.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.129+0000 7f69e3000640 1 -- 172.21.15.69:0/3699195887 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69c40a3d60 msgr2=0x7f69c40a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.129+0000 7f69e3000640 1 --2- 172.21.15.69:0/3699195887 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69c40a3d60 0x7f69c40a6220 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f69d0006010 tx=0x7f69d0031400 comp rx=0 tx=0).stop 2024-04-24T09:09:59.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.129+0000 7f69e3000640 1 -- 172.21.15.69:0/3699195887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69dc10e7f0 msgr2=0x7f69dc19a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.129+0000 7f69e3000640 1 --2- 172.21.15.69:0/3699195887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69dc10e7f0 0x7f69dc19a860 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f69d400b9e0 tx=0x7f69d400bea0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.129+0000 7f69e3000640 1 -- 172.21.15.69:0/3699195887 shutdown_connections 2024-04-24T09:09:59.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.129+0000 7f69e3000640 1 --2- 172.21.15.69:0/3699195887 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69c40a3d60 0x7f69c40a6220 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.129+0000 7f69e3000640 1 --2- 172.21.15.69:0/3699195887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69dc10e7f0 0x7f69dc19a860 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.129+0000 7f69e3000640 1 --2- 172.21.15.69:0/3699195887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f69dc1015f0 0x7f69dc1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.129+0000 7f69e3000640 1 --2- 172.21.15.69:0/3699195887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69dc100c40 0x7f69dc1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.130+0000 7f69e3000640 1 -- 172.21.15.69:0/3699195887 >> 172.21.15.69:0/3699195887 conn(0x7f69dc0fc880 msgr2=0x7f69dc0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:59.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.130+0000 7f69e3000640 1 -- 172.21.15.69:0/3699195887 shutdown_connections 2024-04-24T09:09:59.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.130+0000 7f69e3000640 1 -- 172.21.15.69:0/3699195887 wait complete. 2024-04-24T09:09:59.148 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:09:59.149 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:09:59.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.345+0000 7f824d800640 1 -- 172.21.15.69:0/3227697405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8248069db0 msgr2=0x7f8248106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.345+0000 7f824d800640 1 --2- 172.21.15.69:0/3227697405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8248069db0 0x7f8248106a70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f8230009900 tx=0x7f823002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.345+0000 7f824d800640 1 -- 172.21.15.69:0/3227697405 shutdown_connections 2024-04-24T09:09:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.345+0000 7f824d800640 1 --2- 172.21.15.69:0/3227697405 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82481071a0 0x7f824810ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.345+0000 7f824d800640 1 --2- 172.21.15.69:0/3227697405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8248069db0 0x7f8248106a70 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.345+0000 7f824d800640 1 --2- 172.21.15.69:0/3227697405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8248069400 0x7f82480697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.345+0000 7f824d800640 1 -- 172.21.15.69:0/3227697405 >> 172.21.15.69:0/3227697405 conn(0x7f8248076840 msgr2=0x7f8248076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.346+0000 7f824d800640 1 -- 172.21.15.69:0/3227697405 shutdown_connections 2024-04-24T09:09:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.346+0000 7f824d800640 1 -- 172.21.15.69:0/3227697405 wait complete. 2024-04-24T09:09:59.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.346+0000 7f824d800640 1 Processor -- start 2024-04-24T09:09:59.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.347+0000 7f824d800640 1 -- start start 2024-04-24T09:09:59.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.347+0000 7f824d800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8248069400 0x7f82481010c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.347+0000 7f824d800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82481071a0 0x7f8248101600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.347+0000 7f824d800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8248101b40 0x7f82481a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.347+0000 7f824d800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f82481120c0 con 0x7f8248101b40 2024-04-24T09:09:59.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.347+0000 7f824d800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8248111f40 con 0x7f82481071a0 2024-04-24T09:09:59.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.347+0000 7f824d800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8248112240 con 0x7f8248069400 2024-04-24T09:09:59.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.347+0000 7f824ce00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8248101b40 0x7f82481a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.347+0000 7f824ce00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8248101b40 0x7f82481a6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39422/0 (socket says 172.21.15.69:39422) 2024-04-24T09:09:59.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.347+0000 7f824ce00640 1 -- 172.21.15.69:0/2928867908 learned_addr learned my addr 172.21.15.69:0/2928867908 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:59.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.348+0000 7f823f400640 1 --2- 172.21.15.69:0/2928867908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82481071a0 0x7f8248101600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.348+0000 7f8247e00640 1 --2- 172.21.15.69:0/2928867908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8248069400 0x7f82481010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.348+0000 7f824ce00640 1 -- 172.21.15.69:0/2928867908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8248069400 msgr2=0x7f82481010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.348+0000 7f824ce00640 1 --2- 172.21.15.69:0/2928867908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8248069400 0x7f82481010c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.348+0000 7f824ce00640 1 -- 172.21.15.69:0/2928867908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82481071a0 msgr2=0x7f8248101600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.348+0000 7f824ce00640 1 --2- 172.21.15.69:0/2928867908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82481071a0 0x7f8248101600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.348+0000 7f824ce00640 1 -- 172.21.15.69:0/2928867908 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82481a6e10 con 0x7f8248101b40 2024-04-24T09:09:59.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.348+0000 7f824ce00640 1 --2- 172.21.15.69:0/2928867908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8248101b40 0x7f82481a6770 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f824000cce0 tx=0x7f824000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:59.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.348+0000 7f8245600640 1 -- 172.21.15.69:0/2928867908 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824000f450 con 0x7f8248101b40 2024-04-24T09:09:59.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.349+0000 7f824d800640 1 -- 172.21.15.69:0/2928867908 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82481a7100 con 0x7f8248101b40 2024-04-24T09:09:59.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.349+0000 7f8245600640 1 -- 172.21.15.69:0/2928867908 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f824000faa0 con 0x7f8248101b40 2024-04-24T09:09:59.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.349+0000 7f824d800640 1 -- 172.21.15.69:0/2928867908 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82481a7620 con 0x7f8248101b40 2024-04-24T09:09:59.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.349+0000 7f8245600640 1 -- 172.21.15.69:0/2928867908 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240018b70 con 0x7f8248101b40 2024-04-24T09:09:59.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.350+0000 7f824d800640 1 -- 172.21.15.69:0/2928867908 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8224005290 con 0x7f8248101b40 2024-04-24T09:09:59.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.352+0000 7f8245600640 1 -- 172.21.15.69:0/2928867908 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f824000fc50 con 0x7f8248101b40 2024-04-24T09:09:59.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.353+0000 7f8245600640 1 --2- 172.21.15.69:0/2928867908 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f822c0a3da0 0x7f822c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.353+0000 7f8245600640 1 -- 172.21.15.69:0/2928867908 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f82400d10c0 con 0x7f8248101b40 2024-04-24T09:09:59.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.353+0000 7f8247e00640 1 --2- 172.21.15.69:0/2928867908 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f822c0a3da0 0x7f822c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.354+0000 7f8247e00640 1 --2- 172.21.15.69:0/2928867908 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f822c0a3da0 0x7f822c0a6260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8238005e10 tx=0x7f8238005da0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:59.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.359+0000 7f8245600640 1 -- 172.21.15.69:0/2928867908 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8240014030 con 0x7f8248101b40 2024-04-24T09:09:59.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.627+0000 7f824d800640 1 -- 172.21.15.69:0/2928867908 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8224005850 con 0x7f8248101b40 2024-04-24T09:09:59.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.630+0000 7f8245600640 1 -- 172.21.15.69:0/2928867908 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11880 (secure 0 0 0) 0x7f824009d3c0 con 0x7f8248101b40 2024-04-24T09:09:59.630 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:09:59.630 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/106106849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":106106849},{"type":"v1","addr":"172.21.15.69:6835","nonce":106106849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2217103498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2217103498},{"type":"v1","addr":"172.21.15.69:6837","nonce":2217103498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/3637851934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":3637851934},{"type":"v1","addr":"172.21.15.69:6839","nonce":3637851934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1747080062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1747080062},{"type":"v1","addr":"172.21.15.69:6841","nonce":1747080062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1510374637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1510374637},{"type":"v1","addr":"172.21.15.130:6833","nonce":1510374637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/1889126439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":1889126439},{"type":"v1","addr":"172.21.15.130:6835","nonce":1889126439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:04:01.991559+0000","modified":"2024-04-24T09:09:32.651039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34532,"mds_1":34496,"mds_2":34460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/2790982303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":2790982303},{"type":"v1","addr":"172.21.15.130:6837","nonce":2790982303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6839/4193111366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":4193111366},{"type":"v1","addr":"172.21.15.130:6839","nonce":4193111366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/3240654002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3240654002},{"type":"v1","addr":"172.21.15.81:6833","nonce":3240654002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":92,"addr":"172.21.15.81:6835/3260829578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3260829578},{"type":"v1","addr":"172.21.15.81:6835","nonce":3260829578}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34496":{"gid":34496,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":89,"addr":"172.21.15.81:6837/1973819593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1973819593},{"type":"v1","addr":"172.21.15.81:6837","nonce":1973819593}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34532":{"gid":34532,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/3185704097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":3185704097},{"type":"v1","addr":"172.21.15.81:6839","nonce":3185704097}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34532,"qdb_cluster":[34460,34496,34532]},"id":1}]} 2024-04-24T09:09:59.631 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 29 2024-04-24T09:09:59.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.632+0000 7f824d800640 1 -- 172.21.15.69:0/2928867908 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f822c0a3da0 msgr2=0x7f822c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.632+0000 7f824d800640 1 --2- 172.21.15.69:0/2928867908 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f822c0a3da0 0x7f822c0a6260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8238005e10 tx=0x7f8238005da0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.632+0000 7f824d800640 1 -- 172.21.15.69:0/2928867908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8248101b40 msgr2=0x7f82481a6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.632+0000 7f824d800640 1 --2- 172.21.15.69:0/2928867908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8248101b40 0x7f82481a6770 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f824000cce0 tx=0x7f824000a620 comp rx=0 tx=0).stop 2024-04-24T09:09:59.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.632+0000 7f824d800640 1 -- 172.21.15.69:0/2928867908 shutdown_connections 2024-04-24T09:09:59.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.632+0000 7f824d800640 1 --2- 172.21.15.69:0/2928867908 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f822c0a3da0 0x7f822c0a6260 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.632+0000 7f824d800640 1 --2- 172.21.15.69:0/2928867908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8248101b40 0x7f82481a6770 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.632+0000 7f824d800640 1 --2- 172.21.15.69:0/2928867908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82481071a0 0x7f8248101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.632+0000 7f824d800640 1 --2- 172.21.15.69:0/2928867908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8248069400 0x7f82481010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.632+0000 7f824d800640 1 -- 172.21.15.69:0/2928867908 >> 172.21.15.69:0/2928867908 conn(0x7f8248076840 msgr2=0x7f82480ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:59.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.633+0000 7f824d800640 1 -- 172.21.15.69:0/2928867908 shutdown_connections 2024-04-24T09:09:59.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.633+0000 7f824d800640 1 -- 172.21.15.69:0/2928867908 wait complete. 2024-04-24T09:09:59.652 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:09:59.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:09:59 smithi081 ceph-mon[30961]: pgmap v90: 129 pgs: 129 active+clean; 37 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 204 KiB/s wr, 157 op/s 2024-04-24T09:09:59.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:09:59 smithi069 ceph-mon[28170]: pgmap v90: 129 pgs: 129 active+clean; 37 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 204 KiB/s wr, 157 op/s 2024-04-24T09:09:59.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.852+0000 7fab15c00640 1 -- 172.21.15.69:0/1698076565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab10102ac0 msgr2=0x7fab10106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.852+0000 7fab15c00640 1 --2- 172.21.15.69:0/1698076565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab10102ac0 0x7fab10106b00 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7faafc009900 tx=0x7faafc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.852+0000 7fab15c00640 1 -- 172.21.15.69:0/1698076565 shutdown_connections 2024-04-24T09:09:59.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.852+0000 7fab15c00640 1 --2- 172.21.15.69:0/1698076565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab101071e0 0x7fab1010edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.852+0000 7fab15c00640 1 --2- 172.21.15.69:0/1698076565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab10102ac0 0x7fab10106b00 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.852+0000 7fab15c00640 1 --2- 172.21.15.69:0/1698076565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab10102110 0x7fab101024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.852+0000 7fab15c00640 1 -- 172.21.15.69:0/1698076565 >> 172.21.15.69:0/1698076565 conn(0x7fab100fd9d0 msgr2=0x7fab100ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:59.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.852+0000 7fab15c00640 1 -- 172.21.15.69:0/1698076565 shutdown_connections 2024-04-24T09:09:59.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.852+0000 7fab15c00640 1 -- 172.21.15.69:0/1698076565 wait complete. 2024-04-24T09:09:59.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.853+0000 7fab15c00640 1 Processor -- start 2024-04-24T09:09:59.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.853+0000 7fab15c00640 1 -- start start 2024-04-24T09:09:59.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.854+0000 7fab15c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab10102110 0x7fab10074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.854+0000 7fab15c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab101071e0 0x7fab10074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.854+0000 7fab15c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab10074d80 0x7fab101a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.854+0000 7fab15c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fab101121d0 con 0x7fab10102110 2024-04-24T09:09:59.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.854+0000 7fab15c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fab10112050 con 0x7fab101071e0 2024-04-24T09:09:59.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.854+0000 7fab15c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fab10112350 con 0x7fab10074d80 2024-04-24T09:09:59.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.854+0000 7fab0fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab10102110 0x7fab10074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.854+0000 7fab0fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab10102110 0x7fab10074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39446/0 (socket says 172.21.15.69:39446) 2024-04-24T09:09:59.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.854+0000 7fab0fe00640 1 -- 172.21.15.69:0/1448650494 learned_addr learned my addr 172.21.15.69:0/1448650494 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:59.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.855+0000 7fab15200640 1 --2- 172.21.15.69:0/1448650494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab10074d80 0x7fab101a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.855+0000 7fab0fe00640 1 -- 172.21.15.69:0/1448650494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab10074d80 msgr2=0x7fab101a6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.855+0000 7fab0fe00640 1 --2- 172.21.15.69:0/1448650494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab10074d80 0x7fab101a6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.855+0000 7fab0fe00640 1 -- 172.21.15.69:0/1448650494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab101071e0 msgr2=0x7fab10074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.855+0000 7fab0fe00640 1 --2- 172.21.15.69:0/1448650494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab101071e0 0x7fab10074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.855+0000 7fab0fe00640 1 -- 172.21.15.69:0/1448650494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab101a7060 con 0x7fab10102110 2024-04-24T09:09:59.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.855+0000 7fab15200640 1 --2- 172.21.15.69:0/1448650494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab10074d80 0x7fab101a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:59.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.855+0000 7fab0fe00640 1 --2- 172.21.15.69:0/1448650494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab10102110 0x7fab10074300 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fab0000b4a0 tx=0x7fab0000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:59.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.855+0000 7fab0cc00640 1 -- 172.21.15.69:0/1448650494 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00014070 con 0x7fab10102110 2024-04-24T09:09:59.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.855+0000 7fab0cc00640 1 -- 172.21.15.69:0/1448650494 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab00010ac0 con 0x7fab10102110 2024-04-24T09:09:59.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.855+0000 7fab15c00640 1 -- 172.21.15.69:0/1448650494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab101a7350 con 0x7fab10102110 2024-04-24T09:09:59.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.855+0000 7fab0cc00640 1 -- 172.21.15.69:0/1448650494 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00017c30 con 0x7fab10102110 2024-04-24T09:09:59.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.855+0000 7fab15c00640 1 -- 172.21.15.69:0/1448650494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab101a78a0 con 0x7fab10102110 2024-04-24T09:09:59.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.859+0000 7fab19ff0740 1 Processor -- start 2024-04-24T09:09:59.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.859+0000 7fab19ff0740 1 -- start start 2024-04-24T09:09:59.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.859+0000 7fab19ff0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561871229b40 0x561871229f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.859+0000 7fab19ff0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56187122a460 0x5618712345f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.859+0000 7fab19ff0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561871234b30 0x561871236f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.859+0000 7fab19ff0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56187118f1d0 con 0x561871229b40 2024-04-24T09:09:59.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.859+0000 7fab19ff0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561871135450 con 0x56187122a460 2024-04-24T09:09:59.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.859+0000 7fab19ff0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56187118b190 con 0x561871234b30 2024-04-24T09:09:59.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.859+0000 7fab0cc00640 1 -- 172.21.15.69:0/1448650494 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab00017070 con 0x7fab10102110 2024-04-24T09:09:59.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.859+0000 7fab15200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561871229b40 0x561871229f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.859+0000 7fab15200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561871229b40 0x561871229f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39458/0 (socket says 172.21.15.69:39458) 2024-04-24T09:09:59.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.859+0000 7fab15200640 1 -- 172.21.15.69:0/1271335073 learned_addr learned my addr 172.21.15.69:0/1271335073 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:59.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.860+0000 7fab0f400640 1 --2- 172.21.15.69:0/1271335073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561871234b30 0x561871236f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.860+0000 7fab15200640 1 -- 172.21.15.69:0/1271335073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561871234b30 msgr2=0x561871236f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:59.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.860+0000 7fab15200640 1 --2- 172.21.15.69:0/1271335073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561871234b30 0x561871236f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.860+0000 7fab15200640 1 -- 172.21.15.69:0/1271335073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56187122a460 msgr2=0x5618712345f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:59.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.860+0000 7fab15200640 1 --2- 172.21.15.69:0/1271335073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56187122a460 0x5618712345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.860+0000 7fab15200640 1 -- 172.21.15.69:0/1271335073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618711f2310 con 0x561871229b40 2024-04-24T09:09:59.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.860+0000 7fab15200640 1 --2- 172.21.15.69:0/1271335073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561871229b40 0x561871229f20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fab08008dd0 tx=0x7fab0800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50a401a13c7a300 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:59.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.861+0000 7fab05600640 1 -- 172.21.15.69:0/1271335073 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0800f830 con 0x561871229b40 2024-04-24T09:09:59.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.861+0000 7fab05600640 1 -- 172.21.15.69:0/1271335073 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab0800de20 con 0x561871229b40 2024-04-24T09:09:59.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.861+0000 7fab05600640 1 -- 172.21.15.69:0/1271335073 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08018df0 con 0x561871229b40 2024-04-24T09:09:59.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.861+0000 7fab0cc00640 1 --2- 172.21.15.69:0/1448650494 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7faaf40a3d50 0x7faaf40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.861+0000 7fab0cc00640 1 -- 172.21.15.69:0/1448650494 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fab000cfe60 con 0x7fab10102110 2024-04-24T09:09:59.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.861+0000 7fab0fe00640 1 --2- 172.21.15.69:0/1448650494 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7faaf40a3d50 0x7faaf40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.862+0000 7fab19ff0740 1 -- 172.21.15.69:0/1271335073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561871229b40 msgr2=0x561871229f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.862+0000 7fab19ff0740 1 --2- 172.21.15.69:0/1271335073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561871229b40 0x561871229f20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fab08008dd0 tx=0x7fab0800d710 comp rx=0 tx=0).stop 2024-04-24T09:09:59.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.862+0000 7fab19ff0740 1 -- 172.21.15.69:0/1271335073 shutdown_connections 2024-04-24T09:09:59.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.862+0000 7fab19ff0740 1 --2- 172.21.15.69:0/1271335073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561871234b30 0x561871236f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.862+0000 7fab19ff0740 1 --2- 172.21.15.69:0/1271335073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56187122a460 0x5618712345f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.862+0000 7fab19ff0740 1 --2- 172.21.15.69:0/1271335073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561871229b40 0x561871229f20 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.862+0000 7fab19ff0740 1 -- 172.21.15.69:0/1271335073 >> 172.21.15.69:0/1271335073 conn(0x561871225e60 msgr2=0x561871226240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.862+0000 7fab19ff0740 1 -- 172.21.15.69:0/1271335073 shutdown_connections 2024-04-24T09:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.862+0000 7fab19ff0740 1 -- 172.21.15.69:0/1271335073 wait complete. 2024-04-24T09:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.862+0000 7fab0fe00640 1 --2- 172.21.15.69:0/1448650494 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7faaf40a3d50 0x7faaf40a6210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x56187122b100 tx=0x7fab000047d0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.863+0000 7fab19ff0740 1 -- start start 2024-04-24T09:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.863+0000 7fab19ff0740 1 Processor -- start 2024-04-24T09:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.863+0000 7fab19ff0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561871229b40 0x561871234530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.863+0000 7fab19ff0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56187122a460 0x561871231890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.863+0000 7fab19ff0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561871234b30 0x561871231f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.863+0000 7fab19ff0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5618712379c0 con 0x56187122a460 2024-04-24T09:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.863+0000 7fab19ff0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561871148810 con 0x561871229b40 2024-04-24T09:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.863+0000 7fab19ff0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x561871237b40 con 0x561871234b30 2024-04-24T09:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.863+0000 7fab19ff0740 10 client.0 _ll_register_callbacks cb 0x561871116140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:59.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.864+0000 7fab0f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56187122a460 0x561871231890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.864+0000 7fab0f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56187122a460 0x561871231890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39466/0 (socket says 172.21.15.69:39466) 2024-04-24T09:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.864+0000 7fab0f400640 1 -- 172.21.15.69:0/3498555600 learned_addr learned my addr 172.21.15.69:0/3498555600 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.864+0000 7fab15200640 1 --2- 172.21.15.69:0/3498555600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561871229b40 0x561871234530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.864+0000 7fab15200640 1 --2- 172.21.15.69:0/3498555600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561871234b30 0x561871231f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.864+0000 7fab0f400640 1 -- 172.21.15.69:0/3498555600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561871234b30 msgr2=0x561871231f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.864+0000 7fab0f400640 1 --2- 172.21.15.69:0/3498555600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561871234b30 0x561871231f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.864+0000 7fab0f400640 1 -- 172.21.15.69:0/3498555600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561871229b40 msgr2=0x561871234530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.864+0000 7fab0f400640 1 --2- 172.21.15.69:0/3498555600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561871229b40 0x561871234530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.864+0000 7fab0f400640 1 -- 172.21.15.69:0/3498555600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618711593f0 con 0x56187122a460 2024-04-24T09:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.864+0000 7fab15200640 1 --2- 172.21.15.69:0/3498555600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561871234b30 0x561871231f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.864+0000 7fab0f400640 1 --2- 172.21.15.69:0/3498555600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56187122a460 0x561871231890 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7faafc009900 tx=0x7faafc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291fb39d9c143182 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.864+0000 7faafa000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.865+0000 7faafa000640 1 -- 172.21.15.69:0/3498555600 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc040070 con 0x56187122a460 2024-04-24T09:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.865+0000 7faafa000640 1 -- 172.21.15.69:0/3498555600 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7faafc0046c0 con 0x56187122a460 2024-04-24T09:09:59.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.865+0000 7fab19ff0740 1 -- 172.21.15.69:0/3498555600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab0800de20 con 0x56187122a460 2024-04-24T09:09:59.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.865+0000 7faafa000640 1 -- 172.21.15.69:0/3498555600 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faafc04c050 con 0x56187122a460 2024-04-24T09:09:59.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.865+0000 7fab19ff0740 10 client.16524 fetch_fsmap learned FSMap version 29 2024-04-24T09:09:59.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.865+0000 7fab19ff0740 1 -- 172.21.15.69:0/3498555600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561870ea13f0 con 0x56187122a460 2024-04-24T09:09:59.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.865+0000 7faafa000640 1 -- 172.21.15.69:0/3498555600 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab0800de20 con 0x56187122a460 2024-04-24T09:09:59.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.865+0000 7faafa000640 1 -- 172.21.15.69:0/3498555600 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7faafc03a040 con 0x56187122a460 2024-04-24T09:09:59.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.865+0000 7fab19ff0740 10 client.16524 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:09:59.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.866+0000 7fab19ff0740 10 client.16524 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:09:59.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.866+0000 7fab19ff0740 20 client.16524 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:59.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.866+0000 7fab19ff0740 1 --2- 172.21.15.69:0/3498555600 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x561871244cb0 0x5618712470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.866+0000 7fab19ff0740 4 client.16524 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:59.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.866+0000 7fab19ff0740 1 -- 172.21.15.69:0/3498555600 --> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56187119cbf0 con 0x561871244cb0 2024-04-24T09:09:59.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.866+0000 7fab15200640 1 --2- 172.21.15.69:0/3498555600 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x561871244cb0 0x5618712470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.867+0000 7fab15200640 1 --2- 172.21.15.69:0/3498555600 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x561871244cb0 0x5618712470e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=699f33c77f85c2c1 server_cookie=9500789ccfdd98c4 in_seq=0 out_seq=0 2024-04-24T09:09:59.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.867+0000 7faafa000640 10 client.16524 ms_handle_connect on v2:172.21.15.81:6834/3260829578 2024-04-24T09:09:59.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.869+0000 7faafa000640 1 -- 172.21.15.69:0/3498555600 <== mds.2 v2:172.21.15.81:6834/3260829578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56187119cbf0 con 0x561871244cb0 2024-04-24T09:09:59.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.869+0000 7faafa000640 10 client.16524 handle_command_reply: tid=0 2024-04-24T09:09:59.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.869+0000 7fab19ff0740 1 client.16524 shutdown 2024-04-24T09:09:59.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.869+0000 7fab19ff0740 1 -- 172.21.15.69:0/3498555600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56187122a460 msgr2=0x561871231890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.869+0000 7fab19ff0740 1 --2- 172.21.15.69:0/3498555600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56187122a460 0x561871231890 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7faafc009900 tx=0x7faafc005270 comp rx=0 tx=0).stop 2024-04-24T09:09:59.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.869+0000 7fab19ff0740 1 -- 172.21.15.69:0/3498555600 shutdown_connections 2024-04-24T09:09:59.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.869+0000 7fab19ff0740 1 --2- 172.21.15.69:0/3498555600 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x561871244cb0 0x5618712470e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.869+0000 7fab19ff0740 1 --2- 172.21.15.69:0/3498555600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561871234b30 0x561871231f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.869+0000 7fab19ff0740 1 --2- 172.21.15.69:0/3498555600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56187122a460 0x561871231890 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.869+0000 7fab19ff0740 1 --2- 172.21.15.69:0/3498555600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561871229b40 0x561871234530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.869+0000 7faafa000640 0 client.16524 ms_handle_reset on v2:172.21.15.81:6834/3260829578 2024-04-24T09:09:59.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.869+0000 7fab19ff0740 1 -- 172.21.15.69:0/3498555600 >> 172.21.15.69:0/3498555600 conn(0x561871225e60 msgr2=0x56187122bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:59.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.869+0000 7fab19ff0740 1 -- 172.21.15.69:0/3498555600 shutdown_connections 2024-04-24T09:09:59.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.869+0000 7fab19ff0740 1 -- 172.21.15.69:0/3498555600 wait complete. 2024-04-24T09:09:59.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.870+0000 7fab19ff0740 20 client.16524 trim_cache size 0 max 16384 2024-04-24T09:09:59.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.885+0000 7fab19ff0740 1 Processor -- start 2024-04-24T09:09:59.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.885+0000 7fab19ff0740 1 -- start start 2024-04-24T09:09:59.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.886+0000 7fab19ff0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561871229b40 0x561871240880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.886+0000 7fab19ff0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56187122a460 0x561871231170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.886+0000 7fab19ff0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561871234b30 0x5618712337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.886+0000 7fab19ff0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fab101b3c70 con 0x561871229b40 2024-04-24T09:09:59.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.886+0000 7fab19ff0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fab101b14b0 con 0x561871234b30 2024-04-24T09:09:59.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.886+0000 7fab19ff0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fab101af300 con 0x56187122a460 2024-04-24T09:09:59.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.886+0000 7fab15200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561871229b40 0x561871240880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.886+0000 7fab15200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561871229b40 0x561871240880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39470/0 (socket says 172.21.15.69:39470) 2024-04-24T09:09:59.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.886+0000 7fab15200640 1 -- 172.21.15.69:0/618419293 learned_addr learned my addr 172.21.15.69:0/618419293 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:59.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.886+0000 7fab15200640 1 -- 172.21.15.69:0/618419293 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56187122a460 msgr2=0x561871231170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:59.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.886+0000 7fab15200640 1 --2- 172.21.15.69:0/618419293 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56187122a460 0x561871231170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.886+0000 7fab15200640 1 -- 172.21.15.69:0/618419293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561871234b30 msgr2=0x5618712337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:59.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.886+0000 7fab15200640 1 --2- 172.21.15.69:0/618419293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561871234b30 0x5618712337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.886+0000 7fab15200640 1 -- 172.21.15.69:0/618419293 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561870f6b7d0 con 0x561871229b40 2024-04-24T09:09:59.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.887+0000 7fab15200640 1 --2- 172.21.15.69:0/618419293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561871229b40 0x561871240880 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fab08018a00 tx=0x7fab08008750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2d7242e69ec2b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:59.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.887+0000 7fab04c00640 1 -- 172.21.15.69:0/618419293 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0802f070 con 0x561871229b40 2024-04-24T09:09:59.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.887+0000 7fab04c00640 1 -- 172.21.15.69:0/618419293 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab08018480 con 0x561871229b40 2024-04-24T09:09:59.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.887+0000 7fab04c00640 1 -- 172.21.15.69:0/618419293 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0802a0d0 con 0x561871229b40 2024-04-24T09:09:59.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.888+0000 7fab19ff0740 1 -- 172.21.15.69:0/618419293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561871229b40 msgr2=0x561871240880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.888+0000 7fab19ff0740 1 --2- 172.21.15.69:0/618419293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561871229b40 0x561871240880 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fab08018a00 tx=0x7fab08008750 comp rx=0 tx=0).stop 2024-04-24T09:09:59.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.888+0000 7fab19ff0740 1 -- 172.21.15.69:0/618419293 shutdown_connections 2024-04-24T09:09:59.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.888+0000 7fab19ff0740 1 --2- 172.21.15.69:0/618419293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561871234b30 0x5618712337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.888+0000 7fab19ff0740 1 --2- 172.21.15.69:0/618419293 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56187122a460 0x561871231170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.888+0000 7fab19ff0740 1 --2- 172.21.15.69:0/618419293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561871229b40 0x561871240880 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.888+0000 7fab19ff0740 1 -- 172.21.15.69:0/618419293 >> 172.21.15.69:0/618419293 conn(0x561871225e60 msgr2=0x5618712330c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:59.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.888+0000 7fab19ff0740 1 -- 172.21.15.69:0/618419293 shutdown_connections 2024-04-24T09:09:59.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.888+0000 7fab19ff0740 1 -- 172.21.15.69:0/618419293 wait complete. 2024-04-24T09:09:59.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.888+0000 7fab19ff0740 1 -- start start 2024-04-24T09:09:59.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.889+0000 7fab19ff0740 1 Processor -- start 2024-04-24T09:09:59.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.889+0000 7fab19ff0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561871229b40 0x56187123d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.889+0000 7fab19ff0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56187122a460 0x56187122ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.889+0000 7fab19ff0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561871234b30 0x56187122f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.889+0000 7fab19ff0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fab101af970 con 0x56187122a460 2024-04-24T09:09:59.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.889+0000 7fab19ff0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fab101b0de0 con 0x561871234b30 2024-04-24T09:09:59.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.889+0000 7fab19ff0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fab101b0040 con 0x561871229b40 2024-04-24T09:09:59.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.889+0000 7fab19ff0740 10 client.0 _ll_register_callbacks cb 0x7faafc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:59.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.889+0000 7fab0f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56187122a460 0x56187122ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.889+0000 7fab0f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56187122a460 0x56187122ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39478/0 (socket says 172.21.15.69:39478) 2024-04-24T09:09:59.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.889+0000 7fab0f400640 1 -- 172.21.15.69:0/338625959 learned_addr learned my addr 172.21.15.69:0/338625959 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:09:59.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.890+0000 7fab0f400640 1 -- 172.21.15.69:0/338625959 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561871229b40 msgr2=0x56187123d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:59.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.890+0000 7fab0f400640 1 --2- 172.21.15.69:0/338625959 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561871229b40 0x56187123d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.890+0000 7fab0f400640 1 -- 172.21.15.69:0/338625959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561871234b30 msgr2=0x56187122f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:59.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.890+0000 7fab0f400640 1 --2- 172.21.15.69:0/338625959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561871234b30 0x56187122f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.890+0000 7fab0f400640 1 -- 172.21.15.69:0/338625959 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561871235900 con 0x56187122a460 2024-04-24T09:09:59.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.890+0000 7fab0f400640 1 --2- 172.21.15.69:0/338625959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56187122a460 0x56187122ed40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7faafc009850 tx=0x7faafc031520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c20c545467dd82 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:59.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.890+0000 7faafa000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:09:59.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.890+0000 7faafa000640 1 -- 172.21.15.69:0/338625959 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc055060 con 0x56187122a460 2024-04-24T09:09:59.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.890+0000 7faafa000640 1 -- 172.21.15.69:0/338625959 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7faafc003e40 con 0x56187122a460 2024-04-24T09:09:59.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.890+0000 7fab19ff0740 1 -- 172.21.15.69:0/338625959 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab08018480 con 0x56187122a460 2024-04-24T09:09:59.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.890+0000 7faafa000640 1 -- 172.21.15.69:0/338625959 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faafc05c050 con 0x56187122a460 2024-04-24T09:09:59.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.890+0000 7fab19ff0740 10 client.16530 fetch_fsmap learned FSMap version 29 2024-04-24T09:09:59.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.890+0000 7fab19ff0740 1 -- 172.21.15.69:0/338625959 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5618712ab5a0 con 0x56187122a460 2024-04-24T09:09:59.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.891+0000 7faafa000640 1 -- 172.21.15.69:0/338625959 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab08018480 con 0x56187122a460 2024-04-24T09:09:59.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.891+0000 7faafa000640 1 -- 172.21.15.69:0/338625959 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7faafc0046c0 con 0x56187122a460 2024-04-24T09:09:59.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.891+0000 7fab19ff0740 10 client.16530 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:09:59.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.891+0000 7fab19ff0740 10 client.16530 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:09:59.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.891+0000 7fab19ff0740 20 client.16530 populate_metadata read hostname 'smithi069' 2024-04-24T09:09:59.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.891+0000 7fab19ff0740 1 --2- 172.21.15.69:0/338625959 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x5618712abd80 0x5618712ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:59.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.891+0000 7fab19ff0740 4 client.16530 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:09:59.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.891+0000 7fab19ff0740 1 -- 172.21.15.69:0/338625959 --> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561871196a60 con 0x5618712abd80 2024-04-24T09:09:59.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.892+0000 7fab15200640 1 --2- 172.21.15.69:0/338625959 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x5618712abd80 0x5618712ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:59.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.892+0000 7fab15200640 1 --2- 172.21.15.69:0/338625959 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x5618712abd80 0x5618712ae1a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db2a556603fe19 server_cookie=79c3c063de0a8e0d in_seq=0 out_seq=0 2024-04-24T09:09:59.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.892+0000 7faafa000640 10 client.16530 ms_handle_connect on v2:172.21.15.81:6834/3260829578 2024-04-24T09:09:59.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.895+0000 7faafa000640 1 -- 172.21.15.69:0/338625959 <== mds.2 v2:172.21.15.81:6834/3260829578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+31270 (crc 0 0 0) 0x561871196a60 con 0x5618712abd80 2024-04-24T09:09:59.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.895+0000 7faafa000640 10 client.16530 handle_command_reply: tid=0 2024-04-24T09:09:59.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.896+0000 7fab19ff0740 1 client.16530 shutdown 2024-04-24T09:09:59.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.896+0000 7fab19ff0740 1 -- 172.21.15.69:0/338625959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56187122a460 msgr2=0x56187122ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.896+0000 7fab19ff0740 1 --2- 172.21.15.69:0/338625959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56187122a460 0x56187122ed40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7faafc009850 tx=0x7faafc031520 comp rx=0 tx=0).stop 2024-04-24T09:09:59.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.896+0000 7fab19ff0740 1 -- 172.21.15.69:0/338625959 shutdown_connections 2024-04-24T09:09:59.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.896+0000 7fab19ff0740 1 --2- 172.21.15.69:0/338625959 >> [v2:172.21.15.81:6834/3260829578,v1:172.21.15.81:6835/3260829578] conn(0x5618712abd80 0x5618712ae1a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.896+0000 7fab19ff0740 1 --2- 172.21.15.69:0/338625959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561871234b30 0x56187122f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.896+0000 7fab19ff0740 1 --2- 172.21.15.69:0/338625959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56187122a460 0x56187122ed40 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.896+0000 7fab19ff0740 1 --2- 172.21.15.69:0/338625959 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561871229b40 0x56187123d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.896+0000 7fab19ff0740 1 -- 172.21.15.69:0/338625959 >> 172.21.15.69:0/338625959 conn(0x561871225e60 msgr2=0x561871233950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:59.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.896+0000 7faafa000640 0 client.16530 ms_handle_reset on v2:172.21.15.81:6834/3260829578 2024-04-24T09:09:59.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.896+0000 7fab19ff0740 1 -- 172.21.15.69:0/338625959 shutdown_connections 2024-04-24T09:09:59.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.896+0000 7fab19ff0740 1 -- 172.21.15.69:0/338625959 wait complete. 2024-04-24T09:09:59.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.896+0000 7fab19ff0740 20 client.16530 trim_cache size 0 max 16384 2024-04-24T09:09:59.898 INFO:teuthology.orchestra.run.smithi069.stdout:[ 2024-04-24T09:09:59.898 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.898 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.898 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 16346359, 2024-04-24T09:09:59.898 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628532, 2024-04-24T09:09:59.898 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/attr_replace_test.c" 2024-04-24T09:09:59.898 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.898 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.898 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.898 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 16571550, 2024-04-24T09:09:59.898 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628602, 2024-04-24T09:09:59.899 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mmapcat.c" 2024-04-24T09:09:59.899 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.899 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.899 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.899 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 51813164, 2024-04-24T09:09:59.899 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628424, 2024-04-24T09:09:59.899 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/zoned" 2024-04-24T09:09:59.899 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.899 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.899 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.899 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 60752935, 2024-04-24T09:09:59.899 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628585, 2024-04-24T09:09:59.899 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/locktest.c" 2024-04-24T09:09:59.899 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.899 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.900 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.900 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 112369589, 2024-04-24T09:09:59.900 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628603, 2024-04-24T09:09:59.900 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/multi_open_unlink.c" 2024-04-24T09:09:59.900 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.900 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.900 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.900 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 114305430, 2024-04-24T09:09:59.900 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628543, 2024-04-24T09:09:59.900 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/detached_mounts_propagation.c" 2024-04-24T09:09:59.900 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.900 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.900 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.900 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 193744552, 2024-04-24T09:09:59.901 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628421, 2024-04-24T09:09:59.901 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ubifs" 2024-04-24T09:09:59.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.901 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.901 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.901 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 251521906, 2024-04-24T09:09:59.901 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628378, 2024-04-24T09:09:59.901 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/btrfs" 2024-04-24T09:09:59.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.901 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.901 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.901 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 347365333, 2024-04-24T09:09:59.901 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628320, 2024-04-24T09:09:59.901 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects/pack" 2024-04-24T09:09:59.902 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.902 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.902 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.902 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 354688950, 2024-04-24T09:09:59.902 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628701, 2024-04-24T09:09:59.902 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/vfstest.h" 2024-04-24T09:09:59.902 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.902 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.902 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.902 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 360648059, 2024-04-24T09:09:59.902 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628692, 2024-04-24T09:09:59.902 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/idmapped-mounts.c" 2024-04-24T09:09:59.902 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.902 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.903 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.903 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 397584521, 2024-04-24T09:09:59.903 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628579, 2024-04-24T09:09:59.903 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/helpers.h" 2024-04-24T09:09:59.903 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.903 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.903 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.903 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 423463873, 2024-04-24T09:09:59.903 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628523, 2024-04-24T09:09:59.903 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-io-setup-with-nonwritable-context-pointer.c" 2024-04-24T09:09:59.903 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.903 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.903 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.903 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 425485500, 2024-04-24T09:09:59.903 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628689, 2024-04-24T09:09:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/Makefile" 2024-04-24T09:09:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 568509839, 2024-04-24T09:09:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628338, 2024-04-24T09:09:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs" 2024-04-24T09:09:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 596321650, 2024-04-24T09:09:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628617, 2024-04-24T09:09:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/permname.c" 2024-04-24T09:09:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.904 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.905 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.905 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 623588392, 2024-04-24T09:09:59.905 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628513, 2024-04-24T09:09:59.905 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-cycle-write.c" 2024-04-24T09:09:59.905 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.905 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.905 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.905 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 657599122, 2024-04-24T09:09:59.905 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628410, 2024-04-24T09:09:59.905 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/preamble" 2024-04-24T09:09:59.905 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.905 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.905 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.905 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 661629170, 2024-04-24T09:09:59.905 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628517, 2024-04-24T09:09:59.906 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-hole-filling-race.c" 2024-04-24T09:09:59.906 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.906 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.906 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.906 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 691569761, 2024-04-24T09:09:59.906 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628386, 2024-04-24T09:09:59.906 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmerror" 2024-04-24T09:09:59.906 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.906 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.906 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.906 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 716536554, 2024-04-24T09:09:59.906 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628342, 2024-04-24T09:09:59.906 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/HEAD" 2024-04-24T09:09:59.906 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.906 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.907 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.907 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 739800478, 2024-04-24T09:09:59.907 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628511, 2024-04-24T09:09:59.907 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-append-write-read-race.c" 2024-04-24T09:09:59.907 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.907 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.907 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.907 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 973563093, 2024-04-24T09:09:59.907 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628394, 2024-04-24T09:09:59.907 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/f2fs" 2024-04-24T09:09:59.907 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.907 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.907 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.907 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1062566809, 2024-04-24T09:09:59.907 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628293, 2024-04-24T09:09:59.908 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/branches" 2024-04-24T09:09:59.908 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.908 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.908 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.908 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1071382724, 2024-04-24T09:09:59.908 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628383, 2024-04-24T09:09:59.908 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/defrag" 2024-04-24T09:09:59.908 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.908 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.908 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.908 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1109593105, 2024-04-24T09:09:59.908 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628382, 2024-04-24T09:09:59.908 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/config" 2024-04-24T09:09:59.908 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.909 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.909 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.909 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1138208390, 2024-04-24T09:09:59.909 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628695, 2024-04-24T09:09:59.909 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/mount-idmapped.c" 2024-04-24T09:09:59.909 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.909 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.909 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.909 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1150784194, 2024-04-24T09:09:59.909 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628319, 2024-04-24T09:09:59.909 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects" 2024-04-24T09:09:59.909 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.909 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.909 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.909 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1176805898, 2024-04-24T09:09:59.910 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628562, 2024-04-24T09:09:59.910 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fill.c" 2024-04-24T09:09:59.910 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.910 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.910 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.910 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1220744182, 2024-04-24T09:09:59.910 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628310, 2024-04-24T09:09:59.910 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs/heads" 2024-04-24T09:09:59.910 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.910 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.910 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.910 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1238440630, 2024-04-24T09:09:59.910 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628379, 2024-04-24T09:09:59.910 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/casefold" 2024-04-24T09:09:59.911 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.911 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.911 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.911 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1254632052, 2024-04-24T09:09:59.911 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628667, 2024-04-24T09:09:59.911 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_mtab.c" 2024-04-24T09:09:59.911 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.911 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.911 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.911 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1267301714, 2024-04-24T09:09:59.911 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628381, 2024-04-24T09:09:59.911 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/cgroup2" 2024-04-24T09:09:59.911 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.911 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.911 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.911 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1270955011, 2024-04-24T09:09:59.912 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628698, 2024-04-24T09:09:59.912 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/utils.c" 2024-04-24T09:09:59.912 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.912 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.912 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.912 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1272080746, 2024-04-24T09:09:59.912 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628608, 2024-04-24T09:09:59.912 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/parse-extent-tree.awk" 2024-04-24T09:09:59.912 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.912 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.912 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.912 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1309098796, 2024-04-24T09:09:59.912 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628648, 2024-04-24T09:09:59.912 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_type.c" 2024-04-24T09:09:59.912 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.912 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.913 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.913 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1351815011, 2024-04-24T09:09:59.913 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628347, 2024-04-24T09:09:59.913 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/heads/master" 2024-04-24T09:09:59.913 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.913 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.913 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.913 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1366126793, 2024-04-24T09:09:59.913 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628393, 2024-04-24T09:09:59.913 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ext4" 2024-04-24T09:09:59.913 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.913 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.913 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.914 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1449166465, 2024-04-24T09:09:59.914 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628423, 2024-04-24T09:09:59.914 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/xfs" 2024-04-24T09:09:59.914 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.914 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.914 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.914 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1461104012, 2024-04-24T09:09:59.914 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628405, 2024-04-24T09:09:59.914 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/module" 2024-04-24T09:09:59.914 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.914 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.914 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.914 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1471082224, 2024-04-24T09:09:59.914 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628639, 2024-04-24T09:09:59.915 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/stat_test.c" 2024-04-24T09:09:59.915 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.915 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.915 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.915 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1474779584, 2024-04-24T09:09:59.915 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628656, 2024-04-24T09:09:59.915 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_holes.c" 2024-04-24T09:09:59.915 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.915 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.915 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.915 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1478600361, 2024-04-24T09:09:59.915 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628696, 2024-04-24T09:09:59.915 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/tmpfs-idmapped-mounts.c" 2024-04-24T09:09:59.915 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.915 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.915 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.916 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1555501239, 2024-04-24T09:09:59.916 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628526, 2024-04-24T09:09:59.916 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aiodio_sparse2.c" 2024-04-24T09:09:59.916 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.916 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.916 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.916 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1622951904, 2024-04-24T09:09:59.916 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628351, 2024-04-24T09:09:59.916 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/index.lock" 2024-04-24T09:09:59.916 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.916 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.916 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.916 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1625471636, 2024-04-24T09:09:59.916 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628346, 2024-04-24T09:09:59.916 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/heads" 2024-04-24T09:09:59.917 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.917 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.917 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.917 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1650477561, 2024-04-24T09:09:59.917 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628705, 2024-04-24T09:09:59.917 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/xfsfind.c" 2024-04-24T09:09:59.917 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.917 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.917 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.917 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1691164371, 2024-04-24T09:09:59.917 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628395, 2024-04-24T09:09:59.917 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/fail_make_request" 2024-04-24T09:09:59.917 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.917 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.918 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.918 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1706998959, 2024-04-24T09:09:59.918 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628521, 2024-04-24T09:09:59.918 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-write-verify.c" 2024-04-24T09:09:59.918 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.918 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.918 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.918 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1752912995, 2024-04-24T09:09:59.918 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628525, 2024-04-24T09:09:59.918 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aiocp.c" 2024-04-24T09:09:59.918 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.918 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.918 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1778126529, 2024-04-24T09:09:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628520, 2024-04-24T09:09:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-subblock-eof-read.c" 2024-04-24T09:09:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1859138295, 2024-04-24T09:09:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628514, 2024-04-24T09:09:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-eof-race.c" 2024-04-24T09:09:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1901393888, 2024-04-24T09:09:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628563, 2024-04-24T09:09:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fill2.c" 2024-04-24T09:09:59.919 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.920 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.920 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.920 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1932647416, 2024-04-24T09:09:59.920 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628552, 2024-04-24T09:09:59.920 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/dumpfile" 2024-04-24T09:09:59.920 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.920 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.920 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.920 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1935523013, 2024-04-24T09:09:59.920 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628420, 2024-04-24T09:09:59.920 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/tracing" 2024-04-24T09:09:59.920 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.920 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1937343359, 2024-04-24T09:09:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628402, 2024-04-24T09:09:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/locktest" 2024-04-24T09:09:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1972562517, 2024-04-24T09:09:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628690, 2024-04-24T09:09:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/btrfs-idmapped-mounts.c" 2024-04-24T09:09:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1993416556, 2024-04-24T09:09:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628510, 2024-04-24T09:09:59.921 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-append-write-fallocate-race.c" 2024-04-24T09:09:59.922 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.922 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.922 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.922 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2022371200, 2024-04-24T09:09:59.922 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628700, 2024-04-24T09:09:59.922 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/vfstest.c" 2024-04-24T09:09:59.922 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.922 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.922 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.922 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2029111680, 2024-04-24T09:09:59.922 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628646, 2024-04-24T09:09:59.922 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_offset.c" 2024-04-24T09:09:59.922 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.922 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2058912674, 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628619, 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/popdir.pl" 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2067213811, 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628380, 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ceph" 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2075342033, 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628397, 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filter" 2024-04-24T09:09:59.923 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2135503088, 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628582, 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/iopat.c" 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2153018659, 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628321, 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects/info" 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2200440559, 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628559, 2024-04-24T09:09:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fiemap-fault.c" 2024-04-24T09:09:59.925 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.925 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.925 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.925 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2226745200, 2024-04-24T09:09:59.925 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628340, 2024-04-24T09:09:59.925 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin" 2024-04-24T09:09:59.925 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.925 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.925 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.925 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2252314040, 2024-04-24T09:09:59.925 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628401, 2024-04-24T09:09:59.925 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/inject" 2024-04-24T09:09:59.925 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.926 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.926 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.926 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2348526104, 2024-04-24T09:09:59.926 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628341, 2024-04-24T09:09:59.926 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin/HEAD" 2024-04-24T09:09:59.926 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.926 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.926 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.926 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2363111645, 2024-04-24T09:09:59.926 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628419, 2024-04-24T09:09:59.926 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/test_names" 2024-04-24T09:09:59.926 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.926 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.926 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.926 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2384831929, 2024-04-24T09:09:59.926 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628693, 2024-04-24T09:09:59.927 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/idmapped-mounts.h" 2024-04-24T09:09:59.927 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.927 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.927 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.927 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2406893047, 2024-04-24T09:09:59.927 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628415, 2024-04-24T09:09:59.927 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/renameat2" 2024-04-24T09:09:59.927 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.927 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.927 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.927 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2414428574, 2024-04-24T09:09:59.927 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628388, 2024-04-24T09:09:59.927 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmhugedisk" 2024-04-24T09:09:59.927 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.928 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.928 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.928 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2437225212, 2024-04-24T09:09:59.928 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628509, 2024-04-24T09:09:59.928 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/SOURCE" 2024-04-24T09:09:59.928 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.928 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.928 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.928 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2450297104, 2024-04-24T09:09:59.928 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628691, 2024-04-24T09:09:59.928 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/btrfs-idmapped-mounts.h" 2024-04-24T09:09:59.928 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.928 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.929 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.929 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2460575319, 2024-04-24T09:09:59.929 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628627, 2024-04-24T09:09:59.929 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/renameat2.c" 2024-04-24T09:09:59.929 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.929 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.929 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.929 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2470239890, 2024-04-24T09:09:59.929 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628417, 2024-04-24T09:09:59.929 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/report" 2024-04-24T09:09:59.929 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.929 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.930 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.930 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2493104681, 2024-04-24T09:09:59.930 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628311, 2024-04-24T09:09:59.930 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs/tags" 2024-04-24T09:09:59.930 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.930 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.930 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.930 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2510154718, 2024-04-24T09:09:59.930 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628412, 2024-04-24T09:09:59.930 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/quota" 2024-04-24T09:09:59.930 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.930 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.930 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.931 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2513561305, 2024-04-24T09:09:59.931 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628561, 2024-04-24T09:09:59.931 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fiexchange.h" 2024-04-24T09:09:59.931 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.931 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.931 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.931 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2529868763, 2024-04-24T09:09:59.931 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628512, 2024-04-24T09:09:59.931 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-cow-race.c" 2024-04-24T09:09:59.931 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.931 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.931 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.931 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2537755835, 2024-04-24T09:09:59.931 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628385, 2024-04-24T09:09:59.932 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmdust" 2024-04-24T09:09:59.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.932 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.932 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.932 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2554771788, 2024-04-24T09:09:59.932 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628404, 2024-04-24T09:09:59.932 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/metadump" 2024-04-24T09:09:59.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.932 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.932 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.932 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2616192978, 2024-04-24T09:09:59.932 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628666, 2024-04-24T09:09:59.932 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_mmap_writev_overlap.c" 2024-04-24T09:09:59.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.933 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.933 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.933 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2682881827, 2024-04-24T09:09:59.933 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628396, 2024-04-24T09:09:59.933 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filestreams" 2024-04-24T09:09:59.933 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.933 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.933 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.933 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2757371729, 2024-04-24T09:09:59.933 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628647, 2024-04-24T09:09:59.933 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_offset2.c" 2024-04-24T09:09:59.933 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.933 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.934 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.934 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2764086984, 2024-04-24T09:09:59.934 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628416, 2024-04-24T09:09:59.934 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/repair" 2024-04-24T09:09:59.934 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.934 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.934 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.934 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2766977774, 2024-04-24T09:09:59.934 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628403, 2024-04-24T09:09:59.934 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/log" 2024-04-24T09:09:59.934 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.934 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.934 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.935 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2775104699, 2024-04-24T09:09:59.935 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628392, 2024-04-24T09:09:59.935 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/encrypt" 2024-04-24T09:09:59.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.935 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.935 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.935 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2785148965, 2024-04-24T09:09:59.935 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628694, 2024-04-24T09:09:59.935 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/missing.h" 2024-04-24T09:09:59.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.935 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.935 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.936 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2806222874, 2024-04-24T09:09:59.936 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628640, 2024-04-24T09:09:59.936 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/statx.h" 2024-04-24T09:09:59.936 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.936 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.936 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.936 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2826799643, 2024-04-24T09:09:59.936 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628414, 2024-04-24T09:09:59.936 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/reflink" 2024-04-24T09:09:59.936 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.936 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.936 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.936 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2841452546, 2024-04-24T09:09:59.936 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628581, 2024-04-24T09:09:59.937 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/holetest.c" 2024-04-24T09:09:59.937 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.937 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.937 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.937 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2865704986, 2024-04-24T09:09:59.937 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628507, 2024-04-24T09:09:59.937 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress" 2024-04-24T09:09:59.937 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.937 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.937 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.937 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2895596337, 2024-04-24T09:09:59.937 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628408, 2024-04-24T09:09:59.937 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/perf" 2024-04-24T09:09:59.937 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.937 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.937 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.938 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2956292765, 2024-04-24T09:09:59.938 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628418, 2024-04-24T09:09:59.938 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/scsi_debug" 2024-04-24T09:09:59.938 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.938 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.938 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.938 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2967705891, 2024-04-24T09:09:59.938 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628596, 2024-04-24T09:09:59.938 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/md5.c" 2024-04-24T09:09:59.938 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.938 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.938 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.938 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2994931498, 2024-04-24T09:09:59.938 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628389, 2024-04-24T09:09:59.939 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmlogwrites" 2024-04-24T09:09:59.939 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.939 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.939 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.939 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3036975500, 2024-04-24T09:09:59.939 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628600, 2024-04-24T09:09:59.939 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mmap-rw-fault.c" 2024-04-24T09:09:59.939 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.939 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.939 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.939 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3078982028, 2024-04-24T09:09:59.939 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628577, 2024-04-24T09:09:59.939 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/global.h" 2024-04-24T09:09:59.939 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.939 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.940 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.940 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3110092865, 2024-04-24T09:09:59.940 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628413, 2024-04-24T09:09:59.940 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/rc" 2024-04-24T09:09:59.940 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.940 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.940 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.940 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3150696945, 2024-04-24T09:09:59.940 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628411, 2024-04-24T09:09:59.940 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/punch" 2024-04-24T09:09:59.940 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.940 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.940 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.940 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3175535340, 2024-04-24T09:09:59.940 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511627785, 2024-04-24T09:09:59.940 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/description" 2024-04-24T09:09:59.941 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.941 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.941 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.941 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3189380125, 2024-04-24T09:09:59.941 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628407, 2024-04-24T09:09:59.941 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/overlay" 2024-04-24T09:09:59.941 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.941 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.941 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.941 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3205728076, 2024-04-24T09:09:59.941 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628688, 2024-04-24T09:09:59.941 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs" 2024-04-24T09:09:59.941 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.941 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.941 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.942 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3213359877, 2024-04-24T09:09:59.942 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628409, 2024-04-24T09:09:59.942 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/populate" 2024-04-24T09:09:59.942 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.942 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.942 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.942 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3223676272, 2024-04-24T09:09:59.942 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628391, 2024-04-24T09:09:59.942 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dump" 2024-04-24T09:09:59.942 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.942 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.942 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.942 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3259465019, 2024-04-24T09:09:59.942 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628387, 2024-04-24T09:09:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmflakey" 2024-04-24T09:09:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3352404736, 2024-04-24T09:09:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628516, 2024-04-24T09:09:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-fcntl-race.c" 2024-04-24T09:09:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3364014463, 2024-04-24T09:09:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628625, 2024-04-24T09:09:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/readdir-while-renames.c" 2024-04-24T09:09:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.944 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3389909060, 2024-04-24T09:09:59.944 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628331, 2024-04-24T09:09:59.944 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/shallow" 2024-04-24T09:09:59.944 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.944 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.944 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.944 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3409724052, 2024-04-24T09:09:59.944 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628376, 2024-04-24T09:09:59.944 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/Makefile" 2024-04-24T09:09:59.944 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.944 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.944 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.944 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3436063123, 2024-04-24T09:09:59.944 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628333, 2024-04-24T09:09:59.944 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/packed-refs" 2024-04-24T09:09:59.944 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.945 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.945 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.945 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3487165587, 2024-04-24T09:09:59.945 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628556, 2024-04-24T09:09:59.945 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fake-dump-rootino.c" 2024-04-24T09:09:59.945 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.945 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.945 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.945 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3522762471, 2024-04-24T09:09:59.945 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628337, 2024-04-24T09:09:59.945 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs" 2024-04-24T09:09:59.945 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.945 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.945 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.945 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3535594575, 2024-04-24T09:09:59.946 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628377, 2024-04-24T09:09:59.946 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/attr" 2024-04-24T09:09:59.946 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.946 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.946 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.946 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3554403979, 2024-04-24T09:09:59.946 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628524, 2024-04-24T09:09:59.946 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-last-ref-held-by-io.c" 2024-04-24T09:09:59.946 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.946 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.946 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.946 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3643332925, 2024-04-24T09:09:59.946 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628542, 2024-04-24T09:09:59.946 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/deduperace.c" 2024-04-24T09:09:59.946 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.947 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.947 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.947 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3676432262, 2024-04-24T09:09:59.947 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628345, 2024-04-24T09:09:59.947 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/HEAD" 2024-04-24T09:09:59.947 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.947 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.947 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.947 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3683754233, 2024-04-24T09:09:59.947 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628518, 2024-04-24T09:09:59.947 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-invalidate-failure.c" 2024-04-24T09:09:59.947 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.947 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.947 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.947 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3792303172, 2024-04-24T09:09:59.947 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628390, 2024-04-24T09:09:59.948 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmthin" 2024-04-24T09:09:59.948 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.948 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.948 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.948 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3797619330, 2024-04-24T09:09:59.948 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628697, 2024-04-24T09:09:59.948 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/tmpfs-idmapped-mounts.h" 2024-04-24T09:09:59.948 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.948 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.948 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.948 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3815626013, 2024-04-24T09:09:59.948 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628398, 2024-04-24T09:09:59.948 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filter.btrfs" 2024-04-24T09:09:59.949 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.949 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.949 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.949 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3820118046, 2024-04-24T09:09:59.949 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628294, 2024-04-24T09:09:59.949 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks" 2024-04-24T09:09:59.949 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.949 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.949 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.949 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3850222811, 2024-04-24T09:09:59.949 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628384, 2024-04-24T09:09:59.949 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmdelay" 2024-04-24T09:09:59.949 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.949 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.949 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.949 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3886813943, 2024-04-24T09:09:59.950 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628422, 2024-04-24T09:09:59.950 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/verity" 2024-04-24T09:09:59.950 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.950 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.950 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.950 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3909347137, 2024-04-24T09:09:59.950 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628339, 2024-04-24T09:09:59.950 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes" 2024-04-24T09:09:59.950 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.950 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.950 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.950 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3918391727, 2024-04-24T09:09:59.950 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628599, 2024-04-24T09:09:59.950 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mkswap.c" 2024-04-24T09:09:59.951 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.951 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.951 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.951 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3923855366, 2024-04-24T09:09:59.951 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628399, 2024-04-24T09:09:59.951 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/fuzzy" 2024-04-24T09:09:59.951 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.951 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.951 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.951 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3924492183, 2024-04-24T09:09:59.951 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628309, 2024-04-24T09:09:59.951 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs" 2024-04-24T09:09:59.951 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.951 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.952 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.952 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3951747651, 2024-04-24T09:09:59.952 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628400, 2024-04-24T09:09:59.952 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/gcov" 2024-04-24T09:09:59.952 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.952 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.952 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.952 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3957244530, 2024-04-24T09:09:59.952 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628522, 2024-04-24T09:09:59.952 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-free-ring-with-bogus-nr-pages.c" 2024-04-24T09:09:59.952 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.952 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.952 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.952 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3977505621, 2024-04-24T09:09:59.953 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628350, 2024-04-24T09:09:59.953 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/config" 2024-04-24T09:09:59.953 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.953 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.953 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.953 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3986406120, 2024-04-24T09:09:59.953 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628519, 2024-04-24T09:09:59.953 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-invalidate-readahead.c" 2024-04-24T09:09:59.953 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.953 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.953 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.953 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4040529996, 2024-04-24T09:09:59.953 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628406, 2024-04-24T09:09:59.953 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/nfs" 2024-04-24T09:09:59.954 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.954 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.954 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.954 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4061375483, 2024-04-24T09:09:59.954 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628292, 2024-04-24T09:09:59.954 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/info/exclude" 2024-04-24T09:09:59.954 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.954 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.954 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.954 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4128685640, 2024-04-24T09:09:59.954 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628699, 2024-04-24T09:09:59.954 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/utils.h" 2024-04-24T09:09:59.954 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.954 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4138139763, 2024-04-24T09:09:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628515, 2024-04-24T09:09:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-extend-stat.c" 2024-04-24T09:09:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4151276909, 2024-04-24T09:09:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628508, 2024-04-24T09:09:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/Makefile" 2024-04-24T09:09:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:09:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:09:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:09:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4244343495, 2024-04-24T09:09:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628291, 2024-04-24T09:09:59.955 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/info" 2024-04-24T09:09:59.956 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:09:59.956 INFO:teuthology.orchestra.run.smithi069.stdout:] 2024-04-24T09:09:59.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.897+0000 7fab15c00640 1 -- 172.21.15.69:0/1448650494 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7faaf40a3d50 msgr2=0x7faaf40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.897+0000 7fab15c00640 1 --2- 172.21.15.69:0/1448650494 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7faaf40a3d50 0x7faaf40a6210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x56187122b100 tx=0x7fab000047d0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.897+0000 7fab15c00640 1 -- 172.21.15.69:0/1448650494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab10102110 msgr2=0x7fab10074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:59.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.897+0000 7fab15c00640 1 --2- 172.21.15.69:0/1448650494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab10102110 0x7fab10074300 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fab0000b4a0 tx=0x7fab0000b960 comp rx=0 tx=0).stop 2024-04-24T09:09:59.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.897+0000 7fab15c00640 1 -- 172.21.15.69:0/1448650494 shutdown_connections 2024-04-24T09:09:59.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.897+0000 7fab15c00640 1 --2- 172.21.15.69:0/1448650494 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7faaf40a3d50 0x7faaf40a6210 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.897+0000 7fab15c00640 1 --2- 172.21.15.69:0/1448650494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab10074d80 0x7fab101a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.897+0000 7fab15c00640 1 --2- 172.21.15.69:0/1448650494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab101071e0 0x7fab10074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.897+0000 7fab15c00640 1 --2- 172.21.15.69:0/1448650494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab10102110 0x7fab10074300 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:59.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.897+0000 7fab15c00640 1 -- 172.21.15.69:0/1448650494 >> 172.21.15.69:0/1448650494 conn(0x7fab100fd9d0 msgr2=0x7fab100fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:59.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.898+0000 7fab15c00640 1 -- 172.21.15.69:0/1448650494 shutdown_connections 2024-04-24T09:09:59.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:09:59.898+0000 7fab15c00640 1 -- 172.21.15.69:0/1448650494 wait complete. 2024-04-24T09:09:59.958 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:09:59.959 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:09:59 smithi130 ceph-mon[31553]: pgmap v90: 129 pgs: 129 active+clean; 37 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 204 KiB/s wr, 157 op/s 2024-04-24T09:10:00.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.170+0000 7f7e70a00640 1 -- 172.21.15.69:0/1080153841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e6c06a870 msgr2=0x7f7e6c06ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.170+0000 7f7e70a00640 1 --2- 172.21.15.69:0/1080153841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e6c06a870 0x7f7e6c06ac50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009900 tx=0x7f7e5c02a410 comp rx=0 tx=0).stop 2024-04-24T09:10:00.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.170+0000 7f7e70a00640 1 -- 172.21.15.69:0/1080153841 shutdown_connections 2024-04-24T09:10:00.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.170+0000 7f7e70a00640 1 --2- 172.21.15.69:0/1080153841 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e6c107330 0x7f7e6c10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.170+0000 7f7e70a00640 1 --2- 172.21.15.69:0/1080153841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e6c06b220 0x7f7e6c106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.170+0000 7f7e70a00640 1 --2- 172.21.15.69:0/1080153841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e6c06a870 0x7f7e6c06ac50 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.170+0000 7f7e70a00640 1 -- 172.21.15.69:0/1080153841 >> 172.21.15.69:0/1080153841 conn(0x7f7e6c076d00 msgr2=0x7f7e6c077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:00.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.170+0000 7f7e70a00640 1 -- 172.21.15.69:0/1080153841 shutdown_connections 2024-04-24T09:10:00.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.170+0000 7f7e70a00640 1 -- 172.21.15.69:0/1080153841 wait complete. 2024-04-24T09:10:00.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.171+0000 7f7e70a00640 1 Processor -- start 2024-04-24T09:10:00.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.171+0000 7f7e70a00640 1 -- start start 2024-04-24T09:10:00.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.171+0000 7f7e70a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e6c06a870 0x7f7e6c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.171+0000 7f7e70a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e6c107330 0x7f7e6c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.171+0000 7f7e70a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e6c19a810 0x7f7e6c19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.171+0000 7f7e70a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c112340 con 0x7f7e6c06a870 2024-04-24T09:10:00.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.171+0000 7f7e70a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1121c0 con 0x7f7e6c19a810 2024-04-24T09:10:00.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.171+0000 7f7e70a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1124c0 con 0x7f7e6c107330 2024-04-24T09:10:00.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.172+0000 7f7e69e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e6c06a870 0x7f7e6c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.172+0000 7f7e69e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e6c06a870 0x7f7e6c1a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39500/0 (socket says 172.21.15.69:39500) 2024-04-24T09:10:00.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.172+0000 7f7e69e00640 1 -- 172.21.15.69:0/581927444 learned_addr learned my addr 172.21.15.69:0/581927444 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:00.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.172+0000 7f7e69400640 1 --2- 172.21.15.69:0/581927444 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e6c107330 0x7f7e6c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.172+0000 7f7e6a800640 1 --2- 172.21.15.69:0/581927444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e6c19a810 0x7f7e6c19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.172+0000 7f7e69e00640 1 -- 172.21.15.69:0/581927444 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e6c107330 msgr2=0x7f7e6c1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.172+0000 7f7e69e00640 1 --2- 172.21.15.69:0/581927444 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e6c107330 0x7f7e6c1a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.172+0000 7f7e69e00640 1 -- 172.21.15.69:0/581927444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e6c19a810 msgr2=0x7f7e6c19acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.172+0000 7f7e69e00640 1 --2- 172.21.15.69:0/581927444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e6c19a810 0x7f7e6c19acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.172+0000 7f7e69e00640 1 -- 172.21.15.69:0/581927444 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6c19b3a0 con 0x7f7e6c06a870 2024-04-24T09:10:00.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.172+0000 7f7e6a800640 1 --2- 172.21.15.69:0/581927444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e6c19a810 0x7f7e6c19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:00.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.173+0000 7f7e69e00640 1 --2- 172.21.15.69:0/581927444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e6c06a870 0x7f7e6c1a06a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c005ab0 tx=0x7f7e5c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:00.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.173+0000 7f7e62a00640 1 -- 172.21.15.69:0/581927444 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c03f070 con 0x7f7e6c06a870 2024-04-24T09:10:00.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.173+0000 7f7e70a00640 1 -- 172.21.15.69:0/581927444 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6c19b630 con 0x7f7e6c06a870 2024-04-24T09:10:00.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.173+0000 7f7e62a00640 1 -- 172.21.15.69:0/581927444 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e5c031d60 con 0x7f7e6c06a870 2024-04-24T09:10:00.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.173+0000 7f7e70a00640 1 -- 172.21.15.69:0/581927444 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e6c1a7860 con 0x7f7e6c06a870 2024-04-24T09:10:00.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.173+0000 7f7e62a00640 1 -- 172.21.15.69:0/581927444 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c049060 con 0x7f7e6c06a870 2024-04-24T09:10:00.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.176+0000 7f7e73873740 1 Processor -- start 2024-04-24T09:10:00.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.176+0000 7f7e62a00640 1 -- 172.21.15.69:0/581927444 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7e5c042070 con 0x7f7e6c06a870 2024-04-24T09:10:00.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.177+0000 7f7e73873740 1 -- start start 2024-04-24T09:10:00.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.177+0000 7f7e73873740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.177+0000 7f7e73873740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8b9270 0x55792a8c3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.177+0000 7f7e73873740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8c3940 0x55792a8c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.177+0000 7f7e73873740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55792a81e1d0 con 0x55792a8b8950 2024-04-24T09:10:00.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.177+0000 7f7e73873740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55792a7c4450 con 0x55792a8c3940 2024-04-24T09:10:00.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.177+0000 7f7e73873740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55792a81a190 con 0x55792a8b9270 2024-04-24T09:10:00.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.177+0000 7f7e6a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.177+0000 7f7e6a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8b8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39502/0 (socket says 172.21.15.69:39502) 2024-04-24T09:10:00.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.177+0000 7f7e6a800640 1 -- 172.21.15.69:0/3993252698 learned_addr learned my addr 172.21.15.69:0/3993252698 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:00.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.178+0000 7f7e69400640 1 --2- 172.21.15.69:0/3993252698 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8c3940 0x55792a8c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.178+0000 7f7e69e00640 1 --2- 172.21.15.69:0/3993252698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8b9270 0x55792a8c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.178+0000 7f7e6a800640 1 -- 172.21.15.69:0/3993252698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8b9270 msgr2=0x55792a8c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:00.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.178+0000 7f7e6a800640 1 --2- 172.21.15.69:0/3993252698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8b9270 0x55792a8c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.178+0000 7f7e6a800640 1 -- 172.21.15.69:0/3993252698 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8c3940 msgr2=0x55792a8c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:00.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.178+0000 7f7e6a800640 1 --2- 172.21.15.69:0/3993252698 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8c3940 0x55792a8c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.178+0000 7f7e6a800640 1 -- 172.21.15.69:0/3993252698 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55792a881310 con 0x55792a8b8950 2024-04-24T09:10:00.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.178+0000 7f7e62a00640 1 --2- 172.21.15.69:0/581927444 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7e500a3d20 0x7f7e500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.178+0000 7f7e6a800640 1 --2- 172.21.15.69:0/3993252698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8b8d30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7e64007f30 tx=0x7f7e6400c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ef35320d61bfba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:00.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.178+0000 7f7e62a00640 1 -- 172.21.15.69:0/581927444 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7e5c0f1650 con 0x7f7e6c06a870 2024-04-24T09:10:00.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.179+0000 7f7e69e00640 1 --2- 172.21.15.69:0/581927444 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7e500a3d20 0x7f7e500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.179+0000 7f7e5b000640 1 -- 172.21.15.69:0/3993252698 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64014070 con 0x55792a8b8950 2024-04-24T09:10:00.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.179+0000 7f7e5b000640 1 -- 172.21.15.69:0/3993252698 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e6400fd00 con 0x55792a8b8950 2024-04-24T09:10:00.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.179+0000 7f7e5b000640 1 -- 172.21.15.69:0/3993252698 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400ecb0 con 0x55792a8b8950 2024-04-24T09:10:00.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.180+0000 7f7e73873740 1 -- 172.21.15.69:0/3993252698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 msgr2=0x55792a8b8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.180+0000 7f7e73873740 1 --2- 172.21.15.69:0/3993252698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8b8d30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7e64007f30 tx=0x7f7e6400c750 comp rx=0 tx=0).stop 2024-04-24T09:10:00.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.180+0000 7f7e69e00640 1 --2- 172.21.15.69:0/581927444 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7e500a3d20 0x7f7e500a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55792a8b9f10 tx=0x7f7e5c000bc0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:00.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.180+0000 7f7e73873740 1 -- 172.21.15.69:0/3993252698 shutdown_connections 2024-04-24T09:10:00.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.180+0000 7f7e73873740 1 --2- 172.21.15.69:0/3993252698 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8c3940 0x55792a8c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.180+0000 7f7e73873740 1 --2- 172.21.15.69:0/3993252698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8b9270 0x55792a8c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.180+0000 7f7e73873740 1 --2- 172.21.15.69:0/3993252698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8b8d30 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.180+0000 7f7e73873740 1 -- 172.21.15.69:0/3993252698 >> 172.21.15.69:0/3993252698 conn(0x55792a8b4c70 msgr2=0x55792a8b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:00.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.180+0000 7f7e73873740 1 -- 172.21.15.69:0/3993252698 shutdown_connections 2024-04-24T09:10:00.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.180+0000 7f7e73873740 1 -- 172.21.15.69:0/3993252698 wait complete. 2024-04-24T09:10:00.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.180+0000 7f7e73873740 1 -- start start 2024-04-24T09:10:00.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.181+0000 7f7e73873740 1 Processor -- start 2024-04-24T09:10:00.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.181+0000 7f7e73873740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.181+0000 7f7e73873740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8b9270 0x55792a8c06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.181+0000 7f7e73873740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8c3940 0x55792a8c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.181+0000 7f7e73873740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55792a8c67d0 con 0x55792a8b8950 2024-04-24T09:10:00.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.181+0000 7f7e73873740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55792a7d7810 con 0x55792a8c3940 2024-04-24T09:10:00.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.181+0000 7f7e73873740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55792a8c6950 con 0x55792a8b9270 2024-04-24T09:10:00.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.181+0000 7f7e73873740 10 client.0 _ll_register_callbacks cb 0x55792a7a5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:00.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.181+0000 7f7e6a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.181+0000 7f7e6a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8c33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39510/0 (socket says 172.21.15.69:39510) 2024-04-24T09:10:00.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.181+0000 7f7e6a800640 1 -- 172.21.15.69:0/883070730 learned_addr learned my addr 172.21.15.69:0/883070730 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:00.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.182+0000 7f7e69400640 1 --2- 172.21.15.69:0/883070730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8b9270 0x55792a8c06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.182+0000 7f7e6a800640 1 -- 172.21.15.69:0/883070730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8b9270 msgr2=0x55792a8c06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:00.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.182+0000 7f7e6a800640 1 --2- 172.21.15.69:0/883070730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8b9270 0x55792a8c06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.182+0000 7f7e6a800640 1 -- 172.21.15.69:0/883070730 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8c3940 msgr2=0x55792a8c0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:00.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.182+0000 7f7e6a800640 1 --2- 172.21.15.69:0/883070730 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8c3940 0x55792a8c0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.182+0000 7f7e6a800640 1 -- 172.21.15.69:0/883070730 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55792a7e83f0 con 0x55792a8b8950 2024-04-24T09:10:00.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.182+0000 7f7e6a800640 1 --2- 172.21.15.69:0/883070730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8c33b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7e64004a80 tx=0x7f7e6400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34411105668574fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:00.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.182+0000 7f7e4f400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:10:00.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.182+0000 7f7e4f400640 1 -- 172.21.15.69:0/883070730 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64028060 con 0x55792a8b8950 2024-04-24T09:10:00.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.182+0000 7f7e4f400640 1 -- 172.21.15.69:0/883070730 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7e6400f850 con 0x55792a8b8950 2024-04-24T09:10:00.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.182+0000 7f7e73873740 1 -- 172.21.15.69:0/883070730 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e6400fd00 con 0x55792a8b8950 2024-04-24T09:10:00.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.183+0000 7f7e4f400640 1 -- 172.21.15.69:0/883070730 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7e6402f050 con 0x55792a8b8950 2024-04-24T09:10:00.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.183+0000 7f7e73873740 10 client.16542 fetch_fsmap learned FSMap version 29 2024-04-24T09:10:00.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.183+0000 7f7e73873740 1 -- 172.21.15.69:0/883070730 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55792a5303f0 con 0x55792a8b8950 2024-04-24T09:10:00.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.183+0000 7f7e4f400640 1 -- 172.21.15.69:0/883070730 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e6400fd00 con 0x55792a8b8950 2024-04-24T09:10:00.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.183+0000 7f7e4f400640 1 -- 172.21.15.69:0/883070730 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f7e64023420 con 0x55792a8b8950 2024-04-24T09:10:00.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.183+0000 7f7e73873740 10 client.16542 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:10:00.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.183+0000 7f7e73873740 10 client.16542 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:10:00.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.183+0000 7f7e73873740 20 client.16542 populate_metadata read hostname 'smithi069' 2024-04-24T09:10:00.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.183+0000 7f7e73873740 1 --2- 172.21.15.69:0/883070730 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x55792a8d39e0 0x55792a8d5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.183+0000 7f7e73873740 4 client.16542 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:00.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.183+0000 7f7e73873740 1 -- 172.21.15.69:0/883070730 --> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55792a82bbf0 con 0x55792a8d39e0 2024-04-24T09:10:00.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.184+0000 7f7e69400640 1 --2- 172.21.15.69:0/883070730 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x55792a8d39e0 0x55792a8d5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.186+0000 7f7e69400640 1 --2- 172.21.15.69:0/883070730 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x55792a8d39e0 0x55792a8d5dd0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8f7bb80bc920136 server_cookie=963101e47e24df27 in_seq=0 out_seq=0 2024-04-24T09:10:00.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.186+0000 7f7e4f400640 10 client.16542 ms_handle_connect on v2:172.21.15.81:6836/1973819593 2024-04-24T09:10:00.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.187+0000 7f7e4f400640 1 -- 172.21.15.69:0/883070730 <== mds.1 v2:172.21.15.81:6836/1973819593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55792a82bbf0 con 0x55792a8d39e0 2024-04-24T09:10:00.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.187+0000 7f7e4f400640 10 client.16542 handle_command_reply: tid=0 2024-04-24T09:10:00.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.187+0000 7f7e73873740 1 client.16542 shutdown 2024-04-24T09:10:00.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.188+0000 7f7e73873740 1 -- 172.21.15.69:0/883070730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 msgr2=0x55792a8c33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.188+0000 7f7e73873740 1 --2- 172.21.15.69:0/883070730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8c33b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7e64004a80 tx=0x7f7e6400cf20 comp rx=0 tx=0).stop 2024-04-24T09:10:00.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.188+0000 7f7e73873740 1 -- 172.21.15.69:0/883070730 shutdown_connections 2024-04-24T09:10:00.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.188+0000 7f7e73873740 1 --2- 172.21.15.69:0/883070730 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x55792a8d39e0 0x55792a8d5dd0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.188+0000 7f7e73873740 1 --2- 172.21.15.69:0/883070730 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8c3940 0x55792a8c0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.188+0000 7f7e73873740 1 --2- 172.21.15.69:0/883070730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8b9270 0x55792a8c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.188+0000 7f7e73873740 1 --2- 172.21.15.69:0/883070730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8c33b0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.188+0000 7f7e4f400640 0 client.16542 ms_handle_reset on v2:172.21.15.81:6836/1973819593 2024-04-24T09:10:00.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.188+0000 7f7e73873740 1 -- 172.21.15.69:0/883070730 >> 172.21.15.69:0/883070730 conn(0x55792a8b4c70 msgr2=0x55792a8baa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:00.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.188+0000 7f7e73873740 1 -- 172.21.15.69:0/883070730 shutdown_connections 2024-04-24T09:10:00.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.188+0000 7f7e73873740 1 -- 172.21.15.69:0/883070730 wait complete. 2024-04-24T09:10:00.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.188+0000 7f7e73873740 20 client.16542 trim_cache size 0 max 16384 2024-04-24T09:10:00.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.205+0000 7f7e73873740 1 Processor -- start 2024-04-24T09:10:00.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.205+0000 7f7e73873740 1 -- start start 2024-04-24T09:10:00.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.206+0000 7f7e73873740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8b8950 0x55792a8cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.206+0000 7f7e73873740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8b9270 0x55792a8bff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.206+0000 7f7e73873740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8c3940 0x55792a8bab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.206+0000 7f7e73873740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1b3e60 con 0x55792a8c3940 2024-04-24T09:10:00.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.206+0000 7f7e73873740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1b16a0 con 0x55792a8b9270 2024-04-24T09:10:00.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.206+0000 7f7e73873740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1af4f0 con 0x55792a8b8950 2024-04-24T09:10:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.206+0000 7f7e6a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8c3940 0x55792a8bab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.206+0000 7f7e6a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8c3940 0x55792a8bab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39524/0 (socket says 172.21.15.69:39524) 2024-04-24T09:10:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.206+0000 7f7e6a800640 1 -- 172.21.15.69:0/1995057972 learned_addr learned my addr 172.21.15.69:0/1995057972 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.207+0000 7f7e69400640 1 --2- 172.21.15.69:0/1995057972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8b9270 0x55792a8bff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.207+0000 7f7e6a800640 1 -- 172.21.15.69:0/1995057972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8b8950 msgr2=0x55792a8cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.207+0000 7f7e6a800640 1 --2- 172.21.15.69:0/1995057972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8b8950 0x55792a8cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.207+0000 7f7e6a800640 1 -- 172.21.15.69:0/1995057972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8b9270 msgr2=0x55792a8bff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.207+0000 7f7e6a800640 1 --2- 172.21.15.69:0/1995057972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8b9270 0x55792a8bff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.207+0000 7f7e6a800640 1 -- 172.21.15.69:0/1995057972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55792a5fa7d0 con 0x55792a8c3940 2024-04-24T09:10:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.207+0000 7f7e69400640 1 --2- 172.21.15.69:0/1995057972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8b9270 0x55792a8bff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.207+0000 7f7e6a800640 1 --2- 172.21.15.69:0/1995057972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8c3940 0x55792a8bab70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400ede0 tx=0x7f7e6400c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ecde25ce281ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.207+0000 7f7e5a600640 1 -- 172.21.15.69:0/1995057972 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64028070 con 0x55792a8c3940 2024-04-24T09:10:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.207+0000 7f7e5a600640 1 -- 172.21.15.69:0/1995057972 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e6400e6c0 con 0x55792a8c3940 2024-04-24T09:10:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.207+0000 7f7e5a600640 1 -- 172.21.15.69:0/1995057972 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64014040 con 0x55792a8c3940 2024-04-24T09:10:00.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.208+0000 7f7e73873740 1 -- 172.21.15.69:0/1995057972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8c3940 msgr2=0x55792a8bab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.208+0000 7f7e73873740 1 --2- 172.21.15.69:0/1995057972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8c3940 0x55792a8bab70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400ede0 tx=0x7f7e6400c750 comp rx=0 tx=0).stop 2024-04-24T09:10:00.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.208+0000 7f7e73873740 1 -- 172.21.15.69:0/1995057972 shutdown_connections 2024-04-24T09:10:00.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.208+0000 7f7e73873740 1 --2- 172.21.15.69:0/1995057972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8c3940 0x55792a8bab70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.208+0000 7f7e73873740 1 --2- 172.21.15.69:0/1995057972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8b9270 0x55792a8bff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.208+0000 7f7e73873740 1 --2- 172.21.15.69:0/1995057972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8b8950 0x55792a8cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.208+0000 7f7e73873740 1 -- 172.21.15.69:0/1995057972 >> 172.21.15.69:0/1995057972 conn(0x55792a8b4c70 msgr2=0x55792a8cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.208+0000 7f7e73873740 1 -- 172.21.15.69:0/1995057972 shutdown_connections 2024-04-24T09:10:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.208+0000 7f7e73873740 1 -- 172.21.15.69:0/1995057972 wait complete. 2024-04-24T09:10:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.209+0000 7f7e73873740 1 -- start start 2024-04-24T09:10:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.209+0000 7f7e73873740 1 Processor -- start 2024-04-24T09:10:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.209+0000 7f7e73873740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.209+0000 7f7e73873740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8b9270 0x55792a8bd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.209+0000 7f7e73873740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8c3940 0x55792a8bde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.209+0000 7f7e73873740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1afb60 con 0x55792a8b8950 2024-04-24T09:10:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.209+0000 7f7e73873740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1b0fd0 con 0x55792a8b9270 2024-04-24T09:10:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.209+0000 7f7e73873740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1b0230 con 0x55792a8c3940 2024-04-24T09:10:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.209+0000 7f7e73873740 10 client.0 _ll_register_callbacks cb 0x7f7e64007d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.210+0000 7f7e6a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.210+0000 7f7e6a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8cbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39526/0 (socket says 172.21.15.69:39526) 2024-04-24T09:10:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.210+0000 7f7e6a800640 1 -- 172.21.15.69:0/2570962381 learned_addr learned my addr 172.21.15.69:0/2570962381 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.210+0000 7f7e6a800640 1 -- 172.21.15.69:0/2570962381 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8c3940 msgr2=0x55792a8bde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.210+0000 7f7e6a800640 1 --2- 172.21.15.69:0/2570962381 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8c3940 0x55792a8bde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.210+0000 7f7e6a800640 1 -- 172.21.15.69:0/2570962381 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8b9270 msgr2=0x55792a8bd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.210+0000 7f7e6a800640 1 --2- 172.21.15.69:0/2570962381 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8b9270 0x55792a8bd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.210+0000 7f7e6a800640 1 -- 172.21.15.69:0/2570962381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55792a8be660 con 0x55792a8b8950 2024-04-24T09:10:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.210+0000 7f7e6a800640 1 --2- 172.21.15.69:0/2570962381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8cbf90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f7e640238c0 tx=0x7f7e64007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e27192e19684a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.210+0000 7f7e4f400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:10:00.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.211+0000 7f7e4f400640 1 -- 172.21.15.69:0/2570962381 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64042060 con 0x55792a8b8950 2024-04-24T09:10:00.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.211+0000 7f7e73873740 1 -- 172.21.15.69:0/2570962381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e6400e6c0 con 0x55792a8b8950 2024-04-24T09:10:00.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.211+0000 7f7e4f400640 1 -- 172.21.15.69:0/2570962381 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7e64023420 con 0x55792a8b8950 2024-04-24T09:10:00.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.211+0000 7f7e4f400640 1 -- 172.21.15.69:0/2570962381 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7e64014050 con 0x55792a8b8950 2024-04-24T09:10:00.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.211+0000 7f7e73873740 10 client.16548 fetch_fsmap learned FSMap version 29 2024-04-24T09:10:00.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.211+0000 7f7e73873740 1 -- 172.21.15.69:0/2570962381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55792a939c70 con 0x55792a8b8950 2024-04-24T09:10:00.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.211+0000 7f7e4f400640 1 -- 172.21.15.69:0/2570962381 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e6400e6c0 con 0x55792a8b8950 2024-04-24T09:10:00.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.212+0000 7f7e4f400640 1 -- 172.21.15.69:0/2570962381 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f7e64019650 con 0x55792a8b8950 2024-04-24T09:10:00.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.212+0000 7f7e73873740 10 client.16548 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:10:00.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.212+0000 7f7e73873740 10 client.16548 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:10:00.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.212+0000 7f7e73873740 20 client.16548 populate_metadata read hostname 'smithi069' 2024-04-24T09:10:00.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.212+0000 7f7e73873740 1 --2- 172.21.15.69:0/2570962381 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x55792a93a270 0x55792a93c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.212+0000 7f7e73873740 4 client.16548 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:10:00.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.212+0000 7f7e73873740 1 -- 172.21.15.69:0/2570962381 --> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55792a825a60 con 0x55792a93a270 2024-04-24T09:10:00.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.212+0000 7f7e69400640 1 --2- 172.21.15.69:0/2570962381 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x55792a93a270 0x55792a93c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.213+0000 7f7e69400640 1 --2- 172.21.15.69:0/2570962381 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x55792a93a270 0x55792a93c690 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3d1884eb355a012 server_cookie=cb26c3f18f7c992e in_seq=0 out_seq=0 2024-04-24T09:10:00.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.213+0000 7f7e4f400640 10 client.16548 ms_handle_connect on v2:172.21.15.81:6836/1973819593 2024-04-24T09:10:00.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.217+0000 7f7e4f400640 1 -- 172.21.15.69:0/2570962381 <== mds.1 v2:172.21.15.81:6836/1973819593 1 ==== command_reply(tid 0: 0 ) ==== 8+0+52501 (crc 0 0 0) 0x55792a825a60 con 0x55792a93a270 2024-04-24T09:10:00.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.217+0000 7f7e4f400640 10 client.16548 handle_command_reply: tid=0 2024-04-24T09:10:00.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.218+0000 7f7e73873740 1 client.16548 shutdown 2024-04-24T09:10:00.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.218+0000 7f7e73873740 1 -- 172.21.15.69:0/2570962381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 msgr2=0x55792a8cbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.218+0000 7f7e73873740 1 --2- 172.21.15.69:0/2570962381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8cbf90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f7e640238c0 tx=0x7f7e64007ec0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.218+0000 7f7e73873740 1 -- 172.21.15.69:0/2570962381 shutdown_connections 2024-04-24T09:10:00.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.218+0000 7f7e73873740 1 --2- 172.21.15.69:0/2570962381 >> [v2:172.21.15.81:6836/1973819593,v1:172.21.15.81:6837/1973819593] conn(0x55792a93a270 0x55792a93c690 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.218+0000 7f7e73873740 1 --2- 172.21.15.69:0/2570962381 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55792a8c3940 0x55792a8bde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.218+0000 7f7e73873740 1 --2- 172.21.15.69:0/2570962381 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55792a8b9270 0x55792a8bd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.218+0000 7f7e73873740 1 --2- 172.21.15.69:0/2570962381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55792a8b8950 0x55792a8cbf90 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.218+0000 7f7e4f400640 0 client.16548 ms_handle_reset on v2:172.21.15.81:6836/1973819593 2024-04-24T09:10:00.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.218+0000 7f7e73873740 1 -- 172.21.15.69:0/2570962381 >> 172.21.15.69:0/2570962381 conn(0x55792a8b4c70 msgr2=0x55792a8bf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:00.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.218+0000 7f7e73873740 1 -- 172.21.15.69:0/2570962381 shutdown_connections 2024-04-24T09:10:00.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.218+0000 7f7e73873740 1 -- 172.21.15.69:0/2570962381 wait complete. 2024-04-24T09:10:00.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.218+0000 7f7e73873740 20 client.16548 trim_cache size 0 max 16384 2024-04-24T09:10:00.220 INFO:teuthology.orchestra.run.smithi069.stdout:[ 2024-04-24T09:10:00.220 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.220 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.220 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 6833472, 2024-04-24T09:10:00.220 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628885, 2024-04-24T09:10:00.220 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/089" 2024-04-24T09:10:00.221 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.221 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.221 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.221 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 10197908, 2024-04-24T09:10:00.221 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628764, 2024-04-24T09:10:00.221 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/028.out" 2024-04-24T09:10:00.221 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.221 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.221 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.221 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 35192652, 2024-04-24T09:10:00.221 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628870, 2024-04-24T09:10:00.221 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/081.out" 2024-04-24T09:10:00.221 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.221 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.222 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.222 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 42724336, 2024-04-24T09:10:00.222 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628306, 2024-04-24T09:10:00.222 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/commit-msg.sample" 2024-04-24T09:10:00.222 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.222 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.222 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.222 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 70250420, 2024-04-24T09:10:00.222 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628838, 2024-04-24T09:10:00.222 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/065.out" 2024-04-24T09:10:00.222 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.222 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.222 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.222 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 81870884, 2024-04-24T09:10:00.223 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628829, 2024-04-24T09:10:00.223 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/061" 2024-04-24T09:10:00.223 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.223 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.223 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.223 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 107366220, 2024-04-24T09:10:00.223 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628896, 2024-04-24T09:10:00.223 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/094.out" 2024-04-24T09:10:00.223 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.223 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.223 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.223 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 116639293, 2024-04-24T09:10:00.223 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628761, 2024-04-24T09:10:00.223 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/027" 2024-04-24T09:10:00.223 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.223 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 121570661, 2024-04-24T09:10:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628734, 2024-04-24T09:10:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/013.out" 2024-04-24T09:10:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 151447312, 2024-04-24T09:10:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628912, 2024-04-24T09:10:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/102.out" 2024-04-24T09:10:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 169424799, 2024-04-24T09:10:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628906, 2024-04-24T09:10:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/099.out" 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 198807592, 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628758, 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/025.out" 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 209374472, 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628732, 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/012.out" 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 247095520, 2024-04-24T09:10:00.225 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628895, 2024-04-24T09:10:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/094" 2024-04-24T09:10:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 247475753, 2024-04-24T09:10:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628792, 2024-04-24T09:10:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/042.out" 2024-04-24T09:10:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 274729540, 2024-04-24T09:10:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628855, 2024-04-24T09:10:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/074" 2024-04-24T09:10:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.226 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.227 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 276924496, 2024-04-24T09:10:00.227 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628736, 2024-04-24T09:10:00.227 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/014.out" 2024-04-24T09:10:00.227 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.227 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.227 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.227 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 282137058, 2024-04-24T09:10:00.227 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628730, 2024-04-24T09:10:00.227 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/011.out" 2024-04-24T09:10:00.227 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.227 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.227 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.227 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 345159256, 2024-04-24T09:10:00.227 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628751, 2024-04-24T09:10:00.227 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/022" 2024-04-24T09:10:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 347989433, 2024-04-24T09:10:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628908, 2024-04-24T09:10:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/100.out" 2024-04-24T09:10:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 349773390, 2024-04-24T09:10:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628760, 2024-04-24T09:10:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/026.out" 2024-04-24T09:10:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.228 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 352876795, 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628852, 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/072.out" 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 359522890, 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628800, 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/046.out" 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 370415538, 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628913, 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/103" 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.229 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.230 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 389553222, 2024-04-24T09:10:00.230 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628914, 2024-04-24T09:10:00.230 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/103.out" 2024-04-24T09:10:00.230 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.230 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.230 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.230 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 395987537, 2024-04-24T09:10:00.230 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628904, 2024-04-24T09:10:00.230 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/098.out" 2024-04-24T09:10:00.230 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.230 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.230 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.230 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 403911283, 2024-04-24T09:10:00.230 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628917, 2024-04-24T09:10:00.231 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/105" 2024-04-24T09:10:00.231 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.231 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.231 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.231 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 479910113, 2024-04-24T09:10:00.231 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628890, 2024-04-24T09:10:00.231 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/091.out" 2024-04-24T09:10:00.231 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.231 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.231 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.231 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 486813706, 2024-04-24T09:10:00.231 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628713, 2024-04-24T09:10:00.231 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/003" 2024-04-24T09:10:00.231 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 497422423, 2024-04-24T09:10:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628921, 2024-04-24T09:10:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/107" 2024-04-24T09:10:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 536409285, 2024-04-24T09:10:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628298, 2024-04-24T09:10:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-commit.sample" 2024-04-24T09:10:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 571174634, 2024-04-24T09:10:00.232 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628724, 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/008.out" 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 573041619, 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628791, 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/042" 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 573579736, 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628875, 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/084" 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.233 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 595180975, 2024-04-24T09:10:00.234 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628894, 2024-04-24T09:10:00.234 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/093.out" 2024-04-24T09:10:00.234 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.234 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.234 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.234 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 612464666, 2024-04-24T09:10:00.234 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628822, 2024-04-24T09:10:00.234 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/057.out" 2024-04-24T09:10:00.234 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.234 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.234 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.234 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 621203877, 2024-04-24T09:10:00.234 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628762, 2024-04-24T09:10:00.235 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/027.out" 2024-04-24T09:10:00.235 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.235 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.235 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.235 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 661840137, 2024-04-24T09:10:00.235 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628858, 2024-04-24T09:10:00.235 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/075.out" 2024-04-24T09:10:00.235 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.235 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.235 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.235 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 666721323, 2024-04-24T09:10:00.235 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628796, 2024-04-24T09:10:00.235 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/044.out" 2024-04-24T09:10:00.235 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.236 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.236 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.236 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 682902885, 2024-04-24T09:10:00.236 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628808, 2024-04-24T09:10:00.236 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/050.out" 2024-04-24T09:10:00.236 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.236 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.236 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.236 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 728984796, 2024-04-24T09:10:00.236 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628757, 2024-04-24T09:10:00.236 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/025" 2024-04-24T09:10:00.236 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.236 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.236 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.236 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 746514381, 2024-04-24T09:10:00.237 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628817, 2024-04-24T09:10:00.237 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/055" 2024-04-24T09:10:00.237 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.237 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.237 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.237 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 757949829, 2024-04-24T09:10:00.237 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628880, 2024-04-24T09:10:00.237 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/086.out" 2024-04-24T09:10:00.237 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.237 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.237 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.237 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 780740565, 2024-04-24T09:10:00.237 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628821, 2024-04-24T09:10:00.237 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/057" 2024-04-24T09:10:00.237 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.238 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.238 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.238 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 794937842, 2024-04-24T09:10:00.238 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628863, 2024-04-24T09:10:00.238 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/078" 2024-04-24T09:10:00.238 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.238 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.238 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.238 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 799408602, 2024-04-24T09:10:00.238 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628707, 2024-04-24T09:10:00.238 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/Makefile" 2024-04-24T09:10:00.238 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.238 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.238 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.238 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 861951983, 2024-04-24T09:10:00.239 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628765, 2024-04-24T09:10:00.239 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/029" 2024-04-24T09:10:00.239 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.239 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.239 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.239 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 866919816, 2024-04-24T09:10:00.239 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628819, 2024-04-24T09:10:00.239 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/056" 2024-04-24T09:10:00.239 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.239 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.239 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.239 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 873458021, 2024-04-24T09:10:00.239 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628869, 2024-04-24T09:10:00.239 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/081" 2024-04-24T09:10:00.239 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.240 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.240 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.240 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 900880342, 2024-04-24T09:10:00.240 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628742, 2024-04-24T09:10:00.240 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/017.out" 2024-04-24T09:10:00.240 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.240 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.240 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.240 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 914338860, 2024-04-24T09:10:00.240 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628738, 2024-04-24T09:10:00.240 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/015.out" 2024-04-24T09:10:00.240 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.241 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.241 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.241 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 919970241, 2024-04-24T09:10:00.241 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628777, 2024-04-24T09:10:00.241 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/035" 2024-04-24T09:10:00.241 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.241 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.241 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.241 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 924674784, 2024-04-24T09:10:00.241 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628790, 2024-04-24T09:10:00.241 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/041.out" 2024-04-24T09:10:00.241 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.241 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.242 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.242 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 938267888, 2024-04-24T09:10:00.242 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628728, 2024-04-24T09:10:00.242 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/010.out" 2024-04-24T09:10:00.242 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.242 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.242 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.242 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 993090496, 2024-04-24T09:10:00.242 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628741, 2024-04-24T09:10:00.242 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/017" 2024-04-24T09:10:00.242 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.242 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.242 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.242 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1005868011, 2024-04-24T09:10:00.242 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628909, 2024-04-24T09:10:00.243 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/101" 2024-04-24T09:10:00.243 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.243 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.243 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.243 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1010882015, 2024-04-24T09:10:00.243 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628876, 2024-04-24T09:10:00.243 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/084.out" 2024-04-24T09:10:00.243 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.243 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.243 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.243 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1017009133, 2024-04-24T09:10:00.243 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628731, 2024-04-24T09:10:00.243 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/012" 2024-04-24T09:10:00.243 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.243 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.244 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.244 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1050878852, 2024-04-24T09:10:00.244 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628857, 2024-04-24T09:10:00.244 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/075" 2024-04-24T09:10:00.244 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.244 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.244 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.244 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1095470494, 2024-04-24T09:10:00.244 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628831, 2024-04-24T09:10:00.244 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/062" 2024-04-24T09:10:00.244 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.244 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.244 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.244 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1112913858, 2024-04-24T09:10:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628892, 2024-04-24T09:10:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/092.out" 2024-04-24T09:10:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1145070476, 2024-04-24T09:10:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628891, 2024-04-24T09:10:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/092" 2024-04-24T09:10:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1154134369, 2024-04-24T09:10:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628809, 2024-04-24T09:10:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/051" 2024-04-24T09:10:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.245 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.246 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.246 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1167676811, 2024-04-24T09:10:00.246 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628725, 2024-04-24T09:10:00.246 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/009" 2024-04-24T09:10:00.246 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.246 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.246 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.246 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1182048718, 2024-04-24T09:10:00.246 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628815, 2024-04-24T09:10:00.246 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/054" 2024-04-24T09:10:00.246 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.246 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.246 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.246 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1197026225, 2024-04-24T09:10:00.247 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628866, 2024-04-24T09:10:00.247 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/079.out" 2024-04-24T09:10:00.247 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.247 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.247 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.247 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1211208845, 2024-04-24T09:10:00.247 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628871, 2024-04-24T09:10:00.247 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/082" 2024-04-24T09:10:00.247 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.247 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.247 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.247 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1228888468, 2024-04-24T09:10:00.247 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628926, 2024-04-24T09:10:00.247 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/109.out" 2024-04-24T09:10:00.247 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.247 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1246738782, 2024-04-24T09:10:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628774, 2024-04-24T09:10:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/033.out" 2024-04-24T09:10:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1253792056, 2024-04-24T09:10:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628832, 2024-04-24T09:10:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/062.out" 2024-04-24T09:10:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1277993620, 2024-04-24T09:10:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628776, 2024-04-24T09:10:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/034.out" 2024-04-24T09:10:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1313621005, 2024-04-24T09:10:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628854, 2024-04-24T09:10:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/073.out" 2024-04-24T09:10:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1336522891, 2024-04-24T09:10:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628719, 2024-04-24T09:10:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/006" 2024-04-24T09:10:00.250 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.250 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.250 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.250 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1362312878, 2024-04-24T09:10:00.250 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628735, 2024-04-24T09:10:00.250 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/014" 2024-04-24T09:10:00.250 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.250 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.250 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.250 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1395216955, 2024-04-24T09:10:00.250 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628850, 2024-04-24T09:10:00.250 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/071.out" 2024-04-24T09:10:00.250 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1415592038, 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628887, 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/090" 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1423465270, 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628849, 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/071" 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1490179130, 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628877, 2024-04-24T09:10:00.251 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/085" 2024-04-24T09:10:00.252 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.252 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.252 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.252 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1490521142, 2024-04-24T09:10:00.252 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628711, 2024-04-24T09:10:00.252 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/002" 2024-04-24T09:10:00.252 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.252 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.252 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.252 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1492149154, 2024-04-24T09:10:00.252 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628824, 2024-04-24T09:10:00.252 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/058.out" 2024-04-24T09:10:00.252 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.252 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.252 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1496924110, 2024-04-24T09:10:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628868, 2024-04-24T09:10:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/080.out" 2024-04-24T09:10:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1510783330, 2024-04-24T09:10:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628826, 2024-04-24T09:10:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/059.out" 2024-04-24T09:10:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1511554780, 2024-04-24T09:10:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628888, 2024-04-24T09:10:00.253 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/090.out" 2024-04-24T09:10:00.254 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.254 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.254 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.254 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1531955352, 2024-04-24T09:10:00.254 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628844, 2024-04-24T09:10:00.254 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/068.out" 2024-04-24T09:10:00.254 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.254 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.254 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.254 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1535816944, 2024-04-24T09:10:00.254 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628717, 2024-04-24T09:10:00.254 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/005" 2024-04-24T09:10:00.254 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.254 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.254 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.254 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1565382742, 2024-04-24T09:10:00.255 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628746, 2024-04-24T09:10:00.255 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/019.out" 2024-04-24T09:10:00.255 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.255 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.255 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.255 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1572794069, 2024-04-24T09:10:00.255 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628749, 2024-04-24T09:10:00.255 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/021" 2024-04-24T09:10:00.255 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.255 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.255 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.255 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1596879710, 2024-04-24T09:10:00.255 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628773, 2024-04-24T09:10:00.255 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/033" 2024-04-24T09:10:00.255 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.255 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1619813245, 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628755, 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/024" 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1620929781, 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628710, 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/001.out" 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1623436965, 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628834, 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/063.out" 2024-04-24T09:10:00.256 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.257 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.257 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.257 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1625175505, 2024-04-24T09:10:00.257 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628883, 2024-04-24T09:10:00.257 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/088" 2024-04-24T09:10:00.257 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.257 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.257 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.257 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1650899002, 2024-04-24T09:10:00.257 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628897, 2024-04-24T09:10:00.257 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/095" 2024-04-24T09:10:00.257 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.257 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.257 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.257 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1690181762, 2024-04-24T09:10:00.257 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628840, 2024-04-24T09:10:00.258 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/066.out" 2024-04-24T09:10:00.258 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.258 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.258 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.258 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1690875043, 2024-04-24T09:10:00.258 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628882, 2024-04-24T09:10:00.258 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/087.out" 2024-04-24T09:10:00.258 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.258 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.258 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.258 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1696970307, 2024-04-24T09:10:00.258 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628743, 2024-04-24T09:10:00.258 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/018" 2024-04-24T09:10:00.258 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.258 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.259 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.259 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1716110339, 2024-04-24T09:10:00.259 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628737, 2024-04-24T09:10:00.259 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/015" 2024-04-24T09:10:00.259 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.259 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.259 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.259 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1728333106, 2024-04-24T09:10:00.259 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628874, 2024-04-24T09:10:00.259 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/083.out" 2024-04-24T09:10:00.259 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.259 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.259 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.259 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1753879886, 2024-04-24T09:10:00.260 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628910, 2024-04-24T09:10:00.260 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/101.out" 2024-04-24T09:10:00.260 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.260 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.260 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.260 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1756725107, 2024-04-24T09:10:00.260 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628924, 2024-04-24T09:10:00.260 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/108.out" 2024-04-24T09:10:00.260 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.260 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.260 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.260 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1824056666, 2024-04-24T09:10:00.260 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628766, 2024-04-24T09:10:00.260 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/029.out" 2024-04-24T09:10:00.261 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.261 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.261 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.261 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1844575851, 2024-04-24T09:10:00.261 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628795, 2024-04-24T09:10:00.261 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/044" 2024-04-24T09:10:00.261 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.261 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.261 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.261 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1885275833, 2024-04-24T09:10:00.261 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628922, 2024-04-24T09:10:00.261 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/107.out" 2024-04-24T09:10:00.261 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.261 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.261 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.262 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1888039795, 2024-04-24T09:10:00.262 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628307, 2024-04-24T09:10:00.262 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/applypatch-msg.sample" 2024-04-24T09:10:00.262 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.262 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.262 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.262 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1891456333, 2024-04-24T09:10:00.262 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628799, 2024-04-24T09:10:00.262 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/046" 2024-04-24T09:10:00.262 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.262 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.263 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.263 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1911002322, 2024-04-24T09:10:00.263 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628861, 2024-04-24T09:10:00.263 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/077" 2024-04-24T09:10:00.263 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.263 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.263 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.263 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1922983864, 2024-04-24T09:10:00.263 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628853, 2024-04-24T09:10:00.263 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/073" 2024-04-24T09:10:00.263 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.263 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.263 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.263 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1940093103, 2024-04-24T09:10:00.263 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628928, 2024-04-24T09:10:00.264 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/110.out" 2024-04-24T09:10:00.264 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.264 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.264 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.264 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1952694501, 2024-04-24T09:10:00.264 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628823, 2024-04-24T09:10:00.264 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/058" 2024-04-24T09:10:00.264 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.264 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.264 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.264 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 1994408889, 2024-04-24T09:10:00.264 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628748, 2024-04-24T09:10:00.264 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/020.out" 2024-04-24T09:10:00.265 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.265 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.265 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.265 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2020549417, 2024-04-24T09:10:00.265 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628308, 2024-04-24T09:10:00.265 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-rebase.sample" 2024-04-24T09:10:00.265 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.265 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.265 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.265 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2023486775, 2024-04-24T09:10:00.265 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628901, 2024-04-24T09:10:00.265 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/097" 2024-04-24T09:10:00.265 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.265 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.266 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.266 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2036875585, 2024-04-24T09:10:00.266 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628789, 2024-04-24T09:10:00.266 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/041" 2024-04-24T09:10:00.266 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.266 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.266 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.266 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2093201799, 2024-04-24T09:10:00.266 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628811, 2024-04-24T09:10:00.266 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/052" 2024-04-24T09:10:00.266 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.266 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.267 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.267 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2117163858, 2024-04-24T09:10:00.267 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628720, 2024-04-24T09:10:00.267 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/006.out" 2024-04-24T09:10:00.267 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.267 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.267 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.267 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2125025357, 2024-04-24T09:10:00.267 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628907, 2024-04-24T09:10:00.267 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/100" 2024-04-24T09:10:00.267 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.267 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.267 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.268 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2157719301, 2024-04-24T09:10:00.268 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628780, 2024-04-24T09:10:00.268 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/036.out" 2024-04-24T09:10:00.268 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.268 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.268 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.268 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2164141607, 2024-04-24T09:10:00.268 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628881, 2024-04-24T09:10:00.268 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/087" 2024-04-24T09:10:00.268 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.268 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.268 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.268 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2187450071, 2024-04-24T09:10:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628797, 2024-04-24T09:10:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/045" 2024-04-24T09:10:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2196276658, 2024-04-24T09:10:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628813, 2024-04-24T09:10:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/053" 2024-04-24T09:10:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2271795098, 2024-04-24T09:10:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628843, 2024-04-24T09:10:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/068" 2024-04-24T09:10:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.269 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.270 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.270 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2286410204, 2024-04-24T09:10:00.270 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628873, 2024-04-24T09:10:00.270 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/083" 2024-04-24T09:10:00.270 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.270 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.270 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.270 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2296489530, 2024-04-24T09:10:00.270 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628859, 2024-04-24T09:10:00.270 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/076" 2024-04-24T09:10:00.270 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.270 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.270 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.271 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2299736265, 2024-04-24T09:10:00.271 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628898, 2024-04-24T09:10:00.271 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/095.out" 2024-04-24T09:10:00.271 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.271 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.271 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.271 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2302233637, 2024-04-24T09:10:00.271 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628714, 2024-04-24T09:10:00.271 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/003.out" 2024-04-24T09:10:00.271 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.271 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.271 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.271 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2392938233, 2024-04-24T09:10:00.271 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628763, 2024-04-24T09:10:00.272 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/028" 2024-04-24T09:10:00.272 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.272 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.272 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.272 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2396655253, 2024-04-24T09:10:00.272 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628900, 2024-04-24T09:10:00.272 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/096.out" 2024-04-24T09:10:00.272 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.272 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.272 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.272 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2399602165, 2024-04-24T09:10:00.272 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628830, 2024-04-24T09:10:00.272 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/061.out" 2024-04-24T09:10:00.272 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.272 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.272 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.273 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2405514240, 2024-04-24T09:10:00.273 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628884, 2024-04-24T09:10:00.273 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/088.out" 2024-04-24T09:10:00.273 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.273 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.273 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.273 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2415499212, 2024-04-24T09:10:00.273 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628775, 2024-04-24T09:10:00.273 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/034" 2024-04-24T09:10:00.273 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.273 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.273 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.273 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2427308658, 2024-04-24T09:10:00.273 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628745, 2024-04-24T09:10:00.273 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/019" 2024-04-24T09:10:00.274 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.274 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.274 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.274 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2450332114, 2024-04-24T09:10:00.274 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628772, 2024-04-24T09:10:00.274 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/032.out" 2024-04-24T09:10:00.274 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.274 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.274 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.274 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2458238310, 2024-04-24T09:10:00.274 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628810, 2024-04-24T09:10:00.274 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/051.out" 2024-04-24T09:10:00.274 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.274 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.274 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.274 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2502427690, 2024-04-24T09:10:00.275 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628818, 2024-04-24T09:10:00.275 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/055.out" 2024-04-24T09:10:00.275 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.275 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.275 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.275 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2511052427, 2024-04-24T09:10:00.275 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628740, 2024-04-24T09:10:00.275 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/016.out" 2024-04-24T09:10:00.275 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.275 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.275 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.275 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2519597140, 2024-04-24T09:10:00.275 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628837, 2024-04-24T09:10:00.275 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/065" 2024-04-24T09:10:00.275 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.275 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2522513210, 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628768, 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/030.out" 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2584442558, 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628733, 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/013" 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2589735451, 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628841, 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/067" 2024-04-24T09:10:00.276 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.277 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.277 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.277 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2620759750, 2024-04-24T09:10:00.277 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628750, 2024-04-24T09:10:00.277 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/021.out" 2024-04-24T09:10:00.277 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.277 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.277 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.277 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2626197843, 2024-04-24T09:10:00.277 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628803, 2024-04-24T09:10:00.277 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/048" 2024-04-24T09:10:00.277 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.277 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.277 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.278 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2630782902, 2024-04-24T09:10:00.278 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628919, 2024-04-24T09:10:00.278 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/106" 2024-04-24T09:10:00.278 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.278 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.278 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.278 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2642592601, 2024-04-24T09:10:00.278 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628726, 2024-04-24T09:10:00.278 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/009.out" 2024-04-24T09:10:00.278 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.278 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.278 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.278 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2675663093, 2024-04-24T09:10:00.278 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628836, 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/064.out" 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2742370233, 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628716, 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/004.out" 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2743800880, 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628300, 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/update.sample" 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2752165759, 2024-04-24T09:10:00.280 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628302, 2024-04-24T09:10:00.280 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-receive.sample" 2024-04-24T09:10:00.280 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.280 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.280 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.280 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2758844820, 2024-04-24T09:10:00.280 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628842, 2024-04-24T09:10:00.280 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/067.out" 2024-04-24T09:10:00.280 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.280 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.280 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.280 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2776578948, 2024-04-24T09:10:00.280 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628723, 2024-04-24T09:10:00.280 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/008" 2024-04-24T09:10:00.281 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.281 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.281 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.281 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2796836251, 2024-04-24T09:10:00.281 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628839, 2024-04-24T09:10:00.281 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/066" 2024-04-24T09:10:00.281 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.281 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.281 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.281 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2803015036, 2024-04-24T09:10:00.281 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628918, 2024-04-24T09:10:00.281 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/105.out" 2024-04-24T09:10:00.281 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.281 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.281 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.282 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2822306540, 2024-04-24T09:10:00.282 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628851, 2024-04-24T09:10:00.282 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/072" 2024-04-24T09:10:00.282 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.282 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.282 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.282 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2830481511, 2024-04-24T09:10:00.282 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628754, 2024-04-24T09:10:00.282 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/023.out" 2024-04-24T09:10:00.282 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.282 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.282 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.282 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2833417192, 2024-04-24T09:10:00.282 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628784, 2024-04-24T09:10:00.282 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/038.out" 2024-04-24T09:10:00.283 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.283 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.283 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.283 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2857965714, 2024-04-24T09:10:00.283 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628902, 2024-04-24T09:10:00.283 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/097.out" 2024-04-24T09:10:00.283 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.283 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.283 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.283 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2886936006, 2024-04-24T09:10:00.283 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628709, 2024-04-24T09:10:00.283 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/001" 2024-04-24T09:10:00.283 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.283 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.284 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.284 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2902967193, 2024-04-24T09:10:00.284 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628860, 2024-04-24T09:10:00.284 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/076.out" 2024-04-24T09:10:00.284 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.284 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.284 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.284 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2915961100, 2024-04-24T09:10:00.284 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628295, 2024-04-24T09:10:00.284 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/push-to-checkout.sample" 2024-04-24T09:10:00.284 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.284 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.284 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.284 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 2946522145, 2024-04-24T09:10:00.284 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628798, 2024-04-24T09:10:00.285 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/045.out" 2024-04-24T09:10:00.285 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.285 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.285 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.285 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3011730571, 2024-04-24T09:10:00.285 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628781, 2024-04-24T09:10:00.285 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/037" 2024-04-24T09:10:00.285 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.285 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.285 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.285 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3054119012, 2024-04-24T09:10:00.285 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628721, 2024-04-24T09:10:00.285 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/007" 2024-04-24T09:10:00.285 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.285 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.286 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.286 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3062431121, 2024-04-24T09:10:00.286 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628847, 2024-04-24T09:10:00.286 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/070" 2024-04-24T09:10:00.286 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.286 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.286 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.286 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3079711695, 2024-04-24T09:10:00.286 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628812, 2024-04-24T09:10:00.286 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/052.out" 2024-04-24T09:10:00.286 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.286 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.286 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.287 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3080605787, 2024-04-24T09:10:00.287 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628929, 2024-04-24T09:10:00.287 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/111" 2024-04-24T09:10:00.287 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.287 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.287 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.287 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3104299991, 2024-04-24T09:10:00.287 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628786, 2024-04-24T09:10:00.287 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/039.out" 2024-04-24T09:10:00.287 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.287 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.287 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.287 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3119390004, 2024-04-24T09:10:00.287 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628925, 2024-04-24T09:10:00.287 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/109" 2024-04-24T09:10:00.288 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.288 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.288 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.288 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3131771581, 2024-04-24T09:10:00.288 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628727, 2024-04-24T09:10:00.288 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/010" 2024-04-24T09:10:00.288 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.288 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.288 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.288 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3137985759, 2024-04-24T09:10:00.288 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628708, 2024-04-24T09:10:00.288 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs" 2024-04-24T09:10:00.288 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.288 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.288 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.289 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3162984427, 2024-04-24T09:10:00.289 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628915, 2024-04-24T09:10:00.289 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/104" 2024-04-24T09:10:00.289 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.289 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.289 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.289 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3170658486, 2024-04-24T09:10:00.289 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628920, 2024-04-24T09:10:00.289 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/106.out" 2024-04-24T09:10:00.289 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.289 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.289 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.289 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3223322509, 2024-04-24T09:10:00.289 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628301, 2024-04-24T09:10:00.289 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/prepare-commit-msg.sample" 2024-04-24T09:10:00.290 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.290 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.290 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.290 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3266013302, 2024-04-24T09:10:00.290 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628916, 2024-04-24T09:10:00.290 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/104.out" 2024-04-24T09:10:00.290 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.290 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.290 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.290 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3364301494, 2024-04-24T09:10:00.290 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628769, 2024-04-24T09:10:00.290 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/031" 2024-04-24T09:10:00.290 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.290 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3372466193, 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628886, 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/089.out" 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3378114700, 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628806, 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/049.out" 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3381337763, 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628889, 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/091" 2024-04-24T09:10:00.291 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.292 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.292 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.292 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3391124642, 2024-04-24T09:10:00.292 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628827, 2024-04-24T09:10:00.292 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/060" 2024-04-24T09:10:00.292 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.292 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.292 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.292 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3391992397, 2024-04-24T09:10:00.292 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628782, 2024-04-24T09:10:00.292 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/037.out" 2024-04-24T09:10:00.292 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.292 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.292 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.292 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3397387197, 2024-04-24T09:10:00.292 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628899, 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/096" 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3397913695, 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628785, 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/039" 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3404999741, 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628805, 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/049" 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.293 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3427509089, 2024-04-24T09:10:00.294 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628820, 2024-04-24T09:10:00.294 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/056.out" 2024-04-24T09:10:00.294 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.294 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.294 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.294 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3436280163, 2024-04-24T09:10:00.294 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628848, 2024-04-24T09:10:00.294 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/070.out" 2024-04-24T09:10:00.294 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.294 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.294 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.294 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3449705908, 2024-04-24T09:10:00.294 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628744, 2024-04-24T09:10:00.294 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/018.out" 2024-04-24T09:10:00.294 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.294 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3463935919, 2024-04-24T09:10:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628715, 2024-04-24T09:10:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/004" 2024-04-24T09:10:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3473012535, 2024-04-24T09:10:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628767, 2024-04-24T09:10:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/030" 2024-04-24T09:10:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3484611008, 2024-04-24T09:10:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628864, 2024-04-24T09:10:00.295 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/078.out" 2024-04-24T09:10:00.296 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.296 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.296 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.296 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3490396813, 2024-04-24T09:10:00.296 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628893, 2024-04-24T09:10:00.296 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/093" 2024-04-24T09:10:00.296 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.296 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.296 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.296 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3506123352, 2024-04-24T09:10:00.296 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628878, 2024-04-24T09:10:00.296 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/085.out" 2024-04-24T09:10:00.296 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.296 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.297 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.297 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3557807487, 2024-04-24T09:10:00.297 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628879, 2024-04-24T09:10:00.297 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/086" 2024-04-24T09:10:00.297 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.297 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.297 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.297 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3576863423, 2024-04-24T09:10:00.297 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628856, 2024-04-24T09:10:00.297 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/074.out" 2024-04-24T09:10:00.297 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.297 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.297 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.297 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3581365896, 2024-04-24T09:10:00.298 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628903, 2024-04-24T09:10:00.298 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/098" 2024-04-24T09:10:00.298 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.298 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.298 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.298 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3583268812, 2024-04-24T09:10:00.298 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628305, 2024-04-24T09:10:00.298 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-applypatch.sample" 2024-04-24T09:10:00.298 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.298 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.298 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.298 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3599387029, 2024-04-24T09:10:00.299 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628862, 2024-04-24T09:10:00.299 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/077.out" 2024-04-24T09:10:00.299 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.299 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.299 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.299 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3607091164, 2024-04-24T09:10:00.299 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628794, 2024-04-24T09:10:00.299 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/043.out" 2024-04-24T09:10:00.299 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.299 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.299 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.299 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3609951921, 2024-04-24T09:10:00.299 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628771, 2024-04-24T09:10:00.299 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/032" 2024-04-24T09:10:00.300 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.300 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.300 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.300 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3624683904, 2024-04-24T09:10:00.300 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628722, 2024-04-24T09:10:00.300 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/007.out" 2024-04-24T09:10:00.300 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.300 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.300 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.300 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3633788883, 2024-04-24T09:10:00.300 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628296, 2024-04-24T09:10:00.300 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/post-update.sample" 2024-04-24T09:10:00.300 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.300 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.301 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.301 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3645337550, 2024-04-24T09:10:00.301 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628783, 2024-04-24T09:10:00.301 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/038" 2024-04-24T09:10:00.301 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.301 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.301 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.301 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3649249384, 2024-04-24T09:10:00.301 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628807, 2024-04-24T09:10:00.301 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/050" 2024-04-24T09:10:00.301 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.301 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.301 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.301 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3650457923, 2024-04-24T09:10:00.301 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628802, 2024-04-24T09:10:00.301 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/047.out" 2024-04-24T09:10:00.302 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.302 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.302 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.302 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3677354238, 2024-04-24T09:10:00.302 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628752, 2024-04-24T09:10:00.302 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/022.out" 2024-04-24T09:10:00.302 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.302 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.302 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.302 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3690671135, 2024-04-24T09:10:00.302 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628778, 2024-04-24T09:10:00.302 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/035.out" 2024-04-24T09:10:00.302 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.302 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3724831781, 2024-04-24T09:10:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628845, 2024-04-24T09:10:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/069" 2024-04-24T09:10:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3735687669, 2024-04-24T09:10:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628756, 2024-04-24T09:10:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/024.out" 2024-04-24T09:10:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3747655472, 2024-04-24T09:10:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628788, 2024-04-24T09:10:00.303 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/040.out" 2024-04-24T09:10:00.304 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.304 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.304 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.304 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3772771023, 2024-04-24T09:10:00.304 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628828, 2024-04-24T09:10:00.304 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/060.out" 2024-04-24T09:10:00.304 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.304 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.304 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.304 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3774524065, 2024-04-24T09:10:00.304 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628905, 2024-04-24T09:10:00.304 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/099" 2024-04-24T09:10:00.304 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.304 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.304 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.304 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3774750281, 2024-04-24T09:10:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628814, 2024-04-24T09:10:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/053.out" 2024-04-24T09:10:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3779377616, 2024-04-24T09:10:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628846, 2024-04-24T09:10:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/069.out" 2024-04-24T09:10:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3806606454, 2024-04-24T09:10:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628739, 2024-04-24T09:10:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/016" 2024-04-24T09:10:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.305 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.306 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.306 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3841554550, 2024-04-24T09:10:00.306 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628299, 2024-04-24T09:10:00.306 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-merge-commit.sample" 2024-04-24T09:10:00.306 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.306 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.306 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.306 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3868459657, 2024-04-24T09:10:00.306 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628303, 2024-04-24T09:10:00.306 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/fsmonitor-watchman.sample" 2024-04-24T09:10:00.306 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.306 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.306 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.306 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3876839122, 2024-04-24T09:10:00.306 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628304, 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-push.sample" 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3877076098, 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628787, 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/040" 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3891383064, 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628825, 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/059" 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.307 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3924628958, 2024-04-24T09:10:00.308 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628753, 2024-04-24T09:10:00.308 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/023" 2024-04-24T09:10:00.308 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.308 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.308 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.308 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3943199614, 2024-04-24T09:10:00.308 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628927, 2024-04-24T09:10:00.308 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/110" 2024-04-24T09:10:00.308 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.308 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.308 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.308 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 3977110691, 2024-04-24T09:10:00.308 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628816, 2024-04-24T09:10:00.308 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/054.out" 2024-04-24T09:10:00.308 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.308 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4018165263, 2024-04-24T09:10:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628835, 2024-04-24T09:10:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/064" 2024-04-24T09:10:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4018192250, 2024-04-24T09:10:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628747, 2024-04-24T09:10:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/020" 2024-04-24T09:10:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4019317193, 2024-04-24T09:10:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628718, 2024-04-24T09:10:00.309 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/005.out" 2024-04-24T09:10:00.310 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.310 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.310 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.310 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4023959141, 2024-04-24T09:10:00.310 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628911, 2024-04-24T09:10:00.310 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/102" 2024-04-24T09:10:00.310 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.310 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.310 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.310 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4042103119, 2024-04-24T09:10:00.310 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628801, 2024-04-24T09:10:00.310 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/047" 2024-04-24T09:10:00.310 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.310 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.311 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.311 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4055306945, 2024-04-24T09:10:00.311 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628770, 2024-04-24T09:10:00.311 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/031.out" 2024-04-24T09:10:00.311 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.311 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.311 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.311 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4084457665, 2024-04-24T09:10:00.311 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628297, 2024-04-24T09:10:00.311 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/sendemail-validate.sample" 2024-04-24T09:10:00.311 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.311 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.311 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.311 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4097662745, 2024-04-24T09:10:00.312 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628793, 2024-04-24T09:10:00.312 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/043" 2024-04-24T09:10:00.312 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.312 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.312 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.312 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4105329561, 2024-04-24T09:10:00.312 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628759, 2024-04-24T09:10:00.312 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/026" 2024-04-24T09:10:00.312 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.312 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.312 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.312 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4131234649, 2024-04-24T09:10:00.312 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628872, 2024-04-24T09:10:00.312 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/082.out" 2024-04-24T09:10:00.312 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.312 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.313 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.313 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4144745529, 2024-04-24T09:10:00.313 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628923, 2024-04-24T09:10:00.313 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/108" 2024-04-24T09:10:00.313 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.313 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.313 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.313 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4147292013, 2024-04-24T09:10:00.313 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628804, 2024-04-24T09:10:00.313 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/048.out" 2024-04-24T09:10:00.313 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.313 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.313 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.313 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4199183521, 2024-04-24T09:10:00.313 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628865, 2024-04-24T09:10:00.313 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/079" 2024-04-24T09:10:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4228933487, 2024-04-24T09:10:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628833, 2024-04-24T09:10:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/063" 2024-04-24T09:10:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4257559138, 2024-04-24T09:10:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628779, 2024-04-24T09:10:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/036" 2024-04-24T09:10:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.314 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4284210898, 2024-04-24T09:10:00.315 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628867, 2024-04-24T09:10:00.315 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/080" 2024-04-24T09:10:00.315 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.315 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.315 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.315 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4287812130, 2024-04-24T09:10:00.315 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628712, 2024-04-24T09:10:00.315 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/002.out" 2024-04-24T09:10:00.315 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:10:00.315 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:10:00.315 INFO:teuthology.orchestra.run.smithi069.stdout: "damage_type": "backtrace", 2024-04-24T09:10:00.315 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4289474830, 2024-04-24T09:10:00.315 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1099511628729, 2024-04-24T09:10:00.316 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/011" 2024-04-24T09:10:00.316 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:10:00.316 INFO:teuthology.orchestra.run.smithi069.stdout:] 2024-04-24T09:10:00.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.219+0000 7f7e70a00640 1 -- 172.21.15.69:0/581927444 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7e500a3d20 msgr2=0x7f7e500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.219+0000 7f7e70a00640 1 --2- 172.21.15.69:0/581927444 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7e500a3d20 0x7f7e500a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55792a8b9f10 tx=0x7f7e5c000bc0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.219+0000 7f7e70a00640 1 -- 172.21.15.69:0/581927444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e6c06a870 msgr2=0x7f7e6c1a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.219+0000 7f7e70a00640 1 --2- 172.21.15.69:0/581927444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e6c06a870 0x7f7e6c1a06a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c005ab0 tx=0x7f7e5c003d50 comp rx=0 tx=0).stop 2024-04-24T09:10:00.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.219+0000 7f7e70a00640 1 -- 172.21.15.69:0/581927444 shutdown_connections 2024-04-24T09:10:00.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.219+0000 7f7e70a00640 1 --2- 172.21.15.69:0/581927444 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7e500a3d20 0x7f7e500a61e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.219+0000 7f7e70a00640 1 --2- 172.21.15.69:0/581927444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e6c19a810 0x7f7e6c19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.219+0000 7f7e70a00640 1 --2- 172.21.15.69:0/581927444 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e6c107330 0x7f7e6c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.219+0000 7f7e70a00640 1 --2- 172.21.15.69:0/581927444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e6c06a870 0x7f7e6c1a06a0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.219+0000 7f7e70a00640 1 -- 172.21.15.69:0/581927444 >> 172.21.15.69:0/581927444 conn(0x7f7e6c076d00 msgr2=0x7f7e6c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:00.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.219+0000 7f7e70a00640 1 -- 172.21.15.69:0/581927444 shutdown_connections 2024-04-24T09:10:00.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.219+0000 7f7e70a00640 1 -- 172.21.15.69:0/581927444 wait complete. 2024-04-24T09:10:00.318 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:00 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a[28166]: 2024-04-24T09:09:59.998+0000 7fe78aa00700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-04-24T09:10:00.318 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:00 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a[28166]: 2024-04-24T09:09:59.999+0000 7fe78aa00700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-04-24T09:10:00.318 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:00 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a[28166]: 2024-04-24T09:09:59.999+0000 7fe78aa00700 -1 log_channel(cluster) log [ERR] : mds.e(mds.2): Metadata damage detected 2024-04-24T09:10:00.318 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:00 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a[28166]: 2024-04-24T09:09:59.999+0000 7fe78aa00700 -1 log_channel(cluster) log [ERR] : mds.h(mds.1): Metadata damage detected 2024-04-24T09:10:00.319 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:10:00.507 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:00 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2928867908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:10:00.508 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:00 smithi069 ceph-mon[28170]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-04-24T09:10:00.508 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:00 smithi069 ceph-mon[28170]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-04-24T09:10:00.508 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:00 smithi069 ceph-mon[28170]: mds.e(mds.2): Metadata damage detected 2024-04-24T09:10:00.508 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:00 smithi069 ceph-mon[28170]: mds.h(mds.1): Metadata damage detected 2024-04-24T09:10:00.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.524+0000 7f55a5800640 1 -- 172.21.15.69:0/1512470207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a010c5c0 msgr2=0x7f55a010ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.524+0000 7f55a5800640 1 --2- 172.21.15.69:0/1512470207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a010c5c0 0x7f55a010ea60 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5598009ca0 tx=0x7f559802a9e0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.524+0000 7f55a5800640 1 -- 172.21.15.69:0/1512470207 shutdown_connections 2024-04-24T09:10:00.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.524+0000 7f55a5800640 1 --2- 172.21.15.69:0/1512470207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a010c5c0 0x7f55a010ea60 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.524+0000 7f55a5800640 1 --2- 172.21.15.69:0/1512470207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a0103ba0 0x7f55a010c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.524+0000 7f55a5800640 1 --2- 172.21.15.69:0/1512470207 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55a0103280 0x7f55a0103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.524+0000 7f55a5800640 1 -- 172.21.15.69:0/1512470207 >> 172.21.15.69:0/1512470207 conn(0x7f55a0078000 msgr2=0x7f55a0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:00.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.524+0000 7f55a5800640 1 -- 172.21.15.69:0/1512470207 shutdown_connections 2024-04-24T09:10:00.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.524+0000 7f55a5800640 1 -- 172.21.15.69:0/1512470207 wait complete. 2024-04-24T09:10:00.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.525+0000 7f55a5800640 1 Processor -- start 2024-04-24T09:10:00.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.525+0000 7f55a5800640 1 -- start start 2024-04-24T09:10:00.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.525+0000 7f55a5800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55a0103280 0x7f55a01a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.525+0000 7f55a5800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a0103ba0 0x7f55a01a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.526+0000 7f55a5800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a010c5c0 0x7f55a019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.526+0000 7f55a5800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01122b0 con 0x7f55a010c5c0 2024-04-24T09:10:00.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.526+0000 7f55a5800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f55a0112130 con 0x7f55a0103ba0 2024-04-24T09:10:00.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.526+0000 7f55a5800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f55a0112430 con 0x7f55a0103280 2024-04-24T09:10:00.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.526+0000 7f559f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a010c5c0 0x7f55a019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.526+0000 7f559f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a010c5c0 0x7f55a019a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39548/0 (socket says 172.21.15.69:39548) 2024-04-24T09:10:00.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.526+0000 7f559f400640 1 -- 172.21.15.69:0/1726163757 learned_addr learned my addr 172.21.15.69:0/1726163757 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:00.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.526+0000 7f559ea00640 1 --2- 172.21.15.69:0/1726163757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55a0103280 0x7f55a01a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.526+0000 7f559e000640 1 --2- 172.21.15.69:0/1726163757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a0103ba0 0x7f55a01a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.526+0000 7f559f400640 1 -- 172.21.15.69:0/1726163757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55a0103280 msgr2=0x7f55a01a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.526+0000 7f559f400640 1 --2- 172.21.15.69:0/1726163757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55a0103280 0x7f55a01a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.526+0000 7f559f400640 1 -- 172.21.15.69:0/1726163757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a0103ba0 msgr2=0x7f55a01a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.526+0000 7f559f400640 1 --2- 172.21.15.69:0/1726163757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a0103ba0 0x7f55a01a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.527+0000 7f559f400640 1 -- 172.21.15.69:0/1726163757 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a019afa0 con 0x7f55a010c5c0 2024-04-24T09:10:00.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.527+0000 7f559f400640 1 --2- 172.21.15.69:0/1726163757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a010c5c0 0x7f55a019a7d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5598009dc0 tx=0x7f559802e5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:00.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.527+0000 7f5597400640 1 -- 172.21.15.69:0/1726163757 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598033040 con 0x7f55a010c5c0 2024-04-24T09:10:00.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.527+0000 7f5597400640 1 -- 172.21.15.69:0/1726163757 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5598038b00 con 0x7f55a010c5c0 2024-04-24T09:10:00.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.527+0000 7f5597400640 1 -- 172.21.15.69:0/1726163757 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598043a10 con 0x7f55a010c5c0 2024-04-24T09:10:00.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.527+0000 7f55a5800640 1 -- 172.21.15.69:0/1726163757 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a019b230 con 0x7f55a010c5c0 2024-04-24T09:10:00.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.527+0000 7f55a5800640 1 -- 172.21.15.69:0/1726163757 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a01a74b0 con 0x7f55a010c5c0 2024-04-24T09:10:00.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.531+0000 7f5597400640 1 -- 172.21.15.69:0/1726163757 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5598036070 con 0x7f55a010c5c0 2024-04-24T09:10:00.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.531+0000 7f55a8633740 1 Processor -- start 2024-04-24T09:10:00.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.531+0000 7f55a8633740 1 -- start start 2024-04-24T09:10:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.532+0000 7f55a8633740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994604950 0x558994604d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.532+0000 7f55a8633740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994605270 0x55899460f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.532+0000 7f55a8633740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55899460f940 0x558994611d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.532+0000 7f55a8633740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55899456a1d0 con 0x558994604950 2024-04-24T09:10:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.532+0000 7f55a8633740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x558994510450 con 0x55899460f940 2024-04-24T09:10:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.532+0000 7f55a8633740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558994566190 con 0x558994605270 2024-04-24T09:10:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.532+0000 7f559e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994604950 0x558994604d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.532+0000 7f559e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994604950 0x558994604d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39552/0 (socket says 172.21.15.69:39552) 2024-04-24T09:10:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.532+0000 7f559e000640 1 -- 172.21.15.69:0/385175510 learned_addr learned my addr 172.21.15.69:0/385175510 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.532+0000 7f559ea00640 1 --2- 172.21.15.69:0/385175510 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994605270 0x55899460f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.532+0000 7f559e000640 1 -- 172.21.15.69:0/385175510 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994605270 msgr2=0x55899460f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.532+0000 7f559e000640 1 --2- 172.21.15.69:0/385175510 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994605270 0x55899460f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.532+0000 7f559e000640 1 -- 172.21.15.69:0/385175510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55899460f940 msgr2=0x558994611d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.532+0000 7f559e000640 1 --2- 172.21.15.69:0/385175510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55899460f940 0x558994611d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.532+0000 7f559e000640 1 -- 172.21.15.69:0/385175510 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589945cd310 con 0x558994604950 2024-04-24T09:10:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.533+0000 7f5597400640 1 --2- 172.21.15.69:0/1726163757 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f55840a3da0 0x7f55840a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.533+0000 7f5597400640 1 -- 172.21.15.69:0/1726163757 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f55980f1c20 con 0x7f55a010c5c0 2024-04-24T09:10:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.533+0000 7f559e000640 1 --2- 172.21.15.69:0/385175510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994604950 0x558994604d30 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f558c00cca0 tx=0x7f558c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4131fd43ed0e097b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.533+0000 7f559ea00640 1 --2- 172.21.15.69:0/1726163757 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f55840a3da0 0x7f55840a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.533+0000 7f558be00640 1 -- 172.21.15.69:0/385175510 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c016070 con 0x558994604950 2024-04-24T09:10:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.533+0000 7f558be00640 1 -- 172.21.15.69:0/385175510 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f558c012b30 con 0x558994604950 2024-04-24T09:10:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.533+0000 7f558be00640 1 -- 172.21.15.69:0/385175510 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c011be0 con 0x558994604950 2024-04-24T09:10:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.534+0000 7f55a8633740 1 -- 172.21.15.69:0/385175510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994604950 msgr2=0x558994604d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.534+0000 7f55a8633740 1 --2- 172.21.15.69:0/385175510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994604950 0x558994604d30 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f558c00cca0 tx=0x7f558c010740 comp rx=0 tx=0).stop 2024-04-24T09:10:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.534+0000 7f559ea00640 1 --2- 172.21.15.69:0/1726163757 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f55840a3da0 0x7f55840a6260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x558994605f10 tx=0x7f5590008040 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.534+0000 7f55a8633740 1 -- 172.21.15.69:0/385175510 shutdown_connections 2024-04-24T09:10:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.534+0000 7f55a8633740 1 --2- 172.21.15.69:0/385175510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55899460f940 0x558994611d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.534+0000 7f55a8633740 1 --2- 172.21.15.69:0/385175510 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994605270 0x55899460f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.534+0000 7f55a8633740 1 --2- 172.21.15.69:0/385175510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994604950 0x558994604d30 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.534+0000 7f55a8633740 1 -- 172.21.15.69:0/385175510 >> 172.21.15.69:0/385175510 conn(0x558994600c70 msgr2=0x558994601050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.534+0000 7f55a8633740 1 -- 172.21.15.69:0/385175510 shutdown_connections 2024-04-24T09:10:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.534+0000 7f55a8633740 1 -- 172.21.15.69:0/385175510 wait complete. 2024-04-24T09:10:00.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.535+0000 7f55a8633740 1 -- start start 2024-04-24T09:10:00.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.535+0000 7f55a8633740 1 Processor -- start 2024-04-24T09:10:00.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.535+0000 7f55a8633740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994604950 0x55899460c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.535+0000 7f55a8633740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994605270 0x55899460cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.535+0000 7f55a8633740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55899460f940 0x55899460d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.535+0000 7f55a8633740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5589946127d0 con 0x558994605270 2024-04-24T09:10:00.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.535+0000 7f55a8633740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x558994523810 con 0x55899460f940 2024-04-24T09:10:00.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.535+0000 7f55a8633740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558994612950 con 0x558994604950 2024-04-24T09:10:00.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.535+0000 7f55a8633740 10 client.0 _ll_register_callbacks cb 0x5589944f1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:00.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.536+0000 7f559e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994605270 0x55899460cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.536+0000 7f559e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994605270 0x55899460cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39564/0 (socket says 172.21.15.69:39564) 2024-04-24T09:10:00.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.536+0000 7f559e000640 1 -- 172.21.15.69:0/1148341849 learned_addr learned my addr 172.21.15.69:0/1148341849 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:00.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.536+0000 7f559ea00640 1 --2- 172.21.15.69:0/1148341849 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994604950 0x55899460c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.536+0000 7f559f400640 1 --2- 172.21.15.69:0/1148341849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55899460f940 0x55899460d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.536+0000 7f559e000640 1 -- 172.21.15.69:0/1148341849 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994604950 msgr2=0x55899460c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:00.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.536+0000 7f559e000640 1 --2- 172.21.15.69:0/1148341849 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994604950 0x55899460c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.536+0000 7f559e000640 1 -- 172.21.15.69:0/1148341849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55899460f940 msgr2=0x55899460d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:00.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.536+0000 7f559e000640 1 --2- 172.21.15.69:0/1148341849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55899460f940 0x55899460d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.536+0000 7f559e000640 1 -- 172.21.15.69:0/1148341849 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589945343f0 con 0x558994605270 2024-04-24T09:10:00.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.537+0000 7f559e000640 1 --2- 172.21.15.69:0/1148341849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994605270 0x55899460cc60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f558c004650 tx=0x7f558c0126b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a27a7343a3ee97 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:00.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.537+0000 7f5588c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:10:00.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.537+0000 7f5588c00640 1 -- 172.21.15.69:0/1148341849 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c02a060 con 0x558994605270 2024-04-24T09:10:00.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.537+0000 7f5588c00640 1 -- 172.21.15.69:0/1148341849 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f558c011920 con 0x558994605270 2024-04-24T09:10:00.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.537+0000 7f55a8633740 1 -- 172.21.15.69:0/1148341849 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f558c012b30 con 0x558994605270 2024-04-24T09:10:00.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.537+0000 7f5588c00640 1 -- 172.21.15.69:0/1148341849 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f558c031050 con 0x558994605270 2024-04-24T09:10:00.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.537+0000 7f55a8633740 10 client.16560 fetch_fsmap learned FSMap version 29 2024-04-24T09:10:00.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.537+0000 7f55a8633740 1 -- 172.21.15.69:0/1148341849 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55899427c3f0 con 0x558994605270 2024-04-24T09:10:00.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.537+0000 7f5588c00640 1 -- 172.21.15.69:0/1148341849 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f558c012b30 con 0x558994605270 2024-04-24T09:10:00.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.538+0000 7f5588c00640 1 -- 172.21.15.69:0/1148341849 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f558c012ce0 con 0x558994605270 2024-04-24T09:10:00.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.538+0000 7f55a8633740 10 client.16560 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:10:00.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.538+0000 7f55a8633740 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:00.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.538+0000 7f55a8633740 20 client.16560 populate_metadata read hostname 'smithi069' 2024-04-24T09:10:00.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.538+0000 7f55a8633740 1 --2- 172.21.15.69:0/1148341849 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55899461fb80 0x558994621f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.538+0000 7f55a8633740 4 client.16560 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:00.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.538+0000 7f55a8633740 1 -- 172.21.15.69:0/1148341849 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558994577bf0 con 0x55899461fb80 2024-04-24T09:10:00.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.539+0000 7f559ea00640 1 --2- 172.21.15.69:0/1148341849 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55899461fb80 0x558994621f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.540+0000 7f559ea00640 1 --2- 172.21.15.69:0/1148341849 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55899461fb80 0x558994621f70 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9e056c56743e7a0 server_cookie=bfbd58b5d123987a in_seq=0 out_seq=0 2024-04-24T09:10:00.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.540+0000 7f5588c00640 10 client.16560 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:00.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.542+0000 7f5588c00640 1 -- 172.21.15.69:0/1148341849 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558994577bf0 con 0x55899461fb80 2024-04-24T09:10:00.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.542+0000 7f5588c00640 10 client.16560 handle_command_reply: tid=0 2024-04-24T09:10:00.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.542+0000 7f55a8633740 1 client.16560 shutdown 2024-04-24T09:10:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.542+0000 7f55a8633740 1 -- 172.21.15.69:0/1148341849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994605270 msgr2=0x55899460cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.542+0000 7f55a8633740 1 --2- 172.21.15.69:0/1148341849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994605270 0x55899460cc60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f558c004650 tx=0x7f558c0126b0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.542+0000 7f55a8633740 1 -- 172.21.15.69:0/1148341849 shutdown_connections 2024-04-24T09:10:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.542+0000 7f55a8633740 1 --2- 172.21.15.69:0/1148341849 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55899461fb80 0x558994621f70 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.542+0000 7f55a8633740 1 --2- 172.21.15.69:0/1148341849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55899460f940 0x55899460d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.542+0000 7f55a8633740 1 --2- 172.21.15.69:0/1148341849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994605270 0x55899460cc60 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.542+0000 7f5588c00640 0 client.16560 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.542+0000 7f55a8633740 1 --2- 172.21.15.69:0/1148341849 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994604950 0x55899460c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.542+0000 7f55a8633740 1 -- 172.21.15.69:0/1148341849 >> 172.21.15.69:0/1148341849 conn(0x558994600c70 msgr2=0x558994606970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.542+0000 7f55a8633740 1 -- 172.21.15.69:0/1148341849 shutdown_connections 2024-04-24T09:10:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.542+0000 7f55a8633740 1 -- 172.21.15.69:0/1148341849 wait complete. 2024-04-24T09:10:00.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.542+0000 7f55a8633740 20 client.16560 trim_cache size 0 max 16384 2024-04-24T09:10:00.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.558+0000 7f55a8633740 1 Processor -- start 2024-04-24T09:10:00.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.558+0000 7f55a8633740 1 -- start start 2024-04-24T09:10:00.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.558+0000 7f55a8633740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994604950 0x558994600570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.559+0000 7f55a8633740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994605270 0x558994606f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.559+0000 7f55a8633740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55899460f940 0x55899460cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.559+0000 7f55a8633740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01b3d60 con 0x558994605270 2024-04-24T09:10:00.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.559+0000 7f55a8633740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01b15a0 con 0x55899460f940 2024-04-24T09:10:00.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.559+0000 7f55a8633740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01af3f0 con 0x558994604950 2024-04-24T09:10:00.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.559+0000 7f559e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994605270 0x558994606f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.559+0000 7f559e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994605270 0x558994606f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39572/0 (socket says 172.21.15.69:39572) 2024-04-24T09:10:00.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.559+0000 7f559e000640 1 -- 172.21.15.69:0/628084320 learned_addr learned my addr 172.21.15.69:0/628084320 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:00.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.559+0000 7f559f400640 1 --2- 172.21.15.69:0/628084320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55899460f940 0x55899460cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.559+0000 7f559e000640 1 -- 172.21.15.69:0/628084320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994604950 msgr2=0x558994600570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:00.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.559+0000 7f559e000640 1 --2- 172.21.15.69:0/628084320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994604950 0x558994600570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.559+0000 7f559e000640 1 -- 172.21.15.69:0/628084320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55899460f940 msgr2=0x55899460cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:00.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.559+0000 7f559e000640 1 --2- 172.21.15.69:0/628084320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55899460f940 0x55899460cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.559+0000 7f559e000640 1 -- 172.21.15.69:0/628084320 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589946050a0 con 0x558994605270 2024-04-24T09:10:00.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.559+0000 7f559f400640 1 --2- 172.21.15.69:0/628084320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55899460f940 0x55899460cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:00.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.560+0000 7f559e000640 1 --2- 172.21.15.69:0/628084320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994605270 0x558994606f50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f558c01bd80 tx=0x7f558c00c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577c6fd0f731a5c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:00.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.560+0000 7f558b400640 1 -- 172.21.15.69:0/628084320 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c02a070 con 0x558994605270 2024-04-24T09:10:00.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.560+0000 7f558b400640 1 -- 172.21.15.69:0/628084320 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f558c01b450 con 0x558994605270 2024-04-24T09:10:00.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.560+0000 7f558b400640 1 -- 172.21.15.69:0/628084320 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c016050 con 0x558994605270 2024-04-24T09:10:00.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.561+0000 7f55a8633740 1 -- 172.21.15.69:0/628084320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994605270 msgr2=0x558994606f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.561+0000 7f55a8633740 1 --2- 172.21.15.69:0/628084320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994605270 0x558994606f50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f558c01bd80 tx=0x7f558c00c4c0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.561+0000 7f55a8633740 1 -- 172.21.15.69:0/628084320 shutdown_connections 2024-04-24T09:10:00.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.561+0000 7f55a8633740 1 --2- 172.21.15.69:0/628084320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55899460f940 0x55899460cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.561+0000 7f55a8633740 1 --2- 172.21.15.69:0/628084320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558994605270 0x558994606f50 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.561+0000 7f55a8633740 1 --2- 172.21.15.69:0/628084320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994604950 0x558994600570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.561+0000 7f55a8633740 1 -- 172.21.15.69:0/628084320 >> 172.21.15.69:0/628084320 conn(0x558994600c70 msgr2=0x55899460c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.561+0000 7f55a8633740 1 -- 172.21.15.69:0/628084320 shutdown_connections 2024-04-24T09:10:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.561+0000 7f55a8633740 1 -- 172.21.15.69:0/628084320 wait complete. 2024-04-24T09:10:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.562+0000 7f55a8633740 1 -- start start 2024-04-24T09:10:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.562+0000 7f55a8633740 1 Processor -- start 2024-04-24T09:10:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.562+0000 7f55a8633740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994604950 0x5589946104b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.562+0000 7f55a8633740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558994605270 0x5589946109f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.562+0000 7f55a8633740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55899460f940 0x558994616bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.562+0000 7f55a8633740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01afa60 con 0x55899460f940 2024-04-24T09:10:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.562+0000 7f55a8633740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01b0ed0 con 0x558994605270 2024-04-24T09:10:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.562+0000 7f55a8633740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01b0130 con 0x558994604950 2024-04-24T09:10:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.562+0000 7f55a8633740 10 client.0 _ll_register_callbacks cb 0x7f558c00cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:00.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.563+0000 7f559f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55899460f940 0x558994616bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.563+0000 7f559f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55899460f940 0x558994616bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39586/0 (socket says 172.21.15.69:39586) 2024-04-24T09:10:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.563+0000 7f559f400640 1 -- 172.21.15.69:0/3104437635 learned_addr learned my addr 172.21.15.69:0/3104437635 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.563+0000 7f559e000640 1 --2- 172.21.15.69:0/3104437635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558994605270 0x5589946109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.563+0000 7f559ea00640 1 --2- 172.21.15.69:0/3104437635 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994604950 0x5589946104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.563+0000 7f559f400640 1 -- 172.21.15.69:0/3104437635 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994604950 msgr2=0x5589946104b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.563+0000 7f559f400640 1 --2- 172.21.15.69:0/3104437635 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994604950 0x5589946104b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.563+0000 7f559f400640 1 -- 172.21.15.69:0/3104437635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558994605270 msgr2=0x5589946109f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.563+0000 7f559f400640 1 --2- 172.21.15.69:0/3104437635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558994605270 0x5589946109f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.563+0000 7f559f400640 1 -- 172.21.15.69:0/3104437635 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589943467d0 con 0x55899460f940 2024-04-24T09:10:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.563+0000 7f559ea00640 1 --2- 172.21.15.69:0/3104437635 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994604950 0x5589946104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.563+0000 7f559f400640 1 --2- 172.21.15.69:0/3104437635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55899460f940 0x558994616bd0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f55980bd7b0 tx=0x7f55980bd240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fd6996c049d34a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.563+0000 7f5588c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:10:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.564+0000 7f5588c00640 1 -- 172.21.15.69:0/3104437635 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55980c1bd0 con 0x55899460f940 2024-04-24T09:10:00.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.564+0000 7f5588c00640 1 -- 172.21.15.69:0/3104437635 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f55980b9e90 con 0x55899460f940 2024-04-24T09:10:00.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.564+0000 7f55a8633740 1 -- 172.21.15.69:0/3104437635 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f558c01b450 con 0x55899460f940 2024-04-24T09:10:00.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.564+0000 7f5588c00640 1 -- 172.21.15.69:0/3104437635 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f55980c6490 con 0x55899460f940 2024-04-24T09:10:00.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.564+0000 7f55a8633740 10 client.16566 fetch_fsmap learned FSMap version 29 2024-04-24T09:10:00.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.564+0000 7f55a8633740 1 -- 172.21.15.69:0/3104437635 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589946181b0 con 0x55899460f940 2024-04-24T09:10:00.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.564+0000 7f5588c00640 1 -- 172.21.15.69:0/3104437635 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f558c01b450 con 0x55899460f940 2024-04-24T09:10:00.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.564+0000 7f5588c00640 1 -- 172.21.15.69:0/3104437635 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f5598038650 con 0x55899460f940 2024-04-24T09:10:00.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.564+0000 7f55a8633740 10 client.16566 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:10:00.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.564+0000 7f55a8633740 10 client.16566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:00.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.564+0000 7f55a8633740 20 client.16566 populate_metadata read hostname 'smithi069' 2024-04-24T09:10:00.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.564+0000 7f55a8633740 1 --2- 172.21.15.69:0/3104437635 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5589946860a0 0x5589946884c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.564+0000 7f55a8633740 4 client.16566 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:10:00.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.564+0000 7f55a8633740 1 -- 172.21.15.69:0/3104437635 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558994571a60 con 0x5589946860a0 2024-04-24T09:10:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.565+0000 7f559ea00640 1 --2- 172.21.15.69:0/3104437635 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5589946860a0 0x5589946884c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.566+0000 7f559ea00640 1 --2- 172.21.15.69:0/3104437635 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5589946860a0 0x5589946884c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=954ed1b5fe1b254e server_cookie=3cf453d525873e0a in_seq=0 out_seq=0 2024-04-24T09:10:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.566+0000 7f5588c00640 10 client.16566 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.566+0000 7f5588c00640 1 -- 172.21.15.69:0/3104437635 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558994571a60 con 0x5589946860a0 2024-04-24T09:10:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.566+0000 7f5588c00640 10 client.16566 handle_command_reply: tid=0 2024-04-24T09:10:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.566+0000 7f55a8633740 1 client.16566 shutdown 2024-04-24T09:10:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.567+0000 7f55a8633740 1 -- 172.21.15.69:0/3104437635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55899460f940 msgr2=0x558994616bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.567+0000 7f55a8633740 1 --2- 172.21.15.69:0/3104437635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55899460f940 0x558994616bd0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f55980bd7b0 tx=0x7f55980bd240 comp rx=0 tx=0).stop 2024-04-24T09:10:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.567+0000 7f55a8633740 1 -- 172.21.15.69:0/3104437635 shutdown_connections 2024-04-24T09:10:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.567+0000 7f55a8633740 1 --2- 172.21.15.69:0/3104437635 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5589946860a0 0x5589946884c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.567+0000 7f55a8633740 1 --2- 172.21.15.69:0/3104437635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55899460f940 0x558994616bd0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.567+0000 7f55a8633740 1 --2- 172.21.15.69:0/3104437635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558994605270 0x5589946109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.567+0000 7f5588c00640 0 client.16566 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.567+0000 7f55a8633740 1 --2- 172.21.15.69:0/3104437635 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558994604950 0x5589946104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.567+0000 7f55a8633740 1 -- 172.21.15.69:0/3104437635 >> 172.21.15.69:0/3104437635 conn(0x558994600c70 msgr2=0x55899460cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:00.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.567+0000 7f55a8633740 1 -- 172.21.15.69:0/3104437635 shutdown_connections 2024-04-24T09:10:00.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.567+0000 7f55a8633740 1 -- 172.21.15.69:0/3104437635 wait complete. 2024-04-24T09:10:00.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.567+0000 7f55a8633740 20 client.16566 trim_cache size 0 max 16384 2024-04-24T09:10:00.570 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-04-24T09:10:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.568+0000 7f55a5800640 1 -- 172.21.15.69:0/1726163757 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f55840a3da0 msgr2=0x7f55840a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.568+0000 7f55a5800640 1 --2- 172.21.15.69:0/1726163757 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f55840a3da0 0x7f55840a6260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x558994605f10 tx=0x7f5590008040 comp rx=0 tx=0).stop 2024-04-24T09:10:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.568+0000 7f55a5800640 1 -- 172.21.15.69:0/1726163757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a010c5c0 msgr2=0x7f55a019a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.568+0000 7f55a5800640 1 --2- 172.21.15.69:0/1726163757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a010c5c0 0x7f55a019a7d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5598009dc0 tx=0x7f559802e5c0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.568+0000 7f55a5800640 1 -- 172.21.15.69:0/1726163757 shutdown_connections 2024-04-24T09:10:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.568+0000 7f55a5800640 1 --2- 172.21.15.69:0/1726163757 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f55840a3da0 0x7f55840a6260 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.568+0000 7f55a5800640 1 --2- 172.21.15.69:0/1726163757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a010c5c0 0x7f55a019a7d0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.568+0000 7f55a5800640 1 --2- 172.21.15.69:0/1726163757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a0103ba0 0x7f55a01a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.568+0000 7f55a5800640 1 --2- 172.21.15.69:0/1726163757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55a0103280 0x7f55a01a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.568+0000 7f55a5800640 1 -- 172.21.15.69:0/1726163757 >> 172.21.15.69:0/1726163757 conn(0x7f55a0078000 msgr2=0x7f55a0108490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.568+0000 7f55a5800640 1 -- 172.21.15.69:0/1726163757 shutdown_connections 2024-04-24T09:10:00.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:10:00.568+0000 7f55a5800640 1 -- 172.21.15.69:0/1726163757 wait complete. 2024-04-24T09:10:00.586 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 16346359, 'ino': 1099511628532, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/attr_replace_test.c'}, {'damage_type': 'backtrace', 'id': 16571550, 'ino': 1099511628602, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mmapcat.c'}, {'damage_type': 'backtrace', 'id': 51813164, 'ino': 1099511628424, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/zoned'}, {'damage_type': 'backtrace', 'id': 60752935, 'ino': 1099511628585, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/locktest.c'}, {'damage_type': 'backtrace', 'id': 112369589, 'ino': 1099511628603, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/multi_open_unlink.c'}, {'damage_type': 'backtrace', 'id': 114305430, 'ino': 1099511628543, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/detached_mounts_propagation.c'}, {'damage_type': 'backtrace', 'id': 193744552, 'ino': 1099511628421, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ubifs'}, {'damage_type': 'backtrace', 'id': 251521906, 'ino': 1099511628378, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/btrfs'}, {'damage_type': 'backtrace', 'id': 347365333, 'ino': 1099511628320, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects/pack'}, {'damage_type': 'backtrace', 'id': 354688950, 'ino': 1099511628701, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/vfstest.h'}, {'damage_type': 'backtrace', 'id': 360648059, 'ino': 1099511628692, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/idmapped-mounts.c'}, {'damage_type': 'backtrace', 'id': 397584521, 'ino': 1099511628579, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/helpers.h'}, {'damage_type': 'backtrace', 'id': 423463873, 'ino': 1099511628523, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-io-setup-with-nonwritable-context-pointer.c'}, {'damage_type': 'backtrace', 'id': 425485500, 'ino': 1099511628689, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/Makefile'}, {'damage_type': 'backtrace', 'id': 568509839, 'ino': 1099511628338, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs'}, {'damage_type': 'backtrace', 'id': 596321650, 'ino': 1099511628617, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/permname.c'}, {'damage_type': 'backtrace', 'id': 623588392, 'ino': 1099511628513, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-cycle-write.c'}, {'damage_type': 'backtrace', 'id': 657599122, 'ino': 1099511628410, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/preamble'}, {'damage_type': 'backtrace', 'id': 661629170, 'ino': 1099511628517, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-hole-filling-race.c'}, {'damage_type': 'backtrace', 'id': 691569761, 'ino': 1099511628386, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmerror'}, {'damage_type': 'backtrace', 'id': 716536554, 'ino': 1099511628342, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/HEAD'}, {'damage_type': 'backtrace', 'id': 739800478, 'ino': 1099511628511, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-append-write-read-race.c'}, {'damage_type': 'backtrace', 'id': 973563093, 'ino': 1099511628394, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/f2fs'}, {'damage_type': 'backtrace', 'id': 1062566809, 'ino': 1099511628293, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/branches'}, {'damage_type': 'backtrace', 'id': 1071382724, 'ino': 1099511628383, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/defrag'}, {'damage_type': 'backtrace', 'id': 1109593105, 'ino': 1099511628382, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/config'}, {'damage_type': 'backtrace', 'id': 1138208390, 'ino': 1099511628695, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/mount-idmapped.c'}, {'damage_type': 'backtrace', 'id': 1150784194, 'ino': 1099511628319, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects'}, {'damage_type': 'backtrace', 'id': 1176805898, 'ino': 1099511628562, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fill.c'}, {'damage_type': 'backtrace', 'id': 1220744182, 'ino': 1099511628310, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs/heads'}, {'damage_type': 'backtrace', 'id': 1238440630, 'ino': 1099511628379, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/casefold'}, {'damage_type': 'backtrace', 'id': 1254632052, 'ino': 1099511628667, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_mtab.c'}, {'damage_type': 'backtrace', 'id': 1267301714, 'ino': 1099511628381, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/cgroup2'}, {'damage_type': 'backtrace', 'id': 1270955011, 'ino': 1099511628698, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/utils.c'}, {'damage_type': 'backtrace', 'id': 1272080746, 'ino': 1099511628608, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/parse-extent-tree.awk'}, {'damage_type': 'backtrace', 'id': 1309098796, 'ino': 1099511628648, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_type.c'}, {'damage_type': 'backtrace', 'id': 1351815011, 'ino': 1099511628347, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/heads/master'}, {'damage_type': 'backtrace', 'id': 1366126793, 'ino': 1099511628393, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ext4'}, {'damage_type': 'backtrace', 'id': 1449166465, 'ino': 1099511628423, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/xfs'}, {'damage_type': 'backtrace', 'id': 1461104012, 'ino': 1099511628405, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/module'}, {'damage_type': 'backtrace', 'id': 1471082224, 'ino': 1099511628639, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/stat_test.c'}, {'damage_type': 'backtrace', 'id': 1474779584, 'ino': 1099511628656, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_holes.c'}, {'damage_type': 'backtrace', 'id': 1478600361, 'ino': 1099511628696, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/tmpfs-idmapped-mounts.c'}, {'damage_type': 'backtrace', 'id': 1555501239, 'ino': 1099511628526, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aiodio_sparse2.c'}, {'damage_type': 'backtrace', 'id': 1622951904, 'ino': 1099511628351, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/index.lock'}, {'damage_type': 'backtrace', 'id': 1625471636, 'ino': 1099511628346, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/heads'}, {'damage_type': 'backtrace', 'id': 1650477561, 'ino': 1099511628705, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/xfsfind.c'}, {'damage_type': 'backtrace', 'id': 1691164371, 'ino': 1099511628395, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/fail_make_request'}, {'damage_type': 'backtrace', 'id': 1706998959, 'ino': 1099511628521, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-write-verify.c'}, {'damage_type': 'backtrace', 'id': 1752912995, 'ino': 1099511628525, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aiocp.c'}, {'damage_type': 'backtrace', 'id': 1778126529, 'ino': 1099511628520, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-subblock-eof-read.c'}, {'damage_type': 'backtrace', 'id': 1859138295, 'ino': 1099511628514, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-eof-race.c'}, {'damage_type': 'backtrace', 'id': 1901393888, 'ino': 1099511628563, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fill2.c'}, {'damage_type': 'backtrace', 'id': 1932647416, 'ino': 1099511628552, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/dumpfile'}, {'damage_type': 'backtrace', 'id': 1935523013, 'ino': 1099511628420, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/tracing'}, {'damage_type': 'backtrace', 'id': 1937343359, 'ino': 1099511628402, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/locktest'}, {'damage_type': 'backtrace', 'id': 1972562517, 'ino': 1099511628690, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/btrfs-idmapped-mounts.c'}, {'damage_type': 'backtrace', 'id': 1993416556, 'ino': 1099511628510, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-append-write-fallocate-race.c'}, {'damage_type': 'backtrace', 'id': 2022371200, 'ino': 1099511628700, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/vfstest.c'}, {'damage_type': 'backtrace', 'id': 2029111680, 'ino': 1099511628646, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_offset.c'}, {'damage_type': 'backtrace', 'id': 2058912674, 'ino': 1099511628619, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/popdir.pl'}, {'damage_type': 'backtrace', 'id': 2067213811, 'ino': 1099511628380, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/ceph'}, {'damage_type': 'backtrace', 'id': 2075342033, 'ino': 1099511628397, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filter'}, {'damage_type': 'backtrace', 'id': 2135503088, 'ino': 1099511628582, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/iopat.c'}, {'damage_type': 'backtrace', 'id': 2153018659, 'ino': 1099511628321, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/objects/info'}, {'damage_type': 'backtrace', 'id': 2200440559, 'ino': 1099511628559, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fiemap-fault.c'}, {'damage_type': 'backtrace', 'id': 2226745200, 'ino': 1099511628340, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin'}, {'damage_type': 'backtrace', 'id': 2252314040, 'ino': 1099511628401, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/inject'}, {'damage_type': 'backtrace', 'id': 2348526104, 'ino': 1099511628341, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin/HEAD'}, {'damage_type': 'backtrace', 'id': 2363111645, 'ino': 1099511628419, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/test_names'}, {'damage_type': 'backtrace', 'id': 2384831929, 'ino': 1099511628693, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/idmapped-mounts.h'}, {'damage_type': 'backtrace', 'id': 2406893047, 'ino': 1099511628415, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/renameat2'}, {'damage_type': 'backtrace', 'id': 2414428574, 'ino': 1099511628388, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmhugedisk'}, {'damage_type': 'backtrace', 'id': 2437225212, 'ino': 1099511628509, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/SOURCE'}, {'damage_type': 'backtrace', 'id': 2450297104, 'ino': 1099511628691, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/btrfs-idmapped-mounts.h'}, {'damage_type': 'backtrace', 'id': 2460575319, 'ino': 1099511628627, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/renameat2.c'}, {'damage_type': 'backtrace', 'id': 2470239890, 'ino': 1099511628417, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/report'}, {'damage_type': 'backtrace', 'id': 2493104681, 'ino': 1099511628311, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs/tags'}, {'damage_type': 'backtrace', 'id': 2510154718, 'ino': 1099511628412, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/quota'}, {'damage_type': 'backtrace', 'id': 2513561305, 'ino': 1099511628561, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fiexchange.h'}, {'damage_type': 'backtrace', 'id': 2529868763, 'ino': 1099511628512, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-cow-race.c'}, {'damage_type': 'backtrace', 'id': 2537755835, 'ino': 1099511628385, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmdust'}, {'damage_type': 'backtrace', 'id': 2554771788, 'ino': 1099511628404, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/metadump'}, {'damage_type': 'backtrace', 'id': 2616192978, 'ino': 1099511628666, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_mmap_writev_overlap.c'}, {'damage_type': 'backtrace', 'id': 2682881827, 'ino': 1099511628396, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filestreams'}, {'damage_type': 'backtrace', 'id': 2757371729, 'ino': 1099511628647, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/t_dir_offset2.c'}, {'damage_type': 'backtrace', 'id': 2764086984, 'ino': 1099511628416, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/repair'}, {'damage_type': 'backtrace', 'id': 2766977774, 'ino': 1099511628403, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/log'}, {'damage_type': 'backtrace', 'id': 2775104699, 'ino': 1099511628392, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/encrypt'}, {'damage_type': 'backtrace', 'id': 2785148965, 'ino': 1099511628694, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/missing.h'}, {'damage_type': 'backtrace', 'id': 2806222874, 'ino': 1099511628640, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/statx.h'}, {'damage_type': 'backtrace', 'id': 2826799643, 'ino': 1099511628414, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/reflink'}, {'damage_type': 'backtrace', 'id': 2841452546, 'ino': 1099511628581, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/holetest.c'}, {'damage_type': 'backtrace', 'id': 2865704986, 'ino': 1099511628507, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress'}, {'damage_type': 'backtrace', 'id': 2895596337, 'ino': 1099511628408, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/perf'}, {'damage_type': 'backtrace', 'id': 2956292765, 'ino': 1099511628418, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/scsi_debug'}, {'damage_type': 'backtrace', 'id': 2967705891, 'ino': 1099511628596, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/md5.c'}, {'damage_type': 'backtrace', 'id': 2994931498, 'ino': 1099511628389, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmlogwrites'}, {'damage_type': 'backtrace', 'id': 3036975500, 'ino': 1099511628600, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mmap-rw-fault.c'}, {'damage_type': 'backtrace', 'id': 3078982028, 'ino': 1099511628577, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/global.h'}, {'damage_type': 'backtrace', 'id': 3110092865, 'ino': 1099511628413, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/rc'}, {'damage_type': 'backtrace', 'id': 3150696945, 'ino': 1099511628411, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/punch'}, {'damage_type': 'backtrace', 'id': 3175535340, 'ino': 1099511627785, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/description'}, {'damage_type': 'backtrace', 'id': 3189380125, 'ino': 1099511628407, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/overlay'}, {'damage_type': 'backtrace', 'id': 3205728076, 'ino': 1099511628688, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs'}, {'damage_type': 'backtrace', 'id': 3213359877, 'ino': 1099511628409, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/populate'}, {'damage_type': 'backtrace', 'id': 3223676272, 'ino': 1099511628391, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dump'}, {'damage_type': 'backtrace', 'id': 3259465019, 'ino': 1099511628387, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmflakey'}, {'damage_type': 'backtrace', 'id': 3352404736, 'ino': 1099511628516, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-fcntl-race.c'}, {'damage_type': 'backtrace', 'id': 3364014463, 'ino': 1099511628625, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/readdir-while-renames.c'}, {'damage_type': 'backtrace', 'id': 3389909060, 'ino': 1099511628331, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/shallow'}, {'damage_type': 'backtrace', 'id': 3409724052, 'ino': 1099511628376, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/Makefile'}, {'damage_type': 'backtrace', 'id': 3436063123, 'ino': 1099511628333, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/packed-refs'}, {'damage_type': 'backtrace', 'id': 3487165587, 'ino': 1099511628556, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/fake-dump-rootino.c'}, {'damage_type': 'backtrace', 'id': 3522762471, 'ino': 1099511628337, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs'}, {'damage_type': 'backtrace', 'id': 3535594575, 'ino': 1099511628377, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/attr'}, {'damage_type': 'backtrace', 'id': 3554403979, 'ino': 1099511628524, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-last-ref-held-by-io.c'}, {'damage_type': 'backtrace', 'id': 3643332925, 'ino': 1099511628542, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/deduperace.c'}, {'damage_type': 'backtrace', 'id': 3676432262, 'ino': 1099511628345, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/HEAD'}, {'damage_type': 'backtrace', 'id': 3683754233, 'ino': 1099511628518, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-invalidate-failure.c'}, {'damage_type': 'backtrace', 'id': 3792303172, 'ino': 1099511628390, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmthin'}, {'damage_type': 'backtrace', 'id': 3797619330, 'ino': 1099511628697, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/tmpfs-idmapped-mounts.h'}, {'damage_type': 'backtrace', 'id': 3815626013, 'ino': 1099511628398, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/filter.btrfs'}, {'damage_type': 'backtrace', 'id': 3820118046, 'ino': 1099511628294, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks'}, {'damage_type': 'backtrace', 'id': 3850222811, 'ino': 1099511628384, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/dmdelay'}, {'damage_type': 'backtrace', 'id': 3886813943, 'ino': 1099511628422, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/verity'}, {'damage_type': 'backtrace', 'id': 3909347137, 'ino': 1099511628339, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/logs/refs/remotes'}, {'damage_type': 'backtrace', 'id': 3918391727, 'ino': 1099511628599, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/mkswap.c'}, {'damage_type': 'backtrace', 'id': 3923855366, 'ino': 1099511628399, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/fuzzy'}, {'damage_type': 'backtrace', 'id': 3924492183, 'ino': 1099511628309, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/refs'}, {'damage_type': 'backtrace', 'id': 3951747651, 'ino': 1099511628400, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/gcov'}, {'damage_type': 'backtrace', 'id': 3957244530, 'ino': 1099511628522, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-free-ring-with-bogus-nr-pages.c'}, {'damage_type': 'backtrace', 'id': 3977505621, 'ino': 1099511628350, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/config'}, {'damage_type': 'backtrace', 'id': 3986406120, 'ino': 1099511628519, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-invalidate-readahead.c'}, {'damage_type': 'backtrace', 'id': 4040529996, 'ino': 1099511628406, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/common/nfs'}, {'damage_type': 'backtrace', 'id': 4061375483, 'ino': 1099511628292, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/info/exclude'}, {'damage_type': 'backtrace', 'id': 4128685640, 'ino': 1099511628699, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/vfs/utils.h'}, {'damage_type': 'backtrace', 'id': 4138139763, 'ino': 1099511628515, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/aio-dio-extend-stat.c'}, {'damage_type': 'backtrace', 'id': 4151276909, 'ino': 1099511628508, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/aio-dio-regress/Makefile'}, {'damage_type': 'backtrace', 'id': 4244343495, 'ino': 1099511628291, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/info'}] 2024-04-24T09:10:00.587 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 6833472, 'ino': 1099511628885, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/089'}, {'damage_type': 'backtrace', 'id': 10197908, 'ino': 1099511628764, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/028.out'}, {'damage_type': 'backtrace', 'id': 35192652, 'ino': 1099511628870, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/081.out'}, {'damage_type': 'backtrace', 'id': 42724336, 'ino': 1099511628306, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/commit-msg.sample'}, {'damage_type': 'backtrace', 'id': 70250420, 'ino': 1099511628838, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/065.out'}, {'damage_type': 'backtrace', 'id': 81870884, 'ino': 1099511628829, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/061'}, {'damage_type': 'backtrace', 'id': 107366220, 'ino': 1099511628896, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/094.out'}, {'damage_type': 'backtrace', 'id': 116639293, 'ino': 1099511628761, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/027'}, {'damage_type': 'backtrace', 'id': 121570661, 'ino': 1099511628734, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/013.out'}, {'damage_type': 'backtrace', 'id': 151447312, 'ino': 1099511628912, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/102.out'}, {'damage_type': 'backtrace', 'id': 169424799, 'ino': 1099511628906, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/099.out'}, {'damage_type': 'backtrace', 'id': 198807592, 'ino': 1099511628758, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/025.out'}, {'damage_type': 'backtrace', 'id': 209374472, 'ino': 1099511628732, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/012.out'}, {'damage_type': 'backtrace', 'id': 247095520, 'ino': 1099511628895, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/094'}, {'damage_type': 'backtrace', 'id': 247475753, 'ino': 1099511628792, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/042.out'}, {'damage_type': 'backtrace', 'id': 274729540, 'ino': 1099511628855, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/074'}, {'damage_type': 'backtrace', 'id': 276924496, 'ino': 1099511628736, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/014.out'}, {'damage_type': 'backtrace', 'id': 282137058, 'ino': 1099511628730, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/011.out'}, {'damage_type': 'backtrace', 'id': 345159256, 'ino': 1099511628751, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/022'}, {'damage_type': 'backtrace', 'id': 347989433, 'ino': 1099511628908, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/100.out'}, {'damage_type': 'backtrace', 'id': 349773390, 'ino': 1099511628760, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/026.out'}, {'damage_type': 'backtrace', 'id': 352876795, 'ino': 1099511628852, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/072.out'}, {'damage_type': 'backtrace', 'id': 359522890, 'ino': 1099511628800, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/046.out'}, {'damage_type': 'backtrace', 'id': 370415538, 'ino': 1099511628913, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/103'}, {'damage_type': 'backtrace', 'id': 389553222, 'ino': 1099511628914, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/103.out'}, {'damage_type': 'backtrace', 'id': 395987537, 'ino': 1099511628904, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/098.out'}, {'damage_type': 'backtrace', 'id': 403911283, 'ino': 1099511628917, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/105'}, {'damage_type': 'backtrace', 'id': 479910113, 'ino': 1099511628890, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/091.out'}, {'damage_type': 'backtrace', 'id': 486813706, 'ino': 1099511628713, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/003'}, {'damage_type': 'backtrace', 'id': 497422423, 'ino': 1099511628921, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/107'}, {'damage_type': 'backtrace', 'id': 536409285, 'ino': 1099511628298, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-commit.sample'}, {'damage_type': 'backtrace', 'id': 571174634, 'ino': 1099511628724, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/008.out'}, {'damage_type': 'backtrace', 'id': 573041619, 'ino': 1099511628791, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/042'}, {'damage_type': 'backtrace', 'id': 573579736, 'ino': 1099511628875, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/084'}, {'damage_type': 'backtrace', 'id': 595180975, 'ino': 1099511628894, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/093.out'}, {'damage_type': 'backtrace', 'id': 612464666, 'ino': 1099511628822, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/057.out'}, {'damage_type': 'backtrace', 'id': 621203877, 'ino': 1099511628762, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/027.out'}, {'damage_type': 'backtrace', 'id': 661840137, 'ino': 1099511628858, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/075.out'}, {'damage_type': 'backtrace', 'id': 666721323, 'ino': 1099511628796, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/044.out'}, {'damage_type': 'backtrace', 'id': 682902885, 'ino': 1099511628808, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/050.out'}, {'damage_type': 'backtrace', 'id': 728984796, 'ino': 1099511628757, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/025'}, {'damage_type': 'backtrace', 'id': 746514381, 'ino': 1099511628817, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/055'}, {'damage_type': 'backtrace', 'id': 757949829, 'ino': 1099511628880, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/086.out'}, {'damage_type': 'backtrace', 'id': 780740565, 'ino': 1099511628821, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/057'}, {'damage_type': 'backtrace', 'id': 794937842, 'ino': 1099511628863, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/078'}, {'damage_type': 'backtrace', 'id': 799408602, 'ino': 1099511628707, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/Makefile'}, {'damage_type': 'backtrace', 'id': 861951983, 'ino': 1099511628765, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/029'}, {'damage_type': 'backtrace', 'id': 866919816, 'ino': 1099511628819, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/056'}, {'damage_type': 'backtrace', 'id': 873458021, 'ino': 1099511628869, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/081'}, {'damage_type': 'backtrace', 'id': 900880342, 'ino': 1099511628742, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/017.out'}, {'damage_type': 'backtrace', 'id': 914338860, 'ino': 1099511628738, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/015.out'}, {'damage_type': 'backtrace', 'id': 919970241, 'ino': 1099511628777, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/035'}, {'damage_type': 'backtrace', 'id': 924674784, 'ino': 1099511628790, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/041.out'}, {'damage_type': 'backtrace', 'id': 938267888, 'ino': 1099511628728, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/010.out'}, {'damage_type': 'backtrace', 'id': 993090496, 'ino': 1099511628741, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/017'}, {'damage_type': 'backtrace', 'id': 1005868011, 'ino': 1099511628909, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/101'}, {'damage_type': 'backtrace', 'id': 1010882015, 'ino': 1099511628876, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/084.out'}, {'damage_type': 'backtrace', 'id': 1017009133, 'ino': 1099511628731, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/012'}, {'damage_type': 'backtrace', 'id': 1050878852, 'ino': 1099511628857, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/075'}, {'damage_type': 'backtrace', 'id': 1095470494, 'ino': 1099511628831, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/062'}, {'damage_type': 'backtrace', 'id': 1112913858, 'ino': 1099511628892, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/092.out'}, {'damage_type': 'backtrace', 'id': 1145070476, 'ino': 1099511628891, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/092'}, {'damage_type': 'backtrace', 'id': 1154134369, 'ino': 1099511628809, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/051'}, {'damage_type': 'backtrace', 'id': 1167676811, 'ino': 1099511628725, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/009'}, {'damage_type': 'backtrace', 'id': 1182048718, 'ino': 1099511628815, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/054'}, {'damage_type': 'backtrace', 'id': 1197026225, 'ino': 1099511628866, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/079.out'}, {'damage_type': 'backtrace', 'id': 1211208845, 'ino': 1099511628871, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/082'}, {'damage_type': 'backtrace', 'id': 1228888468, 'ino': 1099511628926, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/109.out'}, {'damage_type': 'backtrace', 'id': 1246738782, 'ino': 1099511628774, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/033.out'}, {'damage_type': 'backtrace', 'id': 1253792056, 'ino': 1099511628832, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/062.out'}, {'damage_type': 'backtrace', 'id': 1277993620, 'ino': 1099511628776, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/034.out'}, {'damage_type': 'backtrace', 'id': 1313621005, 'ino': 1099511628854, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/073.out'}, {'damage_type': 'backtrace', 'id': 1336522891, 'ino': 1099511628719, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/006'}, {'damage_type': 'backtrace', 'id': 1362312878, 'ino': 1099511628735, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/014'}, {'damage_type': 'backtrace', 'id': 1395216955, 'ino': 1099511628850, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/071.out'}, {'damage_type': 'backtrace', 'id': 1415592038, 'ino': 1099511628887, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/090'}, {'damage_type': 'backtrace', 'id': 1423465270, 'ino': 1099511628849, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/071'}, {'damage_type': 'backtrace', 'id': 1490179130, 'ino': 1099511628877, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/085'}, {'damage_type': 'backtrace', 'id': 1490521142, 'ino': 1099511628711, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/002'}, {'damage_type': 'backtrace', 'id': 1492149154, 'ino': 1099511628824, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/058.out'}, {'damage_type': 'backtrace', 'id': 1496924110, 'ino': 1099511628868, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/080.out'}, {'damage_type': 'backtrace', 'id': 1510783330, 'ino': 1099511628826, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/059.out'}, {'damage_type': 'backtrace', 'id': 1511554780, 'ino': 1099511628888, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/090.out'}, {'damage_type': 'backtrace', 'id': 1531955352, 'ino': 1099511628844, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/068.out'}, {'damage_type': 'backtrace', 'id': 1535816944, 'ino': 1099511628717, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/005'}, {'damage_type': 'backtrace', 'id': 1565382742, 'ino': 1099511628746, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/019.out'}, {'damage_type': 'backtrace', 'id': 1572794069, 'ino': 1099511628749, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/021'}, {'damage_type': 'backtrace', 'id': 1596879710, 'ino': 1099511628773, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/033'}, {'damage_type': 'backtrace', 'id': 1619813245, 'ino': 1099511628755, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/024'}, {'damage_type': 'backtrace', 'id': 1620929781, 'ino': 1099511628710, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/001.out'}, {'damage_type': 'backtrace', 'id': 1623436965, 'ino': 1099511628834, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/063.out'}, {'damage_type': 'backtrace', 'id': 1625175505, 'ino': 1099511628883, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/088'}, {'damage_type': 'backtrace', 'id': 1650899002, 'ino': 1099511628897, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/095'}, {'damage_type': 'backtrace', 'id': 1690181762, 'ino': 1099511628840, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/066.out'}, {'damage_type': 'backtrace', 'id': 1690875043, 'ino': 1099511628882, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/087.out'}, {'damage_type': 'backtrace', 'id': 1696970307, 'ino': 1099511628743, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/018'}, {'damage_type': 'backtrace', 'id': 1716110339, 'ino': 1099511628737, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/015'}, {'damage_type': 'backtrace', 'id': 1728333106, 'ino': 1099511628874, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/083.out'}, {'damage_type': 'backtrace', 'id': 1753879886, 'ino': 1099511628910, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/101.out'}, {'damage_type': 'backtrace', 'id': 1756725107, 'ino': 1099511628924, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/108.out'}, {'damage_type': 'backtrace', 'id': 1824056666, 'ino': 1099511628766, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/029.out'}, {'damage_type': 'backtrace', 'id': 1844575851, 'ino': 1099511628795, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/044'}, {'damage_type': 'backtrace', 'id': 1885275833, 'ino': 1099511628922, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/107.out'}, {'damage_type': 'backtrace', 'id': 1888039795, 'ino': 1099511628307, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/applypatch-msg.sample'}, {'damage_type': 'backtrace', 'id': 1891456333, 'ino': 1099511628799, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/046'}, {'damage_type': 'backtrace', 'id': 1911002322, 'ino': 1099511628861, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/077'}, {'damage_type': 'backtrace', 'id': 1922983864, 'ino': 1099511628853, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/073'}, {'damage_type': 'backtrace', 'id': 1940093103, 'ino': 1099511628928, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/110.out'}, {'damage_type': 'backtrace', 'id': 1952694501, 'ino': 1099511628823, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/058'}, {'damage_type': 'backtrace', 'id': 1994408889, 'ino': 1099511628748, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/020.out'}, {'damage_type': 'backtrace', 'id': 2020549417, 'ino': 1099511628308, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-rebase.sample'}, {'damage_type': 'backtrace', 'id': 2023486775, 'ino': 1099511628901, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/097'}, {'damage_type': 'backtrace', 'id': 2036875585, 'ino': 1099511628789, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/041'}, {'damage_type': 'backtrace', 'id': 2093201799, 'ino': 1099511628811, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/052'}, {'damage_type': 'backtrace', 'id': 2117163858, 'ino': 1099511628720, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/006.out'}, {'damage_type': 'backtrace', 'id': 2125025357, 'ino': 1099511628907, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/100'}, {'damage_type': 'backtrace', 'id': 2157719301, 'ino': 1099511628780, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/036.out'}, {'damage_type': 'backtrace', 'id': 2164141607, 'ino': 1099511628881, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/087'}, {'damage_type': 'backtrace', 'id': 2187450071, 'ino': 1099511628797, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/045'}, {'damage_type': 'backtrace', 'id': 2196276658, 'ino': 1099511628813, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/053'}, {'damage_type': 'backtrace', 'id': 2271795098, 'ino': 1099511628843, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/068'}, {'damage_type': 'backtrace', 'id': 2286410204, 'ino': 1099511628873, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/083'}, {'damage_type': 'backtrace', 'id': 2296489530, 'ino': 1099511628859, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/076'}, {'damage_type': 'backtrace', 'id': 2299736265, 'ino': 1099511628898, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/095.out'}, {'damage_type': 'backtrace', 'id': 2302233637, 'ino': 1099511628714, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/003.out'}, {'damage_type': 'backtrace', 'id': 2392938233, 'ino': 1099511628763, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/028'}, {'damage_type': 'backtrace', 'id': 2396655253, 'ino': 1099511628900, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/096.out'}, {'damage_type': 'backtrace', 'id': 2399602165, 'ino': 1099511628830, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/061.out'}, {'damage_type': 'backtrace', 'id': 2405514240, 'ino': 1099511628884, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/088.out'}, {'damage_type': 'backtrace', 'id': 2415499212, 'ino': 1099511628775, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/034'}, {'damage_type': 'backtrace', 'id': 2427308658, 'ino': 1099511628745, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/019'}, {'damage_type': 'backtrace', 'id': 2450332114, 'ino': 1099511628772, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/032.out'}, {'damage_type': 'backtrace', 'id': 2458238310, 'ino': 1099511628810, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/051.out'}, {'damage_type': 'backtrace', 'id': 2502427690, 'ino': 1099511628818, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/055.out'}, {'damage_type': 'backtrace', 'id': 2511052427, 'ino': 1099511628740, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/016.out'}, {'damage_type': 'backtrace', 'id': 2519597140, 'ino': 1099511628837, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/065'}, {'damage_type': 'backtrace', 'id': 2522513210, 'ino': 1099511628768, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/030.out'}, {'damage_type': 'backtrace', 'id': 2584442558, 'ino': 1099511628733, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/013'}, {'damage_type': 'backtrace', 'id': 2589735451, 'ino': 1099511628841, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/067'}, {'damage_type': 'backtrace', 'id': 2620759750, 'ino': 1099511628750, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/021.out'}, {'damage_type': 'backtrace', 'id': 2626197843, 'ino': 1099511628803, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/048'}, {'damage_type': 'backtrace', 'id': 2630782902, 'ino': 1099511628919, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/106'}, {'damage_type': 'backtrace', 'id': 2642592601, 'ino': 1099511628726, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/009.out'}, {'damage_type': 'backtrace', 'id': 2675663093, 'ino': 1099511628836, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/064.out'}, {'damage_type': 'backtrace', 'id': 2742370233, 'ino': 1099511628716, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/004.out'}, {'damage_type': 'backtrace', 'id': 2743800880, 'ino': 1099511628300, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/update.sample'}, {'damage_type': 'backtrace', 'id': 2752165759, 'ino': 1099511628302, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-receive.sample'}, {'damage_type': 'backtrace', 'id': 2758844820, 'ino': 1099511628842, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/067.out'}, {'damage_type': 'backtrace', 'id': 2776578948, 'ino': 1099511628723, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/008'}, {'damage_type': 'backtrace', 'id': 2796836251, 'ino': 1099511628839, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/066'}, {'damage_type': 'backtrace', 'id': 2803015036, 'ino': 1099511628918, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/105.out'}, {'damage_type': 'backtrace', 'id': 2822306540, 'ino': 1099511628851, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/072'}, {'damage_type': 'backtrace', 'id': 2830481511, 'ino': 1099511628754, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/023.out'}, {'damage_type': 'backtrace', 'id': 2833417192, 'ino': 1099511628784, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/038.out'}, {'damage_type': 'backtrace', 'id': 2857965714, 'ino': 1099511628902, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/097.out'}, {'damage_type': 'backtrace', 'id': 2886936006, 'ino': 1099511628709, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/001'}, {'damage_type': 'backtrace', 'id': 2902967193, 'ino': 1099511628860, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/076.out'}, {'damage_type': 'backtrace', 'id': 2915961100, 'ino': 1099511628295, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/push-to-checkout.sample'}, {'damage_type': 'backtrace', 'id': 2946522145, 'ino': 1099511628798, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/045.out'}, {'damage_type': 'backtrace', 'id': 3011730571, 'ino': 1099511628781, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/037'}, {'damage_type': 'backtrace', 'id': 3054119012, 'ino': 1099511628721, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/007'}, {'damage_type': 'backtrace', 'id': 3062431121, 'ino': 1099511628847, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/070'}, {'damage_type': 'backtrace', 'id': 3079711695, 'ino': 1099511628812, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/052.out'}, {'damage_type': 'backtrace', 'id': 3080605787, 'ino': 1099511628929, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/111'}, {'damage_type': 'backtrace', 'id': 3104299991, 'ino': 1099511628786, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/039.out'}, {'damage_type': 'backtrace', 'id': 3119390004, 'ino': 1099511628925, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/109'}, {'damage_type': 'backtrace', 'id': 3131771581, 'ino': 1099511628727, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/010'}, {'damage_type': 'backtrace', 'id': 3137985759, 'ino': 1099511628708, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs'}, {'damage_type': 'backtrace', 'id': 3162984427, 'ino': 1099511628915, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/104'}, {'damage_type': 'backtrace', 'id': 3170658486, 'ino': 1099511628920, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/106.out'}, {'damage_type': 'backtrace', 'id': 3223322509, 'ino': 1099511628301, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/prepare-commit-msg.sample'}, {'damage_type': 'backtrace', 'id': 3266013302, 'ino': 1099511628916, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/104.out'}, {'damage_type': 'backtrace', 'id': 3364301494, 'ino': 1099511628769, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/031'}, {'damage_type': 'backtrace', 'id': 3372466193, 'ino': 1099511628886, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/089.out'}, {'damage_type': 'backtrace', 'id': 3378114700, 'ino': 1099511628806, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/049.out'}, {'damage_type': 'backtrace', 'id': 3381337763, 'ino': 1099511628889, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/091'}, {'damage_type': 'backtrace', 'id': 3391124642, 'ino': 1099511628827, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/060'}, {'damage_type': 'backtrace', 'id': 3391992397, 'ino': 1099511628782, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/037.out'}, {'damage_type': 'backtrace', 'id': 3397387197, 'ino': 1099511628899, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/096'}, {'damage_type': 'backtrace', 'id': 3397913695, 'ino': 1099511628785, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/039'}, {'damage_type': 'backtrace', 'id': 3404999741, 'ino': 1099511628805, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/049'}, {'damage_type': 'backtrace', 'id': 3427509089, 'ino': 1099511628820, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/056.out'}, {'damage_type': 'backtrace', 'id': 3436280163, 'ino': 1099511628848, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/070.out'}, {'damage_type': 'backtrace', 'id': 3449705908, 'ino': 1099511628744, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/018.out'}, {'damage_type': 'backtrace', 'id': 3463935919, 'ino': 1099511628715, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/004'}, {'damage_type': 'backtrace', 'id': 3473012535, 'ino': 1099511628767, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/030'}, {'damage_type': 'backtrace', 'id': 3484611008, 'ino': 1099511628864, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/078.out'}, {'damage_type': 'backtrace', 'id': 3490396813, 'ino': 1099511628893, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/093'}, {'damage_type': 'backtrace', 'id': 3506123352, 'ino': 1099511628878, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/085.out'}, {'damage_type': 'backtrace', 'id': 3557807487, 'ino': 1099511628879, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/086'}, {'damage_type': 'backtrace', 'id': 3576863423, 'ino': 1099511628856, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/074.out'}, {'damage_type': 'backtrace', 'id': 3581365896, 'ino': 1099511628903, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/098'}, {'damage_type': 'backtrace', 'id': 3583268812, 'ino': 1099511628305, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-applypatch.sample'}, {'damage_type': 'backtrace', 'id': 3599387029, 'ino': 1099511628862, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/077.out'}, {'damage_type': 'backtrace', 'id': 3607091164, 'ino': 1099511628794, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/043.out'}, {'damage_type': 'backtrace', 'id': 3609951921, 'ino': 1099511628771, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/032'}, {'damage_type': 'backtrace', 'id': 3624683904, 'ino': 1099511628722, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/007.out'}, {'damage_type': 'backtrace', 'id': 3633788883, 'ino': 1099511628296, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/post-update.sample'}, {'damage_type': 'backtrace', 'id': 3645337550, 'ino': 1099511628783, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/038'}, {'damage_type': 'backtrace', 'id': 3649249384, 'ino': 1099511628807, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/050'}, {'damage_type': 'backtrace', 'id': 3650457923, 'ino': 1099511628802, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/047.out'}, {'damage_type': 'backtrace', 'id': 3677354238, 'ino': 1099511628752, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/022.out'}, {'damage_type': 'backtrace', 'id': 3690671135, 'ino': 1099511628778, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/035.out'}, {'damage_type': 'backtrace', 'id': 3724831781, 'ino': 1099511628845, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/069'}, {'damage_type': 'backtrace', 'id': 3735687669, 'ino': 1099511628756, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/024.out'}, {'damage_type': 'backtrace', 'id': 3747655472, 'ino': 1099511628788, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/040.out'}, {'damage_type': 'backtrace', 'id': 3772771023, 'ino': 1099511628828, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/060.out'}, {'damage_type': 'backtrace', 'id': 3774524065, 'ino': 1099511628905, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/099'}, {'damage_type': 'backtrace', 'id': 3774750281, 'ino': 1099511628814, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/053.out'}, {'damage_type': 'backtrace', 'id': 3779377616, 'ino': 1099511628846, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/069.out'}, {'damage_type': 'backtrace', 'id': 3806606454, 'ino': 1099511628739, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/016'}, {'damage_type': 'backtrace', 'id': 3841554550, 'ino': 1099511628299, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-merge-commit.sample'}, {'damage_type': 'backtrace', 'id': 3868459657, 'ino': 1099511628303, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/fsmonitor-watchman.sample'}, {'damage_type': 'backtrace', 'id': 3876839122, 'ino': 1099511628304, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/pre-push.sample'}, {'damage_type': 'backtrace', 'id': 3877076098, 'ino': 1099511628787, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/040'}, {'damage_type': 'backtrace', 'id': 3891383064, 'ino': 1099511628825, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/059'}, {'damage_type': 'backtrace', 'id': 3924628958, 'ino': 1099511628753, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/023'}, {'damage_type': 'backtrace', 'id': 3943199614, 'ino': 1099511628927, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/110'}, {'damage_type': 'backtrace', 'id': 3977110691, 'ino': 1099511628816, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/054.out'}, {'damage_type': 'backtrace', 'id': 4018165263, 'ino': 1099511628835, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/064'}, {'damage_type': 'backtrace', 'id': 4018192250, 'ino': 1099511628747, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/020'}, {'damage_type': 'backtrace', 'id': 4019317193, 'ino': 1099511628718, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/005.out'}, {'damage_type': 'backtrace', 'id': 4023959141, 'ino': 1099511628911, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/102'}, {'damage_type': 'backtrace', 'id': 4042103119, 'ino': 1099511628801, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/047'}, {'damage_type': 'backtrace', 'id': 4055306945, 'ino': 1099511628770, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/031.out'}, {'damage_type': 'backtrace', 'id': 4084457665, 'ino': 1099511628297, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/.git/hooks/sendemail-validate.sample'}, {'damage_type': 'backtrace', 'id': 4097662745, 'ino': 1099511628793, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/043'}, {'damage_type': 'backtrace', 'id': 4105329561, 'ino': 1099511628759, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/026'}, {'damage_type': 'backtrace', 'id': 4131234649, 'ino': 1099511628872, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/082.out'}, {'damage_type': 'backtrace', 'id': 4144745529, 'ino': 1099511628923, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/108'}, {'damage_type': 'backtrace', 'id': 4147292013, 'ino': 1099511628804, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/048.out'}, {'damage_type': 'backtrace', 'id': 4199183521, 'ino': 1099511628865, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/079'}, {'damage_type': 'backtrace', 'id': 4228933487, 'ino': 1099511628833, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/063'}, {'damage_type': 'backtrace', 'id': 4257559138, 'ino': 1099511628779, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/036'}, {'damage_type': 'backtrace', 'id': 4284210898, 'ino': 1099511628867, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/080'}, {'damage_type': 'backtrace', 'id': 4287812130, 'ino': 1099511628712, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/002.out'}, {'damage_type': 'backtrace', 'id': 4289474830, 'ino': 1099511628729, 'path': '/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/tests/btrfs/011'}] 2024-04-24T09:10:00.587 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-04-24T09:10:00.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:00 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2928867908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:10:00.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:00 smithi081 ceph-mon[30961]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-04-24T09:10:00.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:00 smithi081 ceph-mon[30961]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-04-24T09:10:00.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:00 smithi081 ceph-mon[30961]: mds.e(mds.2): Metadata damage detected 2024-04-24T09:10:00.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:00 smithi081 ceph-mon[30961]: mds.h(mds.1): Metadata damage detected 2024-04-24T09:10:00.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:00 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2928867908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:10:00.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:00 smithi130 ceph-mon[31553]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-04-24T09:10:00.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:00 smithi130 ceph-mon[31553]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-04-24T09:10:00.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:00 smithi130 ceph-mon[31553]: mds.e(mds.2): Metadata damage detected 2024-04-24T09:10:00.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:00 smithi130 ceph-mon[31553]: mds.h(mds.1): Metadata damage detected 2024-04-24T09:10:01.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:01 smithi081 ceph-mon[30961]: pgmap v91: 129 pgs: 129 active+clean; 38 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 360 KiB/s wr, 198 op/s 2024-04-24T09:10:01.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:01 smithi069 ceph-mon[28170]: pgmap v91: 129 pgs: 129 active+clean; 38 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 360 KiB/s wr, 198 op/s 2024-04-24T09:10:01.856 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:01 smithi130 ceph-mon[31553]: pgmap v91: 129 pgs: 129 active+clean; 38 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 360 KiB/s wr, 198 op/s 2024-04-24T09:10:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:03 smithi081 ceph-mon[30961]: pgmap v92: 129 pgs: 129 active+clean; 38 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 283 KiB/s wr, 166 op/s 2024-04-24T09:10:03.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:03 smithi069 ceph-mon[28170]: pgmap v92: 129 pgs: 129 active+clean; 38 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 283 KiB/s wr, 166 op/s 2024-04-24T09:10:03.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:03 smithi130 ceph-mon[31553]: pgmap v92: 129 pgs: 129 active+clean; 38 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 283 KiB/s wr, 166 op/s 2024-04-24T09:10:05.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:05 smithi081 ceph-mon[30961]: pgmap v93: 129 pgs: 129 active+clean; 38 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 283 KiB/s wr, 166 op/s 2024-04-24T09:10:05.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:05 smithi069 ceph-mon[28170]: pgmap v93: 129 pgs: 129 active+clean; 38 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 283 KiB/s wr, 166 op/s 2024-04-24T09:10:05.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:05 smithi130 ceph-mon[31553]: pgmap v93: 129 pgs: 129 active+clean; 38 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 283 KiB/s wr, 166 op/s 2024-04-24T09:10:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.723+0000 7fc6f1800640 1 -- 172.21.15.69:0/3816027699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6ec1042b0 msgr2=0x7fc6ec104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.723+0000 7fc6f1800640 1 --2- 172.21.15.69:0/3816027699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6ec1042b0 0x7fc6ec104690 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009ca0 tx=0x7fc6dc02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.723+0000 7fc6f1800640 1 -- 172.21.15.69:0/3816027699 shutdown_connections 2024-04-24T09:10:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.723+0000 7fc6f1800640 1 --2- 172.21.15.69:0/3816027699 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6ec109380 0x7fc6ec110f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.723+0000 7fc6f1800640 1 --2- 172.21.15.69:0/3816027699 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6ec104c60 0x7fc6ec108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.723+0000 7fc6f1800640 1 --2- 172.21.15.69:0/3816027699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6ec1042b0 0x7fc6ec104690 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.723+0000 7fc6f1800640 1 -- 172.21.15.69:0/3816027699 >> 172.21.15.69:0/3816027699 conn(0x7fc6ec0ffb10 msgr2=0x7fc6ec101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.723+0000 7fc6f1800640 1 -- 172.21.15.69:0/3816027699 shutdown_connections 2024-04-24T09:10:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.723+0000 7fc6f1800640 1 -- 172.21.15.69:0/3816027699 wait complete. 2024-04-24T09:10:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.724+0000 7fc6f1800640 1 Processor -- start 2024-04-24T09:10:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.725+0000 7fc6f1800640 1 -- start start 2024-04-24T09:10:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.725+0000 7fc6f1800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6ec1042b0 0x7fc6ec1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.725+0000 7fc6f1800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6ec109380 0x7fc6ec1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.725+0000 7fc6f1800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6ec19a6c0 0x7fc6ec19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.725+0000 7fc6f1800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec114360 con 0x7fc6ec1042b0 2024-04-24T09:10:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.725+0000 7fc6f1800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec1141e0 con 0x7fc6ec19a6c0 2024-04-24T09:10:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.725+0000 7fc6f1800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec1144e0 con 0x7fc6ec109380 2024-04-24T09:10:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.725+0000 7fc6ebe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6ec1042b0 0x7fc6ec1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.725+0000 7fc6ebe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6ec1042b0 0x7fc6ec1a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59734/0 (socket says 172.21.15.69:59734) 2024-04-24T09:10:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.725+0000 7fc6ebe00640 1 -- 172.21.15.69:0/2628750235 learned_addr learned my addr 172.21.15.69:0/2628750235 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.726+0000 7fc6e3400640 1 --2- 172.21.15.69:0/2628750235 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6ec109380 0x7fc6ec1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.726+0000 7fc6f0e00640 1 --2- 172.21.15.69:0/2628750235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6ec19a6c0 0x7fc6ec19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.726+0000 7fc6ebe00640 1 -- 172.21.15.69:0/2628750235 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6ec109380 msgr2=0x7fc6ec1a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.726+0000 7fc6ebe00640 1 --2- 172.21.15.69:0/2628750235 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6ec109380 0x7fc6ec1a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.726+0000 7fc6ebe00640 1 -- 172.21.15.69:0/2628750235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6ec19a6c0 msgr2=0x7fc6ec19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.726+0000 7fc6ebe00640 1 --2- 172.21.15.69:0/2628750235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6ec19a6c0 0x7fc6ec19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.726+0000 7fc6ebe00640 1 -- 172.21.15.69:0/2628750235 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ec19b2c0 con 0x7fc6ec1042b0 2024-04-24T09:10:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.726+0000 7fc6f0e00640 1 --2- 172.21.15.69:0/2628750235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6ec19a6c0 0x7fc6ec19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.726+0000 7fc6ebe00640 1 --2- 172.21.15.69:0/2628750235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6ec1042b0 0x7fc6ec1a0600 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc031040 tx=0x7fc6dc032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.726+0000 7fc6e9600640 1 -- 172.21.15.69:0/2628750235 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc003cf0 con 0x7fc6ec1042b0 2024-04-24T09:10:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.727+0000 7fc6e9600640 1 -- 172.21.15.69:0/2628750235 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6dc0044f0 con 0x7fc6ec1042b0 2024-04-24T09:10:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.727+0000 7fc6e9600640 1 -- 172.21.15.69:0/2628750235 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc030bd0 con 0x7fc6ec1042b0 2024-04-24T09:10:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.727+0000 7fc6f1800640 1 -- 172.21.15.69:0/2628750235 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ec19b550 con 0x7fc6ec1042b0 2024-04-24T09:10:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.727+0000 7fc6f1800640 1 -- 172.21.15.69:0/2628750235 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ec1a76f0 con 0x7fc6ec1042b0 2024-04-24T09:10:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.730+0000 7fc6f5dd1740 1 Processor -- start 2024-04-24T09:10:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.730+0000 7fc6f5dd1740 1 -- start start 2024-04-24T09:10:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.731+0000 7fc6e9600640 1 -- 172.21.15.69:0/2628750235 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc6dc004040 con 0x7fc6ec1042b0 2024-04-24T09:10:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.731+0000 7fc6f5dd1740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8bb3c0 0x5614bf8bb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.731+0000 7fc6f5dd1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8bbce0 0x5614bf8c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.731+0000 7fc6f5dd1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8c65a0 0x5614bf8c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.731+0000 7fc6f5dd1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5614bf821900 con 0x5614bf8c65a0 2024-04-24T09:10:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.731+0000 7fc6f5dd1740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5614bf7c7f20 con 0x5614bf8bb3c0 2024-04-24T09:10:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.731+0000 7fc6f5dd1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5614bf81d8c0 con 0x5614bf8bbce0 2024-04-24T09:10:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.731+0000 7fc6e3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8c65a0 0x5614bf8c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.731+0000 7fc6e3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8c65a0 0x5614bf8c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59748/0 (socket says 172.21.15.69:59748) 2024-04-24T09:10:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.731+0000 7fc6e3400640 1 -- 172.21.15.69:0/656424007 learned_addr learned my addr 172.21.15.69:0/656424007 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.731+0000 7fc6f0e00640 1 --2- 172.21.15.69:0/656424007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8bb3c0 0x5614bf8bb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.732+0000 7fc6ebe00640 1 --2- 172.21.15.69:0/656424007 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8bbce0 0x5614bf8c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.732+0000 7fc6e3400640 1 -- 172.21.15.69:0/656424007 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8bbce0 msgr2=0x5614bf8c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.732+0000 7fc6e3400640 1 --2- 172.21.15.69:0/656424007 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8bbce0 0x5614bf8c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.732+0000 7fc6e3400640 1 -- 172.21.15.69:0/656424007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8bb3c0 msgr2=0x5614bf8bb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.732+0000 7fc6e3400640 1 --2- 172.21.15.69:0/656424007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8bb3c0 0x5614bf8bb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.732+0000 7fc6e3400640 1 -- 172.21.15.69:0/656424007 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614bf7ece10 con 0x5614bf8c65a0 2024-04-24T09:10:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.732+0000 7fc6e3400640 1 --2- 172.21.15.69:0/656424007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8c65a0 0x5614bf8c8990 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc6d400bc50 tx=0x7fc6d400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4937b916add355b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.732+0000 7fc6e9600640 1 --2- 172.21.15.69:0/2628750235 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc6d00a3d90 0x7fc6d00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.732+0000 7fc6e1600640 1 -- 172.21.15.69:0/656424007 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4015070 con 0x5614bf8c65a0 2024-04-24T09:10:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.733+0000 7fc6e9600640 1 -- 172.21.15.69:0/2628750235 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc6dc0f2010 con 0x7fc6ec1042b0 2024-04-24T09:10:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.733+0000 7fc6e1600640 1 -- 172.21.15.69:0/656424007 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6d401a910 con 0x5614bf8c65a0 2024-04-24T09:10:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.733+0000 7fc6e1600640 1 -- 172.21.15.69:0/656424007 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4011920 con 0x5614bf8c65a0 2024-04-24T09:10:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.733+0000 7fc6f0e00640 1 --2- 172.21.15.69:0/2628750235 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc6d00a3d90 0x7fc6d00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.734+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/656424007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8c65a0 msgr2=0x5614bf8c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.734+0000 7fc6f0e00640 1 --2- 172.21.15.69:0/2628750235 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc6d00a3d90 0x7fc6d00a6250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400cc00 tx=0x7fc6e400c5e0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.734+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/656424007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8c65a0 0x5614bf8c8990 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc6d400bc50 tx=0x7fc6d400f730 comp rx=0 tx=0).stop 2024-04-24T09:10:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.734+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/656424007 shutdown_connections 2024-04-24T09:10:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.734+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/656424007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8c65a0 0x5614bf8c8990 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.734+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/656424007 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8bbce0 0x5614bf8c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.734+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/656424007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8bb3c0 0x5614bf8bb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.734+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/656424007 >> 172.21.15.69:0/656424007 conn(0x5614bf8b78d0 msgr2=0x5614bf8b7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.734+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/656424007 shutdown_connections 2024-04-24T09:10:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.734+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/656424007 wait complete. 2024-04-24T09:10:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.734+0000 7fc6f5dd1740 1 -- start start 2024-04-24T09:10:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.734+0000 7fc6f5dd1740 1 Processor -- start 2024-04-24T09:10:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.735+0000 7fc6f5dd1740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8bb3c0 0x5614bf8c3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.735+0000 7fc6f5dd1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8bbce0 0x5614bf8c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.735+0000 7fc6f5dd1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8c65a0 0x5614bf8c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.735+0000 7fc6f5dd1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5614bf8c90e0 con 0x5614bf8c65a0 2024-04-24T09:10:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.735+0000 7fc6f5dd1740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5614bf7db2c0 con 0x5614bf8bb3c0 2024-04-24T09:10:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.735+0000 7fc6f5dd1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5614bf8c9260 con 0x5614bf8bbce0 2024-04-24T09:10:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.735+0000 7fc6f5dd1740 10 client.0 _ll_register_callbacks cb 0x5614bf533a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.735+0000 7fc6f0e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8c65a0 0x5614bf8c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.735+0000 7fc6f0e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8c65a0 0x5614bf8c3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59754/0 (socket says 172.21.15.69:59754) 2024-04-24T09:10:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.735+0000 7fc6f0e00640 1 -- 172.21.15.69:0/3853829279 learned_addr learned my addr 172.21.15.69:0/3853829279 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.735+0000 7fc6ebe00640 1 --2- 172.21.15.69:0/3853829279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8bb3c0 0x5614bf8c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.736+0000 7fc6e3400640 1 --2- 172.21.15.69:0/3853829279 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8bbce0 0x5614bf8c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.736+0000 7fc6f0e00640 1 -- 172.21.15.69:0/3853829279 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8bbce0 msgr2=0x5614bf8c3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.736+0000 7fc6f0e00640 1 --2- 172.21.15.69:0/3853829279 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8bbce0 0x5614bf8c3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.736+0000 7fc6f0e00640 1 -- 172.21.15.69:0/3853829279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8bb3c0 msgr2=0x5614bf8c3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.736+0000 7fc6f0e00640 1 --2- 172.21.15.69:0/3853829279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8bb3c0 0x5614bf8c3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.736+0000 7fc6f0e00640 1 -- 172.21.15.69:0/3853829279 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614bf7ebfd0 con 0x5614bf8c65a0 2024-04-24T09:10:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.736+0000 7fc6f0e00640 1 --2- 172.21.15.69:0/3853829279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8c65a0 0x5614bf8c3fc0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4012480 tx=0x7fc6e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff617410b0b5b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.736+0000 7fc6da000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:10:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.736+0000 7fc6da000640 1 -- 172.21.15.69:0/3853829279 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4023070 con 0x5614bf8c65a0 2024-04-24T09:10:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.736+0000 7fc6da000640 1 -- 172.21.15.69:0/3853829279 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc6e401f860 con 0x5614bf8c65a0 2024-04-24T09:10:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.736+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/3853829279 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6d401a910 con 0x5614bf8c65a0 2024-04-24T09:10:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.737+0000 7fc6da000640 1 -- 172.21.15.69:0/3853829279 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6e402f050 con 0x5614bf8c65a0 2024-04-24T09:10:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.737+0000 7fc6f5dd1740 10 client.16578 fetch_fsmap learned FSMap version 29 2024-04-24T09:10:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.737+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/3853829279 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5614bf534210 con 0x5614bf8c65a0 2024-04-24T09:10:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.737+0000 7fc6da000640 1 -- 172.21.15.69:0/3853829279 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6d401a910 con 0x5614bf8c65a0 2024-04-24T09:10:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.737+0000 7fc6da000640 1 -- 172.21.15.69:0/3853829279 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fc6e400f040 con 0x5614bf8c65a0 2024-04-24T09:10:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.737+0000 7fc6f5dd1740 10 client.16578 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:10:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.737+0000 7fc6f5dd1740 10 client.16578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.737+0000 7fc6f5dd1740 20 client.16578 populate_metadata read hostname 'smithi069' 2024-04-24T09:10:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.737+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/3853829279 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5614bf8d6830 0x5614bf8d8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.737+0000 7fc6f5dd1740 4 client.16578 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.737+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/3853829279 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614bf82f320 con 0x5614bf8d6830 2024-04-24T09:10:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.738+0000 7fc6ebe00640 1 --2- 172.21.15.69:0/3853829279 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5614bf8d6830 0x5614bf8d8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.739+0000 7fc6ebe00640 1 --2- 172.21.15.69:0/3853829279 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5614bf8d6830 0x5614bf8d8cb0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=204f63eaf71485b9 server_cookie=1fd365525ff4e3a5 in_seq=0 out_seq=0 2024-04-24T09:10:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.739+0000 7fc6da000640 10 client.16578 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.742+0000 7fc6da000640 1 -- 172.21.15.69:0/3853829279 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614bf82f320 con 0x5614bf8d6830 2024-04-24T09:10:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.742+0000 7fc6da000640 10 client.16578 handle_command_reply: tid=0 2024-04-24T09:10:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.742+0000 7fc6f5dd1740 1 client.16578 shutdown 2024-04-24T09:10:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.742+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/3853829279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8c65a0 msgr2=0x5614bf8c3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.742+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/3853829279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8c65a0 0x5614bf8c3fc0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4012480 tx=0x7fc6e4004730 comp rx=0 tx=0).stop 2024-04-24T09:10:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.742+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/3853829279 shutdown_connections 2024-04-24T09:10:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.742+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/3853829279 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5614bf8d6830 0x5614bf8d8cb0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.742+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/3853829279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8c65a0 0x5614bf8c3fc0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.742+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/3853829279 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8bbce0 0x5614bf8c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.742+0000 7fc6da000640 0 client.16578 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.742+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/3853829279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8bb3c0 0x5614bf8c3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.742+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/3853829279 >> 172.21.15.69:0/3853829279 conn(0x5614bf8b78d0 msgr2=0x5614bf8bd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.742+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/3853829279 shutdown_connections 2024-04-24T09:10:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.742+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/3853829279 wait complete. 2024-04-24T09:10:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.743+0000 7fc6f5dd1740 20 client.16578 trim_cache size 0 max 16384 2024-04-24T09:10:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.759+0000 7fc6f5dd1740 1 Processor -- start 2024-04-24T09:10:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.760+0000 7fc6f5dd1740 1 -- start start 2024-04-24T09:10:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.760+0000 7fc6f5dd1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8bb3c0 0x5614bf8d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.760+0000 7fc6f5dd1740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8bbce0 0x5614bf8bdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.760+0000 7fc6f5dd1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8c65a0 0x5614bf8b6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.760+0000 7fc6f5dd1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec1b3c30 con 0x5614bf8bb3c0 2024-04-24T09:10:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.760+0000 7fc6f5dd1740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec1b1470 con 0x5614bf8bbce0 2024-04-24T09:10:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.760+0000 7fc6f5dd1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec1af2c0 con 0x5614bf8c65a0 2024-04-24T09:10:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.760+0000 7fc6ebe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8bb3c0 0x5614bf8d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.760+0000 7fc6ebe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8bb3c0 0x5614bf8d20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59766/0 (socket says 172.21.15.69:59766) 2024-04-24T09:10:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.760+0000 7fc6ebe00640 1 -- 172.21.15.69:0/45153668 learned_addr learned my addr 172.21.15.69:0/45153668 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.761+0000 7fc6e3400640 1 --2- 172.21.15.69:0/45153668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8bbce0 0x5614bf8bdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.761+0000 7fc6ebe00640 1 -- 172.21.15.69:0/45153668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8c65a0 msgr2=0x5614bf8b6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.761+0000 7fc6ebe00640 1 --2- 172.21.15.69:0/45153668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8c65a0 0x5614bf8b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.761+0000 7fc6ebe00640 1 -- 172.21.15.69:0/45153668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8bbce0 msgr2=0x5614bf8bdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.761+0000 7fc6ebe00640 1 --2- 172.21.15.69:0/45153668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8bbce0 0x5614bf8bdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.761+0000 7fc6ebe00640 1 -- 172.21.15.69:0/45153668 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614bf600620 con 0x5614bf8bb3c0 2024-04-24T09:10:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.761+0000 7fc6ebe00640 1 --2- 172.21.15.69:0/45153668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8bb3c0 0x5614bf8d20e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0bdcb0 tx=0x7fc6dc0f43e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b4b3c2263c4fff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.761+0000 7fc6e0c00640 1 -- 172.21.15.69:0/45153668 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc0bd860 con 0x5614bf8bb3c0 2024-04-24T09:10:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.762+0000 7fc6e0c00640 1 -- 172.21.15.69:0/45153668 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6dc036040 con 0x5614bf8bb3c0 2024-04-24T09:10:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.762+0000 7fc6e0c00640 1 -- 172.21.15.69:0/45153668 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc0046a0 con 0x5614bf8bb3c0 2024-04-24T09:10:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.763+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/45153668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8bb3c0 msgr2=0x5614bf8d20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.763+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/45153668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8bb3c0 0x5614bf8d20e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0bdcb0 tx=0x7fc6dc0f43e0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.763+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/45153668 shutdown_connections 2024-04-24T09:10:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.763+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/45153668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8c65a0 0x5614bf8b6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.763+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/45153668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8bbce0 0x5614bf8bdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.763+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/45153668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8bb3c0 0x5614bf8d20e0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.763+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/45153668 >> 172.21.15.69:0/45153668 conn(0x5614bf8b78d0 msgr2=0x5614bf8b6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.763+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/45153668 shutdown_connections 2024-04-24T09:10:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.763+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/45153668 wait complete. 2024-04-24T09:10:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.764+0000 7fc6f5dd1740 1 -- start start 2024-04-24T09:10:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.764+0000 7fc6f5dd1740 1 Processor -- start 2024-04-24T09:10:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.764+0000 7fc6f5dd1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8bb3c0 0x5614bf8ca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.764+0000 7fc6f5dd1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8bbce0 0x5614bf8cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.764+0000 7fc6f5dd1740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8c65a0 0x5614bf93be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.764+0000 7fc6f5dd1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec1af930 con 0x5614bf8bb3c0 2024-04-24T09:10:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.764+0000 7fc6f5dd1740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec1b0da0 con 0x5614bf8c65a0 2024-04-24T09:10:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.764+0000 7fc6f5dd1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec1b0000 con 0x5614bf8bbce0 2024-04-24T09:10:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.764+0000 7fc6f5dd1740 10 client.0 _ll_register_callbacks cb 0x7fc6e4016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.765+0000 7fc6ebe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8bb3c0 0x5614bf8ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.765+0000 7fc6ebe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8bb3c0 0x5614bf8ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59772/0 (socket says 172.21.15.69:59772) 2024-04-24T09:10:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.765+0000 7fc6ebe00640 1 -- 172.21.15.69:0/2794680128 learned_addr learned my addr 172.21.15.69:0/2794680128 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.765+0000 7fc6f0e00640 1 --2- 172.21.15.69:0/2794680128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8c65a0 0x5614bf93be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.765+0000 7fc6ebe00640 1 -- 172.21.15.69:0/2794680128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8bbce0 msgr2=0x5614bf8cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.765+0000 7fc6ebe00640 1 --2- 172.21.15.69:0/2794680128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8bbce0 0x5614bf8cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.765+0000 7fc6ebe00640 1 -- 172.21.15.69:0/2794680128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8c65a0 msgr2=0x5614bf93be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.765+0000 7fc6ebe00640 1 --2- 172.21.15.69:0/2794680128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8c65a0 0x5614bf93be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.765+0000 7fc6ebe00640 1 -- 172.21.15.69:0/2794680128 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614bf93c370 con 0x5614bf8bb3c0 2024-04-24T09:10:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.765+0000 7fc6ebe00640 1 --2- 172.21.15.69:0/2794680128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8bb3c0 0x5614bf8ca720 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0c6460 tx=0x7fc6dc004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1aada47293664 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.765+0000 7fc6da000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:10:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.765+0000 7fc6da000640 1 -- 172.21.15.69:0/2794680128 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc0d6070 con 0x5614bf8bb3c0 2024-04-24T09:10:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.766+0000 7fc6da000640 1 -- 172.21.15.69:0/2794680128 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc6dc030d70 con 0x5614bf8bb3c0 2024-04-24T09:10:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.766+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/2794680128 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6dc036040 con 0x5614bf8bb3c0 2024-04-24T09:10:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.766+0000 7fc6da000640 1 -- 172.21.15.69:0/2794680128 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6dc0bd680 con 0x5614bf8bb3c0 2024-04-24T09:10:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.766+0000 7fc6f5dd1740 10 client.16584 fetch_fsmap learned FSMap version 29 2024-04-24T09:10:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.766+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/2794680128 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5614bf93d650 con 0x5614bf8bb3c0 2024-04-24T09:10:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.766+0000 7fc6da000640 1 -- 172.21.15.69:0/2794680128 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6dc036040 con 0x5614bf8bb3c0 2024-04-24T09:10:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.766+0000 7fc6da000640 1 -- 172.21.15.69:0/2794680128 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fc6dc0ef640 con 0x5614bf8bb3c0 2024-04-24T09:10:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.766+0000 7fc6f5dd1740 10 client.16584 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:10:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.766+0000 7fc6f5dd1740 10 client.16584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.766+0000 7fc6f5dd1740 20 client.16584 populate_metadata read hostname 'smithi069' 2024-04-24T09:10:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.766+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/2794680128 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5614bf93dd10 0x5614bf940130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.766+0000 7fc6f5dd1740 4 client.16584 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:10:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.766+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/2794680128 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5614bf829190 con 0x5614bf93dd10 2024-04-24T09:10:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.767+0000 7fc6e3400640 1 --2- 172.21.15.69:0/2794680128 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5614bf93dd10 0x5614bf940130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.768+0000 7fc6e3400640 1 --2- 172.21.15.69:0/2794680128 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5614bf93dd10 0x5614bf940130 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=786eb71d77b6cace server_cookie=28a5de8d20d528c0 in_seq=0 out_seq=0 2024-04-24T09:10:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.768+0000 7fc6da000640 10 client.16584 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:07.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:07 smithi081 ceph-mon[30961]: pgmap v94: 129 pgs: 129 active+clean; 41 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 556 KiB/s wr, 212 op/s 2024-04-24T09:10:07.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:07 smithi069 ceph-mon[28170]: pgmap v94: 129 pgs: 129 active+clean; 41 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 556 KiB/s wr, 212 op/s 2024-04-24T09:10:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.889+0000 7fc6da000640 1 -- 172.21.15.69:0/2794680128 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5614bf829190 con 0x5614bf93dd10 2024-04-24T09:10:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.889+0000 7fc6da000640 10 client.16584 handle_command_reply: tid=0 2024-04-24T09:10:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.889+0000 7fc6f5dd1740 1 client.16584 shutdown 2024-04-24T09:10:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.889+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/2794680128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8bb3c0 msgr2=0x5614bf8ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.889+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/2794680128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8bb3c0 0x5614bf8ca720 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0c6460 tx=0x7fc6dc004710 comp rx=0 tx=0).stop 2024-04-24T09:10:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.890+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/2794680128 shutdown_connections 2024-04-24T09:10:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.890+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/2794680128 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5614bf93dd10 0x5614bf940130 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.890+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/2794680128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614bf8c65a0 0x5614bf93be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.890+0000 7fc6da000640 0 client.16584 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.890+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/2794680128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5614bf8bbce0 0x5614bf8cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.890+0000 7fc6f5dd1740 1 --2- 172.21.15.69:0/2794680128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5614bf8bb3c0 0x5614bf8ca720 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.890+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/2794680128 >> 172.21.15.69:0/2794680128 conn(0x5614bf8b78d0 msgr2=0x5614bf8b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.890+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/2794680128 shutdown_connections 2024-04-24T09:10:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.890+0000 7fc6f5dd1740 1 -- 172.21.15.69:0/2794680128 wait complete. 2024-04-24T09:10:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.890+0000 7fc6f5dd1740 20 client.16584 trim_cache size 0 max 16384 2024-04-24T09:10:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:10:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:10:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:10:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:10:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.891+0000 7fc6f1800640 1 -- 172.21.15.69:0/2628750235 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc6d00a3d90 msgr2=0x7fc6d00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.891+0000 7fc6f1800640 1 --2- 172.21.15.69:0/2628750235 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc6d00a3d90 0x7fc6d00a6250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400cc00 tx=0x7fc6e400c5e0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.891+0000 7fc6f1800640 1 -- 172.21.15.69:0/2628750235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6ec1042b0 msgr2=0x7fc6ec1a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.891+0000 7fc6f1800640 1 --2- 172.21.15.69:0/2628750235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6ec1042b0 0x7fc6ec1a0600 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc031040 tx=0x7fc6dc032ff0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.891+0000 7fc6f1800640 1 -- 172.21.15.69:0/2628750235 shutdown_connections 2024-04-24T09:10:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.891+0000 7fc6f1800640 1 --2- 172.21.15.69:0/2628750235 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc6d00a3d90 0x7fc6d00a6250 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.891+0000 7fc6f1800640 1 --2- 172.21.15.69:0/2628750235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6ec19a6c0 0x7fc6ec19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.891+0000 7fc6f1800640 1 --2- 172.21.15.69:0/2628750235 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6ec109380 0x7fc6ec1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.891+0000 7fc6f1800640 1 --2- 172.21.15.69:0/2628750235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6ec1042b0 0x7fc6ec1a0600 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.891+0000 7fc6f1800640 1 -- 172.21.15.69:0/2628750235 >> 172.21.15.69:0/2628750235 conn(0x7fc6ec0ffb10 msgr2=0x7fc6ec101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.891+0000 7fc6f1800640 1 -- 172.21.15.69:0/2628750235 shutdown_connections 2024-04-24T09:10:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:07.891+0000 7fc6f1800640 1 -- 172.21.15.69:0/2628750235 wait complete. 2024-04-24T09:10:07.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:07 smithi130 ceph-mon[31553]: pgmap v94: 129 pgs: 129 active+clean; 41 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 556 KiB/s wr, 212 op/s 2024-04-24T09:10:09.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:09 smithi081 ceph-mon[30961]: pgmap v95: 129 pgs: 129 active+clean; 41 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 429 KiB/s wr, 87 op/s 2024-04-24T09:10:09.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:09 smithi069 ceph-mon[28170]: pgmap v95: 129 pgs: 129 active+clean; 41 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 429 KiB/s wr, 87 op/s 2024-04-24T09:10:09.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:09 smithi130 ceph-mon[31553]: pgmap v95: 129 pgs: 129 active+clean; 41 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 429 KiB/s wr, 87 op/s 2024-04-24T09:10:11.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:11 smithi081 ceph-mon[30961]: pgmap v96: 129 pgs: 129 active+clean; 44 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 671 KiB/s wr, 153 op/s 2024-04-24T09:10:11.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:11 smithi069 ceph-mon[28170]: pgmap v96: 129 pgs: 129 active+clean; 44 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 671 KiB/s wr, 153 op/s 2024-04-24T09:10:11.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:11 smithi130 ceph-mon[31553]: pgmap v96: 129 pgs: 129 active+clean; 44 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 671 KiB/s wr, 153 op/s 2024-04-24T09:10:13.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:13 smithi081 ceph-mon[30961]: pgmap v97: 129 pgs: 129 active+clean; 44 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 515 KiB/s wr, 111 op/s 2024-04-24T09:10:13.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:13 smithi069 ceph-mon[28170]: pgmap v97: 129 pgs: 129 active+clean; 44 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 515 KiB/s wr, 111 op/s 2024-04-24T09:10:13.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:13 smithi130 ceph-mon[31553]: pgmap v97: 129 pgs: 129 active+clean; 44 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 515 KiB/s wr, 111 op/s 2024-04-24T09:10:15.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:15 smithi081 ceph-mon[30961]: pgmap v98: 129 pgs: 129 active+clean; 44 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 515 KiB/s wr, 111 op/s 2024-04-24T09:10:15.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:15 smithi069 ceph-mon[28170]: pgmap v98: 129 pgs: 129 active+clean; 44 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 515 KiB/s wr, 111 op/s 2024-04-24T09:10:15.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:15 smithi130 ceph-mon[31553]: pgmap v98: 129 pgs: 129 active+clean; 44 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 515 KiB/s wr, 111 op/s 2024-04-24T09:10:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:17 smithi081 ceph-mon[30961]: pgmap v99: 129 pgs: 129 active+clean; 46 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 736 KiB/s wr, 140 op/s 2024-04-24T09:10:17.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:17 smithi069 ceph-mon[28170]: pgmap v99: 129 pgs: 129 active+clean; 46 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 736 KiB/s wr, 140 op/s 2024-04-24T09:10:17.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:17 smithi130 ceph-mon[31553]: pgmap v99: 129 pgs: 129 active+clean; 46 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 736 KiB/s wr, 140 op/s 2024-04-24T09:10:18.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:18 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:10:18.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:18 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:10:19.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:18 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:10:19.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:19 smithi069 ceph-mon[28170]: pgmap v100: 129 pgs: 129 active+clean; 46 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 462 KiB/s wr, 94 op/s 2024-04-24T09:10:19.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:19 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:10:19.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:19 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:10:19.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:19 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:10:19.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:19 smithi130 ceph-mon[31553]: pgmap v100: 129 pgs: 129 active+clean; 46 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 462 KiB/s wr, 94 op/s 2024-04-24T09:10:19.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:19 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:10:19.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:19 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:10:19.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:19 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:10:20.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:19 smithi081 ceph-mon[30961]: pgmap v100: 129 pgs: 129 active+clean; 46 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 462 KiB/s wr, 94 op/s 2024-04-24T09:10:20.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:19 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:10:20.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:19 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:10:20.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:19 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:10:20.608 INFO:tasks.workunit.client.0.smithi069.stderr:Updating files: 2% (99/4290) Updating files: 3% (129/4290) Updating files: 4% (172/4290) Updating files: 5% (215/4290) Updating files: 5% (225/4290) Updating files: 6% (258/4290) Updating files: 7% (301/4290) Updating files: 7% (320/4290) Updating files: 8% (344/4290) Updating files: 8% (347/4290) Updating files: 9% (387/4290) Updating files: 10% (429/4290) Updating files: 10% (439/4290) Updating files: 11% (472/4290) Updating files: 11% (513/4290) Updating files: 12% (515/4290) Updating files: 13% (558/4290) Updating files: 13% (561/4290) Updating files: 14% (601/4290) Updating files: 14% (632/4290) Updating files: 15% (644/4290) Updating files: 15% (679/4290) Updating files: 16% (687/4290) Updating files: 16% (729/4290) Updating files: 17% (730/4290) Updating files: 18% (773/4290) Updating files: 18% (776/4290) Updating files: 19% (816/4290) Updating files: 19% (826/4290) Updating files: 20% (858/4290) Updating files: 20% (875/4290) Updating files: 21% (901/4290) Updating files: 21% (921/4290) Updating files: 22% (944/4290) Updating files: 22% (972/4290) Updating files: 23% (987/4290) Updating files: 24% (1030/4290) Updating files: 25% (1073/4290) Updating files: 26% (1116/4290) Updating files: 26% (1117/4290) Updating files: 27% (1159/4290) Updating files: 27% (1197/4290) Updating files: 28% (1202/4290) Updating files: 29% (1245/4290) Updating files: 30% (1287/4290) Updating files: 30% (1288/4290) Updating files: 31% (1330/4290) Updating files: 31% (1358/4290) Updating files: 32% (1373/4290) Updating files: 33% (1416/4290) Updating files: 33% (1457/4290) Updating files: 34% (1459/4290) Updating files: 35% (1502/4290) Updating files: 35% (1539/4290) Updating files: 36% (1545/4290) Updating files: 37% (1588/4290) Updating files: 38% (1631/4290) Updating files: 38% (1666/4290) Updating files: 39% (1674/4290) Updating files: 40% (1716/4290) Updating files: 41% (1759/4290) Updating files: 41% (1778/4290) Updating files: 42% (1802/4290) Updating files: 43% (1845/4290) Updating files: 43% (1865/4290) Updating files: 44% (1888/4290) Updating files: 45% (1931/4290) Updating files: 45% (1946/4290) Updating files: 46% (1974/4290) Updating files: 47% (2017/4290) Updating files: 47% (2028/4290) Updating files: 48% (2060/4290) Updating files: 49% (2103/4290) Updating files: 49% (2104/4290) Updating files: 50% (2145/4290) Updating files: 50% (2184/4290) Updating files: 51% (2188/4290) Updating files: 52% (2231/4290) Updating files: 52% (2247/4290) Updating files: 53% (2274/4290) Updating files: 54% (2317/4290) Updating files: 54% (2331/4290) Updating files: 55% (2360/4290) Updating files: 56% (2403/4290) Updating files: 56% (2410/4290) Updating files: 57% (2446/4290) Updating files: 58% (2489/4290) Updating files: 58% (2521/4290) Updating files: 59% (2532/4290) Updating files: 60% (2574/4290) Updating files: 60% (2581/4290) Updating files: 61% (2617/4290) Updating files: 61% (2623/4290) Updating files: 62% (2660/4290) Updating files: 63% (2703/4290) Updating files: 63% (2704/4290) Updating files: 64% (2746/4290) Updating files: 64% (2784/4290) Updating files: 65% (2789/4290) Updating files: 66% (2832/4290) Updating files: 66% (2845/4290) Updating files: 67% (2875/4290) Updating files: 67% (2916/4290) Updating files: 68% (2918/4290) Updating files: 69% (2961/4290) Updating files: 70% (3003/4290) Updating files: 70% (3006/4290) Updating files: 71% (3046/4290) Updating files: 71% (3073/4290) Updating files: 72% (3089/4290) Updating files: 73% (3132/4290) Updating files: 73% (3155/4290) Updating files: 74% (3175/4290) Updating files: 75% (3218/4290) Updating files: 75% (3236/4290) Updating files: 76% (3261/4290) Updating files: 76% (3286/4290) Updating files: 77% (3304/4290) Updating files: 77% (3313/4290) Updating files: 77% (3341/4290) Updating files: 78% (3347/4290) Updating files: 78% (3365/4290) Updating files: 79% (3390/4290) Updating files: 79% (3396/4290) Updating files: 79% (3423/4290) Updating files: 80% (3432/4290) Updating files: 80% (3452/4290) Updating files: 81% (3475/4290) Updating files: 81% (3480/4290) Updating files: 81% (3509/4290) Updating files: 82% (3518/4290) Updating files: 82% (3533/4290) Updating files: 83% (3561/4290) Updating files: 83% (3563/4290) Updating files: 83% (3592/4290) Updating files: 84% (3604/4290) Updating files: 84% (3620/4290) Updating files: 85% (3647/4290) Updating files: 85% (3649/4290) Updating files: 85% (3677/4290) Updating files: 86% (3690/4290) Updating files: 86% (3699/4290) Updating files: 86% (3724/4290) Updating files: 87% (3733/4290) Updating files: 87% (3754/4290) Updating files: 88% (3776/4290) Updating files: 88% (3780/4290) Updating files: 88% (3804/4290) Updating files: 89% (3819/4290) Updating files: 89% (3827/4290) Updating files: 89% (3849/4290) Updating files: 90% (3861/4290) Updating files: 90% (3872/4290) Updating files: 90% (3889/4290) Updating files: 91% (3904/4290) Updating files: 91% (3909/4290) Updating files: 91% (3927/4290) Updating files: 91% (3945/4290) Updating files: 92% (3947/4290) Updating files: 92% (3964/4290) Updating files: 92% (3984/4290) Updating files: 93% (3990/4290) Updating files: 93% (4005/4290) Updating files: 93% (4027/4290) Updating files: 94% (4033/4290) Updating files: 94% (4046/4290) Updating files: 94% (4066/4290) Updating files: 95% (4076/4290) Updating files: 95% (4086/4290) Updating files: 95% (4106/4290) Updating files: 96% (4119/4290) Updating files: 96% (4124/4290) Updating files: 96% (4142/4290) Updating files: 96% (4160/4290) Updating files: 97% (4162/4290) Updating files: 97% (4179/4290) Updating files: 97% (4199/4290) Updating files: 98% (4205/4290) Updating files: 98% (4221/4290) Updating files: 99% (4248/4290) Updating files: 99% (4270/4290) Updating files: 100% (4290/4290) Updating files: 100% (4290/4290), done. 2024-04-24T09:10:20.721 INFO:tasks.workunit.client.0.smithi069.stderr:+ cd xfstests-dev 2024-04-24T09:10:20.727 INFO:tasks.workunit.client.0.smithi069.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2024-04-24T09:10:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.040+0000 7f644ca00640 1 -- 172.21.15.69:0/3345972568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6448100f60 msgr2=0x7f6448101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.040+0000 7f644ca00640 1 --2- 172.21.15.69:0/3345972568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6448100f60 0x7f6448101340 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6438009900 tx=0x7f643802a410 comp rx=0 tx=0).stop 2024-04-24T09:10:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.041+0000 7f644ca00640 1 -- 172.21.15.69:0/3345972568 shutdown_connections 2024-04-24T09:10:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.041+0000 7f644ca00640 1 --2- 172.21.15.69:0/3345972568 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6448106030 0x7f644810dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.041+0000 7f644ca00640 1 --2- 172.21.15.69:0/3345972568 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6448101910 0x7f6448105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.041+0000 7f644ca00640 1 --2- 172.21.15.69:0/3345972568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6448100f60 0x7f6448101340 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.041+0000 7f644ca00640 1 -- 172.21.15.69:0/3345972568 >> 172.21.15.69:0/3345972568 conn(0x7f64480fc820 msgr2=0x7f64480fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.041+0000 7f644ca00640 1 -- 172.21.15.69:0/3345972568 shutdown_connections 2024-04-24T09:10:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.041+0000 7f644ca00640 1 -- 172.21.15.69:0/3345972568 wait complete. 2024-04-24T09:10:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.042+0000 7f644ca00640 1 Processor -- start 2024-04-24T09:10:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.042+0000 7f644ca00640 1 -- start start 2024-04-24T09:10:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.043+0000 7f644ca00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6448100f60 0x7f644810e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.043+0000 7f644ca00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6448106030 0x7f644810ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.043+0000 7f644ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f644810f010 0x7f64481a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.043+0000 7f644ca00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6448112160 con 0x7f644810f010 2024-04-24T09:10:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.043+0000 7f644ca00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6448111fe0 con 0x7f6448100f60 2024-04-24T09:10:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.043+0000 7f644ca00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f64481122e0 con 0x7f6448106030 2024-04-24T09:10:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.043+0000 7f6447200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f644810f010 0x7f64481a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.043+0000 7f6447200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f644810f010 0x7f64481a68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51192/0 (socket says 172.21.15.69:51192) 2024-04-24T09:10:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.043+0000 7f6447200640 1 -- 172.21.15.69:0/3542965576 learned_addr learned my addr 172.21.15.69:0/3542965576 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.044+0000 7f6446800640 1 --2- 172.21.15.69:0/3542965576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6448100f60 0x7f644810e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.044+0000 7f643fe00640 1 --2- 172.21.15.69:0/3542965576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6448106030 0x7f644810ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.044+0000 7f6447200640 1 -- 172.21.15.69:0/3542965576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6448106030 msgr2=0x7f644810ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.044+0000 7f6447200640 1 --2- 172.21.15.69:0/3542965576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6448106030 0x7f644810ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.044+0000 7f6447200640 1 -- 172.21.15.69:0/3542965576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6448100f60 msgr2=0x7f644810e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.044+0000 7f6447200640 1 --2- 172.21.15.69:0/3542965576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6448100f60 0x7f644810e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.044+0000 7f6447200640 1 -- 172.21.15.69:0/3542965576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64481a6fb0 con 0x7f644810f010 2024-04-24T09:10:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.045+0000 7f6447200640 1 --2- 172.21.15.69:0/3542965576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f644810f010 0x7f64481a68b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f644000b970 tx=0x7f644000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.045+0000 7f643f400640 1 -- 172.21.15.69:0/3542965576 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440019070 con 0x7f644810f010 2024-04-24T09:10:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.045+0000 7f643f400640 1 -- 172.21.15.69:0/3542965576 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6440014440 con 0x7f644810f010 2024-04-24T09:10:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.045+0000 7f643f400640 1 -- 172.21.15.69:0/3542965576 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64400075d0 con 0x7f644810f010 2024-04-24T09:10:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.045+0000 7f644ca00640 1 -- 172.21.15.69:0/3542965576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64481a72a0 con 0x7f644810f010 2024-04-24T09:10:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.045+0000 7f644ca00640 1 -- 172.21.15.69:0/3542965576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64481a77a0 con 0x7f644810f010 2024-04-24T09:10:21.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.050+0000 7f64503ae740 1 Processor -- start 2024-04-24T09:10:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.051+0000 7f643f400640 1 -- 172.21.15.69:0/3542965576 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f644001c070 con 0x7f644810f010 2024-04-24T09:10:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.052+0000 7f64503ae740 1 -- start start 2024-04-24T09:10:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.052+0000 7f64503ae740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b2923c0 0x55c70b2927a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.052+0000 7f64503ae740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b292ce0 0x55c70b29d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.052+0000 7f64503ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b29d5a0 0x55c70b29f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.052+0000 7f64503ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c70b1f8900 con 0x55c70b29d5a0 2024-04-24T09:10:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.052+0000 7f64503ae740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55c70b19ef20 con 0x55c70b2923c0 2024-04-24T09:10:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.052+0000 7f64503ae740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c70b1f48c0 con 0x55c70b292ce0 2024-04-24T09:10:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.053+0000 7f643fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b29d5a0 0x55c70b29f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.053+0000 7f643fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b29d5a0 0x55c70b29f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51194/0 (socket says 172.21.15.69:51194) 2024-04-24T09:10:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.053+0000 7f643fe00640 1 -- 172.21.15.69:0/1824305296 learned_addr learned my addr 172.21.15.69:0/1824305296 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.053+0000 7f643fe00640 1 --2- 172.21.15.69:0/1824305296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b2923c0 0x55c70b2927a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.053+0000 7f6446800640 1 --2- 172.21.15.69:0/1824305296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b292ce0 0x55c70b29d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.053+0000 7f643f400640 1 --2- 172.21.15.69:0/3542965576 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f642c0a3d80 0x7f642c0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.053+0000 7f643fe00640 1 -- 172.21.15.69:0/1824305296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b292ce0 msgr2=0x55c70b29d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.053+0000 7f643fe00640 1 --2- 172.21.15.69:0/1824305296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b292ce0 0x55c70b29d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.053+0000 7f643fe00640 1 -- 172.21.15.69:0/1824305296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b2923c0 msgr2=0x55c70b2927a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.053+0000 7f643fe00640 1 --2- 172.21.15.69:0/1824305296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b2923c0 0x55c70b2927a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.053+0000 7f643fe00640 1 -- 172.21.15.69:0/1824305296 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c70b1c3e10 con 0x55c70b29d5a0 2024-04-24T09:10:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.054+0000 7f643f400640 1 -- 172.21.15.69:0/3542965576 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f64400cff80 con 0x7f644810f010 2024-04-24T09:10:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.054+0000 7f6447200640 1 --2- 172.21.15.69:0/3542965576 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f642c0a3d80 0x7f642c0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.054+0000 7f643fe00640 1 --2- 172.21.15.69:0/1824305296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b29d5a0 0x55c70b29f990 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f643000bf00 tx=0x7f6430004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4374d7804a112aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.054+0000 7f6437400640 1 -- 172.21.15.69:0/1824305296 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430004030 con 0x55c70b29d5a0 2024-04-24T09:10:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.054+0000 7f6437400640 1 -- 172.21.15.69:0/1824305296 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6430005020 con 0x55c70b29d5a0 2024-04-24T09:10:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.054+0000 7f6437400640 1 -- 172.21.15.69:0/1824305296 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430004ca0 con 0x55c70b29d5a0 2024-04-24T09:10:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.055+0000 7f64503ae740 1 -- 172.21.15.69:0/1824305296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b29d5a0 msgr2=0x55c70b29f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.055+0000 7f64503ae740 1 --2- 172.21.15.69:0/1824305296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b29d5a0 0x55c70b29f990 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f643000bf00 tx=0x7f6430004730 comp rx=0 tx=0).stop 2024-04-24T09:10:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.055+0000 7f6447200640 1 --2- 172.21.15.69:0/3542965576 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f642c0a3d80 0x7f642c0a6240 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6440000f80 tx=0x7f6440004a40 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.056+0000 7f64503ae740 1 -- 172.21.15.69:0/1824305296 shutdown_connections 2024-04-24T09:10:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.056+0000 7f64503ae740 1 --2- 172.21.15.69:0/1824305296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b29d5a0 0x55c70b29f990 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.056+0000 7f64503ae740 1 --2- 172.21.15.69:0/1824305296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b292ce0 0x55c70b29d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.056+0000 7f64503ae740 1 --2- 172.21.15.69:0/1824305296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b2923c0 0x55c70b2927a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.056+0000 7f64503ae740 1 -- 172.21.15.69:0/1824305296 >> 172.21.15.69:0/1824305296 conn(0x55c70b28e8d0 msgr2=0x55c70b28ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.056+0000 7f64503ae740 1 -- 172.21.15.69:0/1824305296 shutdown_connections 2024-04-24T09:10:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.056+0000 7f64503ae740 1 -- 172.21.15.69:0/1824305296 wait complete. 2024-04-24T09:10:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.056+0000 7f64503ae740 1 -- start start 2024-04-24T09:10:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.056+0000 7f64503ae740 1 Processor -- start 2024-04-24T09:10:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.057+0000 7f64503ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b2923c0 0x55c70b29a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.057+0000 7f64503ae740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b292ce0 0x55c70b29aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.057+0000 7f64503ae740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b29d5a0 0x55c70b29b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.058+0000 7f64503ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c70b2a00e0 con 0x55c70b2923c0 2024-04-24T09:10:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.058+0000 7f643fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b2923c0 0x55c70b29a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.058+0000 7f643fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b2923c0 0x55c70b29a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51196/0 (socket says 172.21.15.69:51196) 2024-04-24T09:10:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.058+0000 7f643fe00640 1 -- 172.21.15.69:0/3553069505 learned_addr learned my addr 172.21.15.69:0/3553069505 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.058+0000 7f64503ae740 1 -- 172.21.15.69:0/3553069505 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55c70b1b22c0 con 0x55c70b292ce0 2024-04-24T09:10:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.058+0000 7f64503ae740 1 -- 172.21.15.69:0/3553069505 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c70b2a0260 con 0x55c70b29d5a0 2024-04-24T09:10:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.058+0000 7f6446800640 1 --2- 172.21.15.69:0/3553069505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b292ce0 0x55c70b29aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.058+0000 7f643fe00640 1 --2- 172.21.15.69:0/3553069505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b29d5a0 0x55c70b29b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.058+0000 7f64503ae740 10 client.0 _ll_register_callbacks cb 0x55c70af0aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.059+0000 7f643fe00640 1 -- 172.21.15.69:0/3553069505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b29d5a0 msgr2=0x55c70b29b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.059+0000 7f643fe00640 1 --2- 172.21.15.69:0/3553069505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b29d5a0 0x55c70b29b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.059+0000 7f643fe00640 1 -- 172.21.15.69:0/3553069505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b292ce0 msgr2=0x55c70b29aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.059+0000 7f643fe00640 1 --2- 172.21.15.69:0/3553069505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b292ce0 0x55c70b29aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.059+0000 7f643fe00640 1 -- 172.21.15.69:0/3553069505 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70b1c2fd0 con 0x55c70b2923c0 2024-04-24T09:10:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.059+0000 7f643fe00640 1 --2- 172.21.15.69:0/3553069505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b2923c0 0x55c70b29a660 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x55c70b28f710 tx=0x7f6430001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f1109b7131be0e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.059+0000 7f642be00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:10:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.059+0000 7f642be00640 1 -- 172.21.15.69:0/3553069505 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430004230 con 0x55c70b2923c0 2024-04-24T09:10:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.059+0000 7f642be00640 1 -- 172.21.15.69:0/3553069505 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f64300043d0 con 0x55c70b2923c0 2024-04-24T09:10:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.060+0000 7f64503ae740 1 -- 172.21.15.69:0/3553069505 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6430005020 con 0x55c70b2923c0 2024-04-24T09:10:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.060+0000 7f642be00640 1 -- 172.21.15.69:0/3553069505 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6430004920 con 0x55c70b2923c0 2024-04-24T09:10:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.060+0000 7f64503ae740 10 client.16596 fetch_fsmap learned FSMap version 29 2024-04-24T09:10:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.060+0000 7f64503ae740 1 -- 172.21.15.69:0/3553069505 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c70af0b210 con 0x55c70b2923c0 2024-04-24T09:10:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.060+0000 7f642be00640 1 -- 172.21.15.69:0/3553069505 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6430005020 con 0x55c70b2923c0 2024-04-24T09:10:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.061+0000 7f642be00640 1 -- 172.21.15.69:0/3553069505 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f6430012450 con 0x55c70b2923c0 2024-04-24T09:10:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.061+0000 7f64503ae740 10 client.16596 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:10:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.061+0000 7f64503ae740 10 client.16596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.061+0000 7f64503ae740 20 client.16596 populate_metadata read hostname 'smithi069' 2024-04-24T09:10:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.061+0000 7f64503ae740 1 --2- 172.21.15.69:0/3553069505 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c70b2b5f10 0x55c70b2b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.062+0000 7f64503ae740 4 client.16596 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.062+0000 7f64503ae740 1 -- 172.21.15.69:0/3553069505 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c70b206320 con 0x55c70b2b5f10 2024-04-24T09:10:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.062+0000 7f6446800640 1 --2- 172.21.15.69:0/3553069505 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c70b2b5f10 0x55c70b2b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.063+0000 7f642be00640 10 client.16596 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.063+0000 7f6446800640 1 --2- 172.21.15.69:0/3553069505 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c70b2b5f10 0x55c70b2b8300 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=350df500fa690f02 server_cookie=1c59c94fa3f87fdc in_seq=0 out_seq=0 2024-04-24T09:10:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.066+0000 7f642be00640 1 -- 172.21.15.69:0/3553069505 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c70b206320 con 0x55c70b2b5f10 2024-04-24T09:10:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.066+0000 7f642be00640 10 client.16596 handle_command_reply: tid=0 2024-04-24T09:10:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.066+0000 7f64503ae740 1 client.16596 shutdown 2024-04-24T09:10:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.066+0000 7f64503ae740 1 -- 172.21.15.69:0/3553069505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b2923c0 msgr2=0x55c70b29a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.066+0000 7f64503ae740 1 --2- 172.21.15.69:0/3553069505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b2923c0 0x55c70b29a660 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x55c70b28f710 tx=0x7f6430001e50 comp rx=0 tx=0).stop 2024-04-24T09:10:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.066+0000 7f64503ae740 1 -- 172.21.15.69:0/3553069505 shutdown_connections 2024-04-24T09:10:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.066+0000 7f64503ae740 1 --2- 172.21.15.69:0/3553069505 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c70b2b5f10 0x55c70b2b8300 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.066+0000 7f64503ae740 1 --2- 172.21.15.69:0/3553069505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b29d5a0 0x55c70b29b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.066+0000 7f64503ae740 1 --2- 172.21.15.69:0/3553069505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b292ce0 0x55c70b29aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.066+0000 7f64503ae740 1 --2- 172.21.15.69:0/3553069505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b2923c0 0x55c70b29a660 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.066+0000 7f642be00640 0 client.16596 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.066+0000 7f64503ae740 1 -- 172.21.15.69:0/3553069505 >> 172.21.15.69:0/3553069505 conn(0x55c70b28e8d0 msgr2=0x55c70b294600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.066+0000 7f64503ae740 1 -- 172.21.15.69:0/3553069505 shutdown_connections 2024-04-24T09:10:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.066+0000 7f64503ae740 1 -- 172.21.15.69:0/3553069505 wait complete. 2024-04-24T09:10:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.066+0000 7f64503ae740 20 client.16596 trim_cache size 0 max 16384 2024-04-24T09:10:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.091+0000 7f64503ae740 1 Processor -- start 2024-04-24T09:10:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.091+0000 7f64503ae740 1 -- start start 2024-04-24T09:10:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.091+0000 7f64503ae740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b2923c0 0x55c70b29cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.091+0000 7f64503ae740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b292ce0 0x55c70b294d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.091+0000 7f64503ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b29d5a0 0x55c70b2bb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.091+0000 7f64503ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f64481af6d0 con 0x55c70b29d5a0 2024-04-24T09:10:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.091+0000 7f64503ae740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f64481acf10 con 0x55c70b292ce0 2024-04-24T09:10:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.091+0000 7f64503ae740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f64481aae10 con 0x55c70b2923c0 2024-04-24T09:10:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.092+0000 7f643fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b29d5a0 0x55c70b2bb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.092+0000 7f6446800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b292ce0 0x55c70b294d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.092+0000 7f643fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b29d5a0 0x55c70b2bb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51204/0 (socket says 172.21.15.69:51204) 2024-04-24T09:10:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.092+0000 7f643fe00640 1 -- 172.21.15.69:0/1755857731 learned_addr learned my addr 172.21.15.69:0/1755857731 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.092+0000 7f643fe00640 1 --2- 172.21.15.69:0/1755857731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b2923c0 0x55c70b29cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.092+0000 7f643fe00640 1 -- 172.21.15.69:0/1755857731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b2923c0 msgr2=0x55c70b29cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.092+0000 7f643fe00640 1 --2- 172.21.15.69:0/1755857731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b2923c0 0x55c70b29cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.092+0000 7f643fe00640 1 -- 172.21.15.69:0/1755857731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b292ce0 msgr2=0x55c70b294d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.092+0000 7f643fe00640 1 --2- 172.21.15.69:0/1755857731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b292ce0 0x55c70b294d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.092+0000 7f643fe00640 1 -- 172.21.15.69:0/1755857731 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c70afd7620 con 0x55c70b29d5a0 2024-04-24T09:10:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.093+0000 7f643fe00640 1 --2- 172.21.15.69:0/1755857731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b29d5a0 0x55c70b2bb320 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6430006010 tx=0x7f64300047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fe0c7f11d804ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.093+0000 7f6436a00640 1 -- 172.21.15.69:0/1755857731 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430012450 con 0x55c70b29d5a0 2024-04-24T09:10:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.093+0000 7f6436a00640 1 -- 172.21.15.69:0/1755857731 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64300044e0 con 0x55c70b29d5a0 2024-04-24T09:10:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.093+0000 7f6436a00640 1 -- 172.21.15.69:0/1755857731 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430004ac0 con 0x55c70b29d5a0 2024-04-24T09:10:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.094+0000 7f64503ae740 1 -- 172.21.15.69:0/1755857731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b29d5a0 msgr2=0x55c70b2bb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.094+0000 7f64503ae740 1 --2- 172.21.15.69:0/1755857731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b29d5a0 0x55c70b2bb320 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6430006010 tx=0x7f64300047a0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.095+0000 7f64503ae740 1 -- 172.21.15.69:0/1755857731 shutdown_connections 2024-04-24T09:10:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.095+0000 7f64503ae740 1 --2- 172.21.15.69:0/1755857731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b29d5a0 0x55c70b2bb320 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.095+0000 7f64503ae740 1 --2- 172.21.15.69:0/1755857731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b292ce0 0x55c70b294d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.095+0000 7f64503ae740 1 --2- 172.21.15.69:0/1755857731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b2923c0 0x55c70b29cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.095+0000 7f64503ae740 1 -- 172.21.15.69:0/1755857731 >> 172.21.15.69:0/1755857731 conn(0x55c70b28e8d0 msgr2=0x55c70b2b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.095+0000 7f64503ae740 1 -- 172.21.15.69:0/1755857731 shutdown_connections 2024-04-24T09:10:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.095+0000 7f64503ae740 1 -- 172.21.15.69:0/1755857731 wait complete. 2024-04-24T09:10:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.096+0000 7f64503ae740 1 -- start start 2024-04-24T09:10:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.096+0000 7f64503ae740 1 Processor -- start 2024-04-24T09:10:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.096+0000 7f64503ae740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b2923c0 0x55c70b2b8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.096+0000 7f64503ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b292ce0 0x55c70b2b92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.096+0000 7f64503ae740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b29d5a0 0x55c70b2b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.096+0000 7f64503ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f64481ab3d0 con 0x55c70b292ce0 2024-04-24T09:10:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.096+0000 7f64503ae740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f64481ac840 con 0x55c70b2923c0 2024-04-24T09:10:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.096+0000 7f64503ae740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f64481abaa0 con 0x55c70b29d5a0 2024-04-24T09:10:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.096+0000 7f64503ae740 10 client.0 _ll_register_callbacks cb 0x7f6438009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.097+0000 7f6446800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b292ce0 0x55c70b2b92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.097+0000 7f6446800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b292ce0 0x55c70b2b92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51210/0 (socket says 172.21.15.69:51210) 2024-04-24T09:10:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.097+0000 7f6446800640 1 -- 172.21.15.69:0/2682714832 learned_addr learned my addr 172.21.15.69:0/2682714832 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.097+0000 7f643fe00640 1 --2- 172.21.15.69:0/2682714832 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b2923c0 0x55c70b2b8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.097+0000 7f6446800640 1 -- 172.21.15.69:0/2682714832 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b29d5a0 msgr2=0x55c70b2b97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.097+0000 7f6446800640 1 --2- 172.21.15.69:0/2682714832 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b29d5a0 0x55c70b2b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.097+0000 7f6446800640 1 -- 172.21.15.69:0/2682714832 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b2923c0 msgr2=0x55c70b2b8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.097+0000 7f6446800640 1 --2- 172.21.15.69:0/2682714832 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b2923c0 0x55c70b2b8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.097+0000 7f6446800640 1 -- 172.21.15.69:0/2682714832 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70b31b0f0 con 0x55c70b292ce0 2024-04-24T09:10:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.098+0000 7f6446800640 1 --2- 172.21.15.69:0/2682714832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b292ce0 0x55c70b2b92b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f64380096c0 tx=0x7f6438005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e475ea0260d8a8c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.098+0000 7f642be00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:10:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.098+0000 7f642be00640 1 -- 172.21.15.69:0/2682714832 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438035040 con 0x55c70b292ce0 2024-04-24T09:10:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.098+0000 7f64503ae740 1 -- 172.21.15.69:0/2682714832 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64300044e0 con 0x55c70b292ce0 2024-04-24T09:10:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.098+0000 7f642be00640 1 -- 172.21.15.69:0/2682714832 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f6438005230 con 0x55c70b292ce0 2024-04-24T09:10:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.098+0000 7f642be00640 1 -- 172.21.15.69:0/2682714832 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6438055050 con 0x55c70b292ce0 2024-04-24T09:10:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.098+0000 7f64503ae740 10 client.16602 fetch_fsmap learned FSMap version 29 2024-04-24T09:10:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.098+0000 7f64503ae740 1 -- 172.21.15.69:0/2682714832 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c70b31c470 con 0x55c70b292ce0 2024-04-24T09:10:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.099+0000 7f642be00640 1 -- 172.21.15.69:0/2682714832 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64300044e0 con 0x55c70b292ce0 2024-04-24T09:10:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.099+0000 7f642be00640 1 -- 172.21.15.69:0/2682714832 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f6438004780 con 0x55c70b292ce0 2024-04-24T09:10:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.099+0000 7f64503ae740 10 client.16602 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:10:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.099+0000 7f64503ae740 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.099+0000 7f64503ae740 20 client.16602 populate_metadata read hostname 'smithi069' 2024-04-24T09:10:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.099+0000 7f64503ae740 1 --2- 172.21.15.69:0/2682714832 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c70b31cc30 0x55c70b31f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.099+0000 7f64503ae740 4 client.16602 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:10:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.099+0000 7f64503ae740 1 -- 172.21.15.69:0/2682714832 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c70b200190 con 0x55c70b31cc30 2024-04-24T09:10:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.099+0000 7f643fe00640 1 --2- 172.21.15.69:0/2682714832 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c70b31cc30 0x55c70b31f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.100+0000 7f643fe00640 1 --2- 172.21.15.69:0/2682714832 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c70b31cc30 0x55c70b31f050 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f0d78183e5b5d37 server_cookie=895540db52d728b7 in_seq=0 out_seq=0 2024-04-24T09:10:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.100+0000 7f642be00640 10 client.16602 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.209+0000 7f642be00640 1 -- 172.21.15.69:0/2682714832 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c70b200190 con 0x55c70b31cc30 2024-04-24T09:10:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.209+0000 7f642be00640 10 client.16602 handle_command_reply: tid=0 2024-04-24T09:10:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.209+0000 7f64503ae740 1 client.16602 shutdown 2024-04-24T09:10:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.210+0000 7f64503ae740 1 -- 172.21.15.69:0/2682714832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b292ce0 msgr2=0x55c70b2b92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.210+0000 7f64503ae740 1 --2- 172.21.15.69:0/2682714832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b292ce0 0x55c70b2b92b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f64380096c0 tx=0x7f6438005df0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.210+0000 7f64503ae740 1 -- 172.21.15.69:0/2682714832 shutdown_connections 2024-04-24T09:10:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.210+0000 7f64503ae740 1 --2- 172.21.15.69:0/2682714832 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c70b31cc30 0x55c70b31f050 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.210+0000 7f64503ae740 1 --2- 172.21.15.69:0/2682714832 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c70b29d5a0 0x55c70b2b97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.210+0000 7f64503ae740 1 --2- 172.21.15.69:0/2682714832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c70b292ce0 0x55c70b2b92b0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.210+0000 7f64503ae740 1 --2- 172.21.15.69:0/2682714832 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c70b2923c0 0x55c70b2b8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.210+0000 7f64503ae740 1 -- 172.21.15.69:0/2682714832 >> 172.21.15.69:0/2682714832 conn(0x55c70b28e8d0 msgr2=0x55c70b28d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.210+0000 7f642be00640 0 client.16602 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.211+0000 7f64503ae740 1 -- 172.21.15.69:0/2682714832 shutdown_connections 2024-04-24T09:10:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.211+0000 7f64503ae740 1 -- 172.21.15.69:0/2682714832 wait complete. 2024-04-24T09:10:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.211+0000 7f64503ae740 20 client.16602 trim_cache size 0 max 16384 2024-04-24T09:10:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:10:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:10:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:10:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:10:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.212+0000 7f644ca00640 1 -- 172.21.15.69:0/3542965576 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f642c0a3d80 msgr2=0x7f642c0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.212+0000 7f644ca00640 1 --2- 172.21.15.69:0/3542965576 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f642c0a3d80 0x7f642c0a6240 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6440000f80 tx=0x7f6440004a40 comp rx=0 tx=0).stop 2024-04-24T09:10:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.212+0000 7f644ca00640 1 -- 172.21.15.69:0/3542965576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f644810f010 msgr2=0x7f64481a68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.212+0000 7f644ca00640 1 --2- 172.21.15.69:0/3542965576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f644810f010 0x7f64481a68b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f644000b970 tx=0x7f644000be30 comp rx=0 tx=0).stop 2024-04-24T09:10:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.212+0000 7f644ca00640 1 -- 172.21.15.69:0/3542965576 shutdown_connections 2024-04-24T09:10:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.212+0000 7f644ca00640 1 --2- 172.21.15.69:0/3542965576 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f642c0a3d80 0x7f642c0a6240 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.212+0000 7f644ca00640 1 --2- 172.21.15.69:0/3542965576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f644810f010 0x7f64481a68b0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.212+0000 7f644ca00640 1 --2- 172.21.15.69:0/3542965576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6448106030 0x7f644810ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.212+0000 7f644ca00640 1 --2- 172.21.15.69:0/3542965576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6448100f60 0x7f644810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.212+0000 7f644ca00640 1 -- 172.21.15.69:0/3542965576 >> 172.21.15.69:0/3542965576 conn(0x7f64480fc820 msgr2=0x7f64480fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.213+0000 7f644ca00640 1 -- 172.21.15.69:0/3542965576 shutdown_connections 2024-04-24T09:10:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:21.213+0000 7f644ca00640 1 -- 172.21.15.69:0/3542965576 wait complete. 2024-04-24T09:10:21.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:21 smithi069 ceph-mon[28170]: pgmap v101: 129 pgs: 129 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 708 KiB/s wr, 127 op/s 2024-04-24T09:10:21.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:21 smithi130 ceph-mon[31553]: pgmap v101: 129 pgs: 129 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 708 KiB/s wr, 127 op/s 2024-04-24T09:10:22.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:21 smithi081 ceph-mon[30961]: pgmap v101: 129 pgs: 129 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 708 KiB/s wr, 127 op/s 2024-04-24T09:10:23.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:23 smithi130 ceph-mon[31553]: pgmap v102: 129 pgs: 129 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 466 KiB/s wr, 62 op/s 2024-04-24T09:10:24.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:23 smithi081 ceph-mon[30961]: pgmap v102: 129 pgs: 129 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 466 KiB/s wr, 62 op/s 2024-04-24T09:10:24.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:23 smithi069 ceph-mon[28170]: pgmap v102: 129 pgs: 129 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 466 KiB/s wr, 62 op/s 2024-04-24T09:10:25.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:25 smithi130 ceph-mon[31553]: pgmap v103: 129 pgs: 129 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 466 KiB/s wr, 62 op/s 2024-04-24T09:10:26.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:25 smithi081 ceph-mon[30961]: pgmap v103: 129 pgs: 129 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 466 KiB/s wr, 62 op/s 2024-04-24T09:10:26.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:25 smithi069 ceph-mon[28170]: pgmap v103: 129 pgs: 129 active+clean; 49 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 466 KiB/s wr, 62 op/s 2024-04-24T09:10:27.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:27 smithi130 ceph-mon[31553]: pgmap v104: 129 pgs: 129 active+clean; 46 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 575 KiB/s wr, 110 op/s 2024-04-24T09:10:28.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:27 smithi081 ceph-mon[30961]: pgmap v104: 129 pgs: 129 active+clean; 46 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 575 KiB/s wr, 110 op/s 2024-04-24T09:10:28.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:27 smithi069 ceph-mon[28170]: pgmap v104: 129 pgs: 129 active+clean; 46 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 575 KiB/s wr, 110 op/s 2024-04-24T09:10:29.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:29 smithi130 ceph-mon[31553]: pgmap v105: 129 pgs: 129 active+clean; 46 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 355 KiB/s wr, 81 op/s 2024-04-24T09:10:30.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:29 smithi081 ceph-mon[30961]: pgmap v105: 129 pgs: 129 active+clean; 46 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 355 KiB/s wr, 81 op/s 2024-04-24T09:10:30.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:29 smithi069 ceph-mon[28170]: pgmap v105: 129 pgs: 129 active+clean; 46 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 355 KiB/s wr, 81 op/s 2024-04-24T09:10:31.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:31 smithi130 ceph-mon[31553]: pgmap v106: 129 pgs: 129 active+clean; 46 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 417 KiB/s wr, 92 op/s 2024-04-24T09:10:32.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:31 smithi081 ceph-mon[30961]: pgmap v106: 129 pgs: 129 active+clean; 46 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 417 KiB/s wr, 92 op/s 2024-04-24T09:10:32.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:31 smithi069 ceph-mon[28170]: pgmap v106: 129 pgs: 129 active+clean; 46 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 417 KiB/s wr, 92 op/s 2024-04-24T09:10:33.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:33 smithi130 ceph-mon[31553]: pgmap v107: 129 pgs: 129 active+clean; 46 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 171 KiB/s wr, 59 op/s 2024-04-24T09:10:34.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:33 smithi081 ceph-mon[30961]: pgmap v107: 129 pgs: 129 active+clean; 46 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 171 KiB/s wr, 59 op/s 2024-04-24T09:10:34.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:33 smithi069 ceph-mon[28170]: pgmap v107: 129 pgs: 129 active+clean; 46 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 171 KiB/s wr, 59 op/s 2024-04-24T09:10:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.355+0000 7fc88f000640 1 -- 172.21.15.69:0/2754696071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc888108ec0 msgr2=0x7fc888115300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.355+0000 7fc88f000640 1 --2- 172.21.15.69:0/2754696071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc888108ec0 0x7fc888115300 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc880009930 tx=0x7fc88002a420 comp rx=0 tx=0).stop 2024-04-24T09:10:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.355+0000 7fc88f000640 1 -- 172.21.15.69:0/2754696071 shutdown_connections 2024-04-24T09:10:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.355+0000 7fc88f000640 1 --2- 172.21.15.69:0/2754696071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc888108ec0 0x7fc888115300 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.355+0000 7fc88f000640 1 --2- 172.21.15.69:0/2754696071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc888108540 0x7fc888108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.355+0000 7fc88f000640 1 --2- 172.21.15.69:0/2754696071 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc88810e4d0 0x7fc88810e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.355+0000 7fc88f000640 1 -- 172.21.15.69:0/2754696071 >> 172.21.15.69:0/2754696071 conn(0x7fc888077de0 msgr2=0x7fc888106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.355+0000 7fc88f000640 1 -- 172.21.15.69:0/2754696071 shutdown_connections 2024-04-24T09:10:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.355+0000 7fc88f000640 1 -- 172.21.15.69:0/2754696071 wait complete. 2024-04-24T09:10:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.356+0000 7fc88f000640 1 Processor -- start 2024-04-24T09:10:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.356+0000 7fc88f000640 1 -- start start 2024-04-24T09:10:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.357+0000 7fc88f000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc888108540 0x7fc8881a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.357+0000 7fc88f000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc888108ec0 0x7fc8881a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.357+0000 7fc88f000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc88810e4d0 0x7fc88819ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.357+0000 7fc88f000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc888118700 con 0x7fc888108540 2024-04-24T09:10:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.357+0000 7fc88f000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc888118580 con 0x7fc888108ec0 2024-04-24T09:10:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.357+0000 7fc88f000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc888118880 con 0x7fc88810e4d0 2024-04-24T09:10:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.357+0000 7fc88dc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc888108540 0x7fc8881a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.357+0000 7fc88dc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc888108540 0x7fc8881a4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49576/0 (socket says 172.21.15.69:49576) 2024-04-24T09:10:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.357+0000 7fc88dc00640 1 -- 172.21.15.69:0/1597116221 learned_addr learned my addr 172.21.15.69:0/1597116221 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.357+0000 7fc88d200640 1 --2- 172.21.15.69:0/1597116221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc888108ec0 0x7fc8881a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.358+0000 7fc88e600640 1 --2- 172.21.15.69:0/1597116221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc88810e4d0 0x7fc88819ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.358+0000 7fc88dc00640 1 -- 172.21.15.69:0/1597116221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc88810e4d0 msgr2=0x7fc88819ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.358+0000 7fc88dc00640 1 --2- 172.21.15.69:0/1597116221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc88810e4d0 0x7fc88819ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.358+0000 7fc88dc00640 1 -- 172.21.15.69:0/1597116221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc888108ec0 msgr2=0x7fc8881a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.358+0000 7fc88dc00640 1 --2- 172.21.15.69:0/1597116221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc888108ec0 0x7fc8881a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.358+0000 7fc88dc00640 1 -- 172.21.15.69:0/1597116221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88819f1a0 con 0x7fc888108540 2024-04-24T09:10:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.358+0000 7fc88dc00640 1 --2- 172.21.15.69:0/1597116221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc888108540 0x7fc8881a4870 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00eab0 tx=0x7fc87c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.358+0000 7fc886000640 1 -- 172.21.15.69:0/1597116221 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c00ce40 con 0x7fc888108540 2024-04-24T09:10:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.358+0000 7fc886000640 1 -- 172.21.15.69:0/1597116221 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc87c011920 con 0x7fc888108540 2024-04-24T09:10:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.358+0000 7fc886000640 1 -- 172.21.15.69:0/1597116221 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c010a00 con 0x7fc888108540 2024-04-24T09:10:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.358+0000 7fc88f000640 1 -- 172.21.15.69:0/1597116221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88819f490 con 0x7fc888108540 2024-04-24T09:10:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.359+0000 7fc88f000640 1 -- 172.21.15.69:0/1597116221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8881ab730 con 0x7fc888108540 2024-04-24T09:10:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.362+0000 7fc8933ea740 1 Processor -- start 2024-04-24T09:10:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.362+0000 7fc886000640 1 -- 172.21.15.69:0/1597116221 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc87c011ad0 con 0x7fc888108540 2024-04-24T09:10:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.362+0000 7fc8933ea740 1 -- start start 2024-04-24T09:10:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.362+0000 7fc8933ea740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c39f3c0 0x55f38c39f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.362+0000 7fc8933ea740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c39fce0 0x55f38c3aa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.362+0000 7fc8933ea740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c3aa5a0 0x55f38c3ac990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.362+0000 7fc8933ea740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f38c305900 con 0x55f38c39fce0 2024-04-24T09:10:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.362+0000 7fc8933ea740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f38c2abf20 con 0x55f38c3aa5a0 2024-04-24T09:10:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.362+0000 7fc8933ea740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f38c3018c0 con 0x55f38c39f3c0 2024-04-24T09:10:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.363+0000 7fc88dc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c39fce0 0x55f38c3aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.363+0000 7fc88dc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c39fce0 0x55f38c3aa060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49578/0 (socket says 172.21.15.69:49578) 2024-04-24T09:10:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.363+0000 7fc88dc00640 1 -- 172.21.15.69:0/3481218373 learned_addr learned my addr 172.21.15.69:0/3481218373 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.363+0000 7fc88e600640 1 --2- 172.21.15.69:0/3481218373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c39f3c0 0x55f38c39f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.363+0000 7fc88d200640 1 --2- 172.21.15.69:0/3481218373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c3aa5a0 0x55f38c3ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.363+0000 7fc88dc00640 1 -- 172.21.15.69:0/3481218373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c39f3c0 msgr2=0x55f38c39f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.363+0000 7fc88dc00640 1 --2- 172.21.15.69:0/3481218373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c39f3c0 0x55f38c39f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.363+0000 7fc88dc00640 1 -- 172.21.15.69:0/3481218373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c3aa5a0 msgr2=0x55f38c3ac990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.363+0000 7fc88dc00640 1 --2- 172.21.15.69:0/3481218373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c3aa5a0 0x55f38c3ac990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.363+0000 7fc88dc00640 1 -- 172.21.15.69:0/3481218373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f38c2d0e10 con 0x55f38c39fce0 2024-04-24T09:10:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.364+0000 7fc886000640 1 --2- 172.21.15.69:0/1597116221 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc8700a3d90 0x7fc8700a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.364+0000 7fc88dc00640 1 --2- 172.21.15.69:0/3481218373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c39fce0 0x55f38c3aa060 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0ce470 tx=0x7fc87c0d9540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfce7ffb5d4876b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.364+0000 7fc88e600640 1 --2- 172.21.15.69:0/1597116221 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc8700a3d90 0x7fc8700a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.364+0000 7fc87aa00640 1 -- 172.21.15.69:0/3481218373 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c0d9e60 con 0x55f38c39fce0 2024-04-24T09:10:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.364+0000 7fc87aa00640 1 -- 172.21.15.69:0/3481218373 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc87c011e00 con 0x55f38c39fce0 2024-04-24T09:10:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.364+0000 7fc886000640 1 -- 172.21.15.69:0/1597116221 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc87c0d1180 con 0x7fc888108540 2024-04-24T09:10:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.364+0000 7fc87aa00640 1 -- 172.21.15.69:0/3481218373 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c09d480 con 0x55f38c39fce0 2024-04-24T09:10:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.365+0000 7fc8933ea740 1 -- 172.21.15.69:0/3481218373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c39fce0 msgr2=0x55f38c3aa060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.365+0000 7fc88e600640 1 --2- 172.21.15.69:0/1597116221 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc8700a3d90 0x7fc8700a6250 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc880006010 tx=0x7fc880005a20 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.365+0000 7fc8933ea740 1 --2- 172.21.15.69:0/3481218373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c39fce0 0x55f38c3aa060 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0ce470 tx=0x7fc87c0d9540 comp rx=0 tx=0).stop 2024-04-24T09:10:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.365+0000 7fc8933ea740 1 -- 172.21.15.69:0/3481218373 shutdown_connections 2024-04-24T09:10:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.365+0000 7fc8933ea740 1 --2- 172.21.15.69:0/3481218373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c3aa5a0 0x55f38c3ac990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.365+0000 7fc8933ea740 1 --2- 172.21.15.69:0/3481218373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c39fce0 0x55f38c3aa060 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.365+0000 7fc8933ea740 1 --2- 172.21.15.69:0/3481218373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c39f3c0 0x55f38c39f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.365+0000 7fc8933ea740 1 -- 172.21.15.69:0/3481218373 >> 172.21.15.69:0/3481218373 conn(0x55f38c39b8d0 msgr2=0x55f38c39bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.365+0000 7fc8933ea740 1 -- 172.21.15.69:0/3481218373 shutdown_connections 2024-04-24T09:10:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.365+0000 7fc8933ea740 1 -- 172.21.15.69:0/3481218373 wait complete. 2024-04-24T09:10:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.365+0000 7fc8933ea740 1 -- start start 2024-04-24T09:10:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.366+0000 7fc8933ea740 1 Processor -- start 2024-04-24T09:10:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.366+0000 7fc8933ea740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c39f3c0 0x55f38c3a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.366+0000 7fc8933ea740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c39fce0 0x55f38c3a7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.366+0000 7fc8933ea740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c3aa5a0 0x55f38c3a8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.366+0000 7fc8933ea740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f38c3ad140 con 0x55f38c3aa5a0 2024-04-24T09:10:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.366+0000 7fc8933ea740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f38c2bf2c0 con 0x55f38c39f3c0 2024-04-24T09:10:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.366+0000 7fc8933ea740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f38c3ad2c0 con 0x55f38c39fce0 2024-04-24T09:10:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.366+0000 7fc8933ea740 10 client.0 _ll_register_callbacks cb 0x55f38c017a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.366+0000 7fc88e600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c3aa5a0 0x55f38c3a8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.367+0000 7fc88e600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c3aa5a0 0x55f38c3a8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49582/0 (socket says 172.21.15.69:49582) 2024-04-24T09:10:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.367+0000 7fc88e600640 1 -- 172.21.15.69:0/1746751930 learned_addr learned my addr 172.21.15.69:0/1746751930 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.367+0000 7fc88dc00640 1 --2- 172.21.15.69:0/1746751930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c39f3c0 0x55f38c3a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.367+0000 7fc88d200640 1 --2- 172.21.15.69:0/1746751930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c39fce0 0x55f38c3a7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.367+0000 7fc88e600640 1 -- 172.21.15.69:0/1746751930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c39fce0 msgr2=0x55f38c3a7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.367+0000 7fc88e600640 1 --2- 172.21.15.69:0/1746751930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c39fce0 0x55f38c3a7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.367+0000 7fc88e600640 1 -- 172.21.15.69:0/1746751930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c39f3c0 msgr2=0x55f38c3a75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.367+0000 7fc88e600640 1 --2- 172.21.15.69:0/1746751930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c39f3c0 0x55f38c3a75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.367+0000 7fc88e600640 1 -- 172.21.15.69:0/1746751930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f38c2cffd0 con 0x55f38c3aa5a0 2024-04-24T09:10:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.367+0000 7fc88e600640 1 --2- 172.21.15.69:0/1746751930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c3aa5a0 0x55f38c3a8040 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc880031440 tx=0x7fc880009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7d88f64d6391ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.367+0000 7fc86f400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:10:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.368+0000 7fc86f400640 1 -- 172.21.15.69:0/1746751930 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88004c070 con 0x55f38c3aa5a0 2024-04-24T09:10:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.368+0000 7fc86f400640 1 -- 172.21.15.69:0/1746751930 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc8800037f0 con 0x55f38c3aa5a0 2024-04-24T09:10:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.368+0000 7fc8933ea740 1 -- 172.21.15.69:0/1746751930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f38c313320 con 0x55f38c3aa5a0 2024-04-24T09:10:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.368+0000 7fc86f400640 1 -- 172.21.15.69:0/1746751930 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc880058050 con 0x55f38c3aa5a0 2024-04-24T09:10:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.368+0000 7fc8933ea740 10 client.16614 fetch_fsmap learned FSMap version 29 2024-04-24T09:10:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.368+0000 7fc8933ea740 1 -- 172.21.15.69:0/1746751930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f38c018210 con 0x55f38c3aa5a0 2024-04-24T09:10:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.368+0000 7fc86f400640 1 -- 172.21.15.69:0/1746751930 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f38c313320 con 0x55f38c3aa5a0 2024-04-24T09:10:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.368+0000 7fc86f400640 1 -- 172.21.15.69:0/1746751930 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fc880032900 con 0x55f38c3aa5a0 2024-04-24T09:10:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.368+0000 7fc8933ea740 10 client.16614 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:10:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.368+0000 7fc8933ea740 10 client.16614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.368+0000 7fc8933ea740 20 client.16614 populate_metadata read hostname 'smithi069' 2024-04-24T09:10:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.369+0000 7fc8933ea740 1 --2- 172.21.15.69:0/1746751930 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f38c3ba760 0x55f38c3bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.369+0000 7fc8933ea740 4 client.16614 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.369+0000 7fc8933ea740 1 -- 172.21.15.69:0/1746751930 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f38c30d190 con 0x55f38c3ba760 2024-04-24T09:10:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.369+0000 7fc88dc00640 1 --2- 172.21.15.69:0/1746751930 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f38c3ba760 0x55f38c3bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.370+0000 7fc88dc00640 1 --2- 172.21.15.69:0/1746751930 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f38c3ba760 0x55f38c3bcb70 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f4252c2d9870d44 server_cookie=67abfc94f9f8f4a8 in_seq=0 out_seq=0 2024-04-24T09:10:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.370+0000 7fc86f400640 10 client.16614 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.373+0000 7fc86f400640 1 -- 172.21.15.69:0/1746751930 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f38c30d190 con 0x55f38c3ba760 2024-04-24T09:10:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.373+0000 7fc86f400640 10 client.16614 handle_command_reply: tid=0 2024-04-24T09:10:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.373+0000 7fc8933ea740 1 client.16614 shutdown 2024-04-24T09:10:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.373+0000 7fc8933ea740 1 -- 172.21.15.69:0/1746751930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c3aa5a0 msgr2=0x55f38c3a8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.373+0000 7fc8933ea740 1 --2- 172.21.15.69:0/1746751930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c3aa5a0 0x55f38c3a8040 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc880031440 tx=0x7fc880009260 comp rx=0 tx=0).stop 2024-04-24T09:10:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.374+0000 7fc8933ea740 1 -- 172.21.15.69:0/1746751930 shutdown_connections 2024-04-24T09:10:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.374+0000 7fc8933ea740 1 --2- 172.21.15.69:0/1746751930 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f38c3ba760 0x55f38c3bcb70 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.374+0000 7fc8933ea740 1 --2- 172.21.15.69:0/1746751930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c3aa5a0 0x55f38c3a8040 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.374+0000 7fc8933ea740 1 --2- 172.21.15.69:0/1746751930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c39fce0 0x55f38c3a7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.374+0000 7fc8933ea740 1 --2- 172.21.15.69:0/1746751930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c39f3c0 0x55f38c3a75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.374+0000 7fc8933ea740 1 -- 172.21.15.69:0/1746751930 >> 172.21.15.69:0/1746751930 conn(0x55f38c39b8d0 msgr2=0x55f38c39aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.374+0000 7fc86f400640 0 client.16614 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.374+0000 7fc8933ea740 1 -- 172.21.15.69:0/1746751930 shutdown_connections 2024-04-24T09:10:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.374+0000 7fc8933ea740 1 -- 172.21.15.69:0/1746751930 wait complete. 2024-04-24T09:10:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.374+0000 7fc8933ea740 20 client.16614 trim_cache size 0 max 16384 2024-04-24T09:10:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.390+0000 7fc8933ea740 1 Processor -- start 2024-04-24T09:10:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.390+0000 7fc8933ea740 1 -- start start 2024-04-24T09:10:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.391+0000 7fc8933ea740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c39f3c0 0x55f38c3a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.391+0000 7fc8933ea740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c39fce0 0x55f38c3b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.391+0000 7fc8933ea740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c3aa5a0 0x55f38c3c2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.391+0000 7fc8933ea740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881b8080 con 0x55f38c3aa5a0 2024-04-24T09:10:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.391+0000 7fc8933ea740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881b58c0 con 0x55f38c39f3c0 2024-04-24T09:10:34.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.391+0000 7fc8933ea740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881b3710 con 0x55f38c39fce0 2024-04-24T09:10:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.391+0000 7fc88e600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c3aa5a0 0x55f38c3c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.391+0000 7fc88e600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c3aa5a0 0x55f38c3c2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49598/0 (socket says 172.21.15.69:49598) 2024-04-24T09:10:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.391+0000 7fc88e600640 1 -- 172.21.15.69:0/229438888 learned_addr learned my addr 172.21.15.69:0/229438888 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.391+0000 7fc88d200640 1 --2- 172.21.15.69:0/229438888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c39fce0 0x55f38c3b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.391+0000 7fc88dc00640 1 --2- 172.21.15.69:0/229438888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c39f3c0 0x55f38c3a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.392+0000 7fc88e600640 1 -- 172.21.15.69:0/229438888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c39fce0 msgr2=0x55f38c3b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.392+0000 7fc88e600640 1 --2- 172.21.15.69:0/229438888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c39fce0 0x55f38c3b60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.392+0000 7fc88e600640 1 -- 172.21.15.69:0/229438888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c39f3c0 msgr2=0x55f38c3a9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.392+0000 7fc88e600640 1 --2- 172.21.15.69:0/229438888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c39f3c0 0x55f38c3a9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.392+0000 7fc88e600640 1 -- 172.21.15.69:0/229438888 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f38c0e4620 con 0x55f38c3aa5a0 2024-04-24T09:10:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.392+0000 7fc88e600640 1 --2- 172.21.15.69:0/229438888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c3aa5a0 0x55f38c3c2480 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc88002a460 tx=0x7fc880005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65ab3ae202dc921 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.392+0000 7fc87a000640 1 -- 172.21.15.69:0/229438888 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88004c070 con 0x55f38c3aa5a0 2024-04-24T09:10:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.392+0000 7fc87a000640 1 -- 172.21.15.69:0/229438888 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc880061070 con 0x55f38c3aa5a0 2024-04-24T09:10:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.392+0000 7fc87a000640 1 -- 172.21.15.69:0/229438888 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880035050 con 0x55f38c3aa5a0 2024-04-24T09:10:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.393+0000 7fc8933ea740 1 -- 172.21.15.69:0/229438888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c3aa5a0 msgr2=0x55f38c3c2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.393+0000 7fc8933ea740 1 --2- 172.21.15.69:0/229438888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c3aa5a0 0x55f38c3c2480 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc88002a460 tx=0x7fc880005460 comp rx=0 tx=0).stop 2024-04-24T09:10:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.393+0000 7fc8933ea740 1 -- 172.21.15.69:0/229438888 shutdown_connections 2024-04-24T09:10:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.393+0000 7fc8933ea740 1 --2- 172.21.15.69:0/229438888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c3aa5a0 0x55f38c3c2480 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.393+0000 7fc8933ea740 1 --2- 172.21.15.69:0/229438888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c39fce0 0x55f38c3b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.393+0000 7fc8933ea740 1 --2- 172.21.15.69:0/229438888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c39f3c0 0x55f38c3a9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.393+0000 7fc8933ea740 1 -- 172.21.15.69:0/229438888 >> 172.21.15.69:0/229438888 conn(0x55f38c39b8d0 msgr2=0x55f38c3a5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.393+0000 7fc8933ea740 1 -- 172.21.15.69:0/229438888 shutdown_connections 2024-04-24T09:10:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.393+0000 7fc8933ea740 1 -- 172.21.15.69:0/229438888 wait complete. 2024-04-24T09:10:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.394+0000 7fc8933ea740 1 -- start start 2024-04-24T09:10:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.394+0000 7fc8933ea740 1 Processor -- start 2024-04-24T09:10:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.394+0000 7fc8933ea740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c39f3c0 0x55f38c3b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.394+0000 7fc8933ea740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c39fce0 0x55f38c3adb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.394+0000 7fc8933ea740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c3aa5a0 0x55f38c3ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.394+0000 7fc8933ea740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881b3d80 con 0x55f38c39f3c0 2024-04-24T09:10:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.394+0000 7fc8933ea740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881b51f0 con 0x55f38c39fce0 2024-04-24T09:10:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.394+0000 7fc8933ea740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881b4450 con 0x55f38c3aa5a0 2024-04-24T09:10:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.394+0000 7fc8933ea740 10 client.0 _ll_register_callbacks cb 0x7fc880009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.394+0000 7fc88dc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c39f3c0 0x55f38c3b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.395+0000 7fc88dc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c39f3c0 0x55f38c3b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49600/0 (socket says 172.21.15.69:49600) 2024-04-24T09:10:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.395+0000 7fc88dc00640 1 -- 172.21.15.69:0/2342608831 learned_addr learned my addr 172.21.15.69:0/2342608831 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.395+0000 7fc88e600640 1 --2- 172.21.15.69:0/2342608831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c3aa5a0 0x55f38c3ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.395+0000 7fc88dc00640 1 -- 172.21.15.69:0/2342608831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c3aa5a0 msgr2=0x55f38c3ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.395+0000 7fc88d200640 1 --2- 172.21.15.69:0/2342608831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c39fce0 0x55f38c3adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.395+0000 7fc88dc00640 1 --2- 172.21.15.69:0/2342608831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c3aa5a0 0x55f38c3ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.395+0000 7fc88dc00640 1 -- 172.21.15.69:0/2342608831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c39fce0 msgr2=0x55f38c3adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.395+0000 7fc88dc00640 1 --2- 172.21.15.69:0/2342608831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c39fce0 0x55f38c3adb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.395+0000 7fc88dc00640 1 -- 172.21.15.69:0/2342608831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f38c3ae6b0 con 0x55f38c39f3c0 2024-04-24T09:10:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.395+0000 7fc88dc00640 1 --2- 172.21.15.69:0/2342608831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c39f3c0 0x55f38c3b27c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc87c09cd10 tx=0x7fc87c09d8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6be6ee3a5ce3715 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.395+0000 7fc86f400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:10:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.395+0000 7fc86f400640 1 -- 172.21.15.69:0/2342608831 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c054030 con 0x55f38c39f3c0 2024-04-24T09:10:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.395+0000 7fc86f400640 1 -- 172.21.15.69:0/2342608831 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc87c0ce7b0 con 0x55f38c39f3c0 2024-04-24T09:10:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.395+0000 7fc8933ea740 1 -- 172.21.15.69:0/2342608831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc880061070 con 0x55f38c39f3c0 2024-04-24T09:10:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.396+0000 7fc86f400640 1 -- 172.21.15.69:0/2342608831 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc87c09d710 con 0x55f38c39f3c0 2024-04-24T09:10:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.396+0000 7fc8933ea740 10 client.16620 fetch_fsmap learned FSMap version 29 2024-04-24T09:10:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.396+0000 7fc8933ea740 1 -- 172.21.15.69:0/2342608831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f38c420b30 con 0x55f38c39f3c0 2024-04-24T09:10:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.396+0000 7fc86f400640 1 -- 172.21.15.69:0/2342608831 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc880061070 con 0x55f38c39f3c0 2024-04-24T09:10:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.396+0000 7fc86f400640 1 -- 172.21.15.69:0/2342608831 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fc87c00f040 con 0x55f38c39f3c0 2024-04-24T09:10:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.396+0000 7fc8933ea740 10 client.16620 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:10:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.396+0000 7fc8933ea740 10 client.16620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.396+0000 7fc8933ea740 20 client.16620 populate_metadata read hostname 'smithi069' 2024-04-24T09:10:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.396+0000 7fc8933ea740 1 --2- 172.21.15.69:0/2342608831 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f38c420ec0 0x55f38c4232b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.396+0000 7fc8933ea740 4 client.16620 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:10:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.396+0000 7fc8933ea740 1 -- 172.21.15.69:0/2342608831 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f38c2894b0 con 0x55f38c420ec0 2024-04-24T09:10:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.397+0000 7fc88d200640 1 --2- 172.21.15.69:0/2342608831 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f38c420ec0 0x55f38c4232b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.398+0000 7fc88d200640 1 --2- 172.21.15.69:0/2342608831 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f38c420ec0 0x55f38c4232b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab9c555f9ae34953 server_cookie=e521b4107ae63b70 in_seq=0 out_seq=0 2024-04-24T09:10:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.398+0000 7fc86f400640 10 client.16620 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.484+0000 7fc86f400640 1 -- 172.21.15.69:0/2342608831 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f38c2894b0 con 0x55f38c420ec0 2024-04-24T09:10:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.484+0000 7fc86f400640 10 client.16620 handle_command_reply: tid=0 2024-04-24T09:10:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.484+0000 7fc8933ea740 1 client.16620 shutdown 2024-04-24T09:10:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.485+0000 7fc8933ea740 1 -- 172.21.15.69:0/2342608831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c39f3c0 msgr2=0x55f38c3b27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.485+0000 7fc8933ea740 1 --2- 172.21.15.69:0/2342608831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c39f3c0 0x55f38c3b27c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc87c09cd10 tx=0x7fc87c09d8b0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.485+0000 7fc8933ea740 1 -- 172.21.15.69:0/2342608831 shutdown_connections 2024-04-24T09:10:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.485+0000 7fc8933ea740 1 --2- 172.21.15.69:0/2342608831 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f38c420ec0 0x55f38c4232b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.485+0000 7fc8933ea740 1 --2- 172.21.15.69:0/2342608831 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f38c3aa5a0 0x55f38c3ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.485+0000 7fc86f400640 0 client.16620 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.485+0000 7fc8933ea740 1 --2- 172.21.15.69:0/2342608831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38c39fce0 0x55f38c3adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.485+0000 7fc8933ea740 1 --2- 172.21.15.69:0/2342608831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f38c39f3c0 0x55f38c3b27c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.485+0000 7fc8933ea740 1 -- 172.21.15.69:0/2342608831 >> 172.21.15.69:0/2342608831 conn(0x55f38c39b8d0 msgr2=0x55f38c39a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.485+0000 7fc8933ea740 1 -- 172.21.15.69:0/2342608831 shutdown_connections 2024-04-24T09:10:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.485+0000 7fc8933ea740 1 -- 172.21.15.69:0/2342608831 wait complete. 2024-04-24T09:10:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.485+0000 7fc8933ea740 20 client.16620 trim_cache size 0 max 16384 2024-04-24T09:10:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:10:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:10:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:10:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:10:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.486+0000 7fc88f000640 1 -- 172.21.15.69:0/1597116221 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc8700a3d90 msgr2=0x7fc8700a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.486+0000 7fc88f000640 1 --2- 172.21.15.69:0/1597116221 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc8700a3d90 0x7fc8700a6250 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc880006010 tx=0x7fc880005a20 comp rx=0 tx=0).stop 2024-04-24T09:10:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.486+0000 7fc88f000640 1 -- 172.21.15.69:0/1597116221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc888108540 msgr2=0x7fc8881a4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.486+0000 7fc88f000640 1 --2- 172.21.15.69:0/1597116221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc888108540 0x7fc8881a4870 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00eab0 tx=0x7fc87c00ef70 comp rx=0 tx=0).stop 2024-04-24T09:10:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.486+0000 7fc88f000640 1 -- 172.21.15.69:0/1597116221 shutdown_connections 2024-04-24T09:10:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.486+0000 7fc88f000640 1 --2- 172.21.15.69:0/1597116221 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc8700a3d90 0x7fc8700a6250 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.486+0000 7fc88f000640 1 --2- 172.21.15.69:0/1597116221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc88810e4d0 0x7fc88819ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.486+0000 7fc88f000640 1 --2- 172.21.15.69:0/1597116221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc888108ec0 0x7fc8881a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.486+0000 7fc88f000640 1 --2- 172.21.15.69:0/1597116221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc888108540 0x7fc8881a4870 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.486+0000 7fc88f000640 1 -- 172.21.15.69:0/1597116221 >> 172.21.15.69:0/1597116221 conn(0x7fc888077de0 msgr2=0x7fc888078400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.487+0000 7fc88f000640 1 -- 172.21.15.69:0/1597116221 shutdown_connections 2024-04-24T09:10:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:34.487+0000 7fc88f000640 1 -- 172.21.15.69:0/1597116221 wait complete. 2024-04-24T09:10:35.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:35 smithi130 ceph-mon[31553]: pgmap v108: 129 pgs: 129 active+clean; 46 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 172 KiB/s wr, 59 op/s 2024-04-24T09:10:36.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:35 smithi081 ceph-mon[30961]: pgmap v108: 129 pgs: 129 active+clean; 46 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 172 KiB/s wr, 59 op/s 2024-04-24T09:10:36.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:35 smithi069 ceph-mon[28170]: pgmap v108: 129 pgs: 129 active+clean; 46 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 172 KiB/s wr, 59 op/s 2024-04-24T09:10:37.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:37 smithi130 ceph-mon[31553]: pgmap v109: 129 pgs: 129 active+clean; 51 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 546 KiB/s wr, 139 op/s 2024-04-24T09:10:38.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:37 smithi081 ceph-mon[30961]: pgmap v109: 129 pgs: 129 active+clean; 51 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 546 KiB/s wr, 139 op/s 2024-04-24T09:10:38.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:37 smithi069 ceph-mon[28170]: pgmap v109: 129 pgs: 129 active+clean; 51 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 546 KiB/s wr, 139 op/s 2024-04-24T09:10:39.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:39 smithi130 ceph-mon[31553]: pgmap v110: 129 pgs: 129 active+clean; 51 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 437 KiB/s wr, 91 op/s 2024-04-24T09:10:40.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:39 smithi081 ceph-mon[30961]: pgmap v110: 129 pgs: 129 active+clean; 51 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 437 KiB/s wr, 91 op/s 2024-04-24T09:10:40.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:39 smithi069 ceph-mon[28170]: pgmap v110: 129 pgs: 129 active+clean; 51 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 437 KiB/s wr, 91 op/s 2024-04-24T09:10:41.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:41 smithi130 ceph-mon[31553]: pgmap v111: 129 pgs: 129 active+clean; 53 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 691 KiB/s wr, 134 op/s 2024-04-24T09:10:42.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:41 smithi081 ceph-mon[30961]: pgmap v111: 129 pgs: 129 active+clean; 53 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 691 KiB/s wr, 134 op/s 2024-04-24T09:10:42.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:41 smithi069 ceph-mon[28170]: pgmap v111: 129 pgs: 129 active+clean; 53 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 691 KiB/s wr, 134 op/s 2024-04-24T09:10:43.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:43 smithi130 ceph-mon[31553]: pgmap v112: 129 pgs: 129 active+clean; 53 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 629 KiB/s wr, 123 op/s 2024-04-24T09:10:44.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:43 smithi081 ceph-mon[30961]: pgmap v112: 129 pgs: 129 active+clean; 53 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 629 KiB/s wr, 123 op/s 2024-04-24T09:10:44.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:43 smithi069 ceph-mon[28170]: pgmap v112: 129 pgs: 129 active+clean; 53 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 629 KiB/s wr, 123 op/s 2024-04-24T09:10:45.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:45 smithi130 ceph-mon[31553]: pgmap v113: 129 pgs: 129 active+clean; 53 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 629 KiB/s wr, 123 op/s 2024-04-24T09:10:46.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:45 smithi081 ceph-mon[30961]: pgmap v113: 129 pgs: 129 active+clean; 53 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 629 KiB/s wr, 123 op/s 2024-04-24T09:10:46.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:45 smithi069 ceph-mon[28170]: pgmap v113: 129 pgs: 129 active+clean; 53 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 629 KiB/s wr, 123 op/s 2024-04-24T09:10:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.636+0000 7f601d400640 1 -- 172.21.15.69:0/3821873917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018108540 msgr2=0x7f6018108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.636+0000 7f601d400640 1 --2- 172.21.15.69:0/3821873917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018108540 0x7f6018108980 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6000009930 tx=0x7f600002a420 comp rx=0 tx=0).stop 2024-04-24T09:10:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.636+0000 7f601d400640 1 -- 172.21.15.69:0/3821873917 shutdown_connections 2024-04-24T09:10:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.636+0000 7f601d400640 1 --2- 172.21.15.69:0/3821873917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6018108ec0 0x7f6018115300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.636+0000 7f601d400640 1 --2- 172.21.15.69:0/3821873917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018108540 0x7f6018108980 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.636+0000 7f601d400640 1 --2- 172.21.15.69:0/3821873917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f601810e4d0 0x7f601810e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.636+0000 7f601d400640 1 -- 172.21.15.69:0/3821873917 >> 172.21.15.69:0/3821873917 conn(0x7f6018077de0 msgr2=0x7f6018106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.636+0000 7f601d400640 1 -- 172.21.15.69:0/3821873917 shutdown_connections 2024-04-24T09:10:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.636+0000 7f601d400640 1 -- 172.21.15.69:0/3821873917 wait complete. 2024-04-24T09:10:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.637+0000 7f601d400640 1 Processor -- start 2024-04-24T09:10:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.637+0000 7f601d400640 1 -- start start 2024-04-24T09:10:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.638+0000 7f601d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018108540 0x7f60181a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.638+0000 7f601d400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6018108ec0 0x7f60181a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.638+0000 7f601d400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f601810e4d0 0x7f601819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.638+0000 7f601d400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f60181186a0 con 0x7f6018108540 2024-04-24T09:10:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.638+0000 7f601d400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6018118520 con 0x7f601810e4d0 2024-04-24T09:10:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.638+0000 7f601d400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6018118820 con 0x7f6018108ec0 2024-04-24T09:10:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.638+0000 7f6017e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018108540 0x7f60181a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.638+0000 7f6017e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018108540 0x7f60181a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40068/0 (socket says 172.21.15.69:40068) 2024-04-24T09:10:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.638+0000 7f6017e00640 1 -- 172.21.15.69:0/1650009319 learned_addr learned my addr 172.21.15.69:0/1650009319 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.638+0000 7f601ca00640 1 --2- 172.21.15.69:0/1650009319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f601810e4d0 0x7f601819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.638+0000 7f600f400640 1 --2- 172.21.15.69:0/1650009319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6018108ec0 0x7f60181a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.638+0000 7f6017e00640 1 -- 172.21.15.69:0/1650009319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6018108ec0 msgr2=0x7f60181a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.638+0000 7f6017e00640 1 --2- 172.21.15.69:0/1650009319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6018108ec0 0x7f60181a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.638+0000 7f6017e00640 1 -- 172.21.15.69:0/1650009319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f601810e4d0 msgr2=0x7f601819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.638+0000 7f6017e00640 1 --2- 172.21.15.69:0/1650009319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f601810e4d0 0x7f601819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.638+0000 7f6017e00640 1 -- 172.21.15.69:0/1650009319 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601819f3e0 con 0x7f6018108540 2024-04-24T09:10:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.639+0000 7f6017e00640 1 --2- 172.21.15.69:0/1650009319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018108540 0x7f60181a4a60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f600800ea80 tx=0x7f600800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.639+0000 7f6015600640 1 -- 172.21.15.69:0/1650009319 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600800ce40 con 0x7f6018108540 2024-04-24T09:10:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.639+0000 7f601d400640 1 -- 172.21.15.69:0/1650009319 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601819f6d0 con 0x7f6018108540 2024-04-24T09:10:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.639+0000 7f6015600640 1 -- 172.21.15.69:0/1650009319 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6008005020 con 0x7f6018108540 2024-04-24T09:10:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.639+0000 7f6015600640 1 -- 172.21.15.69:0/1650009319 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60080116d0 con 0x7f6018108540 2024-04-24T09:10:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.639+0000 7f601d400640 1 -- 172.21.15.69:0/1650009319 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60181ab9e0 con 0x7f6018108540 2024-04-24T09:10:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.643+0000 7f6021854740 1 Processor -- start 2024-04-24T09:10:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.644+0000 7f6021854740 1 -- start start 2024-04-24T09:10:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.644+0000 7f6015600640 1 -- 172.21.15.69:0/1650009319 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f60080051d0 con 0x7f6018108540 2024-04-24T09:10:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.644+0000 7f6021854740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd1e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.644+0000 7f6021854740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd1ece0 0x55bb7bd29060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.644+0000 7f6021854740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd295a0 0x55bb7bd2b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.644+0000 7f6021854740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bb7bc84900 con 0x55bb7bd1e3c0 2024-04-24T09:10:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.644+0000 7f6021854740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55bb7bc2af20 con 0x55bb7bd1ece0 2024-04-24T09:10:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.644+0000 7f6021854740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55bb7bc808c0 con 0x55bb7bd295a0 2024-04-24T09:10:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.644+0000 7f601ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd1e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.644+0000 7f601ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd1e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40080/0 (socket says 172.21.15.69:40080) 2024-04-24T09:10:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.644+0000 7f601ca00640 1 -- 172.21.15.69:0/334254029 learned_addr learned my addr 172.21.15.69:0/334254029 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.645+0000 7f600f400640 1 --2- 172.21.15.69:0/334254029 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd295a0 0x55bb7bd2b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.645+0000 7f6017e00640 1 --2- 172.21.15.69:0/334254029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd1ece0 0x55bb7bd29060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.645+0000 7f601ca00640 1 -- 172.21.15.69:0/334254029 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd295a0 msgr2=0x55bb7bd2b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.645+0000 7f601ca00640 1 --2- 172.21.15.69:0/334254029 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd295a0 0x55bb7bd2b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.645+0000 7f601ca00640 1 -- 172.21.15.69:0/334254029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd1ece0 msgr2=0x55bb7bd29060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.645+0000 7f601ca00640 1 --2- 172.21.15.69:0/334254029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd1ece0 0x55bb7bd29060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.645+0000 7f601ca00640 1 -- 172.21.15.69:0/334254029 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb7bc4fe10 con 0x55bb7bd1e3c0 2024-04-24T09:10:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.645+0000 7f600f400640 1 --2- 172.21.15.69:0/334254029 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd295a0 0x55bb7bd2b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.645+0000 7f6017e00640 1 --2- 172.21.15.69:0/334254029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd1ece0 0x55bb7bd29060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.645+0000 7f601ca00640 1 --2- 172.21.15.69:0/334254029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd1e7a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6010009b40 tx=0x7f601000e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff0b87ac9ffaa57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.646+0000 7f600d200640 1 -- 172.21.15.69:0/334254029 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010016070 con 0x55bb7bd1e3c0 2024-04-24T09:10:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.646+0000 7f600d200640 1 -- 172.21.15.69:0/334254029 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f601000ed20 con 0x55bb7bd1e3c0 2024-04-24T09:10:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.646+0000 7f600d200640 1 -- 172.21.15.69:0/334254029 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010010750 con 0x55bb7bd1e3c0 2024-04-24T09:10:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.646+0000 7f6015600640 1 --2- 172.21.15.69:0/1650009319 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5ffc0a3d90 0x7f5ffc0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.646+0000 7f6015600640 1 -- 172.21.15.69:0/1650009319 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f60080d0df0 con 0x7f6018108540 2024-04-24T09:10:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.646+0000 7f6017e00640 1 --2- 172.21.15.69:0/1650009319 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5ffc0a3d90 0x7f5ffc0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.646+0000 7f6021854740 1 -- 172.21.15.69:0/334254029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd1e3c0 msgr2=0x55bb7bd1e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.646+0000 7f6021854740 1 --2- 172.21.15.69:0/334254029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd1e7a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6010009b40 tx=0x7f601000e2f0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.647+0000 7f6021854740 1 -- 172.21.15.69:0/334254029 shutdown_connections 2024-04-24T09:10:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.647+0000 7f6021854740 1 --2- 172.21.15.69:0/334254029 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd295a0 0x55bb7bd2b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.647+0000 7f6021854740 1 --2- 172.21.15.69:0/334254029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd1ece0 0x55bb7bd29060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.647+0000 7f6021854740 1 --2- 172.21.15.69:0/334254029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd1e7a0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.647+0000 7f6021854740 1 -- 172.21.15.69:0/334254029 >> 172.21.15.69:0/334254029 conn(0x55bb7bd1a8d0 msgr2=0x55bb7bd1acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.647+0000 7f6021854740 1 -- 172.21.15.69:0/334254029 shutdown_connections 2024-04-24T09:10:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.647+0000 7f6017e00640 1 --2- 172.21.15.69:0/1650009319 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5ffc0a3d90 0x7f5ffc0a6250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f60080ce2b0 tx=0x7f6008085f90 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.647+0000 7f6021854740 1 -- 172.21.15.69:0/334254029 wait complete. 2024-04-24T09:10:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.647+0000 7f6021854740 1 -- start start 2024-04-24T09:10:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.648+0000 7f6021854740 1 Processor -- start 2024-04-24T09:10:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.648+0000 7f6021854740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd26480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.648+0000 7f6021854740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd1ece0 0x55bb7bd269c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.648+0000 7f6021854740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd295a0 0x55bb7bd26f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.648+0000 7f6021854740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bb7bd2c0e0 con 0x55bb7bd1ece0 2024-04-24T09:10:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.648+0000 7f6021854740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55bb7bc3e2c0 con 0x55bb7bd295a0 2024-04-24T09:10:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.648+0000 7f6021854740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55bb7bd2c260 con 0x55bb7bd1e3c0 2024-04-24T09:10:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.648+0000 7f6021854740 10 client.0 _ll_register_callbacks cb 0x55bb7b996a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.648+0000 7f600f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd1ece0 0x55bb7bd269c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.648+0000 7f600f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd1ece0 0x55bb7bd269c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40082/0 (socket says 172.21.15.69:40082) 2024-04-24T09:10:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.648+0000 7f600f400640 1 -- 172.21.15.69:0/1501216645 learned_addr learned my addr 172.21.15.69:0/1501216645 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.649+0000 7f601ca00640 1 --2- 172.21.15.69:0/1501216645 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd26480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.649+0000 7f600f400640 1 -- 172.21.15.69:0/1501216645 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 msgr2=0x55bb7bd26480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.649+0000 7f600f400640 1 --2- 172.21.15.69:0/1501216645 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd26480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.649+0000 7f600f400640 1 -- 172.21.15.69:0/1501216645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd295a0 msgr2=0x55bb7bd26f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.649+0000 7f601ca00640 1 --2- 172.21.15.69:0/1501216645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd295a0 0x55bb7bd26f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.649+0000 7f600f400640 1 --2- 172.21.15.69:0/1501216645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd295a0 0x55bb7bd26f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.649+0000 7f600f400640 1 -- 172.21.15.69:0/1501216645 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7bc4efd0 con 0x55bb7bd1ece0 2024-04-24T09:10:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.649+0000 7f600f400640 1 --2- 172.21.15.69:0/1501216645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd1ece0 0x55bb7bd269c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6000006010 tx=0x7f6000032e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f252aebadb53fdc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.649+0000 7f6005600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:10:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.649+0000 7f6005600640 1 -- 172.21.15.69:0/1501216645 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600003f070 con 0x55bb7bd1ece0 2024-04-24T09:10:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.649+0000 7f6005600640 1 -- 172.21.15.69:0/1501216645 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f6000004040 con 0x55bb7bd1ece0 2024-04-24T09:10:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.649+0000 7f6021854740 1 -- 172.21.15.69:0/1501216645 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f601000ed20 con 0x55bb7bd1ece0 2024-04-24T09:10:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.650+0000 7f6005600640 1 -- 172.21.15.69:0/1501216645 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f600004a050 con 0x55bb7bd1ece0 2024-04-24T09:10:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.650+0000 7f6021854740 10 client.16632 fetch_fsmap learned FSMap version 29 2024-04-24T09:10:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.650+0000 7f6021854740 1 -- 172.21.15.69:0/1501216645 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb7b997210 con 0x55bb7bd1ece0 2024-04-24T09:10:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.650+0000 7f6005600640 1 -- 172.21.15.69:0/1501216645 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f601000ed20 con 0x55bb7bd1ece0 2024-04-24T09:10:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.650+0000 7f6005600640 1 -- 172.21.15.69:0/1501216645 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f6000004730 con 0x55bb7bd1ece0 2024-04-24T09:10:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.650+0000 7f6021854740 10 client.16632 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:10:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.650+0000 7f6021854740 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.650+0000 7f6021854740 20 client.16632 populate_metadata read hostname 'smithi069' 2024-04-24T09:10:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.650+0000 7f6021854740 1 --2- 172.21.15.69:0/1501216645 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55bb7bd39580 0x55bb7bd3b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.650+0000 7f6021854740 4 client.16632 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.650+0000 7f6021854740 1 -- 172.21.15.69:0/1501216645 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb7bc92320 con 0x55bb7bd39580 2024-04-24T09:10:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.651+0000 7f601ca00640 1 --2- 172.21.15.69:0/1501216645 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55bb7bd39580 0x55bb7bd3b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.652+0000 7f601ca00640 1 --2- 172.21.15.69:0/1501216645 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55bb7bd39580 0x55bb7bd3b990 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa151cf7181a63c server_cookie=876c814cab89dc11 in_seq=0 out_seq=0 2024-04-24T09:10:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.652+0000 7f6005600640 10 client.16632 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.654+0000 7f6005600640 1 -- 172.21.15.69:0/1501216645 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb7bc92320 con 0x55bb7bd39580 2024-04-24T09:10:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.654+0000 7f6005600640 10 client.16632 handle_command_reply: tid=0 2024-04-24T09:10:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.654+0000 7f6021854740 1 client.16632 shutdown 2024-04-24T09:10:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.654+0000 7f6021854740 1 -- 172.21.15.69:0/1501216645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd1ece0 msgr2=0x55bb7bd269c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.654+0000 7f6021854740 1 --2- 172.21.15.69:0/1501216645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd1ece0 0x55bb7bd269c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6000006010 tx=0x7f6000032e50 comp rx=0 tx=0).stop 2024-04-24T09:10:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.654+0000 7f6021854740 1 -- 172.21.15.69:0/1501216645 shutdown_connections 2024-04-24T09:10:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.654+0000 7f6021854740 1 --2- 172.21.15.69:0/1501216645 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55bb7bd39580 0x55bb7bd3b990 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.654+0000 7f6021854740 1 --2- 172.21.15.69:0/1501216645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd295a0 0x55bb7bd26f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.654+0000 7f6021854740 1 --2- 172.21.15.69:0/1501216645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd1ece0 0x55bb7bd269c0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.654+0000 7f6005600640 0 client.16632 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.654+0000 7f6021854740 1 --2- 172.21.15.69:0/1501216645 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd26480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.654+0000 7f6021854740 1 -- 172.21.15.69:0/1501216645 >> 172.21.15.69:0/1501216645 conn(0x55bb7bd1a8d0 msgr2=0x55bb7bd20650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.655+0000 7f6021854740 1 -- 172.21.15.69:0/1501216645 shutdown_connections 2024-04-24T09:10:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.655+0000 7f6021854740 1 -- 172.21.15.69:0/1501216645 wait complete. 2024-04-24T09:10:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.655+0000 7f6021854740 20 client.16632 trim_cache size 0 max 16384 2024-04-24T09:10:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.672+0000 7f6021854740 1 Processor -- start 2024-04-24T09:10:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.672+0000 7f6021854740 1 -- start start 2024-04-24T09:10:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.672+0000 7f6021854740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd35080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.672+0000 7f6021854740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd295a0 0x55bb7bd20d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.672+0000 7f6021854740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd26b50 0x55bb7bd412a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.672+0000 7f6021854740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f60181b8080 con 0x55bb7bd295a0 2024-04-24T09:10:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.673+0000 7f6021854740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f60181b58c0 con 0x55bb7bd26b50 2024-04-24T09:10:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.673+0000 7f6021854740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f60181b3710 con 0x55bb7bd1e3c0 2024-04-24T09:10:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.673+0000 7f600f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd295a0 0x55bb7bd20d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.673+0000 7f600f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd295a0 0x55bb7bd20d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40092/0 (socket says 172.21.15.69:40092) 2024-04-24T09:10:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.673+0000 7f600f400640 1 -- 172.21.15.69:0/2472392140 learned_addr learned my addr 172.21.15.69:0/2472392140 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.673+0000 7f601ca00640 1 --2- 172.21.15.69:0/2472392140 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd35080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.673+0000 7f600f400640 1 -- 172.21.15.69:0/2472392140 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 msgr2=0x55bb7bd35080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.673+0000 7f600f400640 1 --2- 172.21.15.69:0/2472392140 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd35080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.673+0000 7f600f400640 1 -- 172.21.15.69:0/2472392140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd26b50 msgr2=0x55bb7bd412a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.673+0000 7f600f400640 1 --2- 172.21.15.69:0/2472392140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd26b50 0x55bb7bd412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.673+0000 7f600f400640 1 -- 172.21.15.69:0/2472392140 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb7ba63620 con 0x55bb7bd295a0 2024-04-24T09:10:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.673+0000 7f601ca00640 1 --2- 172.21.15.69:0/2472392140 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd35080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.674+0000 7f600f400640 1 --2- 172.21.15.69:0/2472392140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd295a0 0x55bb7bd20d80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6000004ca0 tx=0x7f6000009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e1cfbf3495c87a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.674+0000 7f6007e00640 1 -- 172.21.15.69:0/2472392140 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600003f070 con 0x55bb7bd295a0 2024-04-24T09:10:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.674+0000 7f6007e00640 1 -- 172.21.15.69:0/2472392140 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6000031730 con 0x55bb7bd295a0 2024-04-24T09:10:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.674+0000 7f6007e00640 1 -- 172.21.15.69:0/2472392140 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000000e60 con 0x55bb7bd295a0 2024-04-24T09:10:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.675+0000 7f6021854740 1 -- 172.21.15.69:0/2472392140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd295a0 msgr2=0x55bb7bd20d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.675+0000 7f6021854740 1 --2- 172.21.15.69:0/2472392140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd295a0 0x55bb7bd20d80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6000004ca0 tx=0x7f6000009230 comp rx=0 tx=0).stop 2024-04-24T09:10:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.675+0000 7f6021854740 1 -- 172.21.15.69:0/2472392140 shutdown_connections 2024-04-24T09:10:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.675+0000 7f6021854740 1 --2- 172.21.15.69:0/2472392140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd26b50 0x55bb7bd412a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.675+0000 7f6021854740 1 --2- 172.21.15.69:0/2472392140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd295a0 0x55bb7bd20d80 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.675+0000 7f6021854740 1 --2- 172.21.15.69:0/2472392140 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd35080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.675+0000 7f6021854740 1 -- 172.21.15.69:0/2472392140 >> 172.21.15.69:0/2472392140 conn(0x55bb7bd1a8d0 msgr2=0x55bb7bd26440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.675+0000 7f6021854740 1 -- 172.21.15.69:0/2472392140 shutdown_connections 2024-04-24T09:10:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.675+0000 7f6021854740 1 -- 172.21.15.69:0/2472392140 wait complete. 2024-04-24T09:10:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.675+0000 7f6021854740 1 -- start start 2024-04-24T09:10:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.676+0000 7f6021854740 1 Processor -- start 2024-04-24T09:10:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.676+0000 7f6021854740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd2d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.676+0000 7f6021854740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd26b50 0x55bb7bd2db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.676+0000 7f6021854740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd295a0 0x55bb7bd9eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.676+0000 7f6021854740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f60181b3d80 con 0x55bb7bd26b50 2024-04-24T09:10:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.676+0000 7f6021854740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f60181b51f0 con 0x55bb7bd295a0 2024-04-24T09:10:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.676+0000 7f6021854740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f60181b4450 con 0x55bb7bd1e3c0 2024-04-24T09:10:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.676+0000 7f6021854740 10 client.0 _ll_register_callbacks cb 0x7f6000009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.676+0000 7f600f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd26b50 0x55bb7bd2db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.676+0000 7f600f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd26b50 0x55bb7bd2db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40098/0 (socket says 172.21.15.69:40098) 2024-04-24T09:10:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.676+0000 7f600f400640 1 -- 172.21.15.69:0/2203226005 learned_addr learned my addr 172.21.15.69:0/2203226005 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:10:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.677+0000 7f601ca00640 1 --2- 172.21.15.69:0/2203226005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd2d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.677+0000 7f600f400640 1 -- 172.21.15.69:0/2203226005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 msgr2=0x55bb7bd2d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.677+0000 7f600f400640 1 --2- 172.21.15.69:0/2203226005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd2d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.677+0000 7f600f400640 1 -- 172.21.15.69:0/2203226005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd295a0 msgr2=0x55bb7bd9eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.677+0000 7f600f400640 1 --2- 172.21.15.69:0/2203226005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd295a0 0x55bb7bd9eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.677+0000 7f600f400640 1 -- 172.21.15.69:0/2203226005 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7bd9f050 con 0x55bb7bd26b50 2024-04-24T09:10:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.677+0000 7f601ca00640 1 --2- 172.21.15.69:0/2203226005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd2d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.677+0000 7f600f400640 1 --2- 172.21.15.69:0/2203226005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd26b50 0x55bb7bd2db50 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6000004c20 tx=0x7f60000048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3809399eedc594 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.677+0000 7f6005600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:10:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.677+0000 7f6005600640 1 -- 172.21.15.69:0/2203226005 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600005a060 con 0x55bb7bd26b50 2024-04-24T09:10:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.677+0000 7f6005600640 1 -- 172.21.15.69:0/2203226005 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f60000051f0 con 0x55bb7bd26b50 2024-04-24T09:10:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.677+0000 7f6021854740 1 -- 172.21.15.69:0/2203226005 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6000031730 con 0x55bb7bd26b50 2024-04-24T09:10:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.678+0000 7f6005600640 1 -- 172.21.15.69:0/2203226005 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6000061050 con 0x55bb7bd26b50 2024-04-24T09:10:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.678+0000 7f6021854740 10 client.16638 fetch_fsmap learned FSMap version 29 2024-04-24T09:10:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.678+0000 7f6021854740 1 -- 172.21.15.69:0/2203226005 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb7bda0200 con 0x55bb7bd26b50 2024-04-24T09:10:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.678+0000 7f6005600640 1 -- 172.21.15.69:0/2203226005 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6000031730 con 0x55bb7bd26b50 2024-04-24T09:10:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.678+0000 7f6005600640 1 -- 172.21.15.69:0/2203226005 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f600003bb40 con 0x55bb7bd26b50 2024-04-24T09:10:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.678+0000 7f6021854740 10 client.16638 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:10:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.678+0000 7f6021854740 10 client.16638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.678+0000 7f6021854740 20 client.16638 populate_metadata read hostname 'smithi069' 2024-04-24T09:10:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.678+0000 7f6021854740 1 --2- 172.21.15.69:0/2203226005 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55bb7bda08e0 0x55bb7bda2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.678+0000 7f6021854740 4 client.16638 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:10:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.678+0000 7f6021854740 1 -- 172.21.15.69:0/2203226005 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb7bc8c190 con 0x55bb7bda08e0 2024-04-24T09:10:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.679+0000 7f601ca00640 1 --2- 172.21.15.69:0/2203226005 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55bb7bda08e0 0x55bb7bda2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.680+0000 7f601ca00640 1 --2- 172.21.15.69:0/2203226005 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55bb7bda08e0 0x55bb7bda2d00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1b4ef0874a23c06 server_cookie=5e00190038d3e2aa in_seq=0 out_seq=0 2024-04-24T09:10:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.680+0000 7f6005600640 10 client.16638 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.775+0000 7f6005600640 1 -- 172.21.15.69:0/2203226005 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb7bc8c190 con 0x55bb7bda08e0 2024-04-24T09:10:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.775+0000 7f6005600640 10 client.16638 handle_command_reply: tid=0 2024-04-24T09:10:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.775+0000 7f6021854740 1 client.16638 shutdown 2024-04-24T09:10:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.776+0000 7f6021854740 1 -- 172.21.15.69:0/2203226005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd26b50 msgr2=0x55bb7bd2db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.776+0000 7f6021854740 1 --2- 172.21.15.69:0/2203226005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd26b50 0x55bb7bd2db50 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6000004c20 tx=0x7f60000048f0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.776+0000 7f6021854740 1 -- 172.21.15.69:0/2203226005 shutdown_connections 2024-04-24T09:10:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.776+0000 7f6021854740 1 --2- 172.21.15.69:0/2203226005 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55bb7bda08e0 0x55bb7bda2d00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.776+0000 7f6021854740 1 --2- 172.21.15.69:0/2203226005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb7bd295a0 0x55bb7bd9eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.776+0000 7f6021854740 1 --2- 172.21.15.69:0/2203226005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb7bd26b50 0x55bb7bd2db50 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.776+0000 7f6021854740 1 --2- 172.21.15.69:0/2203226005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb7bd1e3c0 0x55bb7bd2d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.776+0000 7f6005600640 0 client.16638 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:10:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.776+0000 7f6021854740 1 -- 172.21.15.69:0/2203226005 >> 172.21.15.69:0/2203226005 conn(0x55bb7bd1a8d0 msgr2=0x55bb7bd19200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.776+0000 7f6021854740 1 -- 172.21.15.69:0/2203226005 shutdown_connections 2024-04-24T09:10:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.776+0000 7f6021854740 1 -- 172.21.15.69:0/2203226005 wait complete. 2024-04-24T09:10:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.776+0000 7f6021854740 20 client.16638 trim_cache size 0 max 16384 2024-04-24T09:10:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:10:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:10:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:10:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:10:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.777+0000 7f601d400640 1 -- 172.21.15.69:0/1650009319 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5ffc0a3d90 msgr2=0x7f5ffc0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.777+0000 7f601d400640 1 --2- 172.21.15.69:0/1650009319 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5ffc0a3d90 0x7f5ffc0a6250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f60080ce2b0 tx=0x7f6008085f90 comp rx=0 tx=0).stop 2024-04-24T09:10:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.777+0000 7f601d400640 1 -- 172.21.15.69:0/1650009319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018108540 msgr2=0x7f60181a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.777+0000 7f601d400640 1 --2- 172.21.15.69:0/1650009319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018108540 0x7f60181a4a60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f600800ea80 tx=0x7f600800ef40 comp rx=0 tx=0).stop 2024-04-24T09:10:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.777+0000 7f601d400640 1 -- 172.21.15.69:0/1650009319 shutdown_connections 2024-04-24T09:10:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.777+0000 7f601d400640 1 --2- 172.21.15.69:0/1650009319 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5ffc0a3d90 0x7f5ffc0a6250 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.777+0000 7f601d400640 1 --2- 172.21.15.69:0/1650009319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f601810e4d0 0x7f601819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.777+0000 7f601d400640 1 --2- 172.21.15.69:0/1650009319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6018108ec0 0x7f60181a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.777+0000 7f601d400640 1 --2- 172.21.15.69:0/1650009319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6018108540 0x7f60181a4a60 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.777+0000 7f601d400640 1 -- 172.21.15.69:0/1650009319 >> 172.21.15.69:0/1650009319 conn(0x7f6018077de0 msgr2=0x7f6018078560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.777+0000 7f601d400640 1 -- 172.21.15.69:0/1650009319 shutdown_connections 2024-04-24T09:10:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:10:47.778+0000 7f601d400640 1 -- 172.21.15.69:0/1650009319 wait complete. 2024-04-24T09:10:47.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:47 smithi130 ceph-mon[31553]: pgmap v114: 129 pgs: 129 active+clean; 56 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 877 KiB/s wr, 168 op/s 2024-04-24T09:10:48.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:47 smithi081 ceph-mon[30961]: pgmap v114: 129 pgs: 129 active+clean; 56 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 877 KiB/s wr, 168 op/s 2024-04-24T09:10:48.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:47 smithi069 ceph-mon[28170]: pgmap v114: 129 pgs: 129 active+clean; 56 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 877 KiB/s wr, 168 op/s 2024-04-24T09:10:49.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:49 smithi130 ceph-mon[31553]: pgmap v115: 129 pgs: 129 active+clean; 56 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 502 KiB/s wr, 87 op/s 2024-04-24T09:10:50.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:49 smithi081 ceph-mon[30961]: pgmap v115: 129 pgs: 129 active+clean; 56 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 502 KiB/s wr, 87 op/s 2024-04-24T09:10:50.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:49 smithi069 ceph-mon[28170]: pgmap v115: 129 pgs: 129 active+clean; 56 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 502 KiB/s wr, 87 op/s 2024-04-24T09:10:51.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:51 smithi130 ceph-mon[31553]: pgmap v116: 129 pgs: 129 active+clean; 55 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 797 KiB/s wr, 155 op/s 2024-04-24T09:10:52.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:51 smithi081 ceph-mon[30961]: pgmap v116: 129 pgs: 129 active+clean; 55 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 797 KiB/s wr, 155 op/s 2024-04-24T09:10:52.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:51 smithi069 ceph-mon[28170]: pgmap v116: 129 pgs: 129 active+clean; 55 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 797 KiB/s wr, 155 op/s 2024-04-24T09:10:53.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:53 smithi130 ceph-mon[31553]: pgmap v117: 129 pgs: 129 active+clean; 55 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 543 KiB/s wr, 112 op/s 2024-04-24T09:10:54.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:53 smithi081 ceph-mon[30961]: pgmap v117: 129 pgs: 129 active+clean; 55 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 543 KiB/s wr, 112 op/s 2024-04-24T09:10:54.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:53 smithi069 ceph-mon[28170]: pgmap v117: 129 pgs: 129 active+clean; 55 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 543 KiB/s wr, 112 op/s 2024-04-24T09:10:55.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:55 smithi130 ceph-mon[31553]: pgmap v118: 129 pgs: 129 active+clean; 55 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 543 KiB/s wr, 112 op/s 2024-04-24T09:10:56.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:55 smithi081 ceph-mon[30961]: pgmap v118: 129 pgs: 129 active+clean; 55 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 543 KiB/s wr, 112 op/s 2024-04-24T09:10:56.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:55 smithi069 ceph-mon[28170]: pgmap v118: 129 pgs: 129 active+clean; 55 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 543 KiB/s wr, 112 op/s 2024-04-24T09:10:58.069 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:57 smithi081 ceph-mon[30961]: pgmap v119: 129 pgs: 129 active+clean; 58 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 829 KiB/s wr, 156 op/s 2024-04-24T09:10:58.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:57 smithi069 ceph-mon[28170]: pgmap v119: 129 pgs: 129 active+clean; 58 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 829 KiB/s wr, 156 op/s 2024-04-24T09:10:58.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:57 smithi130 ceph-mon[31553]: pgmap v119: 129 pgs: 129 active+clean; 58 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 829 KiB/s wr, 156 op/s 2024-04-24T09:11:00.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:10:59 smithi081 ceph-mon[30961]: pgmap v120: 129 pgs: 129 active+clean; 58 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 581 KiB/s wr, 111 op/s 2024-04-24T09:11:00.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:10:59 smithi069 ceph-mon[28170]: pgmap v120: 129 pgs: 129 active+clean; 58 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 581 KiB/s wr, 111 op/s 2024-04-24T09:11:00.206 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:10:59 smithi130 ceph-mon[31553]: pgmap v120: 129 pgs: 129 active+clean; 58 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 581 KiB/s wr, 111 op/s 2024-04-24T09:11:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.921+0000 7fae25a00640 1 -- 172.21.15.69:0/2201545073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae20104450 msgr2=0x7fae20104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.921+0000 7fae25a00640 1 --2- 172.21.15.69:0/2201545073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae20104450 0x7fae20104830 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fae10009900 tx=0x7fae1002a410 comp rx=0 tx=0).stop 2024-04-24T09:11:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.921+0000 7fae25a00640 1 -- 172.21.15.69:0/2201545073 shutdown_connections 2024-04-24T09:11:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.921+0000 7fae25a00640 1 --2- 172.21.15.69:0/2201545073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae20109520 0x7fae201110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.921+0000 7fae25a00640 1 --2- 172.21.15.69:0/2201545073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae20104e00 0x7fae20108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.921+0000 7fae25a00640 1 --2- 172.21.15.69:0/2201545073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae20104450 0x7fae20104830 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.921+0000 7fae25a00640 1 -- 172.21.15.69:0/2201545073 >> 172.21.15.69:0/2201545073 conn(0x7fae200ffcf0 msgr2=0x7fae20102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.921+0000 7fae25a00640 1 -- 172.21.15.69:0/2201545073 shutdown_connections 2024-04-24T09:11:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.921+0000 7fae25a00640 1 -- 172.21.15.69:0/2201545073 wait complete. 2024-04-24T09:11:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.922+0000 7fae25a00640 1 Processor -- start 2024-04-24T09:11:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.922+0000 7fae25a00640 1 -- start start 2024-04-24T09:11:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.923+0000 7fae25a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae20104450 0x7fae201a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.923+0000 7fae25a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae20109520 0x7fae201a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.923+0000 7fae25a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae2019a820 0x7fae2019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.923+0000 7fae25a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fae20114500 con 0x7fae2019a820 2024-04-24T09:11:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.923+0000 7fae25a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fae20114380 con 0x7fae20109520 2024-04-24T09:11:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.923+0000 7fae25a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fae20114680 con 0x7fae20104450 2024-04-24T09:11:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.923+0000 7fae1f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae2019a820 0x7fae2019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.923+0000 7fae1f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae2019a820 0x7fae2019acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38490/0 (socket says 172.21.15.69:38490) 2024-04-24T09:11:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.923+0000 7fae1f400640 1 -- 172.21.15.69:0/4127067776 learned_addr learned my addr 172.21.15.69:0/4127067776 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.923+0000 7fae1ea00640 1 --2- 172.21.15.69:0/4127067776 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae20104450 0x7fae201a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.923+0000 7fae16000640 1 --2- 172.21.15.69:0/4127067776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae20109520 0x7fae201a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.923+0000 7fae1f400640 1 -- 172.21.15.69:0/4127067776 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae20104450 msgr2=0x7fae201a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.923+0000 7fae1f400640 1 --2- 172.21.15.69:0/4127067776 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae20104450 0x7fae201a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.923+0000 7fae1f400640 1 -- 172.21.15.69:0/4127067776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae20109520 msgr2=0x7fae201a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.923+0000 7fae1f400640 1 --2- 172.21.15.69:0/4127067776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae20109520 0x7fae201a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.923+0000 7fae1f400640 1 -- 172.21.15.69:0/4127067776 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2019b3b0 con 0x7fae2019a820 2024-04-24T09:11:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.924+0000 7fae1ea00640 1 --2- 172.21.15.69:0/4127067776 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae20104450 0x7fae201a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:11:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.924+0000 7fae1f400640 1 --2- 172.21.15.69:0/4127067776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae2019a820 0x7fae2019acd0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fae1800b970 tx=0x7fae1800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.924+0000 7fae17e00640 1 -- 172.21.15.69:0/4127067776 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18019070 con 0x7fae2019a820 2024-04-24T09:11:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.924+0000 7fae17e00640 1 -- 172.21.15.69:0/4127067776 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae18014440 con 0x7fae2019a820 2024-04-24T09:11:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.924+0000 7fae25a00640 1 -- 172.21.15.69:0/4127067776 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2019b6a0 con 0x7fae2019a820 2024-04-24T09:11:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.924+0000 7fae17e00640 1 -- 172.21.15.69:0/4127067776 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180075d0 con 0x7fae2019a820 2024-04-24T09:11:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.924+0000 7fae25a00640 1 -- 172.21.15.69:0/4127067776 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae201a7830 con 0x7fae2019a820 2024-04-24T09:11:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.928+0000 7fae28764740 1 Processor -- start 2024-04-24T09:11:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.928+0000 7fae17e00640 1 -- 172.21.15.69:0/4127067776 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fae1801c070 con 0x7fae2019a820 2024-04-24T09:11:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.928+0000 7fae28764740 1 -- start start 2024-04-24T09:11:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.929+0000 7fae28764740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdd31b0 0x55bb4bdd3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.929+0000 7fae28764740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdd3ad0 0x55bb4bddde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.929+0000 7fae28764740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdde390 0x55bb4bde0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.929+0000 7fae28764740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bb4bd39900 con 0x55bb4bdd31b0 2024-04-24T09:11:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.929+0000 7fae28764740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55bb4bcdff20 con 0x55bb4bdde390 2024-04-24T09:11:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.929+0000 7fae28764740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55bb4bd358c0 con 0x55bb4bdd3ad0 2024-04-24T09:11:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.929+0000 7fae16000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdd31b0 0x55bb4bdd3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.929+0000 7fae16000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdd31b0 0x55bb4bdd3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38498/0 (socket says 172.21.15.69:38498) 2024-04-24T09:11:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.929+0000 7fae16000640 1 -- 172.21.15.69:0/2136177156 learned_addr learned my addr 172.21.15.69:0/2136177156 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.929+0000 7fae1ea00640 1 --2- 172.21.15.69:0/2136177156 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdd3ad0 0x55bb4bddde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.929+0000 7fae16000640 1 -- 172.21.15.69:0/2136177156 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdd3ad0 msgr2=0x55bb4bddde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.929+0000 7fae16000640 1 --2- 172.21.15.69:0/2136177156 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdd3ad0 0x55bb4bddde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.929+0000 7fae16000640 1 -- 172.21.15.69:0/2136177156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdde390 msgr2=0x55bb4bde0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.929+0000 7fae16000640 1 --2- 172.21.15.69:0/2136177156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdde390 0x55bb4bde0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.929+0000 7fae16000640 1 -- 172.21.15.69:0/2136177156 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4bd04e10 con 0x55bb4bdd31b0 2024-04-24T09:11:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.930+0000 7fae16000640 1 --2- 172.21.15.69:0/2136177156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdd31b0 0x55bb4bdd3590 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fae0800bcb0 tx=0x7fae08011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cb5648b7eea730 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.930+0000 7fae17e00640 1 --2- 172.21.15.69:0/4127067776 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fae040a3dc0 0x7fae040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.930+0000 7fae0fe00640 1 -- 172.21.15.69:0/2136177156 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0801a400 con 0x55bb4bdd31b0 2024-04-24T09:11:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.930+0000 7fae0fe00640 1 -- 172.21.15.69:0/2136177156 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae0801aa50 con 0x55bb4bdd31b0 2024-04-24T09:11:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.930+0000 7fae17e00640 1 -- 172.21.15.69:0/4127067776 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fae180d00d0 con 0x7fae2019a820 2024-04-24T09:11:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.930+0000 7fae0fe00640 1 -- 172.21.15.69:0/2136177156 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08012930 con 0x55bb4bdd31b0 2024-04-24T09:11:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.930+0000 7fae1ea00640 1 --2- 172.21.15.69:0/4127067776 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fae040a3dc0 0x7fae040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.931+0000 7fae28764740 1 -- 172.21.15.69:0/2136177156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdd31b0 msgr2=0x55bb4bdd3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.931+0000 7fae1ea00640 1 --2- 172.21.15.69:0/4127067776 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fae040a3dc0 0x7fae040a6280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fae10005ab0 tx=0x7fae10032cc0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.931+0000 7fae28764740 1 --2- 172.21.15.69:0/2136177156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdd31b0 0x55bb4bdd3590 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fae0800bcb0 tx=0x7fae08011750 comp rx=0 tx=0).stop 2024-04-24T09:11:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.931+0000 7fae28764740 1 -- 172.21.15.69:0/2136177156 shutdown_connections 2024-04-24T09:11:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.931+0000 7fae28764740 1 --2- 172.21.15.69:0/2136177156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdde390 0x55bb4bde0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.931+0000 7fae28764740 1 --2- 172.21.15.69:0/2136177156 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdd3ad0 0x55bb4bddde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.931+0000 7fae28764740 1 --2- 172.21.15.69:0/2136177156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdd31b0 0x55bb4bdd3590 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.931+0000 7fae28764740 1 -- 172.21.15.69:0/2136177156 >> 172.21.15.69:0/2136177156 conn(0x55bb4bdcf6c0 msgr2=0x55bb4bdcfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.931+0000 7fae28764740 1 -- 172.21.15.69:0/2136177156 shutdown_connections 2024-04-24T09:11:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.931+0000 7fae28764740 1 -- 172.21.15.69:0/2136177156 wait complete. 2024-04-24T09:11:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.932+0000 7fae28764740 1 -- start start 2024-04-24T09:11:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.932+0000 7fae28764740 1 Processor -- start 2024-04-24T09:11:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.932+0000 7fae28764740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdd31b0 0x55bb4bddb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.932+0000 7fae28764740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdd3ad0 0x55bb4bddb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.932+0000 7fae28764740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdde390 0x55bb4bddbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.932+0000 7fae28764740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bb4bde0fc0 con 0x55bb4bdde390 2024-04-24T09:11:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.932+0000 7fae28764740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55bb4bcf32c0 con 0x55bb4bdd3ad0 2024-04-24T09:11:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.932+0000 7fae28764740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55bb4bde1140 con 0x55bb4bdd31b0 2024-04-24T09:11:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.932+0000 7fae28764740 10 client.0 _ll_register_callbacks cb 0x55bb4ba4ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.933+0000 7fae1f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdde390 0x55bb4bddbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.933+0000 7fae1f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdde390 0x55bb4bddbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38500/0 (socket says 172.21.15.69:38500) 2024-04-24T09:11:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.933+0000 7fae1f400640 1 -- 172.21.15.69:0/1484808479 learned_addr learned my addr 172.21.15.69:0/1484808479 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.933+0000 7fae16000640 1 --2- 172.21.15.69:0/1484808479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdd3ad0 0x55bb4bddb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.933+0000 7fae1ea00640 1 --2- 172.21.15.69:0/1484808479 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdd31b0 0x55bb4bddb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.933+0000 7fae1f400640 1 -- 172.21.15.69:0/1484808479 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdd31b0 msgr2=0x55bb4bddb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.933+0000 7fae1f400640 1 --2- 172.21.15.69:0/1484808479 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdd31b0 0x55bb4bddb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.933+0000 7fae1f400640 1 -- 172.21.15.69:0/1484808479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdd3ad0 msgr2=0x55bb4bddb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.933+0000 7fae1f400640 1 --2- 172.21.15.69:0/1484808479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdd3ad0 0x55bb4bddb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.933+0000 7fae1f400640 1 -- 172.21.15.69:0/1484808479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4bd03fd0 con 0x55bb4bdde390 2024-04-24T09:11:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.933+0000 7fae16000640 1 --2- 172.21.15.69:0/1484808479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdd3ad0 0x55bb4bddb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:11:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.933+0000 7fae1ea00640 1 --2- 172.21.15.69:0/1484808479 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdd31b0 0x55bb4bddb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:11:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.933+0000 7fae1f400640 1 --2- 172.21.15.69:0/1484808479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdde390 0x55bb4bddbd00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fae180987b0 tx=0x7fae18014aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa68c33c5e3320fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.933+0000 7fae0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:11:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.934+0000 7fae0cc00640 1 -- 172.21.15.69:0/1484808479 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180b5060 con 0x55bb4bdde390 2024-04-24T09:11:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.934+0000 7fae0cc00640 1 -- 172.21.15.69:0/1484808479 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fae180145f0 con 0x55bb4bdde390 2024-04-24T09:11:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.934+0000 7fae28764740 1 -- 172.21.15.69:0/1484808479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae0801aa50 con 0x55bb4bdde390 2024-04-24T09:11:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.934+0000 7fae0cc00640 1 -- 172.21.15.69:0/1484808479 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fae1809f070 con 0x55bb4bdde390 2024-04-24T09:11:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.934+0000 7fae28764740 10 client.16650 fetch_fsmap learned FSMap version 29 2024-04-24T09:11:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.934+0000 7fae28764740 1 -- 172.21.15.69:0/1484808479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb4ba4c210 con 0x55bb4bdde390 2024-04-24T09:11:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.934+0000 7fae0cc00640 1 -- 172.21.15.69:0/1484808479 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae0801aa50 con 0x55bb4bdde390 2024-04-24T09:11:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.934+0000 7fae0cc00640 1 -- 172.21.15.69:0/1484808479 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fae180a45c0 con 0x55bb4bdde390 2024-04-24T09:11:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.934+0000 7fae28764740 10 client.16650 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:11:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.934+0000 7fae28764740 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.935+0000 7fae28764740 20 client.16650 populate_metadata read hostname 'smithi069' 2024-04-24T09:11:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.935+0000 7fae28764740 1 --2- 172.21.15.69:0/1484808479 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55bb4bdee4e0 0x55bb4bdf0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.935+0000 7fae28764740 4 client.16650 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.935+0000 7fae28764740 1 -- 172.21.15.69:0/1484808479 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb4bd47320 con 0x55bb4bdee4e0 2024-04-24T09:11:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.935+0000 7fae1ea00640 1 --2- 172.21.15.69:0/1484808479 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55bb4bdee4e0 0x55bb4bdf0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.936+0000 7fae1ea00640 1 --2- 172.21.15.69:0/1484808479 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55bb4bdee4e0 0x55bb4bdf0980 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dcc2ad556f16b7f server_cookie=96b356ef555615db in_seq=0 out_seq=0 2024-04-24T09:11:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.936+0000 7fae0cc00640 10 client.16650 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.938+0000 7fae0cc00640 1 -- 172.21.15.69:0/1484808479 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb4bd47320 con 0x55bb4bdee4e0 2024-04-24T09:11:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.938+0000 7fae0cc00640 10 client.16650 handle_command_reply: tid=0 2024-04-24T09:11:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.938+0000 7fae28764740 1 client.16650 shutdown 2024-04-24T09:11:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.938+0000 7fae28764740 1 -- 172.21.15.69:0/1484808479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdde390 msgr2=0x55bb4bddbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.938+0000 7fae28764740 1 --2- 172.21.15.69:0/1484808479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdde390 0x55bb4bddbd00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fae180987b0 tx=0x7fae18014aa0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.939+0000 7fae28764740 1 -- 172.21.15.69:0/1484808479 shutdown_connections 2024-04-24T09:11:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.939+0000 7fae28764740 1 --2- 172.21.15.69:0/1484808479 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55bb4bdee4e0 0x55bb4bdf0980 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.939+0000 7fae28764740 1 --2- 172.21.15.69:0/1484808479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdde390 0x55bb4bddbd00 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.939+0000 7fae28764740 1 --2- 172.21.15.69:0/1484808479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdd3ad0 0x55bb4bddb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.939+0000 7fae28764740 1 --2- 172.21.15.69:0/1484808479 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdd31b0 0x55bb4bddb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.939+0000 7fae0cc00640 0 client.16650 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.939+0000 7fae28764740 1 -- 172.21.15.69:0/1484808479 >> 172.21.15.69:0/1484808479 conn(0x55bb4bdcf6c0 msgr2=0x55bb4bdd53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.939+0000 7fae28764740 1 -- 172.21.15.69:0/1484808479 shutdown_connections 2024-04-24T09:11:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.939+0000 7fae28764740 1 -- 172.21.15.69:0/1484808479 wait complete. 2024-04-24T09:11:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.939+0000 7fae28764740 20 client.16650 trim_cache size 0 max 16384 2024-04-24T09:11:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.955+0000 7fae28764740 1 Processor -- start 2024-04-24T09:11:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.956+0000 7fae28764740 1 -- start start 2024-04-24T09:11:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.956+0000 7fae28764740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdd31b0 0x55bb4bde9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.956+0000 7fae28764740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdd3ad0 0x55bb4bddb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.956+0000 7fae28764740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdde390 0x55bb4bdf6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.956+0000 7fae28764740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fae201b3dc0 con 0x55bb4bdd31b0 2024-04-24T09:11:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.956+0000 7fae28764740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fae201b1600 con 0x55bb4bdd3ad0 2024-04-24T09:11:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.956+0000 7fae28764740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fae201af450 con 0x55bb4bdde390 2024-04-24T09:11:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.956+0000 7fae1ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdd31b0 0x55bb4bde9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.956+0000 7fae1ea00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdd31b0 0x55bb4bde9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38504/0 (socket says 172.21.15.69:38504) 2024-04-24T09:11:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.956+0000 7fae1ea00640 1 -- 172.21.15.69:0/247653194 learned_addr learned my addr 172.21.15.69:0/247653194 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.957+0000 7fae1ea00640 1 -- 172.21.15.69:0/247653194 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdde390 msgr2=0x55bb4bdf6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.957+0000 7fae1f400640 1 --2- 172.21.15.69:0/247653194 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdde390 0x55bb4bdf6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.957+0000 7fae1ea00640 1 --2- 172.21.15.69:0/247653194 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdde390 0x55bb4bdf6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.957+0000 7fae1ea00640 1 -- 172.21.15.69:0/247653194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdd3ad0 msgr2=0x55bb4bddb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.957+0000 7fae1ea00640 1 --2- 172.21.15.69:0/247653194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdd3ad0 0x55bb4bddb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.957+0000 7fae1ea00640 1 -- 172.21.15.69:0/247653194 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4bb18620 con 0x55bb4bdd31b0 2024-04-24T09:11:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.957+0000 7fae1f400640 1 --2- 172.21.15.69:0/247653194 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdde390 0x55bb4bdf6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:11:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.957+0000 7fae1ea00640 1 --2- 172.21.15.69:0/247653194 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdd31b0 0x55bb4bde9e80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fae10009230 tx=0x7fae1003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e098ceae9df2c275 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.957+0000 7fae0f400640 1 -- 172.21.15.69:0/247653194 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10035050 con 0x55bb4bdd31b0 2024-04-24T09:11:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.957+0000 7fae0f400640 1 -- 172.21.15.69:0/247653194 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae10053070 con 0x55bb4bdd31b0 2024-04-24T09:11:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.957+0000 7fae0f400640 1 -- 172.21.15.69:0/247653194 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1004d510 con 0x55bb4bdd31b0 2024-04-24T09:11:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.958+0000 7fae28764740 1 -- 172.21.15.69:0/247653194 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdd31b0 msgr2=0x55bb4bde9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.958+0000 7fae28764740 1 --2- 172.21.15.69:0/247653194 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdd31b0 0x55bb4bde9e80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fae10009230 tx=0x7fae1003b000 comp rx=0 tx=0).stop 2024-04-24T09:11:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.958+0000 7fae28764740 1 -- 172.21.15.69:0/247653194 shutdown_connections 2024-04-24T09:11:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.958+0000 7fae28764740 1 --2- 172.21.15.69:0/247653194 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdde390 0x55bb4bdf6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.958+0000 7fae28764740 1 --2- 172.21.15.69:0/247653194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdd3ad0 0x55bb4bddb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.958+0000 7fae28764740 1 --2- 172.21.15.69:0/247653194 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdd31b0 0x55bb4bde9e80 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.958+0000 7fae28764740 1 -- 172.21.15.69:0/247653194 >> 172.21.15.69:0/247653194 conn(0x55bb4bdcf6c0 msgr2=0x55bb4bddb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.958+0000 7fae28764740 1 -- 172.21.15.69:0/247653194 shutdown_connections 2024-04-24T09:11:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.958+0000 7fae28764740 1 -- 172.21.15.69:0/247653194 wait complete. 2024-04-24T09:11:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.959+0000 7fae28764740 1 -- start start 2024-04-24T09:11:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.959+0000 7fae28764740 1 Processor -- start 2024-04-24T09:11:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.959+0000 7fae28764740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdd31b0 0x55bb4bde1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.959+0000 7fae28764740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdd3ad0 0x55bb4bde2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.959+0000 7fae28764740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdde390 0x55bb4bde2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.959+0000 7fae28764740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fae201afac0 con 0x55bb4bdde390 2024-04-24T09:11:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.959+0000 7fae28764740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fae201b0f30 con 0x55bb4bdd31b0 2024-04-24T09:11:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.959+0000 7fae28764740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fae201b0190 con 0x55bb4bdd3ad0 2024-04-24T09:11:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.959+0000 7fae28764740 10 client.0 _ll_register_callbacks cb 0x7fae18098080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.960+0000 7fae1f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdde390 0x55bb4bde2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.960+0000 7fae1f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdde390 0x55bb4bde2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38512/0 (socket says 172.21.15.69:38512) 2024-04-24T09:11:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.960+0000 7fae1f400640 1 -- 172.21.15.69:0/4104629673 learned_addr learned my addr 172.21.15.69:0/4104629673 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.960+0000 7fae1ea00640 1 --2- 172.21.15.69:0/4104629673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdd31b0 0x55bb4bde1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.960+0000 7fae16000640 1 --2- 172.21.15.69:0/4104629673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdd3ad0 0x55bb4bde2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.960+0000 7fae1f400640 1 -- 172.21.15.69:0/4104629673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdd3ad0 msgr2=0x55bb4bde2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.960+0000 7fae1f400640 1 --2- 172.21.15.69:0/4104629673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdd3ad0 0x55bb4bde2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.960+0000 7fae1f400640 1 -- 172.21.15.69:0/4104629673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdd31b0 msgr2=0x55bb4bde1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.960+0000 7fae1f400640 1 --2- 172.21.15.69:0/4104629673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdd31b0 0x55bb4bde1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.960+0000 7fae1f400640 1 -- 172.21.15.69:0/4104629673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4be53b10 con 0x55bb4bdde390 2024-04-24T09:11:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.960+0000 7fae1f400640 1 --2- 172.21.15.69:0/4104629673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdde390 0x55bb4bde2890 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fae18014800 tx=0x7fae18014830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a97de0b468dce3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.960+0000 7fae0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:11:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.960+0000 7fae0cc00640 1 -- 172.21.15.69:0/4104629673 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1809bbb0 con 0x55bb4bdde390 2024-04-24T09:11:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.960+0000 7fae0cc00640 1 -- 172.21.15.69:0/4104629673 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fae180b5060 con 0x55bb4bdde390 2024-04-24T09:11:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.960+0000 7fae28764740 1 -- 172.21.15.69:0/4104629673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae10053070 con 0x55bb4bdde390 2024-04-24T09:11:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.961+0000 7fae0cc00640 1 -- 172.21.15.69:0/4104629673 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fae180a4a50 con 0x55bb4bdde390 2024-04-24T09:11:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.961+0000 7fae28764740 10 client.16656 fetch_fsmap learned FSMap version 29 2024-04-24T09:11:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.961+0000 7fae28764740 1 -- 172.21.15.69:0/4104629673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb4be54d90 con 0x55bb4bdde390 2024-04-24T09:11:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.961+0000 7fae0cc00640 1 -- 172.21.15.69:0/4104629673 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae10053070 con 0x55bb4bdde390 2024-04-24T09:11:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.961+0000 7fae0cc00640 1 -- 172.21.15.69:0/4104629673 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fae180bc110 con 0x55bb4bdde390 2024-04-24T09:11:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.961+0000 7fae28764740 10 client.16656 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T09:11:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.961+0000 7fae28764740 10 client.16656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.961+0000 7fae28764740 20 client.16656 populate_metadata read hostname 'smithi069' 2024-04-24T09:11:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.961+0000 7fae28764740 1 --2- 172.21.15.69:0/4104629673 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55bb4be554e0 0x55bb4be57900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.961+0000 7fae28764740 4 client.16656 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:11:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.961+0000 7fae28764740 1 -- 172.21.15.69:0/4104629673 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb4bd41190 con 0x55bb4be554e0 2024-04-24T09:11:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.962+0000 7fae1ea00640 1 --2- 172.21.15.69:0/4104629673 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55bb4be554e0 0x55bb4be57900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.963+0000 7fae1ea00640 1 --2- 172.21.15.69:0/4104629673 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55bb4be554e0 0x55bb4be57900 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61780db31bb83af7 server_cookie=fb06ab44f995a492 in_seq=0 out_seq=0 2024-04-24T09:11:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:00.963+0000 7fae0cc00640 10 client.16656 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.045+0000 7fae0cc00640 1 -- 172.21.15.69:0/4104629673 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb4bd41190 con 0x55bb4be554e0 2024-04-24T09:11:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.045+0000 7fae0cc00640 10 client.16656 handle_command_reply: tid=0 2024-04-24T09:11:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.045+0000 7fae28764740 1 client.16656 shutdown 2024-04-24T09:11:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.045+0000 7fae28764740 1 -- 172.21.15.69:0/4104629673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdde390 msgr2=0x55bb4bde2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.045+0000 7fae28764740 1 --2- 172.21.15.69:0/4104629673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdde390 0x55bb4bde2890 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fae18014800 tx=0x7fae18014830 comp rx=0 tx=0).stop 2024-04-24T09:11:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.045+0000 7fae28764740 1 -- 172.21.15.69:0/4104629673 shutdown_connections 2024-04-24T09:11:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.045+0000 7fae28764740 1 --2- 172.21.15.69:0/4104629673 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55bb4be554e0 0x55bb4be57900 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.045+0000 7fae28764740 1 --2- 172.21.15.69:0/4104629673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bb4bdde390 0x55bb4bde2890 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.045+0000 7fae28764740 1 --2- 172.21.15.69:0/4104629673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bb4bdd3ad0 0x55bb4bde2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.045+0000 7fae28764740 1 --2- 172.21.15.69:0/4104629673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb4bdd31b0 0x55bb4bde1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.045+0000 7fae0cc00640 0 client.16656 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.045+0000 7fae28764740 1 -- 172.21.15.69:0/4104629673 >> 172.21.15.69:0/4104629673 conn(0x55bb4bdcf6c0 msgr2=0x55bb4bdce840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.046+0000 7fae28764740 1 -- 172.21.15.69:0/4104629673 shutdown_connections 2024-04-24T09:11:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.046+0000 7fae28764740 1 -- 172.21.15.69:0/4104629673 wait complete. 2024-04-24T09:11:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.046+0000 7fae28764740 20 client.16656 trim_cache size 0 max 16384 2024-04-24T09:11:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:11:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:11:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:11:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:11:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.047+0000 7fae25a00640 1 -- 172.21.15.69:0/4127067776 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fae040a3dc0 msgr2=0x7fae040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.047+0000 7fae25a00640 1 --2- 172.21.15.69:0/4127067776 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fae040a3dc0 0x7fae040a6280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fae10005ab0 tx=0x7fae10032cc0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.047+0000 7fae25a00640 1 -- 172.21.15.69:0/4127067776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae2019a820 msgr2=0x7fae2019acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.047+0000 7fae25a00640 1 --2- 172.21.15.69:0/4127067776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae2019a820 0x7fae2019acd0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fae1800b970 tx=0x7fae1800be30 comp rx=0 tx=0).stop 2024-04-24T09:11:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.047+0000 7fae25a00640 1 -- 172.21.15.69:0/4127067776 shutdown_connections 2024-04-24T09:11:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.047+0000 7fae25a00640 1 --2- 172.21.15.69:0/4127067776 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fae040a3dc0 0x7fae040a6280 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.047+0000 7fae25a00640 1 --2- 172.21.15.69:0/4127067776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae2019a820 0x7fae2019acd0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.047+0000 7fae25a00640 1 --2- 172.21.15.69:0/4127067776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae20109520 0x7fae201a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.047+0000 7fae25a00640 1 --2- 172.21.15.69:0/4127067776 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae20104450 0x7fae201a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.047+0000 7fae25a00640 1 -- 172.21.15.69:0/4127067776 >> 172.21.15.69:0/4127067776 conn(0x7fae200ffcf0 msgr2=0x7fae20102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.047+0000 7fae25a00640 1 -- 172.21.15.69:0/4127067776 shutdown_connections 2024-04-24T09:11:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:01.047+0000 7fae25a00640 1 -- 172.21.15.69:0/4127067776 wait complete. 2024-04-24T09:11:02.007 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:01 smithi130 ceph-mon[31553]: pgmap v121: 129 pgs: 129 active+clean; 61 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 860 KiB/s wr, 153 op/s 2024-04-24T09:11:02.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:01 smithi081 ceph-mon[30961]: pgmap v121: 129 pgs: 129 active+clean; 61 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 860 KiB/s wr, 153 op/s 2024-04-24T09:11:02.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:01 smithi069 ceph-mon[28170]: pgmap v121: 129 pgs: 129 active+clean; 61 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 860 KiB/s wr, 153 op/s 2024-04-24T09:11:04.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:03 smithi081 ceph-mon[30961]: pgmap v122: 129 pgs: 129 active+clean; 61 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 565 KiB/s wr, 86 op/s 2024-04-24T09:11:04.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:03 smithi069 ceph-mon[28170]: pgmap v122: 129 pgs: 129 active+clean; 61 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 565 KiB/s wr, 86 op/s 2024-04-24T09:11:04.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:03 smithi130 ceph-mon[31553]: pgmap v122: 129 pgs: 129 active+clean; 61 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 565 KiB/s wr, 86 op/s 2024-04-24T09:11:05.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:04 smithi081 ceph-mon[30961]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:11:05.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:04 smithi069 ceph-mon[28170]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:11:05.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:04 smithi130 ceph-mon[31553]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:11:06.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:05 smithi081 ceph-mon[30961]: pgmap v123: 129 pgs: 129 active+clean; 61 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 565 KiB/s wr, 86 op/s 2024-04-24T09:11:06.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:05 smithi069 ceph-mon[28170]: pgmap v123: 129 pgs: 129 active+clean; 61 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 565 KiB/s wr, 86 op/s 2024-04-24T09:11:06.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:05 smithi130 ceph-mon[31553]: pgmap v123: 129 pgs: 129 active+clean; 61 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 565 KiB/s wr, 86 op/s 2024-04-24T09:11:08.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:07 smithi081 ceph-mon[30961]: pgmap v124: 129 pgs: 129 active+clean; 60 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 831 KiB/s wr, 173 op/s 2024-04-24T09:11:08.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:07 smithi069 ceph-mon[28170]: pgmap v124: 129 pgs: 129 active+clean; 60 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 831 KiB/s wr, 173 op/s 2024-04-24T09:11:08.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:07 smithi130 ceph-mon[31553]: pgmap v124: 129 pgs: 129 active+clean; 60 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 831 KiB/s wr, 173 op/s 2024-04-24T09:11:10.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:09 smithi081 ceph-mon[30961]: pgmap v125: 129 pgs: 129 active+clean; 60 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 545 KiB/s wr, 129 op/s 2024-04-24T09:11:10.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:09 smithi069 ceph-mon[28170]: pgmap v125: 129 pgs: 129 active+clean; 60 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 545 KiB/s wr, 129 op/s 2024-04-24T09:11:10.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:09 smithi130 ceph-mon[31553]: pgmap v125: 129 pgs: 129 active+clean; 60 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 545 KiB/s wr, 129 op/s 2024-04-24T09:11:11.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:10 smithi069 ceph-mon[28170]: pgmap v126: 129 pgs: 129 active+clean; 64 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 918 KiB/s wr, 184 op/s 2024-04-24T09:11:11.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:10 smithi130 ceph-mon[31553]: pgmap v126: 129 pgs: 129 active+clean; 64 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 918 KiB/s wr, 184 op/s 2024-04-24T09:11:11.316 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:10 smithi081 ceph-mon[30961]: pgmap v126: 129 pgs: 129 active+clean; 64 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 918 KiB/s wr, 184 op/s 2024-04-24T09:11:13.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:13 smithi069 ceph-mon[28170]: pgmap v127: 129 pgs: 129 active+clean; 64 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 639 KiB/s wr, 141 op/s 2024-04-24T09:11:13.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:13 smithi130 ceph-mon[31553]: pgmap v127: 129 pgs: 129 active+clean; 64 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 639 KiB/s wr, 141 op/s 2024-04-24T09:11:13.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:13 smithi081 ceph-mon[30961]: pgmap v127: 129 pgs: 129 active+clean; 64 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 639 KiB/s wr, 141 op/s 2024-04-24T09:11:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.198+0000 7f9c40a00640 1 -- 172.21.15.69:0/2328541065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c100e30 msgr2=0x7f9c3c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.198+0000 7f9c40a00640 1 --2- 172.21.15.69:0/2328541065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c100e30 0x7f9c3c101210 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009900 tx=0x7f9c2c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.198+0000 7f9c40a00640 1 -- 172.21.15.69:0/2328541065 shutdown_connections 2024-04-24T09:11:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.198+0000 7f9c40a00640 1 --2- 172.21.15.69:0/2328541065 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c3c10e9e0 0x7f9c3c110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.199+0000 7f9c40a00640 1 --2- 172.21.15.69:0/2328541065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c3c101750 0x7f9c3c10e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.199+0000 7f9c40a00640 1 --2- 172.21.15.69:0/2328541065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c100e30 0x7f9c3c101210 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.199+0000 7f9c40a00640 1 -- 172.21.15.69:0/2328541065 >> 172.21.15.69:0/2328541065 conn(0x7f9c3c078080 msgr2=0x7f9c3c078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.199+0000 7f9c40a00640 1 -- 172.21.15.69:0/2328541065 shutdown_connections 2024-04-24T09:11:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.199+0000 7f9c40a00640 1 -- 172.21.15.69:0/2328541065 wait complete. 2024-04-24T09:11:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.200+0000 7f9c40a00640 1 Processor -- start 2024-04-24T09:11:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.200+0000 7f9c40a00640 1 -- start start 2024-04-24T09:11:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.200+0000 7f9c40a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c100e30 0x7f9c3c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.200+0000 7f9c40a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c3c101750 0x7f9c3c1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.200+0000 7f9c40a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c3c10e9e0 0x7f9c3c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.200+0000 7f9c40a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c114470 con 0x7f9c3c100e30 2024-04-24T09:11:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.200+0000 7f9c40a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c1142f0 con 0x7f9c3c101750 2024-04-24T09:11:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.200+0000 7f9c40a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c1145f0 con 0x7f9c3c10e9e0 2024-04-24T09:11:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.200+0000 7f9c39e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c100e30 0x7f9c3c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.200+0000 7f9c39e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c100e30 0x7f9c3c1a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54246/0 (socket says 172.21.15.69:54246) 2024-04-24T09:11:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.200+0000 7f9c39e00640 1 -- 172.21.15.69:0/2999765887 learned_addr learned my addr 172.21.15.69:0/2999765887 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.201+0000 7f9c3a800640 1 --2- 172.21.15.69:0/2999765887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c3c10e9e0 0x7f9c3c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.201+0000 7f9c39e00640 1 -- 172.21.15.69:0/2999765887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c3c10e9e0 msgr2=0x7f9c3c19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.201+0000 7f9c39e00640 1 --2- 172.21.15.69:0/2999765887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c3c10e9e0 0x7f9c3c19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.201+0000 7f9c39e00640 1 -- 172.21.15.69:0/2999765887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c3c101750 msgr2=0x7f9c3c1a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.201+0000 7f9c39e00640 1 --2- 172.21.15.69:0/2999765887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c3c101750 0x7f9c3c1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.201+0000 7f9c39e00640 1 -- 172.21.15.69:0/2999765887 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c3c19af90 con 0x7f9c3c100e30 2024-04-24T09:11:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.201+0000 7f9c39e00640 1 --2- 172.21.15.69:0/2999765887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c100e30 0x7f9c3c1a0610 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c006010 tx=0x7f9c2c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.201+0000 7f9c32a00640 1 -- 172.21.15.69:0/2999765887 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c03f030 con 0x7f9c3c100e30 2024-04-24T09:11:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.201+0000 7f9c32a00640 1 -- 172.21.15.69:0/2999765887 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c2c0035f0 con 0x7f9c3c100e30 2024-04-24T09:11:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.202+0000 7f9c40a00640 1 -- 172.21.15.69:0/2999765887 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c3c19b220 con 0x7f9c3c100e30 2024-04-24T09:11:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.202+0000 7f9c32a00640 1 -- 172.21.15.69:0/2999765887 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c035070 con 0x7f9c3c100e30 2024-04-24T09:11:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.202+0000 7f9c40a00640 1 -- 172.21.15.69:0/2999765887 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c3c1a7590 con 0x7f9c3c100e30 2024-04-24T09:11:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.206+0000 7f9c437a1740 1 Processor -- start 2024-04-24T09:11:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.206+0000 7f9c437a1740 1 -- start start 2024-04-24T09:11:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.206+0000 7f9c32a00640 1 -- 172.21.15.69:0/2999765887 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9c2c003b10 con 0x7f9c3c100e30 2024-04-24T09:11:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.206+0000 7f9c437a1740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d48311b0 0x5608d4831590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.206+0000 7f9c437a1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d4831ad0 0x5608d483be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.206+0000 7f9c437a1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d483c390 0x5608d483e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.206+0000 7f9c437a1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5608d4797900 con 0x5608d4831ad0 2024-04-24T09:11:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.206+0000 7f9c437a1740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5608d473df20 con 0x5608d48311b0 2024-04-24T09:11:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.206+0000 7f9c437a1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5608d47938c0 con 0x5608d483c390 2024-04-24T09:11:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.207+0000 7f9c39e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d4831ad0 0x5608d483be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.207+0000 7f9c3a800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d48311b0 0x5608d4831590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.207+0000 7f9c39400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d483c390 0x5608d483e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.207+0000 7f9c39e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d4831ad0 0x5608d483be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54256/0 (socket says 172.21.15.69:54256) 2024-04-24T09:11:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.207+0000 7f9c39e00640 1 -- 172.21.15.69:0/4206544691 learned_addr learned my addr 172.21.15.69:0/4206544691 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.208+0000 7f9c39e00640 1 -- 172.21.15.69:0/4206544691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d483c390 msgr2=0x5608d483e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.208+0000 7f9c39e00640 1 --2- 172.21.15.69:0/4206544691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d483c390 0x5608d483e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.208+0000 7f9c39e00640 1 -- 172.21.15.69:0/4206544691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d48311b0 msgr2=0x5608d4831590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.208+0000 7f9c39e00640 1 --2- 172.21.15.69:0/4206544691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d48311b0 0x5608d4831590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.208+0000 7f9c39e00640 1 -- 172.21.15.69:0/4206544691 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608d4762e10 con 0x5608d4831ad0 2024-04-24T09:11:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.208+0000 7f9c39e00640 1 --2- 172.21.15.69:0/4206544691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d4831ad0 0x5608d483be50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0ee4a0 tx=0x7f9c2c0f9640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2578e02dafe6096 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.208+0000 7f9c32a00640 1 --2- 172.21.15.69:0/2999765887 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9c200a3e40 0x7f9c200a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.208+0000 7f9c2b000640 1 -- 172.21.15.69:0/4206544691 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0b9050 con 0x5608d4831ad0 2024-04-24T09:11:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.209+0000 7f9c32a00640 1 -- 172.21.15.69:0/2999765887 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9c2c0f11b0 con 0x7f9c3c100e30 2024-04-24T09:11:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.209+0000 7f9c2b000640 1 -- 172.21.15.69:0/4206544691 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c2c0bd4b0 con 0x5608d4831ad0 2024-04-24T09:11:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.209+0000 7f9c2b000640 1 -- 172.21.15.69:0/4206544691 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0c5380 con 0x5608d4831ad0 2024-04-24T09:11:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.209+0000 7f9c3a800640 1 --2- 172.21.15.69:0/2999765887 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9c200a3e40 0x7f9c200a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.209+0000 7f9c437a1740 1 -- 172.21.15.69:0/4206544691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d4831ad0 msgr2=0x5608d483be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.209+0000 7f9c437a1740 1 --2- 172.21.15.69:0/4206544691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d4831ad0 0x5608d483be50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0ee4a0 tx=0x7f9c2c0f9640 comp rx=0 tx=0).stop 2024-04-24T09:11:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.210+0000 7f9c3a800640 1 --2- 172.21.15.69:0/2999765887 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9c200a3e40 0x7f9c200a6300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c103010 tx=0x7f9c34007600 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.210+0000 7f9c437a1740 1 -- 172.21.15.69:0/4206544691 shutdown_connections 2024-04-24T09:11:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.210+0000 7f9c437a1740 1 --2- 172.21.15.69:0/4206544691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d483c390 0x5608d483e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.210+0000 7f9c437a1740 1 --2- 172.21.15.69:0/4206544691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d4831ad0 0x5608d483be50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.210+0000 7f9c437a1740 1 --2- 172.21.15.69:0/4206544691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d48311b0 0x5608d4831590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.210+0000 7f9c437a1740 1 -- 172.21.15.69:0/4206544691 >> 172.21.15.69:0/4206544691 conn(0x5608d482d6c0 msgr2=0x5608d482daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.210+0000 7f9c437a1740 1 -- 172.21.15.69:0/4206544691 shutdown_connections 2024-04-24T09:11:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.210+0000 7f9c437a1740 1 -- 172.21.15.69:0/4206544691 wait complete. 2024-04-24T09:11:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.210+0000 7f9c437a1740 1 -- start start 2024-04-24T09:11:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.210+0000 7f9c437a1740 1 Processor -- start 2024-04-24T09:11:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.211+0000 7f9c437a1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d48311b0 0x5608d4839310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.211+0000 7f9c437a1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d4831ad0 0x5608d4839850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.211+0000 7f9c437a1740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d483c390 0x5608d4839d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.211+0000 7f9c437a1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5608d483ed80 con 0x5608d4831ad0 2024-04-24T09:11:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.211+0000 7f9c437a1740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5608d47512c0 con 0x5608d483c390 2024-04-24T09:11:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.211+0000 7f9c437a1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5608d483ef00 con 0x5608d48311b0 2024-04-24T09:11:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.211+0000 7f9c437a1740 10 client.0 _ll_register_callbacks cb 0x5608d44a9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.211+0000 7f9c39400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d4831ad0 0x5608d4839850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.211+0000 7f9c39400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d4831ad0 0x5608d4839850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54264/0 (socket says 172.21.15.69:54264) 2024-04-24T09:11:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.211+0000 7f9c39400640 1 -- 172.21.15.69:0/3325250011 learned_addr learned my addr 172.21.15.69:0/3325250011 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.211+0000 7f9c3a800640 1 --2- 172.21.15.69:0/3325250011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d483c390 0x5608d4839d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.211+0000 7f9c39e00640 1 --2- 172.21.15.69:0/3325250011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d48311b0 0x5608d4839310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.212+0000 7f9c39400640 1 -- 172.21.15.69:0/3325250011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d48311b0 msgr2=0x5608d4839310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.212+0000 7f9c39400640 1 --2- 172.21.15.69:0/3325250011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d48311b0 0x5608d4839310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.212+0000 7f9c39400640 1 -- 172.21.15.69:0/3325250011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d483c390 msgr2=0x5608d4839d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.212+0000 7f9c39400640 1 --2- 172.21.15.69:0/3325250011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d483c390 0x5608d4839d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.212+0000 7f9c39400640 1 -- 172.21.15.69:0/3325250011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608d4761fd0 con 0x5608d4831ad0 2024-04-24T09:11:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.212+0000 7f9c39e00640 1 --2- 172.21.15.69:0/3325250011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d48311b0 0x5608d4839310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:11:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.212+0000 7f9c39400640 1 --2- 172.21.15.69:0/3325250011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d4831ad0 0x5608d4839850 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400f6c0 tx=0x7f9c2400fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a33663da054bfe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.212+0000 7f9c1f400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:11:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.212+0000 7f9c1f400640 1 -- 172.21.15.69:0/3325250011 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2401d060 con 0x5608d4831ad0 2024-04-24T09:11:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.212+0000 7f9c1f400640 1 -- 172.21.15.69:0/3325250011 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9c2400db20 con 0x5608d4831ad0 2024-04-24T09:11:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.212+0000 7f9c437a1740 1 -- 172.21.15.69:0/3325250011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c2c0bd4b0 con 0x5608d4831ad0 2024-04-24T09:11:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.212+0000 7f9c1f400640 1 -- 172.21.15.69:0/3325250011 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9c24024050 con 0x5608d4831ad0 2024-04-24T09:11:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.213+0000 7f9c437a1740 10 client.16668 fetch_fsmap learned FSMap version 30 2024-04-24T09:11:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.213+0000 7f9c437a1740 1 -- 172.21.15.69:0/3325250011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5608d44aa210 con 0x5608d4831ad0 2024-04-24T09:11:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.213+0000 7f9c1f400640 1 -- 172.21.15.69:0/3325250011 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c2c0bd4b0 con 0x5608d4831ad0 2024-04-24T09:11:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.213+0000 7f9c1f400640 1 -- 172.21.15.69:0/3325250011 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9c24016340 con 0x5608d4831ad0 2024-04-24T09:11:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.213+0000 7f9c437a1740 10 client.16668 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:11:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.213+0000 7f9c437a1740 10 client.16668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.213+0000 7f9c437a1740 20 client.16668 populate_metadata read hostname 'smithi069' 2024-04-24T09:11:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.213+0000 7f9c437a1740 1 --2- 172.21.15.69:0/3325250011 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5608d484c290 0x5608d484e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.213+0000 7f9c437a1740 4 client.16668 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.213+0000 7f9c437a1740 1 -- 172.21.15.69:0/3325250011 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608d47a5320 con 0x5608d484c290 2024-04-24T09:11:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.214+0000 7f9c39e00640 1 --2- 172.21.15.69:0/3325250011 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5608d484c290 0x5608d484e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.215+0000 7f9c39e00640 1 --2- 172.21.15.69:0/3325250011 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5608d484c290 0x5608d484e700 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed606248f053494 server_cookie=63abf7204f2c7dbb in_seq=0 out_seq=0 2024-04-24T09:11:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.215+0000 7f9c1f400640 10 client.16668 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.217+0000 7f9c1f400640 1 -- 172.21.15.69:0/3325250011 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608d47a5320 con 0x5608d484c290 2024-04-24T09:11:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.217+0000 7f9c1f400640 10 client.16668 handle_command_reply: tid=0 2024-04-24T09:11:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.217+0000 7f9c437a1740 1 client.16668 shutdown 2024-04-24T09:11:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.217+0000 7f9c437a1740 1 -- 172.21.15.69:0/3325250011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d4831ad0 msgr2=0x5608d4839850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.217+0000 7f9c437a1740 1 --2- 172.21.15.69:0/3325250011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d4831ad0 0x5608d4839850 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400f6c0 tx=0x7f9c2400fb80 comp rx=0 tx=0).stop 2024-04-24T09:11:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.217+0000 7f9c437a1740 1 -- 172.21.15.69:0/3325250011 shutdown_connections 2024-04-24T09:11:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.217+0000 7f9c437a1740 1 --2- 172.21.15.69:0/3325250011 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5608d484c290 0x5608d484e700 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.217+0000 7f9c437a1740 1 --2- 172.21.15.69:0/3325250011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d483c390 0x5608d4839d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.217+0000 7f9c437a1740 1 --2- 172.21.15.69:0/3325250011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d4831ad0 0x5608d4839850 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.217+0000 7f9c1f400640 0 client.16668 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.217+0000 7f9c437a1740 1 --2- 172.21.15.69:0/3325250011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d48311b0 0x5608d4839310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.217+0000 7f9c437a1740 1 -- 172.21.15.69:0/3325250011 >> 172.21.15.69:0/3325250011 conn(0x5608d482d6c0 msgr2=0x5608d4833440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.217+0000 7f9c437a1740 1 -- 172.21.15.69:0/3325250011 shutdown_connections 2024-04-24T09:11:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.217+0000 7f9c437a1740 1 -- 172.21.15.69:0/3325250011 wait complete. 2024-04-24T09:11:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.218+0000 7f9c437a1740 20 client.16668 trim_cache size 0 max 16384 2024-04-24T09:11:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.234+0000 7f9c437a1740 1 Processor -- start 2024-04-24T09:11:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.234+0000 7f9c437a1740 1 -- start start 2024-04-24T09:11:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.235+0000 7f9c437a1740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d48311b0 0x5608d4847bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.235+0000 7f9c437a1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d4831ad0 0x5608d48399e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.235+0000 7f9c437a1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d483c390 0x5608d4853fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.235+0000 7f9c437a1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c1b3d00 con 0x5608d483c390 2024-04-24T09:11:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.235+0000 7f9c437a1740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c1b1570 con 0x5608d48311b0 2024-04-24T09:11:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.235+0000 7f9c437a1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c1af3c0 con 0x5608d4831ad0 2024-04-24T09:11:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.235+0000 7f9c3a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d483c390 0x5608d4853fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.235+0000 7f9c3a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d483c390 0x5608d4853fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54266/0 (socket says 172.21.15.69:54266) 2024-04-24T09:11:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.235+0000 7f9c3a800640 1 -- 172.21.15.69:0/4119456679 learned_addr learned my addr 172.21.15.69:0/4119456679 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.236+0000 7f9c39400640 1 --2- 172.21.15.69:0/4119456679 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d4831ad0 0x5608d48399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.236+0000 7f9c39e00640 1 --2- 172.21.15.69:0/4119456679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d48311b0 0x5608d4847bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.236+0000 7f9c3a800640 1 -- 172.21.15.69:0/4119456679 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d4831ad0 msgr2=0x5608d48399e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.236+0000 7f9c3a800640 1 --2- 172.21.15.69:0/4119456679 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d4831ad0 0x5608d48399e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.236+0000 7f9c3a800640 1 -- 172.21.15.69:0/4119456679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d48311b0 msgr2=0x5608d4847bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.236+0000 7f9c3a800640 1 --2- 172.21.15.69:0/4119456679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d48311b0 0x5608d4847bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.236+0000 7f9c3a800640 1 -- 172.21.15.69:0/4119456679 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608d4831900 con 0x5608d483c390 2024-04-24T09:11:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.236+0000 7f9c3a800640 1 --2- 172.21.15.69:0/4119456679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d483c390 0x5608d4853fd0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9c3401e520 tx=0x7f9c34014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777afe9d79d9f4d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.237+0000 7f9c2a600640 1 -- 172.21.15.69:0/4119456679 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3401c840 con 0x5608d483c390 2024-04-24T09:11:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.237+0000 7f9c2a600640 1 -- 172.21.15.69:0/4119456679 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c3401ce90 con 0x5608d483c390 2024-04-24T09:11:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.237+0000 7f9c2a600640 1 -- 172.21.15.69:0/4119456679 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400e3b0 con 0x5608d483c390 2024-04-24T09:11:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.239+0000 7f9c437a1740 1 -- 172.21.15.69:0/4119456679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d483c390 msgr2=0x5608d4853fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.239+0000 7f9c437a1740 1 --2- 172.21.15.69:0/4119456679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d483c390 0x5608d4853fd0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9c3401e520 tx=0x7f9c34014000 comp rx=0 tx=0).stop 2024-04-24T09:11:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.240+0000 7f9c437a1740 1 -- 172.21.15.69:0/4119456679 shutdown_connections 2024-04-24T09:11:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.240+0000 7f9c437a1740 1 --2- 172.21.15.69:0/4119456679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d483c390 0x5608d4853fd0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.240+0000 7f9c437a1740 1 --2- 172.21.15.69:0/4119456679 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d4831ad0 0x5608d48399e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.240+0000 7f9c437a1740 1 --2- 172.21.15.69:0/4119456679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d48311b0 0x5608d4847bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.240+0000 7f9c437a1740 1 -- 172.21.15.69:0/4119456679 >> 172.21.15.69:0/4119456679 conn(0x5608d482d6c0 msgr2=0x5608d48392d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.240+0000 7f9c437a1740 1 -- 172.21.15.69:0/4119456679 shutdown_connections 2024-04-24T09:11:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.240+0000 7f9c437a1740 1 -- 172.21.15.69:0/4119456679 wait complete. 2024-04-24T09:11:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.240+0000 7f9c437a1740 1 -- start start 2024-04-24T09:11:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.241+0000 7f9c437a1740 1 Processor -- start 2024-04-24T09:11:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.241+0000 7f9c437a1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d48311b0 0x5608d48440c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.241+0000 7f9c437a1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d4831ad0 0x5608d4844600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.241+0000 7f9c437a1740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d483c390 0x5608d4841090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.241+0000 7f9c437a1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c1afa30 con 0x5608d48311b0 2024-04-24T09:11:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.241+0000 7f9c437a1740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c1b0ea0 con 0x5608d483c390 2024-04-24T09:11:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.241+0000 7f9c437a1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c1b0100 con 0x5608d4831ad0 2024-04-24T09:11:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.242+0000 7f9c437a1740 10 client.0 _ll_register_callbacks cb 0x7f9c2400a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.242+0000 7f9c39e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d48311b0 0x5608d48440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.242+0000 7f9c39e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d48311b0 0x5608d48440c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54280/0 (socket says 172.21.15.69:54280) 2024-04-24T09:11:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.242+0000 7f9c39e00640 1 -- 172.21.15.69:0/3260804913 learned_addr learned my addr 172.21.15.69:0/3260804913 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.242+0000 7f9c3a800640 1 --2- 172.21.15.69:0/3260804913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d483c390 0x5608d4841090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.243+0000 7f9c39e00640 1 -- 172.21.15.69:0/3260804913 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d4831ad0 msgr2=0x5608d4844600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.243+0000 7f9c39e00640 1 --2- 172.21.15.69:0/3260804913 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d4831ad0 0x5608d4844600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.243+0000 7f9c39e00640 1 -- 172.21.15.69:0/3260804913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d483c390 msgr2=0x5608d4841090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.243+0000 7f9c39e00640 1 --2- 172.21.15.69:0/3260804913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d483c390 0x5608d4841090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.243+0000 7f9c39e00640 1 -- 172.21.15.69:0/3260804913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608d4576620 con 0x5608d48311b0 2024-04-24T09:11:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.243+0000 7f9c39e00640 1 --2- 172.21.15.69:0/3260804913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d48311b0 0x5608d48440c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c031440 tx=0x7f9c2c0c5a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ae7ad8654da677 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.243+0000 7f9c1f400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:11:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.243+0000 7f9c1f400640 1 -- 172.21.15.69:0/3260804913 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c049070 con 0x5608d48311b0 2024-04-24T09:11:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.243+0000 7f9c1f400640 1 -- 172.21.15.69:0/3260804913 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9c2c0bae50 con 0x5608d48311b0 2024-04-24T09:11:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.243+0000 7f9c437a1740 1 -- 172.21.15.69:0/3260804913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c3401ce90 con 0x5608d48311b0 2024-04-24T09:11:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.244+0000 7f9c1f400640 1 -- 172.21.15.69:0/3260804913 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9c2c0c0070 con 0x5608d48311b0 2024-04-24T09:11:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.244+0000 7f9c437a1740 10 client.16674 fetch_fsmap learned FSMap version 30 2024-04-24T09:11:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.244+0000 7f9c437a1740 1 -- 172.21.15.69:0/3260804913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5608d48b3100 con 0x5608d48311b0 2024-04-24T09:11:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.244+0000 7f9c1f400640 1 -- 172.21.15.69:0/3260804913 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c3401ce90 con 0x5608d48311b0 2024-04-24T09:11:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.244+0000 7f9c1f400640 1 -- 172.21.15.69:0/3260804913 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9c2c0a7040 con 0x5608d48311b0 2024-04-24T09:11:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.245+0000 7f9c437a1740 10 client.16674 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:11:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.245+0000 7f9c437a1740 10 client.16674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.245+0000 7f9c437a1740 20 client.16674 populate_metadata read hostname 'smithi069' 2024-04-24T09:11:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.245+0000 7f9c437a1740 1 --2- 172.21.15.69:0/3260804913 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5608d48b3740 0x5608d48b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.245+0000 7f9c437a1740 4 client.16674 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:11:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.245+0000 7f9c437a1740 1 -- 172.21.15.69:0/3260804913 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608d479f190 con 0x5608d48b3740 2024-04-24T09:11:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.245+0000 7f9c39400640 1 --2- 172.21.15.69:0/3260804913 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5608d48b3740 0x5608d48b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.247+0000 7f9c39400640 1 --2- 172.21.15.69:0/3260804913 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5608d48b3740 0x5608d48b5b60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=187728a58c3ecb16 server_cookie=6bc92a0918aa7868 in_seq=0 out_seq=0 2024-04-24T09:11:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.247+0000 7f9c1f400640 10 client.16674 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.386+0000 7f9c1f400640 1 -- 172.21.15.69:0/3260804913 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5608d479f190 con 0x5608d48b3740 2024-04-24T09:11:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.386+0000 7f9c1f400640 10 client.16674 handle_command_reply: tid=0 2024-04-24T09:11:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.386+0000 7f9c437a1740 1 client.16674 shutdown 2024-04-24T09:11:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.386+0000 7f9c437a1740 1 -- 172.21.15.69:0/3260804913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d48311b0 msgr2=0x5608d48440c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.386+0000 7f9c437a1740 1 --2- 172.21.15.69:0/3260804913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d48311b0 0x5608d48440c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c031440 tx=0x7f9c2c0c5a20 comp rx=0 tx=0).stop 2024-04-24T09:11:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.386+0000 7f9c437a1740 1 -- 172.21.15.69:0/3260804913 shutdown_connections 2024-04-24T09:11:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.386+0000 7f9c437a1740 1 --2- 172.21.15.69:0/3260804913 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5608d48b3740 0x5608d48b5b60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.386+0000 7f9c437a1740 1 --2- 172.21.15.69:0/3260804913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608d483c390 0x5608d4841090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.386+0000 7f9c437a1740 1 --2- 172.21.15.69:0/3260804913 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5608d4831ad0 0x5608d4844600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.386+0000 7f9c437a1740 1 --2- 172.21.15.69:0/3260804913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608d48311b0 0x5608d48440c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.386+0000 7f9c1f400640 0 client.16674 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.386+0000 7f9c437a1740 1 -- 172.21.15.69:0/3260804913 >> 172.21.15.69:0/3260804913 conn(0x5608d482d6c0 msgr2=0x5608d482c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.387+0000 7f9c437a1740 1 -- 172.21.15.69:0/3260804913 shutdown_connections 2024-04-24T09:11:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.387+0000 7f9c437a1740 1 -- 172.21.15.69:0/3260804913 wait complete. 2024-04-24T09:11:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.387+0000 7f9c437a1740 20 client.16674 trim_cache size 0 max 16384 2024-04-24T09:11:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:11:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:11:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:11:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:11:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.387+0000 7f9c40a00640 1 -- 172.21.15.69:0/2999765887 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9c200a3e40 msgr2=0x7f9c200a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.388+0000 7f9c40a00640 1 --2- 172.21.15.69:0/2999765887 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9c200a3e40 0x7f9c200a6300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c103010 tx=0x7f9c34007600 comp rx=0 tx=0).stop 2024-04-24T09:11:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.388+0000 7f9c40a00640 1 -- 172.21.15.69:0/2999765887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c100e30 msgr2=0x7f9c3c1a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.388+0000 7f9c40a00640 1 --2- 172.21.15.69:0/2999765887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c100e30 0x7f9c3c1a0610 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c006010 tx=0x7f9c2c004270 comp rx=0 tx=0).stop 2024-04-24T09:11:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.388+0000 7f9c40a00640 1 -- 172.21.15.69:0/2999765887 shutdown_connections 2024-04-24T09:11:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.388+0000 7f9c40a00640 1 --2- 172.21.15.69:0/2999765887 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9c200a3e40 0x7f9c200a6300 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.388+0000 7f9c40a00640 1 --2- 172.21.15.69:0/2999765887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9c3c10e9e0 0x7f9c3c19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.388+0000 7f9c40a00640 1 --2- 172.21.15.69:0/2999765887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c3c101750 0x7f9c3c1a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.388+0000 7f9c40a00640 1 --2- 172.21.15.69:0/2999765887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c100e30 0x7f9c3c1a0610 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.388+0000 7f9c40a00640 1 -- 172.21.15.69:0/2999765887 >> 172.21.15.69:0/2999765887 conn(0x7f9c3c078080 msgr2=0x7f9c3c10fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.388+0000 7f9c40a00640 1 -- 172.21.15.69:0/2999765887 shutdown_connections 2024-04-24T09:11:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:14.388+0000 7f9c40a00640 1 -- 172.21.15.69:0/2999765887 wait complete. 2024-04-24T09:11:15.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:15 smithi069 ceph-mon[28170]: pgmap v128: 129 pgs: 129 active+clean; 64 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 639 KiB/s wr, 141 op/s 2024-04-24T09:11:15.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:15 smithi130 ceph-mon[31553]: pgmap v128: 129 pgs: 129 active+clean; 64 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 639 KiB/s wr, 141 op/s 2024-04-24T09:11:15.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:15 smithi081 ceph-mon[30961]: pgmap v128: 129 pgs: 129 active+clean; 64 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 639 KiB/s wr, 141 op/s 2024-04-24T09:11:17.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:17 smithi069 ceph-mon[28170]: pgmap v129: 129 pgs: 129 active+clean; 66 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 868 KiB/s wr, 250 op/s 2024-04-24T09:11:17.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:17 smithi130 ceph-mon[31553]: pgmap v129: 129 pgs: 129 active+clean; 66 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 868 KiB/s wr, 250 op/s 2024-04-24T09:11:17.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:17 smithi081 ceph-mon[30961]: pgmap v129: 129 pgs: 129 active+clean; 66 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 868 KiB/s wr, 250 op/s 2024-04-24T09:11:19.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:19 smithi130 ceph-mon[31553]: pgmap v130: 129 pgs: 129 active+clean; 66 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 602 KiB/s wr, 163 op/s 2024-04-24T09:11:19.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:19 smithi081 ceph-mon[30961]: pgmap v130: 129 pgs: 129 active+clean; 66 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 602 KiB/s wr, 163 op/s 2024-04-24T09:11:19.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:19 smithi069 ceph-mon[28170]: pgmap v130: 129 pgs: 129 active+clean; 66 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 602 KiB/s wr, 163 op/s 2024-04-24T09:11:20.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:20 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:11:20.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:20 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:11:20.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:20 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:11:21.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:21 smithi130 ceph-mon[31553]: pgmap v131: 129 pgs: 129 active+clean; 68 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 814 KiB/s wr, 210 op/s 2024-04-24T09:11:21.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:21 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:11:21.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:21 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:11:21.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:21 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:11:21.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:21 smithi081 ceph-mon[30961]: pgmap v131: 129 pgs: 129 active+clean; 68 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 814 KiB/s wr, 210 op/s 2024-04-24T09:11:21.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:21 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:11:21.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:21 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:11:21.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:21 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:11:21.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:21 smithi069 ceph-mon[28170]: pgmap v131: 129 pgs: 129 active+clean; 68 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 814 KiB/s wr, 210 op/s 2024-04-24T09:11:21.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:21 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:11:21.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:21 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:11:21.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:21 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:11:23.393 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:23 smithi069 ceph-mon[28170]: pgmap v132: 129 pgs: 129 active+clean; 68 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 441 KiB/s wr, 156 op/s 2024-04-24T09:11:23.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:23 smithi130 ceph-mon[31553]: pgmap v132: 129 pgs: 129 active+clean; 68 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 441 KiB/s wr, 156 op/s 2024-04-24T09:11:23.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:23 smithi081 ceph-mon[30961]: pgmap v132: 129 pgs: 129 active+clean; 68 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 441 KiB/s wr, 156 op/s 2024-04-24T09:11:25.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:25 smithi130 ceph-mon[31553]: pgmap v133: 129 pgs: 129 active+clean; 68 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 441 KiB/s wr, 156 op/s 2024-04-24T09:11:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:25 smithi081 ceph-mon[30961]: pgmap v133: 129 pgs: 129 active+clean; 68 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 441 KiB/s wr, 156 op/s 2024-04-24T09:11:25.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:25 smithi069 ceph-mon[28170]: pgmap v133: 129 pgs: 129 active+clean; 68 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 441 KiB/s wr, 156 op/s 2024-04-24T09:11:27.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:27 smithi130 ceph-mon[31553]: pgmap v134: 129 pgs: 129 active+clean; 71 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 666 KiB/s wr, 201 op/s 2024-04-24T09:11:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.534+0000 7feb45a00640 1 -- 172.21.15.69:0/787939071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb40102c30 msgr2=0x7feb40106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.534+0000 7feb45a00640 1 --2- 172.21.15.69:0/787939071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb40102c30 0x7feb40106c70 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009900 tx=0x7feb2c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.534+0000 7feb45a00640 1 -- 172.21.15.69:0/787939071 shutdown_connections 2024-04-24T09:11:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.534+0000 7feb45a00640 1 --2- 172.21.15.69:0/787939071 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb40107350 0x7feb4010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.534+0000 7feb45a00640 1 --2- 172.21.15.69:0/787939071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb40102c30 0x7feb40106c70 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.534+0000 7feb45a00640 1 --2- 172.21.15.69:0/787939071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb40102280 0x7feb40102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.534+0000 7feb45a00640 1 -- 172.21.15.69:0/787939071 >> 172.21.15.69:0/787939071 conn(0x7feb40079490 msgr2=0x7feb400798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.534+0000 7feb45a00640 1 -- 172.21.15.69:0/787939071 shutdown_connections 2024-04-24T09:11:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.534+0000 7feb45a00640 1 -- 172.21.15.69:0/787939071 wait complete. 2024-04-24T09:11:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.535+0000 7feb45a00640 1 Processor -- start 2024-04-24T09:11:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.536+0000 7feb45a00640 1 -- start start 2024-04-24T09:11:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.536+0000 7feb45a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb40102280 0x7feb401987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.536+0000 7feb45a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb40107350 0x7feb40198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.536+0000 7feb45a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb4019d010 0x7feb4019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.536+0000 7feb45a00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feb401122e0 con 0x7feb4019d010 2024-04-24T09:11:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.536+0000 7feb45a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7feb40112160 con 0x7feb40107350 2024-04-24T09:11:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.536+0000 7feb45a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7feb40112460 con 0x7feb40102280 2024-04-24T09:11:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.536+0000 7feb3f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb4019d010 0x7feb4019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.536+0000 7feb3f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb4019d010 0x7feb4019d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56058/0 (socket says 172.21.15.69:56058) 2024-04-24T09:11:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.536+0000 7feb3f400640 1 -- 172.21.15.69:0/170463453 learned_addr learned my addr 172.21.15.69:0/170463453 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.537+0000 7feb3ea00640 1 --2- 172.21.15.69:0/170463453 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb40102280 0x7feb401987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.537+0000 7feb3e000640 1 --2- 172.21.15.69:0/170463453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb40107350 0x7feb40198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.537+0000 7feb3f400640 1 -- 172.21.15.69:0/170463453 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb40102280 msgr2=0x7feb401987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.537+0000 7feb3f400640 1 --2- 172.21.15.69:0/170463453 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb40102280 0x7feb401987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.537+0000 7feb3f400640 1 -- 172.21.15.69:0/170463453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb40107350 msgr2=0x7feb40198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.537+0000 7feb3f400640 1 --2- 172.21.15.69:0/170463453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb40107350 0x7feb40198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.537+0000 7feb3f400640 1 -- 172.21.15.69:0/170463453 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4019db00 con 0x7feb4019d010 2024-04-24T09:11:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.537+0000 7feb3f400640 1 --2- 172.21.15.69:0/170463453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb4019d010 0x7feb4019d4c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7feb3800e960 tx=0x7feb3800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.537+0000 7feb37400640 1 -- 172.21.15.69:0/170463453 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38004d20 con 0x7feb4019d010 2024-04-24T09:11:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.537+0000 7feb37400640 1 -- 172.21.15.69:0/170463453 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb38005020 con 0x7feb4019d010 2024-04-24T09:11:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.537+0000 7feb45a00640 1 -- 172.21.15.69:0/170463453 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb4019ddf0 con 0x7feb4019d010 2024-04-24T09:11:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.537+0000 7feb37400640 1 -- 172.21.15.69:0/170463453 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb380116d0 con 0x7feb4019d010 2024-04-24T09:11:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.537+0000 7feb45a00640 1 -- 172.21.15.69:0/170463453 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb401a55b0 con 0x7feb4019d010 2024-04-24T09:11:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.541+0000 7feb48787740 1 Processor -- start 2024-04-24T09:11:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.541+0000 7feb37400640 1 -- 172.21.15.69:0/170463453 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feb380051d0 con 0x7feb4019d010 2024-04-24T09:11:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.541+0000 7feb48787740 1 -- start start 2024-04-24T09:11:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.541+0000 7feb48787740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x560759537590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.541+0000 7feb48787740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560759537ad0 0x560759541e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.541+0000 7feb48787740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759542390 0x560759544780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.541+0000 7feb48787740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56075949d900 con 0x560759542390 2024-04-24T09:11:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.541+0000 7feb48787740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x560759443f20 con 0x5607595371b0 2024-04-24T09:11:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.541+0000 7feb48787740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5607594998c0 con 0x560759537ad0 2024-04-24T09:11:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.542+0000 7feb3e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759542390 0x560759544780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.542+0000 7feb3e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759542390 0x560759544780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56068/0 (socket says 172.21.15.69:56068) 2024-04-24T09:11:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.542+0000 7feb3e000640 1 -- 172.21.15.69:0/1031002448 learned_addr learned my addr 172.21.15.69:0/1031002448 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.542+0000 7feb3ea00640 1 --2- 172.21.15.69:0/1031002448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560759537ad0 0x560759541e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.542+0000 7feb3e000640 1 --2- 172.21.15.69:0/1031002448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x560759537590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.542+0000 7feb3e000640 1 -- 172.21.15.69:0/1031002448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560759537ad0 msgr2=0x560759541e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.542+0000 7feb3e000640 1 --2- 172.21.15.69:0/1031002448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560759537ad0 0x560759541e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.542+0000 7feb3e000640 1 -- 172.21.15.69:0/1031002448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 msgr2=0x560759537590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.542+0000 7feb3e000640 1 --2- 172.21.15.69:0/1031002448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x560759537590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.542+0000 7feb3e000640 1 -- 172.21.15.69:0/1031002448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560759468e10 con 0x560759542390 2024-04-24T09:11:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.543+0000 7feb37400640 1 --2- 172.21.15.69:0/170463453 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7feb240a3dc0 0x7feb240a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.543+0000 7feb3e000640 1 --2- 172.21.15.69:0/1031002448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759542390 0x560759544780 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0057c0 tx=0x7feb2c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29dab3b0032baf1b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.543+0000 7feb37400640 1 -- 172.21.15.69:0/170463453 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7feb380d0ff0 con 0x7feb4019d010 2024-04-24T09:11:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.543+0000 7feb2be00640 1 -- 172.21.15.69:0/1031002448 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c035040 con 0x560759542390 2024-04-24T09:11:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.543+0000 7feb3ea00640 1 --2- 172.21.15.69:0/170463453 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7feb240a3dc0 0x7feb240a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.543+0000 7feb2be00640 1 -- 172.21.15.69:0/1031002448 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb2c003de0 con 0x560759542390 2024-04-24T09:11:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.543+0000 7feb2be00640 1 -- 172.21.15.69:0/1031002448 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c03dca0 con 0x560759542390 2024-04-24T09:11:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.544+0000 7feb48787740 1 -- 172.21.15.69:0/1031002448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759542390 msgr2=0x560759544780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.544+0000 7feb48787740 1 --2- 172.21.15.69:0/1031002448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759542390 0x560759544780 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0057c0 tx=0x7feb2c000bc0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.544+0000 7feb3ea00640 1 --2- 172.21.15.69:0/170463453 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7feb240a3dc0 0x7feb240a6280 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x560759538750 tx=0x7feb3000b4d0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.544+0000 7feb48787740 1 -- 172.21.15.69:0/1031002448 shutdown_connections 2024-04-24T09:11:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.544+0000 7feb48787740 1 --2- 172.21.15.69:0/1031002448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759542390 0x560759544780 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.544+0000 7feb48787740 1 --2- 172.21.15.69:0/1031002448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560759537ad0 0x560759541e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.544+0000 7feb48787740 1 --2- 172.21.15.69:0/1031002448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x560759537590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.544+0000 7feb48787740 1 -- 172.21.15.69:0/1031002448 >> 172.21.15.69:0/1031002448 conn(0x5607595336c0 msgr2=0x560759533aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.544+0000 7feb48787740 1 -- 172.21.15.69:0/1031002448 shutdown_connections 2024-04-24T09:11:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.544+0000 7feb48787740 1 -- 172.21.15.69:0/1031002448 wait complete. 2024-04-24T09:11:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.545+0000 7feb48787740 1 -- start start 2024-04-24T09:11:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.545+0000 7feb48787740 1 Processor -- start 2024-04-24T09:11:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.545+0000 7feb48787740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x56075953f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.545+0000 7feb48787740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759537ad0 0x56075953f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.545+0000 7feb48787740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560759542390 0x56075953fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.545+0000 7feb48787740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560759544f60 con 0x560759537ad0 2024-04-24T09:11:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.545+0000 7feb48787740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5607594572c0 con 0x5607595371b0 2024-04-24T09:11:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.545+0000 7feb48787740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5607595450e0 con 0x560759542390 2024-04-24T09:11:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.545+0000 7feb48787740 10 client.0 _ll_register_callbacks cb 0x5607591afa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.546+0000 7feb3e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759537ad0 0x56075953f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.546+0000 7feb3e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759537ad0 0x56075953f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56080/0 (socket says 172.21.15.69:56080) 2024-04-24T09:11:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.546+0000 7feb3e000640 1 -- 172.21.15.69:0/2896643569 learned_addr learned my addr 172.21.15.69:0/2896643569 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.546+0000 7feb3f400640 1 --2- 172.21.15.69:0/2896643569 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560759542390 0x56075953fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.546+0000 7feb3ea00640 1 --2- 172.21.15.69:0/2896643569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x56075953f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.546+0000 7feb3e000640 1 -- 172.21.15.69:0/2896643569 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560759542390 msgr2=0x56075953fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.546+0000 7feb3e000640 1 --2- 172.21.15.69:0/2896643569 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560759542390 0x56075953fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.546+0000 7feb3e000640 1 -- 172.21.15.69:0/2896643569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 msgr2=0x56075953f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.546+0000 7feb3e000640 1 --2- 172.21.15.69:0/2896643569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x56075953f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.546+0000 7feb3e000640 1 -- 172.21.15.69:0/2896643569 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560759467fd0 con 0x560759537ad0 2024-04-24T09:11:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.546+0000 7feb3e000640 1 --2- 172.21.15.69:0/2896643569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759537ad0 0x56075953f910 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7feb2c006010 tx=0x7feb2c03d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e57e2e11e97dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.546+0000 7feb28c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:11:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.547+0000 7feb28c00640 1 -- 172.21.15.69:0/2896643569 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c04d070 con 0x560759537ad0 2024-04-24T09:11:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.547+0000 7feb28c00640 1 -- 172.21.15.69:0/2896643569 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7feb2c03dc20 con 0x560759537ad0 2024-04-24T09:11:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.547+0000 7feb48787740 1 -- 172.21.15.69:0/2896643569 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb2c003de0 con 0x560759537ad0 2024-04-24T09:11:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.547+0000 7feb28c00640 1 -- 172.21.15.69:0/2896643569 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feb2c059050 con 0x560759537ad0 2024-04-24T09:11:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.547+0000 7feb48787740 10 client.16686 fetch_fsmap learned FSMap version 30 2024-04-24T09:11:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.547+0000 7feb48787740 1 -- 172.21.15.69:0/2896643569 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607591b0210 con 0x560759537ad0 2024-04-24T09:11:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.547+0000 7feb28c00640 1 -- 172.21.15.69:0/2896643569 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb2c003de0 con 0x560759537ad0 2024-04-24T09:11:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.548+0000 7feb28c00640 1 -- 172.21.15.69:0/2896643569 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7feb2c03a040 con 0x560759537ad0 2024-04-24T09:11:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.548+0000 7feb48787740 10 client.16686 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:11:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.548+0000 7feb48787740 10 client.16686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.548+0000 7feb48787740 20 client.16686 populate_metadata read hostname 'smithi069' 2024-04-24T09:11:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.548+0000 7feb48787740 1 --2- 172.21.15.69:0/2896643569 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5607595523c0 0x5607595547b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.548+0000 7feb48787740 4 client.16686 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.548+0000 7feb48787740 1 -- 172.21.15.69:0/2896643569 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607594ab320 con 0x5607595523c0 2024-04-24T09:11:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.549+0000 7feb3ea00640 1 --2- 172.21.15.69:0/2896643569 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5607595523c0 0x5607595547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.550+0000 7feb3ea00640 1 --2- 172.21.15.69:0/2896643569 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5607595523c0 0x5607595547b0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8692e3d81139037 server_cookie=9cd5ce31be539cdf in_seq=0 out_seq=0 2024-04-24T09:11:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.550+0000 7feb28c00640 10 client.16686 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.552+0000 7feb28c00640 1 -- 172.21.15.69:0/2896643569 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5607594ab320 con 0x5607595523c0 2024-04-24T09:11:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.552+0000 7feb28c00640 10 client.16686 handle_command_reply: tid=0 2024-04-24T09:11:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.552+0000 7feb48787740 1 client.16686 shutdown 2024-04-24T09:11:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.552+0000 7feb48787740 1 -- 172.21.15.69:0/2896643569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759537ad0 msgr2=0x56075953f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.552+0000 7feb48787740 1 --2- 172.21.15.69:0/2896643569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759537ad0 0x56075953f910 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7feb2c006010 tx=0x7feb2c03d7b0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.552+0000 7feb48787740 1 -- 172.21.15.69:0/2896643569 shutdown_connections 2024-04-24T09:11:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.552+0000 7feb48787740 1 --2- 172.21.15.69:0/2896643569 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5607595523c0 0x5607595547b0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.552+0000 7feb48787740 1 --2- 172.21.15.69:0/2896643569 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560759542390 0x56075953fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.552+0000 7feb48787740 1 --2- 172.21.15.69:0/2896643569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759537ad0 0x56075953f910 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.552+0000 7feb28c00640 0 client.16686 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.552+0000 7feb48787740 1 --2- 172.21.15.69:0/2896643569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x56075953f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.552+0000 7feb48787740 1 -- 172.21.15.69:0/2896643569 >> 172.21.15.69:0/2896643569 conn(0x5607595336c0 msgr2=0x5607595393f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.552+0000 7feb48787740 1 -- 172.21.15.69:0/2896643569 shutdown_connections 2024-04-24T09:11:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.552+0000 7feb48787740 1 -- 172.21.15.69:0/2896643569 wait complete. 2024-04-24T09:11:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.552+0000 7feb48787740 20 client.16686 trim_cache size 0 max 16384 2024-04-24T09:11:27.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:27 smithi081 ceph-mon[30961]: pgmap v134: 129 pgs: 129 active+clean; 71 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 666 KiB/s wr, 201 op/s 2024-04-24T09:11:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.569+0000 7feb48787740 1 Processor -- start 2024-04-24T09:11:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.569+0000 7feb48787740 1 -- start start 2024-04-24T09:11:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.570+0000 7feb48787740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x560759557bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.570+0000 7feb48787740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759537ad0 0x560759532fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.570+0000 7feb48787740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560759542390 0x560759539b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.570+0000 7feb48787740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feb401b1ad0 con 0x560759537ad0 2024-04-24T09:11:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.570+0000 7feb48787740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7feb401af340 con 0x5607595371b0 2024-04-24T09:11:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.570+0000 7feb48787740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7feb401ad190 con 0x560759542390 2024-04-24T09:11:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.570+0000 7feb3e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759537ad0 0x560759532fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.570+0000 7feb3e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759537ad0 0x560759532fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56084/0 (socket says 172.21.15.69:56084) 2024-04-24T09:11:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.570+0000 7feb3e000640 1 -- 172.21.15.69:0/1990196409 learned_addr learned my addr 172.21.15.69:0/1990196409 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.570+0000 7feb3ea00640 1 --2- 172.21.15.69:0/1990196409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x560759557bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.570+0000 7feb3f400640 1 --2- 172.21.15.69:0/1990196409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560759542390 0x560759539b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.570+0000 7feb3e000640 1 -- 172.21.15.69:0/1990196409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560759542390 msgr2=0x560759539b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.571+0000 7feb3e000640 1 --2- 172.21.15.69:0/1990196409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560759542390 0x560759539b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.571+0000 7feb3e000640 1 -- 172.21.15.69:0/1990196409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 msgr2=0x560759557bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.571+0000 7feb3e000640 1 --2- 172.21.15.69:0/1990196409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x560759557bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.571+0000 7feb3e000640 1 -- 172.21.15.69:0/1990196409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56075927c620 con 0x560759537ad0 2024-04-24T09:11:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.571+0000 7feb3ea00640 1 --2- 172.21.15.69:0/1990196409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x560759557bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:11:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.571+0000 7feb3e000640 1 --2- 172.21.15.69:0/1990196409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759537ad0 0x560759532fc0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7feb2c049440 tx=0x7feb2c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62b7175c3a1aab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.571+0000 7feb2b400640 1 -- 172.21.15.69:0/1990196409 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c04d070 con 0x560759537ad0 2024-04-24T09:11:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.571+0000 7feb2b400640 1 -- 172.21.15.69:0/1990196409 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb2c003940 con 0x560759537ad0 2024-04-24T09:11:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.571+0000 7feb2b400640 1 -- 172.21.15.69:0/1990196409 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c035030 con 0x560759537ad0 2024-04-24T09:11:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.572+0000 7feb48787740 1 -- 172.21.15.69:0/1990196409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759537ad0 msgr2=0x560759532fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.572+0000 7feb48787740 1 --2- 172.21.15.69:0/1990196409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759537ad0 0x560759532fc0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7feb2c049440 tx=0x7feb2c000bc0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.572+0000 7feb48787740 1 -- 172.21.15.69:0/1990196409 shutdown_connections 2024-04-24T09:11:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.572+0000 7feb48787740 1 --2- 172.21.15.69:0/1990196409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560759542390 0x560759539b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.572+0000 7feb48787740 1 --2- 172.21.15.69:0/1990196409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759537ad0 0x560759532fc0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.572+0000 7feb48787740 1 --2- 172.21.15.69:0/1990196409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x560759557bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.572+0000 7feb48787740 1 -- 172.21.15.69:0/1990196409 >> 172.21.15.69:0/1990196409 conn(0x5607595336c0 msgr2=0x56075954dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.572+0000 7feb48787740 1 -- 172.21.15.69:0/1990196409 shutdown_connections 2024-04-24T09:11:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.572+0000 7feb48787740 1 -- 172.21.15.69:0/1990196409 wait complete. 2024-04-24T09:11:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.572+0000 7feb48787740 1 -- start start 2024-04-24T09:11:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.573+0000 7feb48787740 1 Processor -- start 2024-04-24T09:11:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.573+0000 7feb48787740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x5607595b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.573+0000 7feb48787740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759542390 0x5607595b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.573+0000 7feb48787740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5607595b95c0 0x5607595bb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.573+0000 7feb48787740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feb401ad800 con 0x560759542390 2024-04-24T09:11:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.573+0000 7feb48787740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7feb401aec70 con 0x5607595371b0 2024-04-24T09:11:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.573+0000 7feb48787740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7feb401aded0 con 0x5607595b95c0 2024-04-24T09:11:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.573+0000 7feb48787740 10 client.0 _ll_register_callbacks cb 0x7feb2c004180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.573+0000 7feb3e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759542390 0x5607595b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.574+0000 7feb3e000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759542390 0x5607595b0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56090/0 (socket says 172.21.15.69:56090) 2024-04-24T09:11:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.574+0000 7feb3e000640 1 -- 172.21.15.69:0/1164332530 learned_addr learned my addr 172.21.15.69:0/1164332530 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.574+0000 7feb3ea00640 1 --2- 172.21.15.69:0/1164332530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x5607595b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.574+0000 7feb3f400640 1 --2- 172.21.15.69:0/1164332530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5607595b95c0 0x5607595bb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.574+0000 7feb3e000640 1 -- 172.21.15.69:0/1164332530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5607595b95c0 msgr2=0x5607595bb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.574+0000 7feb3e000640 1 --2- 172.21.15.69:0/1164332530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5607595b95c0 0x5607595bb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.574+0000 7feb3e000640 1 -- 172.21.15.69:0/1164332530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 msgr2=0x5607595b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.574+0000 7feb3e000640 1 --2- 172.21.15.69:0/1164332530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x5607595b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.574+0000 7feb3e000640 1 -- 172.21.15.69:0/1164332530 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607595bbf20 con 0x560759542390 2024-04-24T09:11:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.574+0000 7feb3e000640 1 --2- 172.21.15.69:0/1164332530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759542390 0x5607595b0c80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7feb2c02a910 tx=0x7feb2c032ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdb3a5b643d19df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.574+0000 7feb28c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:11:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.575+0000 7feb28c00640 1 -- 172.21.15.69:0/1164332530 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c035050 con 0x560759542390 2024-04-24T09:11:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.575+0000 7feb28c00640 1 -- 172.21.15.69:0/1164332530 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7feb2c003af0 con 0x560759542390 2024-04-24T09:11:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.575+0000 7feb48787740 1 -- 172.21.15.69:0/1164332530 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb2c003940 con 0x560759542390 2024-04-24T09:11:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.575+0000 7feb28c00640 1 -- 172.21.15.69:0/1164332530 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feb2c06f050 con 0x560759542390 2024-04-24T09:11:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.575+0000 7feb48787740 10 client.16692 fetch_fsmap learned FSMap version 30 2024-04-24T09:11:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.575+0000 7feb48787740 1 -- 172.21.15.69:0/1164332530 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607595bd1a0 con 0x560759542390 2024-04-24T09:11:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.575+0000 7feb28c00640 1 -- 172.21.15.69:0/1164332530 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb2c003940 con 0x560759542390 2024-04-24T09:11:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.575+0000 7feb28c00640 1 -- 172.21.15.69:0/1164332530 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7feb2c049470 con 0x560759542390 2024-04-24T09:11:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.575+0000 7feb48787740 10 client.16692 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:11:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.575+0000 7feb48787740 10 client.16692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.576+0000 7feb48787740 20 client.16692 populate_metadata read hostname 'smithi069' 2024-04-24T09:11:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.576+0000 7feb48787740 1 --2- 172.21.15.69:0/1164332530 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5607595bd890 0x5607595bfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.576+0000 7feb48787740 4 client.16692 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:11:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.576+0000 7feb48787740 1 -- 172.21.15.69:0/1164332530 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607594a5190 con 0x5607595bd890 2024-04-24T09:11:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.576+0000 7feb3ea00640 1 --2- 172.21.15.69:0/1164332530 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5607595bd890 0x5607595bfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.577+0000 7feb28c00640 10 client.16692 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.577+0000 7feb3ea00640 1 --2- 172.21.15.69:0/1164332530 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5607595bd890 0x5607595bfcb0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfbd1a04d0fd7a06 server_cookie=8687a5744ce28511 in_seq=0 out_seq=0 2024-04-24T09:11:27.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:27 smithi069 ceph-mon[28170]: pgmap v134: 129 pgs: 129 active+clean; 71 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 666 KiB/s wr, 201 op/s 2024-04-24T09:11:27.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.614+0000 7feb28c00640 1 -- 172.21.15.69:0/1164332530 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5607594a5190 con 0x5607595bd890 2024-04-24T09:11:27.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.614+0000 7feb28c00640 10 client.16692 handle_command_reply: tid=0 2024-04-24T09:11:27.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.614+0000 7feb48787740 1 client.16692 shutdown 2024-04-24T09:11:27.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.615+0000 7feb48787740 1 -- 172.21.15.69:0/1164332530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759542390 msgr2=0x5607595b0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:27.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.615+0000 7feb48787740 1 --2- 172.21.15.69:0/1164332530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759542390 0x5607595b0c80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7feb2c02a910 tx=0x7feb2c032ee0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.615+0000 7feb48787740 1 -- 172.21.15.69:0/1164332530 shutdown_connections 2024-04-24T09:11:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.615+0000 7feb48787740 1 --2- 172.21.15.69:0/1164332530 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5607595bd890 0x5607595bfcb0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.615+0000 7feb48787740 1 --2- 172.21.15.69:0/1164332530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5607595b95c0 0x5607595bb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.615+0000 7feb48787740 1 --2- 172.21.15.69:0/1164332530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560759542390 0x5607595b0c80 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.615+0000 7feb48787740 1 --2- 172.21.15.69:0/1164332530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607595371b0 0x5607595b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.615+0000 7feb48787740 1 -- 172.21.15.69:0/1164332530 >> 172.21.15.69:0/1164332530 conn(0x5607595336c0 msgr2=0x56075954d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.615+0000 7feb28c00640 0 client.16692 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.615+0000 7feb48787740 1 -- 172.21.15.69:0/1164332530 shutdown_connections 2024-04-24T09:11:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.615+0000 7feb48787740 1 -- 172.21.15.69:0/1164332530 wait complete. 2024-04-24T09:11:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.616+0000 7feb48787740 20 client.16692 trim_cache size 0 max 16384 2024-04-24T09:11:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:11:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:11:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:11:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:11:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.617+0000 7feb45a00640 1 -- 172.21.15.69:0/170463453 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7feb240a3dc0 msgr2=0x7feb240a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.617+0000 7feb45a00640 1 --2- 172.21.15.69:0/170463453 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7feb240a3dc0 0x7feb240a6280 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x560759538750 tx=0x7feb3000b4d0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.617+0000 7feb45a00640 1 -- 172.21.15.69:0/170463453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb4019d010 msgr2=0x7feb4019d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.617+0000 7feb45a00640 1 --2- 172.21.15.69:0/170463453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb4019d010 0x7feb4019d4c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7feb3800e960 tx=0x7feb3800ee20 comp rx=0 tx=0).stop 2024-04-24T09:11:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.617+0000 7feb45a00640 1 -- 172.21.15.69:0/170463453 shutdown_connections 2024-04-24T09:11:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.617+0000 7feb45a00640 1 --2- 172.21.15.69:0/170463453 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7feb240a3dc0 0x7feb240a6280 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.617+0000 7feb45a00640 1 --2- 172.21.15.69:0/170463453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb4019d010 0x7feb4019d4c0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.617+0000 7feb45a00640 1 --2- 172.21.15.69:0/170463453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb40107350 0x7feb40198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.617+0000 7feb45a00640 1 --2- 172.21.15.69:0/170463453 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb40102280 0x7feb401987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.617+0000 7feb45a00640 1 -- 172.21.15.69:0/170463453 >> 172.21.15.69:0/170463453 conn(0x7feb40079490 msgr2=0x7feb400ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.617+0000 7feb45a00640 1 -- 172.21.15.69:0/170463453 shutdown_connections 2024-04-24T09:11:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:27.617+0000 7feb45a00640 1 -- 172.21.15.69:0/170463453 wait complete. 2024-04-24T09:11:29.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:29 smithi130 ceph-mon[31553]: pgmap v135: 129 pgs: 129 active+clean; 71 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 437 KiB/s wr, 92 op/s 2024-04-24T09:11:29.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:29 smithi081 ceph-mon[30961]: pgmap v135: 129 pgs: 129 active+clean; 71 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 437 KiB/s wr, 92 op/s 2024-04-24T09:11:29.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:29 smithi069 ceph-mon[28170]: pgmap v135: 129 pgs: 129 active+clean; 71 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 437 KiB/s wr, 92 op/s 2024-04-24T09:11:31.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:31 smithi130 ceph-mon[31553]: pgmap v136: 129 pgs: 129 active+clean; 74 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 676 KiB/s wr, 133 op/s 2024-04-24T09:11:31.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:31 smithi081 ceph-mon[30961]: pgmap v136: 129 pgs: 129 active+clean; 74 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 676 KiB/s wr, 133 op/s 2024-04-24T09:11:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:31 smithi069 ceph-mon[28170]: pgmap v136: 129 pgs: 129 active+clean; 74 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 676 KiB/s wr, 133 op/s 2024-04-24T09:11:33.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:33 smithi130 ceph-mon[31553]: pgmap v137: 129 pgs: 129 active+clean; 74 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 464 KiB/s wr, 86 op/s 2024-04-24T09:11:33.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:33 smithi081 ceph-mon[30961]: pgmap v137: 129 pgs: 129 active+clean; 74 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 464 KiB/s wr, 86 op/s 2024-04-24T09:11:33.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:33 smithi069 ceph-mon[28170]: pgmap v137: 129 pgs: 129 active+clean; 74 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 464 KiB/s wr, 86 op/s 2024-04-24T09:11:35.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:35 smithi130 ceph-mon[31553]: pgmap v138: 129 pgs: 129 active+clean; 74 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 464 KiB/s wr, 86 op/s 2024-04-24T09:11:35.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:35 smithi081 ceph-mon[30961]: pgmap v138: 129 pgs: 129 active+clean; 74 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 464 KiB/s wr, 86 op/s 2024-04-24T09:11:35.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:35 smithi069 ceph-mon[28170]: pgmap v138: 129 pgs: 129 active+clean; 74 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 464 KiB/s wr, 86 op/s 2024-04-24T09:11:37.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:37 smithi130 ceph-mon[31553]: pgmap v139: 129 pgs: 129 active+clean; 76 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 668 KiB/s wr, 124 op/s 2024-04-24T09:11:37.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:37 smithi081 ceph-mon[30961]: pgmap v139: 129 pgs: 129 active+clean; 76 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 668 KiB/s wr, 124 op/s 2024-04-24T09:11:37.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:37 smithi069 ceph-mon[28170]: pgmap v139: 129 pgs: 129 active+clean; 76 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 668 KiB/s wr, 124 op/s 2024-04-24T09:11:39.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:39 smithi130 ceph-mon[31553]: pgmap v140: 129 pgs: 129 active+clean; 76 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 443 KiB/s wr, 78 op/s 2024-04-24T09:11:39.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:39 smithi081 ceph-mon[30961]: pgmap v140: 129 pgs: 129 active+clean; 76 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 443 KiB/s wr, 78 op/s 2024-04-24T09:11:39.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:39 smithi069 ceph-mon[28170]: pgmap v140: 129 pgs: 129 active+clean; 76 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 443 KiB/s wr, 78 op/s 2024-04-24T09:11:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.772+0000 7f45c6000640 1 -- 172.21.15.69:0/867687498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45c0108ec0 msgr2=0x7f45c0115300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.772+0000 7f45c6000640 1 --2- 172.21.15.69:0/867687498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45c0108ec0 0x7f45c0115300 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009cd0 tx=0x7f45b802a8e0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.772+0000 7f45c6000640 1 -- 172.21.15.69:0/867687498 shutdown_connections 2024-04-24T09:11:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.772+0000 7f45c6000640 1 --2- 172.21.15.69:0/867687498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45c0108ec0 0x7f45c0115300 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.772+0000 7f45c6000640 1 --2- 172.21.15.69:0/867687498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45c0108540 0x7f45c0108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.772+0000 7f45c6000640 1 --2- 172.21.15.69:0/867687498 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45c010e4d0 0x7f45c010e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.772+0000 7f45c6000640 1 -- 172.21.15.69:0/867687498 >> 172.21.15.69:0/867687498 conn(0x7f45c0077de0 msgr2=0x7f45c0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.772+0000 7f45c6000640 1 -- 172.21.15.69:0/867687498 shutdown_connections 2024-04-24T09:11:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.772+0000 7f45c6000640 1 -- 172.21.15.69:0/867687498 wait complete. 2024-04-24T09:11:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.773+0000 7f45c6000640 1 Processor -- start 2024-04-24T09:11:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.773+0000 7f45c6000640 1 -- start start 2024-04-24T09:11:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.774+0000 7f45c6000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45c0108540 0x7f45c01a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.774+0000 7f45c6000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45c0108ec0 0x7f45c01a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.774+0000 7f45c6000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45c010e4d0 0x7f45c019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.774+0000 7f45c6000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01186b0 con 0x7f45c0108ec0 2024-04-24T09:11:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.774+0000 7f45c6000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f45c0118530 con 0x7f45c0108540 2024-04-24T09:11:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.774+0000 7f45c6000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f45c0118830 con 0x7f45c010e4d0 2024-04-24T09:11:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.774+0000 7f45b7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45c0108ec0 0x7f45c01a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.774+0000 7f45b7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45c0108ec0 0x7f45c01a4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50018/0 (socket says 172.21.15.69:50018) 2024-04-24T09:11:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.774+0000 7f45b7e00640 1 -- 172.21.15.69:0/1959402187 learned_addr learned my addr 172.21.15.69:0/1959402187 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.774+0000 7f45c5600640 1 --2- 172.21.15.69:0/1959402187 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45c010e4d0 0x7f45c019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.774+0000 7f45c4c00640 1 --2- 172.21.15.69:0/1959402187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45c0108540 0x7f45c01a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.775+0000 7f45b7e00640 1 -- 172.21.15.69:0/1959402187 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45c010e4d0 msgr2=0x7f45c019ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.775+0000 7f45b7e00640 1 --2- 172.21.15.69:0/1959402187 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45c010e4d0 0x7f45c019ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.775+0000 7f45b7e00640 1 -- 172.21.15.69:0/1959402187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45c0108540 msgr2=0x7f45c01a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.775+0000 7f45b7e00640 1 --2- 172.21.15.69:0/1959402187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45c0108540 0x7f45c01a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.775+0000 7f45b7e00640 1 -- 172.21.15.69:0/1959402187 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c019f1b0 con 0x7f45c0108ec0 2024-04-24T09:11:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.775+0000 7f45c4c00640 1 --2- 172.21.15.69:0/1959402187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45c0108540 0x7f45c01a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.775+0000 7f45b7e00640 1 --2- 172.21.15.69:0/1959402187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45c0108ec0 0x7f45c01a4dc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f45a800d950 tx=0x7f45a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.775+0000 7f45be000640 1 -- 172.21.15.69:0/1959402187 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8017070 con 0x7f45c0108ec0 2024-04-24T09:11:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.776+0000 7f45be000640 1 -- 172.21.15.69:0/1959402187 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45a8009c30 con 0x7f45c0108ec0 2024-04-24T09:11:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.776+0000 7f45c6000640 1 -- 172.21.15.69:0/1959402187 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c019f4a0 con 0x7f45c0108ec0 2024-04-24T09:11:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.776+0000 7f45be000640 1 -- 172.21.15.69:0/1959402187 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a800f620 con 0x7f45c0108ec0 2024-04-24T09:11:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.776+0000 7f45c6000640 1 -- 172.21.15.69:0/1959402187 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c01ab740 con 0x7f45c0108ec0 2024-04-24T09:11:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.779+0000 7f45be000640 1 -- 172.21.15.69:0/1959402187 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f45a80109a0 con 0x7f45c0108ec0 2024-04-24T09:11:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.779+0000 7f45ca3f7740 1 Processor -- start 2024-04-24T09:11:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.780+0000 7f45ca3f7740 1 -- start start 2024-04-24T09:11:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.780+0000 7f45ca3f7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568977133c0 0x5568977137a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.780+0000 7f45ca3f7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556897713ce0 0x55689771e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.780+0000 7f45ca3f7740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55689771e5a0 0x556897720990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.780+0000 7f45ca3f7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556897679900 con 0x5568977133c0 2024-04-24T09:11:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.780+0000 7f45ca3f7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55689761ff20 con 0x556897713ce0 2024-04-24T09:11:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.780+0000 7f45ca3f7740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5568976758c0 con 0x55689771e5a0 2024-04-24T09:11:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.780+0000 7f45c5600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568977133c0 0x5568977137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.781+0000 7f45c5600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568977133c0 0x5568977137a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50022/0 (socket says 172.21.15.69:50022) 2024-04-24T09:11:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.781+0000 7f45c5600640 1 -- 172.21.15.69:0/3094335208 learned_addr learned my addr 172.21.15.69:0/3094335208 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.781+0000 7f45b7e00640 1 --2- 172.21.15.69:0/3094335208 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55689771e5a0 0x556897720990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.781+0000 7f45c4c00640 1 --2- 172.21.15.69:0/3094335208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556897713ce0 0x55689771e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.781+0000 7f45c5600640 1 -- 172.21.15.69:0/3094335208 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55689771e5a0 msgr2=0x556897720990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.781+0000 7f45c5600640 1 --2- 172.21.15.69:0/3094335208 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55689771e5a0 0x556897720990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.781+0000 7f45c5600640 1 -- 172.21.15.69:0/3094335208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556897713ce0 msgr2=0x55689771e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.781+0000 7f45c5600640 1 --2- 172.21.15.69:0/3094335208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556897713ce0 0x55689771e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.781+0000 7f45c5600640 1 -- 172.21.15.69:0/3094335208 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556897644e10 con 0x5568977133c0 2024-04-24T09:11:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.781+0000 7f45b7e00640 1 --2- 172.21.15.69:0/3094335208 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55689771e5a0 0x556897720990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.781+0000 7f45be000640 1 --2- 172.21.15.69:0/1959402187 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f45a40a3d10 0x7f45a40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.782+0000 7f45be000640 1 -- 172.21.15.69:0/1959402187 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f45a80d0850 con 0x7f45c0108ec0 2024-04-24T09:11:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.782+0000 7f45c5600640 1 --2- 172.21.15.69:0/3094335208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568977133c0 0x5568977137a0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f45b8035000 tx=0x7f45b8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe55fa61841010 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.782+0000 7f45c4c00640 1 --2- 172.21.15.69:0/1959402187 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f45a40a3d10 0x7f45a40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.782+0000 7f45b5c00640 1 -- 172.21.15.69:0/3094335208 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8031850 con 0x5568977133c0 2024-04-24T09:11:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.782+0000 7f45b5c00640 1 -- 172.21.15.69:0/3094335208 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45b8036040 con 0x5568977133c0 2024-04-24T09:11:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.782+0000 7f45b5c00640 1 -- 172.21.15.69:0/3094335208 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b80044a0 con 0x5568977133c0 2024-04-24T09:11:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.783+0000 7f45ca3f7740 1 -- 172.21.15.69:0/3094335208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568977133c0 msgr2=0x5568977137a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.783+0000 7f45c4c00640 1 --2- 172.21.15.69:0/1959402187 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f45a40a3d10 0x7f45a40a61d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f45b0005d80 tx=0x7f45b0005cf0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:40.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.783+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/3094335208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568977133c0 0x5568977137a0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f45b8035000 tx=0x7f45b8009a20 comp rx=0 tx=0).stop 2024-04-24T09:11:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.783+0000 7f45ca3f7740 1 -- 172.21.15.69:0/3094335208 shutdown_connections 2024-04-24T09:11:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.783+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/3094335208 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55689771e5a0 0x556897720990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.783+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/3094335208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556897713ce0 0x55689771e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.783+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/3094335208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568977133c0 0x5568977137a0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.783+0000 7f45ca3f7740 1 -- 172.21.15.69:0/3094335208 >> 172.21.15.69:0/3094335208 conn(0x55689770f8d0 msgr2=0x55689770fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.783+0000 7f45ca3f7740 1 -- 172.21.15.69:0/3094335208 shutdown_connections 2024-04-24T09:11:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.783+0000 7f45ca3f7740 1 -- 172.21.15.69:0/3094335208 wait complete. 2024-04-24T09:11:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.784+0000 7f45ca3f7740 1 -- start start 2024-04-24T09:11:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.784+0000 7f45ca3f7740 1 Processor -- start 2024-04-24T09:11:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45ca3f7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5568977133c0 0x55689771b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45ca3f7740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556897713ce0 0x55689771bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45ca3f7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55689771e5a0 0x55689771c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45ca3f7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556897720fc0 con 0x55689771e5a0 2024-04-24T09:11:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45ca3f7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5568976332c0 con 0x5568977133c0 2024-04-24T09:11:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45ca3f7740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x556897721140 con 0x556897713ce0 2024-04-24T09:11:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45ca3f7740 10 client.0 _ll_register_callbacks cb 0x55689738ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45c5600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55689771e5a0 0x55689771c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45c5600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55689771e5a0 0x55689771c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50036/0 (socket says 172.21.15.69:50036) 2024-04-24T09:11:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45c5600640 1 -- 172.21.15.69:0/3716413733 learned_addr learned my addr 172.21.15.69:0/3716413733 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45c4c00640 1 --2- 172.21.15.69:0/3716413733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5568977133c0 0x55689771b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45b7e00640 1 --2- 172.21.15.69:0/3716413733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556897713ce0 0x55689771bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45c5600640 1 -- 172.21.15.69:0/3716413733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556897713ce0 msgr2=0x55689771bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45c5600640 1 --2- 172.21.15.69:0/3716413733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556897713ce0 0x55689771bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45c5600640 1 -- 172.21.15.69:0/3716413733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5568977133c0 msgr2=0x55689771b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45c5600640 1 --2- 172.21.15.69:0/3716413733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5568977133c0 0x55689771b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.785+0000 7f45c5600640 1 -- 172.21.15.69:0/3716413733 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556897643fd0 con 0x55689771e5a0 2024-04-24T09:11:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.786+0000 7f45c5600640 1 --2- 172.21.15.69:0/3716413733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55689771e5a0 0x55689771c100 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f45b802a920 tx=0x7f45b80095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1971a82286935c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.786+0000 7f45ae000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:11:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.786+0000 7f45ae000640 1 -- 172.21.15.69:0/3716413733 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b804f020 con 0x55689771e5a0 2024-04-24T09:11:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.786+0000 7f45ae000640 1 -- 172.21.15.69:0/3716413733 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f45b8003e80 con 0x55689771e5a0 2024-04-24T09:11:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.786+0000 7f45ca3f7740 1 -- 172.21.15.69:0/3716413733 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b8036040 con 0x55689771e5a0 2024-04-24T09:11:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.786+0000 7f45ae000640 1 -- 172.21.15.69:0/3716413733 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f45b805a050 con 0x55689771e5a0 2024-04-24T09:11:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.786+0000 7f45ca3f7740 10 client.16704 fetch_fsmap learned FSMap version 30 2024-04-24T09:11:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.786+0000 7f45ca3f7740 1 -- 172.21.15.69:0/3716413733 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55689738c210 con 0x55689771e5a0 2024-04-24T09:11:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.787+0000 7f45ae000640 1 -- 172.21.15.69:0/3716413733 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45b8036040 con 0x55689771e5a0 2024-04-24T09:11:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.787+0000 7f45ae000640 1 -- 172.21.15.69:0/3716413733 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f45b8030070 con 0x55689771e5a0 2024-04-24T09:11:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.787+0000 7f45ca3f7740 10 client.16704 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:11:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.787+0000 7f45ca3f7740 10 client.16704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.787+0000 7f45ca3f7740 20 client.16704 populate_metadata read hostname 'smithi069' 2024-04-24T09:11:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.787+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/3716413733 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55689772e510 0x556897730900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.787+0000 7f45ca3f7740 4 client.16704 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.787+0000 7f45ca3f7740 1 -- 172.21.15.69:0/3716413733 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556897687320 con 0x55689772e510 2024-04-24T09:11:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.788+0000 7f45c4c00640 1 --2- 172.21.15.69:0/3716413733 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55689772e510 0x556897730900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.789+0000 7f45c4c00640 1 --2- 172.21.15.69:0/3716413733 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55689772e510 0x556897730900 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7b7f3a1dcdb6e88 server_cookie=cee598fccbab14d9 in_seq=0 out_seq=0 2024-04-24T09:11:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.789+0000 7f45ae000640 10 client.16704 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.790+0000 7f45ae000640 1 -- 172.21.15.69:0/3716413733 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556897687320 con 0x55689772e510 2024-04-24T09:11:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.790+0000 7f45ae000640 10 client.16704 handle_command_reply: tid=0 2024-04-24T09:11:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.791+0000 7f45ca3f7740 1 client.16704 shutdown 2024-04-24T09:11:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.791+0000 7f45ca3f7740 1 -- 172.21.15.69:0/3716413733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55689771e5a0 msgr2=0x55689771c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.791+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/3716413733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55689771e5a0 0x55689771c100 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f45b802a920 tx=0x7f45b80095e0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.791+0000 7f45ca3f7740 1 -- 172.21.15.69:0/3716413733 shutdown_connections 2024-04-24T09:11:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.791+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/3716413733 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55689772e510 0x556897730900 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.791+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/3716413733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55689771e5a0 0x55689771c100 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.791+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/3716413733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556897713ce0 0x55689771bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.791+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/3716413733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5568977133c0 0x55689771b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.791+0000 7f45ae000640 0 client.16704 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.791+0000 7f45ca3f7740 1 -- 172.21.15.69:0/3716413733 >> 172.21.15.69:0/3716413733 conn(0x55689770f8d0 msgr2=0x556897715650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.791+0000 7f45ca3f7740 1 -- 172.21.15.69:0/3716413733 shutdown_connections 2024-04-24T09:11:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.791+0000 7f45ca3f7740 1 -- 172.21.15.69:0/3716413733 wait complete. 2024-04-24T09:11:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.791+0000 7f45ca3f7740 20 client.16704 trim_cache size 0 max 16384 2024-04-24T09:11:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.808+0000 7f45ca3f7740 1 Processor -- start 2024-04-24T09:11:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.808+0000 7f45ca3f7740 1 -- start start 2024-04-24T09:11:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.808+0000 7f45ca3f7740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568977133c0 0x55689771ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.808+0000 7f45ca3f7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556897713ce0 0x556897729f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.809+0000 7f45ca3f7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55689771e5a0 0x55689771bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.809+0000 7f45ca3f7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01b7fe0 con 0x55689771e5a0 2024-04-24T09:11:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.809+0000 7f45ca3f7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01b5850 con 0x556897713ce0 2024-04-24T09:11:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.809+0000 7f45ca3f7740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01b36a0 con 0x5568977133c0 2024-04-24T09:11:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.809+0000 7f45c5600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55689771e5a0 0x55689771bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.809+0000 7f45c5600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55689771e5a0 0x55689771bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50050/0 (socket says 172.21.15.69:50050) 2024-04-24T09:11:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.809+0000 7f45c5600640 1 -- 172.21.15.69:0/1861213825 learned_addr learned my addr 172.21.15.69:0/1861213825 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.809+0000 7f45b7e00640 1 --2- 172.21.15.69:0/1861213825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556897713ce0 0x556897729f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.809+0000 7f45c5600640 1 -- 172.21.15.69:0/1861213825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568977133c0 msgr2=0x55689771ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.809+0000 7f45c4c00640 1 --2- 172.21.15.69:0/1861213825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568977133c0 0x55689771ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.810+0000 7f45c5600640 1 --2- 172.21.15.69:0/1861213825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568977133c0 0x55689771ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.810+0000 7f45c5600640 1 -- 172.21.15.69:0/1861213825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556897713ce0 msgr2=0x556897729f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.810+0000 7f45c5600640 1 --2- 172.21.15.69:0/1861213825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556897713ce0 0x556897729f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.810+0000 7f45c5600640 1 -- 172.21.15.69:0/1861213825 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556897720d00 con 0x55689771e5a0 2024-04-24T09:11:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.810+0000 7f45b7e00640 1 --2- 172.21.15.69:0/1861213825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556897713ce0 0x556897729f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:11:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.810+0000 7f45c5600640 1 --2- 172.21.15.69:0/1861213825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55689771e5a0 0x55689771bef0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f45b8005f70 tx=0x7f45b8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df83e34919935d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.810+0000 7f45b5200640 1 -- 172.21.15.69:0/1861213825 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b804f040 con 0x55689771e5a0 2024-04-24T09:11:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.810+0000 7f45b5200640 1 -- 172.21.15.69:0/1861213825 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45b8036040 con 0x55689771e5a0 2024-04-24T09:11:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.810+0000 7f45b5200640 1 -- 172.21.15.69:0/1861213825 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8046880 con 0x55689771e5a0 2024-04-24T09:11:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.811+0000 7f45ca3f7740 1 -- 172.21.15.69:0/1861213825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55689771e5a0 msgr2=0x55689771bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.811+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/1861213825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55689771e5a0 0x55689771bef0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f45b8005f70 tx=0x7f45b8009a20 comp rx=0 tx=0).stop 2024-04-24T09:11:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.811+0000 7f45ca3f7740 1 -- 172.21.15.69:0/1861213825 shutdown_connections 2024-04-24T09:11:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.811+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/1861213825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55689771e5a0 0x55689771bef0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.811+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/1861213825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556897713ce0 0x556897729f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.811+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/1861213825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5568977133c0 0x55689771ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.811+0000 7f45ca3f7740 1 -- 172.21.15.69:0/1861213825 >> 172.21.15.69:0/1861213825 conn(0x55689770f8d0 msgr2=0x556897715600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.811+0000 7f45ca3f7740 1 -- 172.21.15.69:0/1861213825 shutdown_connections 2024-04-24T09:11:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.811+0000 7f45ca3f7740 1 -- 172.21.15.69:0/1861213825 wait complete. 2024-04-24T09:11:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.812+0000 7f45ca3f7740 1 -- start start 2024-04-24T09:11:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.812+0000 7f45ca3f7740 1 Processor -- start 2024-04-24T09:11:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.812+0000 7f45ca3f7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568977133c0 0x556897726550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.812+0000 7f45ca3f7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556897713ce0 0x556897721a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.812+0000 7f45ca3f7740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55689771e5a0 0x556897721fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.812+0000 7f45ca3f7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01b3d10 con 0x5568977133c0 2024-04-24T09:11:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.812+0000 7f45ca3f7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01b5180 con 0x556897713ce0 2024-04-24T09:11:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.812+0000 7f45ca3f7740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01b43e0 con 0x55689771e5a0 2024-04-24T09:11:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.812+0000 7f45ca3f7740 10 client.0 _ll_register_callbacks cb 0x7f45b8009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.813+0000 7f45c4c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568977133c0 0x556897726550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.813+0000 7f45c4c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568977133c0 0x556897726550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50056/0 (socket says 172.21.15.69:50056) 2024-04-24T09:11:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.813+0000 7f45c4c00640 1 -- 172.21.15.69:0/2962117097 learned_addr learned my addr 172.21.15.69:0/2962117097 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.813+0000 7f45c4c00640 1 -- 172.21.15.69:0/2962117097 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55689771e5a0 msgr2=0x556897721fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.813+0000 7f45c4c00640 1 --2- 172.21.15.69:0/2962117097 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55689771e5a0 0x556897721fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.813+0000 7f45c4c00640 1 -- 172.21.15.69:0/2962117097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556897713ce0 msgr2=0x556897721a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.813+0000 7f45c4c00640 1 --2- 172.21.15.69:0/2962117097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556897713ce0 0x556897721a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.813+0000 7f45c4c00640 1 -- 172.21.15.69:0/2962117097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556897458620 con 0x5568977133c0 2024-04-24T09:11:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.813+0000 7f45c4c00640 1 --2- 172.21.15.69:0/2962117097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568977133c0 0x556897726550 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f45b000bef0 tx=0x7f45b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d412576ab32c88 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.813+0000 7f45ae000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:11:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.814+0000 7f45ae000640 1 -- 172.21.15.69:0/2962117097 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0029330 con 0x5568977133c0 2024-04-24T09:11:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.814+0000 7f45ae000640 1 -- 172.21.15.69:0/2962117097 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f45b00294d0 con 0x5568977133c0 2024-04-24T09:11:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.814+0000 7f45ca3f7740 1 -- 172.21.15.69:0/2962117097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b8036040 con 0x5568977133c0 2024-04-24T09:11:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.814+0000 7f45ae000640 1 -- 172.21.15.69:0/2962117097 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f45b0039050 con 0x5568977133c0 2024-04-24T09:11:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.814+0000 7f45ca3f7740 10 client.16710 fetch_fsmap learned FSMap version 30 2024-04-24T09:11:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.814+0000 7f45ca3f7740 1 -- 172.21.15.69:0/2962117097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556897794310 con 0x5568977133c0 2024-04-24T09:11:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.814+0000 7f45ae000640 1 -- 172.21.15.69:0/2962117097 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45b8036040 con 0x5568977133c0 2024-04-24T09:11:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.814+0000 7f45ae000640 1 -- 172.21.15.69:0/2962117097 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f45b001f420 con 0x5568977133c0 2024-04-24T09:11:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.815+0000 7f45ca3f7740 10 client.16710 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:11:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.815+0000 7f45ca3f7740 10 client.16710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.815+0000 7f45ca3f7740 20 client.16710 populate_metadata read hostname 'smithi069' 2024-04-24T09:11:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.815+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/2962117097 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x556897794ad0 0x556897796f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.815+0000 7f45ca3f7740 4 client.16710 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:11:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.815+0000 7f45ca3f7740 1 -- 172.21.15.69:0/2962117097 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556897681190 con 0x556897794ad0 2024-04-24T09:11:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.815+0000 7f45b7e00640 1 --2- 172.21.15.69:0/2962117097 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x556897794ad0 0x556897796f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.816+0000 7f45b7e00640 1 --2- 172.21.15.69:0/2962117097 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x556897794ad0 0x556897796f50 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=237068cfc4212221 server_cookie=e02f2111eb8f3ba6 in_seq=0 out_seq=0 2024-04-24T09:11:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.816+0000 7f45ae000640 10 client.16710 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.867+0000 7f45ae000640 1 -- 172.21.15.69:0/2962117097 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556897681190 con 0x556897794ad0 2024-04-24T09:11:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.867+0000 7f45ae000640 10 client.16710 handle_command_reply: tid=0 2024-04-24T09:11:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.867+0000 7f45ca3f7740 1 client.16710 shutdown 2024-04-24T09:11:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.867+0000 7f45ca3f7740 1 -- 172.21.15.69:0/2962117097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568977133c0 msgr2=0x556897726550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.867+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/2962117097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568977133c0 0x556897726550 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f45b000bef0 tx=0x7f45b00046c0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.867+0000 7f45ca3f7740 1 -- 172.21.15.69:0/2962117097 shutdown_connections 2024-04-24T09:11:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.867+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/2962117097 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x556897794ad0 0x556897796f50 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.867+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/2962117097 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55689771e5a0 0x556897721fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.867+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/2962117097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556897713ce0 0x556897721a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.867+0000 7f45ae000640 0 client.16710 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.867+0000 7f45ca3f7740 1 --2- 172.21.15.69:0/2962117097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5568977133c0 0x556897726550 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.867+0000 7f45ca3f7740 1 -- 172.21.15.69:0/2962117097 >> 172.21.15.69:0/2962117097 conn(0x55689770f8d0 msgr2=0x556897728f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.867+0000 7f45ca3f7740 1 -- 172.21.15.69:0/2962117097 shutdown_connections 2024-04-24T09:11:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.867+0000 7f45ca3f7740 1 -- 172.21.15.69:0/2962117097 wait complete. 2024-04-24T09:11:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.867+0000 7f45ca3f7740 20 client.16710 trim_cache size 0 max 16384 2024-04-24T09:11:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:11:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:11:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:11:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:11:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.868+0000 7f45c6000640 1 -- 172.21.15.69:0/1959402187 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f45a40a3d10 msgr2=0x7f45a40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.868+0000 7f45c6000640 1 --2- 172.21.15.69:0/1959402187 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f45a40a3d10 0x7f45a40a61d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f45b0005d80 tx=0x7f45b0005cf0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.868+0000 7f45c6000640 1 -- 172.21.15.69:0/1959402187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45c0108ec0 msgr2=0x7f45c01a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.868+0000 7f45c6000640 1 --2- 172.21.15.69:0/1959402187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45c0108ec0 0x7f45c01a4dc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f45a800d950 tx=0x7f45a800de10 comp rx=0 tx=0).stop 2024-04-24T09:11:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.868+0000 7f45c6000640 1 -- 172.21.15.69:0/1959402187 shutdown_connections 2024-04-24T09:11:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.868+0000 7f45c6000640 1 --2- 172.21.15.69:0/1959402187 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f45a40a3d10 0x7f45a40a61d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.868+0000 7f45c6000640 1 --2- 172.21.15.69:0/1959402187 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45c010e4d0 0x7f45c019ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.868+0000 7f45c6000640 1 --2- 172.21.15.69:0/1959402187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45c0108ec0 0x7f45c01a4dc0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.868+0000 7f45c6000640 1 --2- 172.21.15.69:0/1959402187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45c0108540 0x7f45c01a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.868+0000 7f45c6000640 1 -- 172.21.15.69:0/1959402187 >> 172.21.15.69:0/1959402187 conn(0x7f45c0077de0 msgr2=0x7f45c00784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.868+0000 7f45c6000640 1 -- 172.21.15.69:0/1959402187 shutdown_connections 2024-04-24T09:11:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:40.868+0000 7f45c6000640 1 -- 172.21.15.69:0/1959402187 wait complete. 2024-04-24T09:11:41.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:41 smithi130 ceph-mon[31553]: pgmap v141: 129 pgs: 129 active+clean; 78 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 650 KiB/s wr, 116 op/s 2024-04-24T09:11:41.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:41 smithi081 ceph-mon[30961]: pgmap v141: 129 pgs: 129 active+clean; 78 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 650 KiB/s wr, 116 op/s 2024-04-24T09:11:41.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:41 smithi069 ceph-mon[28170]: pgmap v141: 129 pgs: 129 active+clean; 78 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 650 KiB/s wr, 116 op/s 2024-04-24T09:11:43.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:43 smithi130 ceph-mon[31553]: pgmap v142: 129 pgs: 129 active+clean; 78 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 411 KiB/s wr, 75 op/s 2024-04-24T09:11:43.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:43 smithi081 ceph-mon[30961]: pgmap v142: 129 pgs: 129 active+clean; 78 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 411 KiB/s wr, 75 op/s 2024-04-24T09:11:43.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:43 smithi069 ceph-mon[28170]: pgmap v142: 129 pgs: 129 active+clean; 78 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 411 KiB/s wr, 75 op/s 2024-04-24T09:11:45.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:45 smithi130 ceph-mon[31553]: pgmap v143: 129 pgs: 129 active+clean; 78 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 411 KiB/s wr, 75 op/s 2024-04-24T09:11:45.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:45 smithi081 ceph-mon[30961]: pgmap v143: 129 pgs: 129 active+clean; 78 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 411 KiB/s wr, 75 op/s 2024-04-24T09:11:45.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:45 smithi069 ceph-mon[28170]: pgmap v143: 129 pgs: 129 active+clean; 78 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 411 KiB/s wr, 75 op/s 2024-04-24T09:11:47.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:47 smithi081 ceph-mon[30961]: pgmap v144: 129 pgs: 129 active+clean; 80 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 629 KiB/s wr, 126 op/s 2024-04-24T09:11:47.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:47 smithi069 ceph-mon[28170]: pgmap v144: 129 pgs: 129 active+clean; 80 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 629 KiB/s wr, 126 op/s 2024-04-24T09:11:47.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:47 smithi130 ceph-mon[31553]: pgmap v144: 129 pgs: 129 active+clean; 80 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 629 KiB/s wr, 126 op/s 2024-04-24T09:11:49.570 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:49 smithi081 ceph-mon[30961]: pgmap v145: 129 pgs: 129 active+clean; 80 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 424 KiB/s wr, 88 op/s 2024-04-24T09:11:49.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:49 smithi069 ceph-mon[28170]: pgmap v145: 129 pgs: 129 active+clean; 80 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 424 KiB/s wr, 88 op/s 2024-04-24T09:11:49.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:49 smithi130 ceph-mon[31553]: pgmap v145: 129 pgs: 129 active+clean; 80 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 424 KiB/s wr, 88 op/s 2024-04-24T09:11:51.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:51 smithi081 ceph-mon[30961]: pgmap v146: 129 pgs: 129 active+clean; 83 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 636 KiB/s wr, 126 op/s 2024-04-24T09:11:51.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:51 smithi069 ceph-mon[28170]: pgmap v146: 129 pgs: 129 active+clean; 83 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 636 KiB/s wr, 126 op/s 2024-04-24T09:11:51.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:51 smithi130 ceph-mon[31553]: pgmap v146: 129 pgs: 129 active+clean; 83 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 636 KiB/s wr, 126 op/s 2024-04-24T09:11:53.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:53 smithi081 ceph-mon[30961]: pgmap v147: 129 pgs: 129 active+clean; 83 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 429 KiB/s wr, 89 op/s 2024-04-24T09:11:53.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:53 smithi069 ceph-mon[28170]: pgmap v147: 129 pgs: 129 active+clean; 83 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 429 KiB/s wr, 89 op/s 2024-04-24T09:11:53.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:53 smithi130 ceph-mon[31553]: pgmap v147: 129 pgs: 129 active+clean; 83 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 429 KiB/s wr, 89 op/s 2024-04-24T09:11:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.010+0000 7f8435800640 1 -- 172.21.15.69:0/4292456141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8430109350 msgr2=0x7f8430110f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.010+0000 7f8435800640 1 --2- 172.21.15.69:0/4292456141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8430109350 0x7f8430110f20 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8428009ca0 tx=0x7f842802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.010+0000 7f8435800640 1 -- 172.21.15.69:0/4292456141 shutdown_connections 2024-04-24T09:11:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.010+0000 7f8435800640 1 --2- 172.21.15.69:0/4292456141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8430109350 0x7f8430110f20 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.010+0000 7f8435800640 1 --2- 172.21.15.69:0/4292456141 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8430104c30 0x7f8430108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.010+0000 7f8435800640 1 --2- 172.21.15.69:0/4292456141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8430104280 0x7f8430104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.010+0000 7f8435800640 1 -- 172.21.15.69:0/4292456141 >> 172.21.15.69:0/4292456141 conn(0x7f84300ffb20 msgr2=0x7f8430101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.010+0000 7f8435800640 1 -- 172.21.15.69:0/4292456141 shutdown_connections 2024-04-24T09:11:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.010+0000 7f8435800640 1 -- 172.21.15.69:0/4292456141 wait complete. 2024-04-24T09:11:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.011+0000 7f8435800640 1 Processor -- start 2024-04-24T09:11:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.011+0000 7f8435800640 1 -- start start 2024-04-24T09:11:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.011+0000 7f8435800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8430104280 0x7f84301a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.011+0000 7f8435800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8430109350 0x7f84301a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.011+0000 7f8435800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f843019a640 0x7f843019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.011+0000 7f8435800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f84301142a0 con 0x7f8430109350 2024-04-24T09:11:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.011+0000 7f8435800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8430114120 con 0x7f8430104280 2024-04-24T09:11:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.011+0000 7f8435800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8430114420 con 0x7f843019a640 2024-04-24T09:11:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.012+0000 7f842f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8430109350 0x7f84301a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.012+0000 7f842f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8430109350 0x7f84301a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52124/0 (socket says 172.21.15.69:52124) 2024-04-24T09:11:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.012+0000 7f842f400640 1 -- 172.21.15.69:0/2691588086 learned_addr learned my addr 172.21.15.69:0/2691588086 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.012+0000 7f842fe00640 1 --2- 172.21.15.69:0/2691588086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8430104280 0x7f84301a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.012+0000 7f842f400640 1 -- 172.21.15.69:0/2691588086 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f843019a640 msgr2=0x7f843019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.012+0000 7f842f400640 1 --2- 172.21.15.69:0/2691588086 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f843019a640 0x7f843019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.012+0000 7f842f400640 1 -- 172.21.15.69:0/2691588086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8430104280 msgr2=0x7f84301a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.012+0000 7f842f400640 1 --2- 172.21.15.69:0/2691588086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8430104280 0x7f84301a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.012+0000 7f842f400640 1 -- 172.21.15.69:0/2691588086 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843019b160 con 0x7f8430109350 2024-04-24T09:11:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.012+0000 7f842f400640 1 --2- 172.21.15.69:0/2691588086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8430109350 0x7f84301a0980 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f841c00c970 tx=0x7f841c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.013+0000 7f842cc00640 1 -- 172.21.15.69:0/2691588086 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c007bb0 con 0x7f8430109350 2024-04-24T09:11:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.013+0000 7f842cc00640 1 -- 172.21.15.69:0/2691588086 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f841c007d50 con 0x7f8430109350 2024-04-24T09:11:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.013+0000 7f8435800640 1 -- 172.21.15.69:0/2691588086 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843019b450 con 0x7f8430109350 2024-04-24T09:11:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.013+0000 7f842cc00640 1 -- 172.21.15.69:0/2691588086 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c00f450 con 0x7f8430109350 2024-04-24T09:11:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.013+0000 7f8435800640 1 -- 172.21.15.69:0/2691588086 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84301a7650 con 0x7f8430109350 2024-04-24T09:11:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.016+0000 7f8439db3740 1 Processor -- start 2024-04-24T09:11:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.016+0000 7f8439db3740 1 -- start start 2024-04-24T09:11:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.016+0000 7f8439db3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31ced93c0 0x55b31ced97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.016+0000 7f8439db3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31cee4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.016+0000 7f8439db3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31cee45a0 0x55b31cee6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.017+0000 7f8439db3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b31ce3f900 con 0x55b31ced9ce0 2024-04-24T09:11:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.017+0000 7f8439db3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55b31cde5f20 con 0x55b31ced93c0 2024-04-24T09:11:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.017+0000 7f8439db3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b31ce3b8c0 con 0x55b31cee45a0 2024-04-24T09:11:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.017+0000 7f842cc00640 1 -- 172.21.15.69:0/2691588086 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f841c004f20 con 0x7f8430109350 2024-04-24T09:11:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.017+0000 7f842fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31cee4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.017+0000 7f842fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31cee4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52128/0 (socket says 172.21.15.69:52128) 2024-04-24T09:11:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.017+0000 7f842fe00640 1 -- 172.21.15.69:0/1145806343 learned_addr learned my addr 172.21.15.69:0/1145806343 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.017+0000 7f8434e00640 1 --2- 172.21.15.69:0/1145806343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31ced93c0 0x55b31ced97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.017+0000 7f842fe00640 1 -- 172.21.15.69:0/1145806343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31cee45a0 msgr2=0x55b31cee6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.017+0000 7f842fe00640 1 --2- 172.21.15.69:0/1145806343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31cee45a0 0x55b31cee6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.017+0000 7f842fe00640 1 -- 172.21.15.69:0/1145806343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31ced93c0 msgr2=0x55b31ced97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.017+0000 7f842fe00640 1 --2- 172.21.15.69:0/1145806343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31ced93c0 0x55b31ced97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.017+0000 7f842fe00640 1 -- 172.21.15.69:0/1145806343 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31ce0ae10 con 0x55b31ced9ce0 2024-04-24T09:11:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.017+0000 7f8434e00640 1 --2- 172.21.15.69:0/1145806343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31ced93c0 0x55b31ced97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.018+0000 7f842fe00640 1 --2- 172.21.15.69:0/1145806343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31cee4060 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f842000bcb0 tx=0x7f842000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55d657fd7f2f04a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.018+0000 7f8425600640 1 -- 172.21.15.69:0/1145806343 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420015070 con 0x55b31ced9ce0 2024-04-24T09:11:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.018+0000 7f8425600640 1 -- 172.21.15.69:0/1145806343 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f842001a910 con 0x55b31ced9ce0 2024-04-24T09:11:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.018+0000 7f8425600640 1 -- 172.21.15.69:0/1145806343 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420011a00 con 0x55b31ced9ce0 2024-04-24T09:11:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.019+0000 7f842cc00640 1 --2- 172.21.15.69:0/2691588086 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f84140a3d10 0x7f84140a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.019+0000 7f842cc00640 1 -- 172.21.15.69:0/2691588086 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f841c0d0c10 con 0x7f8430109350 2024-04-24T09:11:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.019+0000 7f8434e00640 1 --2- 172.21.15.69:0/2691588086 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f84140a3d10 0x7f84140a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.019+0000 7f8439db3740 1 -- 172.21.15.69:0/1145806343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 msgr2=0x55b31cee4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.019+0000 7f8439db3740 1 --2- 172.21.15.69:0/1145806343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31cee4060 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f842000bcb0 tx=0x7f842000f730 comp rx=0 tx=0).stop 2024-04-24T09:11:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.019+0000 7f8439db3740 1 -- 172.21.15.69:0/1145806343 shutdown_connections 2024-04-24T09:11:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.019+0000 7f8439db3740 1 --2- 172.21.15.69:0/1145806343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31cee45a0 0x55b31cee6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.019+0000 7f8439db3740 1 --2- 172.21.15.69:0/1145806343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31cee4060 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.019+0000 7f8439db3740 1 --2- 172.21.15.69:0/1145806343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31ced93c0 0x55b31ced97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.019+0000 7f8439db3740 1 -- 172.21.15.69:0/1145806343 >> 172.21.15.69:0/1145806343 conn(0x55b31ced58d0 msgr2=0x55b31ced5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.019+0000 7f8439db3740 1 -- 172.21.15.69:0/1145806343 shutdown_connections 2024-04-24T09:11:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.019+0000 7f8439db3740 1 -- 172.21.15.69:0/1145806343 wait complete. 2024-04-24T09:11:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.020+0000 7f8434e00640 1 --2- 172.21.15.69:0/2691588086 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f84140a3d10 0x7f84140a61d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8428009a60 tx=0x7f8428005c50 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.020+0000 7f8439db3740 1 -- start start 2024-04-24T09:11:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.020+0000 7f8439db3740 1 Processor -- start 2024-04-24T09:11:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.020+0000 7f8439db3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced93c0 0x55b31cee1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.020+0000 7f8439db3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31ced9ce0 0x55b31cee1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.020+0000 7f8439db3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31cee45a0 0x55b31cee1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.020+0000 7f8439db3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b31cee7140 con 0x55b31ced93c0 2024-04-24T09:11:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.021+0000 7f8439db3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55b31cdf92c0 con 0x55b31ced9ce0 2024-04-24T09:11:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.021+0000 7f8439db3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b31cee72c0 con 0x55b31cee45a0 2024-04-24T09:11:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.021+0000 7f8439db3740 10 client.0 _ll_register_callbacks cb 0x55b31cb51a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.021+0000 7f842fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced93c0 0x55b31cee1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.021+0000 7f842fe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced93c0 0x55b31cee1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52134/0 (socket says 172.21.15.69:52134) 2024-04-24T09:11:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.021+0000 7f842fe00640 1 -- 172.21.15.69:0/949178805 learned_addr learned my addr 172.21.15.69:0/949178805 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.021+0000 7f842fe00640 1 -- 172.21.15.69:0/949178805 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31cee45a0 msgr2=0x55b31cee1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.021+0000 7f842fe00640 1 --2- 172.21.15.69:0/949178805 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31cee45a0 0x55b31cee1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.021+0000 7f842fe00640 1 -- 172.21.15.69:0/949178805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31ced9ce0 msgr2=0x55b31cee1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.021+0000 7f842fe00640 1 --2- 172.21.15.69:0/949178805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31ced9ce0 0x55b31cee1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.021+0000 7f842fe00640 1 -- 172.21.15.69:0/949178805 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b31ce09fd0 con 0x55b31ced93c0 2024-04-24T09:11:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.022+0000 7f842fe00640 1 --2- 172.21.15.69:0/949178805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced93c0 0x55b31cee1560 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8420000c00 tx=0x7f842000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8955815c77bfac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.022+0000 7f841a000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:11:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.022+0000 7f841a000640 1 -- 172.21.15.69:0/949178805 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420015050 con 0x55b31ced93c0 2024-04-24T09:11:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.022+0000 7f841a000640 1 -- 172.21.15.69:0/949178805 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f842001aac0 con 0x55b31ced93c0 2024-04-24T09:11:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.022+0000 7f8439db3740 1 -- 172.21.15.69:0/949178805 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f842001a910 con 0x55b31ced93c0 2024-04-24T09:11:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.022+0000 7f841a000640 1 -- 172.21.15.69:0/949178805 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8420031050 con 0x55b31ced93c0 2024-04-24T09:11:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.022+0000 7f8439db3740 10 client.16722 fetch_fsmap learned FSMap version 30 2024-04-24T09:11:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.022+0000 7f8439db3740 1 -- 172.21.15.69:0/949178805 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b31cb52210 con 0x55b31ced93c0 2024-04-24T09:11:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.023+0000 7f841a000640 1 -- 172.21.15.69:0/949178805 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f842001a910 con 0x55b31ced93c0 2024-04-24T09:11:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.023+0000 7f841a000640 1 -- 172.21.15.69:0/949178805 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f842002c030 con 0x55b31ced93c0 2024-04-24T09:11:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.023+0000 7f8439db3740 10 client.16722 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:11:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.023+0000 7f8439db3740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.023+0000 7f8439db3740 20 client.16722 populate_metadata read hostname 'smithi069' 2024-04-24T09:11:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.023+0000 7f8439db3740 1 --2- 172.21.15.69:0/949178805 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55b31cef4660 0x55b31cef6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.023+0000 7f8439db3740 4 client.16722 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.023+0000 7f8439db3740 1 -- 172.21.15.69:0/949178805 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b31ce4d320 con 0x55b31cef4660 2024-04-24T09:11:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.024+0000 7f842f400640 1 --2- 172.21.15.69:0/949178805 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55b31cef4660 0x55b31cef6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.025+0000 7f842f400640 1 --2- 172.21.15.69:0/949178805 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55b31cef4660 0x55b31cef6a50 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62bb2ab5d0f6e67b server_cookie=e3048b6772a9bbc5 in_seq=0 out_seq=0 2024-04-24T09:11:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.025+0000 7f841a000640 10 client.16722 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.027+0000 7f841a000640 1 -- 172.21.15.69:0/949178805 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b31ce4d320 con 0x55b31cef4660 2024-04-24T09:11:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.027+0000 7f841a000640 10 client.16722 handle_command_reply: tid=0 2024-04-24T09:11:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.027+0000 7f8439db3740 1 client.16722 shutdown 2024-04-24T09:11:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.027+0000 7f8439db3740 1 -- 172.21.15.69:0/949178805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced93c0 msgr2=0x55b31cee1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.027+0000 7f8439db3740 1 --2- 172.21.15.69:0/949178805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced93c0 0x55b31cee1560 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8420000c00 tx=0x7f842000bfd0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.027+0000 7f8439db3740 1 -- 172.21.15.69:0/949178805 shutdown_connections 2024-04-24T09:11:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.027+0000 7f8439db3740 1 --2- 172.21.15.69:0/949178805 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55b31cef4660 0x55b31cef6a50 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.027+0000 7f8439db3740 1 --2- 172.21.15.69:0/949178805 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31cee45a0 0x55b31cee1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.027+0000 7f8439db3740 1 --2- 172.21.15.69:0/949178805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31ced9ce0 0x55b31cee1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.027+0000 7f8439db3740 1 --2- 172.21.15.69:0/949178805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced93c0 0x55b31cee1560 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.027+0000 7f841a000640 0 client.16722 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.027+0000 7f8439db3740 1 -- 172.21.15.69:0/949178805 >> 172.21.15.69:0/949178805 conn(0x55b31ced58d0 msgr2=0x55b31cedb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.027+0000 7f8439db3740 1 -- 172.21.15.69:0/949178805 shutdown_connections 2024-04-24T09:11:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.027+0000 7f8439db3740 1 -- 172.21.15.69:0/949178805 wait complete. 2024-04-24T09:11:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.027+0000 7f8439db3740 20 client.16722 trim_cache size 0 max 16384 2024-04-24T09:11:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.044+0000 7f8439db3740 1 Processor -- start 2024-04-24T09:11:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.044+0000 7f8439db3740 1 -- start start 2024-04-24T09:11:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.044+0000 7f8439db3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31ced93c0 0x55b31cef0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.044+0000 7f8439db3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31cee1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.044+0000 7f8439db3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31cee45a0 0x55b31cefc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.044+0000 7f8439db3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f84301b3bf0 con 0x55b31ced9ce0 2024-04-24T09:11:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.044+0000 7f8439db3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f84301b1460 con 0x55b31cee45a0 2024-04-24T09:11:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.044+0000 7f8439db3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f84301af2b0 con 0x55b31ced93c0 2024-04-24T09:11:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.044+0000 7f842f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31cee1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.044+0000 7f842f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31cee1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52138/0 (socket says 172.21.15.69:52138) 2024-04-24T09:11:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.044+0000 7f842f400640 1 -- 172.21.15.69:0/513441184 learned_addr learned my addr 172.21.15.69:0/513441184 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.045+0000 7f842fe00640 1 --2- 172.21.15.69:0/513441184 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31ced93c0 0x55b31cef0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.045+0000 7f8434e00640 1 --2- 172.21.15.69:0/513441184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31cee45a0 0x55b31cefc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.045+0000 7f842f400640 1 -- 172.21.15.69:0/513441184 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31ced93c0 msgr2=0x55b31cef0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.045+0000 7f842f400640 1 --2- 172.21.15.69:0/513441184 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31ced93c0 0x55b31cef0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.045+0000 7f842f400640 1 -- 172.21.15.69:0/513441184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31cee45a0 msgr2=0x55b31cefc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.045+0000 7f842f400640 1 --2- 172.21.15.69:0/513441184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31cee45a0 0x55b31cefc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.045+0000 7f842f400640 1 -- 172.21.15.69:0/513441184 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31cc1e620 con 0x55b31ced9ce0 2024-04-24T09:11:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.045+0000 7f842fe00640 1 --2- 172.21.15.69:0/513441184 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31ced93c0 0x55b31cef0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:11:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.045+0000 7f8434e00640 1 --2- 172.21.15.69:0/513441184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31cee45a0 0x55b31cefc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:11:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.045+0000 7f842f400640 1 --2- 172.21.15.69:0/513441184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31cee1c30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f841c09c7a0 tx=0x7f841c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa884a97a9952b0f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.045+0000 7f8424c00640 1 -- 172.21.15.69:0/513441184 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c01d040 con 0x55b31ced9ce0 2024-04-24T09:11:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.045+0000 7f8424c00640 1 -- 172.21.15.69:0/513441184 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f841c0a0bc0 con 0x55b31ced9ce0 2024-04-24T09:11:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.045+0000 7f8424c00640 1 -- 172.21.15.69:0/513441184 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c0870c0 con 0x55b31ced9ce0 2024-04-24T09:11:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.046+0000 7f8439db3740 1 -- 172.21.15.69:0/513441184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 msgr2=0x55b31cee1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.046+0000 7f8439db3740 1 --2- 172.21.15.69:0/513441184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31cee1c30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f841c09c7a0 tx=0x7f841c00cf60 comp rx=0 tx=0).stop 2024-04-24T09:11:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.046+0000 7f8439db3740 1 -- 172.21.15.69:0/513441184 shutdown_connections 2024-04-24T09:11:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.046+0000 7f8439db3740 1 --2- 172.21.15.69:0/513441184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31cee45a0 0x55b31cefc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.046+0000 7f8439db3740 1 --2- 172.21.15.69:0/513441184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31cee1c30 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.046+0000 7f8439db3740 1 --2- 172.21.15.69:0/513441184 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31ced93c0 0x55b31cef0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.046+0000 7f8439db3740 1 -- 172.21.15.69:0/513441184 >> 172.21.15.69:0/513441184 conn(0x55b31ced58d0 msgr2=0x55b31cee1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.046+0000 7f8439db3740 1 -- 172.21.15.69:0/513441184 shutdown_connections 2024-04-24T09:11:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.046+0000 7f8439db3740 1 -- 172.21.15.69:0/513441184 wait complete. 2024-04-24T09:11:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.047+0000 7f8439db3740 1 -- start start 2024-04-24T09:11:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.047+0000 7f8439db3740 1 Processor -- start 2024-04-24T09:11:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.047+0000 7f8439db3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31ced93c0 0x55b31ceec620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.047+0000 7f8439db3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31ceecb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.047+0000 7f8439db3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31cee45a0 0x55b31cee9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.047+0000 7f8439db3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f84301af920 con 0x55b31ced9ce0 2024-04-24T09:11:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.047+0000 7f8439db3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f84301b0d90 con 0x55b31ced93c0 2024-04-24T09:11:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.047+0000 7f8439db3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f84301afff0 con 0x55b31cee45a0 2024-04-24T09:11:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.047+0000 7f8439db3740 10 client.0 _ll_register_callbacks cb 0x7f842000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.048+0000 7f842f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31ceecb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.048+0000 7f842f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31ceecb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52152/0 (socket says 172.21.15.69:52152) 2024-04-24T09:11:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.048+0000 7f842f400640 1 -- 172.21.15.69:0/3240368187 learned_addr learned my addr 172.21.15.69:0/3240368187 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:11:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.048+0000 7f842f400640 1 -- 172.21.15.69:0/3240368187 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31cee45a0 msgr2=0x55b31cee9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.048+0000 7f842f400640 1 --2- 172.21.15.69:0/3240368187 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31cee45a0 0x55b31cee9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.048+0000 7f842f400640 1 -- 172.21.15.69:0/3240368187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31ced93c0 msgr2=0x55b31ceec620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.048+0000 7f842f400640 1 --2- 172.21.15.69:0/3240368187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31ced93c0 0x55b31ceec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.048+0000 7f842f400640 1 -- 172.21.15.69:0/3240368187 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b31cee9ea0 con 0x55b31ced9ce0 2024-04-24T09:11:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.048+0000 7f842f400640 1 --2- 172.21.15.69:0/3240368187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31ceecb60 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f841c0991e0 tx=0x7f841c09c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a90a4f8723b98b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.048+0000 7f841a000640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:11:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.049+0000 7f841a000640 1 -- 172.21.15.69:0/3240368187 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c0b9020 con 0x55b31ced9ce0 2024-04-24T09:11:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.049+0000 7f841a000640 1 -- 172.21.15.69:0/3240368187 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f841c098e80 con 0x55b31ced9ce0 2024-04-24T09:11:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.049+0000 7f8439db3740 1 -- 172.21.15.69:0/3240368187 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f841c0a0bc0 con 0x55b31ced9ce0 2024-04-24T09:11:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.049+0000 7f841a000640 1 -- 172.21.15.69:0/3240368187 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f841c01d040 con 0x55b31ced9ce0 2024-04-24T09:11:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.049+0000 7f8439db3740 10 client.16728 fetch_fsmap learned FSMap version 30 2024-04-24T09:11:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.049+0000 7f8439db3740 1 -- 172.21.15.69:0/3240368187 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b31cf5b4a0 con 0x55b31ced9ce0 2024-04-24T09:11:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.049+0000 7f841a000640 1 -- 172.21.15.69:0/3240368187 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f841c0a0bc0 con 0x55b31ced9ce0 2024-04-24T09:11:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.049+0000 7f841a000640 1 -- 172.21.15.69:0/3240368187 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f841c0ce240 con 0x55b31ced9ce0 2024-04-24T09:11:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.049+0000 7f8439db3740 10 client.16728 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:11:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.049+0000 7f8439db3740 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.049+0000 7f8439db3740 20 client.16728 populate_metadata read hostname 'smithi069' 2024-04-24T09:11:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.049+0000 7f8439db3740 1 --2- 172.21.15.69:0/3240368187 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55b31cf5bc80 0x55b31cf5e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.049+0000 7f8439db3740 4 client.16728 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:11:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.049+0000 7f8439db3740 1 -- 172.21.15.69:0/3240368187 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b31ce47190 con 0x55b31cf5bc80 2024-04-24T09:11:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.050+0000 7f842fe00640 1 --2- 172.21.15.69:0/3240368187 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55b31cf5bc80 0x55b31cf5e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.051+0000 7f842fe00640 1 --2- 172.21.15.69:0/3240368187 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55b31cf5bc80 0x55b31cf5e0a0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e25a2ff694a0bb12 server_cookie=c29fc09301bc3b9e in_seq=0 out_seq=0 2024-04-24T09:11:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.051+0000 7f841a000640 10 client.16728 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.180+0000 7f841a000640 1 -- 172.21.15.69:0/3240368187 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b31ce47190 con 0x55b31cf5bc80 2024-04-24T09:11:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.181+0000 7f841a000640 10 client.16728 handle_command_reply: tid=0 2024-04-24T09:11:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.181+0000 7f8439db3740 1 client.16728 shutdown 2024-04-24T09:11:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.181+0000 7f8439db3740 1 -- 172.21.15.69:0/3240368187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 msgr2=0x55b31ceecb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.181+0000 7f8439db3740 1 --2- 172.21.15.69:0/3240368187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31ceecb60 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f841c0991e0 tx=0x7f841c09c710 comp rx=0 tx=0).stop 2024-04-24T09:11:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.181+0000 7f8439db3740 1 -- 172.21.15.69:0/3240368187 shutdown_connections 2024-04-24T09:11:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.181+0000 7f8439db3740 1 --2- 172.21.15.69:0/3240368187 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55b31cf5bc80 0x55b31cf5e0a0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.181+0000 7f8439db3740 1 --2- 172.21.15.69:0/3240368187 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b31cee45a0 0x55b31cee9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.181+0000 7f841a000640 0 client.16728 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:11:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.181+0000 7f8439db3740 1 --2- 172.21.15.69:0/3240368187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b31ced9ce0 0x55b31ceecb60 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.181+0000 7f8439db3740 1 --2- 172.21.15.69:0/3240368187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b31ced93c0 0x55b31ceec620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.181+0000 7f8439db3740 1 -- 172.21.15.69:0/3240368187 >> 172.21.15.69:0/3240368187 conn(0x55b31ced58d0 msgr2=0x55b31cee1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.181+0000 7f8439db3740 1 -- 172.21.15.69:0/3240368187 shutdown_connections 2024-04-24T09:11:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.181+0000 7f8439db3740 1 -- 172.21.15.69:0/3240368187 wait complete. 2024-04-24T09:11:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.181+0000 7f8439db3740 20 client.16728 trim_cache size 0 max 16384 2024-04-24T09:11:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:11:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:11:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:11:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:11:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.182+0000 7f8435800640 1 -- 172.21.15.69:0/2691588086 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f84140a3d10 msgr2=0x7f84140a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.182+0000 7f8435800640 1 --2- 172.21.15.69:0/2691588086 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f84140a3d10 0x7f84140a61d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8428009a60 tx=0x7f8428005c50 comp rx=0 tx=0).stop 2024-04-24T09:11:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.182+0000 7f8435800640 1 -- 172.21.15.69:0/2691588086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8430109350 msgr2=0x7f84301a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.182+0000 7f8435800640 1 --2- 172.21.15.69:0/2691588086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8430109350 0x7f84301a0980 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f841c00c970 tx=0x7f841c00ce30 comp rx=0 tx=0).stop 2024-04-24T09:11:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.182+0000 7f8435800640 1 -- 172.21.15.69:0/2691588086 shutdown_connections 2024-04-24T09:11:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.182+0000 7f8435800640 1 --2- 172.21.15.69:0/2691588086 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f84140a3d10 0x7f84140a61d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.182+0000 7f8435800640 1 --2- 172.21.15.69:0/2691588086 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f843019a640 0x7f843019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.182+0000 7f8435800640 1 --2- 172.21.15.69:0/2691588086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8430109350 0x7f84301a0980 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.182+0000 7f8435800640 1 --2- 172.21.15.69:0/2691588086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8430104280 0x7f84301a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.182+0000 7f8435800640 1 -- 172.21.15.69:0/2691588086 >> 172.21.15.69:0/2691588086 conn(0x7f84300ffb20 msgr2=0x7f8430101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.182+0000 7f8435800640 1 -- 172.21.15.69:0/2691588086 shutdown_connections 2024-04-24T09:11:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:11:54.182+0000 7f8435800640 1 -- 172.21.15.69:0/2691588086 wait complete. 2024-04-24T09:11:55.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:55 smithi081 ceph-mon[30961]: pgmap v148: 129 pgs: 129 active+clean; 83 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 429 KiB/s wr, 89 op/s 2024-04-24T09:11:55.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:55 smithi069 ceph-mon[28170]: pgmap v148: 129 pgs: 129 active+clean; 83 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 429 KiB/s wr, 89 op/s 2024-04-24T09:11:55.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:55 smithi130 ceph-mon[31553]: pgmap v148: 129 pgs: 129 active+clean; 83 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 429 KiB/s wr, 89 op/s 2024-04-24T09:11:57.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:57 smithi081 ceph-mon[30961]: pgmap v149: 129 pgs: 129 active+clean; 86 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 697 KiB/s wr, 161 op/s 2024-04-24T09:11:57.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:57 smithi069 ceph-mon[28170]: pgmap v149: 129 pgs: 129 active+clean; 86 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 697 KiB/s wr, 161 op/s 2024-04-24T09:11:57.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:57 smithi130 ceph-mon[31553]: pgmap v149: 129 pgs: 129 active+clean; 86 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 697 KiB/s wr, 161 op/s 2024-04-24T09:11:59.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:11:59 smithi081 ceph-mon[30961]: pgmap v150: 129 pgs: 129 active+clean; 86 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 480 KiB/s wr, 110 op/s 2024-04-24T09:11:59.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:11:59 smithi069 ceph-mon[28170]: pgmap v150: 129 pgs: 129 active+clean; 86 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 480 KiB/s wr, 110 op/s 2024-04-24T09:11:59.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:11:59 smithi130 ceph-mon[31553]: pgmap v150: 129 pgs: 129 active+clean; 86 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 480 KiB/s wr, 110 op/s 2024-04-24T09:12:01.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:01 smithi081 ceph-mon[30961]: pgmap v151: 129 pgs: 129 active+clean; 88 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 729 KiB/s wr, 151 op/s 2024-04-24T09:12:01.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:01 smithi069 ceph-mon[28170]: pgmap v151: 129 pgs: 129 active+clean; 88 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 729 KiB/s wr, 151 op/s 2024-04-24T09:12:01.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:01 smithi130 ceph-mon[31553]: pgmap v151: 129 pgs: 129 active+clean; 88 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 729 KiB/s wr, 151 op/s 2024-04-24T09:12:03.506 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:03 smithi130 ceph-mon[31553]: pgmap v152: 129 pgs: 129 active+clean; 88 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 517 KiB/s wr, 112 op/s 2024-04-24T09:12:03.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:03 smithi081 ceph-mon[30961]: pgmap v152: 129 pgs: 129 active+clean; 88 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 517 KiB/s wr, 112 op/s 2024-04-24T09:12:03.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:03 smithi069 ceph-mon[28170]: pgmap v152: 129 pgs: 129 active+clean; 88 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 517 KiB/s wr, 112 op/s 2024-04-24T09:12:05.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:05 smithi081 ceph-mon[30961]: pgmap v153: 129 pgs: 129 active+clean; 88 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 517 KiB/s wr, 112 op/s 2024-04-24T09:12:05.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:05 smithi069 ceph-mon[28170]: pgmap v153: 129 pgs: 129 active+clean; 88 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 517 KiB/s wr, 112 op/s 2024-04-24T09:12:05.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:05 smithi130 ceph-mon[31553]: pgmap v153: 129 pgs: 129 active+clean; 88 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 517 KiB/s wr, 112 op/s 2024-04-24T09:12:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.326+0000 7fe7f9200640 1 -- 172.21.15.69:0/2368585728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7f41038e0 msgr2=0x7fe7f410bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.326+0000 7fe7f9200640 1 --2- 172.21.15.69:0/2368585728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7f41038e0 0x7fe7f410bdc0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009900 tx=0x7fe7dc02a480 comp rx=0 tx=0).stop 2024-04-24T09:12:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.326+0000 7fe7f9200640 1 -- 172.21.15.69:0/2368585728 shutdown_connections 2024-04-24T09:12:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.326+0000 7fe7f9200640 1 --2- 172.21.15.69:0/2368585728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7f410c3e0 0x7fe7f410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.326+0000 7fe7f9200640 1 --2- 172.21.15.69:0/2368585728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7f41038e0 0x7fe7f410bdc0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.326+0000 7fe7f9200640 1 --2- 172.21.15.69:0/2368585728 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f4102fc0 0x7fe7f41033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.326+0000 7fe7f9200640 1 -- 172.21.15.69:0/2368585728 >> 172.21.15.69:0/2368585728 conn(0x7fe7f40fc820 msgr2=0x7fe7f40fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.326+0000 7fe7f9200640 1 -- 172.21.15.69:0/2368585728 shutdown_connections 2024-04-24T09:12:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.326+0000 7fe7f9200640 1 -- 172.21.15.69:0/2368585728 wait complete. 2024-04-24T09:12:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.327+0000 7fe7f9200640 1 Processor -- start 2024-04-24T09:12:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.328+0000 7fe7f9200640 1 -- start start 2024-04-24T09:12:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.328+0000 7fe7f9200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7f4102fc0 0x7fe7f41a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.328+0000 7fe7f9200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f41038e0 0x7fe7f41a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.328+0000 7fe7f9200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7f410c3e0 0x7fe7f419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.328+0000 7fe7f9200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f41120c0 con 0x7fe7f4102fc0 2024-04-24T09:12:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.328+0000 7fe7f9200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4111f40 con 0x7fe7f410c3e0 2024-04-24T09:12:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.328+0000 7fe7f9200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4112240 con 0x7fe7f41038e0 2024-04-24T09:12:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.328+0000 7fe7f3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7f4102fc0 0x7fe7f41a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.328+0000 7fe7f3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7f4102fc0 0x7fe7f41a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37176/0 (socket says 172.21.15.69:37176) 2024-04-24T09:12:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.328+0000 7fe7f3400640 1 -- 172.21.15.69:0/3475413237 learned_addr learned my addr 172.21.15.69:0/3475413237 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.329+0000 7fe7ebe00640 1 --2- 172.21.15.69:0/3475413237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f41038e0 0x7fe7f41a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.329+0000 7fe7f3e00640 1 --2- 172.21.15.69:0/3475413237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7f410c3e0 0x7fe7f419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.329+0000 7fe7f3400640 1 -- 172.21.15.69:0/3475413237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f41038e0 msgr2=0x7fe7f41a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.329+0000 7fe7f3400640 1 --2- 172.21.15.69:0/3475413237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f41038e0 0x7fe7f41a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.329+0000 7fe7f3400640 1 -- 172.21.15.69:0/3475413237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7f410c3e0 msgr2=0x7fe7f419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.329+0000 7fe7f3400640 1 --2- 172.21.15.69:0/3475413237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7f410c3e0 0x7fe7f419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.329+0000 7fe7f3400640 1 -- 172.21.15.69:0/3475413237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f419af80 con 0x7fe7f4102fc0 2024-04-24T09:12:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.329+0000 7fe7f3400640 1 --2- 172.21.15.69:0/3475413237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7f4102fc0 0x7fe7f41a05b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400d770 tx=0x7fe7e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.329+0000 7fe7f0c00640 1 -- 172.21.15.69:0/3475413237 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4017070 con 0x7fe7f4102fc0 2024-04-24T09:12:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.330+0000 7fe7f0c00640 1 -- 172.21.15.69:0/3475413237 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7e40054d0 con 0x7fe7f4102fc0 2024-04-24T09:12:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.330+0000 7fe7f0c00640 1 -- 172.21.15.69:0/3475413237 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4004cf0 con 0x7fe7f4102fc0 2024-04-24T09:12:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.330+0000 7fe7f9200640 1 -- 172.21.15.69:0/3475413237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f419b270 con 0x7fe7f4102fc0 2024-04-24T09:12:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.330+0000 7fe7f9200640 1 -- 172.21.15.69:0/3475413237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f41a7530 con 0x7fe7f4102fc0 2024-04-24T09:12:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.333+0000 7fe7fd660740 1 Processor -- start 2024-04-24T09:12:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.333+0000 7fe7f0c00640 1 -- 172.21.15.69:0/3475413237 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe7e4004e90 con 0x7fe7f4102fc0 2024-04-24T09:12:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.333+0000 7fe7fd660740 1 -- start start 2024-04-24T09:12:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.333+0000 7fe7fd660740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18aec3c0 0x561e18aec7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.333+0000 7fe7fd660740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18af7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.333+0000 7fe7fd660740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18af75a0 0x561e18af9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.333+0000 7fe7fd660740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561e18a52900 con 0x561e18aecce0 2024-04-24T09:12:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.333+0000 7fe7fd660740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561e189f8f20 con 0x561e18aec3c0 2024-04-24T09:12:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.333+0000 7fe7fd660740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x561e18a4e8c0 con 0x561e18af75a0 2024-04-24T09:12:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.334+0000 7fe7f3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18af7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.334+0000 7fe7f3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18af7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37180/0 (socket says 172.21.15.69:37180) 2024-04-24T09:12:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.334+0000 7fe7f3400640 1 -- 172.21.15.69:0/2358892921 learned_addr learned my addr 172.21.15.69:0/2358892921 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.334+0000 7fe7f3e00640 1 --2- 172.21.15.69:0/2358892921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18aec3c0 0x561e18aec7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.334+0000 7fe7ebe00640 1 --2- 172.21.15.69:0/2358892921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18af75a0 0x561e18af9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.334+0000 7fe7f3400640 1 -- 172.21.15.69:0/2358892921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18af75a0 msgr2=0x561e18af9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.334+0000 7fe7f3400640 1 --2- 172.21.15.69:0/2358892921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18af75a0 0x561e18af9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.334+0000 7fe7f3400640 1 -- 172.21.15.69:0/2358892921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18aec3c0 msgr2=0x561e18aec7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.334+0000 7fe7f3400640 1 --2- 172.21.15.69:0/2358892921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18aec3c0 0x561e18aec7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.334+0000 7fe7f3400640 1 -- 172.21.15.69:0/2358892921 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e18a1de10 con 0x561e18aecce0 2024-04-24T09:12:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.334+0000 7fe7f3e00640 1 --2- 172.21.15.69:0/2358892921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18aec3c0 0x561e18aec7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.335+0000 7fe7f3400640 1 --2- 172.21.15.69:0/2358892921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18af7060 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40cd940 tx=0x7fe7e40d6cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cac419c6f91324 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.335+0000 7fe7f0c00640 1 --2- 172.21.15.69:0/3475413237 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe7d80a3d90 0x7fe7d80a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.335+0000 7fe7e8c00640 1 -- 172.21.15.69:0/2358892921 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40df070 con 0x561e18aecce0 2024-04-24T09:12:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.335+0000 7fe7e8c00640 1 -- 172.21.15.69:0/2358892921 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7e40d0e50 con 0x561e18aecce0 2024-04-24T09:12:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.335+0000 7fe7e8c00640 1 -- 172.21.15.69:0/2358892921 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40984f0 con 0x561e18aecce0 2024-04-24T09:12:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.335+0000 7fe7f0c00640 1 -- 172.21.15.69:0/3475413237 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe7e40d0650 con 0x7fe7f4102fc0 2024-04-24T09:12:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.335+0000 7fe7f3e00640 1 --2- 172.21.15.69:0/3475413237 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe7d80a3d90 0x7fe7d80a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.336+0000 7fe7fd660740 1 -- 172.21.15.69:0/2358892921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 msgr2=0x561e18af7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.336+0000 7fe7fd660740 1 --2- 172.21.15.69:0/2358892921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18af7060 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40cd940 tx=0x7fe7e40d6cb0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.336+0000 7fe7f3e00640 1 --2- 172.21.15.69:0/3475413237 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe7d80a3d90 0x7fe7d80a6250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0092f0 tx=0x7fe7ec009280 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.336+0000 7fe7fd660740 1 -- 172.21.15.69:0/2358892921 shutdown_connections 2024-04-24T09:12:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.336+0000 7fe7fd660740 1 --2- 172.21.15.69:0/2358892921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18af75a0 0x561e18af9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.336+0000 7fe7fd660740 1 --2- 172.21.15.69:0/2358892921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18af7060 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.336+0000 7fe7fd660740 1 --2- 172.21.15.69:0/2358892921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18aec3c0 0x561e18aec7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.336+0000 7fe7fd660740 1 -- 172.21.15.69:0/2358892921 >> 172.21.15.69:0/2358892921 conn(0x561e18ae88d0 msgr2=0x561e18ae8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.336+0000 7fe7fd660740 1 -- 172.21.15.69:0/2358892921 shutdown_connections 2024-04-24T09:12:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.336+0000 7fe7fd660740 1 -- 172.21.15.69:0/2358892921 wait complete. 2024-04-24T09:12:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.337+0000 7fe7fd660740 1 -- start start 2024-04-24T09:12:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.337+0000 7fe7fd660740 1 Processor -- start 2024-04-24T09:12:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.337+0000 7fe7fd660740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18aec3c0 0x561e18af4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.337+0000 7fe7fd660740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18af4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.337+0000 7fe7fd660740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18af75a0 0x561e18af4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.337+0000 7fe7fd660740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561e18afa140 con 0x561e18aecce0 2024-04-24T09:12:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.337+0000 7fe7fd660740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561e18a0c2c0 con 0x561e18af75a0 2024-04-24T09:12:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.337+0000 7fe7fd660740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x561e18afa2c0 con 0x561e18aec3c0 2024-04-24T09:12:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.337+0000 7fe7fd660740 10 client.0 _ll_register_callbacks cb 0x561e18764a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.338+0000 7fe7ebe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18af4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.338+0000 7fe7ebe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18af4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37194/0 (socket says 172.21.15.69:37194) 2024-04-24T09:12:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.338+0000 7fe7ebe00640 1 -- 172.21.15.69:0/3282480726 learned_addr learned my addr 172.21.15.69:0/3282480726 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.338+0000 7fe7f3400640 1 --2- 172.21.15.69:0/3282480726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18aec3c0 0x561e18af4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.338+0000 7fe7ebe00640 1 -- 172.21.15.69:0/3282480726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18aec3c0 msgr2=0x561e18af4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.338+0000 7fe7ebe00640 1 --2- 172.21.15.69:0/3282480726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18aec3c0 0x561e18af4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.338+0000 7fe7ebe00640 1 -- 172.21.15.69:0/3282480726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18af75a0 msgr2=0x561e18af4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.338+0000 7fe7ebe00640 1 --2- 172.21.15.69:0/3282480726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18af75a0 0x561e18af4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.338+0000 7fe7ebe00640 1 -- 172.21.15.69:0/3282480726 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e18a1cfd0 con 0x561e18aecce0 2024-04-24T09:12:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.338+0000 7fe7ebe00640 1 --2- 172.21.15.69:0/3282480726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18af4a60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009230 tx=0x7fe7dc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3236ecab1efca13c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.338+0000 7fe7e1600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:12:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.339+0000 7fe7e1600640 1 -- 172.21.15.69:0/3282480726 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc035030 con 0x561e18aecce0 2024-04-24T09:12:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.339+0000 7fe7e1600640 1 -- 172.21.15.69:0/3282480726 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe7dc0048c0 con 0x561e18aecce0 2024-04-24T09:12:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.339+0000 7fe7fd660740 1 -- 172.21.15.69:0/3282480726 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e40d0e50 con 0x561e18aecce0 2024-04-24T09:12:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.339+0000 7fe7e1600640 1 -- 172.21.15.69:0/3282480726 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe7dc005d50 con 0x561e18aecce0 2024-04-24T09:12:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.339+0000 7fe7fd660740 10 client.16740 fetch_fsmap learned FSMap version 30 2024-04-24T09:12:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.339+0000 7fe7fd660740 1 -- 172.21.15.69:0/3282480726 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e18765210 con 0x561e18aecce0 2024-04-24T09:12:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.339+0000 7fe7e1600640 1 -- 172.21.15.69:0/3282480726 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7e40d0e50 con 0x561e18aecce0 2024-04-24T09:12:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.339+0000 7fe7e1600640 1 -- 172.21.15.69:0/3282480726 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe7dc0314a0 con 0x561e18aecce0 2024-04-24T09:12:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.340+0000 7fe7fd660740 10 client.16740 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:12:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.340+0000 7fe7fd660740 10 client.16740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.340+0000 7fe7fd660740 20 client.16740 populate_metadata read hostname 'smithi069' 2024-04-24T09:12:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.340+0000 7fe7fd660740 1 --2- 172.21.15.69:0/3282480726 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x561e18b07770 0x561e18b09ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.340+0000 7fe7fd660740 4 client.16740 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.340+0000 7fe7fd660740 1 -- 172.21.15.69:0/3282480726 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e18a60320 con 0x561e18b07770 2024-04-24T09:12:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.340+0000 7fe7f3400640 1 --2- 172.21.15.69:0/3282480726 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x561e18b07770 0x561e18b09ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.341+0000 7fe7f3400640 1 --2- 172.21.15.69:0/3282480726 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x561e18b07770 0x561e18b09ba0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa90ec72f6e5ee64 server_cookie=e87788085d2e733 in_seq=0 out_seq=0 2024-04-24T09:12:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.341+0000 7fe7e1600640 10 client.16740 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:12:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.343+0000 7fe7e1600640 1 -- 172.21.15.69:0/3282480726 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e18a60320 con 0x561e18b07770 2024-04-24T09:12:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.343+0000 7fe7e1600640 10 client.16740 handle_command_reply: tid=0 2024-04-24T09:12:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.343+0000 7fe7fd660740 1 client.16740 shutdown 2024-04-24T09:12:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.343+0000 7fe7fd660740 1 -- 172.21.15.69:0/3282480726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 msgr2=0x561e18af4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.343+0000 7fe7fd660740 1 --2- 172.21.15.69:0/3282480726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18af4a60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009230 tx=0x7fe7dc005270 comp rx=0 tx=0).stop 2024-04-24T09:12:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.343+0000 7fe7fd660740 1 -- 172.21.15.69:0/3282480726 shutdown_connections 2024-04-24T09:12:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.343+0000 7fe7fd660740 1 --2- 172.21.15.69:0/3282480726 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x561e18b07770 0x561e18b09ba0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.343+0000 7fe7fd660740 1 --2- 172.21.15.69:0/3282480726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18af75a0 0x561e18af4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.343+0000 7fe7e1600640 0 client.16740 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:12:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.343+0000 7fe7fd660740 1 --2- 172.21.15.69:0/3282480726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18af4a60 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.343+0000 7fe7fd660740 1 --2- 172.21.15.69:0/3282480726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18aec3c0 0x561e18af4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.343+0000 7fe7fd660740 1 -- 172.21.15.69:0/3282480726 >> 172.21.15.69:0/3282480726 conn(0x561e18ae88d0 msgr2=0x561e18aee650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.344+0000 7fe7fd660740 1 -- 172.21.15.69:0/3282480726 shutdown_connections 2024-04-24T09:12:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.344+0000 7fe7fd660740 1 -- 172.21.15.69:0/3282480726 wait complete. 2024-04-24T09:12:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.344+0000 7fe7fd660740 20 client.16740 trim_cache size 0 max 16384 2024-04-24T09:12:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.361+0000 7fe7fd660740 1 Processor -- start 2024-04-24T09:12:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.361+0000 7fe7fd660740 1 -- start start 2024-04-24T09:12:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.361+0000 7fe7fd660740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18aec3c0 0x561e18b030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.361+0000 7fe7fd660740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18aeed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.361+0000 7fe7fd660740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18af75a0 0x561e18af4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.361+0000 7fe7fd660740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f41b3c00 con 0x561e18aecce0 2024-04-24T09:12:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.361+0000 7fe7fd660740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f41b1470 con 0x561e18af75a0 2024-04-24T09:12:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.361+0000 7fe7fd660740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f41af2c0 con 0x561e18aec3c0 2024-04-24T09:12:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.361+0000 7fe7ebe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18aeed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.362+0000 7fe7ebe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18aeed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37198/0 (socket says 172.21.15.69:37198) 2024-04-24T09:12:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.362+0000 7fe7ebe00640 1 -- 172.21.15.69:0/1698595706 learned_addr learned my addr 172.21.15.69:0/1698595706 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.362+0000 7fe7f3e00640 1 --2- 172.21.15.69:0/1698595706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18af75a0 0x561e18af4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.362+0000 7fe7f3400640 1 --2- 172.21.15.69:0/1698595706 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18aec3c0 0x561e18b030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.362+0000 7fe7ebe00640 1 -- 172.21.15.69:0/1698595706 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18aec3c0 msgr2=0x561e18b030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.362+0000 7fe7ebe00640 1 --2- 172.21.15.69:0/1698595706 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18aec3c0 0x561e18b030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.362+0000 7fe7ebe00640 1 -- 172.21.15.69:0/1698595706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18af75a0 msgr2=0x561e18af4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.362+0000 7fe7ebe00640 1 --2- 172.21.15.69:0/1698595706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18af75a0 0x561e18af4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.362+0000 7fe7ebe00640 1 -- 172.21.15.69:0/1698595706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e18af34d0 con 0x561e18aecce0 2024-04-24T09:12:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.362+0000 7fe7ebe00640 1 --2- 172.21.15.69:0/1698595706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18aeed80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc031a60 tx=0x7fe7dc0097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77a1287e0c76aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.363+0000 7fe7e3e00640 1 -- 172.21.15.69:0/1698595706 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc035030 con 0x561e18aecce0 2024-04-24T09:12:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.363+0000 7fe7e3e00640 1 -- 172.21.15.69:0/1698595706 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7dc03f070 con 0x561e18aecce0 2024-04-24T09:12:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.363+0000 7fe7e3e00640 1 -- 172.21.15.69:0/1698595706 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc004dd0 con 0x561e18aecce0 2024-04-24T09:12:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.363+0000 7fe7fd660740 1 -- 172.21.15.69:0/1698595706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 msgr2=0x561e18aeed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.363+0000 7fe7fd660740 1 --2- 172.21.15.69:0/1698595706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18aeed80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc031a60 tx=0x7fe7dc0097c0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.364+0000 7fe7fd660740 1 -- 172.21.15.69:0/1698595706 shutdown_connections 2024-04-24T09:12:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.364+0000 7fe7fd660740 1 --2- 172.21.15.69:0/1698595706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18af75a0 0x561e18af4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.364+0000 7fe7fd660740 1 --2- 172.21.15.69:0/1698595706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aecce0 0x561e18aeed80 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.364+0000 7fe7fd660740 1 --2- 172.21.15.69:0/1698595706 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18aec3c0 0x561e18b030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.364+0000 7fe7fd660740 1 -- 172.21.15.69:0/1698595706 >> 172.21.15.69:0/1698595706 conn(0x561e18ae88d0 msgr2=0x561e18ae8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.364+0000 7fe7fd660740 1 -- 172.21.15.69:0/1698595706 shutdown_connections 2024-04-24T09:12:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.364+0000 7fe7fd660740 1 -- 172.21.15.69:0/1698595706 wait complete. 2024-04-24T09:12:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.364+0000 7fe7fd660740 1 -- start start 2024-04-24T09:12:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.364+0000 7fe7fd660740 1 Processor -- start 2024-04-24T09:12:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.365+0000 7fe7fd660740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aec3c0 0x561e18aff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.365+0000 7fe7fd660740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18aecce0 0x561e18affb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.365+0000 7fe7fd660740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18af75a0 0x561e18afc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.365+0000 7fe7fd660740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f41af930 con 0x561e18aec3c0 2024-04-24T09:12:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.365+0000 7fe7fd660740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f41b0da0 con 0x561e18af75a0 2024-04-24T09:12:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.365+0000 7fe7fd660740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f41b0000 con 0x561e18aecce0 2024-04-24T09:12:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.365+0000 7fe7fd660740 10 client.0 _ll_register_callbacks cb 0x7fe7f406a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.365+0000 7fe7f3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aec3c0 0x561e18aff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.365+0000 7fe7f3400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aec3c0 0x561e18aff600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37208/0 (socket says 172.21.15.69:37208) 2024-04-24T09:12:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.365+0000 7fe7f3400640 1 -- 172.21.15.69:0/2528872456 learned_addr learned my addr 172.21.15.69:0/2528872456 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.365+0000 7fe7ebe00640 1 --2- 172.21.15.69:0/2528872456 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18aecce0 0x561e18affb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.365+0000 7fe7f3e00640 1 --2- 172.21.15.69:0/2528872456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18af75a0 0x561e18afc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.366+0000 7fe7f3400640 1 -- 172.21.15.69:0/2528872456 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18aecce0 msgr2=0x561e18affb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.366+0000 7fe7f3400640 1 --2- 172.21.15.69:0/2528872456 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18aecce0 0x561e18affb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.366+0000 7fe7f3400640 1 -- 172.21.15.69:0/2528872456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18af75a0 msgr2=0x561e18afc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.366+0000 7fe7f3400640 1 --2- 172.21.15.69:0/2528872456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18af75a0 0x561e18afc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.366+0000 7fe7f3400640 1 -- 172.21.15.69:0/2528872456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e18831620 con 0x561e18aec3c0 2024-04-24T09:12:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.366+0000 7fe7f3e00640 1 --2- 172.21.15.69:0/2528872456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18af75a0 0x561e18afc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.366+0000 7fe7ebe00640 1 --2- 172.21.15.69:0/2528872456 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18aecce0 0x561e18affb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.366+0000 7fe7f3400640 1 --2- 172.21.15.69:0/2528872456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aec3c0 0x561e18aff600 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4098c20 tx=0x7fe7e4016fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb8e90000b1cef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.366+0000 7fe7e1600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:12:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.366+0000 7fe7e1600640 1 -- 172.21.15.69:0/2528872456 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e404a640 con 0x561e18aec3c0 2024-04-24T09:12:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.366+0000 7fe7e1600640 1 -- 172.21.15.69:0/2528872456 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe7e40d66d0 con 0x561e18aec3c0 2024-04-24T09:12:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.366+0000 7fe7fd660740 1 -- 172.21.15.69:0/2528872456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7dc03f070 con 0x561e18aec3c0 2024-04-24T09:12:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.366+0000 7fe7e1600640 1 -- 172.21.15.69:0/2528872456 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe7e40488f0 con 0x561e18aec3c0 2024-04-24T09:12:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.367+0000 7fe7fd660740 10 client.16746 fetch_fsmap learned FSMap version 30 2024-04-24T09:12:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.367+0000 7fe7fd660740 1 -- 172.21.15.69:0/2528872456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e18b6e530 con 0x561e18aec3c0 2024-04-24T09:12:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.367+0000 7fe7e1600640 1 -- 172.21.15.69:0/2528872456 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7dc03f070 con 0x561e18aec3c0 2024-04-24T09:12:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.367+0000 7fe7e1600640 1 -- 172.21.15.69:0/2528872456 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe7e40988c0 con 0x561e18aec3c0 2024-04-24T09:12:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.367+0000 7fe7fd660740 10 client.16746 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:12:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.367+0000 7fe7fd660740 10 client.16746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.367+0000 7fe7fd660740 20 client.16746 populate_metadata read hostname 'smithi069' 2024-04-24T09:12:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.367+0000 7fe7fd660740 1 --2- 172.21.15.69:0/2528872456 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x561e18b6e830 0x561e18b70c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.367+0000 7fe7fd660740 4 client.16746 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:12:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.367+0000 7fe7fd660740 1 -- 172.21.15.69:0/2528872456 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e18a5a190 con 0x561e18b6e830 2024-04-24T09:12:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.368+0000 7fe7ebe00640 1 --2- 172.21.15.69:0/2528872456 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x561e18b6e830 0x561e18b70c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.369+0000 7fe7ebe00640 1 --2- 172.21.15.69:0/2528872456 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x561e18b6e830 0x561e18b70c20 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b33d2d341df4452 server_cookie=e4fd44ffcb886d60 in_seq=0 out_seq=0 2024-04-24T09:12:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.369+0000 7fe7e1600640 10 client.16746 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:12:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.469+0000 7fe7e1600640 1 -- 172.21.15.69:0/2528872456 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e18a5a190 con 0x561e18b6e830 2024-04-24T09:12:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.469+0000 7fe7e1600640 10 client.16746 handle_command_reply: tid=0 2024-04-24T09:12:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.469+0000 7fe7fd660740 1 client.16746 shutdown 2024-04-24T09:12:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.469+0000 7fe7fd660740 1 -- 172.21.15.69:0/2528872456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aec3c0 msgr2=0x561e18aff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.469+0000 7fe7fd660740 1 --2- 172.21.15.69:0/2528872456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aec3c0 0x561e18aff600 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4098c20 tx=0x7fe7e4016fe0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.469+0000 7fe7fd660740 1 -- 172.21.15.69:0/2528872456 shutdown_connections 2024-04-24T09:12:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.469+0000 7fe7fd660740 1 --2- 172.21.15.69:0/2528872456 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x561e18b6e830 0x561e18b70c20 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.469+0000 7fe7fd660740 1 --2- 172.21.15.69:0/2528872456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e18af75a0 0x561e18afc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.469+0000 7fe7fd660740 1 --2- 172.21.15.69:0/2528872456 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561e18aecce0 0x561e18affb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.469+0000 7fe7fd660740 1 --2- 172.21.15.69:0/2528872456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e18aec3c0 0x561e18aff600 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.469+0000 7fe7fd660740 1 -- 172.21.15.69:0/2528872456 >> 172.21.15.69:0/2528872456 conn(0x561e18ae88d0 msgr2=0x561e18ae7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.469+0000 7fe7e1600640 0 client.16746 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:12:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.469+0000 7fe7fd660740 1 -- 172.21.15.69:0/2528872456 shutdown_connections 2024-04-24T09:12:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.469+0000 7fe7fd660740 1 -- 172.21.15.69:0/2528872456 wait complete. 2024-04-24T09:12:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.470+0000 7fe7fd660740 20 client.16746 trim_cache size 0 max 16384 2024-04-24T09:12:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:12:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:12:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:12:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:12:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.470+0000 7fe7f9200640 1 -- 172.21.15.69:0/3475413237 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe7d80a3d90 msgr2=0x7fe7d80a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.470+0000 7fe7f9200640 1 --2- 172.21.15.69:0/3475413237 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe7d80a3d90 0x7fe7d80a6250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0092f0 tx=0x7fe7ec009280 comp rx=0 tx=0).stop 2024-04-24T09:12:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.470+0000 7fe7f9200640 1 -- 172.21.15.69:0/3475413237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7f4102fc0 msgr2=0x7fe7f41a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.470+0000 7fe7f9200640 1 --2- 172.21.15.69:0/3475413237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7f4102fc0 0x7fe7f41a05b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400d770 tx=0x7fe7e400dc30 comp rx=0 tx=0).stop 2024-04-24T09:12:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.470+0000 7fe7f9200640 1 -- 172.21.15.69:0/3475413237 shutdown_connections 2024-04-24T09:12:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.470+0000 7fe7f9200640 1 --2- 172.21.15.69:0/3475413237 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe7d80a3d90 0x7fe7d80a6250 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.470+0000 7fe7f9200640 1 --2- 172.21.15.69:0/3475413237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7f410c3e0 0x7fe7f419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.470+0000 7fe7f9200640 1 --2- 172.21.15.69:0/3475413237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7f41038e0 0x7fe7f41a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.470+0000 7fe7f9200640 1 --2- 172.21.15.69:0/3475413237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7f4102fc0 0x7fe7f41a05b0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.470+0000 7fe7f9200640 1 -- 172.21.15.69:0/3475413237 >> 172.21.15.69:0/3475413237 conn(0x7fe7f40fc820 msgr2=0x7fe7f4101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.471+0000 7fe7f9200640 1 -- 172.21.15.69:0/3475413237 shutdown_connections 2024-04-24T09:12:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:07.471+0000 7fe7f9200640 1 -- 172.21.15.69:0/3475413237 wait complete. 2024-04-24T09:12:07.566 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:07 smithi081 ceph-mon[30961]: pgmap v154: 129 pgs: 129 active+clean; 91 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 806 KiB/s wr, 167 op/s 2024-04-24T09:12:07.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:07 smithi069 ceph-mon[28170]: pgmap v154: 129 pgs: 129 active+clean; 91 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 806 KiB/s wr, 167 op/s 2024-04-24T09:12:07.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:07 smithi130 ceph-mon[31553]: pgmap v154: 129 pgs: 129 active+clean; 91 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 806 KiB/s wr, 167 op/s 2024-04-24T09:12:09.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:09 smithi081 ceph-mon[30961]: pgmap v155: 129 pgs: 129 active+clean; 91 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 538 KiB/s wr, 95 op/s 2024-04-24T09:12:09.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:09 smithi069 ceph-mon[28170]: pgmap v155: 129 pgs: 129 active+clean; 91 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 538 KiB/s wr, 95 op/s 2024-04-24T09:12:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:09 smithi130 ceph-mon[31553]: pgmap v155: 129 pgs: 129 active+clean; 91 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 538 KiB/s wr, 95 op/s 2024-04-24T09:12:11.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:11 smithi081 ceph-mon[30961]: pgmap v156: 129 pgs: 129 active+clean; 90 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 817 KiB/s wr, 159 op/s 2024-04-24T09:12:11.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:11 smithi069 ceph-mon[28170]: pgmap v156: 129 pgs: 129 active+clean; 90 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 817 KiB/s wr, 159 op/s 2024-04-24T09:12:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:11 smithi130 ceph-mon[31553]: pgmap v156: 129 pgs: 129 active+clean; 90 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 817 KiB/s wr, 159 op/s 2024-04-24T09:12:13.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:13 smithi081 ceph-mon[30961]: pgmap v157: 129 pgs: 129 active+clean; 90 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 568 KiB/s wr, 119 op/s 2024-04-24T09:12:13.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:13 smithi069 ceph-mon[28170]: pgmap v157: 129 pgs: 129 active+clean; 90 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 568 KiB/s wr, 119 op/s 2024-04-24T09:12:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:13 smithi130 ceph-mon[31553]: pgmap v157: 129 pgs: 129 active+clean; 90 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 568 KiB/s wr, 119 op/s 2024-04-24T09:12:15.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:15 smithi081 ceph-mon[30961]: pgmap v158: 129 pgs: 129 active+clean; 90 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 568 KiB/s wr, 119 op/s 2024-04-24T09:12:15.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:15 smithi069 ceph-mon[28170]: pgmap v158: 129 pgs: 129 active+clean; 90 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 568 KiB/s wr, 119 op/s 2024-04-24T09:12:15.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:15 smithi130 ceph-mon[31553]: pgmap v158: 129 pgs: 129 active+clean; 90 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 568 KiB/s wr, 119 op/s 2024-04-24T09:12:17.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:17 smithi081 ceph-mon[30961]: pgmap v159: 129 pgs: 129 active+clean; 92 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 734 KiB/s wr, 167 op/s 2024-04-24T09:12:17.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:17 smithi069 ceph-mon[28170]: pgmap v159: 129 pgs: 129 active+clean; 92 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 734 KiB/s wr, 167 op/s 2024-04-24T09:12:17.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:17 smithi130 ceph-mon[31553]: pgmap v159: 129 pgs: 129 active+clean; 92 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 734 KiB/s wr, 167 op/s 2024-04-24T09:12:19.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:19 smithi069 ceph-mon[28170]: pgmap v160: 129 pgs: 129 active+clean; 92 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 445 KiB/s wr, 112 op/s 2024-04-24T09:12:19.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:19 smithi130 ceph-mon[31553]: pgmap v160: 129 pgs: 129 active+clean; 92 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 445 KiB/s wr, 112 op/s 2024-04-24T09:12:19.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:19 smithi081 ceph-mon[30961]: pgmap v160: 129 pgs: 129 active+clean; 92 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 445 KiB/s wr, 112 op/s 2024-04-24T09:12:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.621+0000 7f946a800640 1 -- 172.21.15.69:0/3128210122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464108ec0 msgr2=0x7f9464115300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.621+0000 7f946a800640 1 --2- 172.21.15.69:0/3128210122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464108ec0 0x7f9464115300 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f945c009cd0 tx=0x7f945c02a8e0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.621+0000 7f946a800640 1 -- 172.21.15.69:0/3128210122 shutdown_connections 2024-04-24T09:12:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.621+0000 7f946a800640 1 --2- 172.21.15.69:0/3128210122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464108ec0 0x7f9464115300 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.621+0000 7f946a800640 1 --2- 172.21.15.69:0/3128210122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9464108540 0x7f9464108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.621+0000 7f946a800640 1 --2- 172.21.15.69:0/3128210122 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f946410e4d0 0x7f946410e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.621+0000 7f946a800640 1 -- 172.21.15.69:0/3128210122 >> 172.21.15.69:0/3128210122 conn(0x7f9464077de0 msgr2=0x7f9464106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.621+0000 7f946a800640 1 -- 172.21.15.69:0/3128210122 shutdown_connections 2024-04-24T09:12:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.622+0000 7f946a800640 1 -- 172.21.15.69:0/3128210122 wait complete. 2024-04-24T09:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.622+0000 7f946a800640 1 Processor -- start 2024-04-24T09:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.623+0000 7f946a800640 1 -- start start 2024-04-24T09:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.623+0000 7f946a800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9464108540 0x7f94641a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.623+0000 7f946a800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464108ec0 0x7f94641a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.623+0000 7f946a800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f946410e4d0 0x7f946419e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.623+0000 7f946a800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f94641186e0 con 0x7f9464108ec0 2024-04-24T09:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.623+0000 7f946a800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9464118560 con 0x7f946410e4d0 2024-04-24T09:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.623+0000 7f946a800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9464118860 con 0x7f9464108540 2024-04-24T09:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.623+0000 7f9468a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464108ec0 0x7f94641a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.623+0000 7f9468a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464108ec0 0x7f94641a4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56678/0 (socket says 172.21.15.69:56678) 2024-04-24T09:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.623+0000 7f9468a00640 1 -- 172.21.15.69:0/3995947883 learned_addr learned my addr 172.21.15.69:0/3995947883 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.624+0000 7f9469400640 1 --2- 172.21.15.69:0/3995947883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9464108540 0x7f94641a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.624+0000 7f9469e00640 1 --2- 172.21.15.69:0/3995947883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f946410e4d0 0x7f946419e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.624+0000 7f9468a00640 1 -- 172.21.15.69:0/3995947883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9464108540 msgr2=0x7f94641a47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.624+0000 7f9468a00640 1 --2- 172.21.15.69:0/3995947883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9464108540 0x7f94641a47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.624+0000 7f9468a00640 1 -- 172.21.15.69:0/3995947883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f946410e4d0 msgr2=0x7f946419e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.624+0000 7f9468a00640 1 --2- 172.21.15.69:0/3995947883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f946410e4d0 0x7f946419e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.624+0000 7f9468a00640 1 -- 172.21.15.69:0/3995947883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946419f160 con 0x7f9464108ec0 2024-04-24T09:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.624+0000 7f9469e00640 1 --2- 172.21.15.69:0/3995947883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f946410e4d0 0x7f946419e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.624+0000 7f9469400640 1 --2- 172.21.15.69:0/3995947883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9464108540 0x7f94641a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.624+0000 7f9468a00640 1 --2- 172.21.15.69:0/3995947883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464108ec0 0x7f94641a4d20 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f945000d950 tx=0x7f945000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.624+0000 7f9462000640 1 -- 172.21.15.69:0/3995947883 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450017070 con 0x7f9464108ec0 2024-04-24T09:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.625+0000 7f9462000640 1 -- 172.21.15.69:0/3995947883 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9450009c30 con 0x7f9464108ec0 2024-04-24T09:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.625+0000 7f946a800640 1 -- 172.21.15.69:0/3995947883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946419f450 con 0x7f9464108ec0 2024-04-24T09:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.625+0000 7f9462000640 1 -- 172.21.15.69:0/3995947883 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945000f620 con 0x7f9464108ec0 2024-04-24T09:12:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.625+0000 7f946a800640 1 -- 172.21.15.69:0/3995947883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94641ab6a0 con 0x7f9464108ec0 2024-04-24T09:12:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.628+0000 7f9462000640 1 -- 172.21.15.69:0/3995947883 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f94500109a0 con 0x7f9464108ec0 2024-04-24T09:12:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.628+0000 7f946ede5740 1 Processor -- start 2024-04-24T09:12:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.628+0000 7f946ede5740 1 -- start start 2024-04-24T09:12:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.629+0000 7f946ede5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ec3c0 0x5640d64ec7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.629+0000 7f946ede5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ecce0 0x5640d64f7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.629+0000 7f946ede5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64f75a0 0x5640d64f9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.629+0000 7f946ede5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5640d6452900 con 0x5640d64ec3c0 2024-04-24T09:12:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.629+0000 7f946ede5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5640d63f8f20 con 0x5640d64f75a0 2024-04-24T09:12:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.629+0000 7f946ede5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5640d644e8c0 con 0x5640d64ecce0 2024-04-24T09:12:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.629+0000 7f9469e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ec3c0 0x5640d64ec7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.629+0000 7f9469e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ec3c0 0x5640d64ec7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56680/0 (socket says 172.21.15.69:56680) 2024-04-24T09:12:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.629+0000 7f9469e00640 1 -- 172.21.15.69:0/34214420 learned_addr learned my addr 172.21.15.69:0/34214420 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.629+0000 7f9468a00640 1 --2- 172.21.15.69:0/34214420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64f75a0 0x5640d64f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.629+0000 7f9469400640 1 --2- 172.21.15.69:0/34214420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ecce0 0x5640d64f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.630+0000 7f9469e00640 1 -- 172.21.15.69:0/34214420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ecce0 msgr2=0x5640d64f7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.630+0000 7f9469e00640 1 --2- 172.21.15.69:0/34214420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ecce0 0x5640d64f7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.630+0000 7f9469e00640 1 -- 172.21.15.69:0/34214420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64f75a0 msgr2=0x5640d64f9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.630+0000 7f9469e00640 1 --2- 172.21.15.69:0/34214420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64f75a0 0x5640d64f9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.630+0000 7f9469e00640 1 -- 172.21.15.69:0/34214420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640d641de10 con 0x5640d64ec3c0 2024-04-24T09:12:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.630+0000 7f9462000640 1 --2- 172.21.15.69:0/3995947883 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f944c0a3d10 0x7f944c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.630+0000 7f9469e00640 1 --2- 172.21.15.69:0/34214420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ec3c0 0x5640d64ec7a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f945c009df0 tx=0x7f945c031430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17248ad2639a31bd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.630+0000 7f9462000640 1 -- 172.21.15.69:0/3995947883 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f94500d0850 con 0x7f9464108ec0 2024-04-24T09:12:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.630+0000 7f9469400640 1 --2- 172.21.15.69:0/3995947883 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f944c0a3d10 0x7f944c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.630+0000 7f9456600640 1 -- 172.21.15.69:0/34214420 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c031850 con 0x5640d64ec3c0 2024-04-24T09:12:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.630+0000 7f9456600640 1 -- 172.21.15.69:0/34214420 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f945c036040 con 0x5640d64ec3c0 2024-04-24T09:12:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.631+0000 7f9456600640 1 -- 172.21.15.69:0/34214420 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c004510 con 0x5640d64ec3c0 2024-04-24T09:12:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.631+0000 7f946ede5740 1 -- 172.21.15.69:0/34214420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ec3c0 msgr2=0x5640d64ec7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.631+0000 7f9469400640 1 --2- 172.21.15.69:0/3995947883 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f944c0a3d10 0x7f944c0a61d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9458005fd0 tx=0x7f945800b650 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.631+0000 7f946ede5740 1 --2- 172.21.15.69:0/34214420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ec3c0 0x5640d64ec7a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f945c009df0 tx=0x7f945c031430 comp rx=0 tx=0).stop 2024-04-24T09:12:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.631+0000 7f946ede5740 1 -- 172.21.15.69:0/34214420 shutdown_connections 2024-04-24T09:12:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.631+0000 7f946ede5740 1 --2- 172.21.15.69:0/34214420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64f75a0 0x5640d64f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.631+0000 7f946ede5740 1 --2- 172.21.15.69:0/34214420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ecce0 0x5640d64f7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.631+0000 7f946ede5740 1 --2- 172.21.15.69:0/34214420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ec3c0 0x5640d64ec7a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.631+0000 7f946ede5740 1 -- 172.21.15.69:0/34214420 >> 172.21.15.69:0/34214420 conn(0x5640d64e88d0 msgr2=0x5640d64e8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.631+0000 7f946ede5740 1 -- 172.21.15.69:0/34214420 shutdown_connections 2024-04-24T09:12:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.632+0000 7f946ede5740 1 -- 172.21.15.69:0/34214420 wait complete. 2024-04-24T09:12:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.632+0000 7f946ede5740 1 -- start start 2024-04-24T09:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.632+0000 7f946ede5740 1 Processor -- start 2024-04-24T09:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f946ede5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ec3c0 0x5640d64f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f946ede5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ecce0 0x5640d64f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f946ede5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64f75a0 0x5640d64f5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f946ede5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5640d64f9fc0 con 0x5640d64ecce0 2024-04-24T09:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f946ede5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5640d640c2c0 con 0x5640d64f75a0 2024-04-24T09:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f946ede5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5640d64fa140 con 0x5640d64ec3c0 2024-04-24T09:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f946ede5740 10 client.0 _ll_register_callbacks cb 0x5640d6164a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f9468a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ecce0 0x5640d64f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f9468a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ecce0 0x5640d64f4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56686/0 (socket says 172.21.15.69:56686) 2024-04-24T09:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f9468a00640 1 -- 172.21.15.69:0/674705566 learned_addr learned my addr 172.21.15.69:0/674705566 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f9469e00640 1 --2- 172.21.15.69:0/674705566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64f75a0 0x5640d64f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f9469400640 1 --2- 172.21.15.69:0/674705566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ec3c0 0x5640d64f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f9468a00640 1 -- 172.21.15.69:0/674705566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ec3c0 msgr2=0x5640d64f46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f9468a00640 1 --2- 172.21.15.69:0/674705566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ec3c0 0x5640d64f46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f9468a00640 1 -- 172.21.15.69:0/674705566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64f75a0 msgr2=0x5640d64f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f9468a00640 1 --2- 172.21.15.69:0/674705566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64f75a0 0x5640d64f5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f9468a00640 1 -- 172.21.15.69:0/674705566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640d641cfd0 con 0x5640d64ecce0 2024-04-24T09:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.633+0000 7f9469e00640 1 --2- 172.21.15.69:0/674705566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64f75a0 0x5640d64f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.634+0000 7f9468a00640 1 --2- 172.21.15.69:0/674705566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ecce0 0x5640d64f4be0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f945009c150 tx=0x7f945009bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f2fe83d27e8707 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.634+0000 7f944aa00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.634+0000 7f944aa00640 1 -- 172.21.15.69:0/674705566 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450020050 con 0x5640d64ecce0 2024-04-24T09:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.634+0000 7f944aa00640 1 -- 172.21.15.69:0/674705566 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f94500a0500 con 0x5640d64ecce0 2024-04-24T09:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.634+0000 7f946ede5740 1 -- 172.21.15.69:0/674705566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f945c036040 con 0x5640d64ecce0 2024-04-24T09:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.634+0000 7f944aa00640 1 -- 172.21.15.69:0/674705566 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f94500ce080 con 0x5640d64ecce0 2024-04-24T09:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.634+0000 7f946ede5740 10 client.16758 fetch_fsmap learned FSMap version 30 2024-04-24T09:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.634+0000 7f946ede5740 1 -- 172.21.15.69:0/674705566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5640d6165210 con 0x5640d64ecce0 2024-04-24T09:12:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.634+0000 7f944aa00640 1 -- 172.21.15.69:0/674705566 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f945c036040 con 0x5640d64ecce0 2024-04-24T09:12:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.635+0000 7f944aa00640 1 -- 172.21.15.69:0/674705566 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f94500a01b0 con 0x5640d64ecce0 2024-04-24T09:12:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.635+0000 7f946ede5740 10 client.16758 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:12:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.635+0000 7f946ede5740 10 client.16758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.635+0000 7f946ede5740 20 client.16758 populate_metadata read hostname 'smithi069' 2024-04-24T09:12:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.635+0000 7f946ede5740 1 --2- 172.21.15.69:0/674705566 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5640d6507790 0x5640d6509be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.635+0000 7f946ede5740 4 client.16758 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.635+0000 7f946ede5740 1 -- 172.21.15.69:0/674705566 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640d6460320 con 0x5640d6507790 2024-04-24T09:12:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.635+0000 7f9469400640 1 --2- 172.21.15.69:0/674705566 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5640d6507790 0x5640d6509be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.636+0000 7f9469400640 1 --2- 172.21.15.69:0/674705566 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5640d6507790 0x5640d6509be0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=481faf290fcbed6e server_cookie=68a8684fee10cab9 in_seq=0 out_seq=0 2024-04-24T09:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.636+0000 7f944aa00640 10 client.16758 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.638+0000 7f944aa00640 1 -- 172.21.15.69:0/674705566 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640d6460320 con 0x5640d6507790 2024-04-24T09:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.638+0000 7f944aa00640 10 client.16758 handle_command_reply: tid=0 2024-04-24T09:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.638+0000 7f946ede5740 1 client.16758 shutdown 2024-04-24T09:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.638+0000 7f946ede5740 1 -- 172.21.15.69:0/674705566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ecce0 msgr2=0x5640d64f4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.638+0000 7f946ede5740 1 --2- 172.21.15.69:0/674705566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ecce0 0x5640d64f4be0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f945009c150 tx=0x7f945009bf90 comp rx=0 tx=0).stop 2024-04-24T09:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.638+0000 7f946ede5740 1 -- 172.21.15.69:0/674705566 shutdown_connections 2024-04-24T09:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.638+0000 7f946ede5740 1 --2- 172.21.15.69:0/674705566 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5640d6507790 0x5640d6509be0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.638+0000 7f946ede5740 1 --2- 172.21.15.69:0/674705566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64f75a0 0x5640d64f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.638+0000 7f944aa00640 0 client.16758 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.638+0000 7f946ede5740 1 --2- 172.21.15.69:0/674705566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ecce0 0x5640d64f4be0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.638+0000 7f946ede5740 1 --2- 172.21.15.69:0/674705566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ec3c0 0x5640d64f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.638+0000 7f946ede5740 1 -- 172.21.15.69:0/674705566 >> 172.21.15.69:0/674705566 conn(0x5640d64e88d0 msgr2=0x5640d64ee600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.639+0000 7f946ede5740 1 -- 172.21.15.69:0/674705566 shutdown_connections 2024-04-24T09:12:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.639+0000 7f946ede5740 1 -- 172.21.15.69:0/674705566 wait complete. 2024-04-24T09:12:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.639+0000 7f946ede5740 20 client.16758 trim_cache size 0 max 16384 2024-04-24T09:12:20.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.656+0000 7f946ede5740 1 Processor -- start 2024-04-24T09:12:20.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.656+0000 7f946ede5740 1 -- start start 2024-04-24T09:12:20.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.656+0000 7f946ede5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ec3c0 0x5640d64f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.656+0000 7f946ede5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ecce0 0x5640d64f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.656+0000 7f946ede5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64f75a0 0x5640d64e7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.657+0000 7f946ede5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f94641b7ea0 con 0x5640d64ec3c0 2024-04-24T09:12:20.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.657+0000 7f946ede5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f94641b5710 con 0x5640d64f75a0 2024-04-24T09:12:20.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.657+0000 7f946ede5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f94641b3560 con 0x5640d64ecce0 2024-04-24T09:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.657+0000 7f9469400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ec3c0 0x5640d64f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.657+0000 7f9469400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ec3c0 0x5640d64f6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56688/0 (socket says 172.21.15.69:56688) 2024-04-24T09:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.657+0000 7f9469400640 1 -- 172.21.15.69:0/2684638949 learned_addr learned my addr 172.21.15.69:0/2684638949 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.657+0000 7f9469e00640 1 --2- 172.21.15.69:0/2684638949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64f75a0 0x5640d64e7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.657+0000 7f9468a00640 1 --2- 172.21.15.69:0/2684638949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ecce0 0x5640d64f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.657+0000 7f9469400640 1 -- 172.21.15.69:0/2684638949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ecce0 msgr2=0x5640d64f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.657+0000 7f9469400640 1 --2- 172.21.15.69:0/2684638949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ecce0 0x5640d64f4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.657+0000 7f9469400640 1 -- 172.21.15.69:0/2684638949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64f75a0 msgr2=0x5640d64e7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.657+0000 7f9469400640 1 --2- 172.21.15.69:0/2684638949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64f75a0 0x5640d64e7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.657+0000 7f9469400640 1 -- 172.21.15.69:0/2684638949 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640d64f9d00 con 0x5640d64ec3c0 2024-04-24T09:12:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.658+0000 7f9469400640 1 --2- 172.21.15.69:0/2684638949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ec3c0 0x5640d64f6dd0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x5640d650a490 tx=0x7f9458012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8105a7a7b06f7cce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.658+0000 7f9455c00640 1 -- 172.21.15.69:0/2684638949 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94580263a0 con 0x5640d64ec3c0 2024-04-24T09:12:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.658+0000 7f9455c00640 1 -- 172.21.15.69:0/2684638949 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f945802c070 con 0x5640d64ec3c0 2024-04-24T09:12:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.658+0000 7f9455c00640 1 -- 172.21.15.69:0/2684638949 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945801dbe0 con 0x5640d64ec3c0 2024-04-24T09:12:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.659+0000 7f946ede5740 1 -- 172.21.15.69:0/2684638949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ec3c0 msgr2=0x5640d64f6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.659+0000 7f946ede5740 1 --2- 172.21.15.69:0/2684638949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ec3c0 0x5640d64f6dd0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x5640d650a490 tx=0x7f9458012000 comp rx=0 tx=0).stop 2024-04-24T09:12:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.659+0000 7f946ede5740 1 -- 172.21.15.69:0/2684638949 shutdown_connections 2024-04-24T09:12:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.659+0000 7f946ede5740 1 --2- 172.21.15.69:0/2684638949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64f75a0 0x5640d64e7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.659+0000 7f946ede5740 1 --2- 172.21.15.69:0/2684638949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ecce0 0x5640d64f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.659+0000 7f946ede5740 1 --2- 172.21.15.69:0/2684638949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64ec3c0 0x5640d64f6dd0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.659+0000 7f946ede5740 1 -- 172.21.15.69:0/2684638949 >> 172.21.15.69:0/2684638949 conn(0x5640d64e88d0 msgr2=0x5640d64e7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.659+0000 7f946ede5740 1 -- 172.21.15.69:0/2684638949 shutdown_connections 2024-04-24T09:12:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.659+0000 7f946ede5740 1 -- 172.21.15.69:0/2684638949 wait complete. 2024-04-24T09:12:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.660+0000 7f946ede5740 1 -- start start 2024-04-24T09:12:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.660+0000 7f946ede5740 1 Processor -- start 2024-04-24T09:12:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.660+0000 7f946ede5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ec3c0 0x5640d64ff860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.660+0000 7f946ede5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64ecce0 0x5640d64fc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.660+0000 7f946ede5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64f75a0 0x5640d64fc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.660+0000 7f946ede5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f94641b3bd0 con 0x5640d64f75a0 2024-04-24T09:12:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.660+0000 7f946ede5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f94641b5040 con 0x5640d64ecce0 2024-04-24T09:12:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.660+0000 7f946ede5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f94641b42a0 con 0x5640d64ec3c0 2024-04-24T09:12:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.660+0000 7f946ede5740 10 client.0 _ll_register_callbacks cb 0x7f945000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.661+0000 7f9469e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64f75a0 0x5640d64fc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.661+0000 7f9469e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64f75a0 0x5640d64fc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56696/0 (socket says 172.21.15.69:56696) 2024-04-24T09:12:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.661+0000 7f9469e00640 1 -- 172.21.15.69:0/3970908016 learned_addr learned my addr 172.21.15.69:0/3970908016 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.661+0000 7f9468a00640 1 --2- 172.21.15.69:0/3970908016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64ecce0 0x5640d64fc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.661+0000 7f9469400640 1 --2- 172.21.15.69:0/3970908016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ec3c0 0x5640d64ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.661+0000 7f9469e00640 1 -- 172.21.15.69:0/3970908016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ec3c0 msgr2=0x5640d64ff860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.661+0000 7f9469e00640 1 --2- 172.21.15.69:0/3970908016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ec3c0 0x5640d64ff860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.661+0000 7f9469e00640 1 -- 172.21.15.69:0/3970908016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64ecce0 msgr2=0x5640d64fc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.661+0000 7f9469e00640 1 --2- 172.21.15.69:0/3970908016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64ecce0 0x5640d64fc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.661+0000 7f9469e00640 1 -- 172.21.15.69:0/3970908016 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640d6231620 con 0x5640d64f75a0 2024-04-24T09:12:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.661+0000 7f9469e00640 1 --2- 172.21.15.69:0/3970908016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64f75a0 0x5640d64fc980 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f945c005e30 tx=0x7f945c004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc75e7ab6fa4dab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.661+0000 7f944aa00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:12:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.662+0000 7f944aa00640 1 -- 172.21.15.69:0/3970908016 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c04e070 con 0x5640d64f75a0 2024-04-24T09:12:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.662+0000 7f944aa00640 1 -- 172.21.15.69:0/3970908016 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f945c003d80 con 0x5640d64f75a0 2024-04-24T09:12:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.662+0000 7f946ede5740 1 -- 172.21.15.69:0/3970908016 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f945802c070 con 0x5640d64f75a0 2024-04-24T09:12:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.662+0000 7f944aa00640 1 -- 172.21.15.69:0/3970908016 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f945c057050 con 0x5640d64f75a0 2024-04-24T09:12:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.662+0000 7f946ede5740 10 client.16764 fetch_fsmap learned FSMap version 30 2024-04-24T09:12:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.662+0000 7f946ede5740 1 -- 172.21.15.69:0/3970908016 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5640d656ec10 con 0x5640d64f75a0 2024-04-24T09:12:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.662+0000 7f944aa00640 1 -- 172.21.15.69:0/3970908016 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f945802c070 con 0x5640d64f75a0 2024-04-24T09:12:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.663+0000 7f944aa00640 1 -- 172.21.15.69:0/3970908016 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f945c051020 con 0x5640d64f75a0 2024-04-24T09:12:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.663+0000 7f946ede5740 10 client.16764 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:12:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.663+0000 7f946ede5740 10 client.16764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.663+0000 7f946ede5740 20 client.16764 populate_metadata read hostname 'smithi069' 2024-04-24T09:12:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.663+0000 7f946ede5740 1 --2- 172.21.15.69:0/3970908016 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5640d656ef10 0x5640d6571330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.663+0000 7f946ede5740 4 client.16764 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:12:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.663+0000 7f946ede5740 1 -- 172.21.15.69:0/3970908016 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5640d645a190 con 0x5640d656ef10 2024-04-24T09:12:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.663+0000 7f9469400640 1 --2- 172.21.15.69:0/3970908016 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5640d656ef10 0x5640d6571330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.664+0000 7f9469400640 1 --2- 172.21.15.69:0/3970908016 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5640d656ef10 0x5640d6571330 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c72c65fdcb099091 server_cookie=6ee5d842ac1c44b8 in_seq=0 out_seq=0 2024-04-24T09:12:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.664+0000 7f944aa00640 10 client.16764 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:12:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.747+0000 7f944aa00640 1 -- 172.21.15.69:0/3970908016 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5640d645a190 con 0x5640d656ef10 2024-04-24T09:12:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.747+0000 7f944aa00640 10 client.16764 handle_command_reply: tid=0 2024-04-24T09:12:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.747+0000 7f946ede5740 1 client.16764 shutdown 2024-04-24T09:12:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.747+0000 7f946ede5740 1 -- 172.21.15.69:0/3970908016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64f75a0 msgr2=0x5640d64fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.747+0000 7f946ede5740 1 --2- 172.21.15.69:0/3970908016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64f75a0 0x5640d64fc980 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f945c005e30 tx=0x7f945c004aa0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.747+0000 7f946ede5740 1 -- 172.21.15.69:0/3970908016 shutdown_connections 2024-04-24T09:12:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.747+0000 7f946ede5740 1 --2- 172.21.15.69:0/3970908016 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5640d656ef10 0x5640d6571330 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.747+0000 7f946ede5740 1 --2- 172.21.15.69:0/3970908016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640d64f75a0 0x5640d64fc980 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.747+0000 7f946ede5740 1 --2- 172.21.15.69:0/3970908016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5640d64ecce0 0x5640d64fc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.747+0000 7f946ede5740 1 --2- 172.21.15.69:0/3970908016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640d64ec3c0 0x5640d64ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.748+0000 7f946ede5740 1 -- 172.21.15.69:0/3970908016 >> 172.21.15.69:0/3970908016 conn(0x5640d64e88d0 msgr2=0x5640d64ee600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.748+0000 7f944aa00640 0 client.16764 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:12:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.748+0000 7f946ede5740 1 -- 172.21.15.69:0/3970908016 shutdown_connections 2024-04-24T09:12:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.748+0000 7f946ede5740 1 -- 172.21.15.69:0/3970908016 wait complete. 2024-04-24T09:12:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.748+0000 7f946ede5740 20 client.16764 trim_cache size 0 max 16384 2024-04-24T09:12:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:12:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:12:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:12:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:12:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.749+0000 7f946a800640 1 -- 172.21.15.69:0/3995947883 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f944c0a3d10 msgr2=0x7f944c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.749+0000 7f946a800640 1 --2- 172.21.15.69:0/3995947883 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f944c0a3d10 0x7f944c0a61d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9458005fd0 tx=0x7f945800b650 comp rx=0 tx=0).stop 2024-04-24T09:12:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.749+0000 7f946a800640 1 -- 172.21.15.69:0/3995947883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464108ec0 msgr2=0x7f94641a4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.749+0000 7f946a800640 1 --2- 172.21.15.69:0/3995947883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464108ec0 0x7f94641a4d20 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f945000d950 tx=0x7f945000de10 comp rx=0 tx=0).stop 2024-04-24T09:12:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.749+0000 7f946a800640 1 -- 172.21.15.69:0/3995947883 shutdown_connections 2024-04-24T09:12:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.749+0000 7f946a800640 1 --2- 172.21.15.69:0/3995947883 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f944c0a3d10 0x7f944c0a61d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.749+0000 7f946a800640 1 --2- 172.21.15.69:0/3995947883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f946410e4d0 0x7f946419e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.749+0000 7f946a800640 1 --2- 172.21.15.69:0/3995947883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9464108ec0 0x7f94641a4d20 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.749+0000 7f946a800640 1 --2- 172.21.15.69:0/3995947883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9464108540 0x7f94641a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.749+0000 7f946a800640 1 -- 172.21.15.69:0/3995947883 >> 172.21.15.69:0/3995947883 conn(0x7f9464077de0 msgr2=0x7f94640783a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.749+0000 7f946a800640 1 -- 172.21.15.69:0/3995947883 shutdown_connections 2024-04-24T09:12:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:20.749+0000 7f946a800640 1 -- 172.21.15.69:0/3995947883 wait complete. 2024-04-24T09:12:21.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:21 smithi069 ceph-mon[28170]: pgmap v161: 129 pgs: 129 active+clean; 94 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 658 KiB/s wr, 154 op/s 2024-04-24T09:12:21.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:21 smithi130 ceph-mon[31553]: pgmap v161: 129 pgs: 129 active+clean; 94 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 658 KiB/s wr, 154 op/s 2024-04-24T09:12:21.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:21 smithi081 ceph-mon[30961]: pgmap v161: 129 pgs: 129 active+clean; 94 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 658 KiB/s wr, 154 op/s 2024-04-24T09:12:22.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:22 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:22.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:22 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:22.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:22 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:23 smithi069 ceph-mon[28170]: pgmap v162: 129 pgs: 129 active+clean; 94 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 379 KiB/s wr, 90 op/s 2024-04-24T09:12:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:23 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:23 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:23.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:23 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:12:23.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:23 smithi130 ceph-mon[31553]: pgmap v162: 129 pgs: 129 active+clean; 94 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 379 KiB/s wr, 90 op/s 2024-04-24T09:12:23.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:23 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:23.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:23 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:23.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:23 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:12:23.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:23 smithi081 ceph-mon[30961]: pgmap v162: 129 pgs: 129 active+clean; 94 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 379 KiB/s wr, 90 op/s 2024-04-24T09:12:23.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:23 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:23.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:23 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:23.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:23 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:12:25.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:25 smithi130 ceph-mon[31553]: pgmap v163: 129 pgs: 129 active+clean; 94 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 379 KiB/s wr, 90 op/s 2024-04-24T09:12:25.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:25 smithi081 ceph-mon[30961]: pgmap v163: 129 pgs: 129 active+clean; 94 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 379 KiB/s wr, 90 op/s 2024-04-24T09:12:25.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:25 smithi069 ceph-mon[28170]: pgmap v163: 129 pgs: 129 active+clean; 94 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 379 KiB/s wr, 90 op/s 2024-04-24T09:12:27.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:27 smithi130 ceph-mon[31553]: pgmap v164: 129 pgs: 129 active+clean; 97 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 669 KiB/s wr, 159 op/s 2024-04-24T09:12:27.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:27 smithi081 ceph-mon[30961]: pgmap v164: 129 pgs: 129 active+clean; 97 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 669 KiB/s wr, 159 op/s 2024-04-24T09:12:27.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:27 smithi069 ceph-mon[28170]: pgmap v164: 129 pgs: 129 active+clean; 97 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 669 KiB/s wr, 159 op/s 2024-04-24T09:12:29.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:29 smithi130 ceph-mon[31553]: pgmap v165: 129 pgs: 129 active+clean; 97 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 503 KiB/s wr, 111 op/s 2024-04-24T09:12:29.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:29 smithi081 ceph-mon[30961]: pgmap v165: 129 pgs: 129 active+clean; 97 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 503 KiB/s wr, 111 op/s 2024-04-24T09:12:29.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:29 smithi069 ceph-mon[28170]: pgmap v165: 129 pgs: 129 active+clean; 97 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 503 KiB/s wr, 111 op/s 2024-04-24T09:12:31.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:31 smithi130 ceph-mon[31553]: pgmap v166: 129 pgs: 129 active+clean; 100 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 733 KiB/s wr, 153 op/s 2024-04-24T09:12:31.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:31 smithi081 ceph-mon[30961]: pgmap v166: 129 pgs: 129 active+clean; 100 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 733 KiB/s wr, 153 op/s 2024-04-24T09:12:31.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:31 smithi069 ceph-mon[28170]: pgmap v166: 129 pgs: 129 active+clean; 100 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 733 KiB/s wr, 153 op/s 2024-04-24T09:12:33.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:33 smithi130 ceph-mon[31553]: pgmap v167: 129 pgs: 129 active+clean; 100 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 520 KiB/s wr, 111 op/s 2024-04-24T09:12:33.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:33 smithi081 ceph-mon[30961]: pgmap v167: 129 pgs: 129 active+clean; 100 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 520 KiB/s wr, 111 op/s 2024-04-24T09:12:33.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:33 smithi069 ceph-mon[28170]: pgmap v167: 129 pgs: 129 active+clean; 100 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 520 KiB/s wr, 111 op/s 2024-04-24T09:12:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.897+0000 7fe0dcc00640 1 -- 172.21.15.69:0/3339526479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d8103020 msgr2=0x7fe0d8103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.897+0000 7fe0dcc00640 1 --2- 172.21.15.69:0/3339526479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d8103020 0x7fe0d8103400 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009900 tx=0x7fe0c802a480 comp rx=0 tx=0).stop 2024-04-24T09:12:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.897+0000 7fe0dcc00640 1 -- 172.21.15.69:0/3339526479 shutdown_connections 2024-04-24T09:12:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.897+0000 7fe0dcc00640 1 --2- 172.21.15.69:0/3339526479 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0d810c3f0 0x7fe0d810e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.897+0000 7fe0dcc00640 1 --2- 172.21.15.69:0/3339526479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0d8103940 0x7fe0d810be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.897+0000 7fe0dcc00640 1 --2- 172.21.15.69:0/3339526479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d8103020 0x7fe0d8103400 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.897+0000 7fe0dcc00640 1 -- 172.21.15.69:0/3339526479 >> 172.21.15.69:0/3339526479 conn(0x7fe0d80fc820 msgr2=0x7fe0d80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.897+0000 7fe0dcc00640 1 -- 172.21.15.69:0/3339526479 shutdown_connections 2024-04-24T09:12:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.897+0000 7fe0dcc00640 1 -- 172.21.15.69:0/3339526479 wait complete. 2024-04-24T09:12:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.898+0000 7fe0dcc00640 1 Processor -- start 2024-04-24T09:12:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.898+0000 7fe0dcc00640 1 -- start start 2024-04-24T09:12:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0dcc00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0d8103020 0x7fe0d81a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0dcc00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0d8103940 0x7fe0d81a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0dcc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d810c3f0 0x7fe0d819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0dcc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d81120a0 con 0x7fe0d810c3f0 2024-04-24T09:12:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0dcc00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d8111f20 con 0x7fe0d8103940 2024-04-24T09:12:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0dcc00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d8112220 con 0x7fe0d8103020 2024-04-24T09:12:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0d7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d810c3f0 0x7fe0d819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0d7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d810c3f0 0x7fe0d819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45554/0 (socket says 172.21.15.69:45554) 2024-04-24T09:12:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0d7e00640 1 -- 172.21.15.69:0/115866870 learned_addr learned my addr 172.21.15.69:0/115866870 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0d7400640 1 --2- 172.21.15.69:0/115866870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0d8103020 0x7fe0d81a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0d6a00640 1 --2- 172.21.15.69:0/115866870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0d8103940 0x7fe0d81a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0d7e00640 1 -- 172.21.15.69:0/115866870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0d8103020 msgr2=0x7fe0d81a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0d7e00640 1 --2- 172.21.15.69:0/115866870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0d8103020 0x7fe0d81a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0d7e00640 1 -- 172.21.15.69:0/115866870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0d8103940 msgr2=0x7fe0d81a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0d7e00640 1 --2- 172.21.15.69:0/115866870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0d8103940 0x7fe0d81a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0d7e00640 1 -- 172.21.15.69:0/115866870 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d819ad40 con 0x7fe0d810c3f0 2024-04-24T09:12:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.899+0000 7fe0d6a00640 1 --2- 172.21.15.69:0/115866870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0d8103940 0x7fe0d81a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.900+0000 7fe0d7e00640 1 --2- 172.21.15.69:0/115866870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d810c3f0 0x7fe0d819a5c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe0d000cce0 tx=0x7fe0d000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.900+0000 7fe0cfe00640 1 -- 172.21.15.69:0/115866870 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d000f450 con 0x7fe0d810c3f0 2024-04-24T09:12:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.900+0000 7fe0cfe00640 1 -- 172.21.15.69:0/115866870 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0d000faa0 con 0x7fe0d810c3f0 2024-04-24T09:12:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.900+0000 7fe0dcc00640 1 -- 172.21.15.69:0/115866870 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d819b030 con 0x7fe0d810c3f0 2024-04-24T09:12:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.900+0000 7fe0cfe00640 1 -- 172.21.15.69:0/115866870 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0018b70 con 0x7fe0d810c3f0 2024-04-24T09:12:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.900+0000 7fe0dcc00640 1 -- 172.21.15.69:0/115866870 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d81a72d0 con 0x7fe0d810c3f0 2024-04-24T09:12:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.904+0000 7fe0e0f88740 1 Processor -- start 2024-04-24T09:12:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.904+0000 7fe0cfe00640 1 -- 172.21.15.69:0/115866870 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe0d0018d10 con 0x7fe0d810c3f0 2024-04-24T09:12:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.904+0000 7fe0e0f88740 1 -- start start 2024-04-24T09:12:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.905+0000 7fe0e0f88740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe53c0 0x55ef70fe57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.905+0000 7fe0e0f88740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe5ce0 0x55ef70ff0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.905+0000 7fe0e0f88740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70ff05a0 0x55ef70ff2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.905+0000 7fe0e0f88740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ef70f4b900 con 0x55ef70fe5ce0 2024-04-24T09:12:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.905+0000 7fe0e0f88740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ef70ef1f20 con 0x55ef70fe53c0 2024-04-24T09:12:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.905+0000 7fe0e0f88740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ef70f478c0 con 0x55ef70ff05a0 2024-04-24T09:12:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.905+0000 7fe0d7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe5ce0 0x55ef70ff0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.905+0000 7fe0d7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe5ce0 0x55ef70ff0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45556/0 (socket says 172.21.15.69:45556) 2024-04-24T09:12:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.905+0000 7fe0d7400640 1 -- 172.21.15.69:0/2663512127 learned_addr learned my addr 172.21.15.69:0/2663512127 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.905+0000 7fe0d6a00640 1 --2- 172.21.15.69:0/2663512127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe53c0 0x55ef70fe57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.905+0000 7fe0d6a00640 1 --2- 172.21.15.69:0/2663512127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70ff05a0 0x55ef70ff2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.905+0000 7fe0d7400640 1 -- 172.21.15.69:0/2663512127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70ff05a0 msgr2=0x55ef70ff2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.905+0000 7fe0d7400640 1 --2- 172.21.15.69:0/2663512127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70ff05a0 0x55ef70ff2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.906+0000 7fe0d7400640 1 -- 172.21.15.69:0/2663512127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe53c0 msgr2=0x55ef70fe57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.906+0000 7fe0d7400640 1 --2- 172.21.15.69:0/2663512127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe53c0 0x55ef70fe57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.906+0000 7fe0d7400640 1 -- 172.21.15.69:0/2663512127 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef70f16e10 con 0x55ef70fe5ce0 2024-04-24T09:12:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.906+0000 7fe0d6a00640 1 --2- 172.21.15.69:0/2663512127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70ff05a0 0x55ef70ff2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.906+0000 7fe0cfe00640 1 --2- 172.21.15.69:0/115866870 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe0bc0a3d10 0x7fe0bc0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.906+0000 7fe0d7400640 1 --2- 172.21.15.69:0/2663512127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe5ce0 0x55ef70ff0060 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8031440 tx=0x7fe0c8004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663d7adb03a5456c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.906+0000 7fe0cfe00640 1 -- 172.21.15.69:0/115866870 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe0d00d0fe0 con 0x7fe0d810c3f0 2024-04-24T09:12:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.906+0000 7fe0d6a00640 1 --2- 172.21.15.69:0/115866870 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe0bc0a3d10 0x7fe0bc0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.906+0000 7fe0c3e00640 1 -- 172.21.15.69:0/2663512127 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c803f070 con 0x55ef70fe5ce0 2024-04-24T09:12:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.906+0000 7fe0c3e00640 1 -- 172.21.15.69:0/2663512127 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0c80047a0 con 0x55ef70fe5ce0 2024-04-24T09:12:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.906+0000 7fe0c3e00640 1 -- 172.21.15.69:0/2663512127 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8035070 con 0x55ef70fe5ce0 2024-04-24T09:12:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.907+0000 7fe0e0f88740 1 -- 172.21.15.69:0/2663512127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe5ce0 msgr2=0x55ef70ff0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.907+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/2663512127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe5ce0 0x55ef70ff0060 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8031440 tx=0x7fe0c8004390 comp rx=0 tx=0).stop 2024-04-24T09:12:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.907+0000 7fe0d6a00640 1 --2- 172.21.15.69:0/115866870 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe0bc0a3d10 0x7fe0bc0a61d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4005fd0 tx=0x7fe0c4005c80 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.907+0000 7fe0e0f88740 1 -- 172.21.15.69:0/2663512127 shutdown_connections 2024-04-24T09:12:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.907+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/2663512127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70ff05a0 0x55ef70ff2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.907+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/2663512127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe5ce0 0x55ef70ff0060 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.907+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/2663512127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe53c0 0x55ef70fe57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.907+0000 7fe0e0f88740 1 -- 172.21.15.69:0/2663512127 >> 172.21.15.69:0/2663512127 conn(0x55ef70fe18d0 msgr2=0x55ef70fe1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.907+0000 7fe0e0f88740 1 -- 172.21.15.69:0/2663512127 shutdown_connections 2024-04-24T09:12:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.907+0000 7fe0e0f88740 1 -- 172.21.15.69:0/2663512127 wait complete. 2024-04-24T09:12:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.908+0000 7fe0e0f88740 1 -- start start 2024-04-24T09:12:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.908+0000 7fe0e0f88740 1 Processor -- start 2024-04-24T09:12:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.908+0000 7fe0e0f88740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe53c0 0x55ef70fed650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.908+0000 7fe0e0f88740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe5ce0 0x55ef70fedb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.908+0000 7fe0e0f88740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70ff05a0 0x55ef70fee0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.909+0000 7fe0e0f88740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ef70ff30e0 con 0x55ef70fe53c0 2024-04-24T09:12:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.909+0000 7fe0e0f88740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ef70f052c0 con 0x55ef70fe5ce0 2024-04-24T09:12:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.909+0000 7fe0e0f88740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ef70ff3260 con 0x55ef70ff05a0 2024-04-24T09:12:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.909+0000 7fe0e0f88740 10 client.0 _ll_register_callbacks cb 0x55ef70c5da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.909+0000 7fe0d7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe53c0 0x55ef70fed650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.909+0000 7fe0d7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe53c0 0x55ef70fed650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45572/0 (socket says 172.21.15.69:45572) 2024-04-24T09:12:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.909+0000 7fe0d7400640 1 -- 172.21.15.69:0/770047541 learned_addr learned my addr 172.21.15.69:0/770047541 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.909+0000 7fe0d6a00640 1 --2- 172.21.15.69:0/770047541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe5ce0 0x55ef70fedb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.909+0000 7fe0d7e00640 1 --2- 172.21.15.69:0/770047541 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70ff05a0 0x55ef70fee0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.909+0000 7fe0d7400640 1 -- 172.21.15.69:0/770047541 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70ff05a0 msgr2=0x55ef70fee0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.909+0000 7fe0d7400640 1 --2- 172.21.15.69:0/770047541 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70ff05a0 0x55ef70fee0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.909+0000 7fe0d7400640 1 -- 172.21.15.69:0/770047541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe5ce0 msgr2=0x55ef70fedb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.909+0000 7fe0d7400640 1 --2- 172.21.15.69:0/770047541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe5ce0 0x55ef70fedb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.909+0000 7fe0d7400640 1 -- 172.21.15.69:0/770047541 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef70f15fd0 con 0x55ef70fe53c0 2024-04-24T09:12:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.910+0000 7fe0d7400640 1 --2- 172.21.15.69:0/770047541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe53c0 0x55ef70fed650 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe0c802a9a0 tx=0x7fe0c80052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3c8887c360921f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.910+0000 7fe0c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:12:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.910+0000 7fe0c0c00640 1 -- 172.21.15.69:0/770047541 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8035070 con 0x55ef70fe53c0 2024-04-24T09:12:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.910+0000 7fe0e0f88740 1 -- 172.21.15.69:0/770047541 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0c80047a0 con 0x55ef70fe53c0 2024-04-24T09:12:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.910+0000 7fe0c0c00640 1 -- 172.21.15.69:0/770047541 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe0c80316a0 con 0x55ef70fe53c0 2024-04-24T09:12:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.910+0000 7fe0c0c00640 1 -- 172.21.15.69:0/770047541 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe0c8057050 con 0x55ef70fe53c0 2024-04-24T09:12:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.910+0000 7fe0e0f88740 10 client.16776 fetch_fsmap learned FSMap version 30 2024-04-24T09:12:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.910+0000 7fe0e0f88740 1 -- 172.21.15.69:0/770047541 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ef70c5e210 con 0x55ef70fe53c0 2024-04-24T09:12:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.911+0000 7fe0c0c00640 1 -- 172.21.15.69:0/770047541 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0c80047a0 con 0x55ef70fe53c0 2024-04-24T09:12:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.911+0000 7fe0c0c00640 1 -- 172.21.15.69:0/770047541 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe0c804c720 con 0x55ef70fe53c0 2024-04-24T09:12:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.911+0000 7fe0e0f88740 10 client.16776 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:12:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.911+0000 7fe0e0f88740 10 client.16776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.911+0000 7fe0e0f88740 20 client.16776 populate_metadata read hostname 'smithi069' 2024-04-24T09:12:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.911+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/770047541 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55ef71000540 0x55ef71002930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.911+0000 7fe0e0f88740 4 client.16776 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.911+0000 7fe0e0f88740 1 -- 172.21.15.69:0/770047541 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef70f59320 con 0x55ef71000540 2024-04-24T09:12:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.911+0000 7fe0d6a00640 1 --2- 172.21.15.69:0/770047541 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55ef71000540 0x55ef71002930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.913+0000 7fe0d6a00640 1 --2- 172.21.15.69:0/770047541 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55ef71000540 0x55ef71002930 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6523aaf87d0318f server_cookie=bb413307fdf8a594 in_seq=0 out_seq=0 2024-04-24T09:12:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.913+0000 7fe0c0c00640 10 client.16776 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:12:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.915+0000 7fe0c0c00640 1 -- 172.21.15.69:0/770047541 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef70f59320 con 0x55ef71000540 2024-04-24T09:12:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.915+0000 7fe0c0c00640 10 client.16776 handle_command_reply: tid=0 2024-04-24T09:12:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.915+0000 7fe0e0f88740 1 client.16776 shutdown 2024-04-24T09:12:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.915+0000 7fe0e0f88740 1 -- 172.21.15.69:0/770047541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe53c0 msgr2=0x55ef70fed650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.915+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/770047541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe53c0 0x55ef70fed650 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe0c802a9a0 tx=0x7fe0c80052d0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.915+0000 7fe0e0f88740 1 -- 172.21.15.69:0/770047541 shutdown_connections 2024-04-24T09:12:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.915+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/770047541 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55ef71000540 0x55ef71002930 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.915+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/770047541 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70ff05a0 0x55ef70fee0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.915+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/770047541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe5ce0 0x55ef70fedb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.915+0000 7fe0c0c00640 0 client.16776 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:12:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.915+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/770047541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe53c0 0x55ef70fed650 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.915+0000 7fe0e0f88740 1 -- 172.21.15.69:0/770047541 >> 172.21.15.69:0/770047541 conn(0x55ef70fe18d0 msgr2=0x55ef70fe7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.916+0000 7fe0e0f88740 1 -- 172.21.15.69:0/770047541 shutdown_connections 2024-04-24T09:12:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.916+0000 7fe0e0f88740 1 -- 172.21.15.69:0/770047541 wait complete. 2024-04-24T09:12:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.916+0000 7fe0e0f88740 20 client.16776 trim_cache size 0 max 16384 2024-04-24T09:12:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.932+0000 7fe0e0f88740 1 Processor -- start 2024-04-24T09:12:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.932+0000 7fe0e0f88740 1 -- start start 2024-04-24T09:12:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.932+0000 7fe0e0f88740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70fe53c0 0x55ef70fefd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.932+0000 7fe0e0f88740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe5ce0 0x55ef71007fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.932+0000 7fe0e0f88740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70ff05a0 0x55ef71008500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.932+0000 7fe0e0f88740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d81b3a70 con 0x55ef70ff05a0 2024-04-24T09:12:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.932+0000 7fe0e0f88740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d81b12e0 con 0x55ef70fe5ce0 2024-04-24T09:12:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.932+0000 7fe0e0f88740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d81af130 con 0x55ef70fe53c0 2024-04-24T09:12:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.933+0000 7fe0d7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70ff05a0 0x55ef71008500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.933+0000 7fe0d7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70ff05a0 0x55ef71008500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45588/0 (socket says 172.21.15.69:45588) 2024-04-24T09:12:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.933+0000 7fe0d7e00640 1 -- 172.21.15.69:0/3842205008 learned_addr learned my addr 172.21.15.69:0/3842205008 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.933+0000 7fe0d6a00640 1 --2- 172.21.15.69:0/3842205008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe5ce0 0x55ef71007fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.933+0000 7fe0d7400640 1 --2- 172.21.15.69:0/3842205008 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70fe53c0 0x55ef70fefd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.933+0000 7fe0d7e00640 1 -- 172.21.15.69:0/3842205008 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70fe53c0 msgr2=0x55ef70fefd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.933+0000 7fe0d7e00640 1 --2- 172.21.15.69:0/3842205008 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70fe53c0 0x55ef70fefd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.933+0000 7fe0d7e00640 1 -- 172.21.15.69:0/3842205008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe5ce0 msgr2=0x55ef71007fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.933+0000 7fe0d7e00640 1 --2- 172.21.15.69:0/3842205008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe5ce0 0x55ef71007fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.933+0000 7fe0d7e00640 1 -- 172.21.15.69:0/3842205008 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef70ff2d00 con 0x55ef70ff05a0 2024-04-24T09:12:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.933+0000 7fe0d7400640 1 --2- 172.21.15.69:0/3842205008 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70fe53c0 0x55ef70fefd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.933+0000 7fe0d7e00640 1 --2- 172.21.15.69:0/3842205008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70ff05a0 0x55ef71008500 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fe0d009c900 tx=0x7fe0d009c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfdb200c96fdc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.933+0000 7fe0c3400640 1 -- 172.21.15.69:0/3842205008 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d00b5020 con 0x55ef70ff05a0 2024-04-24T09:12:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.934+0000 7fe0c3400640 1 -- 172.21.15.69:0/3842205008 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0d00a5870 con 0x55ef70ff05a0 2024-04-24T09:12:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.934+0000 7fe0c3400640 1 -- 172.21.15.69:0/3842205008 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d009d0c0 con 0x55ef70ff05a0 2024-04-24T09:12:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.934+0000 7fe0e0f88740 1 -- 172.21.15.69:0/3842205008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70ff05a0 msgr2=0x55ef71008500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.934+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/3842205008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70ff05a0 0x55ef71008500 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fe0d009c900 tx=0x7fe0d009c870 comp rx=0 tx=0).stop 2024-04-24T09:12:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.934+0000 7fe0e0f88740 1 -- 172.21.15.69:0/3842205008 shutdown_connections 2024-04-24T09:12:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.934+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/3842205008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70ff05a0 0x55ef71008500 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.934+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/3842205008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe5ce0 0x55ef71007fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.934+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/3842205008 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70fe53c0 0x55ef70fefd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.934+0000 7fe0e0f88740 1 -- 172.21.15.69:0/3842205008 >> 172.21.15.69:0/3842205008 conn(0x55ef70fe18d0 msgr2=0x55ef70fe89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.934+0000 7fe0e0f88740 1 -- 172.21.15.69:0/3842205008 shutdown_connections 2024-04-24T09:12:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.934+0000 7fe0e0f88740 1 -- 172.21.15.69:0/3842205008 wait complete. 2024-04-24T09:12:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.935+0000 7fe0e0f88740 1 -- start start 2024-04-24T09:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.935+0000 7fe0e0f88740 1 Processor -- start 2024-04-24T09:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.935+0000 7fe0e0f88740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe53c0 0x55ef70ff89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.935+0000 7fe0e0f88740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe5ce0 0x55ef70fea0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.935+0000 7fe0e0f88740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70ff05a0 0x55ef70fea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.935+0000 7fe0e0f88740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d81af7a0 con 0x55ef70fe5ce0 2024-04-24T09:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.935+0000 7fe0e0f88740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d81b0c10 con 0x55ef70fe53c0 2024-04-24T09:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.935+0000 7fe0e0f88740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d81afe70 con 0x55ef70ff05a0 2024-04-24T09:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.935+0000 7fe0e0f88740 10 client.0 _ll_register_callbacks cb 0x7fe0c8004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.936+0000 7fe0d6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe5ce0 0x55ef70fea0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.936+0000 7fe0d6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe5ce0 0x55ef70fea0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45596/0 (socket says 172.21.15.69:45596) 2024-04-24T09:12:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.936+0000 7fe0d6a00640 1 -- 172.21.15.69:0/1497980822 learned_addr learned my addr 172.21.15.69:0/1497980822 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.936+0000 7fe0d7400640 1 --2- 172.21.15.69:0/1497980822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe53c0 0x55ef70ff89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.936+0000 7fe0d7e00640 1 --2- 172.21.15.69:0/1497980822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70ff05a0 0x55ef70fea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.936+0000 7fe0d6a00640 1 -- 172.21.15.69:0/1497980822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70ff05a0 msgr2=0x55ef70fea650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.936+0000 7fe0d6a00640 1 --2- 172.21.15.69:0/1497980822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70ff05a0 0x55ef70fea650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.936+0000 7fe0d6a00640 1 -- 172.21.15.69:0/1497980822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe53c0 msgr2=0x55ef70ff89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.936+0000 7fe0d6a00640 1 --2- 172.21.15.69:0/1497980822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe53c0 0x55ef70ff89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.936+0000 7fe0d6a00640 1 -- 172.21.15.69:0/1497980822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef70d2a620 con 0x55ef70fe5ce0 2024-04-24T09:12:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.937+0000 7fe0d6a00640 1 --2- 172.21.15.69:0/1497980822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe5ce0 0x55ef70fea0f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe0d819b700 tx=0x7fe0c4005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819bd6dd3601835f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.937+0000 7fe0c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:12:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.937+0000 7fe0c0c00640 1 -- 172.21.15.69:0/1497980822 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4027500 con 0x55ef70fe5ce0 2024-04-24T09:12:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.937+0000 7fe0c0c00640 1 -- 172.21.15.69:0/1497980822 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe0c40276a0 con 0x55ef70fe5ce0 2024-04-24T09:12:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.937+0000 7fe0e0f88740 1 -- 172.21.15.69:0/1497980822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0d00a5870 con 0x55ef70fe5ce0 2024-04-24T09:12:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.937+0000 7fe0c0c00640 1 -- 172.21.15.69:0/1497980822 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe0c4038050 con 0x55ef70fe5ce0 2024-04-24T09:12:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.937+0000 7fe0e0f88740 10 client.16782 fetch_fsmap learned FSMap version 30 2024-04-24T09:12:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.937+0000 7fe0e0f88740 1 -- 172.21.15.69:0/1497980822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ef710666c0 con 0x55ef70fe5ce0 2024-04-24T09:12:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.937+0000 7fe0c0c00640 1 -- 172.21.15.69:0/1497980822 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0d00a5870 con 0x55ef70fe5ce0 2024-04-24T09:12:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.938+0000 7fe0c0c00640 1 -- 172.21.15.69:0/1497980822 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe0c402f410 con 0x55ef70fe5ce0 2024-04-24T09:12:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.938+0000 7fe0e0f88740 10 client.16782 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:12:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.938+0000 7fe0e0f88740 10 client.16782 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.938+0000 7fe0e0f88740 20 client.16782 populate_metadata read hostname 'smithi069' 2024-04-24T09:12:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.938+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/1497980822 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55ef71066e50 0x55ef71069270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.938+0000 7fe0e0f88740 4 client.16782 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:12:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.938+0000 7fe0e0f88740 1 -- 172.21.15.69:0/1497980822 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef70f53190 con 0x55ef71066e50 2024-04-24T09:12:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.938+0000 7fe0d7400640 1 --2- 172.21.15.69:0/1497980822 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55ef71066e50 0x55ef71069270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.939+0000 7fe0d7400640 1 --2- 172.21.15.69:0/1497980822 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55ef71066e50 0x55ef71069270 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da59df761b57ac38 server_cookie=53a1ef878a62f00d in_seq=0 out_seq=0 2024-04-24T09:12:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:33.939+0000 7fe0c0c00640 10 client.16782 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:12:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.029+0000 7fe0c0c00640 1 -- 172.21.15.69:0/1497980822 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef70f53190 con 0x55ef71066e50 2024-04-24T09:12:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.029+0000 7fe0c0c00640 10 client.16782 handle_command_reply: tid=0 2024-04-24T09:12:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.029+0000 7fe0e0f88740 1 client.16782 shutdown 2024-04-24T09:12:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.029+0000 7fe0e0f88740 1 -- 172.21.15.69:0/1497980822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe5ce0 msgr2=0x55ef70fea0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.029+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/1497980822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe5ce0 0x55ef70fea0f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe0d819b700 tx=0x7fe0c4005b70 comp rx=0 tx=0).stop 2024-04-24T09:12:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.029+0000 7fe0e0f88740 1 -- 172.21.15.69:0/1497980822 shutdown_connections 2024-04-24T09:12:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.029+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/1497980822 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55ef71066e50 0x55ef71069270 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.029+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/1497980822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ef70ff05a0 0x55ef70fea650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.029+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/1497980822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef70fe5ce0 0x55ef70fea0f0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.029+0000 7fe0c0c00640 0 client.16782 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:12:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.029+0000 7fe0e0f88740 1 --2- 172.21.15.69:0/1497980822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ef70fe53c0 0x55ef70ff89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.029+0000 7fe0e0f88740 1 -- 172.21.15.69:0/1497980822 >> 172.21.15.69:0/1497980822 conn(0x55ef70fe18d0 msgr2=0x55ef70fe0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.029+0000 7fe0e0f88740 1 -- 172.21.15.69:0/1497980822 shutdown_connections 2024-04-24T09:12:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.029+0000 7fe0e0f88740 1 -- 172.21.15.69:0/1497980822 wait complete. 2024-04-24T09:12:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.029+0000 7fe0e0f88740 20 client.16782 trim_cache size 0 max 16384 2024-04-24T09:12:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:12:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:12:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:12:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:12:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.030+0000 7fe0dcc00640 1 -- 172.21.15.69:0/115866870 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe0bc0a3d10 msgr2=0x7fe0bc0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.030+0000 7fe0dcc00640 1 --2- 172.21.15.69:0/115866870 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe0bc0a3d10 0x7fe0bc0a61d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4005fd0 tx=0x7fe0c4005c80 comp rx=0 tx=0).stop 2024-04-24T09:12:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.030+0000 7fe0dcc00640 1 -- 172.21.15.69:0/115866870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d810c3f0 msgr2=0x7fe0d819a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.030+0000 7fe0dcc00640 1 --2- 172.21.15.69:0/115866870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d810c3f0 0x7fe0d819a5c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe0d000cce0 tx=0x7fe0d000a620 comp rx=0 tx=0).stop 2024-04-24T09:12:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.030+0000 7fe0dcc00640 1 -- 172.21.15.69:0/115866870 shutdown_connections 2024-04-24T09:12:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.030+0000 7fe0dcc00640 1 --2- 172.21.15.69:0/115866870 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fe0bc0a3d10 0x7fe0bc0a61d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.030+0000 7fe0dcc00640 1 --2- 172.21.15.69:0/115866870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d810c3f0 0x7fe0d819a5c0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.030+0000 7fe0dcc00640 1 --2- 172.21.15.69:0/115866870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0d8103940 0x7fe0d81a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.030+0000 7fe0dcc00640 1 --2- 172.21.15.69:0/115866870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0d8103020 0x7fe0d81a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.030+0000 7fe0dcc00640 1 -- 172.21.15.69:0/115866870 >> 172.21.15.69:0/115866870 conn(0x7fe0d80fc820 msgr2=0x7fe0d8100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.030+0000 7fe0dcc00640 1 -- 172.21.15.69:0/115866870 shutdown_connections 2024-04-24T09:12:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:34.030+0000 7fe0dcc00640 1 -- 172.21.15.69:0/115866870 wait complete. 2024-04-24T09:12:35.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:35 smithi130 ceph-mon[31553]: pgmap v168: 129 pgs: 129 active+clean; 100 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 520 KiB/s wr, 111 op/s 2024-04-24T09:12:35.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:35 smithi081 ceph-mon[30961]: pgmap v168: 129 pgs: 129 active+clean; 100 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 520 KiB/s wr, 111 op/s 2024-04-24T09:12:35.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:35 smithi069 ceph-mon[28170]: pgmap v168: 129 pgs: 129 active+clean; 100 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 520 KiB/s wr, 111 op/s 2024-04-24T09:12:37.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:37 smithi130 ceph-mon[31553]: pgmap v169: 129 pgs: 129 active+clean; 102 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 752 KiB/s wr, 180 op/s 2024-04-24T09:12:37.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:37 smithi081 ceph-mon[30961]: pgmap v169: 129 pgs: 129 active+clean; 102 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 752 KiB/s wr, 180 op/s 2024-04-24T09:12:37.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:37 smithi069 ceph-mon[28170]: pgmap v169: 129 pgs: 129 active+clean; 102 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 752 KiB/s wr, 180 op/s 2024-04-24T09:12:39.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:39 smithi130 ceph-mon[31553]: pgmap v170: 129 pgs: 129 active+clean; 102 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 462 KiB/s wr, 111 op/s 2024-04-24T09:12:39.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:39 smithi081 ceph-mon[30961]: pgmap v170: 129 pgs: 129 active+clean; 102 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 462 KiB/s wr, 111 op/s 2024-04-24T09:12:39.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:39 smithi069 ceph-mon[28170]: pgmap v170: 129 pgs: 129 active+clean; 102 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 462 KiB/s wr, 111 op/s 2024-04-24T09:12:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:41 smithi130 ceph-mon[31553]: pgmap v171: 129 pgs: 129 active+clean; 105 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 686 KiB/s wr, 153 op/s 2024-04-24T09:12:41.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:41 smithi081 ceph-mon[30961]: pgmap v171: 129 pgs: 129 active+clean; 105 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 686 KiB/s wr, 153 op/s 2024-04-24T09:12:41.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:41 smithi069 ceph-mon[28170]: pgmap v171: 129 pgs: 129 active+clean; 105 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 686 KiB/s wr, 153 op/s 2024-04-24T09:12:43.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:43 smithi130 ceph-mon[31553]: pgmap v172: 129 pgs: 129 active+clean; 105 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 456 KiB/s wr, 111 op/s 2024-04-24T09:12:43.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:43 smithi081 ceph-mon[30961]: pgmap v172: 129 pgs: 129 active+clean; 105 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 456 KiB/s wr, 111 op/s 2024-04-24T09:12:43.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:43 smithi069 ceph-mon[28170]: pgmap v172: 129 pgs: 129 active+clean; 105 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 456 KiB/s wr, 111 op/s 2024-04-24T09:12:45.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:45 smithi130 ceph-mon[31553]: pgmap v173: 129 pgs: 129 active+clean; 105 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 456 KiB/s wr, 111 op/s 2024-04-24T09:12:45.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:45 smithi081 ceph-mon[30961]: pgmap v173: 129 pgs: 129 active+clean; 105 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 456 KiB/s wr, 111 op/s 2024-04-24T09:12:45.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:45 smithi069 ceph-mon[28170]: pgmap v173: 129 pgs: 129 active+clean; 105 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 456 KiB/s wr, 111 op/s 2024-04-24T09:12:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.173+0000 7f7a62600640 1 -- 172.21.15.69:0/4118607230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a5c102280 msgr2=0x7f7a5c102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.173+0000 7f7a62600640 1 --2- 172.21.15.69:0/4118607230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a5c102280 0x7f7a5c102660 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009900 tx=0x7f7a4c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.173+0000 7f7a62600640 1 -- 172.21.15.69:0/4118607230 shutdown_connections 2024-04-24T09:12:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.173+0000 7f7a62600640 1 --2- 172.21.15.69:0/4118607230 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a5c107350 0x7f7a5c10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.173+0000 7f7a62600640 1 --2- 172.21.15.69:0/4118607230 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a5c102c30 0x7f7a5c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.173+0000 7f7a62600640 1 --2- 172.21.15.69:0/4118607230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a5c102280 0x7f7a5c102660 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.173+0000 7f7a62600640 1 -- 172.21.15.69:0/4118607230 >> 172.21.15.69:0/4118607230 conn(0x7f7a5c079490 msgr2=0x7f7a5c0798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.173+0000 7f7a62600640 1 -- 172.21.15.69:0/4118607230 shutdown_connections 2024-04-24T09:12:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.173+0000 7f7a62600640 1 -- 172.21.15.69:0/4118607230 wait complete. 2024-04-24T09:12:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.174+0000 7f7a62600640 1 Processor -- start 2024-04-24T09:12:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.174+0000 7f7a62600640 1 -- start start 2024-04-24T09:12:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.175+0000 7f7a62600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a5c102280 0x7f7a5c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.175+0000 7f7a62600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a5c107350 0x7f7a5c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.175+0000 7f7a62600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a5c19d020 0x7f7a5c19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.175+0000 7f7a62600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1122c0 con 0x7f7a5c102280 2024-04-24T09:12:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.175+0000 7f7a62600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c112140 con 0x7f7a5c19d020 2024-04-24T09:12:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.175+0000 7f7a62600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c112440 con 0x7f7a5c107350 2024-04-24T09:12:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.175+0000 7f7a5b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a5c102280 0x7f7a5c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.175+0000 7f7a5b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a5c102280 0x7f7a5c198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42106/0 (socket says 172.21.15.69:42106) 2024-04-24T09:12:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.175+0000 7f7a5b400640 1 -- 172.21.15.69:0/2091865429 learned_addr learned my addr 172.21.15.69:0/2091865429 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.175+0000 7f7a5aa00640 1 --2- 172.21.15.69:0/2091865429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a5c107350 0x7f7a5c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.176+0000 7f7a5b400640 1 -- 172.21.15.69:0/2091865429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a5c107350 msgr2=0x7f7a5c198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.176+0000 7f7a5be00640 1 --2- 172.21.15.69:0/2091865429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a5c19d020 0x7f7a5c19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.176+0000 7f7a5b400640 1 --2- 172.21.15.69:0/2091865429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a5c107350 0x7f7a5c198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.176+0000 7f7a5b400640 1 -- 172.21.15.69:0/2091865429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a5c19d020 msgr2=0x7f7a5c19d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.176+0000 7f7a5b400640 1 --2- 172.21.15.69:0/2091865429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a5c19d020 0x7f7a5c19d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.176+0000 7f7a5b400640 1 -- 172.21.15.69:0/2091865429 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c19db10 con 0x7f7a5c102280 2024-04-24T09:12:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.176+0000 7f7a5aa00640 1 --2- 172.21.15.69:0/2091865429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a5c107350 0x7f7a5c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.176+0000 7f7a5be00640 1 --2- 172.21.15.69:0/2091865429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a5c19d020 0x7f7a5c19d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.176+0000 7f7a5b400640 1 --2- 172.21.15.69:0/2091865429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a5c102280 0x7f7a5c198760 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c005940 tx=0x7f7a4c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.176+0000 7f7a53e00640 1 -- 172.21.15.69:0/2091865429 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c03f030 con 0x7f7a5c102280 2024-04-24T09:12:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.176+0000 7f7a53e00640 1 -- 172.21.15.69:0/2091865429 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a4c003aa0 con 0x7f7a5c102280 2024-04-24T09:12:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.176+0000 7f7a62600640 1 -- 172.21.15.69:0/2091865429 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5c19dda0 con 0x7f7a5c102280 2024-04-24T09:12:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.176+0000 7f7a53e00640 1 -- 172.21.15.69:0/2091865429 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c035070 con 0x7f7a5c102280 2024-04-24T09:12:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.177+0000 7f7a62600640 1 -- 172.21.15.69:0/2091865429 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a5c1a5560 con 0x7f7a5c102280 2024-04-24T09:12:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.180+0000 7f7a53e00640 1 -- 172.21.15.69:0/2091865429 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7a4c042030 con 0x7f7a5c102280 2024-04-24T09:12:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.180+0000 7f7a653d9740 1 Processor -- start 2024-04-24T09:12:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.180+0000 7f7a653d9740 1 -- start start 2024-04-24T09:12:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.180+0000 7f7a653d9740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67feb45c0 0x55c67feb49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.180+0000 7f7a653d9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb4ee0 0x55c67febf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.180+0000 7f7a653d9740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67febf7a0 0x55c67fec1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.181+0000 7f7a653d9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c67fe16900 con 0x55c67feb4ee0 2024-04-24T09:12:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.181+0000 7f7a653d9740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55c67fdbcf20 con 0x55c67febf7a0 2024-04-24T09:12:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.181+0000 7f7a653d9740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c67fe128c0 con 0x55c67feb45c0 2024-04-24T09:12:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.181+0000 7f7a5b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb4ee0 0x55c67febf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.181+0000 7f7a5b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb4ee0 0x55c67febf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42112/0 (socket says 172.21.15.69:42112) 2024-04-24T09:12:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.181+0000 7f7a5b400640 1 -- 172.21.15.69:0/4170646690 learned_addr learned my addr 172.21.15.69:0/4170646690 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.181+0000 7f7a5be00640 1 --2- 172.21.15.69:0/4170646690 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67feb45c0 0x55c67feb49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.181+0000 7f7a5aa00640 1 --2- 172.21.15.69:0/4170646690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67febf7a0 0x55c67fec1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.182+0000 7f7a5b400640 1 -- 172.21.15.69:0/4170646690 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67feb45c0 msgr2=0x55c67feb49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.182+0000 7f7a5b400640 1 --2- 172.21.15.69:0/4170646690 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67feb45c0 0x55c67feb49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.182+0000 7f7a5b400640 1 -- 172.21.15.69:0/4170646690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67febf7a0 msgr2=0x55c67fec1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.182+0000 7f7a5b400640 1 --2- 172.21.15.69:0/4170646690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67febf7a0 0x55c67fec1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.182+0000 7f7a5b400640 1 -- 172.21.15.69:0/4170646690 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67fde1e10 con 0x55c67feb4ee0 2024-04-24T09:12:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.182+0000 7f7a53e00640 1 --2- 172.21.15.69:0/2091865429 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7a400a3dc0 0x7f7a400a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.182+0000 7f7a53e00640 1 -- 172.21.15.69:0/2091865429 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7a4c0f13f0 con 0x7f7a5c102280 2024-04-24T09:12:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.182+0000 7f7a5be00640 1 --2- 172.21.15.69:0/2091865429 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7a400a3dc0 0x7f7a400a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.182+0000 7f7a5b400640 1 --2- 172.21.15.69:0/4170646690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb4ee0 0x55c67febf260 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0ee710 tx=0x7f7a4c0f9620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d66f9acd18f931 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.182+0000 7f7a50c00640 1 -- 172.21.15.69:0/4170646690 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c049070 con 0x55c67feb4ee0 2024-04-24T09:12:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.182+0000 7f7a50c00640 1 -- 172.21.15.69:0/4170646690 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a4c03b460 con 0x55c67feb4ee0 2024-04-24T09:12:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.183+0000 7f7a50c00640 1 -- 172.21.15.69:0/4170646690 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0bc950 con 0x55c67feb4ee0 2024-04-24T09:12:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.183+0000 7f7a653d9740 1 -- 172.21.15.69:0/4170646690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb4ee0 msgr2=0x55c67febf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.183+0000 7f7a653d9740 1 --2- 172.21.15.69:0/4170646690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb4ee0 0x55c67febf260 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0ee710 tx=0x7f7a4c0f9620 comp rx=0 tx=0).stop 2024-04-24T09:12:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.183+0000 7f7a5be00640 1 --2- 172.21.15.69:0/2091865429 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7a400a3dc0 0x7f7a400a6280 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7a54005fd0 tx=0x7f7a54009700 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.183+0000 7f7a653d9740 1 -- 172.21.15.69:0/4170646690 shutdown_connections 2024-04-24T09:12:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.183+0000 7f7a653d9740 1 --2- 172.21.15.69:0/4170646690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67febf7a0 0x55c67fec1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.183+0000 7f7a653d9740 1 --2- 172.21.15.69:0/4170646690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb4ee0 0x55c67febf260 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.183+0000 7f7a653d9740 1 --2- 172.21.15.69:0/4170646690 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67feb45c0 0x55c67feb49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.183+0000 7f7a653d9740 1 -- 172.21.15.69:0/4170646690 >> 172.21.15.69:0/4170646690 conn(0x55c67feb0ad0 msgr2=0x55c67feb0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.183+0000 7f7a653d9740 1 -- 172.21.15.69:0/4170646690 shutdown_connections 2024-04-24T09:12:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.183+0000 7f7a653d9740 1 -- 172.21.15.69:0/4170646690 wait complete. 2024-04-24T09:12:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.184+0000 7f7a653d9740 1 -- start start 2024-04-24T09:12:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.184+0000 7f7a653d9740 1 Processor -- start 2024-04-24T09:12:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.184+0000 7f7a653d9740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67feb45c0 0x55c67febc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.184+0000 7f7a653d9740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67feb4ee0 0x55c67febcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.184+0000 7f7a653d9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67febf7a0 0x55c67febd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.184+0000 7f7a653d9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c67fec2190 con 0x55c67febf7a0 2024-04-24T09:12:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.185+0000 7f7a653d9740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55c67fdd02c0 con 0x55c67feb45c0 2024-04-24T09:12:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.185+0000 7f7a653d9740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c67fec2310 con 0x55c67feb4ee0 2024-04-24T09:12:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.185+0000 7f7a653d9740 10 client.0 _ll_register_callbacks cb 0x55c67fb28a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.185+0000 7f7a5be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67febf7a0 0x55c67febd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.185+0000 7f7a5be00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67febf7a0 0x55c67febd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42126/0 (socket says 172.21.15.69:42126) 2024-04-24T09:12:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.185+0000 7f7a5be00640 1 -- 172.21.15.69:0/1486666210 learned_addr learned my addr 172.21.15.69:0/1486666210 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.185+0000 7f7a5aa00640 1 --2- 172.21.15.69:0/1486666210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67feb4ee0 0x55c67febcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.185+0000 7f7a5b400640 1 --2- 172.21.15.69:0/1486666210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67feb45c0 0x55c67febc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.185+0000 7f7a5be00640 1 -- 172.21.15.69:0/1486666210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67feb4ee0 msgr2=0x55c67febcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.185+0000 7f7a5be00640 1 --2- 172.21.15.69:0/1486666210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67feb4ee0 0x55c67febcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.185+0000 7f7a5be00640 1 -- 172.21.15.69:0/1486666210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67feb45c0 msgr2=0x55c67febc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.185+0000 7f7a5be00640 1 --2- 172.21.15.69:0/1486666210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67feb45c0 0x55c67febc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.185+0000 7f7a5be00640 1 -- 172.21.15.69:0/1486666210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67fde0fd0 con 0x55c67febf7a0 2024-04-24T09:12:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.185+0000 7f7a5b400640 1 --2- 172.21.15.69:0/1486666210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67feb45c0 0x55c67febc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.186+0000 7f7a5be00640 1 --2- 172.21.15.69:0/1486666210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67febf7a0 0x55c67febd140 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009d50 tx=0x7f7a54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0f2573928d587b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.186+0000 7f7a45600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:12:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.186+0000 7f7a45600640 1 -- 172.21.15.69:0/1486666210 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5401db80 con 0x55c67febf7a0 2024-04-24T09:12:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.186+0000 7f7a45600640 1 -- 172.21.15.69:0/1486666210 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7a5401dd20 con 0x55c67febf7a0 2024-04-24T09:12:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.186+0000 7f7a653d9740 1 -- 172.21.15.69:0/1486666210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c67fe24320 con 0x55c67febf7a0 2024-04-24T09:12:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.186+0000 7f7a45600640 1 -- 172.21.15.69:0/1486666210 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7a5401b460 con 0x55c67febf7a0 2024-04-24T09:12:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.186+0000 7f7a653d9740 10 client.16794 fetch_fsmap learned FSMap version 30 2024-04-24T09:12:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.186+0000 7f7a653d9740 1 -- 172.21.15.69:0/1486666210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c67fb29210 con 0x55c67febf7a0 2024-04-24T09:12:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.187+0000 7f7a45600640 1 -- 172.21.15.69:0/1486666210 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55c67fe24320 con 0x55c67febf7a0 2024-04-24T09:12:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.187+0000 7f7a45600640 1 -- 172.21.15.69:0/1486666210 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7a54024070 con 0x55c67febf7a0 2024-04-24T09:12:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.187+0000 7f7a653d9740 10 client.16794 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:12:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.187+0000 7f7a653d9740 10 client.16794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.187+0000 7f7a653d9740 20 client.16794 populate_metadata read hostname 'smithi069' 2024-04-24T09:12:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.187+0000 7f7a653d9740 1 --2- 172.21.15.69:0/1486666210 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c67fecf7f0 0x55c67fed1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.187+0000 7f7a653d9740 4 client.16794 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.187+0000 7f7a653d9740 1 -- 172.21.15.69:0/1486666210 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c67fe1e190 con 0x55c67fecf7f0 2024-04-24T09:12:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.188+0000 7f7a5b400640 1 --2- 172.21.15.69:0/1486666210 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c67fecf7f0 0x55c67fed1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.188+0000 7f7a5b400640 1 --2- 172.21.15.69:0/1486666210 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c67fecf7f0 0x55c67fed1c60 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd3c661cbbb97cbf server_cookie=3bb063a783fc4571 in_seq=0 out_seq=0 2024-04-24T09:12:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.188+0000 7f7a45600640 10 client.16794 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:12:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.190+0000 7f7a45600640 1 -- 172.21.15.69:0/1486666210 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c67fe1e190 con 0x55c67fecf7f0 2024-04-24T09:12:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.190+0000 7f7a45600640 10 client.16794 handle_command_reply: tid=0 2024-04-24T09:12:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.190+0000 7f7a653d9740 1 client.16794 shutdown 2024-04-24T09:12:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.190+0000 7f7a653d9740 1 -- 172.21.15.69:0/1486666210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67febf7a0 msgr2=0x55c67febd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.190+0000 7f7a653d9740 1 --2- 172.21.15.69:0/1486666210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67febf7a0 0x55c67febd140 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009d50 tx=0x7f7a54004a40 comp rx=0 tx=0).stop 2024-04-24T09:12:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.191+0000 7f7a653d9740 1 -- 172.21.15.69:0/1486666210 shutdown_connections 2024-04-24T09:12:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.191+0000 7f7a653d9740 1 --2- 172.21.15.69:0/1486666210 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c67fecf7f0 0x55c67fed1c60 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.191+0000 7f7a653d9740 1 --2- 172.21.15.69:0/1486666210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67febf7a0 0x55c67febd140 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.191+0000 7f7a653d9740 1 --2- 172.21.15.69:0/1486666210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67feb4ee0 0x55c67febcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.191+0000 7f7a653d9740 1 --2- 172.21.15.69:0/1486666210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67feb45c0 0x55c67febc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.191+0000 7f7a45600640 0 client.16794 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:12:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.191+0000 7f7a653d9740 1 -- 172.21.15.69:0/1486666210 >> 172.21.15.69:0/1486666210 conn(0x55c67feb0ad0 msgr2=0x55c67feab840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.191+0000 7f7a653d9740 1 -- 172.21.15.69:0/1486666210 shutdown_connections 2024-04-24T09:12:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.191+0000 7f7a653d9740 1 -- 172.21.15.69:0/1486666210 wait complete. 2024-04-24T09:12:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.191+0000 7f7a653d9740 20 client.16794 trim_cache size 0 max 16384 2024-04-24T09:12:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.207+0000 7f7a653d9740 1 Processor -- start 2024-04-24T09:12:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.208+0000 7f7a653d9740 1 -- start start 2024-04-24T09:12:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.208+0000 7f7a653d9740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67feb45c0 0x55c67feaffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.208+0000 7f7a653d9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb4ee0 0x55c67feb6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.208+0000 7f7a653d9740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67febf7a0 0x55c67febcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.208+0000 7f7a653d9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1af8b0 con 0x55c67feb4ee0 2024-04-24T09:12:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.208+0000 7f7a653d9740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1ad120 con 0x55c67feb45c0 2024-04-24T09:12:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.208+0000 7f7a653d9740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1aaf70 con 0x55c67febf7a0 2024-04-24T09:12:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.208+0000 7f7a5aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb4ee0 0x55c67feb6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.208+0000 7f7a5aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb4ee0 0x55c67feb6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42134/0 (socket says 172.21.15.69:42134) 2024-04-24T09:12:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.208+0000 7f7a5aa00640 1 -- 172.21.15.69:0/2249415482 learned_addr learned my addr 172.21.15.69:0/2249415482 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.208+0000 7f7a5b400640 1 --2- 172.21.15.69:0/2249415482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67feb45c0 0x55c67feaffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.209+0000 7f7a5aa00640 1 -- 172.21.15.69:0/2249415482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67febf7a0 msgr2=0x55c67febcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.209+0000 7f7a5aa00640 1 --2- 172.21.15.69:0/2249415482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67febf7a0 0x55c67febcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.209+0000 7f7a5aa00640 1 -- 172.21.15.69:0/2249415482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67feb45c0 msgr2=0x55c67feaffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.209+0000 7f7a5aa00640 1 --2- 172.21.15.69:0/2249415482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67feb45c0 0x55c67feaffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.209+0000 7f7a5aa00640 1 -- 172.21.15.69:0/2249415482 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67fbf5620 con 0x55c67feb4ee0 2024-04-24T09:12:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.209+0000 7f7a5aa00640 1 --2- 172.21.15.69:0/2249415482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb4ee0 0x55c67feb6f80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c1018a0 tx=0x7f7a48013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7b17ec4ed508d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.209+0000 7f7a47e00640 1 -- 172.21.15.69:0/2249415482 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48004220 con 0x55c67feb4ee0 2024-04-24T09:12:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.209+0000 7f7a47e00640 1 -- 172.21.15.69:0/2249415482 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a480043c0 con 0x55c67feb4ee0 2024-04-24T09:12:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.209+0000 7f7a47e00640 1 -- 172.21.15.69:0/2249415482 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48011a70 con 0x55c67feb4ee0 2024-04-24T09:12:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.210+0000 7f7a653d9740 1 -- 172.21.15.69:0/2249415482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb4ee0 msgr2=0x55c67feb6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.210+0000 7f7a653d9740 1 --2- 172.21.15.69:0/2249415482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb4ee0 0x55c67feb6f80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c1018a0 tx=0x7f7a48013830 comp rx=0 tx=0).stop 2024-04-24T09:12:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.210+0000 7f7a653d9740 1 -- 172.21.15.69:0/2249415482 shutdown_connections 2024-04-24T09:12:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.210+0000 7f7a653d9740 1 --2- 172.21.15.69:0/2249415482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67febf7a0 0x55c67febcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.210+0000 7f7a653d9740 1 --2- 172.21.15.69:0/2249415482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb4ee0 0x55c67feb6f80 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.210+0000 7f7a653d9740 1 --2- 172.21.15.69:0/2249415482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67feb45c0 0x55c67feaffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.210+0000 7f7a653d9740 1 -- 172.21.15.69:0/2249415482 >> 172.21.15.69:0/2249415482 conn(0x55c67feb0ad0 msgr2=0x55c67febc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.210+0000 7f7a653d9740 1 -- 172.21.15.69:0/2249415482 shutdown_connections 2024-04-24T09:12:47.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.210+0000 7f7a653d9740 1 -- 172.21.15.69:0/2249415482 wait complete. 2024-04-24T09:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.211+0000 7f7a653d9740 1 -- start start 2024-04-24T09:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.211+0000 7f7a653d9740 1 Processor -- start 2024-04-24T09:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.211+0000 7f7a653d9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb45c0 0x55c67feaffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.211+0000 7f7a653d9740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67feb4ee0 0x55c67feac360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.211+0000 7f7a653d9740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67febf7a0 0x55c67feacaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.211+0000 7f7a653d9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1ab5e0 con 0x55c67feb45c0 2024-04-24T09:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.211+0000 7f7a653d9740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1aca50 con 0x55c67febf7a0 2024-04-24T09:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.211+0000 7f7a653d9740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1abcb0 con 0x55c67feb4ee0 2024-04-24T09:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.211+0000 7f7a653d9740 10 client.0 _ll_register_callbacks cb 0x7f7a54005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.212+0000 7f7a5b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb45c0 0x55c67feaffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.212+0000 7f7a5b400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb45c0 0x55c67feaffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42138/0 (socket says 172.21.15.69:42138) 2024-04-24T09:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.212+0000 7f7a5b400640 1 -- 172.21.15.69:0/1059727422 learned_addr learned my addr 172.21.15.69:0/1059727422 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.212+0000 7f7a5aa00640 1 --2- 172.21.15.69:0/1059727422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67feb4ee0 0x55c67feac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.212+0000 7f7a5be00640 1 --2- 172.21.15.69:0/1059727422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67febf7a0 0x55c67feacaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.212+0000 7f7a5b400640 1 -- 172.21.15.69:0/1059727422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67feb4ee0 msgr2=0x55c67feac360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.212+0000 7f7a5b400640 1 --2- 172.21.15.69:0/1059727422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67feb4ee0 0x55c67feac360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.212+0000 7f7a5b400640 1 -- 172.21.15.69:0/1059727422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67febf7a0 msgr2=0x55c67feacaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.212+0000 7f7a5b400640 1 --2- 172.21.15.69:0/1059727422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67febf7a0 0x55c67feacaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.212+0000 7f7a5b400640 1 -- 172.21.15.69:0/1059727422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67fec64c0 con 0x55c67feb45c0 2024-04-24T09:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.212+0000 7f7a5b400640 1 --2- 172.21.15.69:0/1059727422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb45c0 0x55c67feaffe0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c006010 tx=0x7f7a4c0bcb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8e614f3cf43ed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.212+0000 7f7a45600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.213+0000 7f7a45600640 1 -- 172.21.15.69:0/1059727422 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c049040 con 0x55c67feb45c0 2024-04-24T09:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.213+0000 7f7a45600640 1 -- 172.21.15.69:0/1059727422 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7a4c0f1120 con 0x55c67feb45c0 2024-04-24T09:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.213+0000 7f7a653d9740 1 -- 172.21.15.69:0/1059727422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a480043c0 con 0x55c67feb45c0 2024-04-24T09:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.213+0000 7f7a45600640 1 -- 172.21.15.69:0/1059727422 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7a4c0bc950 con 0x55c67feb45c0 2024-04-24T09:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.213+0000 7f7a653d9740 10 client.16800 fetch_fsmap learned FSMap version 30 2024-04-24T09:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.213+0000 7f7a653d9740 1 -- 172.21.15.69:0/1059727422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c67fec7830 con 0x55c67feb45c0 2024-04-24T09:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.213+0000 7f7a45600640 1 -- 172.21.15.69:0/1059727422 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a480043c0 con 0x55c67feb45c0 2024-04-24T09:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.213+0000 7f7a45600640 1 -- 172.21.15.69:0/1059727422 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7a4c0b9660 con 0x55c67feb45c0 2024-04-24T09:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.213+0000 7f7a653d9740 10 client.16800 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.213+0000 7f7a653d9740 10 client.16800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.213+0000 7f7a653d9740 20 client.16800 populate_metadata read hostname 'smithi069' 2024-04-24T09:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.213+0000 7f7a653d9740 1 --2- 172.21.15.69:0/1059727422 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c67ff3a5d0 0x55c67ff3c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.213+0000 7f7a653d9740 4 client.16800 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.213+0000 7f7a653d9740 1 -- 172.21.15.69:0/1059727422 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c67fd9a4b0 con 0x55c67ff3a5d0 2024-04-24T09:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.214+0000 7f7a5aa00640 1 --2- 172.21.15.69:0/1059727422 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c67ff3a5d0 0x55c67ff3c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.215+0000 7f7a5aa00640 1 --2- 172.21.15.69:0/1059727422 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c67ff3a5d0 0x55c67ff3c9c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fc56deb16525f5a server_cookie=3607215a4421ae61 in_seq=0 out_seq=0 2024-04-24T09:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.215+0000 7f7a45600640 10 client.16800 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:12:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.340+0000 7f7a45600640 1 -- 172.21.15.69:0/1059727422 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c67fd9a4b0 con 0x55c67ff3a5d0 2024-04-24T09:12:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.340+0000 7f7a45600640 10 client.16800 handle_command_reply: tid=0 2024-04-24T09:12:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.340+0000 7f7a653d9740 1 client.16800 shutdown 2024-04-24T09:12:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.341+0000 7f7a653d9740 1 -- 172.21.15.69:0/1059727422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb45c0 msgr2=0x55c67feaffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.341+0000 7f7a653d9740 1 --2- 172.21.15.69:0/1059727422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb45c0 0x55c67feaffe0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c006010 tx=0x7f7a4c0bcb30 comp rx=0 tx=0).stop 2024-04-24T09:12:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.341+0000 7f7a653d9740 1 -- 172.21.15.69:0/1059727422 shutdown_connections 2024-04-24T09:12:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.341+0000 7f7a653d9740 1 --2- 172.21.15.69:0/1059727422 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55c67ff3a5d0 0x55c67ff3c9c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.341+0000 7f7a653d9740 1 --2- 172.21.15.69:0/1059727422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c67febf7a0 0x55c67feacaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.341+0000 7f7a653d9740 1 --2- 172.21.15.69:0/1059727422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c67feb4ee0 0x55c67feac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.341+0000 7f7a653d9740 1 --2- 172.21.15.69:0/1059727422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c67feb45c0 0x55c67feaffe0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.341+0000 7f7a45600640 0 client.16800 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:12:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.341+0000 7f7a653d9740 1 -- 172.21.15.69:0/1059727422 >> 172.21.15.69:0/1059727422 conn(0x55c67feb0ad0 msgr2=0x55c67febcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.341+0000 7f7a653d9740 1 -- 172.21.15.69:0/1059727422 shutdown_connections 2024-04-24T09:12:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.341+0000 7f7a653d9740 1 -- 172.21.15.69:0/1059727422 wait complete. 2024-04-24T09:12:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.341+0000 7f7a653d9740 20 client.16800 trim_cache size 0 max 16384 2024-04-24T09:12:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:12:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:12:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:12:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:12:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.342+0000 7f7a62600640 1 -- 172.21.15.69:0/2091865429 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7a400a3dc0 msgr2=0x7f7a400a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.342+0000 7f7a62600640 1 --2- 172.21.15.69:0/2091865429 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7a400a3dc0 0x7f7a400a6280 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7a54005fd0 tx=0x7f7a54009700 comp rx=0 tx=0).stop 2024-04-24T09:12:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.342+0000 7f7a62600640 1 -- 172.21.15.69:0/2091865429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a5c102280 msgr2=0x7f7a5c198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.342+0000 7f7a62600640 1 --2- 172.21.15.69:0/2091865429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a5c102280 0x7f7a5c198760 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c005940 tx=0x7f7a4c004270 comp rx=0 tx=0).stop 2024-04-24T09:12:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.342+0000 7f7a62600640 1 -- 172.21.15.69:0/2091865429 shutdown_connections 2024-04-24T09:12:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.342+0000 7f7a62600640 1 --2- 172.21.15.69:0/2091865429 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f7a400a3dc0 0x7f7a400a6280 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.342+0000 7f7a62600640 1 --2- 172.21.15.69:0/2091865429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a5c19d020 0x7f7a5c19d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.342+0000 7f7a62600640 1 --2- 172.21.15.69:0/2091865429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7a5c107350 0x7f7a5c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.342+0000 7f7a62600640 1 --2- 172.21.15.69:0/2091865429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a5c102280 0x7f7a5c198760 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.342+0000 7f7a62600640 1 -- 172.21.15.69:0/2091865429 >> 172.21.15.69:0/2091865429 conn(0x7f7a5c079490 msgr2=0x7f7a5c0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.342+0000 7f7a62600640 1 -- 172.21.15.69:0/2091865429 shutdown_connections 2024-04-24T09:12:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:12:47.342+0000 7f7a62600640 1 -- 172.21.15.69:0/2091865429 wait complete. 2024-04-24T09:12:47.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:47 smithi130 ceph-mon[31553]: pgmap v174: 129 pgs: 129 active+clean; 108 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 695 KiB/s wr, 153 op/s 2024-04-24T09:12:47.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:47 smithi081 ceph-mon[30961]: pgmap v174: 129 pgs: 129 active+clean; 108 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 695 KiB/s wr, 153 op/s 2024-04-24T09:12:47.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:47 smithi069 ceph-mon[28170]: pgmap v174: 129 pgs: 129 active+clean; 108 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 695 KiB/s wr, 153 op/s 2024-04-24T09:12:49.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:49 smithi130 ceph-mon[31553]: pgmap v175: 129 pgs: 129 active+clean; 108 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 463 KiB/s wr, 84 op/s 2024-04-24T09:12:49.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:49 smithi081 ceph-mon[30961]: pgmap v175: 129 pgs: 129 active+clean; 108 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 463 KiB/s wr, 84 op/s 2024-04-24T09:12:49.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:49 smithi069 ceph-mon[28170]: pgmap v175: 129 pgs: 129 active+clean; 108 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 463 KiB/s wr, 84 op/s 2024-04-24T09:12:51.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:51 smithi130 ceph-mon[31553]: pgmap v176: 129 pgs: 129 active+clean; 110 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 720 KiB/s wr, 161 op/s 2024-04-24T09:12:51.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:51 smithi081 ceph-mon[30961]: pgmap v176: 129 pgs: 129 active+clean; 110 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 720 KiB/s wr, 161 op/s 2024-04-24T09:12:51.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:51 smithi069 ceph-mon[28170]: pgmap v176: 129 pgs: 129 active+clean; 110 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 720 KiB/s wr, 161 op/s 2024-04-24T09:12:53.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:53 smithi130 ceph-mon[31553]: pgmap v177: 129 pgs: 129 active+clean; 110 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 495 KiB/s wr, 119 op/s 2024-04-24T09:12:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:53 smithi081 ceph-mon[30961]: pgmap v177: 129 pgs: 129 active+clean; 110 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 495 KiB/s wr, 119 op/s 2024-04-24T09:12:53.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:53 smithi069 ceph-mon[28170]: pgmap v177: 129 pgs: 129 active+clean; 110 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 495 KiB/s wr, 119 op/s 2024-04-24T09:12:55.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:55 smithi130 ceph-mon[31553]: pgmap v178: 129 pgs: 129 active+clean; 110 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 496 KiB/s wr, 119 op/s 2024-04-24T09:12:55.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:55 smithi081 ceph-mon[30961]: pgmap v178: 129 pgs: 129 active+clean; 110 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 496 KiB/s wr, 119 op/s 2024-04-24T09:12:55.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:55 smithi069 ceph-mon[28170]: pgmap v178: 129 pgs: 129 active+clean; 110 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 496 KiB/s wr, 119 op/s 2024-04-24T09:12:57.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:57 smithi130 ceph-mon[31553]: pgmap v179: 129 pgs: 129 active+clean; 113 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 737 KiB/s wr, 162 op/s 2024-04-24T09:12:57.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:57 smithi081 ceph-mon[30961]: pgmap v179: 129 pgs: 129 active+clean; 113 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 737 KiB/s wr, 162 op/s 2024-04-24T09:12:57.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:57 smithi069 ceph-mon[28170]: pgmap v179: 129 pgs: 129 active+clean; 113 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 737 KiB/s wr, 162 op/s 2024-04-24T09:12:58.192 INFO:tasks.workunit.client.0.smithi069.stderr:Note: switching to '12973fc04fd10d4af086901e10ffa8e48866b735'. 2024-04-24T09:12:58.192 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-24T09:12:58.192 INFO:tasks.workunit.client.0.smithi069.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T09:12:58.193 INFO:tasks.workunit.client.0.smithi069.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T09:12:58.193 INFO:tasks.workunit.client.0.smithi069.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T09:12:58.193 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-24T09:12:58.193 INFO:tasks.workunit.client.0.smithi069.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T09:12:58.193 INFO:tasks.workunit.client.0.smithi069.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T09:12:58.193 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-24T09:12:58.193 INFO:tasks.workunit.client.0.smithi069.stderr: git switch -c 2024-04-24T09:12:58.193 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-24T09:12:58.193 INFO:tasks.workunit.client.0.smithi069.stderr:Or undo this operation with: 2024-04-24T09:12:58.193 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-24T09:12:58.193 INFO:tasks.workunit.client.0.smithi069.stderr: git switch - 2024-04-24T09:12:58.193 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-24T09:12:58.193 INFO:tasks.workunit.client.0.smithi069.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T09:12:58.193 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-24T09:12:58.198 INFO:tasks.workunit.client.0.smithi069.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2024-04-24T09:12:58.248 INFO:tasks.workunit.client.0.smithi069.stderr:+ make -j4 2024-04-24T09:12:58.353 INFO:tasks.workunit.client.0.smithi069.stdout:libtoolize -cfi 2024-04-24T09:12:59.237 INFO:tasks.workunit.client.0.smithi069.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-24T09:12:59.237 INFO:tasks.workunit.client.0.smithi069.stdout:libtoolize: copying file './config.guess' 2024-04-24T09:12:59.285 INFO:tasks.workunit.client.0.smithi069.stdout:libtoolize: copying file './config.sub' 2024-04-24T09:12:59.325 INFO:tasks.workunit.client.0.smithi069.stdout:libtoolize: copying file './install-sh' 2024-04-24T09:12:59.363 INFO:tasks.workunit.client.0.smithi069.stdout:libtoolize: copying file './ltmain.sh' 2024-04-24T09:12:59.493 INFO:tasks.workunit.client.0.smithi069.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-24T09:12:59.494 INFO:tasks.workunit.client.0.smithi069.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-24T09:12:59.536 INFO:tasks.workunit.client.0.smithi069.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-24T09:12:59.568 INFO:tasks.workunit.client.0.smithi069.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-24T09:12:59.598 INFO:tasks.workunit.client.0.smithi069.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-24T09:12:59.634 INFO:tasks.workunit.client.0.smithi069.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-24T09:12:59.665 INFO:tasks.workunit.client.0.smithi069.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-24T09:12:59.669 INFO:tasks.workunit.client.0.smithi069.stdout:cp include/install-sh . 2024-04-24T09:12:59.718 INFO:tasks.workunit.client.0.smithi069.stdout:aclocal -I m4 2024-04-24T09:12:59.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:12:59 smithi081 ceph-mon[30961]: pgmap v180: 129 pgs: 129 active+clean; 113 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 498 KiB/s wr, 119 op/s 2024-04-24T09:12:59.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:12:59 smithi069 ceph-mon[28170]: pgmap v180: 129 pgs: 129 active+clean; 113 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 498 KiB/s wr, 119 op/s 2024-04-24T09:12:59.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:12:59 smithi130 ceph-mon[31553]: pgmap v180: 129 pgs: 129 active+clean; 113 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 498 KiB/s wr, 119 op/s 2024-04-24T09:13:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.484+0000 7fc6de000640 1 -- 172.21.15.69:0/4086106132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d8104e00 msgr2=0x7fc6d8108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.484+0000 7fc6de000640 1 --2- 172.21.15.69:0/4086106132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d8104e00 0x7fc6d8108e40 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009900 tx=0x7fc6c402a410 comp rx=0 tx=0).stop 2024-04-24T09:13:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.484+0000 7fc6de000640 1 -- 172.21.15.69:0/4086106132 shutdown_connections 2024-04-24T09:13:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.484+0000 7fc6de000640 1 --2- 172.21.15.69:0/4086106132 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6d8109520 0x7fc6d81110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.484+0000 7fc6de000640 1 --2- 172.21.15.69:0/4086106132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d8104e00 0x7fc6d8108e40 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.484+0000 7fc6de000640 1 --2- 172.21.15.69:0/4086106132 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6d8104450 0x7fc6d8104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.484+0000 7fc6de000640 1 -- 172.21.15.69:0/4086106132 >> 172.21.15.69:0/4086106132 conn(0x7fc6d80ffcf0 msgr2=0x7fc6d8102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.484+0000 7fc6de000640 1 -- 172.21.15.69:0/4086106132 shutdown_connections 2024-04-24T09:13:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.484+0000 7fc6de000640 1 -- 172.21.15.69:0/4086106132 wait complete. 2024-04-24T09:13:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.485+0000 7fc6de000640 1 Processor -- start 2024-04-24T09:13:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.485+0000 7fc6de000640 1 -- start start 2024-04-24T09:13:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.485+0000 7fc6de000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6d8104450 0x7fc6d81a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.485+0000 7fc6de000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6d8109520 0x7fc6d81a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.486+0000 7fc6de000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d819a7e0 0x7fc6d819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.486+0000 7fc6de000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d81144c0 con 0x7fc6d819a7e0 2024-04-24T09:13:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.486+0000 7fc6de000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d8114340 con 0x7fc6d8104450 2024-04-24T09:13:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.486+0000 7fc6de000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d8114640 con 0x7fc6d8109520 2024-04-24T09:13:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.486+0000 7fc6d7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d819a7e0 0x7fc6d819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.486+0000 7fc6d7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d819a7e0 0x7fc6d819ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51250/0 (socket says 172.21.15.69:51250) 2024-04-24T09:13:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.486+0000 7fc6d7e00640 1 -- 172.21.15.69:0/747578567 learned_addr learned my addr 172.21.15.69:0/747578567 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.486+0000 7fc6d6a00640 1 --2- 172.21.15.69:0/747578567 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6d8109520 0x7fc6d81a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.486+0000 7fc6d7400640 1 --2- 172.21.15.69:0/747578567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6d8104450 0x7fc6d81a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.486+0000 7fc6d7e00640 1 -- 172.21.15.69:0/747578567 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6d8109520 msgr2=0x7fc6d81a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.486+0000 7fc6d7e00640 1 --2- 172.21.15.69:0/747578567 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6d8109520 0x7fc6d81a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.486+0000 7fc6d7e00640 1 -- 172.21.15.69:0/747578567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6d8104450 msgr2=0x7fc6d81a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.486+0000 7fc6d7e00640 1 --2- 172.21.15.69:0/747578567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6d8104450 0x7fc6d81a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.486+0000 7fc6d7e00640 1 -- 172.21.15.69:0/747578567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d819b370 con 0x7fc6d819a7e0 2024-04-24T09:13:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.487+0000 7fc6d7e00640 1 --2- 172.21.15.69:0/747578567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d819a7e0 0x7fc6d819ac90 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000b970 tx=0x7fc6d000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.487+0000 7fc6cfe00640 1 -- 172.21.15.69:0/747578567 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0019070 con 0x7fc6d819a7e0 2024-04-24T09:13:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.487+0000 7fc6cfe00640 1 -- 172.21.15.69:0/747578567 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6d0014440 con 0x7fc6d819a7e0 2024-04-24T09:13:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.487+0000 7fc6cfe00640 1 -- 172.21.15.69:0/747578567 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d00075d0 con 0x7fc6d819a7e0 2024-04-24T09:13:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.487+0000 7fc6de000640 1 -- 172.21.15.69:0/747578567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d819b660 con 0x7fc6d819a7e0 2024-04-24T09:13:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.487+0000 7fc6de000640 1 -- 172.21.15.69:0/747578567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d81a77c0 con 0x7fc6d819a7e0 2024-04-24T09:13:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.491+0000 7fc6e0e33740 1 Processor -- start 2024-04-24T09:13:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.491+0000 7fc6cfe00640 1 -- 172.21.15.69:0/747578567 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc6d001c070 con 0x7fc6d819a7e0 2024-04-24T09:13:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.491+0000 7fc6e0e33740 1 -- start start 2024-04-24T09:13:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.491+0000 7fc6e0e33740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5632893621b0 0x563289362590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.492+0000 7fc6e0e33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563289362ad0 0x56328936ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.492+0000 7fc6e0e33740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56328936d390 0x56328936f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.492+0000 7fc6e0e33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5632892c8900 con 0x563289362ad0 2024-04-24T09:13:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.492+0000 7fc6e0e33740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56328926ef20 con 0x5632893621b0 2024-04-24T09:13:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.492+0000 7fc6e0e33740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5632892c48c0 con 0x56328936d390 2024-04-24T09:13:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.492+0000 7fc6d7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563289362ad0 0x56328936ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.492+0000 7fc6d7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563289362ad0 0x56328936ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51252/0 (socket says 172.21.15.69:51252) 2024-04-24T09:13:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.492+0000 7fc6d7400640 1 -- 172.21.15.69:0/3011033851 learned_addr learned my addr 172.21.15.69:0/3011033851 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.492+0000 7fc6d6a00640 1 --2- 172.21.15.69:0/3011033851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5632893621b0 0x563289362590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.492+0000 7fc6d6a00640 1 --2- 172.21.15.69:0/3011033851 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56328936d390 0x56328936f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.493+0000 7fc6d7400640 1 -- 172.21.15.69:0/3011033851 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56328936d390 msgr2=0x56328936f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.493+0000 7fc6d7400640 1 --2- 172.21.15.69:0/3011033851 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56328936d390 0x56328936f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.493+0000 7fc6d7400640 1 -- 172.21.15.69:0/3011033851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5632893621b0 msgr2=0x563289362590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.493+0000 7fc6d7400640 1 --2- 172.21.15.69:0/3011033851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5632893621b0 0x563289362590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.493+0000 7fc6d7400640 1 -- 172.21.15.69:0/3011033851 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563289293e10 con 0x563289362ad0 2024-04-24T09:13:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.493+0000 7fc6d6a00640 1 --2- 172.21.15.69:0/3011033851 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56328936d390 0x56328936f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.493+0000 7fc6d7400640 1 --2- 172.21.15.69:0/3011033851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563289362ad0 0x56328936ce50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800cbf0 tx=0x7fc6c800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b2db5e19f84c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.493+0000 7fc6cfe00640 1 --2- 172.21.15.69:0/747578567 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc6bc0a3dc0 0x7fc6bc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.493+0000 7fc6cfe00640 1 -- 172.21.15.69:0/747578567 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc6d00d0080 con 0x7fc6d819a7e0 2024-04-24T09:13:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.493+0000 7fc6c3e00640 1 -- 172.21.15.69:0/3011033851 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8010630 con 0x563289362ad0 2024-04-24T09:13:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.493+0000 7fc6d6a00640 1 --2- 172.21.15.69:0/747578567 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc6bc0a3dc0 0x7fc6bc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.493+0000 7fc6c3e00640 1 -- 172.21.15.69:0/3011033851 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6c8010c80 con 0x563289362ad0 2024-04-24T09:13:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.493+0000 7fc6c3e00640 1 -- 172.21.15.69:0/3011033851 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8019b80 con 0x563289362ad0 2024-04-24T09:13:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.494+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3011033851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563289362ad0 msgr2=0x56328936ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.494+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/3011033851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563289362ad0 0x56328936ce50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800cbf0 tx=0x7fc6c800e720 comp rx=0 tx=0).stop 2024-04-24T09:13:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.494+0000 7fc6d6a00640 1 --2- 172.21.15.69:0/747578567 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc6bc0a3dc0 0x7fc6bc0a6280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4000c00 tx=0x7fc6c40058b0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.495+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3011033851 shutdown_connections 2024-04-24T09:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.495+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/3011033851 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56328936d390 0x56328936f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.495+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/3011033851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563289362ad0 0x56328936ce50 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.495+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/3011033851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5632893621b0 0x563289362590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.495+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3011033851 >> 172.21.15.69:0/3011033851 conn(0x56328935e6c0 msgr2=0x56328935eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.495+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3011033851 shutdown_connections 2024-04-24T09:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.495+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3011033851 wait complete. 2024-04-24T09:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.495+0000 7fc6e0e33740 1 -- start start 2024-04-24T09:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.495+0000 7fc6e0e33740 1 Processor -- start 2024-04-24T09:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.496+0000 7fc6e0e33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632893621b0 0x56328936a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.496+0000 7fc6e0e33740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563289362ad0 0x56328936a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.496+0000 7fc6e0e33740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56328936d390 0x56328936ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.496+0000 7fc6e0e33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56328936ffc0 con 0x5632893621b0 2024-04-24T09:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.496+0000 7fc6e0e33740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5632892822c0 con 0x56328936d390 2024-04-24T09:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.496+0000 7fc6e0e33740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563289370140 con 0x563289362ad0 2024-04-24T09:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.496+0000 7fc6e0e33740 10 client.0 _ll_register_callbacks cb 0x563288fdaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.496+0000 7fc6d7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632893621b0 0x56328936a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.496+0000 7fc6d7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632893621b0 0x56328936a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51256/0 (socket says 172.21.15.69:51256) 2024-04-24T09:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.496+0000 7fc6d7400640 1 -- 172.21.15.69:0/3913626825 learned_addr learned my addr 172.21.15.69:0/3913626825 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.496+0000 7fc6d7e00640 1 --2- 172.21.15.69:0/3913626825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56328936d390 0x56328936ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.497+0000 7fc6d6a00640 1 --2- 172.21.15.69:0/3913626825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563289362ad0 0x56328936a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.497+0000 7fc6d7400640 1 -- 172.21.15.69:0/3913626825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563289362ad0 msgr2=0x56328936a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.497+0000 7fc6d7400640 1 --2- 172.21.15.69:0/3913626825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563289362ad0 0x56328936a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.497+0000 7fc6d7400640 1 -- 172.21.15.69:0/3913626825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56328936d390 msgr2=0x56328936ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.497+0000 7fc6d7400640 1 --2- 172.21.15.69:0/3913626825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56328936d390 0x56328936ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.497+0000 7fc6d7400640 1 -- 172.21.15.69:0/3913626825 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563289292fd0 con 0x5632893621b0 2024-04-24T09:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.497+0000 7fc6d7e00640 1 --2- 172.21.15.69:0/3913626825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56328936d390 0x56328936ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.497+0000 7fc6d6a00640 1 --2- 172.21.15.69:0/3913626825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563289362ad0 0x56328936a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.497+0000 7fc6d7400640 1 --2- 172.21.15.69:0/3913626825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632893621b0 0x56328936a300 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800c960 tx=0x7fc6c8005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159724849560b399 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.497+0000 7fc6c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.497+0000 7fc6c0c00640 1 -- 172.21.15.69:0/3913626825 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8019e50 con 0x5632893621b0 2024-04-24T09:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.497+0000 7fc6c0c00640 1 -- 172.21.15.69:0/3913626825 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc6c8005250 con 0x5632893621b0 2024-04-24T09:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.497+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3913626825 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6c8010c80 con 0x5632893621b0 2024-04-24T09:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.498+0000 7fc6c0c00640 1 -- 172.21.15.69:0/3913626825 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc6c8005450 con 0x5632893621b0 2024-04-24T09:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.498+0000 7fc6e0e33740 10 client.16812 fetch_fsmap learned FSMap version 30 2024-04-24T09:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.498+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3913626825 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563288fdb210 con 0x5632893621b0 2024-04-24T09:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.498+0000 7fc6c0c00640 1 -- 172.21.15.69:0/3913626825 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6c8010c80 con 0x5632893621b0 2024-04-24T09:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.498+0000 7fc6c0c00640 1 -- 172.21.15.69:0/3913626825 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc6c8010630 con 0x5632893621b0 2024-04-24T09:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.498+0000 7fc6e0e33740 10 client.16812 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.498+0000 7fc6e0e33740 10 client.16812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.498+0000 7fc6e0e33740 20 client.16812 populate_metadata read hostname 'smithi069' 2024-04-24T09:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.498+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/3913626825 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x56328937d360 0x56328937f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.498+0000 7fc6e0e33740 4 client.16812 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.498+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3913626825 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632892d6320 con 0x56328937d360 2024-04-24T09:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.499+0000 7fc6d6a00640 1 --2- 172.21.15.69:0/3913626825 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x56328937d360 0x56328937f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.500+0000 7fc6d6a00640 1 --2- 172.21.15.69:0/3913626825 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x56328937d360 0x56328937f750 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34da6bf30c6146c1 server_cookie=a7b020b619a94fce in_seq=0 out_seq=0 2024-04-24T09:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.500+0000 7fc6c0c00640 10 client.16812 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.502+0000 7fc6c0c00640 1 -- 172.21.15.69:0/3913626825 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5632892d6320 con 0x56328937d360 2024-04-24T09:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.502+0000 7fc6c0c00640 10 client.16812 handle_command_reply: tid=0 2024-04-24T09:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.502+0000 7fc6e0e33740 1 client.16812 shutdown 2024-04-24T09:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.502+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3913626825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632893621b0 msgr2=0x56328936a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.502+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/3913626825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632893621b0 0x56328936a300 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800c960 tx=0x7fc6c8005fc0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.502+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3913626825 shutdown_connections 2024-04-24T09:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.502+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/3913626825 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x56328937d360 0x56328937f750 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.502+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/3913626825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56328936d390 0x56328936ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.502+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/3913626825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563289362ad0 0x56328936a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.502+0000 7fc6c0c00640 0 client.16812 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.502+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/3913626825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632893621b0 0x56328936a300 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.502+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3913626825 >> 172.21.15.69:0/3913626825 conn(0x56328935e6c0 msgr2=0x563289364440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.502+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3913626825 shutdown_connections 2024-04-24T09:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.502+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3913626825 wait complete. 2024-04-24T09:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.502+0000 7fc6e0e33740 20 client.16812 trim_cache size 0 max 16384 2024-04-24T09:13:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.521+0000 7fc6e0e33740 1 Processor -- start 2024-04-24T09:13:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.521+0000 7fc6e0e33740 1 -- start start 2024-04-24T09:13:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.521+0000 7fc6e0e33740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5632893621b0 0x563289378e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.522+0000 7fc6e0e33740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563289362ad0 0x563289384f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.522+0000 7fc6e0e33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56328936d390 0x563289385470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.522+0000 7fc6e0e33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d81b3d30 con 0x56328936d390 2024-04-24T09:13:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.522+0000 7fc6e0e33740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d81b15a0 con 0x563289362ad0 2024-04-24T09:13:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.522+0000 7fc6e0e33740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d81af3f0 con 0x5632893621b0 2024-04-24T09:13:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.522+0000 7fc6d7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56328936d390 0x563289385470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.522+0000 7fc6d7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56328936d390 0x563289385470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51258/0 (socket says 172.21.15.69:51258) 2024-04-24T09:13:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.522+0000 7fc6d7e00640 1 -- 172.21.15.69:0/4269167556 learned_addr learned my addr 172.21.15.69:0/4269167556 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.522+0000 7fc6d7400640 1 --2- 172.21.15.69:0/4269167556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5632893621b0 0x563289378e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.522+0000 7fc6d7e00640 1 -- 172.21.15.69:0/4269167556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5632893621b0 msgr2=0x563289378e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.522+0000 7fc6d7e00640 1 --2- 172.21.15.69:0/4269167556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5632893621b0 0x563289378e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.522+0000 7fc6d7e00640 1 -- 172.21.15.69:0/4269167556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563289362ad0 msgr2=0x563289384f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.522+0000 7fc6d7e00640 1 --2- 172.21.15.69:0/4269167556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563289362ad0 0x563289384f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.522+0000 7fc6d7e00640 1 -- 172.21.15.69:0/4269167556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563289362900 con 0x56328936d390 2024-04-24T09:13:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.523+0000 7fc6d7e00640 1 --2- 172.21.15.69:0/4269167556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56328936d390 0x563289385470 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000b940 tx=0x7fc6d0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea4c9f66efd4ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.523+0000 7fc6c3400640 1 -- 172.21.15.69:0/4269167556 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d00b5070 con 0x56328936d390 2024-04-24T09:13:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.523+0000 7fc6c3400640 1 -- 172.21.15.69:0/4269167556 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6d009bd90 con 0x56328936d390 2024-04-24T09:13:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.523+0000 7fc6c3400640 1 -- 172.21.15.69:0/4269167556 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d00cd840 con 0x56328936d390 2024-04-24T09:13:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.524+0000 7fc6e0e33740 1 -- 172.21.15.69:0/4269167556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56328936d390 msgr2=0x563289385470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.524+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/4269167556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56328936d390 0x563289385470 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000b940 tx=0x7fc6d0004b50 comp rx=0 tx=0).stop 2024-04-24T09:13:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.524+0000 7fc6e0e33740 1 -- 172.21.15.69:0/4269167556 shutdown_connections 2024-04-24T09:13:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.524+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/4269167556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56328936d390 0x563289385470 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.524+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/4269167556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563289362ad0 0x563289384f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.524+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/4269167556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5632893621b0 0x563289378e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.524+0000 7fc6e0e33740 1 -- 172.21.15.69:0/4269167556 >> 172.21.15.69:0/4269167556 conn(0x56328935e6c0 msgr2=0x56328935eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.524+0000 7fc6e0e33740 1 -- 172.21.15.69:0/4269167556 shutdown_connections 2024-04-24T09:13:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.524+0000 7fc6e0e33740 1 -- 172.21.15.69:0/4269167556 wait complete. 2024-04-24T09:13:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.524+0000 7fc6e0e33740 1 -- start start 2024-04-24T09:13:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.524+0000 7fc6e0e33740 1 Processor -- start 2024-04-24T09:13:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.525+0000 7fc6e0e33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632893621b0 0x563289366f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.525+0000 7fc6e0e33740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563289362ad0 0x5632893674d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.525+0000 7fc6e0e33740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56328936d390 0x563289367a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.525+0000 7fc6e0e33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d81afa60 con 0x5632893621b0 2024-04-24T09:13:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.525+0000 7fc6e0e33740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d81b0ed0 con 0x563289362ad0 2024-04-24T09:13:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.525+0000 7fc6e0e33740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d81b0130 con 0x56328936d390 2024-04-24T09:13:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.525+0000 7fc6e0e33740 10 client.0 _ll_register_callbacks cb 0x7fc6c8014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.525+0000 7fc6d7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632893621b0 0x563289366f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.525+0000 7fc6d7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632893621b0 0x563289366f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51260/0 (socket says 172.21.15.69:51260) 2024-04-24T09:13:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.525+0000 7fc6d7400640 1 -- 172.21.15.69:0/3363555115 learned_addr learned my addr 172.21.15.69:0/3363555115 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.526+0000 7fc6d6a00640 1 --2- 172.21.15.69:0/3363555115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563289362ad0 0x5632893674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.526+0000 7fc6d7400640 1 -- 172.21.15.69:0/3363555115 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56328936d390 msgr2=0x563289367a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.526+0000 7fc6d7400640 1 --2- 172.21.15.69:0/3363555115 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56328936d390 0x563289367a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.526+0000 7fc6d7400640 1 -- 172.21.15.69:0/3363555115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563289362ad0 msgr2=0x5632893674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.526+0000 7fc6d7400640 1 --2- 172.21.15.69:0/3363555115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563289362ad0 0x5632893674d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.526+0000 7fc6d7400640 1 -- 172.21.15.69:0/3363555115 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56328937fac0 con 0x5632893621b0 2024-04-24T09:13:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.526+0000 7fc6d7400640 1 --2- 172.21.15.69:0/3363555115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632893621b0 0x563289366f90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800cf50 tx=0x7fc6c800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0601dc4ab3305f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.526+0000 7fc6c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:13:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.526+0000 7fc6c0c00640 1 -- 172.21.15.69:0/3363555115 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8005250 con 0x5632893621b0 2024-04-24T09:13:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.527+0000 7fc6c0c00640 1 -- 172.21.15.69:0/3363555115 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc6c8004660 con 0x5632893621b0 2024-04-24T09:13:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.527+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3363555115 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6d009bd90 con 0x5632893621b0 2024-04-24T09:13:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.527+0000 7fc6c0c00640 1 -- 172.21.15.69:0/3363555115 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc6c80193c0 con 0x5632893621b0 2024-04-24T09:13:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.527+0000 7fc6e0e33740 10 client.16818 fetch_fsmap learned FSMap version 30 2024-04-24T09:13:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.527+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3363555115 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5632890a7620 con 0x5632893621b0 2024-04-24T09:13:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.527+0000 7fc6c0c00640 1 -- 172.21.15.69:0/3363555115 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6d009bd90 con 0x5632893621b0 2024-04-24T09:13:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.527+0000 7fc6c0c00640 1 -- 172.21.15.69:0/3363555115 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc6c8032070 con 0x5632893621b0 2024-04-24T09:13:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.527+0000 7fc6e0e33740 10 client.16818 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:13:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.527+0000 7fc6e0e33740 10 client.16818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.527+0000 7fc6e0e33740 20 client.16818 populate_metadata read hostname 'smithi069' 2024-04-24T09:13:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.528+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/3363555115 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5632893e3cc0 0x5632893e60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.528+0000 7fc6e0e33740 4 client.16818 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:13:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.528+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3363555115 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5632892d0190 con 0x5632893e3cc0 2024-04-24T09:13:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.528+0000 7fc6d6a00640 1 --2- 172.21.15.69:0/3363555115 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5632893e3cc0 0x5632893e60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.529+0000 7fc6d6a00640 1 --2- 172.21.15.69:0/3363555115 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5632893e3cc0 0x5632893e60e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21091a6fb52957ea server_cookie=5df36e7339e46edb in_seq=0 out_seq=0 2024-04-24T09:13:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.529+0000 7fc6c0c00640 10 client.16818 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.938+0000 7fc6c0c00640 1 -- 172.21.15.69:0/3363555115 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5632892d0190 con 0x5632893e3cc0 2024-04-24T09:13:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.938+0000 7fc6c0c00640 10 client.16818 handle_command_reply: tid=0 2024-04-24T09:13:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.939+0000 7fc6e0e33740 1 client.16818 shutdown 2024-04-24T09:13:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.939+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3363555115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632893621b0 msgr2=0x563289366f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.939+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/3363555115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632893621b0 0x563289366f90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800cf50 tx=0x7fc6c800ed60 comp rx=0 tx=0).stop 2024-04-24T09:13:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.939+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3363555115 shutdown_connections 2024-04-24T09:13:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.939+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/3363555115 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x5632893e3cc0 0x5632893e60e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.939+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/3363555115 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56328936d390 0x563289367a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.939+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/3363555115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563289362ad0 0x5632893674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.939+0000 7fc6e0e33740 1 --2- 172.21.15.69:0/3363555115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632893621b0 0x563289366f90 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.939+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3363555115 >> 172.21.15.69:0/3363555115 conn(0x56328935e6c0 msgr2=0x563289363ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.940+0000 7fc6c0c00640 0 client.16818 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.940+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3363555115 shutdown_connections 2024-04-24T09:13:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.940+0000 7fc6e0e33740 1 -- 172.21.15.69:0/3363555115 wait complete. 2024-04-24T09:13:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.940+0000 7fc6e0e33740 20 client.16818 trim_cache size 0 max 16384 2024-04-24T09:13:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:13:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:13:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:13:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:13:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.941+0000 7fc6de000640 1 -- 172.21.15.69:0/747578567 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc6bc0a3dc0 msgr2=0x7fc6bc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.941+0000 7fc6de000640 1 --2- 172.21.15.69:0/747578567 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc6bc0a3dc0 0x7fc6bc0a6280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4000c00 tx=0x7fc6c40058b0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.941+0000 7fc6de000640 1 -- 172.21.15.69:0/747578567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d819a7e0 msgr2=0x7fc6d819ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.941+0000 7fc6de000640 1 --2- 172.21.15.69:0/747578567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d819a7e0 0x7fc6d819ac90 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000b970 tx=0x7fc6d000be30 comp rx=0 tx=0).stop 2024-04-24T09:13:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.941+0000 7fc6de000640 1 -- 172.21.15.69:0/747578567 shutdown_connections 2024-04-24T09:13:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.941+0000 7fc6de000640 1 --2- 172.21.15.69:0/747578567 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fc6bc0a3dc0 0x7fc6bc0a6280 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.941+0000 7fc6de000640 1 --2- 172.21.15.69:0/747578567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d819a7e0 0x7fc6d819ac90 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.941+0000 7fc6de000640 1 --2- 172.21.15.69:0/747578567 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6d8109520 0x7fc6d81a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.941+0000 7fc6de000640 1 --2- 172.21.15.69:0/747578567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6d8104450 0x7fc6d81a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.941+0000 7fc6de000640 1 -- 172.21.15.69:0/747578567 >> 172.21.15.69:0/747578567 conn(0x7fc6d80ffcf0 msgr2=0x7fc6d8101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.942+0000 7fc6de000640 1 -- 172.21.15.69:0/747578567 shutdown_connections 2024-04-24T09:13:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:00.942+0000 7fc6de000640 1 -- 172.21.15.69:0/747578567 wait complete. 2024-04-24T09:13:01.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:01 smithi081 ceph-mon[30961]: pgmap v181: 129 pgs: 129 active+clean; 116 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 778 KiB/s wr, 156 op/s 2024-04-24T09:13:01.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:01 smithi069 ceph-mon[28170]: pgmap v181: 129 pgs: 129 active+clean; 116 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 778 KiB/s wr, 156 op/s 2024-04-24T09:13:01.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:01 smithi130 ceph-mon[31553]: pgmap v181: 129 pgs: 129 active+clean; 116 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 778 KiB/s wr, 156 op/s 2024-04-24T09:13:02.509 INFO:tasks.workunit.client.0.smithi069.stdout:autoheader 2024-04-24T09:13:03.282 INFO:tasks.workunit.client.0.smithi069.stdout:autoconf 2024-04-24T09:13:03.757 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:03 smithi130 ceph-mon[31553]: pgmap v182: 129 pgs: 129 active+clean; 116 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 521 KiB/s wr, 79 op/s 2024-04-24T09:13:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:03 smithi081 ceph-mon[30961]: pgmap v182: 129 pgs: 129 active+clean; 116 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 521 KiB/s wr, 79 op/s 2024-04-24T09:13:03.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:03 smithi069 ceph-mon[28170]: pgmap v182: 129 pgs: 129 active+clean; 116 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 521 KiB/s wr, 79 op/s 2024-04-24T09:13:03.970 INFO:tasks.workunit.client.0.smithi069.stdout:./configure \ 2024-04-24T09:13:03.970 INFO:tasks.workunit.client.0.smithi069.stdout: --libexecdir=/usr/lib \ 2024-04-24T09:13:03.970 INFO:tasks.workunit.client.0.smithi069.stdout: --exec_prefix=/var/lib 2024-04-24T09:13:04.666 INFO:tasks.workunit.client.0.smithi069.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-24T09:13:04.667 INFO:tasks.workunit.client.0.smithi069.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-24T09:13:04.672 INFO:tasks.workunit.client.0.smithi069.stdout:checking how to print strings... printf 2024-04-24T09:13:04.673 INFO:tasks.workunit.client.0.smithi069.stdout:checking for gcc... gcc 2024-04-24T09:13:05.109 INFO:tasks.workunit.client.0.smithi069.stdout:checking whether the C compiler works... yes 2024-04-24T09:13:05.110 INFO:tasks.workunit.client.0.smithi069.stdout:checking for C compiler default output file name... a.out 2024-04-24T09:13:05.341 INFO:tasks.workunit.client.0.smithi069.stdout:checking for suffix of executables... 2024-04-24T09:13:05.525 INFO:tasks.workunit.client.0.smithi069.stdout:checking whether we are cross compiling... no 2024-04-24T09:13:05.736 INFO:tasks.workunit.client.0.smithi069.stdout:checking for suffix of object files... o 2024-04-24T09:13:05.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:05 smithi081 ceph-mon[30961]: pgmap v183: 129 pgs: 129 active+clean; 116 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 521 KiB/s wr, 79 op/s 2024-04-24T09:13:05.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:05 smithi069 ceph-mon[28170]: pgmap v183: 129 pgs: 129 active+clean; 116 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 521 KiB/s wr, 79 op/s 2024-04-24T09:13:05.893 INFO:tasks.workunit.client.0.smithi069.stdout:checking whether we are using the GNU C compiler... yes 2024-04-24T09:13:05.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:05 smithi130 ceph-mon[31553]: pgmap v183: 129 pgs: 129 active+clean; 116 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 521 KiB/s wr, 79 op/s 2024-04-24T09:13:06.017 INFO:tasks.workunit.client.0.smithi069.stdout:checking whether gcc accepts -g... yes 2024-04-24T09:13:06.220 INFO:tasks.workunit.client.0.smithi069.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-24T09:13:06.240 INFO:tasks.workunit.client.0.smithi069.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-24T09:13:06.256 INFO:tasks.workunit.client.0.smithi069.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-24T09:13:06.262 INFO:tasks.workunit.client.0.smithi069.stdout:checking for egrep... /usr/bin/grep -E 2024-04-24T09:13:06.268 INFO:tasks.workunit.client.0.smithi069.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-24T09:13:06.282 INFO:tasks.workunit.client.0.smithi069.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-24T09:13:06.291 INFO:tasks.workunit.client.0.smithi069.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-24T09:13:06.320 INFO:tasks.workunit.client.0.smithi069.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-24T09:13:07.400 INFO:tasks.workunit.client.0.smithi069.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-24T09:13:07.402 INFO:tasks.workunit.client.0.smithi069.stdout:checking whether ln -s works... yes 2024-04-24T09:13:07.437 INFO:tasks.workunit.client.0.smithi069.stdout:checking the maximum length of command line arguments... 1572864 2024-04-24T09:13:07.449 INFO:tasks.workunit.client.0.smithi069.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-24T09:13:07.451 INFO:tasks.workunit.client.0.smithi069.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-24T09:13:07.452 INFO:tasks.workunit.client.0.smithi069.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-24T09:13:07.454 INFO:tasks.workunit.client.0.smithi069.stdout:checking for objdump... objdump 2024-04-24T09:13:07.456 INFO:tasks.workunit.client.0.smithi069.stdout:checking how to recognize dependent libraries... pass_all 2024-04-24T09:13:07.457 INFO:tasks.workunit.client.0.smithi069.stdout:checking for dlltool... no 2024-04-24T09:13:07.458 INFO:tasks.workunit.client.0.smithi069.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-24T09:13:07.461 INFO:tasks.workunit.client.0.smithi069.stdout:checking for ar... ar 2024-04-24T09:13:07.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:07 smithi081 ceph-mon[30961]: pgmap v184: 129 pgs: 129 active+clean; 114 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 806 KiB/s wr, 134 op/s 2024-04-24T09:13:07.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:07 smithi069 ceph-mon[28170]: pgmap v184: 129 pgs: 129 active+clean; 114 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 806 KiB/s wr, 134 op/s 2024-04-24T09:13:07.855 INFO:tasks.workunit.client.0.smithi069.stdout:checking for archiver @FILE support... @ 2024-04-24T09:13:07.859 INFO:tasks.workunit.client.0.smithi069.stdout:checking for strip... strip 2024-04-24T09:13:07.863 INFO:tasks.workunit.client.0.smithi069.stdout:checking for ranlib... ranlib 2024-04-24T09:13:07.867 INFO:tasks.workunit.client.0.smithi069.stdout:checking for gawk... gawk 2024-04-24T09:13:07.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:07 smithi130 ceph-mon[31553]: pgmap v184: 129 pgs: 129 active+clean; 114 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 806 KiB/s wr, 134 op/s 2024-04-24T09:13:08.479 INFO:tasks.workunit.client.0.smithi069.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-24T09:13:08.487 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sysroot... no 2024-04-24T09:13:08.578 INFO:tasks.workunit.client.0.smithi069.stdout:checking for a working dd... /usr/bin/dd 2024-04-24T09:13:08.628 INFO:tasks.workunit.client.0.smithi069.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-24T09:13:08.799 INFO:tasks.workunit.client.0.smithi069.stdout:checking for mt... no 2024-04-24T09:13:08.877 INFO:tasks.workunit.client.0.smithi069.stdout:checking if : is a manifest tool... no 2024-04-24T09:13:09.181 INFO:tasks.workunit.client.0.smithi069.stdout:checking how to run the C preprocessor... gcc -E 2024-04-24T09:13:09.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:09 smithi081 ceph-mon[30961]: pgmap v185: 129 pgs: 129 active+clean; 114 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 565 KiB/s wr, 91 op/s 2024-04-24T09:13:09.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:09 smithi069 ceph-mon[28170]: pgmap v185: 129 pgs: 129 active+clean; 114 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 565 KiB/s wr, 91 op/s 2024-04-24T09:13:09.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:09 smithi130 ceph-mon[31553]: pgmap v185: 129 pgs: 129 active+clean; 114 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 565 KiB/s wr, 91 op/s 2024-04-24T09:13:10.106 INFO:tasks.workunit.client.0.smithi069.stdout:checking for ANSI C header files... yes 2024-04-24T09:13:10.280 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/types.h... yes 2024-04-24T09:13:10.514 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/stat.h... yes 2024-04-24T09:13:10.743 INFO:tasks.workunit.client.0.smithi069.stdout:checking for stdlib.h... yes 2024-04-24T09:13:10.966 INFO:tasks.workunit.client.0.smithi069.stdout:checking for string.h... yes 2024-04-24T09:13:11.185 INFO:tasks.workunit.client.0.smithi069.stdout:checking for memory.h... yes 2024-04-24T09:13:11.410 INFO:tasks.workunit.client.0.smithi069.stdout:checking for strings.h... yes 2024-04-24T09:13:11.646 INFO:tasks.workunit.client.0.smithi069.stdout:checking for inttypes.h... yes 2024-04-24T09:13:11.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:11 smithi081 ceph-mon[30961]: pgmap v186: 129 pgs: 129 active+clean; 116 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 694 KiB/s wr, 114 op/s 2024-04-24T09:13:11.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:11 smithi069 ceph-mon[28170]: pgmap v186: 129 pgs: 129 active+clean; 116 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 694 KiB/s wr, 114 op/s 2024-04-24T09:13:11.866 INFO:tasks.workunit.client.0.smithi069.stdout:checking for stdint.h... yes 2024-04-24T09:13:11.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:11 smithi130 ceph-mon[31553]: pgmap v186: 129 pgs: 129 active+clean; 116 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 694 KiB/s wr, 114 op/s 2024-04-24T09:13:12.108 INFO:tasks.workunit.client.0.smithi069.stdout:checking for unistd.h... yes 2024-04-24T09:13:12.322 INFO:tasks.workunit.client.0.smithi069.stdout:checking for dlfcn.h... yes 2024-04-24T09:13:12.367 INFO:tasks.workunit.client.0.smithi069.stdout:checking for objdir... .libs 2024-04-24T09:13:12.899 INFO:tasks.workunit.client.0.smithi069.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-24T09:13:12.901 INFO:tasks.workunit.client.0.smithi069.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-24T09:13:13.101 INFO:tasks.workunit.client.0.smithi069.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-24T09:13:13.319 INFO:tasks.workunit.client.0.smithi069.stdout:checking if gcc static flag -static works... no 2024-04-24T09:13:13.654 INFO:tasks.workunit.client.0.smithi069.stdout:checking if gcc supports -c -o file.o... yes 2024-04-24T09:13:13.656 INFO:tasks.workunit.client.0.smithi069.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-24T09:13:13.684 INFO:tasks.workunit.client.0.smithi069.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-24T09:13:13.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:13 smithi081 ceph-mon[30961]: pgmap v187: 129 pgs: 129 active+clean; 116 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 414 KiB/s wr, 77 op/s 2024-04-24T09:13:13.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:13 smithi069 ceph-mon[28170]: pgmap v187: 129 pgs: 129 active+clean; 116 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 414 KiB/s wr, 77 op/s 2024-04-24T09:13:13.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:13 smithi130 ceph-mon[31553]: pgmap v187: 129 pgs: 129 active+clean; 116 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 414 KiB/s wr, 77 op/s 2024-04-24T09:13:14.005 INFO:tasks.workunit.client.0.smithi069.stdout:checking whether -lc should be explicitly linked in... no 2024-04-24T09:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.096+0000 7f5dae800640 1 -- 172.21.15.69:0/2265539636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5da810e9e0 msgr2=0x7f5da8110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.096+0000 7f5dae800640 1 --2- 172.21.15.69:0/2265539636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5da810e9e0 0x7f5da8110dd0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009ca0 tx=0x7f5da002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.096+0000 7f5dae800640 1 -- 172.21.15.69:0/2265539636 shutdown_connections 2024-04-24T09:13:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.096+0000 7f5dae800640 1 --2- 172.21.15.69:0/2265539636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5da810e9e0 0x7f5da8110dd0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.096+0000 7f5dae800640 1 --2- 172.21.15.69:0/2265539636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5da8101750 0x7f5da810e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.096+0000 7f5dae800640 1 --2- 172.21.15.69:0/2265539636 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5da8100e30 0x7f5da8101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.096+0000 7f5dae800640 1 -- 172.21.15.69:0/2265539636 >> 172.21.15.69:0/2265539636 conn(0x7f5da8078080 msgr2=0x7f5da8078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.097+0000 7f5dae800640 1 -- 172.21.15.69:0/2265539636 shutdown_connections 2024-04-24T09:13:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.097+0000 7f5dae800640 1 -- 172.21.15.69:0/2265539636 wait complete. 2024-04-24T09:13:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.097+0000 7f5dae800640 1 Processor -- start 2024-04-24T09:13:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.098+0000 7f5dae800640 1 -- start start 2024-04-24T09:13:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.098+0000 7f5dae800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5da8100e30 0x7f5da81a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.098+0000 7f5dae800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5da8101750 0x7f5da81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.098+0000 7f5dae800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5da810e9e0 0x7f5da819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.098+0000 7f5dae800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5da8114490 con 0x7f5da810e9e0 2024-04-24T09:13:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.098+0000 7f5dae800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5da8114310 con 0x7f5da8100e30 2024-04-24T09:13:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.098+0000 7f5dae800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5da8114610 con 0x7f5da8101750 2024-04-24T09:13:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.099+0000 7f5da7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5da810e9e0 0x7f5da819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.099+0000 7f5da7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5da810e9e0 0x7f5da819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49908/0 (socket says 172.21.15.69:49908) 2024-04-24T09:13:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.099+0000 7f5da7e00640 1 -- 172.21.15.69:0/630801321 learned_addr learned my addr 172.21.15.69:0/630801321 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.099+0000 7f5da7400640 1 --2- 172.21.15.69:0/630801321 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5da8100e30 0x7f5da81a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.099+0000 7f5da6a00640 1 --2- 172.21.15.69:0/630801321 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5da8101750 0x7f5da81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.099+0000 7f5da7e00640 1 -- 172.21.15.69:0/630801321 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5da8101750 msgr2=0x7f5da81a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.099+0000 7f5da7e00640 1 --2- 172.21.15.69:0/630801321 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5da8101750 0x7f5da81a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.099+0000 7f5da7e00640 1 -- 172.21.15.69:0/630801321 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5da8100e30 msgr2=0x7f5da81a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.099+0000 7f5da7e00640 1 --2- 172.21.15.69:0/630801321 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5da8100e30 0x7f5da81a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.099+0000 7f5da7e00640 1 -- 172.21.15.69:0/630801321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da819b010 con 0x7f5da810e9e0 2024-04-24T09:13:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.099+0000 7f5da7400640 1 --2- 172.21.15.69:0/630801321 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5da8100e30 0x7f5da81a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.100+0000 7f5da7e00640 1 --2- 172.21.15.69:0/630801321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5da810e9e0 0x7f5da819a750 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5da0031040 tx=0x7f5da0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.100+0000 7f5d9fe00640 1 -- 172.21.15.69:0/630801321 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da0003ce0 con 0x7f5da810e9e0 2024-04-24T09:13:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.100+0000 7f5d9fe00640 1 -- 172.21.15.69:0/630801321 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5da00044f0 con 0x7f5da810e9e0 2024-04-24T09:13:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.100+0000 7f5dae800640 1 -- 172.21.15.69:0/630801321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da819b2a0 con 0x7f5da810e9e0 2024-04-24T09:13:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.100+0000 7f5d9fe00640 1 -- 172.21.15.69:0/630801321 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da0030c80 con 0x7f5da810e9e0 2024-04-24T09:13:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.100+0000 7f5dae800640 1 -- 172.21.15.69:0/630801321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da81a7610 con 0x7f5da810e9e0 2024-04-24T09:13:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.104+0000 7f5d9fe00640 1 -- 172.21.15.69:0/630801321 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5da00046a0 con 0x7f5da810e9e0 2024-04-24T09:13:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.104+0000 7f5db1567740 1 Processor -- start 2024-04-24T09:13:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.104+0000 7f5db1567740 1 -- start start 2024-04-24T09:13:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.104+0000 7f5db1567740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a525291b0 0x559a52529590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.104+0000 7f5db1567740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a52529ad0 0x559a52533e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.104+0000 7f5db1567740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a52534390 0x559a52536780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.104+0000 7f5db1567740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559a5248f900 con 0x559a525291b0 2024-04-24T09:13:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.104+0000 7f5db1567740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x559a52435f20 con 0x559a52529ad0 2024-04-24T09:13:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.104+0000 7f5db1567740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559a5248b8c0 con 0x559a52534390 2024-04-24T09:13:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.105+0000 7f5da6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a525291b0 0x559a52529590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.105+0000 7f5da6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a525291b0 0x559a52529590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49918/0 (socket says 172.21.15.69:49918) 2024-04-24T09:13:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.105+0000 7f5da6a00640 1 -- 172.21.15.69:0/1083974896 learned_addr learned my addr 172.21.15.69:0/1083974896 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.105+0000 7f5da7400640 1 --2- 172.21.15.69:0/1083974896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a52529ad0 0x559a52533e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.105+0000 7f5da6a00640 1 --2- 172.21.15.69:0/1083974896 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a52534390 0x559a52536780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.105+0000 7f5da6a00640 1 -- 172.21.15.69:0/1083974896 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a52534390 msgr2=0x559a52536780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.105+0000 7f5da6a00640 1 --2- 172.21.15.69:0/1083974896 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a52534390 0x559a52536780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.105+0000 7f5da6a00640 1 -- 172.21.15.69:0/1083974896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a52529ad0 msgr2=0x559a52533e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.105+0000 7f5da6a00640 1 --2- 172.21.15.69:0/1083974896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a52529ad0 0x559a52533e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.105+0000 7f5da6a00640 1 -- 172.21.15.69:0/1083974896 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a5245ae10 con 0x559a525291b0 2024-04-24T09:13:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.105+0000 7f5da7400640 1 --2- 172.21.15.69:0/1083974896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a52529ad0 0x559a52533e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.105+0000 7f5da6a00640 1 --2- 172.21.15.69:0/1083974896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a525291b0 0x559a52529590 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5d94005fd0 tx=0x7f5d9400b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5afda200188ed3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.105+0000 7f5d9fe00640 1 --2- 172.21.15.69:0/630801321 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5d8c0a3e40 0x7f5d8c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.106+0000 7f5d9fe00640 1 -- 172.21.15.69:0/630801321 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f5da00f1f10 con 0x7f5da810e9e0 2024-04-24T09:13:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.106+0000 7f5d9cc00640 1 -- 172.21.15.69:0/1083974896 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d940110d0 con 0x559a525291b0 2024-04-24T09:13:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.106+0000 7f5da7400640 1 --2- 172.21.15.69:0/630801321 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5d8c0a3e40 0x7f5d8c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.106+0000 7f5d9cc00640 1 -- 172.21.15.69:0/1083974896 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d94005bf0 con 0x559a525291b0 2024-04-24T09:13:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.106+0000 7f5d9cc00640 1 -- 172.21.15.69:0/1083974896 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d940126c0 con 0x559a525291b0 2024-04-24T09:13:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.107+0000 7f5db1567740 1 -- 172.21.15.69:0/1083974896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a525291b0 msgr2=0x559a52529590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.107+0000 7f5da7400640 1 --2- 172.21.15.69:0/630801321 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5d8c0a3e40 0x7f5d8c0a6300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x559a5252a750 tx=0x7f5d9800b7d0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.107+0000 7f5db1567740 1 --2- 172.21.15.69:0/1083974896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a525291b0 0x559a52529590 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5d94005fd0 tx=0x7f5d9400b610 comp rx=0 tx=0).stop 2024-04-24T09:13:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.107+0000 7f5db1567740 1 -- 172.21.15.69:0/1083974896 shutdown_connections 2024-04-24T09:13:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.107+0000 7f5db1567740 1 --2- 172.21.15.69:0/1083974896 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a52534390 0x559a52536780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.107+0000 7f5db1567740 1 --2- 172.21.15.69:0/1083974896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a52529ad0 0x559a52533e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.107+0000 7f5db1567740 1 --2- 172.21.15.69:0/1083974896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a525291b0 0x559a52529590 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.107+0000 7f5db1567740 1 -- 172.21.15.69:0/1083974896 >> 172.21.15.69:0/1083974896 conn(0x559a525256c0 msgr2=0x559a52525aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.107+0000 7f5db1567740 1 -- 172.21.15.69:0/1083974896 shutdown_connections 2024-04-24T09:13:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.107+0000 7f5db1567740 1 -- 172.21.15.69:0/1083974896 wait complete. 2024-04-24T09:13:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.107+0000 7f5db1567740 1 -- start start 2024-04-24T09:13:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.108+0000 7f5db1567740 1 Processor -- start 2024-04-24T09:13:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.108+0000 7f5db1567740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a525291b0 0x559a525312c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.108+0000 7f5db1567740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a52529ad0 0x559a52531800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.108+0000 7f5db1567740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a52534390 0x559a52531d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.108+0000 7f5db1567740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559a52536f60 con 0x559a525291b0 2024-04-24T09:13:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.108+0000 7f5db1567740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x559a524492c0 con 0x559a52529ad0 2024-04-24T09:13:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.108+0000 7f5db1567740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559a525370e0 con 0x559a52534390 2024-04-24T09:13:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.108+0000 7f5db1567740 10 client.0 _ll_register_callbacks cb 0x559a521a1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.109+0000 7f5da7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a525291b0 0x559a525312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.109+0000 7f5da7400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a525291b0 0x559a525312c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49922/0 (socket says 172.21.15.69:49922) 2024-04-24T09:13:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.109+0000 7f5da7400640 1 -- 172.21.15.69:0/4219661758 learned_addr learned my addr 172.21.15.69:0/4219661758 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.109+0000 7f5da7e00640 1 --2- 172.21.15.69:0/4219661758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a52534390 0x559a52531d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.109+0000 7f5da6a00640 1 --2- 172.21.15.69:0/4219661758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a52529ad0 0x559a52531800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.109+0000 7f5da7400640 1 -- 172.21.15.69:0/4219661758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a52534390 msgr2=0x559a52531d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.109+0000 7f5da7400640 1 --2- 172.21.15.69:0/4219661758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a52534390 0x559a52531d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.109+0000 7f5da7400640 1 -- 172.21.15.69:0/4219661758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a52529ad0 msgr2=0x559a52531800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.109+0000 7f5da7400640 1 --2- 172.21.15.69:0/4219661758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a52529ad0 0x559a52531800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.109+0000 7f5da7400640 1 -- 172.21.15.69:0/4219661758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a52459fd0 con 0x559a525291b0 2024-04-24T09:13:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.110+0000 7f5da7400640 1 --2- 172.21.15.69:0/4219661758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a525291b0 0x559a525312c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5d98012800 tx=0x7f5d980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b097be3ab237aebd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.110+0000 7f5d91600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:13:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.110+0000 7f5d91600640 1 -- 172.21.15.69:0/4219661758 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9800fad0 con 0x559a525291b0 2024-04-24T09:13:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.110+0000 7f5d91600640 1 -- 172.21.15.69:0/4219661758 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f5d9800fc70 con 0x559a525291b0 2024-04-24T09:13:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.110+0000 7f5db1567740 1 -- 172.21.15.69:0/4219661758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a5249d320 con 0x559a525291b0 2024-04-24T09:13:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.110+0000 7f5d91600640 1 -- 172.21.15.69:0/4219661758 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d98030050 con 0x559a525291b0 2024-04-24T09:13:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.110+0000 7f5db1567740 10 client.16830 fetch_fsmap learned FSMap version 30 2024-04-24T09:13:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.110+0000 7f5db1567740 1 -- 172.21.15.69:0/4219661758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a521a2210 con 0x559a525291b0 2024-04-24T09:13:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.111+0000 7f5d91600640 1 -- 172.21.15.69:0/4219661758 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559a5249d320 con 0x559a525291b0 2024-04-24T09:13:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.111+0000 7f5d91600640 1 -- 172.21.15.69:0/4219661758 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5d980274b0 con 0x559a525291b0 2024-04-24T09:13:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.111+0000 7f5db1567740 10 client.16830 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:13:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.111+0000 7f5db1567740 10 client.16830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.111+0000 7f5db1567740 20 client.16830 populate_metadata read hostname 'smithi069' 2024-04-24T09:13:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.111+0000 7f5db1567740 1 --2- 172.21.15.69:0/4219661758 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x559a52544530 0x559a52546980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.111+0000 7f5db1567740 4 client.16830 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.111+0000 7f5db1567740 1 -- 172.21.15.69:0/4219661758 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a52497190 con 0x559a52544530 2024-04-24T09:13:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.112+0000 7f5da6a00640 1 --2- 172.21.15.69:0/4219661758 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x559a52544530 0x559a52546980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.112+0000 7f5da6a00640 1 --2- 172.21.15.69:0/4219661758 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x559a52544530 0x559a52546980 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae4bf77dc19d82c0 server_cookie=7e8b7754f262c79e in_seq=0 out_seq=0 2024-04-24T09:13:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.112+0000 7f5d91600640 10 client.16830 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.115+0000 7f5d91600640 1 -- 172.21.15.69:0/4219661758 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a52497190 con 0x559a52544530 2024-04-24T09:13:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.115+0000 7f5d91600640 10 client.16830 handle_command_reply: tid=0 2024-04-24T09:13:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.115+0000 7f5db1567740 1 client.16830 shutdown 2024-04-24T09:13:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.116+0000 7f5db1567740 1 -- 172.21.15.69:0/4219661758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a525291b0 msgr2=0x559a525312c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.116+0000 7f5db1567740 1 --2- 172.21.15.69:0/4219661758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a525291b0 0x559a525312c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5d98012800 tx=0x7f5d980046c0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.116+0000 7f5db1567740 1 -- 172.21.15.69:0/4219661758 shutdown_connections 2024-04-24T09:13:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.116+0000 7f5db1567740 1 --2- 172.21.15.69:0/4219661758 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x559a52544530 0x559a52546980 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.116+0000 7f5db1567740 1 --2- 172.21.15.69:0/4219661758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a52534390 0x559a52531d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.116+0000 7f5db1567740 1 --2- 172.21.15.69:0/4219661758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a52529ad0 0x559a52531800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.116+0000 7f5db1567740 1 --2- 172.21.15.69:0/4219661758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a525291b0 0x559a525312c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.116+0000 7f5d91600640 0 client.16830 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.116+0000 7f5db1567740 1 -- 172.21.15.69:0/4219661758 >> 172.21.15.69:0/4219661758 conn(0x559a525256c0 msgr2=0x559a5252b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.116+0000 7f5db1567740 1 -- 172.21.15.69:0/4219661758 shutdown_connections 2024-04-24T09:13:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.116+0000 7f5db1567740 1 -- 172.21.15.69:0/4219661758 wait complete. 2024-04-24T09:13:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.116+0000 7f5db1567740 20 client.16830 trim_cache size 0 max 16384 2024-04-24T09:13:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.133+0000 7f5db1567740 1 Processor -- start 2024-04-24T09:13:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.134+0000 7f5db1567740 1 -- start start 2024-04-24T09:13:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.134+0000 7f5db1567740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a525291b0 0x559a5252bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.134+0000 7f5db1567740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a52529ad0 0x559a525246c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.134+0000 7f5db1567740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a52534390 0x559a5254c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.134+0000 7f5db1567740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81b3d50 con 0x559a52529ad0 2024-04-24T09:13:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.134+0000 7f5db1567740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81b15c0 con 0x559a525291b0 2024-04-24T09:13:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.134+0000 7f5db1567740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81af410 con 0x559a52534390 2024-04-24T09:13:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.134+0000 7f5da6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a52529ad0 0x559a525246c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.134+0000 7f5da6a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a52529ad0 0x559a525246c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49938/0 (socket says 172.21.15.69:49938) 2024-04-24T09:13:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.134+0000 7f5da6a00640 1 -- 172.21.15.69:0/2497110628 learned_addr learned my addr 172.21.15.69:0/2497110628 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.134+0000 7f5da7400640 1 --2- 172.21.15.69:0/2497110628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a525291b0 0x559a5252bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.135+0000 7f5da7e00640 1 --2- 172.21.15.69:0/2497110628 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a52534390 0x559a5254c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.135+0000 7f5da6a00640 1 -- 172.21.15.69:0/2497110628 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a52534390 msgr2=0x559a5254c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.135+0000 7f5da6a00640 1 --2- 172.21.15.69:0/2497110628 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a52534390 0x559a5254c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.135+0000 7f5da6a00640 1 -- 172.21.15.69:0/2497110628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a525291b0 msgr2=0x559a5252bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.135+0000 7f5da6a00640 1 --2- 172.21.15.69:0/2497110628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a525291b0 0x559a5252bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.135+0000 7f5da6a00640 1 -- 172.21.15.69:0/2497110628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a5226e620 con 0x559a52529ad0 2024-04-24T09:13:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.135+0000 7f5da7400640 1 --2- 172.21.15.69:0/2497110628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a525291b0 0x559a5252bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.135+0000 7f5da6a00640 1 --2- 172.21.15.69:0/2497110628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a52529ad0 0x559a525246c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5d94011350 tx=0x7f5d94005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95097c24ef83798 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.135+0000 7f5d93e00640 1 -- 172.21.15.69:0/2497110628 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94005240 con 0x559a52529ad0 2024-04-24T09:13:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.136+0000 7f5d93e00640 1 -- 172.21.15.69:0/2497110628 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d940277f0 con 0x559a52529ad0 2024-04-24T09:13:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.136+0000 7f5d93e00640 1 -- 172.21.15.69:0/2497110628 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94004c10 con 0x559a52529ad0 2024-04-24T09:13:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.136+0000 7f5db1567740 1 -- 172.21.15.69:0/2497110628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a52529ad0 msgr2=0x559a525246c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.136+0000 7f5db1567740 1 --2- 172.21.15.69:0/2497110628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a52529ad0 0x559a525246c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5d94011350 tx=0x7f5d94005850 comp rx=0 tx=0).stop 2024-04-24T09:13:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.137+0000 7f5db1567740 1 -- 172.21.15.69:0/2497110628 shutdown_connections 2024-04-24T09:13:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.137+0000 7f5db1567740 1 --2- 172.21.15.69:0/2497110628 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a52534390 0x559a5254c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.137+0000 7f5db1567740 1 --2- 172.21.15.69:0/2497110628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a52529ad0 0x559a525246c0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.137+0000 7f5db1567740 1 --2- 172.21.15.69:0/2497110628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a525291b0 0x559a5252bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.137+0000 7f5db1567740 1 -- 172.21.15.69:0/2497110628 >> 172.21.15.69:0/2497110628 conn(0x559a525256c0 msgr2=0x559a52523fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.137+0000 7f5db1567740 1 -- 172.21.15.69:0/2497110628 shutdown_connections 2024-04-24T09:13:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.137+0000 7f5db1567740 1 -- 172.21.15.69:0/2497110628 wait complete. 2024-04-24T09:13:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.137+0000 7f5db1567740 1 -- start start 2024-04-24T09:13:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.138+0000 7f5db1567740 1 Processor -- start 2024-04-24T09:13:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.138+0000 7f5db1567740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a525291b0 0x559a5253c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.138+0000 7f5db1567740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a52529ad0 0x559a52537aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.138+0000 7f5db1567740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a52534390 0x559a52537fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.138+0000 7f5db1567740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81afa80 con 0x559a52534390 2024-04-24T09:13:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.138+0000 7f5db1567740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81b0ef0 con 0x559a52529ad0 2024-04-24T09:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.138+0000 7f5db1567740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81b0150 con 0x559a525291b0 2024-04-24T09:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.138+0000 7f5db1567740 10 client.0 _ll_register_callbacks cb 0x7f5d9800ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.139+0000 7f5da7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a52534390 0x559a52537fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.139+0000 7f5da7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a52534390 0x559a52537fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49952/0 (socket says 172.21.15.69:49952) 2024-04-24T09:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.139+0000 7f5da7e00640 1 -- 172.21.15.69:0/772591166 learned_addr learned my addr 172.21.15.69:0/772591166 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.139+0000 7f5da7400640 1 --2- 172.21.15.69:0/772591166 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a525291b0 0x559a5253c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.139+0000 7f5da6a00640 1 --2- 172.21.15.69:0/772591166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a52529ad0 0x559a52537aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.139+0000 7f5da7e00640 1 -- 172.21.15.69:0/772591166 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a525291b0 msgr2=0x559a5253c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.139+0000 7f5da7e00640 1 --2- 172.21.15.69:0/772591166 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a525291b0 0x559a5253c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.139+0000 7f5da7e00640 1 -- 172.21.15.69:0/772591166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a52529ad0 msgr2=0x559a52537aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.139+0000 7f5da7e00640 1 --2- 172.21.15.69:0/772591166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a52529ad0 0x559a52537aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.139+0000 7f5da7e00640 1 -- 172.21.15.69:0/772591166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a52538810 con 0x559a52534390 2024-04-24T09:13:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.140+0000 7f5da7e00640 1 --2- 172.21.15.69:0/772591166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a52534390 0x559a52537fe0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5da00bdc10 tx=0x7f5da00bd880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4201552542f5d5ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.140+0000 7f5d91600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:13:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.140+0000 7f5d91600640 1 -- 172.21.15.69:0/772591166 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da00098b0 con 0x559a52534390 2024-04-24T09:13:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.140+0000 7f5d91600640 1 -- 172.21.15.69:0/772591166 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f5da00ba2f0 con 0x559a52534390 2024-04-24T09:13:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.140+0000 7f5db1567740 1 -- 172.21.15.69:0/772591166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d940277f0 con 0x559a52534390 2024-04-24T09:13:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.141+0000 7f5d91600640 1 -- 172.21.15.69:0/772591166 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5da00bdc80 con 0x559a52534390 2024-04-24T09:13:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.141+0000 7f5db1567740 10 client.16836 fetch_fsmap learned FSMap version 30 2024-04-24T09:13:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.141+0000 7f5db1567740 1 -- 172.21.15.69:0/772591166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a525aac20 con 0x559a52534390 2024-04-24T09:13:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.141+0000 7f5d91600640 1 -- 172.21.15.69:0/772591166 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d940277f0 con 0x559a52534390 2024-04-24T09:13:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.141+0000 7f5d91600640 1 -- 172.21.15.69:0/772591166 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5da00de0b0 con 0x559a52534390 2024-04-24T09:13:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.141+0000 7f5db1567740 10 client.16836 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:13:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.141+0000 7f5db1567740 10 client.16836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.141+0000 7f5db1567740 20 client.16836 populate_metadata read hostname 'smithi069' 2024-04-24T09:13:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.142+0000 7f5db1567740 1 --2- 172.21.15.69:0/772591166 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x559a525ab3a0 0x559a525ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.142+0000 7f5db1567740 4 client.16836 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:13:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.142+0000 7f5db1567740 1 -- 172.21.15.69:0/772591166 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a524134b0 con 0x559a525ab3a0 2024-04-24T09:13:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.142+0000 7f5da7400640 1 --2- 172.21.15.69:0/772591166 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x559a525ab3a0 0x559a525ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.143+0000 7f5d91600640 10 client.16836 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.143+0000 7f5da7400640 1 --2- 172.21.15.69:0/772591166 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x559a525ab3a0 0x559a525ad7c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d88ef70cc1c27f62 server_cookie=3791a2f3167c2f8a in_seq=0 out_seq=0 2024-04-24T09:13:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.222+0000 7f5d91600640 1 -- 172.21.15.69:0/772591166 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a524134b0 con 0x559a525ab3a0 2024-04-24T09:13:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.222+0000 7f5d91600640 10 client.16836 handle_command_reply: tid=0 2024-04-24T09:13:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.222+0000 7f5db1567740 1 client.16836 shutdown 2024-04-24T09:13:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.222+0000 7f5db1567740 1 -- 172.21.15.69:0/772591166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a52534390 msgr2=0x559a52537fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.222+0000 7f5db1567740 1 --2- 172.21.15.69:0/772591166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a52534390 0x559a52537fe0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5da00bdc10 tx=0x7f5da00bd880 comp rx=0 tx=0).stop 2024-04-24T09:13:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.222+0000 7f5db1567740 1 -- 172.21.15.69:0/772591166 shutdown_connections 2024-04-24T09:13:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.222+0000 7f5db1567740 1 --2- 172.21.15.69:0/772591166 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x559a525ab3a0 0x559a525ad7c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.222+0000 7f5db1567740 1 --2- 172.21.15.69:0/772591166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559a52534390 0x559a52537fe0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.222+0000 7f5db1567740 1 --2- 172.21.15.69:0/772591166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a52529ad0 0x559a52537aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.222+0000 7f5db1567740 1 --2- 172.21.15.69:0/772591166 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559a525291b0 0x559a5253c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.222+0000 7f5db1567740 1 -- 172.21.15.69:0/772591166 >> 172.21.15.69:0/772591166 conn(0x559a525256c0 msgr2=0x559a52524840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.222+0000 7f5d91600640 0 client.16836 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.222+0000 7f5db1567740 1 -- 172.21.15.69:0/772591166 shutdown_connections 2024-04-24T09:13:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.222+0000 7f5db1567740 1 -- 172.21.15.69:0/772591166 wait complete. 2024-04-24T09:13:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.222+0000 7f5db1567740 20 client.16836 trim_cache size 0 max 16384 2024-04-24T09:13:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:13:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:13:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:13:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:13:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.223+0000 7f5dae800640 1 -- 172.21.15.69:0/630801321 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5d8c0a3e40 msgr2=0x7f5d8c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.223+0000 7f5dae800640 1 --2- 172.21.15.69:0/630801321 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5d8c0a3e40 0x7f5d8c0a6300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x559a5252a750 tx=0x7f5d9800b7d0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.223+0000 7f5dae800640 1 -- 172.21.15.69:0/630801321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5da810e9e0 msgr2=0x7f5da819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.223+0000 7f5dae800640 1 --2- 172.21.15.69:0/630801321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5da810e9e0 0x7f5da819a750 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5da0031040 tx=0x7f5da0003970 comp rx=0 tx=0).stop 2024-04-24T09:13:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.223+0000 7f5dae800640 1 -- 172.21.15.69:0/630801321 shutdown_connections 2024-04-24T09:13:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.223+0000 7f5dae800640 1 --2- 172.21.15.69:0/630801321 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5d8c0a3e40 0x7f5d8c0a6300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.223+0000 7f5dae800640 1 --2- 172.21.15.69:0/630801321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5da810e9e0 0x7f5da819a750 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.223+0000 7f5dae800640 1 --2- 172.21.15.69:0/630801321 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5da8101750 0x7f5da81a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.223+0000 7f5dae800640 1 --2- 172.21.15.69:0/630801321 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5da8100e30 0x7f5da81a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.223+0000 7f5dae800640 1 -- 172.21.15.69:0/630801321 >> 172.21.15.69:0/630801321 conn(0x7f5da8078080 msgr2=0x7f5da810fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.223+0000 7f5dae800640 1 -- 172.21.15.69:0/630801321 shutdown_connections 2024-04-24T09:13:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:14.223+0000 7f5dae800640 1 -- 172.21.15.69:0/630801321 wait complete. 2024-04-24T09:13:14.281 INFO:tasks.workunit.client.0.smithi069.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-24T09:13:14.282 INFO:tasks.workunit.client.0.smithi069.stdout:checking how to hardcode library paths into programs... immediate 2024-04-24T09:13:14.286 INFO:tasks.workunit.client.0.smithi069.stdout:checking whether stripping libraries is possible... yes 2024-04-24T09:13:14.286 INFO:tasks.workunit.client.0.smithi069.stdout:checking if libtool supports shared libraries... yes 2024-04-24T09:13:14.287 INFO:tasks.workunit.client.0.smithi069.stdout:checking whether to build shared libraries... yes 2024-04-24T09:13:14.288 INFO:tasks.workunit.client.0.smithi069.stdout:checking whether to build static libraries... yes 2024-04-24T09:13:14.309 INFO:tasks.workunit.client.0.smithi069.stdout:checking for gcc... (cached) gcc 2024-04-24T09:13:14.498 INFO:tasks.workunit.client.0.smithi069.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-24T09:13:14.500 INFO:tasks.workunit.client.0.smithi069.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-24T09:13:14.503 INFO:tasks.workunit.client.0.smithi069.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-24T09:13:14.506 INFO:tasks.workunit.client.0.smithi069.stdout:checking for gmake... /usr/bin/gmake 2024-04-24T09:13:14.506 INFO:tasks.workunit.client.0.smithi069.stderr:./configure: line 11968: test: $(SHELL): binary operator expected 2024-04-24T09:13:14.510 INFO:tasks.workunit.client.0.smithi069.stdout:checking for tar... /bin/tar 2024-04-24T09:13:14.514 INFO:tasks.workunit.client.0.smithi069.stdout:checking for gzip... /bin/gzip 2024-04-24T09:13:14.590 INFO:tasks.workunit.client.0.smithi069.stdout:checking whether gcc -MM is supported... yes 2024-04-24T09:13:14.591 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sort... /bin/sort 2024-04-24T09:13:14.592 INFO:tasks.workunit.client.0.smithi069.stdout:checking whether ln -s works... yes 2024-04-24T09:13:14.595 INFO:tasks.workunit.client.0.smithi069.stdout:checking for rpm... /bin/rpm 2024-04-24T09:13:14.610 INFO:tasks.workunit.client.0.smithi069.stdout:checking for rpmbuild... no 2024-04-24T09:13:14.611 INFO:tasks.workunit.client.0.smithi069.stdout:checking for ANSI C header files... (cached) yes 2024-04-24T09:13:14.853 INFO:tasks.workunit.client.0.smithi069.stdout:checking assert.h usability... yes 2024-04-24T09:13:14.937 INFO:tasks.workunit.client.0.smithi069.stdout:checking assert.h presence... yes 2024-04-24T09:13:14.939 INFO:tasks.workunit.client.0.smithi069.stdout:checking for assert.h... yes 2024-04-24T09:13:15.213 INFO:tasks.workunit.client.0.smithi069.stdout:checking bstring.h usability... no 2024-04-24T09:13:15.380 INFO:tasks.workunit.client.0.smithi069.stdout:checking bstring.h presence... no 2024-04-24T09:13:15.383 INFO:tasks.workunit.client.0.smithi069.stdout:checking for bstring.h... no 2024-04-24T09:13:15.575 INFO:tasks.workunit.client.0.smithi069.stdout:checking libgen.h usability... yes 2024-04-24T09:13:15.650 INFO:tasks.workunit.client.0.smithi069.stdout:checking libgen.h presence... yes 2024-04-24T09:13:15.651 INFO:tasks.workunit.client.0.smithi069.stdout:checking for libgen.h... yes 2024-04-24T09:13:15.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:15 smithi081 ceph-mon[30961]: pgmap v188: 129 pgs: 129 active+clean; 116 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 414 KiB/s wr, 77 op/s 2024-04-24T09:13:15.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:15 smithi069 ceph-mon[28170]: pgmap v188: 129 pgs: 129 active+clean; 116 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 414 KiB/s wr, 77 op/s 2024-04-24T09:13:15.866 INFO:tasks.workunit.client.0.smithi069.stdout:checking dirent.h usability... yes 2024-04-24T09:13:15.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:15 smithi130 ceph-mon[31553]: pgmap v188: 129 pgs: 129 active+clean; 116 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 414 KiB/s wr, 77 op/s 2024-04-24T09:13:15.958 INFO:tasks.workunit.client.0.smithi069.stdout:checking dirent.h presence... yes 2024-04-24T09:13:15.960 INFO:tasks.workunit.client.0.smithi069.stdout:checking for dirent.h... yes 2024-04-24T09:13:16.200 INFO:tasks.workunit.client.0.smithi069.stdout:checking errno.h usability... yes 2024-04-24T09:13:16.284 INFO:tasks.workunit.client.0.smithi069.stdout:checking errno.h presence... yes 2024-04-24T09:13:16.287 INFO:tasks.workunit.client.0.smithi069.stdout:checking for errno.h... yes 2024-04-24T09:13:16.532 INFO:tasks.workunit.client.0.smithi069.stdout:checking malloc.h usability... yes 2024-04-24T09:13:16.620 INFO:tasks.workunit.client.0.smithi069.stdout:checking malloc.h presence... yes 2024-04-24T09:13:16.621 INFO:tasks.workunit.client.0.smithi069.stdout:checking for malloc.h... yes 2024-04-24T09:13:16.899 INFO:tasks.workunit.client.0.smithi069.stdout:checking uuid.h usability... no 2024-04-24T09:13:17.048 INFO:tasks.workunit.client.0.smithi069.stdout:checking uuid.h presence... no 2024-04-24T09:13:17.050 INFO:tasks.workunit.client.0.smithi069.stdout:checking for uuid.h... no 2024-04-24T09:13:17.244 INFO:tasks.workunit.client.0.smithi069.stdout:checking uuid/uuid.h usability... yes 2024-04-24T09:13:17.332 INFO:tasks.workunit.client.0.smithi069.stdout:checking uuid/uuid.h presence... yes 2024-04-24T09:13:17.334 INFO:tasks.workunit.client.0.smithi069.stdout:checking for uuid/uuid.h... yes 2024-04-24T09:13:17.610 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/uuid.h usability... no 2024-04-24T09:13:17.745 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/uuid.h presence... no 2024-04-24T09:13:17.747 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/uuid.h... no 2024-04-24T09:13:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:17 smithi081 ceph-mon[30961]: pgmap v189: 129 pgs: 129 active+clean; 115 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 715 KiB/s wr, 125 op/s 2024-04-24T09:13:17.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:17 smithi069 ceph-mon[28170]: pgmap v189: 129 pgs: 129 active+clean; 115 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 715 KiB/s wr, 125 op/s 2024-04-24T09:13:17.942 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/file.h usability... yes 2024-04-24T09:13:17.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:17 smithi130 ceph-mon[31553]: pgmap v189: 129 pgs: 129 active+clean; 115 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 715 KiB/s wr, 125 op/s 2024-04-24T09:13:18.030 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/file.h presence... yes 2024-04-24T09:13:18.031 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/file.h... yes 2024-04-24T09:13:18.258 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/fcntl.h usability... yes 2024-04-24T09:13:18.345 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/fcntl.h presence... yes 2024-04-24T09:13:18.347 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/fcntl.h... yes 2024-04-24T09:13:18.637 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/syssgi.h usability... no 2024-04-24T09:13:18.786 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/syssgi.h presence... no 2024-04-24T09:13:18.788 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/syssgi.h... no 2024-04-24T09:13:19.024 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/param.h usability... yes 2024-04-24T09:13:19.117 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/param.h presence... yes 2024-04-24T09:13:19.119 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/param.h... yes 2024-04-24T09:13:19.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:18 smithi130 ceph-mon[31553]: pgmap v190: 129 pgs: 129 active+clean; 115 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 429 KiB/s wr, 70 op/s 2024-04-24T09:13:19.206 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/stat.h... (cached) yes 2024-04-24T09:13:19.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:18 smithi081 ceph-mon[30961]: pgmap v190: 129 pgs: 129 active+clean; 115 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 429 KiB/s wr, 70 op/s 2024-04-24T09:13:19.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:18 smithi069 ceph-mon[28170]: pgmap v190: 129 pgs: 129 active+clean; 115 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 429 KiB/s wr, 70 op/s 2024-04-24T09:13:19.434 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/statvfs.h usability... yes 2024-04-24T09:13:19.525 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/statvfs.h presence... yes 2024-04-24T09:13:19.527 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/statvfs.h... yes 2024-04-24T09:13:19.760 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/time.h usability... yes 2024-04-24T09:13:19.850 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/time.h presence... yes 2024-04-24T09:13:19.852 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/time.h... yes 2024-04-24T09:13:20.106 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/ioctl.h usability... yes 2024-04-24T09:13:20.189 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/ioctl.h presence... yes 2024-04-24T09:13:20.190 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/ioctl.h... yes 2024-04-24T09:13:20.446 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/wait.h usability... yes 2024-04-24T09:13:20.537 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/wait.h presence... yes 2024-04-24T09:13:20.538 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/wait.h... yes 2024-04-24T09:13:20.641 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/types.h... (cached) yes 2024-04-24T09:13:20.719 INFO:tasks.workunit.client.0.smithi069.stdout:checking for strings.h... (cached) yes 2024-04-24T09:13:20.953 INFO:tasks.workunit.client.0.smithi069.stdout:checking err.h usability... yes 2024-04-24T09:13:21.055 INFO:tasks.workunit.client.0.smithi069.stdout:checking err.h presence... yes 2024-04-24T09:13:21.058 INFO:tasks.workunit.client.0.smithi069.stdout:checking for err.h... yes 2024-04-24T09:13:21.296 INFO:tasks.workunit.client.0.smithi069.stdout:checking linux/falloc.h usability... yes 2024-04-24T09:13:21.380 INFO:tasks.workunit.client.0.smithi069.stdout:checking linux/falloc.h presence... yes 2024-04-24T09:13:21.383 INFO:tasks.workunit.client.0.smithi069.stdout:checking for linux/falloc.h... yes 2024-04-24T09:13:21.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:21 smithi130 ceph-mon[31553]: pgmap v191: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 597 KiB/s wr, 96 op/s 2024-04-24T09:13:21.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:21 smithi081 ceph-mon[30961]: pgmap v191: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 597 KiB/s wr, 96 op/s 2024-04-24T09:13:21.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:21 smithi069 ceph-mon[28170]: pgmap v191: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 597 KiB/s wr, 96 op/s 2024-04-24T09:13:21.652 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-24T09:13:21.808 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-24T09:13:21.811 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-24T09:13:22.020 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-24T09:13:22.191 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-24T09:13:22.193 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-24T09:13:22.414 INFO:tasks.workunit.client.0.smithi069.stdout:checking xfs/platform_defs.h usability... no 2024-04-24T09:13:22.566 INFO:tasks.workunit.client.0.smithi069.stdout:checking xfs/platform_defs.h presence... no 2024-04-24T09:13:22.568 INFO:tasks.workunit.client.0.smithi069.stdout:checking for xfs/platform_defs.h... no 2024-04-24T09:13:22.769 INFO:tasks.workunit.client.0.smithi069.stdout:checking btrfs/ioctl.h usability... no 2024-04-24T09:13:22.911 INFO:tasks.workunit.client.0.smithi069.stdout:checking btrfs/ioctl.h presence... no 2024-04-24T09:13:22.913 INFO:tasks.workunit.client.0.smithi069.stdout:checking for btrfs/ioctl.h... no 2024-04-24T09:13:23.128 INFO:tasks.workunit.client.0.smithi069.stdout:checking cifs/ioctl.h usability... no 2024-04-24T09:13:23.254 INFO:tasks.workunit.client.0.smithi069.stdout:checking cifs/ioctl.h presence... no 2024-04-24T09:13:23.255 INFO:tasks.workunit.client.0.smithi069.stdout:checking for cifs/ioctl.h... no 2024-04-24T09:13:23.393 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:23 smithi069 ceph-mon[28170]: pgmap v192: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 468 KiB/s wr, 74 op/s 2024-04-24T09:13:23.401 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/mman.h usability... yes 2024-04-24T09:13:23.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:23 smithi130 ceph-mon[31553]: pgmap v192: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 468 KiB/s wr, 74 op/s 2024-04-24T09:13:23.470 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/mman.h presence... yes 2024-04-24T09:13:23.472 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/mman.h... yes 2024-04-24T09:13:23.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:23 smithi081 ceph-mon[30961]: pgmap v192: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 468 KiB/s wr, 74 op/s 2024-04-24T09:13:23.645 INFO:tasks.workunit.client.0.smithi069.stdout:checking for xfs/xfs_log_format.h... no 2024-04-24T09:13:23.873 INFO:tasks.workunit.client.0.smithi069.stdout:checking for uuid_compare... no 2024-04-24T09:13:24.036 INFO:tasks.workunit.client.0.smithi069.stdout:checking for uuid_compare in -luuid... yes 2024-04-24T09:13:24.284 INFO:tasks.workunit.client.0.smithi069.stdout:checking for xfs/xfs.h... yes 2024-04-24T09:13:24.437 INFO:tasks.workunit.client.0.smithi069.stdout:checking for xfs/libxfs.h... no 2024-04-24T09:13:24.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:24 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:24.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:24 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:24.574 INFO:tasks.workunit.client.0.smithi069.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-24T09:13:24.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:24 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:24.704 INFO:tasks.workunit.client.0.smithi069.stdout:checking for xfs/xqm.h... yes 2024-04-24T09:13:24.945 INFO:tasks.workunit.client.0.smithi069.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-24T09:13:25.107 INFO:tasks.workunit.client.0.smithi069.stdout:checking xfs/handle.h usability... yes 2024-04-24T09:13:25.186 INFO:tasks.workunit.client.0.smithi069.stdout:checking xfs/handle.h presence... yes 2024-04-24T09:13:25.188 INFO:tasks.workunit.client.0.smithi069.stdout:checking for xfs/handle.h... yes 2024-04-24T09:13:25.434 INFO:tasks.workunit.client.0.smithi069.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-24T09:13:25.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:25 smithi130 ceph-mon[31553]: pgmap v193: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 468 KiB/s wr, 74 op/s 2024-04-24T09:13:25.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:25 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:25.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:25 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:25.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:25 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:13:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:25 smithi081 ceph-mon[30961]: pgmap v193: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 468 KiB/s wr, 74 op/s 2024-04-24T09:13:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:25 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:25 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:25 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:13:25.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:25 smithi069 ceph-mon[28170]: pgmap v193: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 468 KiB/s wr, 74 op/s 2024-04-24T09:13:25.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:25 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:25.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:25 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:25.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:25 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:13:25.653 INFO:tasks.workunit.client.0.smithi069.stdout:checking attr/attributes.h usability... yes 2024-04-24T09:13:25.714 INFO:tasks.workunit.client.0.smithi069.stdout:checking attr/attributes.h presence... yes 2024-04-24T09:13:25.715 INFO:tasks.workunit.client.0.smithi069.stdout:checking for attr/attributes.h... yes 2024-04-24T09:13:25.996 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/attributes.h usability... no 2024-04-24T09:13:26.163 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/attributes.h presence... no 2024-04-24T09:13:26.166 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/attributes.h... no 2024-04-24T09:13:26.417 INFO:tasks.workunit.client.0.smithi069.stdout:checking for attr_list in -lattr... yes 2024-04-24T09:13:26.658 INFO:tasks.workunit.client.0.smithi069.stdout:checking for attr_set in -lattr... yes 2024-04-24T09:13:26.796 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/acl.h usability... yes 2024-04-24T09:13:26.883 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/acl.h presence... yes 2024-04-24T09:13:26.885 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/acl.h... yes 2024-04-24T09:13:27.037 INFO:tasks.workunit.client.0.smithi069.stdout:checking acl/libacl.h usability... yes 2024-04-24T09:13:27.123 INFO:tasks.workunit.client.0.smithi069.stdout:checking acl/libacl.h presence... yes 2024-04-24T09:13:27.123 INFO:tasks.workunit.client.0.smithi069.stdout:checking for acl/libacl.h... yes 2024-04-24T09:13:27.351 INFO:tasks.workunit.client.0.smithi069.stdout:checking for acl_init in -lacl... yes 2024-04-24T09:13:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.363+0000 7f4e1a400640 1 -- 172.21.15.69:0/3850810769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e14074c20 msgr2=0x7f4e1410e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.363+0000 7f4e1a400640 1 --2- 172.21.15.69:0/3850810769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e14074c20 0x7f4e1410e3b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009900 tx=0x7f4dfc02a480 comp rx=0 tx=0).stop 2024-04-24T09:13:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.363+0000 7f4e1a400640 1 -- 172.21.15.69:0/3850810769 shutdown_connections 2024-04-24T09:13:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.363+0000 7f4e1a400640 1 --2- 172.21.15.69:0/3850810769 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e1410e8f0 0x7f4e14110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.363+0000 7f4e1a400640 1 --2- 172.21.15.69:0/3850810769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e14074c20 0x7f4e1410e3b0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.363+0000 7f4e1a400640 1 --2- 172.21.15.69:0/3850810769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e14074300 0x7f4e140746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.363+0000 7f4e1a400640 1 -- 172.21.15.69:0/3850810769 >> 172.21.15.69:0/3850810769 conn(0x7f4e14079490 msgr2=0x7f4e140798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.363+0000 7f4e1a400640 1 -- 172.21.15.69:0/3850810769 shutdown_connections 2024-04-24T09:13:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.363+0000 7f4e1a400640 1 -- 172.21.15.69:0/3850810769 wait complete. 2024-04-24T09:13:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.364+0000 7f4e1a400640 1 Processor -- start 2024-04-24T09:13:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.365+0000 7f4e1a400640 1 -- start start 2024-04-24T09:13:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.365+0000 7f4e1a400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e14074300 0x7f4e141a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.365+0000 7f4e1a400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e14074c20 0x7f4e141a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.365+0000 7f4e1a400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e1410e8f0 0x7f4e1419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.365+0000 7f4e1a400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141144f0 con 0x7f4e14074300 2024-04-24T09:13:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.365+0000 7f4e1a400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4e14114370 con 0x7f4e14074c20 2024-04-24T09:13:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.365+0000 7f4e1a400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4e14114670 con 0x7f4e1410e8f0 2024-04-24T09:13:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.365+0000 7f4e13400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e14074300 0x7f4e141a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.365+0000 7f4e13400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e14074300 0x7f4e141a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35964/0 (socket says 172.21.15.69:35964) 2024-04-24T09:13:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.365+0000 7f4e13400640 1 -- 172.21.15.69:0/3769015918 learned_addr learned my addr 172.21.15.69:0/3769015918 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.366+0000 7f4e13e00640 1 --2- 172.21.15.69:0/3769015918 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e1410e8f0 0x7f4e1419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.366+0000 7f4e0aa00640 1 --2- 172.21.15.69:0/3769015918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e14074c20 0x7f4e141a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.366+0000 7f4e13400640 1 -- 172.21.15.69:0/3769015918 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e1410e8f0 msgr2=0x7f4e1419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.366+0000 7f4e13400640 1 --2- 172.21.15.69:0/3769015918 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e1410e8f0 0x7f4e1419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.366+0000 7f4e13400640 1 -- 172.21.15.69:0/3769015918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e14074c20 msgr2=0x7f4e141a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.366+0000 7f4e13400640 1 --2- 172.21.15.69:0/3769015918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e14074c20 0x7f4e141a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.366+0000 7f4e13400640 1 -- 172.21.15.69:0/3769015918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1419aff0 con 0x7f4e14074300 2024-04-24T09:13:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.366+0000 7f4e13e00640 1 --2- 172.21.15.69:0/3769015918 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e1410e8f0 0x7f4e1419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.366+0000 7f4e13400640 1 --2- 172.21.15.69:0/3769015918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e14074300 0x7f4e141a0620 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400eab0 tx=0x7f4e0400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.366+0000 7f4e10c00640 1 -- 172.21.15.69:0/3769015918 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0400ce40 con 0x7f4e14074300 2024-04-24T09:13:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.366+0000 7f4e10c00640 1 -- 172.21.15.69:0/3769015918 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e04011920 con 0x7f4e14074300 2024-04-24T09:13:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.366+0000 7f4e1a400640 1 -- 172.21.15.69:0/3769015918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1419b2e0 con 0x7f4e14074300 2024-04-24T09:13:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.366+0000 7f4e10c00640 1 -- 172.21.15.69:0/3769015918 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e04010a00 con 0x7f4e14074300 2024-04-24T09:13:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.367+0000 7f4e1a400640 1 -- 172.21.15.69:0/3769015918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e141a75a0 con 0x7f4e14074300 2024-04-24T09:13:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.370+0000 7f4e10c00640 1 -- 172.21.15.69:0/3769015918 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4e04011ad0 con 0x7f4e14074300 2024-04-24T09:13:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.370+0000 7f4e1d19b740 1 Processor -- start 2024-04-24T09:13:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.371+0000 7f4e1d19b740 1 -- start start 2024-04-24T09:13:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.372+0000 7f4e1d19b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34c71b0 0x55eab34c7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.372+0000 7f4e1d19b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.372+0000 7f4e1d19b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34d2390 0x55eab34d4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.372+0000 7f4e1d19b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55eab342d900 con 0x55eab34c7ad0 2024-04-24T09:13:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.372+0000 7f4e1d19b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55eab33d3f20 con 0x55eab34c71b0 2024-04-24T09:13:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.372+0000 7f4e1d19b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55eab34298c0 con 0x55eab34d2390 2024-04-24T09:13:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.372+0000 7f4e13400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.372+0000 7f4e13400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35974/0 (socket says 172.21.15.69:35974) 2024-04-24T09:13:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.372+0000 7f4e13400640 1 -- 172.21.15.69:0/2701261366 learned_addr learned my addr 172.21.15.69:0/2701261366 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.372+0000 7f4e13e00640 1 --2- 172.21.15.69:0/2701261366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34c71b0 0x55eab34c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.372+0000 7f4e10c00640 1 --2- 172.21.15.69:0/3769015918 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4df80a3ec0 0x7f4df80a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.372+0000 7f4e0aa00640 1 --2- 172.21.15.69:0/2701261366 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34d2390 0x55eab34d4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.373+0000 7f4e13400640 1 -- 172.21.15.69:0/2701261366 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34d2390 msgr2=0x55eab34d4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.373+0000 7f4e13400640 1 --2- 172.21.15.69:0/2701261366 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34d2390 0x55eab34d4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.373+0000 7f4e13400640 1 -- 172.21.15.69:0/2701261366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34c71b0 msgr2=0x55eab34c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.373+0000 7f4e13400640 1 --2- 172.21.15.69:0/2701261366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34c71b0 0x55eab34c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.373+0000 7f4e13400640 1 -- 172.21.15.69:0/2701261366 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eab33f8e10 con 0x55eab34c7ad0 2024-04-24T09:13:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.373+0000 7f4e0aa00640 1 --2- 172.21.15.69:0/2701261366 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34d2390 0x55eab34d4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.373+0000 7f4e10c00640 1 -- 172.21.15.69:0/3769015918 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4e040d1260 con 0x7f4e14074300 2024-04-24T09:13:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.373+0000 7f4e13e00640 1 --2- 172.21.15.69:0/3769015918 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4df80a3ec0 0x7f4df80a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.373+0000 7f4e13400640 1 --2- 172.21.15.69:0/2701261366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34d1e50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f4e040ce550 tx=0x7f4e040094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bac9cfcd43d435c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.374+0000 7f4e08c00640 1 -- 172.21.15.69:0/2701261366 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0409c9a0 con 0x55eab34c7ad0 2024-04-24T09:13:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.374+0000 7f4e08c00640 1 -- 172.21.15.69:0/2701261366 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e04011e00 con 0x55eab34c7ad0 2024-04-24T09:13:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.374+0000 7f4e08c00640 1 -- 172.21.15.69:0/2701261366 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e040d4e60 con 0x55eab34c7ad0 2024-04-24T09:13:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.375+0000 7f4e1d19b740 1 -- 172.21.15.69:0/2701261366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 msgr2=0x55eab34d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.375+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/2701261366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34d1e50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f4e040ce550 tx=0x7f4e040094a0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.375+0000 7f4e13e00640 1 --2- 172.21.15.69:0/3769015918 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4df80a3ec0 0x7f4df80a6380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e14101810 tx=0x7f4e0c0046c0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.375+0000 7f4e1d19b740 1 -- 172.21.15.69:0/2701261366 shutdown_connections 2024-04-24T09:13:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.375+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/2701261366 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34d2390 0x55eab34d4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.375+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/2701261366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34d1e50 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.375+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/2701261366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34c71b0 0x55eab34c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.375+0000 7f4e1d19b740 1 -- 172.21.15.69:0/2701261366 >> 172.21.15.69:0/2701261366 conn(0x55eab34c36c0 msgr2=0x55eab34c3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.375+0000 7f4e1d19b740 1 -- 172.21.15.69:0/2701261366 shutdown_connections 2024-04-24T09:13:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.375+0000 7f4e1d19b740 1 -- 172.21.15.69:0/2701261366 wait complete. 2024-04-24T09:13:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.376+0000 7f4e1d19b740 1 -- start start 2024-04-24T09:13:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.376+0000 7f4e1d19b740 1 Processor -- start 2024-04-24T09:13:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e1d19b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c71b0 0x55eab34cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e1d19b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34c7ad0 0x55eab34cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e1d19b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34d2390 0x55eab34cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e1d19b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55eab34d4f00 con 0x55eab34c71b0 2024-04-24T09:13:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e1d19b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55eab33e72c0 con 0x55eab34c7ad0 2024-04-24T09:13:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e1d19b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55eab34d5080 con 0x55eab34d2390 2024-04-24T09:13:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e1d19b740 10 client.0 _ll_register_callbacks cb 0x55eab313fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e13400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c71b0 0x55eab34cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e13400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c71b0 0x55eab34cf370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35984/0 (socket says 172.21.15.69:35984) 2024-04-24T09:13:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e13400640 1 -- 172.21.15.69:0/3173480544 learned_addr learned my addr 172.21.15.69:0/3173480544 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e13e00640 1 --2- 172.21.15.69:0/3173480544 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34d2390 0x55eab34cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e0aa00640 1 --2- 172.21.15.69:0/3173480544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34c7ad0 0x55eab34cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e13400640 1 -- 172.21.15.69:0/3173480544 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34d2390 msgr2=0x55eab34cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e13400640 1 --2- 172.21.15.69:0/3173480544 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34d2390 0x55eab34cfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e13400640 1 -- 172.21.15.69:0/3173480544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34c7ad0 msgr2=0x55eab34cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e13400640 1 --2- 172.21.15.69:0/3173480544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34c7ad0 0x55eab34cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e13400640 1 -- 172.21.15.69:0/3173480544 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eab33f7fd0 con 0x55eab34c71b0 2024-04-24T09:13:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.377+0000 7f4e0aa00640 1 --2- 172.21.15.69:0/3173480544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34c7ad0 0x55eab34cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.378+0000 7f4e13400640 1 --2- 172.21.15.69:0/3173480544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c71b0 0x55eab34cf370 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f4e0401e040 tx=0x7f4e040ceac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b32f81611b6948 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.378+0000 7f4e01600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:13:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.378+0000 7f4e01600640 1 -- 172.21.15.69:0/3173480544 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e040d4e60 con 0x55eab34c71b0 2024-04-24T09:13:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.378+0000 7f4e01600640 1 -- 172.21.15.69:0/3173480544 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4e04098d30 con 0x55eab34c71b0 2024-04-24T09:13:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.378+0000 7f4e1d19b740 1 -- 172.21.15.69:0/3173480544 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eab343b320 con 0x55eab34c71b0 2024-04-24T09:13:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.378+0000 7f4e01600640 1 -- 172.21.15.69:0/3173480544 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e0409c7c0 con 0x55eab34c71b0 2024-04-24T09:13:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.378+0000 7f4e1d19b740 10 client.16848 fetch_fsmap learned FSMap version 30 2024-04-24T09:13:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.378+0000 7f4e1d19b740 1 -- 172.21.15.69:0/3173480544 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eab3140210 con 0x55eab34c71b0 2024-04-24T09:13:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.379+0000 7f4e01600640 1 -- 172.21.15.69:0/3173480544 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55eab343b320 con 0x55eab34c71b0 2024-04-24T09:13:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.379+0000 7f4e01600640 1 -- 172.21.15.69:0/3173480544 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4e0400f040 con 0x55eab34c71b0 2024-04-24T09:13:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.379+0000 7f4e1d19b740 10 client.16848 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:13:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.379+0000 7f4e1d19b740 10 client.16848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.379+0000 7f4e1d19b740 20 client.16848 populate_metadata read hostname 'smithi069' 2024-04-24T09:13:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.379+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/3173480544 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55eab34e2260 0x55eab34e4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.379+0000 7f4e1d19b740 4 client.16848 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.379+0000 7f4e1d19b740 1 -- 172.21.15.69:0/3173480544 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eab3435190 con 0x55eab34e2260 2024-04-24T09:13:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.380+0000 7f4e0aa00640 1 --2- 172.21.15.69:0/3173480544 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55eab34e2260 0x55eab34e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.381+0000 7f4e0aa00640 1 --2- 172.21.15.69:0/3173480544 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55eab34e2260 0x55eab34e4650 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d50cc5f91f031000 server_cookie=5a58084929b16504 in_seq=0 out_seq=0 2024-04-24T09:13:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.381+0000 7f4e01600640 10 client.16848 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.383+0000 7f4e01600640 1 -- 172.21.15.69:0/3173480544 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eab3435190 con 0x55eab34e2260 2024-04-24T09:13:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.383+0000 7f4e01600640 10 client.16848 handle_command_reply: tid=0 2024-04-24T09:13:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.383+0000 7f4e1d19b740 1 client.16848 shutdown 2024-04-24T09:13:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.383+0000 7f4e1d19b740 1 -- 172.21.15.69:0/3173480544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c71b0 msgr2=0x55eab34cf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.383+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/3173480544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c71b0 0x55eab34cf370 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f4e0401e040 tx=0x7f4e040ceac0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.383+0000 7f4e1d19b740 1 -- 172.21.15.69:0/3173480544 shutdown_connections 2024-04-24T09:13:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.383+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/3173480544 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55eab34e2260 0x55eab34e4650 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.383+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/3173480544 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34d2390 0x55eab34cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.383+0000 7f4e01600640 0 client.16848 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.383+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/3173480544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34c7ad0 0x55eab34cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.383+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/3173480544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c71b0 0x55eab34cf370 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.383+0000 7f4e1d19b740 1 -- 172.21.15.69:0/3173480544 >> 172.21.15.69:0/3173480544 conn(0x55eab34c36c0 msgr2=0x55eab34c9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.383+0000 7f4e1d19b740 1 -- 172.21.15.69:0/3173480544 shutdown_connections 2024-04-24T09:13:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.383+0000 7f4e1d19b740 1 -- 172.21.15.69:0/3173480544 wait complete. 2024-04-24T09:13:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.383+0000 7f4e1d19b740 20 client.16848 trim_cache size 0 max 16384 2024-04-24T09:13:27.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.400+0000 7f4e1d19b740 1 Processor -- start 2024-04-24T09:13:27.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.400+0000 7f4e1d19b740 1 -- start start 2024-04-24T09:13:27.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.400+0000 7f4e1d19b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34c71b0 0x55eab34dddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.400+0000 7f4e1d19b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.401+0000 7f4e1d19b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34d2390 0x55eab34c26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.401+0000 7f4e1d19b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141b3d50 con 0x55eab34c7ad0 2024-04-24T09:13:27.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.401+0000 7f4e1d19b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141b15c0 con 0x55eab34d2390 2024-04-24T09:13:27.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.401+0000 7f4e1d19b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141af410 con 0x55eab34c71b0 2024-04-24T09:13:27.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.401+0000 7f4e0aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.401+0000 7f4e0aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35992/0 (socket says 172.21.15.69:35992) 2024-04-24T09:13:27.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.401+0000 7f4e0aa00640 1 -- 172.21.15.69:0/3431666050 learned_addr learned my addr 172.21.15.69:0/3431666050 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:27.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.401+0000 7f4e13400640 1 --2- 172.21.15.69:0/3431666050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34c71b0 0x55eab34dddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.401+0000 7f4e13e00640 1 --2- 172.21.15.69:0/3431666050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34d2390 0x55eab34c26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.401+0000 7f4e0aa00640 1 -- 172.21.15.69:0/3431666050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34c71b0 msgr2=0x55eab34dddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:27.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.401+0000 7f4e0aa00640 1 --2- 172.21.15.69:0/3431666050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34c71b0 0x55eab34dddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.401+0000 7f4e0aa00640 1 -- 172.21.15.69:0/3431666050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34d2390 msgr2=0x55eab34c26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:27.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.401+0000 7f4e0aa00640 1 --2- 172.21.15.69:0/3431666050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34d2390 0x55eab34c26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.401+0000 7f4e0aa00640 1 -- 172.21.15.69:0/3431666050 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eab320c620 con 0x55eab34c7ad0 2024-04-24T09:13:27.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.402+0000 7f4e0aa00640 1 --2- 172.21.15.69:0/3431666050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34c9b70 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc005ab0 tx=0x7f4dfc005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ca64f8323029a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.402+0000 7f4e03e00640 1 -- 172.21.15.69:0/3431666050 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc035040 con 0x55eab34c7ad0 2024-04-24T09:13:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.402+0000 7f4e03e00640 1 -- 172.21.15.69:0/3431666050 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dfc0054f0 con 0x55eab34c7ad0 2024-04-24T09:13:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.402+0000 7f4e03e00640 1 -- 172.21.15.69:0/3431666050 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc0046c0 con 0x55eab34c7ad0 2024-04-24T09:13:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.403+0000 7f4e1d19b740 1 -- 172.21.15.69:0/3431666050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 msgr2=0x55eab34c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.403+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/3431666050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34c9b70 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc005ab0 tx=0x7f4dfc005c00 comp rx=0 tx=0).stop 2024-04-24T09:13:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.403+0000 7f4e1d19b740 1 -- 172.21.15.69:0/3431666050 shutdown_connections 2024-04-24T09:13:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.403+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/3431666050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34d2390 0x55eab34c26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.403+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/3431666050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34c9b70 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.403+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/3431666050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34c71b0 0x55eab34dddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.403+0000 7f4e1d19b740 1 -- 172.21.15.69:0/3431666050 >> 172.21.15.69:0/3431666050 conn(0x55eab34c36c0 msgr2=0x55eab34c1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.403+0000 7f4e1d19b740 1 -- 172.21.15.69:0/3431666050 shutdown_connections 2024-04-24T09:13:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.403+0000 7f4e1d19b740 1 -- 172.21.15.69:0/3431666050 wait complete. 2024-04-24T09:13:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.403+0000 7f4e1d19b740 1 -- start start 2024-04-24T09:13:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.404+0000 7f4e1d19b740 1 Processor -- start 2024-04-24T09:13:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.404+0000 7f4e1d19b740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34c71b0 0x55eab34cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.404+0000 7f4e1d19b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.404+0000 7f4e1d19b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34d2390 0x55eab34ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.404+0000 7f4e1d19b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141afa80 con 0x55eab34c7ad0 2024-04-24T09:13:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.404+0000 7f4e1d19b740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141b0ef0 con 0x55eab34c71b0 2024-04-24T09:13:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.404+0000 7f4e1d19b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141b0150 con 0x55eab34d2390 2024-04-24T09:13:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.404+0000 7f4e1d19b740 10 client.0 _ll_register_callbacks cb 0x7f4e04098fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.405+0000 7f4e0aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.405+0000 7f4e0aa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36004/0 (socket says 172.21.15.69:36004) 2024-04-24T09:13:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.405+0000 7f4e0aa00640 1 -- 172.21.15.69:0/1712687994 learned_addr learned my addr 172.21.15.69:0/1712687994 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.405+0000 7f4e13e00640 1 --2- 172.21.15.69:0/1712687994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34d2390 0x55eab34ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.405+0000 7f4e13400640 1 --2- 172.21.15.69:0/1712687994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34c71b0 0x55eab34cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.405+0000 7f4e0aa00640 1 -- 172.21.15.69:0/1712687994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34d2390 msgr2=0x55eab34ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.405+0000 7f4e0aa00640 1 --2- 172.21.15.69:0/1712687994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34d2390 0x55eab34ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.405+0000 7f4e0aa00640 1 -- 172.21.15.69:0/1712687994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34c71b0 msgr2=0x55eab34cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.405+0000 7f4e0aa00640 1 --2- 172.21.15.69:0/1712687994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34c71b0 0x55eab34cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.405+0000 7f4e0aa00640 1 -- 172.21.15.69:0/1712687994 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eab3547770 con 0x55eab34c7ad0 2024-04-24T09:13:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.405+0000 7f4e0aa00640 1 --2- 172.21.15.69:0/1712687994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34cc7d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc02a9a0 tx=0x7f4dfc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c54bd89ed7dbcc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.405+0000 7f4e01600640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:13:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.406+0000 7f4e01600640 1 -- 172.21.15.69:0/1712687994 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc035040 con 0x55eab34c7ad0 2024-04-24T09:13:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.406+0000 7f4e01600640 1 -- 172.21.15.69:0/1712687994 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4dfc040520 con 0x55eab34c7ad0 2024-04-24T09:13:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.406+0000 7f4e1d19b740 1 -- 172.21.15.69:0/1712687994 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dfc0054f0 con 0x55eab34c7ad0 2024-04-24T09:13:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.406+0000 7f4e01600640 1 -- 172.21.15.69:0/1712687994 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4dfc059050 con 0x55eab34c7ad0 2024-04-24T09:13:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.406+0000 7f4e1d19b740 10 client.16854 fetch_fsmap learned FSMap version 30 2024-04-24T09:13:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.406+0000 7f4e1d19b740 1 -- 172.21.15.69:0/1712687994 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eab3548a60 con 0x55eab34c7ad0 2024-04-24T09:13:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.406+0000 7f4e01600640 1 -- 172.21.15.69:0/1712687994 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dfc0054f0 con 0x55eab34c7ad0 2024-04-24T09:13:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.407+0000 7f4e01600640 1 -- 172.21.15.69:0/1712687994 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4dfc04a050 con 0x55eab34c7ad0 2024-04-24T09:13:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.407+0000 7f4e1d19b740 10 client.16854 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:13:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.407+0000 7f4e1d19b740 10 client.16854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.407+0000 7f4e1d19b740 20 client.16854 populate_metadata read hostname 'smithi069' 2024-04-24T09:13:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.407+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/1712687994 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55eab35491d0 0x55eab354b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.407+0000 7f4e1d19b740 4 client.16854 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:13:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.407+0000 7f4e1d19b740 1 -- 172.21.15.69:0/1712687994 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eab33b14b0 con 0x55eab35491d0 2024-04-24T09:13:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.407+0000 7f4e13400640 1 --2- 172.21.15.69:0/1712687994 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55eab35491d0 0x55eab354b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.408+0000 7f4e13400640 1 --2- 172.21.15.69:0/1712687994 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55eab35491d0 0x55eab354b5f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84fa893fa4a97ecd server_cookie=fc6dec0c3794762e in_seq=0 out_seq=0 2024-04-24T09:13:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.408+0000 7f4e01600640 10 client.16854 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:27.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:27 smithi130 ceph-mon[31553]: pgmap v194: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 645 KiB/s wr, 101 op/s 2024-04-24T09:13:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.489+0000 7f4e01600640 1 -- 172.21.15.69:0/1712687994 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eab33b14b0 con 0x55eab35491d0 2024-04-24T09:13:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.489+0000 7f4e01600640 10 client.16854 handle_command_reply: tid=0 2024-04-24T09:13:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.489+0000 7f4e1d19b740 1 client.16854 shutdown 2024-04-24T09:13:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.489+0000 7f4e1d19b740 1 -- 172.21.15.69:0/1712687994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 msgr2=0x55eab34cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.489+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/1712687994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34cc7d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc02a9a0 tx=0x7f4dfc005e80 comp rx=0 tx=0).stop 2024-04-24T09:13:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.489+0000 7f4e1d19b740 1 -- 172.21.15.69:0/1712687994 shutdown_connections 2024-04-24T09:13:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.489+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/1712687994 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55eab35491d0 0x55eab354b5f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.489+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/1712687994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eab34d2390 0x55eab34ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.489+0000 7f4e01600640 0 client.16854 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.489+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/1712687994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eab34c7ad0 0x55eab34cc7d0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.489+0000 7f4e1d19b740 1 --2- 172.21.15.69:0/1712687994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eab34c71b0 0x55eab34cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.489+0000 7f4e1d19b740 1 -- 172.21.15.69:0/1712687994 >> 172.21.15.69:0/1712687994 conn(0x55eab34c36c0 msgr2=0x55eab34cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.490+0000 7f4e1d19b740 1 -- 172.21.15.69:0/1712687994 shutdown_connections 2024-04-24T09:13:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.490+0000 7f4e1d19b740 1 -- 172.21.15.69:0/1712687994 wait complete. 2024-04-24T09:13:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.490+0000 7f4e1d19b740 20 client.16854 trim_cache size 0 max 16384 2024-04-24T09:13:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:13:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:13:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:13:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:13:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.490+0000 7f4e1a400640 1 -- 172.21.15.69:0/3769015918 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4df80a3ec0 msgr2=0x7f4df80a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.490+0000 7f4e1a400640 1 --2- 172.21.15.69:0/3769015918 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4df80a3ec0 0x7f4df80a6380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e14101810 tx=0x7f4e0c0046c0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.491+0000 7f4e1a400640 1 -- 172.21.15.69:0/3769015918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e14074300 msgr2=0x7f4e141a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.491+0000 7f4e1a400640 1 --2- 172.21.15.69:0/3769015918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e14074300 0x7f4e141a0620 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400eab0 tx=0x7f4e0400ef70 comp rx=0 tx=0).stop 2024-04-24T09:13:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.491+0000 7f4e1a400640 1 -- 172.21.15.69:0/3769015918 shutdown_connections 2024-04-24T09:13:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.491+0000 7f4e1a400640 1 --2- 172.21.15.69:0/3769015918 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4df80a3ec0 0x7f4df80a6380 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.491+0000 7f4e1a400640 1 --2- 172.21.15.69:0/3769015918 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e1410e8f0 0x7f4e1419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.491+0000 7f4e1a400640 1 --2- 172.21.15.69:0/3769015918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e14074c20 0x7f4e141a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.491+0000 7f4e1a400640 1 --2- 172.21.15.69:0/3769015918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e14074300 0x7f4e141a0620 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.491+0000 7f4e1a400640 1 -- 172.21.15.69:0/3769015918 >> 172.21.15.69:0/3769015918 conn(0x7f4e14079490 msgr2=0x7f4e140feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.491+0000 7f4e1a400640 1 -- 172.21.15.69:0/3769015918 shutdown_connections 2024-04-24T09:13:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:27.491+0000 7f4e1a400640 1 -- 172.21.15.69:0/3769015918 wait complete. 2024-04-24T09:13:27.505 INFO:tasks.workunit.client.0.smithi069.stdout:checking gdbm-ndbm.h usability... no 2024-04-24T09:13:27.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:27 smithi081 ceph-mon[30961]: pgmap v194: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 645 KiB/s wr, 101 op/s 2024-04-24T09:13:27.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:27 smithi069 ceph-mon[28170]: pgmap v194: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 645 KiB/s wr, 101 op/s 2024-04-24T09:13:27.591 INFO:tasks.workunit.client.0.smithi069.stdout:checking gdbm-ndbm.h presence... no 2024-04-24T09:13:27.592 INFO:tasks.workunit.client.0.smithi069.stdout:checking for gdbm-ndbm.h... no 2024-04-24T09:13:27.743 INFO:tasks.workunit.client.0.smithi069.stdout:checking gdbm/ndbm.h usability... no 2024-04-24T09:13:27.886 INFO:tasks.workunit.client.0.smithi069.stdout:checking gdbm/ndbm.h presence... no 2024-04-24T09:13:27.888 INFO:tasks.workunit.client.0.smithi069.stdout:checking for gdbm/ndbm.h... no 2024-04-24T09:13:28.067 INFO:tasks.workunit.client.0.smithi069.stdout:checking ndbm.h usability... no 2024-04-24T09:13:28.215 INFO:tasks.workunit.client.0.smithi069.stdout:checking ndbm.h presence... no 2024-04-24T09:13:28.217 INFO:tasks.workunit.client.0.smithi069.stdout:checking for ndbm.h... no 2024-04-24T09:13:28.354 INFO:tasks.workunit.client.0.smithi069.stdout:checking libaio.h usability... yes 2024-04-24T09:13:28.437 INFO:tasks.workunit.client.0.smithi069.stdout:checking libaio.h presence... yes 2024-04-24T09:13:28.439 INFO:tasks.workunit.client.0.smithi069.stdout:checking for libaio.h... yes 2024-04-24T09:13:28.673 INFO:tasks.workunit.client.0.smithi069.stdout:checking for dm_make_handle in -ldm... no 2024-04-24T09:13:28.673 INFO:tasks.workunit.client.0.smithi069.stdout: 2024-04-24T09:13:28.673 INFO:tasks.workunit.client.0.smithi069.stdout:WARNING: could not find a valid DMAPI base library. 2024-04-24T09:13:28.673 INFO:tasks.workunit.client.0.smithi069.stdout:If you want DMAPI support please install the data migration 2024-04-24T09:13:28.673 INFO:tasks.workunit.client.0.smithi069.stdout:API (dmapi) library package. Alternatively, run "make install" 2024-04-24T09:13:28.674 INFO:tasks.workunit.client.0.smithi069.stdout:from the dmapi source. 2024-04-24T09:13:28.674 INFO:tasks.workunit.client.0.smithi069.stdout: 2024-04-24T09:13:28.781 INFO:tasks.workunit.client.0.smithi069.stdout:checking linux/fiemap.h usability... yes 2024-04-24T09:13:28.854 INFO:tasks.workunit.client.0.smithi069.stdout:checking linux/fiemap.h presence... yes 2024-04-24T09:13:28.856 INFO:tasks.workunit.client.0.smithi069.stdout:checking for linux/fiemap.h... yes 2024-04-24T09:13:29.095 INFO:tasks.workunit.client.0.smithi069.stdout:checking for fallocate... yes 2024-04-24T09:13:29.353 INFO:tasks.workunit.client.0.smithi069.stdout:checking for open_by_handle_at... yes 2024-04-24T09:13:29.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:29 smithi130 ceph-mon[31553]: pgmap v195: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 345 KiB/s wr, 53 op/s 2024-04-24T09:13:29.539 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/prctl.h usability... yes 2024-04-24T09:13:29.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:29 smithi081 ceph-mon[30961]: pgmap v195: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 345 KiB/s wr, 53 op/s 2024-04-24T09:13:29.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:29 smithi069 ceph-mon[28170]: pgmap v195: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 345 KiB/s wr, 53 op/s 2024-04-24T09:13:29.623 INFO:tasks.workunit.client.0.smithi069.stdout:checking sys/prctl.h presence... yes 2024-04-24T09:13:29.625 INFO:tasks.workunit.client.0.smithi069.stdout:checking for sys/prctl.h... yes 2024-04-24T09:13:29.773 INFO:tasks.workunit.client.0.smithi069.stdout:checking linux/fs.h usability... yes 2024-04-24T09:13:29.855 INFO:tasks.workunit.client.0.smithi069.stdout:checking linux/fs.h presence... yes 2024-04-24T09:13:29.856 INFO:tasks.workunit.client.0.smithi069.stdout:checking for linux/fs.h... yes 2024-04-24T09:13:30.051 INFO:tasks.workunit.client.0.smithi069.stdout:checking btrfsutil.h usability... no 2024-04-24T09:13:30.199 INFO:tasks.workunit.client.0.smithi069.stdout:checking btrfsutil.h presence... no 2024-04-24T09:13:30.201 INFO:tasks.workunit.client.0.smithi069.stdout:checking for btrfsutil.h... no 2024-04-24T09:13:30.610 INFO:tasks.workunit.client.0.smithi069.stdout:checking for copy_file_range... yes 2024-04-24T09:13:30.856 INFO:tasks.workunit.client.0.smithi069.stdout:checking for renameat2... yes 2024-04-24T09:13:30.926 INFO:tasks.workunit.client.0.smithi069.stdout:configure: creating ./config.status 2024-04-24T09:13:31.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:31 smithi130 ceph-mon[31553]: pgmap v196: 129 pgs: 129 active+clean; 116 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 534 KiB/s wr, 105 op/s 2024-04-24T09:13:31.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:31 smithi081 ceph-mon[30961]: pgmap v196: 129 pgs: 129 active+clean; 116 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 534 KiB/s wr, 105 op/s 2024-04-24T09:13:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:31 smithi069 ceph-mon[28170]: pgmap v196: 129 pgs: 129 active+clean; 116 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 534 KiB/s wr, 105 op/s 2024-04-24T09:13:32.322 INFO:tasks.workunit.client.0.smithi069.stdout:config.status: creating include/builddefs 2024-04-24T09:13:32.487 INFO:tasks.workunit.client.0.smithi069.stdout:config.status: creating include/config.h 2024-04-24T09:13:33.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:33 smithi130 ceph-mon[31553]: pgmap v197: 129 pgs: 129 active+clean; 116 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 366 KiB/s wr, 79 op/s 2024-04-24T09:13:33.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:33 smithi081 ceph-mon[30961]: pgmap v197: 129 pgs: 129 active+clean; 116 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 366 KiB/s wr, 79 op/s 2024-04-24T09:13:33.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:33 smithi069 ceph-mon[28170]: pgmap v197: 129 pgs: 129 active+clean; 116 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 366 KiB/s wr, 79 op/s 2024-04-24T09:13:35.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:35 smithi130 ceph-mon[31553]: pgmap v198: 129 pgs: 129 active+clean; 116 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 366 KiB/s wr, 79 op/s 2024-04-24T09:13:35.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:35 smithi081 ceph-mon[30961]: pgmap v198: 129 pgs: 129 active+clean; 116 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 366 KiB/s wr, 79 op/s 2024-04-24T09:13:35.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:35 smithi069 ceph-mon[28170]: pgmap v198: 129 pgs: 129 active+clean; 116 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 366 KiB/s wr, 79 op/s 2024-04-24T09:13:37.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:37 smithi130 ceph-mon[31553]: pgmap v199: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 447 KiB/s wr, 97 op/s 2024-04-24T09:13:37.557 INFO:tasks.workunit.client.0.smithi069.stdout:config.status: executing libtool commands 2024-04-24T09:13:37.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:37 smithi081 ceph-mon[30961]: pgmap v199: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 447 KiB/s wr, 97 op/s 2024-04-24T09:13:37.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:37 smithi069 ceph-mon[28170]: pgmap v199: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 447 KiB/s wr, 97 op/s 2024-04-24T09:13:38.329 INFO:tasks.workunit.client.0.smithi069.stdout:Building include 2024-04-24T09:13:38.329 INFO:tasks.workunit.client.0.smithi069.stdout:Building lib 2024-04-24T09:13:38.330 INFO:tasks.workunit.client.0.smithi069.stdout:Building tests 2024-04-24T09:13:38.389 INFO:tasks.workunit.client.0.smithi069.stdout: [LTDEP] 2024-04-24T09:13:39.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:39 smithi130 ceph-mon[31553]: pgmap v200: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 270 KiB/s wr, 70 op/s 2024-04-24T09:13:39.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:39 smithi081 ceph-mon[30961]: pgmap v200: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 270 KiB/s wr, 70 op/s 2024-04-24T09:13:39.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:39 smithi069 ceph-mon[28170]: pgmap v200: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 270 KiB/s wr, 70 op/s 2024-04-24T09:13:40.087 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] dataascii.lo 2024-04-24T09:13:40.090 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] databin.lo 2024-04-24T09:13:40.090 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] datapid.lo 2024-04-24T09:13:40.092 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] file_lock.lo 2024-04-24T09:13:40.467 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] forker.lo 2024-04-24T09:13:40.530 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] pattern.lo 2024-04-24T09:13:40.607 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] open_flags.lo 2024-04-24T09:13:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.658+0000 7f6984c00640 1 -- 172.21.15.69:0/3971487115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6980106480 msgr2=0x7f6980106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.658+0000 7f6984c00640 1 --2- 172.21.15.69:0/3971487115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6980106480 0x7f6980106930 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f6978009cd0 tx=0x7f697802a970 comp rx=0 tx=0).stop 2024-04-24T09:13:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.658+0000 7f6984c00640 1 -- 172.21.15.69:0/3971487115 shutdown_connections 2024-04-24T09:13:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.658+0000 7f6984c00640 1 --2- 172.21.15.69:0/3971487115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6980106480 0x7f6980106930 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.658+0000 7f6984c00640 1 --2- 172.21.15.69:0/3971487115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69801019a0 0x7f6980105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.658+0000 7f6984c00640 1 --2- 172.21.15.69:0/3971487115 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6980101080 0x7f6980101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.658+0000 7f6984c00640 1 -- 172.21.15.69:0/3971487115 >> 172.21.15.69:0/3971487115 conn(0x7f6980078010 msgr2=0x7f6980078420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.659+0000 7f6984c00640 1 -- 172.21.15.69:0/3971487115 shutdown_connections 2024-04-24T09:13:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.659+0000 7f6984c00640 1 -- 172.21.15.69:0/3971487115 wait complete. 2024-04-24T09:13:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.660+0000 7f6984c00640 1 Processor -- start 2024-04-24T09:13:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.660+0000 7f6984c00640 1 -- start start 2024-04-24T09:13:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.661+0000 7f6984c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6980101080 0x7f698019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.661+0000 7f6984c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f69801019a0 0x7f698019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.661+0000 7f6984c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6980106480 0x7f69801a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.661+0000 7f6984c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6980118910 con 0x7f6980101080 2024-04-24T09:13:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.661+0000 7f6984c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6980118790 con 0x7f6980106480 2024-04-24T09:13:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.661+0000 7f6984c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6980118a90 con 0x7f69801019a0 2024-04-24T09:13:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.661+0000 7f697de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6980101080 0x7f698019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.661+0000 7f697de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6980101080 0x7f698019edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36604/0 (socket says 172.21.15.69:36604) 2024-04-24T09:13:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.661+0000 7f697de00640 1 -- 172.21.15.69:0/3986120525 learned_addr learned my addr 172.21.15.69:0/3986120525 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.661+0000 7f697d400640 1 --2- 172.21.15.69:0/3986120525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f69801019a0 0x7f698019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.662+0000 7f697e800640 1 --2- 172.21.15.69:0/3986120525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6980106480 0x7f69801a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.662+0000 7f697de00640 1 -- 172.21.15.69:0/3986120525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f69801019a0 msgr2=0x7f698019f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.662+0000 7f697de00640 1 --2- 172.21.15.69:0/3986120525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f69801019a0 0x7f698019f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.662+0000 7f697de00640 1 -- 172.21.15.69:0/3986120525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6980106480 msgr2=0x7f69801a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.662+0000 7f697de00640 1 --2- 172.21.15.69:0/3986120525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6980106480 0x7f69801a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.662+0000 7f697de00640 1 -- 172.21.15.69:0/3986120525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69801a3d70 con 0x7f6980101080 2024-04-24T09:13:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.663+0000 7f697de00640 1 --2- 172.21.15.69:0/3986120525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6980101080 0x7f698019edd0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f697000e9f0 tx=0x7f697000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.664+0000 7f6976a00640 1 -- 172.21.15.69:0/3986120525 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697000cd80 con 0x7f6980101080 2024-04-24T09:13:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.664+0000 7f6976a00640 1 -- 172.21.15.69:0/3986120525 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69700054d0 con 0x7f6980101080 2024-04-24T09:13:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.664+0000 7f6976a00640 1 -- 172.21.15.69:0/3986120525 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970010490 con 0x7f6980101080 2024-04-24T09:13:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.664+0000 7f6984c00640 1 -- 172.21.15.69:0/3986120525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69801a4060 con 0x7f6980101080 2024-04-24T09:13:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.664+0000 7f6984c00640 1 -- 172.21.15.69:0/3986120525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69801ab8f0 con 0x7f6980101080 2024-04-24T09:13:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.668+0000 7f69879a5740 1 Processor -- start 2024-04-24T09:13:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.668+0000 7f69879a5740 1 -- start start 2024-04-24T09:13:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.668+0000 7f69879a5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e4110501b0 0x55e411050590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.668+0000 7f69879a5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e411050ad0 0x55e41105ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.668+0000 7f69879a5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e41105b390 0x55e41105d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.668+0000 7f69879a5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e410fb6900 con 0x55e41105b390 2024-04-24T09:13:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.668+0000 7f69879a5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e410f5cf20 con 0x55e411050ad0 2024-04-24T09:13:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.668+0000 7f69879a5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55e410fb28c0 con 0x55e4110501b0 2024-04-24T09:13:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.669+0000 7f6976a00640 1 -- 172.21.15.69:0/3986120525 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6970005020 con 0x7f6980101080 2024-04-24T09:13:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.669+0000 7f697d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e41105b390 0x55e41105d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.669+0000 7f697d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e41105b390 0x55e41105d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36618/0 (socket says 172.21.15.69:36618) 2024-04-24T09:13:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.669+0000 7f697d400640 1 -- 172.21.15.69:0/2349854441 learned_addr learned my addr 172.21.15.69:0/2349854441 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.669+0000 7f697e800640 1 --2- 172.21.15.69:0/2349854441 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e4110501b0 0x55e411050590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.670+0000 7f697d400640 1 -- 172.21.15.69:0/2349854441 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e4110501b0 msgr2=0x55e411050590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.670+0000 7f697d400640 1 --2- 172.21.15.69:0/2349854441 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e4110501b0 0x55e411050590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.670+0000 7f697d400640 1 -- 172.21.15.69:0/2349854441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e411050ad0 msgr2=0x55e41105ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.670+0000 7f697de00640 1 --2- 172.21.15.69:0/2349854441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e411050ad0 0x55e41105ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.670+0000 7f697d400640 1 --2- 172.21.15.69:0/2349854441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e411050ad0 0x55e41105ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.670+0000 7f697d400640 1 -- 172.21.15.69:0/2349854441 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e410f81e10 con 0x55e41105b390 2024-04-24T09:13:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.670+0000 7f697d400640 1 --2- 172.21.15.69:0/2349854441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e41105b390 0x55e41105d780 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f696c009c10 tx=0x7f696c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2170540525777e0b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.671+0000 7f696b000640 1 -- 172.21.15.69:0/2349854441 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c004230 con 0x55e41105b390 2024-04-24T09:13:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.671+0000 7f6976a00640 1 --2- 172.21.15.69:0/3986120525 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69640a3fa0 0x7f69640a6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.671+0000 7f696b000640 1 -- 172.21.15.69:0/2349854441 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f696c0043d0 con 0x55e41105b390 2024-04-24T09:13:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.671+0000 7f6976a00640 1 -- 172.21.15.69:0/3986120525 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f69700d0e00 con 0x7f6980101080 2024-04-24T09:13:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.672+0000 7f69879a5740 1 -- 172.21.15.69:0/2349854441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e41105b390 msgr2=0x55e41105d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.672+0000 7f69879a5740 1 --2- 172.21.15.69:0/2349854441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e41105b390 0x55e41105d780 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f696c009c10 tx=0x7f696c011810 comp rx=0 tx=0).stop 2024-04-24T09:13:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.672+0000 7f696b000640 1 -- 172.21.15.69:0/2349854441 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c004d80 con 0x55e41105b390 2024-04-24T09:13:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.673+0000 7f697e800640 1 --2- 172.21.15.69:0/3986120525 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69640a3fa0 0x7f69640a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.674+0000 7f697e800640 1 --2- 172.21.15.69:0/3986120525 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69640a3fa0 0x7f69640a6460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6978004af0 tx=0x7f697803c040 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.675+0000 7f69879a5740 1 -- 172.21.15.69:0/2349854441 shutdown_connections 2024-04-24T09:13:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.675+0000 7f69879a5740 1 --2- 172.21.15.69:0/2349854441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e41105b390 0x55e41105d780 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.675+0000 7f69879a5740 1 --2- 172.21.15.69:0/2349854441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e411050ad0 0x55e41105ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.675+0000 7f69879a5740 1 --2- 172.21.15.69:0/2349854441 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e4110501b0 0x55e411050590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.675+0000 7f69879a5740 1 -- 172.21.15.69:0/2349854441 >> 172.21.15.69:0/2349854441 conn(0x55e41104c6c0 msgr2=0x55e41104caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:40.675 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] random_range.lo 2024-04-24T09:13:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.675+0000 7f69879a5740 1 -- 172.21.15.69:0/2349854441 shutdown_connections 2024-04-24T09:13:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.675+0000 7f69879a5740 1 -- 172.21.15.69:0/2349854441 wait complete. 2024-04-24T09:13:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.676+0000 7f69879a5740 1 -- start start 2024-04-24T09:13:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.677+0000 7f69879a5740 1 Processor -- start 2024-04-24T09:13:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.677+0000 7f69879a5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4110501b0 0x55e4110582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.677+0000 7f69879a5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e411050ad0 0x55e411058820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.678+0000 7f69879a5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 0x55e411058d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.678+0000 7f69879a5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e41105dfb0 con 0x55e411050ad0 2024-04-24T09:13:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.678+0000 7f69879a5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e410f702c0 con 0x55e4110501b0 2024-04-24T09:13:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.678+0000 7f69879a5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55e41105e130 con 0x55e41105b390 2024-04-24T09:13:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.678+0000 7f69879a5740 10 client.0 _ll_register_callbacks cb 0x55e410cc8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.678+0000 7f697d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e411050ad0 0x55e411058820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.678+0000 7f697d400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e411050ad0 0x55e411058820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36632/0 (socket says 172.21.15.69:36632) 2024-04-24T09:13:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.678+0000 7f697d400640 1 -- 172.21.15.69:0/4007858773 learned_addr learned my addr 172.21.15.69:0/4007858773 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.678+0000 7f697de00640 1 --2- 172.21.15.69:0/4007858773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4110501b0 0x55e4110582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.678+0000 7f697d400640 1 -- 172.21.15.69:0/4007858773 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 msgr2=0x55e411058d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.678+0000 7f697e800640 1 --2- 172.21.15.69:0/4007858773 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 0x55e411058d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.679+0000 7f697d400640 1 --2- 172.21.15.69:0/4007858773 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 0x55e411058d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.679+0000 7f697d400640 1 -- 172.21.15.69:0/4007858773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4110501b0 msgr2=0x55e4110582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.679+0000 7f697d400640 1 --2- 172.21.15.69:0/4007858773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4110501b0 0x55e4110582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.679+0000 7f697d400640 1 -- 172.21.15.69:0/4007858773 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e410f80fd0 con 0x55e411050ad0 2024-04-24T09:13:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.680+0000 7f697e800640 1 --2- 172.21.15.69:0/4007858773 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 0x55e411058d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.680+0000 7f697d400640 1 --2- 172.21.15.69:0/4007858773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e411050ad0 0x55e411058820 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f696c000c00 tx=0x7f696c019a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec51d7b63116b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.680+0000 7f695f400640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:13:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.679+0000 7f697de00640 1 --2- 172.21.15.69:0/4007858773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4110501b0 0x55e4110582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.681+0000 7f695f400640 1 -- 172.21.15.69:0/4007858773 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c028020 con 0x55e411050ad0 2024-04-24T09:13:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.681+0000 7f695f400640 1 -- 172.21.15.69:0/4007858773 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f696c024420 con 0x55e411050ad0 2024-04-24T09:13:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.681+0000 7f69879a5740 1 -- 172.21.15.69:0/4007858773 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f696c0043d0 con 0x55e411050ad0 2024-04-24T09:13:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.682+0000 7f695f400640 1 -- 172.21.15.69:0/4007858773 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f696c009850 con 0x55e411050ad0 2024-04-24T09:13:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.683+0000 7f69879a5740 10 client.16866 fetch_fsmap learned FSMap version 30 2024-04-24T09:13:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.683+0000 7f69879a5740 1 -- 172.21.15.69:0/4007858773 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e410cc9210 con 0x55e411050ad0 2024-04-24T09:13:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.685+0000 7f695f400640 1 -- 172.21.15.69:0/4007858773 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f696c0043d0 con 0x55e411050ad0 2024-04-24T09:13:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.685+0000 7f695f400640 1 -- 172.21.15.69:0/4007858773 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f696c018070 con 0x55e411050ad0 2024-04-24T09:13:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.685+0000 7f69879a5740 10 client.16866 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:13:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.685+0000 7f69879a5740 10 client.16866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.685+0000 7f69879a5740 20 client.16866 populate_metadata read hostname 'smithi069' 2024-04-24T09:13:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.685+0000 7f69879a5740 1 --2- 172.21.15.69:0/4007858773 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e41106b490 0x55e41106d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.685+0000 7f69879a5740 4 client.16866 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.685+0000 7f69879a5740 1 -- 172.21.15.69:0/4007858773 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e410fc4320 con 0x55e41106b490 2024-04-24T09:13:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.686+0000 7f697de00640 1 --2- 172.21.15.69:0/4007858773 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e41106b490 0x55e41106d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.688+0000 7f697de00640 1 --2- 172.21.15.69:0/4007858773 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e41106b490 0x55e41106d880 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20ef2ada68109618 server_cookie=9363de5335a5615e in_seq=0 out_seq=0 2024-04-24T09:13:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.688+0000 7f695f400640 10 client.16866 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.691+0000 7f695f400640 1 -- 172.21.15.69:0/4007858773 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e410fc4320 con 0x55e41106b490 2024-04-24T09:13:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.691+0000 7f695f400640 10 client.16866 handle_command_reply: tid=0 2024-04-24T09:13:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.691+0000 7f69879a5740 1 client.16866 shutdown 2024-04-24T09:13:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.692+0000 7f69879a5740 1 -- 172.21.15.69:0/4007858773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e411050ad0 msgr2=0x55e411058820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.692+0000 7f69879a5740 1 --2- 172.21.15.69:0/4007858773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e411050ad0 0x55e411058820 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f696c000c00 tx=0x7f696c019a50 comp rx=0 tx=0).stop 2024-04-24T09:13:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.692+0000 7f69879a5740 1 -- 172.21.15.69:0/4007858773 shutdown_connections 2024-04-24T09:13:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.692+0000 7f69879a5740 1 --2- 172.21.15.69:0/4007858773 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e41106b490 0x55e41106d880 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.692+0000 7f695f400640 0 client.16866 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.692+0000 7f69879a5740 1 --2- 172.21.15.69:0/4007858773 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 0x55e411058d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.692+0000 7f69879a5740 1 --2- 172.21.15.69:0/4007858773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e411050ad0 0x55e411058820 secure :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f696c000c00 tx=0x7f696c019a50 comp rx=0 tx=0).stop 2024-04-24T09:13:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.692+0000 7f69879a5740 1 --2- 172.21.15.69:0/4007858773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4110501b0 0x55e4110582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.692+0000 7f69879a5740 1 -- 172.21.15.69:0/4007858773 >> 172.21.15.69:0/4007858773 conn(0x55e41104c6c0 msgr2=0x55e4110523f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.692+0000 7f69879a5740 1 -- 172.21.15.69:0/4007858773 shutdown_connections 2024-04-24T09:13:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.693+0000 7f69879a5740 1 -- 172.21.15.69:0/4007858773 wait complete. 2024-04-24T09:13:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.693+0000 7f69879a5740 20 client.16866 trim_cache size 0 max 16384 2024-04-24T09:13:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.717+0000 7f69879a5740 1 Processor -- start 2024-04-24T09:13:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.717+0000 7f69879a5740 1 -- start start 2024-04-24T09:13:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.718+0000 7f69879a5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4110501b0 0x55e411066ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.718+0000 7f69879a5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 0x55e411058b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.718+0000 7f69879a5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e41106b490 0x55e4110730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.718+0000 7f69879a5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b8200 con 0x55e4110501b0 2024-04-24T09:13:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.718+0000 7f69879a5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b5a70 con 0x55e41106b490 2024-04-24T09:13:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.718+0000 7f69879a5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b38c0 con 0x55e41105b390 2024-04-24T09:13:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.718+0000 7f697de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4110501b0 0x55e411066ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.718+0000 7f697de00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4110501b0 0x55e411066ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36648/0 (socket says 172.21.15.69:36648) 2024-04-24T09:13:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.718+0000 7f697de00640 1 -- 172.21.15.69:0/940227760 learned_addr learned my addr 172.21.15.69:0/940227760 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.719+0000 7f697e800640 1 --2- 172.21.15.69:0/940227760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e41106b490 0x55e4110730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.719+0000 7f697d400640 1 --2- 172.21.15.69:0/940227760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 0x55e411058b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.719+0000 7f697de00640 1 -- 172.21.15.69:0/940227760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 msgr2=0x55e411058b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.719+0000 7f697de00640 1 --2- 172.21.15.69:0/940227760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 0x55e411058b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.719+0000 7f697de00640 1 -- 172.21.15.69:0/940227760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e41106b490 msgr2=0x55e4110730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.719+0000 7f697de00640 1 --2- 172.21.15.69:0/940227760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e41106b490 0x55e4110730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.719+0000 7f697de00640 1 -- 172.21.15.69:0/940227760 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e411050900 con 0x55e4110501b0 2024-04-24T09:13:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.719+0000 7f697d400640 1 --2- 172.21.15.69:0/940227760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 0x55e411058b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.719+0000 7f697e800640 1 --2- 172.21.15.69:0/940227760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e41106b490 0x55e4110730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.720+0000 7f697de00640 1 --2- 172.21.15.69:0/940227760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4110501b0 0x55e411066ec0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f697009c700 tx=0x7f69700d3250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d410d7836b2a0087 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.720+0000 7f696a600640 1 -- 172.21.15.69:0/940227760 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970005680 con 0x55e4110501b0 2024-04-24T09:13:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.720+0000 7f696a600640 1 -- 172.21.15.69:0/940227760 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69700b7020 con 0x55e4110501b0 2024-04-24T09:13:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.720+0000 7f696a600640 1 -- 172.21.15.69:0/940227760 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697009ad00 con 0x55e4110501b0 2024-04-24T09:13:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.721+0000 7f69879a5740 1 -- 172.21.15.69:0/940227760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4110501b0 msgr2=0x55e411066ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.721+0000 7f69879a5740 1 --2- 172.21.15.69:0/940227760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4110501b0 0x55e411066ec0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f697009c700 tx=0x7f69700d3250 comp rx=0 tx=0).stop 2024-04-24T09:13:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.721+0000 7f69879a5740 1 -- 172.21.15.69:0/940227760 shutdown_connections 2024-04-24T09:13:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.721+0000 7f69879a5740 1 --2- 172.21.15.69:0/940227760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e41106b490 0x55e4110730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.721+0000 7f69879a5740 1 --2- 172.21.15.69:0/940227760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 0x55e411058b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.721+0000 7f69879a5740 1 --2- 172.21.15.69:0/940227760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e4110501b0 0x55e411066ec0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.721+0000 7f69879a5740 1 -- 172.21.15.69:0/940227760 >> 172.21.15.69:0/940227760 conn(0x55e41104c6c0 msgr2=0x55e411053610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.722+0000 7f69879a5740 1 -- 172.21.15.69:0/940227760 shutdown_connections 2024-04-24T09:13:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.722+0000 7f69879a5740 1 -- 172.21.15.69:0/940227760 wait complete. 2024-04-24T09:13:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.722+0000 7f69879a5740 1 -- start start 2024-04-24T09:13:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.723+0000 7f69879a5740 1 Processor -- start 2024-04-24T09:13:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.723+0000 7f69879a5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4110501b0 0x55e4110633d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.723+0000 7f69879a5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 0x55e411063910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.723+0000 7f69879a5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e41106b490 0x55e411054a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.723+0000 7f69879a5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b3f30 con 0x55e41106b490 2024-04-24T09:13:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.723+0000 7f69879a5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b53a0 con 0x55e4110501b0 2024-04-24T09:13:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.723+0000 7f69879a5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b4600 con 0x55e41105b390 2024-04-24T09:13:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.723+0000 7f69879a5740 10 client.0 _ll_register_callbacks cb 0x7f6970005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.724+0000 7f697d400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 0x55e411063910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.724+0000 7f697d400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 0x55e411063910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.69:35156/0 (socket says 172.21.15.69:35156) 2024-04-24T09:13:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.724+0000 7f697d400640 1 -- 172.21.15.69:0/4290419874 learned_addr learned my addr 172.21.15.69:0/4290419874 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.724+0000 7f697de00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4110501b0 0x55e4110633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.724+0000 7f697e800640 1 --2- 172.21.15.69:0/4290419874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e41106b490 0x55e411054a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.725+0000 7f697d400640 1 -- 172.21.15.69:0/4290419874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4110501b0 msgr2=0x55e4110633d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.725+0000 7f697d400640 1 --2- 172.21.15.69:0/4290419874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4110501b0 0x55e4110633d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.725+0000 7f697d400640 1 -- 172.21.15.69:0/4290419874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e41106b490 msgr2=0x55e411054a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.725+0000 7f697d400640 1 --2- 172.21.15.69:0/4290419874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e41106b490 0x55e411054a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.725+0000 7f697d400640 1 -- 172.21.15.69:0/4290419874 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e410d95620 con 0x55e41105b390 2024-04-24T09:13:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.725+0000 7f697d400640 1 --2- 172.21.15.69:0/4290419874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 0x55e411063910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f696c011850 tx=0x7f696c02c6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68bb422118f9b436 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.725+0000 7f695f400640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-24T09:13:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.726+0000 7f695f400640 1 -- 172.21.15.69:0/4290419874 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c009e80 con 0x55e41105b390 2024-04-24T09:13:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.726+0000 7f69879a5740 1 -- 172.21.15.69:0/4290419874 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69700b7020 con 0x55e41105b390 2024-04-24T09:13:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.726+0000 7f695f400640 1 -- 172.21.15.69:0/4290419874 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f696c004020 con 0x55e41105b390 2024-04-24T09:13:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.726+0000 7f695f400640 1 -- 172.21.15.69:0/4290419874 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f696c03f050 con 0x55e41105b390 2024-04-24T09:13:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.726+0000 7f69879a5740 10 client.25864 fetch_fsmap learned FSMap version 30 2024-04-24T09:13:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.726+0000 7f69879a5740 1 -- 172.21.15.69:0/4290419874 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e4110d1290 con 0x55e41105b390 2024-04-24T09:13:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.727+0000 7f695f400640 1 -- 172.21.15.69:0/4290419874 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69700b7020 con 0x55e41105b390 2024-04-24T09:13:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.727+0000 7f695f400640 1 -- 172.21.15.69:0/4290419874 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f696c018070 con 0x55e41105b390 2024-04-24T09:13:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.727+0000 7f69879a5740 10 client.25864 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:13:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.727+0000 7f69879a5740 10 client.25864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.727+0000 7f69879a5740 20 client.25864 populate_metadata read hostname 'smithi069' 2024-04-24T09:13:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.727+0000 7f69879a5740 1 --2- 172.21.15.69:0/4290419874 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e4110d1a10 0x55e4110d3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.727+0000 7f69879a5740 4 client.25864 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:13:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.727+0000 7f69879a5740 1 -- 172.21.15.69:0/4290419874 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e410fbe190 con 0x55e4110d1a10 2024-04-24T09:13:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.728+0000 7f697de00640 1 --2- 172.21.15.69:0/4290419874 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e4110d1a10 0x55e4110d3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:40.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.729+0000 7f697de00640 1 --2- 172.21.15.69:0/4290419874 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e4110d1a10 0x55e4110d3e30 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee54aa05377d7b0c server_cookie=89952507171473e in_seq=0 out_seq=0 2024-04-24T09:13:40.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.729+0000 7f695f400640 10 client.25864 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.816+0000 7f695f400640 1 -- 172.21.15.69:0/4290419874 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e410fbe190 con 0x55e4110d1a10 2024-04-24T09:13:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.816+0000 7f695f400640 10 client.25864 handle_command_reply: tid=0 2024-04-24T09:13:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.816+0000 7f69879a5740 1 client.25864 shutdown 2024-04-24T09:13:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.816+0000 7f69879a5740 1 -- 172.21.15.69:0/4290419874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 msgr2=0x55e411063910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.816+0000 7f69879a5740 1 --2- 172.21.15.69:0/4290419874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 0x55e411063910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f696c011850 tx=0x7f696c02c6b0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.816+0000 7f69879a5740 1 -- 172.21.15.69:0/4290419874 shutdown_connections 2024-04-24T09:13:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.816+0000 7f69879a5740 1 --2- 172.21.15.69:0/4290419874 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55e4110d1a10 0x55e4110d3e30 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.816+0000 7f69879a5740 1 --2- 172.21.15.69:0/4290419874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e41106b490 0x55e411054a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.816+0000 7f69879a5740 1 --2- 172.21.15.69:0/4290419874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e41105b390 0x55e411063910 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.816+0000 7f69879a5740 1 --2- 172.21.15.69:0/4290419874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4110501b0 0x55e4110633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.816+0000 7f695f400640 0 client.25864 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.816+0000 7f69879a5740 1 -- 172.21.15.69:0/4290419874 >> 172.21.15.69:0/4290419874 conn(0x55e41104c6c0 msgr2=0x55e411051b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.817+0000 7f69879a5740 1 -- 172.21.15.69:0/4290419874 shutdown_connections 2024-04-24T09:13:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.817+0000 7f69879a5740 1 -- 172.21.15.69:0/4290419874 wait complete. 2024-04-24T09:13:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.817+0000 7f69879a5740 20 client.25864 trim_cache size 0 max 16384 2024-04-24T09:13:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:13:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:13:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:13:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:13:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.818+0000 7f6984c00640 1 -- 172.21.15.69:0/3986120525 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69640a3fa0 msgr2=0x7f69640a6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.818+0000 7f6984c00640 1 --2- 172.21.15.69:0/3986120525 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69640a3fa0 0x7f69640a6460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6978004af0 tx=0x7f697803c040 comp rx=0 tx=0).stop 2024-04-24T09:13:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.818+0000 7f6984c00640 1 -- 172.21.15.69:0/3986120525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6980101080 msgr2=0x7f698019edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.818+0000 7f6984c00640 1 --2- 172.21.15.69:0/3986120525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6980101080 0x7f698019edd0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f697000e9f0 tx=0x7f697000edf0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.818+0000 7f6984c00640 1 -- 172.21.15.69:0/3986120525 shutdown_connections 2024-04-24T09:13:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.818+0000 7f6984c00640 1 --2- 172.21.15.69:0/3986120525 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f69640a3fa0 0x7f69640a6460 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.818+0000 7f6984c00640 1 --2- 172.21.15.69:0/3986120525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6980106480 0x7f69801a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.818+0000 7f6984c00640 1 --2- 172.21.15.69:0/3986120525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f69801019a0 0x7f698019f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.818+0000 7f6984c00640 1 --2- 172.21.15.69:0/3986120525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6980101080 0x7f698019edd0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.818+0000 7f6984c00640 1 -- 172.21.15.69:0/3986120525 >> 172.21.15.69:0/3986120525 conn(0x7f6980078010 msgr2=0x7f6980103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.818+0000 7f6984c00640 1 -- 172.21.15.69:0/3986120525 shutdown_connections 2024-04-24T09:13:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:40.818+0000 7f6984c00640 1 -- 172.21.15.69:0/3986120525 wait complete. 2024-04-24T09:13:40.941 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] string_to_tokens.lo 2024-04-24T09:13:40.985 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] str_to_bytes.lo 2024-04-24T09:13:41.134 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] tlibio.lo 2024-04-24T09:13:41.250 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] write_log.lo 2024-04-24T09:13:41.277 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] random.lo 2024-04-24T09:13:41.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:41 smithi130 ceph-mon[31553]: pgmap v201: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 315 KiB/s wr, 76 op/s 2024-04-24T09:13:41.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:41 smithi081 ceph-mon[30961]: pgmap v201: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 315 KiB/s wr, 76 op/s 2024-04-24T09:13:41.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:41 smithi069 ceph-mon[28170]: pgmap v201: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 315 KiB/s wr, 76 op/s 2024-04-24T09:13:41.861 INFO:tasks.workunit.client.0.smithi069.stdout: [LD] 2024-04-24T09:13:42.643 INFO:tasks.workunit.client.0.smithi069.stdout:Building ltp 2024-04-24T09:13:42.644 INFO:tasks.workunit.client.0.smithi069.stdout:Building src 2024-04-24T09:13:42.644 INFO:tasks.workunit.client.0.smithi069.stdout:Building common 2024-04-24T09:13:42.645 INFO:tasks.workunit.client.0.smithi069.stdout:Building m4 2024-04-24T09:13:42.726 INFO:tasks.workunit.client.0.smithi069.stdout: [DEP] 2024-04-24T09:13:42.898 INFO:tasks.workunit.client.0.smithi069.stdout: [DEP] 2024-04-24T09:13:43.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:43 smithi081 ceph-mon[30961]: pgmap v202: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 126 KiB/s wr, 23 op/s 2024-04-24T09:13:43.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:43 smithi069 ceph-mon[28170]: pgmap v202: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 126 KiB/s wr, 23 op/s 2024-04-24T09:13:43.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:43 smithi130 ceph-mon[31553]: pgmap v202: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 126 KiB/s wr, 23 op/s 2024-04-24T09:13:45.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:45 smithi081 ceph-mon[30961]: pgmap v203: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 126 KiB/s wr, 23 op/s 2024-04-24T09:13:45.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:45 smithi069 ceph-mon[28170]: pgmap v203: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 126 KiB/s wr, 23 op/s 2024-04-24T09:13:45.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:45 smithi130 ceph-mon[31553]: pgmap v203: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 126 KiB/s wr, 23 op/s 2024-04-24T09:13:47.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:47 smithi081 ceph-mon[30961]: pgmap v204: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 257 KiB/s wr, 55 op/s 2024-04-24T09:13:47.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:47 smithi069 ceph-mon[28170]: pgmap v204: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 257 KiB/s wr, 55 op/s 2024-04-24T09:13:47.683 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] doio 2024-04-24T09:13:47.686 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] fsstress 2024-04-24T09:13:47.686 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] fsx 2024-04-24T09:13:47.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:47 smithi130 ceph-mon[31553]: pgmap v204: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 257 KiB/s wr, 55 op/s 2024-04-24T09:13:48.999 INFO:tasks.workunit.client.0.smithi069.stderr:fsstress.c: In function 'attr_list_path': 2024-04-24T09:13:49.000 INFO:tasks.workunit.client.0.smithi069.stderr:fsstress.c:859:9: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2024-04-24T09:13:49.000 INFO:tasks.workunit.client.0.smithi069.stderr: 859 | rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2024-04-24T09:13:49.000 INFO:tasks.workunit.client.0.smithi069.stderr: | ^~~~ 2024-04-24T09:13:49.001 INFO:tasks.workunit.client.0.smithi069.stderr:In file included from ../src/global.h:25, 2024-04-24T09:13:49.001 INFO:tasks.workunit.client.0.smithi069.stderr: from fsstress.c:12: 2024-04-24T09:13:49.001 INFO:tasks.workunit.client.0.smithi069.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2024-04-24T09:13:49.001 INFO:tasks.workunit.client.0.smithi069.stderr: 165 | extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2024-04-24T09:13:49.001 INFO:tasks.workunit.client.0.smithi069.stderr: | ^~~~~~~~~ 2024-04-24T09:13:49.001 INFO:tasks.workunit.client.0.smithi069.stderr:fsstress.c: In function 'attr_remove_path': 2024-04-24T09:13:49.001 INFO:tasks.workunit.client.0.smithi069.stderr:fsstress.c:878:9: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2024-04-24T09:13:49.001 INFO:tasks.workunit.client.0.smithi069.stderr: 878 | rval = attr_remove(name->path, attrname, flags); 2024-04-24T09:13:49.001 INFO:tasks.workunit.client.0.smithi069.stderr: | ^~~~ 2024-04-24T09:13:49.002 INFO:tasks.workunit.client.0.smithi069.stderr:In file included from ../src/global.h:25, 2024-04-24T09:13:49.002 INFO:tasks.workunit.client.0.smithi069.stderr: from fsstress.c:12: 2024-04-24T09:13:49.002 INFO:tasks.workunit.client.0.smithi069.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2024-04-24T09:13:49.002 INFO:tasks.workunit.client.0.smithi069.stderr: 152 | extern int attr_remove (const char *__path, const char *__attrname, 2024-04-24T09:13:49.002 INFO:tasks.workunit.client.0.smithi069.stderr: | ^~~~~~~~~~~ 2024-04-24T09:13:49.002 INFO:tasks.workunit.client.0.smithi069.stderr:fsstress.c: In function 'attr_set_path': 2024-04-24T09:13:49.002 INFO:tasks.workunit.client.0.smithi069.stderr:fsstress.c:898:9: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2024-04-24T09:13:49.002 INFO:tasks.workunit.client.0.smithi069.stderr: 898 | rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2024-04-24T09:13:49.002 INFO:tasks.workunit.client.0.smithi069.stderr: | ^~~~ 2024-04-24T09:13:49.002 INFO:tasks.workunit.client.0.smithi069.stderr:In file included from ../src/global.h:25, 2024-04-24T09:13:49.002 INFO:tasks.workunit.client.0.smithi069.stderr: from fsstress.c:12: 2024-04-24T09:13:49.003 INFO:tasks.workunit.client.0.smithi069.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2024-04-24T09:13:49.003 INFO:tasks.workunit.client.0.smithi069.stderr: 139 | extern int attr_set (const char *__path, const char *__attrname, 2024-04-24T09:13:49.003 INFO:tasks.workunit.client.0.smithi069.stderr: | ^~~~~~~~ 2024-04-24T09:13:49.017 INFO:tasks.workunit.client.0.smithi069.stderr:fsstress.c: In function 'allocsp_f': 2024-04-24T09:13:49.018 INFO:tasks.workunit.client.0.smithi069.stderr:fsstress.c:2047:32: error: 'XFS_IOC_ALLOCSP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-04-24T09:13:49.018 INFO:tasks.workunit.client.0.smithi069.stderr: 2047 | e = xfsctl(f.path, fd, XFS_IOC_ALLOCSP64, &fl) < 0 ? errno : 0; 2024-04-24T09:13:49.018 INFO:tasks.workunit.client.0.smithi069.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-24T09:13:49.018 INFO:tasks.workunit.client.0.smithi069.stderr: | XFS_IOC_RESVSP64 2024-04-24T09:13:49.018 INFO:tasks.workunit.client.0.smithi069.stderr:fsstress.c:2047:32: note: each undeclared identifier is reported only once for each function it appears in 2024-04-24T09:13:49.050 INFO:tasks.workunit.client.0.smithi069.stderr:fsstress.c: In function 'freesp_f': 2024-04-24T09:13:49.051 INFO:tasks.workunit.client.0.smithi069.stderr:fsstress.c:3640:32: error: 'XFS_IOC_FREESP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-04-24T09:13:49.051 INFO:tasks.workunit.client.0.smithi069.stderr: 3640 | e = xfsctl(f.path, fd, XFS_IOC_FREESP64, &fl) < 0 ? errno : 0; 2024-04-24T09:13:49.051 INFO:tasks.workunit.client.0.smithi069.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-24T09:13:49.051 INFO:tasks.workunit.client.0.smithi069.stderr: | XFS_IOC_RESVSP64 2024-04-24T09:13:49.124 INFO:tasks.workunit.client.0.smithi069.stderr:gmake[3]: *** [Makefile:46: fsstress] Error 1 2024-04-24T09:13:49.124 INFO:tasks.workunit.client.0.smithi069.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-04-24T09:13:49.479 INFO:tasks.workunit.client.0.smithi069.stderr:fsx.c: In function 'main': 2024-04-24T09:13:49.479 INFO:tasks.workunit.client.0.smithi069.stderr:fsx.c:2662:33: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2024-04-24T09:13:49.479 INFO:tasks.workunit.client.0.smithi069.stderr: 2662 | strncpy(opsfile, optarg, sizeof(opsfile)); 2024-04-24T09:13:49.479 INFO:tasks.workunit.client.0.smithi069.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T09:13:49.480 INFO:tasks.workunit.client.0.smithi069.stderr:fsx.c:2632:25: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2024-04-24T09:13:49.480 INFO:tasks.workunit.client.0.smithi069.stderr: 2632 | strncpy(dname, optarg, sizeof(dname)); 2024-04-24T09:13:49.480 INFO:tasks.workunit.client.0.smithi069.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T09:13:49.550 INFO:tasks.workunit.client.0.smithi069.stderr:doio.c: In function 'parse_cmdline': 2024-04-24T09:13:49.550 INFO:tasks.workunit.client.0.smithi069.stderr:doio.c:3113:33: warning: '%s' directive argument is null [-Wformat-overflow=] 2024-04-24T09:13:49.550 INFO:tasks.workunit.client.0.smithi069.stderr: 3113 | fprintf(stderr, 2024-04-24T09:13:49.550 INFO:tasks.workunit.client.0.smithi069.stderr: | ^~~~~~~~~~~~~~~ 2024-04-24T09:13:49.551 INFO:tasks.workunit.client.0.smithi069.stderr: 3114 | "%s%s: Illegal -C arg (%s). Must be one of: ", 2024-04-24T09:13:49.551 INFO:tasks.workunit.client.0.smithi069.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T09:13:49.551 INFO:tasks.workunit.client.0.smithi069.stderr: 3115 | Prog, TagName, tok); 2024-04-24T09:13:49.551 INFO:tasks.workunit.client.0.smithi069.stderr: | ~~~~~~~~~~~~~~~~~~~ 2024-04-24T09:13:49.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:49 smithi081 ceph-mon[30961]: pgmap v205: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 175 KiB/s wr, 37 op/s 2024-04-24T09:13:49.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:49 smithi069 ceph-mon[28170]: pgmap v205: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 175 KiB/s wr, 37 op/s 2024-04-24T09:13:49.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:49 smithi130 ceph-mon[31553]: pgmap v205: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 175 KiB/s wr, 37 op/s 2024-04-24T09:13:49.816 INFO:tasks.workunit.client.0.smithi069.stderr:gmake[2]: *** [include/buildrules:30: ltp] Error 2 2024-04-24T09:13:49.816 INFO:tasks.workunit.client.0.smithi069.stderr:gmake[2]: *** Waiting for unfinished jobs.... 2024-04-24T09:13:51.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:51 smithi081 ceph-mon[30961]: pgmap v206: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 232 KiB/s wr, 46 op/s 2024-04-24T09:13:51.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:51 smithi069 ceph-mon[28170]: pgmap v206: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 232 KiB/s wr, 46 op/s 2024-04-24T09:13:51.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:51 smithi130 ceph-mon[31553]: pgmap v206: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 232 KiB/s wr, 46 op/s 2024-04-24T09:13:53.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:53 smithi081 ceph-mon[30961]: pgmap v207: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 187 KiB/s wr, 40 op/s 2024-04-24T09:13:53.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:53 smithi069 ceph-mon[28170]: pgmap v207: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 187 KiB/s wr, 40 op/s 2024-04-24T09:13:53.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:53 smithi130 ceph-mon[31553]: pgmap v207: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 187 KiB/s wr, 40 op/s 2024-04-24T09:13:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.968+0000 7f93c3800640 1 -- 172.21.15.69:0/1080665218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc104e10 msgr2=0x7f93bc108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.968+0000 7f93c3800640 1 --2- 172.21.15.69:0/1080665218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc104e10 0x7f93bc108e50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009900 tx=0x7f93a802a410 comp rx=0 tx=0).stop 2024-04-24T09:13:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.968+0000 7f93c3800640 1 -- 172.21.15.69:0/1080665218 shutdown_connections 2024-04-24T09:13:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.968+0000 7f93c3800640 1 --2- 172.21.15.69:0/1080665218 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93bc109530 0x7f93bc111100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.968+0000 7f93c3800640 1 --2- 172.21.15.69:0/1080665218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc104e10 0x7f93bc108e50 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.968+0000 7f93c3800640 1 --2- 172.21.15.69:0/1080665218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93bc104460 0x7f93bc104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.968+0000 7f93c3800640 1 -- 172.21.15.69:0/1080665218 >> 172.21.15.69:0/1080665218 conn(0x7f93bc0ffd00 msgr2=0x7f93bc102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.968+0000 7f93c3800640 1 -- 172.21.15.69:0/1080665218 shutdown_connections 2024-04-24T09:13:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.968+0000 7f93c3800640 1 -- 172.21.15.69:0/1080665218 wait complete. 2024-04-24T09:13:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.969+0000 7f93c3800640 1 Processor -- start 2024-04-24T09:13:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.969+0000 7f93c3800640 1 -- start start 2024-04-24T09:13:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.969+0000 7f93c3800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93bc104460 0x7f93bc1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.969+0000 7f93c3800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93bc109530 0x7f93bc1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.969+0000 7f93c3800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc19a800 0x7f93bc19acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.970+0000 7f93c3800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc1144c0 con 0x7f93bc19a800 2024-04-24T09:13:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.970+0000 7f93c3800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc114340 con 0x7f93bc109530 2024-04-24T09:13:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.970+0000 7f93c3800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc114640 con 0x7f93bc104460 2024-04-24T09:13:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.970+0000 7f93c1800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc19a800 0x7f93bc19acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.970+0000 7f93c1800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc19a800 0x7f93bc19acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41788/0 (socket says 172.21.15.69:41788) 2024-04-24T09:13:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.970+0000 7f93c1800640 1 -- 172.21.15.69:0/3646908040 learned_addr learned my addr 172.21.15.69:0/3646908040 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.970+0000 7f93bbe00640 1 --2- 172.21.15.69:0/3646908040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93bc109530 0x7f93bc1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.970+0000 7f93c0e00640 1 --2- 172.21.15.69:0/3646908040 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93bc104460 0x7f93bc1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.970+0000 7f93c1800640 1 -- 172.21.15.69:0/3646908040 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93bc104460 msgr2=0x7f93bc1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.970+0000 7f93c1800640 1 --2- 172.21.15.69:0/3646908040 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93bc104460 0x7f93bc1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.970+0000 7f93c1800640 1 -- 172.21.15.69:0/3646908040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93bc109530 msgr2=0x7f93bc1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.970+0000 7f93c1800640 1 --2- 172.21.15.69:0/3646908040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93bc109530 0x7f93bc1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.970+0000 7f93c1800640 1 -- 172.21.15.69:0/3646908040 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93bc19b390 con 0x7f93bc19a800 2024-04-24T09:13:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.970+0000 7f93bbe00640 1 --2- 172.21.15.69:0/3646908040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93bc109530 0x7f93bc1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.971+0000 7f93c1800640 1 --2- 172.21.15.69:0/3646908040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc19a800 0x7f93bc19acb0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f93b400eaa0 tx=0x7f93b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.971+0000 7f93b9600640 1 -- 172.21.15.69:0/3646908040 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b400cdb0 con 0x7f93bc19a800 2024-04-24T09:13:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.971+0000 7f93b9600640 1 -- 172.21.15.69:0/3646908040 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b4005020 con 0x7f93bc19a800 2024-04-24T09:13:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.971+0000 7f93c3800640 1 -- 172.21.15.69:0/3646908040 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93bc19b680 con 0x7f93bc19a800 2024-04-24T09:13:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.971+0000 7f93b9600640 1 -- 172.21.15.69:0/3646908040 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b40116d0 con 0x7f93bc19a800 2024-04-24T09:13:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.971+0000 7f93c3800640 1 -- 172.21.15.69:0/3646908040 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93bc1a77e0 con 0x7f93bc19a800 2024-04-24T09:13:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.975+0000 7f93b9600640 1 -- 172.21.15.69:0/3646908040 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f93b40051d0 con 0x7f93bc19a800 2024-04-24T09:13:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.975+0000 7f93c65f0740 1 Processor -- start 2024-04-24T09:13:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.975+0000 7f93c65f0740 1 -- start start 2024-04-24T09:13:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.975+0000 7f93c65f0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f421b0 0x564f06f42590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.975+0000 7f93c65f0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f42ad0 0x564f06f4ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.975+0000 7f93c65f0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f4d390 0x564f06f4f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.975+0000 7f93c65f0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564f06ea8900 con 0x564f06f421b0 2024-04-24T09:13:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.975+0000 7f93c65f0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x564f06e4ef20 con 0x564f06f4d390 2024-04-24T09:13:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.975+0000 7f93c65f0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564f06ea48c0 con 0x564f06f42ad0 2024-04-24T09:13:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.976+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f421b0 0x564f06f42590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.976+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f421b0 0x564f06f42590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41796/0 (socket says 172.21.15.69:41796) 2024-04-24T09:13:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.976+0000 7f93bbe00640 1 -- 172.21.15.69:0/1441374883 learned_addr learned my addr 172.21.15.69:0/1441374883 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.976+0000 7f93c0e00640 1 --2- 172.21.15.69:0/1441374883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f42ad0 0x564f06f4ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.976+0000 7f93bbe00640 1 --2- 172.21.15.69:0/1441374883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f4d390 0x564f06f4f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.976+0000 7f93bbe00640 1 -- 172.21.15.69:0/1441374883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f42ad0 msgr2=0x564f06f4ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.976+0000 7f93bbe00640 1 --2- 172.21.15.69:0/1441374883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f42ad0 0x564f06f4ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.976+0000 7f93bbe00640 1 -- 172.21.15.69:0/1441374883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f4d390 msgr2=0x564f06f4f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.976+0000 7f93bbe00640 1 --2- 172.21.15.69:0/1441374883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f4d390 0x564f06f4f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.976+0000 7f93bbe00640 1 -- 172.21.15.69:0/1441374883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f06e73e10 con 0x564f06f421b0 2024-04-24T09:13:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.977+0000 7f93b9600640 1 --2- 172.21.15.69:0/3646908040 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f93a40a3dc0 0x7f93a40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.977+0000 7f93bbe00640 1 --2- 172.21.15.69:0/1441374883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f421b0 0x564f06f42590 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f93a8005ec0 tx=0x7f93a8005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54043be6f3e1e28d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.977+0000 7f93b9600640 1 -- 172.21.15.69:0/3646908040 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f93b40d0d90 con 0x7f93bc19a800 2024-04-24T09:13:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.977+0000 7f93c0e00640 1 --2- 172.21.15.69:0/3646908040 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f93a40a3dc0 0x7f93a40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.977+0000 7f93ae000640 1 -- 172.21.15.69:0/1441374883 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8043020 con 0x564f06f421b0 2024-04-24T09:13:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.977+0000 7f93ae000640 1 -- 172.21.15.69:0/1441374883 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93a80046c0 con 0x564f06f421b0 2024-04-24T09:13:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.977+0000 7f93ae000640 1 -- 172.21.15.69:0/1441374883 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8035070 con 0x564f06f421b0 2024-04-24T09:13:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.978+0000 7f93c65f0740 1 -- 172.21.15.69:0/1441374883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f421b0 msgr2=0x564f06f42590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.978+0000 7f93c65f0740 1 --2- 172.21.15.69:0/1441374883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f421b0 0x564f06f42590 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f93a8005ec0 tx=0x7f93a8005bc0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.978+0000 7f93c0e00640 1 --2- 172.21.15.69:0/3646908040 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f93a40a3dc0 0x7f93a40a6280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x564f06f43750 tx=0x7f93b000d340 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.978+0000 7f93c65f0740 1 -- 172.21.15.69:0/1441374883 shutdown_connections 2024-04-24T09:13:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.978+0000 7f93c65f0740 1 --2- 172.21.15.69:0/1441374883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f4d390 0x564f06f4f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.978+0000 7f93c65f0740 1 --2- 172.21.15.69:0/1441374883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f42ad0 0x564f06f4ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.978+0000 7f93c65f0740 1 --2- 172.21.15.69:0/1441374883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f421b0 0x564f06f42590 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.978+0000 7f93c65f0740 1 -- 172.21.15.69:0/1441374883 >> 172.21.15.69:0/1441374883 conn(0x564f06f3e6c0 msgr2=0x564f06f3eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.978+0000 7f93c65f0740 1 -- 172.21.15.69:0/1441374883 shutdown_connections 2024-04-24T09:13:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.978+0000 7f93c65f0740 1 -- 172.21.15.69:0/1441374883 wait complete. 2024-04-24T09:13:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.979+0000 7f93c65f0740 1 -- start start 2024-04-24T09:13:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.979+0000 7f93c65f0740 1 Processor -- start 2024-04-24T09:13:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c65f0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f421b0 0x564f06f4a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c65f0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f42ad0 0x564f06f4a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c65f0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f4d390 0x564f06f4ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c65f0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564f06f4ff60 con 0x564f06f4d390 2024-04-24T09:13:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c65f0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x564f06e622c0 con 0x564f06f421b0 2024-04-24T09:13:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c65f0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564f06f500e0 con 0x564f06f42ad0 2024-04-24T09:13:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c65f0740 10 client.0 _ll_register_callbacks cb 0x564f06bbaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c1800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f4d390 0x564f06f4ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c1800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f4d390 0x564f06f4ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41810/0 (socket says 172.21.15.69:41810) 2024-04-24T09:13:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c1800640 1 -- 172.21.15.69:0/1354544698 learned_addr learned my addr 172.21.15.69:0/1354544698 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c0e00640 1 --2- 172.21.15.69:0/1354544698 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f421b0 0x564f06f4a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93bbe00640 1 --2- 172.21.15.69:0/1354544698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f42ad0 0x564f06f4a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c1800640 1 -- 172.21.15.69:0/1354544698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f42ad0 msgr2=0x564f06f4a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c1800640 1 --2- 172.21.15.69:0/1354544698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f42ad0 0x564f06f4a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c1800640 1 -- 172.21.15.69:0/1354544698 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f421b0 msgr2=0x564f06f4a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c1800640 1 --2- 172.21.15.69:0/1354544698 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f421b0 0x564f06f4a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c1800640 1 -- 172.21.15.69:0/1354544698 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f06e72fd0 con 0x564f06f4d390 2024-04-24T09:13:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.980+0000 7f93c0e00640 1 --2- 172.21.15.69:0/1354544698 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f421b0 0x564f06f4a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.981+0000 7f93c1800640 1 --2- 172.21.15.69:0/1354544698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f4d390 0x564f06f4ade0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f93b4099360 tx=0x7f93b409d1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e95d70c045bae7e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.981+0000 7f93a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:13:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.981+0000 7f93a2a00640 1 -- 172.21.15.69:0/1354544698 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b401f040 con 0x564f06f4d390 2024-04-24T09:13:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.981+0000 7f93a2a00640 1 -- 172.21.15.69:0/1354544698 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f93b40870c0 con 0x564f06f4d390 2024-04-24T09:13:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.981+0000 7f93c65f0740 1 -- 172.21.15.69:0/1354544698 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93a80046c0 con 0x564f06f4d390 2024-04-24T09:13:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.981+0000 7f93a2a00640 1 -- 172.21.15.69:0/1354544698 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f93b40a5600 con 0x564f06f4d390 2024-04-24T09:13:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.981+0000 7f93c65f0740 10 client.16884 fetch_fsmap learned FSMap version 30 2024-04-24T09:13:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.981+0000 7f93c65f0740 1 -- 172.21.15.69:0/1354544698 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564f06bbb210 con 0x564f06f4d390 2024-04-24T09:13:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.982+0000 7f93a2a00640 1 -- 172.21.15.69:0/1354544698 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93a80046c0 con 0x564f06f4d390 2024-04-24T09:13:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.982+0000 7f93a2a00640 1 -- 172.21.15.69:0/1354544698 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f93b400f040 con 0x564f06f4d390 2024-04-24T09:13:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.982+0000 7f93c65f0740 10 client.16884 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:13:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.982+0000 7f93c65f0740 10 client.16884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.982+0000 7f93c65f0740 20 client.16884 populate_metadata read hostname 'smithi069' 2024-04-24T09:13:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.982+0000 7f93c65f0740 1 --2- 172.21.15.69:0/1354544698 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x564f06f5d530 0x564f06f5f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.982+0000 7f93c65f0740 4 client.16884 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.982+0000 7f93c65f0740 1 -- 172.21.15.69:0/1354544698 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f06eb6320 con 0x564f06f5d530 2024-04-24T09:13:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.983+0000 7f93c0e00640 1 --2- 172.21.15.69:0/1354544698 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x564f06f5d530 0x564f06f5f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.984+0000 7f93c0e00640 1 --2- 172.21.15.69:0/1354544698 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x564f06f5d530 0x564f06f5f940 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b79aa43934bfaea5 server_cookie=c1c97edc202c96dc in_seq=0 out_seq=0 2024-04-24T09:13:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.984+0000 7f93a2a00640 10 client.16884 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.985+0000 7f93a2a00640 1 -- 172.21.15.69:0/1354544698 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f06eb6320 con 0x564f06f5d530 2024-04-24T09:13:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.985+0000 7f93a2a00640 10 client.16884 handle_command_reply: tid=0 2024-04-24T09:13:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.985+0000 7f93c65f0740 1 client.16884 shutdown 2024-04-24T09:13:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.986+0000 7f93c65f0740 1 -- 172.21.15.69:0/1354544698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f4d390 msgr2=0x564f06f4ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.986+0000 7f93c65f0740 1 --2- 172.21.15.69:0/1354544698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f4d390 0x564f06f4ade0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f93b4099360 tx=0x7f93b409d1d0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.986+0000 7f93c65f0740 1 -- 172.21.15.69:0/1354544698 shutdown_connections 2024-04-24T09:13:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.986+0000 7f93c65f0740 1 --2- 172.21.15.69:0/1354544698 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x564f06f5d530 0x564f06f5f940 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.986+0000 7f93c65f0740 1 --2- 172.21.15.69:0/1354544698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f4d390 0x564f06f4ade0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.986+0000 7f93c65f0740 1 --2- 172.21.15.69:0/1354544698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f42ad0 0x564f06f4a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.986+0000 7f93c65f0740 1 --2- 172.21.15.69:0/1354544698 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f421b0 0x564f06f4a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.986+0000 7f93c65f0740 1 -- 172.21.15.69:0/1354544698 >> 172.21.15.69:0/1354544698 conn(0x564f06f3e6c0 msgr2=0x564f06f44440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.986+0000 7f93a2a00640 0 client.16884 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.986+0000 7f93c65f0740 1 -- 172.21.15.69:0/1354544698 shutdown_connections 2024-04-24T09:13:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.986+0000 7f93c65f0740 1 -- 172.21.15.69:0/1354544698 wait complete. 2024-04-24T09:13:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:53.986+0000 7f93c65f0740 20 client.16884 trim_cache size 0 max 16384 2024-04-24T09:13:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.005+0000 7f93c65f0740 1 Processor -- start 2024-04-24T09:13:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.005+0000 7f93c65f0740 1 -- start start 2024-04-24T09:13:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.005+0000 7f93c65f0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f421b0 0x564f06f58e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.005+0000 7f93c65f0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f42ad0 0x564f06f44b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.005+0000 7f93c65f0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f4d390 0x564f06f3d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.005+0000 7f93c65f0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc1b3c90 con 0x564f06f421b0 2024-04-24T09:13:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.005+0000 7f93c65f0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc1b1500 con 0x564f06f4d390 2024-04-24T09:13:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.005+0000 7f93c65f0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc1af350 con 0x564f06f42ad0 2024-04-24T09:13:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.006+0000 7f93c0e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f421b0 0x564f06f58e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.006+0000 7f93c0e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f421b0 0x564f06f58e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41816/0 (socket says 172.21.15.69:41816) 2024-04-24T09:13:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.006+0000 7f93c0e00640 1 -- 172.21.15.69:0/819027658 learned_addr learned my addr 172.21.15.69:0/819027658 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.006+0000 7f93c0e00640 1 -- 172.21.15.69:0/819027658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f42ad0 msgr2=0x564f06f44b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.006+0000 7f93c1800640 1 --2- 172.21.15.69:0/819027658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f4d390 0x564f06f3d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.006+0000 7f93bbe00640 1 --2- 172.21.15.69:0/819027658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f42ad0 0x564f06f44b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.006+0000 7f93c0e00640 1 --2- 172.21.15.69:0/819027658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f42ad0 0x564f06f44b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.006+0000 7f93c0e00640 1 -- 172.21.15.69:0/819027658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f4d390 msgr2=0x564f06f3d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.006+0000 7f93c0e00640 1 --2- 172.21.15.69:0/819027658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f4d390 0x564f06f3d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.006+0000 7f93c0e00640 1 -- 172.21.15.69:0/819027658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f06c87620 con 0x564f06f421b0 2024-04-24T09:13:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.006+0000 7f93c1800640 1 --2- 172.21.15.69:0/819027658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f4d390 0x564f06f3d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.006+0000 7f93c0e00640 1 --2- 172.21.15.69:0/819027658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f421b0 0x564f06f58e20 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f93b00124c0 tx=0x7f93b0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc515804cf9172ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.007+0000 7f93ad600640 1 -- 172.21.15.69:0/819027658 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b00284b0 con 0x564f06f421b0 2024-04-24T09:13:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.007+0000 7f93ad600640 1 -- 172.21.15.69:0/819027658 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b00054d0 con 0x564f06f421b0 2024-04-24T09:13:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.007+0000 7f93ad600640 1 -- 172.21.15.69:0/819027658 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0004c60 con 0x564f06f421b0 2024-04-24T09:13:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.007+0000 7f93c65f0740 1 -- 172.21.15.69:0/819027658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f421b0 msgr2=0x564f06f58e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.007+0000 7f93c65f0740 1 --2- 172.21.15.69:0/819027658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f421b0 0x564f06f58e20 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f93b00124c0 tx=0x7f93b0014000 comp rx=0 tx=0).stop 2024-04-24T09:13:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.008+0000 7f93c65f0740 1 -- 172.21.15.69:0/819027658 shutdown_connections 2024-04-24T09:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.008+0000 7f93c65f0740 1 --2- 172.21.15.69:0/819027658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f4d390 0x564f06f3d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.008+0000 7f93c65f0740 1 --2- 172.21.15.69:0/819027658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f42ad0 0x564f06f44b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.008+0000 7f93c65f0740 1 --2- 172.21.15.69:0/819027658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f421b0 0x564f06f58e20 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.008+0000 7f93c65f0740 1 -- 172.21.15.69:0/819027658 >> 172.21.15.69:0/819027658 conn(0x564f06f3e6c0 msgr2=0x564f06f3cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.008+0000 7f93c65f0740 1 -- 172.21.15.69:0/819027658 shutdown_connections 2024-04-24T09:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.008+0000 7f93c65f0740 1 -- 172.21.15.69:0/819027658 wait complete. 2024-04-24T09:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.008+0000 7f93c65f0740 1 -- start start 2024-04-24T09:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.008+0000 7f93c65f0740 1 Processor -- start 2024-04-24T09:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93c65f0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f421b0 0x564f06f55540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93c65f0740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f42ad0 0x564f06f508b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93c65f0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f4d390 0x564f06f50df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93c65f0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc1af9c0 con 0x564f06f4d390 2024-04-24T09:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93c65f0740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc1b0e30 con 0x564f06f42ad0 2024-04-24T09:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93c65f0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc1b0090 con 0x564f06f421b0 2024-04-24T09:13:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93c65f0740 10 client.0 _ll_register_callbacks cb 0x7f93b4005830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93c1800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f4d390 0x564f06f50df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93c1800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f4d390 0x564f06f50df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41824/0 (socket says 172.21.15.69:41824) 2024-04-24T09:13:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93c1800640 1 -- 172.21.15.69:0/3980748051 learned_addr learned my addr 172.21.15.69:0/3980748051 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:13:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93bbe00640 1 --2- 172.21.15.69:0/3980748051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f42ad0 0x564f06f508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93c1800640 1 -- 172.21.15.69:0/3980748051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f421b0 msgr2=0x564f06f55540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93c1800640 1 --2- 172.21.15.69:0/3980748051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f421b0 0x564f06f55540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93c1800640 1 -- 172.21.15.69:0/3980748051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f42ad0 msgr2=0x564f06f508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93c1800640 1 --2- 172.21.15.69:0/3980748051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f42ad0 0x564f06f508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93c1800640 1 -- 172.21.15.69:0/3980748051 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f06f51430 con 0x564f06f4d390 2024-04-24T09:13:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.009+0000 7f93bbe00640 1 --2- 172.21.15.69:0/3980748051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f42ad0 0x564f06f508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.010+0000 7f93c1800640 1 --2- 172.21.15.69:0/3980748051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f4d390 0x564f06f50df0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f93b4098df0 tx=0x7f93b4098d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2e1114a21232d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.010+0000 7f93a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:13:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.010+0000 7f93a2a00640 1 -- 172.21.15.69:0/3980748051 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b401f030 con 0x564f06f4d390 2024-04-24T09:13:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.010+0000 7f93a2a00640 1 -- 172.21.15.69:0/3980748051 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f93b4067260 con 0x564f06f4d390 2024-04-24T09:13:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.010+0000 7f93c65f0740 1 -- 172.21.15.69:0/3980748051 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b00054d0 con 0x564f06f4d390 2024-04-24T09:13:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.010+0000 7f93a2a00640 1 -- 172.21.15.69:0/3980748051 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f93b4082050 con 0x564f06f4d390 2024-04-24T09:13:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.010+0000 7f93c65f0740 10 client.16890 fetch_fsmap learned FSMap version 30 2024-04-24T09:13:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.010+0000 7f93c65f0740 1 -- 172.21.15.69:0/3980748051 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564f06fc3890 con 0x564f06f4d390 2024-04-24T09:13:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.010+0000 7f93a2a00640 1 -- 172.21.15.69:0/3980748051 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b00054d0 con 0x564f06f4d390 2024-04-24T09:13:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.011+0000 7f93a2a00640 1 -- 172.21.15.69:0/3980748051 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f93b409b850 con 0x564f06f4d390 2024-04-24T09:13:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.011+0000 7f93c65f0740 10 client.16890 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:13:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.011+0000 7f93c65f0740 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.011+0000 7f93c65f0740 20 client.16890 populate_metadata read hostname 'smithi069' 2024-04-24T09:13:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.011+0000 7f93c65f0740 1 --2- 172.21.15.69:0/3980748051 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x564f06fc3f80 0x564f06fc63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.011+0000 7f93c65f0740 4 client.16890 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:13:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.011+0000 7f93c65f0740 1 -- 172.21.15.69:0/3980748051 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f06eb0190 con 0x564f06fc3f80 2024-04-24T09:13:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.011+0000 7f93c0e00640 1 --2- 172.21.15.69:0/3980748051 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x564f06fc3f80 0x564f06fc63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.012+0000 7f93c0e00640 1 --2- 172.21.15.69:0/3980748051 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x564f06fc3f80 0x564f06fc63c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdb98049c448c9c9 server_cookie=4a4feba2e0afd8f1 in_seq=0 out_seq=0 2024-04-24T09:13:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:54.012+0000 7f93a2a00640 10 client.16890 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:55.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:55 smithi081 ceph-mon[30961]: pgmap v208: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 187 KiB/s wr, 40 op/s 2024-04-24T09:13:55.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:55 smithi069 ceph-mon[28170]: pgmap v208: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 187 KiB/s wr, 40 op/s 2024-04-24T09:13:55.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:55 smithi130 ceph-mon[31553]: pgmap v208: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 187 KiB/s wr, 40 op/s 2024-04-24T09:13:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.341+0000 7f93a2a00640 1 -- 172.21.15.69:0/3980748051 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564f06eb0190 con 0x564f06fc3f80 2024-04-24T09:13:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.341+0000 7f93a2a00640 10 client.16890 handle_command_reply: tid=0 2024-04-24T09:13:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.341+0000 7f93c65f0740 1 client.16890 shutdown 2024-04-24T09:13:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.342+0000 7f93c65f0740 1 -- 172.21.15.69:0/3980748051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f4d390 msgr2=0x564f06f50df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.342+0000 7f93c65f0740 1 --2- 172.21.15.69:0/3980748051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f4d390 0x564f06f50df0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f93b4098df0 tx=0x7f93b4098d60 comp rx=0 tx=0).stop 2024-04-24T09:13:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.342+0000 7f93c65f0740 1 -- 172.21.15.69:0/3980748051 shutdown_connections 2024-04-24T09:13:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.342+0000 7f93c65f0740 1 --2- 172.21.15.69:0/3980748051 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x564f06fc3f80 0x564f06fc63c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.342+0000 7f93c65f0740 1 --2- 172.21.15.69:0/3980748051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564f06f4d390 0x564f06f50df0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.342+0000 7f93c65f0740 1 --2- 172.21.15.69:0/3980748051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f06f42ad0 0x564f06f508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.342+0000 7f93c65f0740 1 --2- 172.21.15.69:0/3980748051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f06f421b0 0x564f06f55540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.342+0000 7f93c65f0740 1 -- 172.21.15.69:0/3980748051 >> 172.21.15.69:0/3980748051 conn(0x564f06f3e6c0 msgr2=0x564f06f3d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.342+0000 7f93a2a00640 0 client.16890 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:13:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.342+0000 7f93c65f0740 1 -- 172.21.15.69:0/3980748051 shutdown_connections 2024-04-24T09:13:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.342+0000 7f93c65f0740 1 -- 172.21.15.69:0/3980748051 wait complete. 2024-04-24T09:13:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.343+0000 7f93c65f0740 20 client.16890 trim_cache size 0 max 16384 2024-04-24T09:13:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:13:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:13:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:13:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:13:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.344+0000 7f93c3800640 1 -- 172.21.15.69:0/3646908040 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f93a40a3dc0 msgr2=0x7f93a40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.344+0000 7f93c3800640 1 --2- 172.21.15.69:0/3646908040 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f93a40a3dc0 0x7f93a40a6280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x564f06f43750 tx=0x7f93b000d340 comp rx=0 tx=0).stop 2024-04-24T09:13:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.344+0000 7f93c3800640 1 -- 172.21.15.69:0/3646908040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc19a800 msgr2=0x7f93bc19acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.344+0000 7f93c3800640 1 --2- 172.21.15.69:0/3646908040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc19a800 0x7f93bc19acb0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f93b400eaa0 tx=0x7f93b400ef60 comp rx=0 tx=0).stop 2024-04-24T09:13:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.344+0000 7f93c3800640 1 -- 172.21.15.69:0/3646908040 shutdown_connections 2024-04-24T09:13:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.344+0000 7f93c3800640 1 --2- 172.21.15.69:0/3646908040 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f93a40a3dc0 0x7f93a40a6280 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.344+0000 7f93c3800640 1 --2- 172.21.15.69:0/3646908040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc19a800 0x7f93bc19acb0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.344+0000 7f93c3800640 1 --2- 172.21.15.69:0/3646908040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93bc109530 0x7f93bc1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.344+0000 7f93c3800640 1 --2- 172.21.15.69:0/3646908040 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93bc104460 0x7f93bc1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.344+0000 7f93c3800640 1 -- 172.21.15.69:0/3646908040 >> 172.21.15.69:0/3646908040 conn(0x7f93bc0ffd00 msgr2=0x7f93bc101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.344+0000 7f93c3800640 1 -- 172.21.15.69:0/3646908040 shutdown_connections 2024-04-24T09:13:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:13:56.344+0000 7f93c3800640 1 -- 172.21.15.69:0/3646908040 wait complete. 2024-04-24T09:13:57.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:57 smithi081 ceph-mon[30961]: pgmap v209: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 205 KiB/s wr, 44 op/s 2024-04-24T09:13:57.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:57 smithi069 ceph-mon[28170]: pgmap v209: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 205 KiB/s wr, 44 op/s 2024-04-24T09:13:57.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:57 smithi130 ceph-mon[31553]: pgmap v209: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 205 KiB/s wr, 44 op/s 2024-04-24T09:13:59.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:13:59 smithi081 ceph-mon[30961]: pgmap v210: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 74 KiB/s wr, 12 op/s 2024-04-24T09:13:59.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:13:59 smithi069 ceph-mon[28170]: pgmap v210: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 74 KiB/s wr, 12 op/s 2024-04-24T09:13:59.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:13:59 smithi130 ceph-mon[31553]: pgmap v210: 129 pgs: 129 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 74 KiB/s wr, 12 op/s 2024-04-24T09:14:01.166 INFO:tasks.workunit.client.0.smithi069.stdout: [DEP] 2024-04-24T09:14:01.170 INFO:tasks.workunit.client.0.smithi069.stdout: [DEP] 2024-04-24T09:14:01.176 INFO:tasks.workunit.client.0.smithi069.stdout: [DEP] 2024-04-24T09:14:01.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:01 smithi081 ceph-mon[30961]: pgmap v211: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 90 KiB/s wr, 19 op/s 2024-04-24T09:14:01.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:01 smithi069 ceph-mon[28170]: pgmap v211: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 90 KiB/s wr, 19 op/s 2024-04-24T09:14:01.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:01 smithi130 ceph-mon[31553]: pgmap v211: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 90 KiB/s wr, 19 op/s 2024-04-24T09:14:03.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:03 smithi081 ceph-mon[30961]: pgmap v212: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-04-24T09:14:03.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:03 smithi069 ceph-mon[28170]: pgmap v212: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-04-24T09:14:03.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:03 smithi130 ceph-mon[31553]: pgmap v212: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-04-24T09:14:03.976 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] dirstress 2024-04-24T09:14:04.002 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] fill2 2024-04-24T09:14:04.002 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] fill 2024-04-24T09:14:04.012 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] getpagesize 2024-04-24T09:14:04.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:04 smithi081 ceph-mon[30961]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T09:14:04.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:04 smithi069 ceph-mon[28170]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T09:14:04.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:04 smithi130 ceph-mon[31553]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T09:14:04.903 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] holes 2024-04-24T09:14:04.920 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] lstat64 2024-04-24T09:14:05.081 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] nametest 2024-04-24T09:14:05.110 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] permname 2024-04-24T09:14:05.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:05 smithi081 ceph-mon[30961]: pgmap v214: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-04-24T09:14:05.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:05 smithi069 ceph-mon[28170]: pgmap v214: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-04-24T09:14:05.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:05 smithi130 ceph-mon[31553]: pgmap v214: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-04-24T09:14:05.713 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] randholes 2024-04-24T09:14:05.784 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] runas 2024-04-24T09:14:05.926 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] truncfile 2024-04-24T09:14:05.969 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] usemem 2024-04-24T09:14:06.522 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] mmapcat 2024-04-24T09:14:06.572 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] append_reader 2024-04-24T09:14:06.594 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] append_writer 2024-04-24T09:14:06.646 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] dirperf 2024-04-24T09:14:07.280 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] metaperf 2024-04-24T09:14:07.308 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] devzero 2024-04-24T09:14:07.316 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] feature 2024-04-24T09:14:07.487 INFO:tasks.workunit.client.0.smithi069.stdout: [CC] alloc 2024-04-24T09:14:07.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:07 smithi081 ceph-mon[30961]: pgmap v215: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 117 KiB/s wr, 40 op/s 2024-04-24T09:14:07.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:07 smithi069 ceph-mon[28170]: pgmap v215: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 117 KiB/s wr, 40 op/s 2024-04-24T09:14:07.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:07 smithi130 ceph-mon[31553]: pgmap v215: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 117 KiB/s wr, 40 op/s 2024-04-24T09:14:08.076 INFO:tasks.workunit.client.0.smithi069.stderr:alloc.c: In function 'main': 2024-04-24T09:14:08.076 INFO:tasks.workunit.client.0.smithi069.stderr:alloc.c:154:1: error: 'bozo' undeclared (first use in this function) 2024-04-24T09:14:08.076 INFO:tasks.workunit.client.0.smithi069.stderr: 154 | bozo! 2024-04-24T09:14:08.076 INFO:tasks.workunit.client.0.smithi069.stderr: | ^~~~ 2024-04-24T09:14:08.076 INFO:tasks.workunit.client.0.smithi069.stderr:alloc.c:154:1: note: each undeclared identifier is reported only once for each function it appears in 2024-04-24T09:14:08.076 INFO:tasks.workunit.client.0.smithi069.stderr:alloc.c:154:5: error: expected ';' before '!' token 2024-04-24T09:14:08.076 INFO:tasks.workunit.client.0.smithi069.stderr: 154 | bozo! 2024-04-24T09:14:08.077 INFO:tasks.workunit.client.0.smithi069.stderr: | ^ 2024-04-24T09:14:08.077 INFO:tasks.workunit.client.0.smithi069.stderr: | ; 2024-04-24T09:14:08.077 INFO:tasks.workunit.client.0.smithi069.stderr:alloc.c:184:25: error: 'rflag' undeclared (first use in this function); did you mean 'nflag'? 2024-04-24T09:14:08.077 INFO:tasks.workunit.client.0.smithi069.stderr: 184 | rflag = 1; 2024-04-24T09:14:08.077 INFO:tasks.workunit.client.0.smithi069.stderr: | ^~~~~ 2024-04-24T09:14:08.077 INFO:tasks.workunit.client.0.smithi069.stderr: | nflag 2024-04-24T09:14:08.077 INFO:tasks.workunit.client.0.smithi069.stderr:alloc.c:320:5: error: expected ';' before '!' token 2024-04-24T09:14:08.078 INFO:tasks.workunit.client.0.smithi069.stderr: 320 | bozo! 2024-04-24T09:14:08.078 INFO:tasks.workunit.client.0.smithi069.stderr: | ^ 2024-04-24T09:14:08.078 INFO:tasks.workunit.client.0.smithi069.stderr: | ; 2024-04-24T09:14:08.078 INFO:tasks.workunit.client.0.smithi069.stderr:alloc.c:128:25: warning: variable 'f' set but not used [-Wunused-but-set-variable] 2024-04-24T09:14:08.078 INFO:tasks.workunit.client.0.smithi069.stderr: 128 | struct flock64 f; 2024-04-24T09:14:08.078 INFO:tasks.workunit.client.0.smithi069.stderr: | ^ 2024-04-24T09:14:08.084 INFO:tasks.workunit.client.0.smithi069.stderr:gmake[3]: *** [Makefile:83: alloc] Error 1 2024-04-24T09:14:08.084 INFO:tasks.workunit.client.0.smithi069.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-04-24T09:14:08.155 INFO:tasks.workunit.client.0.smithi069.stderr:gmake[2]: *** [include/buildrules:30: src] Error 2 2024-04-24T09:14:08.156 INFO:tasks.workunit.client.0.smithi069.stderr:make[1]: *** [Makefile:53: default] Error 2 2024-04-24T09:14:08.158 INFO:tasks.workunit.client.0.smithi069.stderr:make: *** [Makefile:51: default] Error 2 2024-04-24T09:14:08.160 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-24T09:14:08.161 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-04-24T09:14:08.161 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-24T09:14:09.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:08 smithi130 ceph-mon[31553]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T09:14:09.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:08 smithi130 ceph-mon[31553]: pgmap v217: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 124 KiB/s wr, 39 op/s 2024-04-24T09:14:09.226 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:08 smithi069 ceph-mon[28170]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T09:14:09.227 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:08 smithi069 ceph-mon[28170]: pgmap v217: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 124 KiB/s wr, 39 op/s 2024-04-24T09:14:09.230 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi069 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1f81fb397ae98da7563d451a78c61574c8f4e6e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-04-24T09:14:12.006 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b41adb3f1d194a7e85a4978d6ee2bb6e Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi069 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1f81fb397ae98da7563d451a78c61574c8f4e6e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-04-24T09:14:12.010 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-24T09:14:12.168 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi069.front.sepia.ceph.com 2024-04-24T09:14:12.173 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:11 smithi130 ceph-mon[31553]: pgmap v218: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 235 KiB/s wr, 87 op/s 2024-04-24T09:14:12.174 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:08 smithi081 ceph-mon[30961]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T09:14:12.174 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:08 smithi081 ceph-mon[30961]: pgmap v217: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 124 KiB/s wr, 39 op/s 2024-04-24T09:14:12.174 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:11 smithi081 ceph-mon[30961]: pgmap v218: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 235 KiB/s wr, 87 op/s 2024-04-24T09:14:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.490+0000 7f074bc00640 1 -- 172.21.15.69:0/249911172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07440742d0 msgr2=0x7f07440746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.490+0000 7f074bc00640 1 --2- 172.21.15.69:0/249911172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07440742d0 0x7f07440746b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0738009900 tx=0x7f073802a480 comp rx=0 tx=0).stop 2024-04-24T09:14:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.490+0000 7f074bc00640 1 -- 172.21.15.69:0/249911172 shutdown_connections 2024-04-24T09:14:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.490+0000 7f074bc00640 1 --2- 172.21.15.69:0/249911172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f074410e8f0 0x7f0744110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.490+0000 7f074bc00640 1 --2- 172.21.15.69:0/249911172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0744074bf0 0x7f074410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.490+0000 7f074bc00640 1 --2- 172.21.15.69:0/249911172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07440742d0 0x7f07440746b0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.490+0000 7f074bc00640 1 -- 172.21.15.69:0/249911172 >> 172.21.15.69:0/249911172 conn(0x7f0744079490 msgr2=0x7f07440798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.490+0000 7f074bc00640 1 -- 172.21.15.69:0/249911172 shutdown_connections 2024-04-24T09:14:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.490+0000 7f074bc00640 1 -- 172.21.15.69:0/249911172 wait complete. 2024-04-24T09:14:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.491+0000 7f074bc00640 1 Processor -- start 2024-04-24T09:14:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.491+0000 7f074bc00640 1 -- start start 2024-04-24T09:14:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.492+0000 7f074bc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07440742d0 0x7f07441a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.492+0000 7f074bc00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0744074bf0 0x7f07441a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.492+0000 7f074bc00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f074410e8f0 0x7f074419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.492+0000 7f074bc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0744114500 con 0x7f07440742d0 2024-04-24T09:14:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.492+0000 7f074bc00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0744114380 con 0x7f074410e8f0 2024-04-24T09:14:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.492+0000 7f074bc00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0744114680 con 0x7f0744074bf0 2024-04-24T09:14:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.492+0000 7f0749200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07440742d0 0x7f07441a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.492+0000 7f0749200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07440742d0 0x7f07441a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44930/0 (socket says 172.21.15.69:44930) 2024-04-24T09:14:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.492+0000 7f0749200640 1 -- 172.21.15.69:0/2326036491 learned_addr learned my addr 172.21.15.69:0/2326036491 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:14:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.493+0000 7f0743e00640 1 --2- 172.21.15.69:0/2326036491 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0744074bf0 0x7f07441a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.493+0000 7f0749c00640 1 --2- 172.21.15.69:0/2326036491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f074410e8f0 0x7f074419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.493+0000 7f0749200640 1 -- 172.21.15.69:0/2326036491 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0744074bf0 msgr2=0x7f07441a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.493+0000 7f0749200640 1 --2- 172.21.15.69:0/2326036491 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0744074bf0 0x7f07441a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.493+0000 7f0749200640 1 -- 172.21.15.69:0/2326036491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f074410e8f0 msgr2=0x7f074419a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.493+0000 7f0749200640 1 --2- 172.21.15.69:0/2326036491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f074410e8f0 0x7f074419a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.493+0000 7f0749200640 1 -- 172.21.15.69:0/2326036491 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074419afb0 con 0x7f07440742d0 2024-04-24T09:14:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.493+0000 7f0749200640 1 --2- 172.21.15.69:0/2326036491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07440742d0 0x7f07441a0620 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f0738005ab0 tx=0x7f0738003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.493+0000 7f0741600640 1 -- 172.21.15.69:0/2326036491 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073803f070 con 0x7f07440742d0 2024-04-24T09:14:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.493+0000 7f0741600640 1 -- 172.21.15.69:0/2326036491 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0738031d60 con 0x7f07440742d0 2024-04-24T09:14:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.493+0000 7f074bc00640 1 -- 172.21.15.69:0/2326036491 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074419b240 con 0x7f07440742d0 2024-04-24T09:14:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.493+0000 7f0741600640 1 -- 172.21.15.69:0/2326036491 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738035030 con 0x7f07440742d0 2024-04-24T09:14:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.493+0000 7f074bc00640 1 -- 172.21.15.69:0/2326036491 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07441a7470 con 0x7f07440742d0 2024-04-24T09:14:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.496+0000 7f074e9fd740 1 Processor -- start 2024-04-24T09:14:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.497+0000 7f0741600640 1 -- 172.21.15.69:0/2326036491 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0738042070 con 0x7f07440742d0 2024-04-24T09:14:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.497+0000 7f074e9fd740 1 -- start start 2024-04-24T09:14:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.497+0000 7f074e9fd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fb1b0 0x55f93b4fb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.497+0000 7f074e9fd740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b4fbad0 0x55f93b505e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.497+0000 7f074e9fd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b506390 0x55f93b508780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.497+0000 7f074e9fd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f93b461900 con 0x55f93b4fb1b0 2024-04-24T09:14:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.497+0000 7f074e9fd740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f93b407f20 con 0x55f93b4fbad0 2024-04-24T09:14:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.497+0000 7f074e9fd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f93b45d8c0 con 0x55f93b506390 2024-04-24T09:14:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.497+0000 7f0749c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fb1b0 0x55f93b4fb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.497+0000 7f0749c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fb1b0 0x55f93b4fb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44940/0 (socket says 172.21.15.69:44940) 2024-04-24T09:14:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.497+0000 7f0749c00640 1 -- 172.21.15.69:0/460123788 learned_addr learned my addr 172.21.15.69:0/460123788 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:14:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.498+0000 7f0743e00640 1 --2- 172.21.15.69:0/460123788 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b506390 0x55f93b508780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.498+0000 7f0749200640 1 --2- 172.21.15.69:0/460123788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b4fbad0 0x55f93b505e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.498+0000 7f0749c00640 1 -- 172.21.15.69:0/460123788 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b506390 msgr2=0x55f93b508780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.498+0000 7f0749c00640 1 --2- 172.21.15.69:0/460123788 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b506390 0x55f93b508780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.498+0000 7f0749c00640 1 -- 172.21.15.69:0/460123788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b4fbad0 msgr2=0x55f93b505e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.498+0000 7f0749c00640 1 --2- 172.21.15.69:0/460123788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b4fbad0 0x55f93b505e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.498+0000 7f0749c00640 1 -- 172.21.15.69:0/460123788 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f93b42ce10 con 0x55f93b4fb1b0 2024-04-24T09:14:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.498+0000 7f0749c00640 1 --2- 172.21.15.69:0/460123788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fb1b0 0x55f93b4fb590 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f073c009dc0 tx=0x7f073c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8a1206d327c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.498+0000 7f0741600640 1 --2- 172.21.15.69:0/2326036491 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f072c0a3e40 0x7f072c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.498+0000 7f0741600640 1 -- 172.21.15.69:0/2326036491 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f07380f11d0 con 0x7f07440742d0 2024-04-24T09:14:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.499+0000 7f0736000640 1 -- 172.21.15.69:0/460123788 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c01a480 con 0x55f93b4fb1b0 2024-04-24T09:14:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.499+0000 7f0736000640 1 -- 172.21.15.69:0/460123788 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f073c01aad0 con 0x55f93b4fb1b0 2024-04-24T09:14:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.499+0000 7f0736000640 1 -- 172.21.15.69:0/460123788 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c011c00 con 0x55f93b4fb1b0 2024-04-24T09:14:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.499+0000 7f0749200640 1 --2- 172.21.15.69:0/2326036491 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f072c0a3e40 0x7f072c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.499+0000 7f074e9fd740 1 -- 172.21.15.69:0/460123788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fb1b0 msgr2=0x55f93b4fb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.499+0000 7f0749200640 1 --2- 172.21.15.69:0/2326036491 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f072c0a3e40 0x7f072c0a6300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55f93b4fc750 tx=0x7f0738009260 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.499+0000 7f074e9fd740 1 --2- 172.21.15.69:0/460123788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fb1b0 0x55f93b4fb590 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f073c009dc0 tx=0x7f073c00f730 comp rx=0 tx=0).stop 2024-04-24T09:14:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.499+0000 7f074e9fd740 1 -- 172.21.15.69:0/460123788 shutdown_connections 2024-04-24T09:14:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.499+0000 7f074e9fd740 1 --2- 172.21.15.69:0/460123788 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b506390 0x55f93b508780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.499+0000 7f074e9fd740 1 --2- 172.21.15.69:0/460123788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b4fbad0 0x55f93b505e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.499+0000 7f074e9fd740 1 --2- 172.21.15.69:0/460123788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fb1b0 0x55f93b4fb590 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.499+0000 7f074e9fd740 1 -- 172.21.15.69:0/460123788 >> 172.21.15.69:0/460123788 conn(0x55f93b4f76c0 msgr2=0x55f93b4f7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.500+0000 7f074e9fd740 1 -- 172.21.15.69:0/460123788 shutdown_connections 2024-04-24T09:14:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.500+0000 7f074e9fd740 1 -- 172.21.15.69:0/460123788 wait complete. 2024-04-24T09:14:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.500+0000 7f074e9fd740 1 -- start start 2024-04-24T09:14:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.500+0000 7f074e9fd740 1 Processor -- start 2024-04-24T09:14:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.501+0000 7f074e9fd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fb1b0 0x55f93b5032b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.501+0000 7f074e9fd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b4fbad0 0x55f93b5037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.501+0000 7f074e9fd740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b506390 0x55f93b503d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.501+0000 7f074e9fd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f93b508fc0 con 0x55f93b4fb1b0 2024-04-24T09:14:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.501+0000 7f074e9fd740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f93b41b2c0 con 0x55f93b506390 2024-04-24T09:14:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.501+0000 7f074e9fd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f93b509140 con 0x55f93b4fbad0 2024-04-24T09:14:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.501+0000 7f074e9fd740 10 client.0 _ll_register_callbacks cb 0x55f93b173a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.501+0000 7f0749c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fb1b0 0x55f93b5032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.501+0000 7f0749c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fb1b0 0x55f93b5032b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44950/0 (socket says 172.21.15.69:44950) 2024-04-24T09:14:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.501+0000 7f0749c00640 1 -- 172.21.15.69:0/1176192325 learned_addr learned my addr 172.21.15.69:0/1176192325 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:14:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.501+0000 7f0743e00640 1 --2- 172.21.15.69:0/1176192325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b4fbad0 0x55f93b5037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.501+0000 7f0749c00640 1 -- 172.21.15.69:0/1176192325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b4fbad0 msgr2=0x55f93b5037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.501+0000 7f0749c00640 1 --2- 172.21.15.69:0/1176192325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b4fbad0 0x55f93b5037f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.501+0000 7f0749c00640 1 -- 172.21.15.69:0/1176192325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b506390 msgr2=0x55f93b503d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.501+0000 7f0749c00640 1 --2- 172.21.15.69:0/1176192325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b506390 0x55f93b503d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.501+0000 7f0749c00640 1 -- 172.21.15.69:0/1176192325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f93b42bfd0 con 0x55f93b4fb1b0 2024-04-24T09:14:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.502+0000 7f0749c00640 1 --2- 172.21.15.69:0/1176192325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fb1b0 0x55f93b5032b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f073c0116e0 tx=0x7f073c01aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f72c99d3461e30 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.502+0000 7f072aa00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:14:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.502+0000 7f072aa00640 1 -- 172.21.15.69:0/1176192325 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c020070 con 0x55f93b4fb1b0 2024-04-24T09:14:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.502+0000 7f072aa00640 1 -- 172.21.15.69:0/1176192325 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f073c01a480 con 0x55f93b4fb1b0 2024-04-24T09:14:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.502+0000 7f074e9fd740 1 -- 172.21.15.69:0/1176192325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f073c01aad0 con 0x55f93b4fb1b0 2024-04-24T09:14:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.502+0000 7f072aa00640 1 -- 172.21.15.69:0/1176192325 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f073c004e50 con 0x55f93b4fb1b0 2024-04-24T09:14:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.502+0000 7f074e9fd740 10 client.16902 fetch_fsmap learned FSMap version 30 2024-04-24T09:14:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.502+0000 7f074e9fd740 1 -- 172.21.15.69:0/1176192325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f93b174210 con 0x55f93b4fb1b0 2024-04-24T09:14:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.503+0000 7f072aa00640 1 -- 172.21.15.69:0/1176192325 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f073c01aad0 con 0x55f93b4fb1b0 2024-04-24T09:14:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.503+0000 7f072aa00640 1 -- 172.21.15.69:0/1176192325 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f073c005750 con 0x55f93b4fb1b0 2024-04-24T09:14:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.503+0000 7f074e9fd740 10 client.16902 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:14:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.503+0000 7f074e9fd740 10 client.16902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.503+0000 7f074e9fd740 20 client.16902 populate_metadata read hostname 'smithi069' 2024-04-24T09:14:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.503+0000 7f074e9fd740 1 --2- 172.21.15.69:0/1176192325 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f93b5163b0 0x55f93b5187a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.503+0000 7f074e9fd740 4 client.16902 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:14:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.503+0000 7f074e9fd740 1 -- 172.21.15.69:0/1176192325 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f93b46f320 con 0x55f93b5163b0 2024-04-24T09:14:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.504+0000 7f0743e00640 1 --2- 172.21.15.69:0/1176192325 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f93b5163b0 0x55f93b5187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.505+0000 7f0743e00640 1 --2- 172.21.15.69:0/1176192325 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f93b5163b0 0x55f93b5187a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1afbb76b1b9f64e8 server_cookie=bb2b2e2fb320b70e in_seq=0 out_seq=0 2024-04-24T09:14:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.505+0000 7f072aa00640 10 client.16902 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:14:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.506+0000 7f072aa00640 1 -- 172.21.15.69:0/1176192325 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f93b46f320 con 0x55f93b5163b0 2024-04-24T09:14:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.506+0000 7f072aa00640 10 client.16902 handle_command_reply: tid=0 2024-04-24T09:14:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.507+0000 7f074e9fd740 1 client.16902 shutdown 2024-04-24T09:14:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.507+0000 7f074e9fd740 1 -- 172.21.15.69:0/1176192325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fb1b0 msgr2=0x55f93b5032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.507+0000 7f074e9fd740 1 --2- 172.21.15.69:0/1176192325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fb1b0 0x55f93b5032b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f073c0116e0 tx=0x7f073c01aeb0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.507+0000 7f074e9fd740 1 -- 172.21.15.69:0/1176192325 shutdown_connections 2024-04-24T09:14:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.507+0000 7f074e9fd740 1 --2- 172.21.15.69:0/1176192325 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f93b5163b0 0x55f93b5187a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.507+0000 7f074e9fd740 1 --2- 172.21.15.69:0/1176192325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b506390 0x55f93b503d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.507+0000 7f074e9fd740 1 --2- 172.21.15.69:0/1176192325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b4fbad0 0x55f93b5037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.507+0000 7f074e9fd740 1 --2- 172.21.15.69:0/1176192325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fb1b0 0x55f93b5032b0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.507+0000 7f072aa00640 0 client.16902 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:14:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.507+0000 7f074e9fd740 1 -- 172.21.15.69:0/1176192325 >> 172.21.15.69:0/1176192325 conn(0x55f93b4f76c0 msgr2=0x55f93b4fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.507+0000 7f074e9fd740 1 -- 172.21.15.69:0/1176192325 shutdown_connections 2024-04-24T09:14:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.507+0000 7f074e9fd740 1 -- 172.21.15.69:0/1176192325 wait complete. 2024-04-24T09:14:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.507+0000 7f074e9fd740 20 client.16902 trim_cache size 0 max 16384 2024-04-24T09:14:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.523+0000 7f074e9fd740 1 Processor -- start 2024-04-24T09:14:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.523+0000 7f074e9fd740 1 -- start start 2024-04-24T09:14:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.524+0000 7f074e9fd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b4fb1b0 0x55f93b511e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.524+0000 7f074e9fd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fbad0 0x55f93b51dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.524+0000 7f074e9fd740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b506390 0x55f93b51e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.524+0000 7f074e9fd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f07441b3e20 con 0x55f93b4fbad0 2024-04-24T09:14:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.524+0000 7f074e9fd740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f07441b1690 con 0x55f93b506390 2024-04-24T09:14:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.524+0000 7f074e9fd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f07441af500 con 0x55f93b4fb1b0 2024-04-24T09:14:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.524+0000 7f0743e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fbad0 0x55f93b51dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.524+0000 7f0749c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b506390 0x55f93b51e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.524+0000 7f0749c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b4fb1b0 0x55f93b511e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.525+0000 7f0749c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b506390 0x55f93b51e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.69:54366/0 (socket says 172.21.15.69:54366) 2024-04-24T09:14:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.525+0000 7f0743e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fbad0 0x55f93b51dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44960/0 (socket says 172.21.15.69:44960) 2024-04-24T09:14:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.525+0000 7f0749c00640 1 -- 172.21.15.69:0/4116562139 learned_addr learned my addr 172.21.15.69:0/4116562139 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:14:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.525+0000 7f0743e00640 1 -- 172.21.15.69:0/4116562139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b4fb1b0 msgr2=0x55f93b511e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.525+0000 7f0743e00640 1 --2- 172.21.15.69:0/4116562139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b4fb1b0 0x55f93b511e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.525+0000 7f0743e00640 1 -- 172.21.15.69:0/4116562139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b506390 msgr2=0x55f93b51e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.525+0000 7f0743e00640 1 --2- 172.21.15.69:0/4116562139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b506390 0x55f93b51e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.525+0000 7f0743e00640 1 -- 172.21.15.69:0/4116562139 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f93b240620 con 0x55f93b4fbad0 2024-04-24T09:14:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.525+0000 7f0743e00640 1 --2- 172.21.15.69:0/4116562139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fbad0 0x55f93b51dfa0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x55f93b4f8ae0 tx=0x7f073000c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2030d5c44bad54 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.526+0000 7f0735600640 1 -- 172.21.15.69:0/4116562139 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730013b70 con 0x55f93b4fbad0 2024-04-24T09:14:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.526+0000 7f0735600640 1 -- 172.21.15.69:0/4116562139 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f073001b8a0 con 0x55f93b4fbad0 2024-04-24T09:14:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.526+0000 7f0735600640 1 -- 172.21.15.69:0/4116562139 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730004a00 con 0x55f93b4fbad0 2024-04-24T09:14:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.526+0000 7f074e9fd740 1 -- 172.21.15.69:0/4116562139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fbad0 msgr2=0x55f93b51dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.526+0000 7f074e9fd740 1 --2- 172.21.15.69:0/4116562139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fbad0 0x55f93b51dfa0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x55f93b4f8ae0 tx=0x7f073000c560 comp rx=0 tx=0).stop 2024-04-24T09:14:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.526+0000 7f074e9fd740 1 -- 172.21.15.69:0/4116562139 shutdown_connections 2024-04-24T09:14:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.526+0000 7f074e9fd740 1 --2- 172.21.15.69:0/4116562139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b506390 0x55f93b51e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.526+0000 7f074e9fd740 1 --2- 172.21.15.69:0/4116562139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b4fbad0 0x55f93b51dfa0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.526+0000 7f074e9fd740 1 --2- 172.21.15.69:0/4116562139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b4fb1b0 0x55f93b511e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.526+0000 7f074e9fd740 1 -- 172.21.15.69:0/4116562139 >> 172.21.15.69:0/4116562139 conn(0x55f93b4f76c0 msgr2=0x55f93b4fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.526+0000 7f074e9fd740 1 -- 172.21.15.69:0/4116562139 shutdown_connections 2024-04-24T09:14:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.527+0000 7f074e9fd740 1 -- 172.21.15.69:0/4116562139 wait complete. 2024-04-24T09:14:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.527+0000 7f074e9fd740 1 -- start start 2024-04-24T09:14:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.527+0000 7f074e9fd740 1 Processor -- start 2024-04-24T09:14:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.527+0000 7f074e9fd740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b4fb1b0 0x55f93b50e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.527+0000 7f074e9fd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b4fbad0 0x55f93b4ffb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.527+0000 7f074e9fd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b506390 0x55f93b5000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.527+0000 7f074e9fd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f07441afb50 con 0x55f93b506390 2024-04-24T09:14:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.527+0000 7f074e9fd740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f07441b0fc0 con 0x55f93b4fb1b0 2024-04-24T09:14:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.527+0000 7f074e9fd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f07441b0220 con 0x55f93b4fbad0 2024-04-24T09:14:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.528+0000 7f074e9fd740 10 client.0 _ll_register_callbacks cb 0x7f073c009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.528+0000 7f0749c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b506390 0x55f93b5000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.528+0000 7f0749c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b506390 0x55f93b5000a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44964/0 (socket says 172.21.15.69:44964) 2024-04-24T09:14:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.528+0000 7f0749c00640 1 -- 172.21.15.69:0/643152460 learned_addr learned my addr 172.21.15.69:0/643152460 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:14:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.528+0000 7f0749c00640 1 --2- 172.21.15.69:0/643152460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b4fb1b0 0x55f93b50e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.528+0000 7f0743e00640 1 --2- 172.21.15.69:0/643152460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b4fbad0 0x55f93b4ffb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.528+0000 7f0749c00640 1 -- 172.21.15.69:0/643152460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b4fbad0 msgr2=0x55f93b4ffb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.528+0000 7f0749c00640 1 --2- 172.21.15.69:0/643152460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b4fbad0 0x55f93b4ffb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.528+0000 7f0749c00640 1 -- 172.21.15.69:0/643152460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b4fb1b0 msgr2=0x55f93b50e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.528+0000 7f0749c00640 1 --2- 172.21.15.69:0/643152460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b4fb1b0 0x55f93b50e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.528+0000 7f0749c00640 1 -- 172.21.15.69:0/643152460 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f93b500710 con 0x55f93b506390 2024-04-24T09:14:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.528+0000 7f0749c00640 1 --2- 172.21.15.69:0/643152460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b506390 0x55f93b5000a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f073c009ce0 tx=0x7f073c01aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40e1f3946133f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.529+0000 7f072aa00640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-24T09:14:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.529+0000 7f072aa00640 1 -- 172.21.15.69:0/643152460 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c041060 con 0x55f93b506390 2024-04-24T09:14:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.529+0000 7f072aa00640 1 -- 172.21.15.69:0/643152460 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f073c027b10 con 0x55f93b506390 2024-04-24T09:14:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.529+0000 7f074e9fd740 1 -- 172.21.15.69:0/643152460 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f073001b8a0 con 0x55f93b506390 2024-04-24T09:14:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.529+0000 7f072aa00640 1 -- 172.21.15.69:0/643152460 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f073c048050 con 0x55f93b506390 2024-04-24T09:14:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.529+0000 7f074e9fd740 10 client.16908 fetch_fsmap learned FSMap version 30 2024-04-24T09:14:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.529+0000 7f074e9fd740 1 -- 172.21.15.69:0/643152460 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f93b57c6b0 con 0x55f93b506390 2024-04-24T09:14:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.529+0000 7f072aa00640 1 -- 172.21.15.69:0/643152460 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f073001b8a0 con 0x55f93b506390 2024-04-24T09:14:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.530+0000 7f072aa00640 1 -- 172.21.15.69:0/643152460 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f073c0117b0 con 0x55f93b506390 2024-04-24T09:14:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.530+0000 7f074e9fd740 10 client.16908 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T09:14:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.530+0000 7f074e9fd740 10 client.16908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.530+0000 7f074e9fd740 20 client.16908 populate_metadata read hostname 'smithi069' 2024-04-24T09:14:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.530+0000 7f074e9fd740 1 --2- 172.21.15.69:0/643152460 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f93b57ce30 0x55f93b57f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.530+0000 7f074e9fd740 4 client.16908 mds_command: new command op to 34532 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:14:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.530+0000 7f074e9fd740 1 -- 172.21.15.69:0/643152460 --> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f93b469190 con 0x55f93b57ce30 2024-04-24T09:14:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.530+0000 7f0743e00640 1 --2- 172.21.15.69:0/643152460 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f93b57ce30 0x55f93b57f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.531+0000 7f0743e00640 1 --2- 172.21.15.69:0/643152460 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f93b57ce30 0x55f93b57f250 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3fe27c095d1c4f6 server_cookie=35a045d0840e9a13 in_seq=0 out_seq=0 2024-04-24T09:14:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.531+0000 7f072aa00640 10 client.16908 ms_handle_connect on v2:172.21.15.81:6838/3185704097 2024-04-24T09:14:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.576+0000 7f072aa00640 1 -- 172.21.15.69:0/643152460 <== mds.0 v2:172.21.15.81:6838/3185704097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f93b469190 con 0x55f93b57ce30 2024-04-24T09:14:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.576+0000 7f072aa00640 10 client.16908 handle_command_reply: tid=0 2024-04-24T09:14:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.576+0000 7f074e9fd740 1 client.16908 shutdown 2024-04-24T09:14:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.577+0000 7f074e9fd740 1 -- 172.21.15.69:0/643152460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b506390 msgr2=0x55f93b5000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.577+0000 7f074e9fd740 1 --2- 172.21.15.69:0/643152460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b506390 0x55f93b5000a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f073c009ce0 tx=0x7f073c01aeb0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.577+0000 7f074e9fd740 1 -- 172.21.15.69:0/643152460 shutdown_connections 2024-04-24T09:14:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.577+0000 7f074e9fd740 1 --2- 172.21.15.69:0/643152460 >> [v2:172.21.15.81:6838/3185704097,v1:172.21.15.81:6839/3185704097] conn(0x55f93b57ce30 0x55f93b57f250 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.577+0000 7f074e9fd740 1 --2- 172.21.15.69:0/643152460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f93b506390 0x55f93b5000a0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.577+0000 7f074e9fd740 1 --2- 172.21.15.69:0/643152460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f93b4fbad0 0x55f93b4ffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.577+0000 7f074e9fd740 1 --2- 172.21.15.69:0/643152460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f93b4fb1b0 0x55f93b50e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.577+0000 7f074e9fd740 1 -- 172.21.15.69:0/643152460 >> 172.21.15.69:0/643152460 conn(0x55f93b4f76c0 msgr2=0x55f93b4f5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.577+0000 7f072aa00640 0 client.16908 ms_handle_reset on v2:172.21.15.81:6838/3185704097 2024-04-24T09:14:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.577+0000 7f074e9fd740 1 -- 172.21.15.69:0/643152460 shutdown_connections 2024-04-24T09:14:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.577+0000 7f074e9fd740 1 -- 172.21.15.69:0/643152460 wait complete. 2024-04-24T09:14:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.578+0000 7f074e9fd740 20 client.16908 trim_cache size 0 max 16384 2024-04-24T09:14:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-04-24T09:14:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-04-24T09:14:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-04-24T09:14:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-04-24T09:14:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.578+0000 7f074bc00640 1 -- 172.21.15.69:0/2326036491 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f072c0a3e40 msgr2=0x7f072c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.578+0000 7f074bc00640 1 --2- 172.21.15.69:0/2326036491 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f072c0a3e40 0x7f072c0a6300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55f93b4fc750 tx=0x7f0738009260 comp rx=0 tx=0).stop 2024-04-24T09:14:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.578+0000 7f074bc00640 1 -- 172.21.15.69:0/2326036491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07440742d0 msgr2=0x7f07441a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.578+0000 7f074bc00640 1 --2- 172.21.15.69:0/2326036491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07440742d0 0x7f07441a0620 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f0738005ab0 tx=0x7f0738003cd0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.579+0000 7f074bc00640 1 -- 172.21.15.69:0/2326036491 shutdown_connections 2024-04-24T09:14:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.579+0000 7f074bc00640 1 --2- 172.21.15.69:0/2326036491 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f072c0a3e40 0x7f072c0a6300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.579+0000 7f074bc00640 1 --2- 172.21.15.69:0/2326036491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f074410e8f0 0x7f074419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.579+0000 7f074bc00640 1 --2- 172.21.15.69:0/2326036491 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0744074bf0 0x7f07441a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.579+0000 7f074bc00640 1 --2- 172.21.15.69:0/2326036491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07440742d0 0x7f07441a0620 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.579+0000 7f074bc00640 1 -- 172.21.15.69:0/2326036491 >> 172.21.15.69:0/2326036491 conn(0x7f0744079490 msgr2=0x7f07440feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.579+0000 7f074bc00640 1 -- 172.21.15.69:0/2326036491 shutdown_connections 2024-04-24T09:14:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-04-24T09:14:09.579+0000 7f074bc00640 1 -- 172.21.15.69:0/2326036491 wait complete. 2024-04-24T09:14:12.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:11 smithi069 ceph-mon[28170]: pgmap v218: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 235 KiB/s wr, 87 op/s 2024-04-24T09:14:12.223 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-24T09:14:12.240 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-24T09:14:12.240 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T09:14:12.241 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-04-24T09:14:12.241 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-24T09:14:12.359 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-24T09:14:13.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:13 smithi130 ceph-mon[31553]: pgmap v219: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 213 KiB/s wr, 79 op/s 2024-04-24T09:14:13.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:13 smithi081 ceph-mon[30961]: pgmap v219: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 213 KiB/s wr, 79 op/s 2024-04-24T09:14:13.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:13 smithi069 ceph-mon[28170]: pgmap v219: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 213 KiB/s wr, 79 op/s 2024-04-24T09:14:15.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:15 smithi130 ceph-mon[31553]: pgmap v220: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 188 KiB/s wr, 69 op/s 2024-04-24T09:14:15.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:15 smithi081 ceph-mon[30961]: pgmap v220: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 188 KiB/s wr, 69 op/s 2024-04-24T09:14:15.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:15 smithi069 ceph-mon[28170]: pgmap v220: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 188 KiB/s wr, 69 op/s 2024-04-24T09:14:17.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:17 smithi130 ceph-mon[31553]: pgmap v221: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 104 KiB/s wr, 44 op/s 2024-04-24T09:14:17.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:17 smithi081 ceph-mon[30961]: pgmap v221: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 104 KiB/s wr, 44 op/s 2024-04-24T09:14:17.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:17 smithi069 ceph-mon[28170]: pgmap v221: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 104 KiB/s wr, 44 op/s 2024-04-24T09:14:19.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:19 smithi130 ceph-mon[31553]: pgmap v222: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 102 KiB/s wr, 43 op/s 2024-04-24T09:14:19.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:19 smithi081 ceph-mon[30961]: pgmap v222: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 102 KiB/s wr, 43 op/s 2024-04-24T09:14:19.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:19 smithi069 ceph-mon[28170]: pgmap v222: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 102 KiB/s wr, 43 op/s 2024-04-24T09:14:21.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:21 smithi130 ceph-mon[31553]: pgmap v223: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 100 KiB/s wr, 41 op/s 2024-04-24T09:14:21.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:21 smithi081 ceph-mon[30961]: pgmap v223: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 100 KiB/s wr, 41 op/s 2024-04-24T09:14:21.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:21 smithi069 ceph-mon[28170]: pgmap v223: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 100 KiB/s wr, 41 op/s 2024-04-24T09:14:23.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:23 smithi130 ceph-mon[31553]: pgmap v224: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-24T09:14:23.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:23 smithi081 ceph-mon[30961]: pgmap v224: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-24T09:14:23.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:23 smithi069 ceph-mon[28170]: pgmap v224: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-24T09:14:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:25 smithi081 ceph-mon[30961]: pgmap v225: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-24T09:14:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:25 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:25.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:25 smithi069 ceph-mon[28170]: pgmap v225: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-24T09:14:25.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:25 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:25.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:25 smithi130 ceph-mon[31553]: pgmap v225: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-24T09:14:25.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:25 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:26.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:26 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:26.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:26 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:26.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:26 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:26.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:26 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:26.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:26 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:26.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:26 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:27.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:27 smithi081 ceph-mon[30961]: pgmap v226: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-24T09:14:27.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:27 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:14:27.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:27 smithi069 ceph-mon[28170]: pgmap v226: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-24T09:14:27.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:27 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:14:27.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:27 smithi130 ceph-mon[31553]: pgmap v226: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-24T09:14:27.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:27 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:14:29.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:29 smithi081 ceph-mon[30961]: pgmap v227: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-04-24T09:14:29.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:29 smithi069 ceph-mon[28170]: pgmap v227: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-04-24T09:14:29.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:29 smithi130 ceph-mon[31553]: pgmap v227: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-04-24T09:14:31.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:31 smithi081 ceph-mon[30961]: pgmap v228: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-04-24T09:14:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:31 smithi069 ceph-mon[28170]: pgmap v228: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-04-24T09:14:31.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:31 smithi130 ceph-mon[31553]: pgmap v228: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-04-24T09:14:33.507 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:33 smithi130 ceph-mon[31553]: pgmap v229: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.8 KiB/s wr, 6 op/s 2024-04-24T09:14:33.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:33 smithi081 ceph-mon[30961]: pgmap v229: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.8 KiB/s wr, 6 op/s 2024-04-24T09:14:33.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:33 smithi069 ceph-mon[28170]: pgmap v229: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.8 KiB/s wr, 6 op/s 2024-04-24T09:14:35.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:35 smithi081 ceph-mon[30961]: pgmap v230: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.8 KiB/s wr, 6 op/s 2024-04-24T09:14:35.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:35 smithi069 ceph-mon[28170]: pgmap v230: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.8 KiB/s wr, 6 op/s 2024-04-24T09:14:35.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:35 smithi130 ceph-mon[31553]: pgmap v230: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.8 KiB/s wr, 6 op/s 2024-04-24T09:14:37.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:37 smithi081 ceph-mon[30961]: pgmap v231: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-04-24T09:14:37.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:37 smithi069 ceph-mon[28170]: pgmap v231: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-04-24T09:14:37.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:37 smithi130 ceph-mon[31553]: pgmap v231: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-04-24T09:14:39.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:39 smithi081 ceph-mon[30961]: pgmap v232: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-04-24T09:14:39.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:39 smithi069 ceph-mon[28170]: pgmap v232: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-04-24T09:14:39.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:39 smithi130 ceph-mon[31553]: pgmap v232: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-04-24T09:14:41.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:41 smithi081 ceph-mon[30961]: pgmap v233: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-04-24T09:14:41.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:41 smithi069 ceph-mon[28170]: pgmap v233: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-04-24T09:14:41.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:41 smithi130 ceph-mon[31553]: pgmap v233: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-04-24T09:14:43.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:43 smithi081 ceph-mon[30961]: pgmap v234: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-04-24T09:14:43.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:43 smithi069 ceph-mon[28170]: pgmap v234: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-04-24T09:14:43.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:43 smithi130 ceph-mon[31553]: pgmap v234: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-04-24T09:14:45.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:45 smithi081 ceph-mon[30961]: pgmap v235: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-04-24T09:14:45.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:45 smithi069 ceph-mon[28170]: pgmap v235: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-04-24T09:14:45.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:45 smithi130 ceph-mon[31553]: pgmap v235: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-04-24T09:14:47.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:47 smithi081 ceph-mon[30961]: pgmap v236: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-24T09:14:47.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:47 smithi069 ceph-mon[28170]: pgmap v236: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-24T09:14:47.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:47 smithi130 ceph-mon[31553]: pgmap v236: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-24T09:14:49.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:49 smithi081 ceph-mon[30961]: pgmap v237: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-04-24T09:14:49.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:49 smithi069 ceph-mon[28170]: pgmap v237: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-04-24T09:14:49.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:49 smithi130 ceph-mon[31553]: pgmap v237: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-04-24T09:14:51.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:51 smithi081 ceph-mon[30961]: pgmap v238: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-04-24T09:14:51.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:51 smithi069 ceph-mon[28170]: pgmap v238: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-04-24T09:14:51.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:51 smithi130 ceph-mon[31553]: pgmap v238: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-04-24T09:14:53.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:53 smithi081 ceph-mon[30961]: pgmap v239: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-24T09:14:53.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:53 smithi069 ceph-mon[28170]: pgmap v239: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-24T09:14:53.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:53 smithi130 ceph-mon[31553]: pgmap v239: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-24T09:14:55.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:55 smithi069 ceph-mon[28170]: pgmap v240: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-24T09:14:55.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:55 smithi130 ceph-mon[31553]: pgmap v240: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-24T09:14:55.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:55 smithi081 ceph-mon[30961]: pgmap v240: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-24T09:14:57.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:57 smithi069 ceph-mon[28170]: pgmap v241: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-04-24T09:14:57.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:57 smithi130 ceph-mon[31553]: pgmap v241: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-04-24T09:14:57.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:57 smithi081 ceph-mon[30961]: pgmap v241: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-04-24T09:14:59.598 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:14:59 smithi069 ceph-mon[28170]: pgmap v242: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-24T09:14:59.826 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:14:59 smithi081 ceph-mon[30961]: pgmap v242: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-24T09:14:59.912 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:14:59 smithi130 ceph-mon[31553]: pgmap v242: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-24T09:15:01.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:01 smithi069 ceph-mon[28170]: pgmap v243: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-04-24T09:15:01.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:01 smithi130 ceph-mon[31553]: pgmap v243: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-04-24T09:15:01.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:01 smithi081 ceph-mon[30961]: pgmap v243: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-04-24T09:15:03.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:03 smithi130 ceph-mon[31553]: pgmap v244: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-24T09:15:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:03 smithi081 ceph-mon[30961]: pgmap v244: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-24T09:15:03.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:03 smithi069 ceph-mon[28170]: pgmap v244: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-24T09:15:05.620 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:05 smithi081 ceph-mon[30961]: pgmap v245: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-24T09:15:05.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:05 smithi130 ceph-mon[31553]: pgmap v245: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-24T09:15:05.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:05 smithi069 ceph-mon[28170]: pgmap v245: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-24T09:15:07.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:07 smithi130 ceph-mon[31553]: pgmap v246: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-24T09:15:07.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:07 smithi081 ceph-mon[30961]: pgmap v246: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-24T09:15:07.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:07 smithi069 ceph-mon[28170]: pgmap v246: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-24T09:15:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:09 smithi130 ceph-mon[31553]: pgmap v247: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-24T09:15:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:09 smithi130 ceph-mon[31553]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T09:15:09.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:09 smithi081 ceph-mon[30961]: pgmap v247: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-24T09:15:09.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:09 smithi081 ceph-mon[30961]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T09:15:09.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:09 smithi069 ceph-mon[28170]: pgmap v247: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-24T09:15:09.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:09 smithi069 ceph-mon[28170]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T09:15:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:11 smithi130 ceph-mon[31553]: pgmap v249: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:15:11.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:11 smithi081 ceph-mon[30961]: pgmap v249: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:15:11.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:11 smithi069 ceph-mon[28170]: pgmap v249: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:15:13.705 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:13 smithi130 ceph-mon[31553]: pgmap v250: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:15:13.706 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:13 smithi130 ceph-mon[31553]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T09:15:13.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:13 smithi081 ceph-mon[30961]: pgmap v250: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:15:13.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:13 smithi081 ceph-mon[30961]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T09:15:13.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:13 smithi069 ceph-mon[28170]: pgmap v250: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:15:13.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:13 smithi069 ceph-mon[28170]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T09:15:15.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:15 smithi130 ceph-mon[31553]: pgmap v252: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 255 B/s wr, 4 op/s 2024-04-24T09:15:15.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:15 smithi081 ceph-mon[30961]: pgmap v252: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 255 B/s wr, 4 op/s 2024-04-24T09:15:15.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:15 smithi069 ceph-mon[28170]: pgmap v252: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 255 B/s wr, 4 op/s 2024-04-24T09:15:17.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:17 smithi130 ceph-mon[31553]: pgmap v253: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 8 op/s 2024-04-24T09:15:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:17 smithi081 ceph-mon[30961]: pgmap v253: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 8 op/s 2024-04-24T09:15:17.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:17 smithi069 ceph-mon[28170]: pgmap v253: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 8 op/s 2024-04-24T09:15:19.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:18 smithi130 ceph-mon[31553]: pgmap v254: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 314 B/s wr, 6 op/s 2024-04-24T09:15:19.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:18 smithi081 ceph-mon[30961]: pgmap v254: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 314 B/s wr, 6 op/s 2024-04-24T09:15:19.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:18 smithi069 ceph-mon[28170]: pgmap v254: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 314 B/s wr, 6 op/s 2024-04-24T09:15:21.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:21 smithi130 ceph-mon[31553]: pgmap v255: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T09:15:21.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:21 smithi081 ceph-mon[30961]: pgmap v255: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T09:15:21.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:21 smithi069 ceph-mon[28170]: pgmap v255: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T09:15:23.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:23 smithi130 ceph-mon[31553]: pgmap v256: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T09:15:23.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:23 smithi081 ceph-mon[30961]: pgmap v256: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T09:15:23.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:23 smithi069 ceph-mon[28170]: pgmap v256: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T09:15:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:25 smithi081 ceph-mon[30961]: pgmap v257: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2024-04-24T09:15:25.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:25 smithi069 ceph-mon[28170]: pgmap v257: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2024-04-24T09:15:25.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:25 smithi130 ceph-mon[31553]: pgmap v257: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2024-04-24T09:15:27.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:27 smithi081 ceph-mon[30961]: pgmap v258: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-24T09:15:27.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:27 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:27.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:27 smithi069 ceph-mon[28170]: pgmap v258: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-24T09:15:27.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:27 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:27.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:27 smithi130 ceph-mon[31553]: pgmap v258: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-24T09:15:27.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:27 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:28.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:28 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:15:28.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:28 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:15:28.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:28 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:15:28.842 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:28 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:15:28.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:28 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:15:28.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:28 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:15:28.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:28 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:15:28.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:28 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:15:29.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:28 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:15:29.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:28 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:15:29.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:28 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:15:29.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:28 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:15:29.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:29 smithi130 ceph-mon[31553]: pgmap v259: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:29.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:29 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:29.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:29 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:29.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:29 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:15:30.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:29 smithi081 ceph-mon[30961]: pgmap v259: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:30.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:29 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:30.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:29 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:30.068 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:29 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:15:30.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:29 smithi069 ceph-mon[28170]: pgmap v259: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:30.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:29 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:30.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:29 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:30.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:29 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:15:31.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:31 smithi130 ceph-mon[31553]: pgmap v260: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:32.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:31 smithi081 ceph-mon[30961]: pgmap v260: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:32.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:31 smithi069 ceph-mon[28170]: pgmap v260: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:32.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:32 smithi130 ceph-mon[31553]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:15:33.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:32 smithi081 ceph-mon[30961]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:15:33.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:32 smithi069 ceph-mon[28170]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:15:33.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:33 smithi130 ceph-mon[31553]: pgmap v261: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:34.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:33 smithi081 ceph-mon[30961]: pgmap v261: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:34.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:33 smithi069 ceph-mon[28170]: pgmap v261: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:35.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:35 smithi130 ceph-mon[31553]: pgmap v262: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:36.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:35 smithi081 ceph-mon[30961]: pgmap v262: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:36.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:35 smithi069 ceph-mon[28170]: pgmap v262: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:37.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:37 smithi130 ceph-mon[31553]: pgmap v263: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:38.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:37 smithi081 ceph-mon[30961]: pgmap v263: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:38.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:37 smithi069 ceph-mon[28170]: pgmap v263: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:39.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:39 smithi130 ceph-mon[31553]: pgmap v264: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:40.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:39 smithi081 ceph-mon[30961]: pgmap v264: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:40.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:39 smithi069 ceph-mon[28170]: pgmap v264: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:41.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:41 smithi130 ceph-mon[31553]: pgmap v265: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:42.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:41 smithi081 ceph-mon[30961]: pgmap v265: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:42.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:41 smithi069 ceph-mon[28170]: pgmap v265: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:43.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:43 smithi130 ceph-mon[31553]: pgmap v266: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:44.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:43 smithi081 ceph-mon[30961]: pgmap v266: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:44.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:43 smithi069 ceph-mon[28170]: pgmap v266: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:45.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:45 smithi130 ceph-mon[31553]: pgmap v267: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:46.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:45 smithi081 ceph-mon[30961]: pgmap v267: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:46.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:45 smithi069 ceph-mon[28170]: pgmap v267: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:47.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:47 smithi130 ceph-mon[31553]: pgmap v268: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:47.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:47 smithi130 ceph-mon[31553]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:15:48.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:47 smithi081 ceph-mon[30961]: pgmap v268: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:48.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:47 smithi081 ceph-mon[30961]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:15:48.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:47 smithi069 ceph-mon[28170]: pgmap v268: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:48.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:47 smithi069 ceph-mon[28170]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:15:50.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:49 smithi081 ceph-mon[30961]: pgmap v269: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:50.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:49 smithi069 ceph-mon[28170]: pgmap v269: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:50.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:49 smithi130 ceph-mon[31553]: pgmap v269: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:52.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:51 smithi081 ceph-mon[30961]: pgmap v270: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:52.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:51 smithi069 ceph-mon[28170]: pgmap v270: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:52.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:51 smithi130 ceph-mon[31553]: pgmap v270: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:54.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:53 smithi081 ceph-mon[30961]: pgmap v271: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:54.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:53 smithi069 ceph-mon[28170]: pgmap v271: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:54.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:53 smithi130 ceph-mon[31553]: pgmap v271: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:56.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:55 smithi081 ceph-mon[30961]: pgmap v272: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:56.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:55 smithi069 ceph-mon[28170]: pgmap v272: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:56.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:55 smithi130 ceph-mon[31553]: pgmap v272: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:15:58.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:57 smithi081 ceph-mon[30961]: pgmap v273: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:58.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:57 smithi069 ceph-mon[28170]: pgmap v273: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:15:58.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:57 smithi130 ceph-mon[31553]: pgmap v273: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:00.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:15:59 smithi081 ceph-mon[30961]: pgmap v274: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:00.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:15:59 smithi069 ceph-mon[28170]: pgmap v274: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:00.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:15:59 smithi130 ceph-mon[31553]: pgmap v274: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:02.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:01 smithi081 ceph-mon[30961]: pgmap v275: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:02.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:01 smithi069 ceph-mon[28170]: pgmap v275: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:02.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:01 smithi130 ceph-mon[31553]: pgmap v275: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:04.007 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:03 smithi130 ceph-mon[31553]: pgmap v276: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:04.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:03 smithi081 ceph-mon[30961]: pgmap v276: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:04.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:03 smithi069 ceph-mon[28170]: pgmap v276: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:06.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:05 smithi081 ceph-mon[30961]: pgmap v277: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:06.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:05 smithi069 ceph-mon[28170]: pgmap v277: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:06.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:05 smithi130 ceph-mon[31553]: pgmap v277: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:08.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:07 smithi081 ceph-mon[30961]: pgmap v278: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T09:16:08.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:07 smithi069 ceph-mon[28170]: pgmap v278: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T09:16:08.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:07 smithi130 ceph-mon[31553]: pgmap v278: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T09:16:09.066 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:08 smithi081 ceph-mon[30961]: osdmap e98: 12 total, 12 up, 12 in 2024-04-24T09:16:09.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:08 smithi069 ceph-mon[28170]: osdmap e98: 12 total, 12 up, 12 in 2024-04-24T09:16:09.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:08 smithi130 ceph-mon[31553]: osdmap e98: 12 total, 12 up, 12 in 2024-04-24T09:16:10.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:09 smithi081 ceph-mon[30961]: pgmap v280: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:16:10.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:09 smithi069 ceph-mon[28170]: pgmap v280: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:16:10.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:09 smithi130 ceph-mon[31553]: pgmap v280: 129 pgs: 129 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:16:12.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:11 smithi069 ceph-mon[28170]: pgmap v281: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:16:12.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:11 smithi130 ceph-mon[31553]: pgmap v281: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:16:12.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:11 smithi081 ceph-mon[30961]: pgmap v281: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:16:14.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:13 smithi069 ceph-mon[28170]: pgmap v282: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:16:14.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:13 smithi069 ceph-mon[28170]: osdmap e99: 12 total, 12 up, 12 in 2024-04-24T09:16:14.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:13 smithi130 ceph-mon[31553]: pgmap v282: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:16:14.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:13 smithi130 ceph-mon[31553]: osdmap e99: 12 total, 12 up, 12 in 2024-04-24T09:16:14.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:13 smithi081 ceph-mon[30961]: pgmap v282: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:16:14.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:13 smithi081 ceph-mon[30961]: osdmap e99: 12 total, 12 up, 12 in 2024-04-24T09:16:15.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:14 smithi130 ceph-mon[31553]: pgmap v284: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T09:16:15.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:14 smithi081 ceph-mon[30961]: pgmap v284: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T09:16:15.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:14 smithi069 ceph-mon[28170]: pgmap v284: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T09:16:17.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:17 smithi081 ceph-mon[30961]: pgmap v285: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 244 B/s wr, 7 op/s 2024-04-24T09:16:17.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:17 smithi069 ceph-mon[28170]: pgmap v285: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 244 B/s wr, 7 op/s 2024-04-24T09:16:17.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:17 smithi130 ceph-mon[31553]: pgmap v285: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 244 B/s wr, 7 op/s 2024-04-24T09:16:19.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:19 smithi081 ceph-mon[30961]: pgmap v286: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T09:16:19.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:19 smithi069 ceph-mon[28170]: pgmap v286: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T09:16:19.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:19 smithi130 ceph-mon[31553]: pgmap v286: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T09:16:21.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:21 smithi081 ceph-mon[30961]: pgmap v287: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T09:16:21.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:21 smithi069 ceph-mon[28170]: pgmap v287: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T09:16:21.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:21 smithi130 ceph-mon[31553]: pgmap v287: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T09:16:23.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:23 smithi081 ceph-mon[30961]: pgmap v288: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T09:16:23.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:23 smithi069 ceph-mon[28170]: pgmap v288: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T09:16:23.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:23 smithi130 ceph-mon[31553]: pgmap v288: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T09:16:25.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:25 smithi081 ceph-mon[30961]: pgmap v289: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 90 B/s wr, 5 op/s 2024-04-24T09:16:25.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:25 smithi069 ceph-mon[28170]: pgmap v289: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 90 B/s wr, 5 op/s 2024-04-24T09:16:25.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:25 smithi130 ceph-mon[31553]: pgmap v289: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 90 B/s wr, 5 op/s 2024-04-24T09:16:27.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:27 smithi081 ceph-mon[30961]: pgmap v290: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-24T09:16:27.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:27 smithi069 ceph-mon[28170]: pgmap v290: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-24T09:16:27.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:27 smithi130 ceph-mon[31553]: pgmap v290: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-24T09:16:29.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:29 smithi081 ceph-mon[30961]: pgmap v291: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:29.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:29 smithi069 ceph-mon[28170]: pgmap v291: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:29.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:29 smithi130 ceph-mon[31553]: pgmap v291: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:30.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:30 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:30.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:30 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:30.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:30 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:31.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:31 smithi081 ceph-mon[30961]: pgmap v292: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:31.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:31 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:31.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:31 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:16:31.568 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:31 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:16:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:31 smithi069 ceph-mon[28170]: pgmap v292: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:31 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:31.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:31 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:16:31.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:31 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:16:31.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:31 smithi130 ceph-mon[31553]: pgmap v292: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:31.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:31 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:31.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:31 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:16:31.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:31 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:16:33.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:33 smithi081 ceph-mon[30961]: pgmap v293: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:33.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:33 smithi069 ceph-mon[28170]: pgmap v293: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:33.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:33 smithi130 ceph-mon[31553]: pgmap v293: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:35.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:35 smithi081 ceph-mon[30961]: pgmap v294: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:35.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:35 smithi069 ceph-mon[28170]: pgmap v294: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:35.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:35 smithi130 ceph-mon[31553]: pgmap v294: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:37.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:37 smithi081 ceph-mon[30961]: pgmap v295: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:37.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:37 smithi069 ceph-mon[28170]: pgmap v295: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:37.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:37 smithi130 ceph-mon[31553]: pgmap v295: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:39.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:39 smithi081 ceph-mon[30961]: pgmap v296: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:39.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:39 smithi081 ceph-mon[30961]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:16:39.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:39 smithi069 ceph-mon[28170]: pgmap v296: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:39.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:39 smithi069 ceph-mon[28170]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:16:39.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:39 smithi130 ceph-mon[31553]: pgmap v296: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:39.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:39 smithi130 ceph-mon[31553]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:16:41.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:41 smithi081 ceph-mon[30961]: pgmap v297: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:41.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:41 smithi069 ceph-mon[28170]: pgmap v297: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:41.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:41 smithi130 ceph-mon[31553]: pgmap v297: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:43.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:43 smithi069 ceph-mon[28170]: pgmap v298: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:43.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:43 smithi130 ceph-mon[31553]: pgmap v298: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:43.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:43 smithi081 ceph-mon[30961]: pgmap v298: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:45.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:45 smithi069 ceph-mon[28170]: pgmap v299: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:45.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:45 smithi130 ceph-mon[31553]: pgmap v299: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:45.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:45 smithi081 ceph-mon[30961]: pgmap v299: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:47.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:47 smithi069 ceph-mon[28170]: pgmap v300: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:47.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:47 smithi130 ceph-mon[31553]: pgmap v300: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:47.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:47 smithi081 ceph-mon[30961]: pgmap v300: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:49.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:49 smithi130 ceph-mon[31553]: pgmap v301: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:49.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:49 smithi130 ceph-mon[31553]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:16:49.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:49 smithi081 ceph-mon[30961]: pgmap v301: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:49.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:49 smithi081 ceph-mon[30961]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:16:49.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:49 smithi069 ceph-mon[28170]: pgmap v301: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:49.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:49 smithi069 ceph-mon[28170]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T09:16:51.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:51 smithi130 ceph-mon[31553]: pgmap v302: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:51.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:51 smithi081 ceph-mon[30961]: pgmap v302: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:51.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:51 smithi069 ceph-mon[28170]: pgmap v302: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:53.644 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:53 smithi069 ceph-mon[28170]: pgmap v303: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:53.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:53 smithi130 ceph-mon[31553]: pgmap v303: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:53.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:53 smithi081 ceph-mon[30961]: pgmap v303: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:55.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:55 smithi130 ceph-mon[31553]: pgmap v304: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:55.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:55 smithi081 ceph-mon[30961]: pgmap v304: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:55.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:55 smithi069 ceph-mon[28170]: pgmap v304: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:57.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:57 smithi130 ceph-mon[31553]: pgmap v305: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:57.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:57 smithi081 ceph-mon[30961]: pgmap v305: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:57.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:57 smithi069 ceph-mon[28170]: pgmap v305: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:16:59.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:16:59 smithi130 ceph-mon[31553]: pgmap v306: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:59.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:16:59 smithi081 ceph-mon[30961]: pgmap v306: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:16:59.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:16:59 smithi069 ceph-mon[28170]: pgmap v306: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:01.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:01 smithi130 ceph-mon[31553]: pgmap v307: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:01.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:01 smithi081 ceph-mon[30961]: pgmap v307: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:01.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:01 smithi069 ceph-mon[28170]: pgmap v307: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:03.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:03 smithi130 ceph-mon[31553]: pgmap v308: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:03 smithi081 ceph-mon[30961]: pgmap v308: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:03.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:03 smithi069 ceph-mon[28170]: pgmap v308: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:05.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:05 smithi130 ceph-mon[31553]: pgmap v309: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:05.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:05 smithi081 ceph-mon[30961]: pgmap v309: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:05.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:05 smithi069 ceph-mon[28170]: pgmap v309: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:07.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:07 smithi130 ceph-mon[31553]: pgmap v310: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T09:17:07.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:07 smithi081 ceph-mon[30961]: pgmap v310: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T09:17:07.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:07 smithi069 ceph-mon[28170]: pgmap v310: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T09:17:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:09 smithi130 ceph-mon[31553]: pgmap v311: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T09:17:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:09 smithi130 ceph-mon[31553]: osdmap e100: 12 total, 12 up, 12 in 2024-04-24T09:17:09.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:09 smithi081 ceph-mon[30961]: pgmap v311: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T09:17:09.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:09 smithi081 ceph-mon[30961]: osdmap e100: 12 total, 12 up, 12 in 2024-04-24T09:17:09.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:09 smithi069 ceph-mon[28170]: pgmap v311: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T09:17:09.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:09 smithi069 ceph-mon[28170]: osdmap e100: 12 total, 12 up, 12 in 2024-04-24T09:17:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:11 smithi130 ceph-mon[31553]: pgmap v313: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:17:11.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:11 smithi081 ceph-mon[30961]: pgmap v313: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:17:11.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:11 smithi069 ceph-mon[28170]: pgmap v313: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:17:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:13 smithi130 ceph-mon[31553]: pgmap v314: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:17:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:13 smithi130 ceph-mon[31553]: osdmap e101: 12 total, 12 up, 12 in 2024-04-24T09:17:13.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:13 smithi081 ceph-mon[30961]: pgmap v314: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:17:13.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:13 smithi081 ceph-mon[30961]: osdmap e101: 12 total, 12 up, 12 in 2024-04-24T09:17:13.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:13 smithi069 ceph-mon[28170]: pgmap v314: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T09:17:13.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:13 smithi069 ceph-mon[28170]: osdmap e101: 12 total, 12 up, 12 in 2024-04-24T09:17:14.455 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-24T09:17:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.655+0000 7f05f2e00640 1 -- 172.21.15.69:0/731300948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05ec102250 msgr2=0x7f05ec102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.655+0000 7f05f2e00640 1 --2- 172.21.15.69:0/731300948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05ec102250 0x7f05ec102630 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009ca0 tx=0x7f05dc02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.656+0000 7f05f2e00640 1 -- 172.21.15.69:0/731300948 shutdown_connections 2024-04-24T09:17:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.656+0000 7f05f2e00640 1 --2- 172.21.15.69:0/731300948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05ec107320 0x7f05ec10eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.656+0000 7f05f2e00640 1 --2- 172.21.15.69:0/731300948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05ec102c00 0x7f05ec106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.656+0000 7f05f2e00640 1 --2- 172.21.15.69:0/731300948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05ec102250 0x7f05ec102630 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.656+0000 7f05f2e00640 1 -- 172.21.15.69:0/731300948 >> 172.21.15.69:0/731300948 conn(0x7f05ec079490 msgr2=0x7f05ec0798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:14.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.656+0000 7f05f2e00640 1 -- 172.21.15.69:0/731300948 shutdown_connections 2024-04-24T09:17:14.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.656+0000 7f05f2e00640 1 -- 172.21.15.69:0/731300948 wait complete. 2024-04-24T09:17:14.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.657+0000 7f05f2e00640 1 Processor -- start 2024-04-24T09:17:14.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.657+0000 7f05f2e00640 1 -- start start 2024-04-24T09:17:14.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05f2e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05ec102250 0x7f05ec198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:14.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05f2e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05ec107320 0x7f05ec198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:14.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05f2e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05ec19cfc0 0x7f05ec19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:14.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05f2e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec112290 con 0x7f05ec107320 2024-04-24T09:17:14.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05f2e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec112110 con 0x7f05ec19cfc0 2024-04-24T09:17:14.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05f2e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec112410 con 0x7f05ec102250 2024-04-24T09:17:14.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05eb400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05ec107320 0x7f05ec198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:14.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05eb400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05ec107320 0x7f05ec198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37674/0 (socket says 172.21.15.69:37674) 2024-04-24T09:17:14.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05eb400640 1 -- 172.21.15.69:0/1362847551 learned_addr learned my addr 172.21.15.69:0/1362847551 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:17:14.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05ebe00640 1 --2- 172.21.15.69:0/1362847551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05ec102250 0x7f05ec198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:14.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05f0e00640 1 --2- 172.21.15.69:0/1362847551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05ec19cfc0 0x7f05ec19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:14.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05eb400640 1 -- 172.21.15.69:0/1362847551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05ec102250 msgr2=0x7f05ec198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:14.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05eb400640 1 --2- 172.21.15.69:0/1362847551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05ec102250 0x7f05ec198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05eb400640 1 -- 172.21.15.69:0/1362847551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05ec19cfc0 msgr2=0x7f05ec19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:14.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05eb400640 1 --2- 172.21.15.69:0/1362847551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05ec19cfc0 0x7f05ec19d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05eb400640 1 -- 172.21.15.69:0/1362847551 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec19dab0 con 0x7f05ec107320 2024-04-24T09:17:14.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.658+0000 7f05f0e00640 1 --2- 172.21.15.69:0/1362847551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05ec19cfc0 0x7f05ec19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:14.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.659+0000 7f05eb400640 1 --2- 172.21.15.69:0/1362847551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05ec107320 0x7f05ec198c90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f05d800c970 tx=0x7f05d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:14.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.659+0000 7f05e8c00640 1 -- 172.21.15.69:0/1362847551 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d8014070 con 0x7f05ec107320 2024-04-24T09:17:14.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.659+0000 7f05e8c00640 1 -- 172.21.15.69:0/1362847551 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05d8004440 con 0x7f05ec107320 2024-04-24T09:17:14.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.659+0000 7f05e8c00640 1 -- 172.21.15.69:0/1362847551 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d8003e50 con 0x7f05ec107320 2024-04-24T09:17:14.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.659+0000 7f05f2e00640 1 -- 172.21.15.69:0/1362847551 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ec19dda0 con 0x7f05ec107320 2024-04-24T09:17:14.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.659+0000 7f05f2e00640 1 -- 172.21.15.69:0/1362847551 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ec1a55e0 con 0x7f05ec107320 2024-04-24T09:17:14.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.660+0000 7f05f2e00640 1 -- 172.21.15.69:0/1362847551 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ec04fd90 con 0x7f05ec107320 2024-04-24T09:17:14.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.663+0000 7f05e8c00640 1 -- 172.21.15.69:0/1362847551 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f05d8017070 con 0x7f05ec107320 2024-04-24T09:17:14.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.665+0000 7f05e8c00640 1 --2- 172.21.15.69:0/1362847551 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f05d00a3fe0 0x7f05d00a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:14.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.665+0000 7f05e8c00640 1 -- 172.21.15.69:0/1362847551 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f05d80d0a80 con 0x7f05ec107320 2024-04-24T09:17:14.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.665+0000 7f05ebe00640 1 --2- 172.21.15.69:0/1362847551 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f05d00a3fe0 0x7f05d00a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:14.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.666+0000 7f05ebe00640 1 --2- 172.21.15.69:0/1362847551 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f05d00a3fe0 0x7f05d00a64a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f05dc000f80 tx=0x7f05dc005ef0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:14.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.669+0000 7f05e8c00640 1 -- 172.21.15.69:0/1362847551 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05d80d2e20 con 0x7f05ec107320 2024-04-24T09:17:14.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.895+0000 7f05f2e00640 1 -- 172.21.15.69:0/1362847551 --> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f05ec102c00 con 0x7f05d00a3fe0 2024-04-24T09:17:14.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.913+0000 7f05e8c00640 1 -- 172.21.15.69:0/1362847551 <== mgr.15465 v2:172.21.15.69:6800/2822728880 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f05ec102c00 con 0x7f05d00a3fe0 2024-04-24T09:17:14.913 INFO:teuthology.orchestra.run.smithi069.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-24T00:00:00", "created": "2024-04-24T09:07:12", "first": "2024-04-24T09:08:00", "last": "2024-04-24T09:17:00", "last_pruned": "2024-04-24T09:17:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-24T09:17:14.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.915+0000 7f05f2e00640 1 -- 172.21.15.69:0/1362847551 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f05d00a3fe0 msgr2=0x7f05d00a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:14.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.915+0000 7f05f2e00640 1 --2- 172.21.15.69:0/1362847551 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f05d00a3fe0 0x7f05d00a64a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f05dc000f80 tx=0x7f05dc005ef0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.916+0000 7f05f2e00640 1 -- 172.21.15.69:0/1362847551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05ec107320 msgr2=0x7f05ec198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:14.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.916+0000 7f05f2e00640 1 --2- 172.21.15.69:0/1362847551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05ec107320 0x7f05ec198c90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f05d800c970 tx=0x7f05d800ce30 comp rx=0 tx=0).stop 2024-04-24T09:17:14.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.916+0000 7f05f2e00640 1 -- 172.21.15.69:0/1362847551 shutdown_connections 2024-04-24T09:17:14.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.916+0000 7f05f2e00640 1 --2- 172.21.15.69:0/1362847551 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f05d00a3fe0 0x7f05d00a64a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.916+0000 7f05f2e00640 1 --2- 172.21.15.69:0/1362847551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05ec19cfc0 0x7f05ec19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.916+0000 7f05f2e00640 1 --2- 172.21.15.69:0/1362847551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05ec107320 0x7f05ec198c90 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.916+0000 7f05f2e00640 1 --2- 172.21.15.69:0/1362847551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05ec102250 0x7f05ec198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.916+0000 7f05f2e00640 1 -- 172.21.15.69:0/1362847551 >> 172.21.15.69:0/1362847551 conn(0x7f05ec079490 msgr2=0x7f05ec0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:14.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.916+0000 7f05f2e00640 1 -- 172.21.15.69:0/1362847551 shutdown_connections 2024-04-24T09:17:14.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:14.916+0000 7f05f2e00640 1 -- 172.21.15.69:0/1362847551 wait complete. 2024-04-24T09:17:14.935 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-24T09:17:15.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.130+0000 7f5ff2400640 1 -- 172.21.15.69:0/1008338060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec0743d0 msgr2=0x7f5fec0747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:15.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.130+0000 7f5ff2400640 1 --2- 172.21.15.69:0/1008338060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec0743d0 0x7f5fec0747b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009900 tx=0x7f5fdc02a480 comp rx=0 tx=0).stop 2024-04-24T09:17:15.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.130+0000 7f5ff2400640 1 -- 172.21.15.69:0/1008338060 shutdown_connections 2024-04-24T09:17:15.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.130+0000 7f5ff2400640 1 --2- 172.21.15.69:0/1008338060 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5fec10e6b0 0x7f5fec110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.130+0000 7f5ff2400640 1 --2- 172.21.15.69:0/1008338060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fec074cf0 0x7f5fec10e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.130+0000 7f5ff2400640 1 --2- 172.21.15.69:0/1008338060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec0743d0 0x7f5fec0747b0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.130+0000 7f5ff2400640 1 -- 172.21.15.69:0/1008338060 >> 172.21.15.69:0/1008338060 conn(0x7f5fec0fd940 msgr2=0x7f5fec0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:15.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.130+0000 7f5ff2400640 1 -- 172.21.15.69:0/1008338060 shutdown_connections 2024-04-24T09:17:15.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.130+0000 7f5ff2400640 1 -- 172.21.15.69:0/1008338060 wait complete. 2024-04-24T09:17:15.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.131+0000 7f5ff2400640 1 Processor -- start 2024-04-24T09:17:15.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.131+0000 7f5ff2400640 1 -- start start 2024-04-24T09:17:15.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.132+0000 7f5ff2400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec0743d0 0x7f5fec1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:15.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.132+0000 7f5ff2400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5fec074cf0 0x7f5fec1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:15.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.132+0000 7f5ff2400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fec10e6b0 0x7f5fec19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:15.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.132+0000 7f5ff2400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1142e0 con 0x7f5fec0743d0 2024-04-24T09:17:15.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.132+0000 7f5ff2400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec114160 con 0x7f5fec10e6b0 2024-04-24T09:17:15.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.132+0000 7f5ff2400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec114460 con 0x7f5fec074cf0 2024-04-24T09:17:15.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.132+0000 7f5ff1000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec0743d0 0x7f5fec1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:15.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.132+0000 7f5ff1000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec0743d0 0x7f5fec1a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37702/0 (socket says 172.21.15.69:37702) 2024-04-24T09:17:15.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.132+0000 7f5ff1000640 1 -- 172.21.15.69:0/802589615 learned_addr learned my addr 172.21.15.69:0/802589615 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:17:15.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.132+0000 7f5fe3e00640 1 --2- 172.21.15.69:0/802589615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5fec074cf0 0x7f5fec1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:15.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.132+0000 7f5ff1a00640 1 --2- 172.21.15.69:0/802589615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fec10e6b0 0x7f5fec19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:15.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.133+0000 7f5ff1000640 1 -- 172.21.15.69:0/802589615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5fec074cf0 msgr2=0x7f5fec1a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:15.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.133+0000 7f5ff1000640 1 --2- 172.21.15.69:0/802589615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5fec074cf0 0x7f5fec1a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.133+0000 7f5ff1000640 1 -- 172.21.15.69:0/802589615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fec10e6b0 msgr2=0x7f5fec19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:15.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.133+0000 7f5ff1000640 1 --2- 172.21.15.69:0/802589615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fec10e6b0 0x7f5fec19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.133+0000 7f5ff1000640 1 -- 172.21.15.69:0/802589615 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fec19af80 con 0x7f5fec0743d0 2024-04-24T09:17:15.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.133+0000 7f5fe3e00640 1 --2- 172.21.15.69:0/802589615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5fec074cf0 0x7f5fec1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:15.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.133+0000 7f5ff1a00640 1 --2- 172.21.15.69:0/802589615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fec10e6b0 0x7f5fec19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:15.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.133+0000 7f5ff1000640 1 --2- 172.21.15.69:0/802589615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec0743d0 0x7f5fec1a0600 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc005940 tx=0x7f5fdc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:15.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.133+0000 7f5fea000640 1 -- 172.21.15.69:0/802589615 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc03f030 con 0x7f5fec0743d0 2024-04-24T09:17:15.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.133+0000 7f5fea000640 1 -- 172.21.15.69:0/802589615 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fdc003aa0 con 0x7f5fec0743d0 2024-04-24T09:17:15.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.133+0000 7f5ff2400640 1 -- 172.21.15.69:0/802589615 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fec19b210 con 0x7f5fec0743d0 2024-04-24T09:17:15.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.133+0000 7f5fea000640 1 -- 172.21.15.69:0/802589615 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc035070 con 0x7f5fec0743d0 2024-04-24T09:17:15.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.133+0000 7f5ff2400640 1 -- 172.21.15.69:0/802589615 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fec1a73a0 con 0x7f5fec0743d0 2024-04-24T09:17:15.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.135+0000 7f5ff2400640 1 -- 172.21.15.69:0/802589615 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fec075a80 con 0x7f5fec0743d0 2024-04-24T09:17:15.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.137+0000 7f5fea000640 1 -- 172.21.15.69:0/802589615 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5fdc042030 con 0x7f5fec0743d0 2024-04-24T09:17:15.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.138+0000 7f5fea000640 1 --2- 172.21.15.69:0/802589615 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5fd00a3d10 0x7f5fd00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:15.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.138+0000 7f5fea000640 1 -- 172.21.15.69:0/802589615 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f5fdc0f1230 con 0x7f5fec0743d0 2024-04-24T09:17:15.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.138+0000 7f5fe3e00640 1 --2- 172.21.15.69:0/802589615 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5fd00a3d10 0x7f5fd00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:15.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.139+0000 7f5fe3e00640 1 --2- 172.21.15.69:0/802589615 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5fd00a3d10 0x7f5fd00a61d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5fec19bf10 tx=0x7f5fd4005d30 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:15.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.144+0000 7f5fea000640 1 -- 172.21.15.69:0/802589615 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fdc0bd4e0 con 0x7f5fec0743d0 2024-04-24T09:17:15.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.371+0000 7f5ff2400640 1 -- 172.21.15.69:0/802589615 --> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5fec1098a0 con 0x7f5fd00a3d10 2024-04-24T09:17:15.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.388+0000 7f5fea000640 1 -- 172.21.15.69:0/802589615 <== mgr.15465 v2:172.21.15.69:6800/2822728880 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5fd400b6b0 con 0x7f5fd00a3d10 2024-04-24T09:17:15.388 INFO:teuthology.orchestra.run.smithi069.stdout:/ 1m 6m3h 2024-04-24T09:17:15.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.391+0000 7f5ff2400640 1 -- 172.21.15.69:0/802589615 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5fd00a3d10 msgr2=0x7f5fd00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:15.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.391+0000 7f5ff2400640 1 --2- 172.21.15.69:0/802589615 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5fd00a3d10 0x7f5fd00a61d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5fec19bf10 tx=0x7f5fd4005d30 comp rx=0 tx=0).stop 2024-04-24T09:17:15.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.391+0000 7f5ff2400640 1 -- 172.21.15.69:0/802589615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec0743d0 msgr2=0x7f5fec1a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:15.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.391+0000 7f5ff2400640 1 --2- 172.21.15.69:0/802589615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec0743d0 0x7f5fec1a0600 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc005940 tx=0x7f5fdc004270 comp rx=0 tx=0).stop 2024-04-24T09:17:15.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.391+0000 7f5ff2400640 1 -- 172.21.15.69:0/802589615 shutdown_connections 2024-04-24T09:17:15.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.391+0000 7f5ff2400640 1 --2- 172.21.15.69:0/802589615 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f5fd00a3d10 0x7f5fd00a61d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.391+0000 7f5ff2400640 1 --2- 172.21.15.69:0/802589615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fec10e6b0 0x7f5fec19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.391+0000 7f5ff2400640 1 --2- 172.21.15.69:0/802589615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5fec074cf0 0x7f5fec1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.391+0000 7f5ff2400640 1 --2- 172.21.15.69:0/802589615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec0743d0 0x7f5fec1a0600 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.391+0000 7f5ff2400640 1 -- 172.21.15.69:0/802589615 >> 172.21.15.69:0/802589615 conn(0x7f5fec0fd940 msgr2=0x7f5fec0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:15.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.392+0000 7f5ff2400640 1 -- 172.21.15.69:0/802589615 shutdown_connections 2024-04-24T09:17:15.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.392+0000 7f5ff2400640 1 -- 172.21.15.69:0/802589615 wait complete. 2024-04-24T09:17:15.412 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-24T09:17:15.423 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-24T09:17:15.423 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-04-24T09:17:15.429 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-24T09:17:15.440 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:17:15.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.653+0000 7f85b0e00640 1 -- 172.21.15.69:0/2150096029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85ac0743a0 msgr2=0x7f85ac074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:15.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.653+0000 7f85b0e00640 1 --2- 172.21.15.69:0/2150096029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85ac0743a0 0x7f85ac074780 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f859c009900 tx=0x7f859c02a490 comp rx=0 tx=0).stop 2024-04-24T09:17:15.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.653+0000 7f85b0e00640 1 -- 172.21.15.69:0/2150096029 shutdown_connections 2024-04-24T09:17:15.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.653+0000 7f85b0e00640 1 --2- 172.21.15.69:0/2150096029 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85ac112e70 0x7f85ac115260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.653+0000 7f85b0e00640 1 --2- 172.21.15.69:0/2150096029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ac074cc0 0x7f85ac112930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.653+0000 7f85b0e00640 1 --2- 172.21.15.69:0/2150096029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85ac0743a0 0x7f85ac074780 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.653+0000 7f85b0e00640 1 -- 172.21.15.69:0/2150096029 >> 172.21.15.69:0/2150096029 conn(0x7f85ac0fd970 msgr2=0x7f85ac0ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:15.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.653+0000 7f85b0e00640 1 -- 172.21.15.69:0/2150096029 shutdown_connections 2024-04-24T09:17:15.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.653+0000 7f85b0e00640 1 -- 172.21.15.69:0/2150096029 wait complete. 2024-04-24T09:17:15.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.654+0000 7f85b0e00640 1 Processor -- start 2024-04-24T09:17:15.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.654+0000 7f85b0e00640 1 -- start start 2024-04-24T09:17:15.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.654+0000 7f85b0e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ac0743a0 0x7f85ac19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:15.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.654+0000 7f85b0e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85ac074cc0 0x7f85ac19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:15.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.654+0000 7f85b0e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85ac112e70 0x7f85ac1a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:15.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.654+0000 7f85b0e00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1187b0 con 0x7f85ac074cc0 2024-04-24T09:17:15.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.654+0000 7f85b0e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac118630 con 0x7f85ac0743a0 2024-04-24T09:17:15.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.654+0000 7f85b0e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac118930 con 0x7f85ac112e70 2024-04-24T09:17:15.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.655+0000 7f85aaa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85ac074cc0 0x7f85ac19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:15.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.655+0000 7f85aaa00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85ac074cc0 0x7f85ac19f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37706/0 (socket says 172.21.15.69:37706) 2024-04-24T09:17:15.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.655+0000 7f85aaa00640 1 -- 172.21.15.69:0/352900272 learned_addr learned my addr 172.21.15.69:0/352900272 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:17:15.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.655+0000 7f85ab400640 1 --2- 172.21.15.69:0/352900272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ac0743a0 0x7f85ac19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:15.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.655+0000 7f85abe00640 1 --2- 172.21.15.69:0/352900272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85ac112e70 0x7f85ac1a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:15.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.655+0000 7f85aaa00640 1 -- 172.21.15.69:0/352900272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85ac112e70 msgr2=0x7f85ac1a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:15.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.655+0000 7f85aaa00640 1 --2- 172.21.15.69:0/352900272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85ac112e70 0x7f85ac1a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.655+0000 7f85aaa00640 1 -- 172.21.15.69:0/352900272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ac0743a0 msgr2=0x7f85ac19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:15.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.655+0000 7f85aaa00640 1 --2- 172.21.15.69:0/352900272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ac0743a0 0x7f85ac19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.655+0000 7f85aaa00640 1 -- 172.21.15.69:0/352900272 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ac1a3d10 con 0x7f85ac074cc0 2024-04-24T09:17:15.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.655+0000 7f85abe00640 1 --2- 172.21.15.69:0/352900272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85ac112e70 0x7f85ac1a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:15.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.655+0000 7f85aaa00640 1 --2- 172.21.15.69:0/352900272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85ac074cc0 0x7f85ac19f2b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f859800c890 tx=0x7f859800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:15.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.656+0000 7f85a3e00640 1 -- 172.21.15.69:0/352900272 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598004220 con 0x7f85ac074cc0 2024-04-24T09:17:15.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.656+0000 7f85a3e00640 1 -- 172.21.15.69:0/352900272 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85980043c0 con 0x7f85ac074cc0 2024-04-24T09:17:15.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.656+0000 7f85a3e00640 1 -- 172.21.15.69:0/352900272 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598004a00 con 0x7f85ac074cc0 2024-04-24T09:17:15.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.656+0000 7f85b0e00640 1 -- 172.21.15.69:0/352900272 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ac1a4000 con 0x7f85ac074cc0 2024-04-24T09:17:15.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.656+0000 7f85b0e00640 1 -- 172.21.15.69:0/352900272 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ac1ab890 con 0x7f85ac074cc0 2024-04-24T09:17:15.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.657+0000 7f85b0e00640 1 -- 172.21.15.69:0/352900272 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85ac069190 con 0x7f85ac074cc0 2024-04-24T09:17:15.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.660+0000 7f85a3e00640 1 -- 172.21.15.69:0/352900272 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8598004ba0 con 0x7f85ac074cc0 2024-04-24T09:17:15.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.661+0000 7f85a3e00640 1 --2- 172.21.15.69:0/352900272 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f85900a3da0 0x7f85900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:15.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.661+0000 7f85a3e00640 1 -- 172.21.15.69:0/352900272 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f85980d08c0 con 0x7f85ac074cc0 2024-04-24T09:17:15.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.662+0000 7f85ab400640 1 --2- 172.21.15.69:0/352900272 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f85900a3da0 0x7f85900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:15.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.662+0000 7f85ab400640 1 --2- 172.21.15.69:0/352900272 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f85900a3da0 0x7f85900a6260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f859c000c00 tx=0x7f859c031400 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:15.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.666+0000 7f85a3e00640 1 -- 172.21.15.69:0/352900272 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8598014030 con 0x7f85ac074cc0 2024-04-24T09:17:15.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:15 smithi130 ceph-mon[31553]: pgmap v316: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T09:17:15.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:15 smithi081 ceph-mon[30961]: pgmap v316: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T09:17:15.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:15 smithi069 ceph-mon[28170]: pgmap v316: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T09:17:15.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.869+0000 7f85b0e00640 1 -- 172.21.15.69:0/352900272 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85ac104d50 con 0x7f85ac074cc0 2024-04-24T09:17:15.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.873+0000 7f85a3e00640 1 -- 172.21.15.69:0/352900272 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+25127 (secure 0 0 0) 0x7f859809cb70 con 0x7f85ac074cc0 2024-04-24T09:17:15.873 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:17:15.874 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":101,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:17:12.920525+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/660205566":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/1098677190":"2024-04-25T09:07:03.863940+0000","172.21.15.69:0/3145713060":"2024-04-25T09:07:03.863940+0000","172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/3008138297":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/2901345957":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/1098677190":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:17:15.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.876+0000 7f85b0e00640 1 -- 172.21.15.69:0/352900272 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f85900a3da0 msgr2=0x7f85900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:15.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.876+0000 7f85b0e00640 1 --2- 172.21.15.69:0/352900272 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f85900a3da0 0x7f85900a6260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f859c000c00 tx=0x7f859c031400 comp rx=0 tx=0).stop 2024-04-24T09:17:15.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.876+0000 7f85b0e00640 1 -- 172.21.15.69:0/352900272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85ac074cc0 msgr2=0x7f85ac19f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:15.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.876+0000 7f85b0e00640 1 --2- 172.21.15.69:0/352900272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85ac074cc0 0x7f85ac19f2b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f859800c890 tx=0x7f859800cd50 comp rx=0 tx=0).stop 2024-04-24T09:17:15.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.876+0000 7f85b0e00640 1 -- 172.21.15.69:0/352900272 shutdown_connections 2024-04-24T09:17:15.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.876+0000 7f85b0e00640 1 --2- 172.21.15.69:0/352900272 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f85900a3da0 0x7f85900a6260 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.876+0000 7f85b0e00640 1 --2- 172.21.15.69:0/352900272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85ac112e70 0x7f85ac1a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.876+0000 7f85b0e00640 1 --2- 172.21.15.69:0/352900272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85ac074cc0 0x7f85ac19f2b0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.876+0000 7f85b0e00640 1 --2- 172.21.15.69:0/352900272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ac0743a0 0x7f85ac19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.876+0000 7f85b0e00640 1 -- 172.21.15.69:0/352900272 >> 172.21.15.69:0/352900272 conn(0x7f85ac0fd970 msgr2=0x7f85ac0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:15.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.876+0000 7f85b0e00640 1 -- 172.21.15.69:0/352900272 shutdown_connections 2024-04-24T09:17:15.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:15.876+0000 7f85b0e00640 1 -- 172.21.15.69:0/352900272 wait complete. 2024-04-24T09:17:15.894 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:57:37.377599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:04:00.299827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:04:00.958498+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T09:04:04.105314+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T09:17:15.895 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:17:16.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.101+0000 7f40c4c00640 1 -- 172.21.15.69:0/1153596041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40c0109520 msgr2=0x7f40c01110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:16.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.101+0000 7f40c4c00640 1 --2- 172.21.15.69:0/1153596041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40c0109520 0x7f40c01110f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009ca0 tx=0x7f40b802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.101+0000 7f40c4c00640 1 -- 172.21.15.69:0/1153596041 shutdown_connections 2024-04-24T09:17:16.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.101+0000 7f40c4c00640 1 --2- 172.21.15.69:0/1153596041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40c0109520 0x7f40c01110f0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.101+0000 7f40c4c00640 1 --2- 172.21.15.69:0/1153596041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40c0104e00 0x7f40c0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.101+0000 7f40c4c00640 1 --2- 172.21.15.69:0/1153596041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40c0104450 0x7f40c0104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.101+0000 7f40c4c00640 1 -- 172.21.15.69:0/1153596041 >> 172.21.15.69:0/1153596041 conn(0x7f40c00ffc70 msgr2=0x7f40c0102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:16.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.101+0000 7f40c4c00640 1 -- 172.21.15.69:0/1153596041 shutdown_connections 2024-04-24T09:17:16.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.101+0000 7f40c4c00640 1 -- 172.21.15.69:0/1153596041 wait complete. 2024-04-24T09:17:16.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.102+0000 7f40c4c00640 1 Processor -- start 2024-04-24T09:17:16.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.102+0000 7f40c4c00640 1 -- start start 2024-04-24T09:17:16.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.103+0000 7f40c4c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40c0104450 0x7f40c01a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:16.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.103+0000 7f40c4c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40c0109520 0x7f40c01a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:16.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.103+0000 7f40c4c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40c019a7b0 0x7f40c019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:16.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.103+0000 7f40c4c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f40c0114490 con 0x7f40c019a7b0 2024-04-24T09:17:16.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.103+0000 7f40c4c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f40c0114310 con 0x7f40c0104450 2024-04-24T09:17:16.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.103+0000 7f40c4c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f40c0114610 con 0x7f40c0109520 2024-04-24T09:17:16.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.103+0000 7f40be800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40c019a7b0 0x7f40c019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:16.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.103+0000 7f40be800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40c019a7b0 0x7f40c019ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37718/0 (socket says 172.21.15.69:37718) 2024-04-24T09:17:16.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.103+0000 7f40be800640 1 -- 172.21.15.69:0/949771591 learned_addr learned my addr 172.21.15.69:0/949771591 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:17:16.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.103+0000 7f40bde00640 1 --2- 172.21.15.69:0/949771591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40c0104450 0x7f40c01a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:16.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.103+0000 7f40bd400640 1 --2- 172.21.15.69:0/949771591 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40c0109520 0x7f40c01a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:16.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.104+0000 7f40be800640 1 -- 172.21.15.69:0/949771591 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40c0109520 msgr2=0x7f40c01a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:16.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.104+0000 7f40be800640 1 --2- 172.21.15.69:0/949771591 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40c0109520 0x7f40c01a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.104+0000 7f40be800640 1 -- 172.21.15.69:0/949771591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40c0104450 msgr2=0x7f40c01a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:16.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.104+0000 7f40be800640 1 --2- 172.21.15.69:0/949771591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40c0104450 0x7f40c01a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.104+0000 7f40be800640 1 -- 172.21.15.69:0/949771591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c019b3b0 con 0x7f40c019a7b0 2024-04-24T09:17:16.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.104+0000 7f40be800640 1 --2- 172.21.15.69:0/949771591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40c019a7b0 0x7f40c019ac30 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009dc0 tx=0x7f40b8032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:16.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.104+0000 7f40b6a00640 1 -- 172.21.15.69:0/949771591 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8003cf0 con 0x7f40c019a7b0 2024-04-24T09:17:16.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.104+0000 7f40b6a00640 1 -- 172.21.15.69:0/949771591 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40b80044f0 con 0x7f40c019a7b0 2024-04-24T09:17:16.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.104+0000 7f40c4c00640 1 -- 172.21.15.69:0/949771591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c019b640 con 0x7f40c019a7b0 2024-04-24T09:17:16.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.104+0000 7f40b6a00640 1 -- 172.21.15.69:0/949771591 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8030bf0 con 0x7f40c019a7b0 2024-04-24T09:17:16.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.104+0000 7f40c4c00640 1 -- 172.21.15.69:0/949771591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c01a7790 con 0x7f40c019a7b0 2024-04-24T09:17:16.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.106+0000 7f40c4c00640 1 -- 172.21.15.69:0/949771591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40c0105920 con 0x7f40c019a7b0 2024-04-24T09:17:16.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.108+0000 7f40b6a00640 1 -- 172.21.15.69:0/949771591 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f40b8004040 con 0x7f40c019a7b0 2024-04-24T09:17:16.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.109+0000 7f40b6a00640 1 --2- 172.21.15.69:0/949771591 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f40a40a3d20 0x7f40a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:16.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.109+0000 7f40b6a00640 1 -- 172.21.15.69:0/949771591 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f40b80f21f0 con 0x7f40c019a7b0 2024-04-24T09:17:16.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.109+0000 7f40bde00640 1 --2- 172.21.15.69:0/949771591 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f40a40a3d20 0x7f40a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:16.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.110+0000 7f40bde00640 1 --2- 172.21.15.69:0/949771591 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f40a40a3d20 0x7f40a40a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005fd0 tx=0x7f40b0005850 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:16.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.117+0000 7f40b6a00640 1 -- 172.21.15.69:0/949771591 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40b80be5b0 con 0x7f40c019a7b0 2024-04-24T09:17:16.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.319+0000 7f40c4c00640 1 -- 172.21.15.69:0/949771591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f40c0104e00 con 0x7f40c019a7b0 2024-04-24T09:17:16.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.320+0000 7f40b6a00640 1 -- 172.21.15.69:0/949771591 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f40b80bdeb0 con 0x7f40c019a7b0 2024-04-24T09:17:16.320 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-24T09:17:16.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.323+0000 7f40c4c00640 1 -- 172.21.15.69:0/949771591 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f40a40a3d20 msgr2=0x7f40a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:16.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.323+0000 7f40c4c00640 1 --2- 172.21.15.69:0/949771591 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f40a40a3d20 0x7f40a40a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005fd0 tx=0x7f40b0005850 comp rx=0 tx=0).stop 2024-04-24T09:17:16.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.323+0000 7f40c4c00640 1 -- 172.21.15.69:0/949771591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40c019a7b0 msgr2=0x7f40c019ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:16.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.323+0000 7f40c4c00640 1 --2- 172.21.15.69:0/949771591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40c019a7b0 0x7f40c019ac30 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009dc0 tx=0x7f40b8032ff0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.323+0000 7f40c4c00640 1 -- 172.21.15.69:0/949771591 shutdown_connections 2024-04-24T09:17:16.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.323+0000 7f40c4c00640 1 --2- 172.21.15.69:0/949771591 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f40a40a3d20 0x7f40a40a61e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.323+0000 7f40c4c00640 1 --2- 172.21.15.69:0/949771591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40c019a7b0 0x7f40c019ac30 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.323+0000 7f40c4c00640 1 --2- 172.21.15.69:0/949771591 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40c0109520 0x7f40c01a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.323+0000 7f40c4c00640 1 --2- 172.21.15.69:0/949771591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40c0104450 0x7f40c01a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.323+0000 7f40c4c00640 1 -- 172.21.15.69:0/949771591 >> 172.21.15.69:0/949771591 conn(0x7f40c00ffc70 msgr2=0x7f40c0101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:16.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.323+0000 7f40c4c00640 1 -- 172.21.15.69:0/949771591 shutdown_connections 2024-04-24T09:17:16.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.323+0000 7f40c4c00640 1 -- 172.21.15.69:0/949771591 wait complete. 2024-04-24T09:17:16.343 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:17:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.555+0000 7fa0b5000640 1 -- 172.21.15.69:0/1422298143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0b0109520 msgr2=0x7fa0b01110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.555+0000 7fa0b5000640 1 --2- 172.21.15.69:0/1422298143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0b0109520 0x7fa0b01110f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009ca0 tx=0x7fa0a802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.555+0000 7fa0b5000640 1 -- 172.21.15.69:0/1422298143 shutdown_connections 2024-04-24T09:17:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.555+0000 7fa0b5000640 1 --2- 172.21.15.69:0/1422298143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0b0109520 0x7fa0b01110f0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.555+0000 7fa0b5000640 1 --2- 172.21.15.69:0/1422298143 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0b0104e00 0x7fa0b0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.555+0000 7fa0b5000640 1 --2- 172.21.15.69:0/1422298143 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0b0104450 0x7fa0b0104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.555+0000 7fa0b5000640 1 -- 172.21.15.69:0/1422298143 >> 172.21.15.69:0/1422298143 conn(0x7fa0b00ffc70 msgr2=0x7fa0b0102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.555+0000 7fa0b5000640 1 -- 172.21.15.69:0/1422298143 shutdown_connections 2024-04-24T09:17:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.555+0000 7fa0b5000640 1 -- 172.21.15.69:0/1422298143 wait complete. 2024-04-24T09:17:16.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.556+0000 7fa0b5000640 1 Processor -- start 2024-04-24T09:17:16.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.557+0000 7fa0b5000640 1 -- start start 2024-04-24T09:17:16.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.557+0000 7fa0b5000640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0b0104450 0x7fa0b01a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:16.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.557+0000 7fa0b5000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0b0109520 0x7fa0b01a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:16.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.557+0000 7fa0b5000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0b019a880 0x7fa0b019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:16.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.557+0000 7fa0b5000640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01144e0 con 0x7fa0b0104450 2024-04-24T09:17:16.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.557+0000 7fa0b5000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b0114360 con 0x7fa0b0109520 2024-04-24T09:17:16.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.557+0000 7fa0b5000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b0114660 con 0x7fa0b019a880 2024-04-24T09:17:16.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.557+0000 7fa0ade00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0b0104450 0x7fa0b01a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:16.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.557+0000 7fa0ade00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0b0104450 0x7fa0b01a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37748/0 (socket says 172.21.15.69:37748) 2024-04-24T09:17:16.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.557+0000 7fa0ade00640 1 -- 172.21.15.69:0/2287672486 learned_addr learned my addr 172.21.15.69:0/2287672486 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:17:16.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.558+0000 7fa0ad400640 1 --2- 172.21.15.69:0/2287672486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0b0109520 0x7fa0b01a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:16.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.558+0000 7fa0ae800640 1 --2- 172.21.15.69:0/2287672486 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0b019a880 0x7fa0b019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:16.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.558+0000 7fa0ade00640 1 -- 172.21.15.69:0/2287672486 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0b019a880 msgr2=0x7fa0b019ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:16.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.558+0000 7fa0ade00640 1 --2- 172.21.15.69:0/2287672486 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0b019a880 0x7fa0b019ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.558+0000 7fa0ade00640 1 -- 172.21.15.69:0/2287672486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0b0109520 msgr2=0x7fa0b01a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:16.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.558+0000 7fa0ade00640 1 --2- 172.21.15.69:0/2287672486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0b0109520 0x7fa0b01a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.558+0000 7fa0ade00640 1 -- 172.21.15.69:0/2287672486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b019b410 con 0x7fa0b0104450 2024-04-24T09:17:16.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.558+0000 7fa0ad400640 1 --2- 172.21.15.69:0/2287672486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0b0109520 0x7fa0b01a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:16.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.558+0000 7fa0ade00640 1 --2- 172.21.15.69:0/2287672486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0b0104450 0x7fa0b01a06c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000d770 tx=0x7fa0a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:16.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.558+0000 7fa0a6a00640 1 -- 172.21.15.69:0/2287672486 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0017070 con 0x7fa0b0104450 2024-04-24T09:17:16.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.558+0000 7fa0a6a00640 1 -- 172.21.15.69:0/2287672486 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0a00054d0 con 0x7fa0b0104450 2024-04-24T09:17:16.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.558+0000 7fa0b5000640 1 -- 172.21.15.69:0/2287672486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b019b700 con 0x7fa0b0104450 2024-04-24T09:17:16.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.558+0000 7fa0a6a00640 1 -- 172.21.15.69:0/2287672486 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0004cf0 con 0x7fa0b0104450 2024-04-24T09:17:16.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.559+0000 7fa0b5000640 1 -- 172.21.15.69:0/2287672486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b01a7890 con 0x7fa0b0104450 2024-04-24T09:17:16.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.560+0000 7fa0b5000640 1 -- 172.21.15.69:0/2287672486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b0105920 con 0x7fa0b0104450 2024-04-24T09:17:16.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.562+0000 7fa0a6a00640 1 -- 172.21.15.69:0/2287672486 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa0a0004e90 con 0x7fa0b0104450 2024-04-24T09:17:16.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.564+0000 7fa0a6a00640 1 --2- 172.21.15.69:0/2287672486 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa0940a3da0 0x7fa0940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:16.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.564+0000 7fa0a6a00640 1 -- 172.21.15.69:0/2287672486 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fa0a00d0380 con 0x7fa0b0104450 2024-04-24T09:17:16.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.564+0000 7fa0ad400640 1 --2- 172.21.15.69:0/2287672486 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa0940a3da0 0x7fa0940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:16.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.565+0000 7fa0ad400640 1 --2- 172.21.15.69:0/2287672486 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa0940a3da0 0x7fa0940a6260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa09c005fd0 tx=0x7fa09c0094d0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:16.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.569+0000 7fa0a6a00640 1 -- 172.21.15.69:0/2287672486 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0b019b700 con 0x7fa0b0104450 2024-04-24T09:17:16.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:16 smithi069 ceph-mon[28170]: from='client.16914 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:16.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:16 smithi069 ceph-mon[28170]: from='client.16920 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:16.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:16 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/352900272' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:17:16.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:16 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/949771591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:17:16.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:16 smithi130 ceph-mon[31553]: from='client.16914 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:16.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:16 smithi130 ceph-mon[31553]: from='client.16920 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:16.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:16 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/352900272' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:17:16.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:16 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/949771591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:17:16.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.771+0000 7fa0b5000640 1 -- 172.21.15.69:0/2287672486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa0b019bd60 con 0x7fa0b0104450 2024-04-24T09:17:16.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.772+0000 7fa0a6a00640 1 -- 172.21.15.69:0/2287672486 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7fa0b019bd60 con 0x7fa0b0104450 2024-04-24T09:17:16.772 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:17:16.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.774+0000 7fa0b5000640 1 -- 172.21.15.69:0/2287672486 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa0940a3da0 msgr2=0x7fa0940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:16.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.774+0000 7fa0b5000640 1 --2- 172.21.15.69:0/2287672486 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa0940a3da0 0x7fa0940a6260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa09c005fd0 tx=0x7fa09c0094d0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.774+0000 7fa0b5000640 1 -- 172.21.15.69:0/2287672486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0b0104450 msgr2=0x7fa0b01a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:16.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.774+0000 7fa0b5000640 1 --2- 172.21.15.69:0/2287672486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0b0104450 0x7fa0b01a06c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000d770 tx=0x7fa0a000dc30 comp rx=0 tx=0).stop 2024-04-24T09:17:16.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.775+0000 7fa0b5000640 1 -- 172.21.15.69:0/2287672486 shutdown_connections 2024-04-24T09:17:16.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.775+0000 7fa0b5000640 1 --2- 172.21.15.69:0/2287672486 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fa0940a3da0 0x7fa0940a6260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.775+0000 7fa0b5000640 1 --2- 172.21.15.69:0/2287672486 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0b019a880 0x7fa0b019ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.775+0000 7fa0b5000640 1 --2- 172.21.15.69:0/2287672486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0b0109520 0x7fa0b01a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.775+0000 7fa0b5000640 1 --2- 172.21.15.69:0/2287672486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0b0104450 0x7fa0b01a06c0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:16.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.775+0000 7fa0b5000640 1 -- 172.21.15.69:0/2287672486 >> 172.21.15.69:0/2287672486 conn(0x7fa0b00ffc70 msgr2=0x7fa0b0101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:16.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.775+0000 7fa0b5000640 1 -- 172.21.15.69:0/2287672486 shutdown_connections 2024-04-24T09:17:16.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.775+0000 7fa0b5000640 1 -- 172.21.15.69:0/2287672486 wait complete. 2024-04-24T09:17:16.793 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:17:16.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:16 smithi081 ceph-mon[30961]: from='client.16914 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:16.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:16 smithi081 ceph-mon[30961]: from='client.16920 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:16.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:16 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/352900272' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:17:16.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:16 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/949771591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:17:16.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.999+0000 7f9cbcc00640 1 -- 172.21.15.69:0/2103268874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cb8104cc0 msgr2=0x7f9cb8108d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:16.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.999+0000 7f9cbcc00640 1 --2- 172.21.15.69:0/2103268874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cb8104cc0 0x7f9cb8108d00 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009900 tx=0x7f9ca002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.999+0000 7f9cbcc00640 1 -- 172.21.15.69:0/2103268874 shutdown_connections 2024-04-24T09:17:17.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.999+0000 7f9cbcc00640 1 --2- 172.21.15.69:0/2103268874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cb81093e0 0x7f9cb8110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.999+0000 7f9cbcc00640 1 --2- 172.21.15.69:0/2103268874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cb8104cc0 0x7f9cb8108d00 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.999+0000 7f9cbcc00640 1 --2- 172.21.15.69:0/2103268874 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9cb8104310 0x7f9cb81046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.999+0000 7f9cbcc00640 1 -- 172.21.15.69:0/2103268874 >> 172.21.15.69:0/2103268874 conn(0x7f9cb80ffb10 msgr2=0x7f9cb8101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:17.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.999+0000 7f9cbcc00640 1 -- 172.21.15.69:0/2103268874 shutdown_connections 2024-04-24T09:17:17.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:16.999+0000 7f9cbcc00640 1 -- 172.21.15.69:0/2103268874 wait complete. 2024-04-24T09:17:17.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.000+0000 7f9cbcc00640 1 Processor -- start 2024-04-24T09:17:17.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.000+0000 7f9cbcc00640 1 -- start start 2024-04-24T09:17:17.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.001+0000 7f9cbcc00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cb8104310 0x7f9cb81a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.001+0000 7f9cbcc00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9cb81093e0 0x7f9cb81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.001+0000 7f9cbcc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cb819a750 0x7f9cb819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.001+0000 7f9cbcc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb8114370 con 0x7f9cb819a750 2024-04-24T09:17:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.001+0000 7f9cbcc00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb81141f0 con 0x7f9cb8104310 2024-04-24T09:17:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.001+0000 7f9cbcc00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb81144f0 con 0x7f9cb81093e0 2024-04-24T09:17:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.001+0000 7f9cb7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cb819a750 0x7f9cb819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.001+0000 7f9cb7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cb819a750 0x7f9cb819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37766/0 (socket says 172.21.15.69:37766) 2024-04-24T09:17:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.001+0000 7f9cb7e00640 1 -- 172.21.15.69:0/1338054615 learned_addr learned my addr 172.21.15.69:0/1338054615 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:17:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.001+0000 7f9cb7400640 1 --2- 172.21.15.69:0/1338054615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cb8104310 0x7f9cb81a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:17.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.001+0000 7f9cafe00640 1 --2- 172.21.15.69:0/1338054615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9cb81093e0 0x7f9cb81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:17.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.001+0000 7f9cb7e00640 1 -- 172.21.15.69:0/1338054615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9cb81093e0 msgr2=0x7f9cb81a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:17.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.001+0000 7f9cb7e00640 1 --2- 172.21.15.69:0/1338054615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9cb81093e0 0x7f9cb81a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.001+0000 7f9cb7e00640 1 -- 172.21.15.69:0/1338054615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cb8104310 msgr2=0x7f9cb81a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:17.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.001+0000 7f9cb7e00640 1 --2- 172.21.15.69:0/1338054615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cb8104310 0x7f9cb81a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.001+0000 7f9cb7e00640 1 -- 172.21.15.69:0/1338054615 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb819b350 con 0x7f9cb819a750 2024-04-24T09:17:17.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.002+0000 7f9cb7e00640 1 --2- 172.21.15.69:0/1338054615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cb819a750 0x7f9cb819abd0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000cad0 tx=0x7f9cb000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:17.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.002+0000 7f9cb4c00640 1 -- 172.21.15.69:0/1338054615 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb000ae20 con 0x7f9cb819a750 2024-04-24T09:17:17.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.002+0000 7f9cb4c00640 1 -- 172.21.15.69:0/1338054615 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cb0005020 con 0x7f9cb819a750 2024-04-24T09:17:17.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.002+0000 7f9cbcc00640 1 -- 172.21.15.69:0/1338054615 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb81a7430 con 0x7f9cb819a750 2024-04-24T09:17:17.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.002+0000 7f9cb4c00640 1 -- 172.21.15.69:0/1338054615 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb000f6b0 con 0x7f9cb819a750 2024-04-24T09:17:17.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.002+0000 7f9cbcc00640 1 -- 172.21.15.69:0/1338054615 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb81a77e0 con 0x7f9cb819a750 2024-04-24T09:17:17.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.003+0000 7f9cbcc00640 1 -- 172.21.15.69:0/1338054615 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb81057e0 con 0x7f9cb819a750 2024-04-24T09:17:17.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.005+0000 7f9cb4c00640 1 -- 172.21.15.69:0/1338054615 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9cb00051d0 con 0x7f9cb819a750 2024-04-24T09:17:17.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.007+0000 7f9cb4c00640 1 --2- 172.21.15.69:0/1338054615 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9c9c0a3d20 0x7f9c9c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:17.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.007+0000 7f9cb4c00640 1 -- 172.21.15.69:0/1338054615 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f9cb00d0c90 con 0x7f9cb819a750 2024-04-24T09:17:17.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.007+0000 7f9cb7400640 1 --2- 172.21.15.69:0/1338054615 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9c9c0a3d20 0x7f9c9c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:17.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.008+0000 7f9cb7400640 1 --2- 172.21.15.69:0/1338054615 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9c9c0a3d20 0x7f9c9c0a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8005fd0 tx=0x7f9ca8005850 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:17.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.011+0000 7f9cb4c00640 1 -- 172.21.15.69:0/1338054615 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9cb0014030 con 0x7f9cb819a750 2024-04-24T09:17:17.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.201+0000 7f9cbcc00640 1 -- 172.21.15.69:0/1338054615 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9cb810c610 con 0x7f9cb819a750 2024-04-24T09:17:17.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.202+0000 7f9cb4c00640 1 -- 172.21.15.69:0/1338054615 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7f9cb009cf40 con 0x7f9cb819a750 2024-04-24T09:17:17.202 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-24T09:17:17.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.205+0000 7f9cbcc00640 1 -- 172.21.15.69:0/1338054615 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9c9c0a3d20 msgr2=0x7f9c9c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:17.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.205+0000 7f9cbcc00640 1 --2- 172.21.15.69:0/1338054615 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9c9c0a3d20 0x7f9c9c0a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8005fd0 tx=0x7f9ca8005850 comp rx=0 tx=0).stop 2024-04-24T09:17:17.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.205+0000 7f9cbcc00640 1 -- 172.21.15.69:0/1338054615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cb819a750 msgr2=0x7f9cb819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:17.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.205+0000 7f9cbcc00640 1 --2- 172.21.15.69:0/1338054615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cb819a750 0x7f9cb819abd0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000cad0 tx=0x7f9cb000cf90 comp rx=0 tx=0).stop 2024-04-24T09:17:17.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.205+0000 7f9cbcc00640 1 -- 172.21.15.69:0/1338054615 shutdown_connections 2024-04-24T09:17:17.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.205+0000 7f9cbcc00640 1 --2- 172.21.15.69:0/1338054615 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9c9c0a3d20 0x7f9c9c0a61e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.205+0000 7f9cbcc00640 1 --2- 172.21.15.69:0/1338054615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cb819a750 0x7f9cb819abd0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.205+0000 7f9cbcc00640 1 --2- 172.21.15.69:0/1338054615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9cb81093e0 0x7f9cb81a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.205+0000 7f9cbcc00640 1 --2- 172.21.15.69:0/1338054615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cb8104310 0x7f9cb81a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.205+0000 7f9cbcc00640 1 -- 172.21.15.69:0/1338054615 >> 172.21.15.69:0/1338054615 conn(0x7f9cb80ffb10 msgr2=0x7f9cb8100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:17.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.205+0000 7f9cbcc00640 1 -- 172.21.15.69:0/1338054615 shutdown_connections 2024-04-24T09:17:17.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.205+0000 7f9cbcc00640 1 -- 172.21.15.69:0/1338054615 wait complete. 2024-04-24T09:17:17.223 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T09:17:17.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.419+0000 7f350f400640 1 -- 172.21.15.69:0/663538264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3508109520 msgr2=0x7f35081110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.419+0000 7f350f400640 1 --2- 172.21.15.69:0/663538264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3508109520 0x7f35081110f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f3500009ca0 tx=0x7f350002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.419+0000 7f350f400640 1 -- 172.21.15.69:0/663538264 shutdown_connections 2024-04-24T09:17:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.419+0000 7f350f400640 1 --2- 172.21.15.69:0/663538264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3508109520 0x7f35081110f0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.419+0000 7f350f400640 1 --2- 172.21.15.69:0/663538264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3508104e00 0x7f3508108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.419+0000 7f350f400640 1 --2- 172.21.15.69:0/663538264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3508104450 0x7f3508104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.419+0000 7f350f400640 1 -- 172.21.15.69:0/663538264 >> 172.21.15.69:0/663538264 conn(0x7f35080ffc70 msgr2=0x7f3508102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.419+0000 7f350f400640 1 -- 172.21.15.69:0/663538264 shutdown_connections 2024-04-24T09:17:17.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.419+0000 7f350f400640 1 -- 172.21.15.69:0/663538264 wait complete. 2024-04-24T09:17:17.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.420+0000 7f350f400640 1 Processor -- start 2024-04-24T09:17:17.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.420+0000 7f350f400640 1 -- start start 2024-04-24T09:17:17.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.420+0000 7f350f400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3508104450 0x7f35081a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:17.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.420+0000 7f350f400640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3508109520 0x7f35081a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:17.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.420+0000 7f350f400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f350819a830 0x7f350819ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:17.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.420+0000 7f350f400640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3508114490 con 0x7f3508104450 2024-04-24T09:17:17.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.420+0000 7f350f400640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3508114310 con 0x7f3508109520 2024-04-24T09:17:17.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.420+0000 7f350f400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3508114610 con 0x7f350819a830 2024-04-24T09:17:17.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.421+0000 7f350ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3508104450 0x7f35081a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:17.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.421+0000 7f350ca00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3508104450 0x7f35081a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37782/0 (socket says 172.21.15.69:37782) 2024-04-24T09:17:17.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.421+0000 7f350ca00640 1 -- 172.21.15.69:0/4036017596 learned_addr learned my addr 172.21.15.69:0/4036017596 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:17:17.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.421+0000 7f350d400640 1 --2- 172.21.15.69:0/4036017596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f350819a830 0x7f350819ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:17.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.421+0000 7f3507e00640 1 --2- 172.21.15.69:0/4036017596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3508109520 0x7f35081a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:17.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.421+0000 7f350ca00640 1 -- 172.21.15.69:0/4036017596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f350819a830 msgr2=0x7f350819ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:17.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.421+0000 7f350ca00640 1 --2- 172.21.15.69:0/4036017596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f350819a830 0x7f350819ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.421+0000 7f350ca00640 1 -- 172.21.15.69:0/4036017596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3508109520 msgr2=0x7f35081a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:17.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.421+0000 7f350ca00640 1 --2- 172.21.15.69:0/4036017596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3508109520 0x7f35081a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.421+0000 7f350ca00640 1 -- 172.21.15.69:0/4036017596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350819b3c0 con 0x7f3508104450 2024-04-24T09:17:17.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.421+0000 7f350ca00640 1 --2- 172.21.15.69:0/4036017596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3508104450 0x7f35081a06c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00d950 tx=0x7f34fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:17.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.422+0000 7f3505600640 1 -- 172.21.15.69:0/4036017596 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc017030 con 0x7f3508104450 2024-04-24T09:17:17.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.422+0000 7f3505600640 1 -- 172.21.15.69:0/4036017596 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34fc004020 con 0x7f3508104450 2024-04-24T09:17:17.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.422+0000 7f3505600640 1 -- 172.21.15.69:0/4036017596 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0047a0 con 0x7f3508104450 2024-04-24T09:17:17.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.422+0000 7f350f400640 1 -- 172.21.15.69:0/4036017596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350819b6b0 con 0x7f3508104450 2024-04-24T09:17:17.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.422+0000 7f350f400640 1 -- 172.21.15.69:0/4036017596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35081a7840 con 0x7f3508104450 2024-04-24T09:17:17.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.423+0000 7f350f400640 1 -- 172.21.15.69:0/4036017596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3508105920 con 0x7f3508104450 2024-04-24T09:17:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.425+0000 7f3505600640 1 -- 172.21.15.69:0/4036017596 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f34fc0041d0 con 0x7f3508104450 2024-04-24T09:17:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.426+0000 7f3505600640 1 --2- 172.21.15.69:0/4036017596 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f34f00a3d20 0x7f34f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.427+0000 7f3505600640 1 -- 172.21.15.69:0/4036017596 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f34fc0d0750 con 0x7f3508104450 2024-04-24T09:17:17.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.427+0000 7f3507e00640 1 --2- 172.21.15.69:0/4036017596 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f34f00a3d20 0x7f34f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:17.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.427+0000 7f3507e00640 1 --2- 172.21.15.69:0/4036017596 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f34f00a3d20 0x7f34f00a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f350810aab0 tx=0x7f34f4005f90 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:17.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.431+0000 7f3505600640 1 -- 172.21.15.69:0/4036017596 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f350819b6b0 con 0x7f3508104450 2024-04-24T09:17:17.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:17 smithi069 ceph-mon[28170]: pgmap v317: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-24T09:17:17.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:17 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2287672486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:17:17.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:17 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1338054615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:17:17.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.632+0000 7f350f400640 1 -- 172.21.15.69:0/4036017596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3508104e00 con 0x7f3508104450 2024-04-24T09:17:17.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.633+0000 7f3505600640 1 -- 172.21.15.69:0/4036017596 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v101) ==== 103+0+11 (secure 0 0 0) 0x7f34fc09ca00 con 0x7f3508104450 2024-04-24T09:17:17.633 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:17:17.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.636+0000 7f350f400640 1 -- 172.21.15.69:0/4036017596 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f34f00a3d20 msgr2=0x7f34f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:17.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.636+0000 7f350f400640 1 --2- 172.21.15.69:0/4036017596 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f34f00a3d20 0x7f34f00a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f350810aab0 tx=0x7f34f4005f90 comp rx=0 tx=0).stop 2024-04-24T09:17:17.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.636+0000 7f350f400640 1 -- 172.21.15.69:0/4036017596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3508104450 msgr2=0x7f35081a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:17.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.636+0000 7f350f400640 1 --2- 172.21.15.69:0/4036017596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3508104450 0x7f35081a06c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00d950 tx=0x7f34fc00de10 comp rx=0 tx=0).stop 2024-04-24T09:17:17.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.636+0000 7f350f400640 1 -- 172.21.15.69:0/4036017596 shutdown_connections 2024-04-24T09:17:17.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.636+0000 7f350f400640 1 --2- 172.21.15.69:0/4036017596 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f34f00a3d20 0x7f34f00a61e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.636+0000 7f350f400640 1 --2- 172.21.15.69:0/4036017596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f350819a830 0x7f350819ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.636+0000 7f350f400640 1 --2- 172.21.15.69:0/4036017596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3508109520 0x7f35081a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.636+0000 7f350f400640 1 --2- 172.21.15.69:0/4036017596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3508104450 0x7f35081a06c0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.636+0000 7f350f400640 1 -- 172.21.15.69:0/4036017596 >> 172.21.15.69:0/4036017596 conn(0x7f35080ffc70 msgr2=0x7f3508100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:17.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.636+0000 7f350f400640 1 -- 172.21.15.69:0/4036017596 shutdown_connections 2024-04-24T09:17:17.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.636+0000 7f350f400640 1 -- 172.21.15.69:0/4036017596 wait complete. 2024-04-24T09:17:17.653 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-24T09:17:17.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:17 smithi130 ceph-mon[31553]: pgmap v317: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-24T09:17:17.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:17 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2287672486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:17:17.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:17 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1338054615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:17:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:17 smithi081 ceph-mon[30961]: pgmap v317: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-24T09:17:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:17 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2287672486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:17:17.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:17 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1338054615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:17:17.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.848+0000 7fcab7200640 1 -- 172.21.15.69:0/2354640538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab81071b0 msgr2=0x7fcab80696d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:17.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.848+0000 7fcab7200640 1 --2- 172.21.15.69:0/2354640538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab81071b0 0x7fcab80696d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009900 tx=0x7fcaa402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.849+0000 7fcab7200640 1 -- 172.21.15.69:0/2354640538 shutdown_connections 2024-04-24T09:17:17.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.849+0000 7fcab7200640 1 --2- 172.21.15.69:0/2354640538 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcab8104890 0x7fcab8104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.849+0000 7fcab7200640 1 --2- 172.21.15.69:0/2354640538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab81071b0 0x7fcab80696d0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.849+0000 7fcab7200640 1 --2- 172.21.15.69:0/2354640538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab8106d40 0x7fcab8069190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.849+0000 7fcab7200640 1 -- 172.21.15.69:0/2354640538 >> 172.21.15.69:0/2354640538 conn(0x7fcab8076840 msgr2=0x7fcab8076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:17.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.849+0000 7fcab7200640 1 -- 172.21.15.69:0/2354640538 shutdown_connections 2024-04-24T09:17:17.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.849+0000 7fcab7200640 1 -- 172.21.15.69:0/2354640538 wait complete. 2024-04-24T09:17:17.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.850+0000 7fcab7200640 1 Processor -- start 2024-04-24T09:17:17.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.850+0000 7fcab7200640 1 -- start start 2024-04-24T09:17:17.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.850+0000 7fcab7200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab8104890 0x7fcab81a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:17.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.850+0000 7fcab7200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab8106d40 0x7fcab81a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:17.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.850+0000 7fcab7200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcab81071b0 0x7fcab819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:17.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.850+0000 7fcab7200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81185e0 con 0x7fcab8106d40 2024-04-24T09:17:17.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.850+0000 7fcab7200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcab8118460 con 0x7fcab8104890 2024-04-24T09:17:17.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.850+0000 7fcab7200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcab8118760 con 0x7fcab81071b0 2024-04-24T09:17:17.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.850+0000 7fcab5400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab8106d40 0x7fcab81a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:17.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.850+0000 7fcab5400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab8106d40 0x7fcab81a4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37790/0 (socket says 172.21.15.69:37790) 2024-04-24T09:17:17.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.850+0000 7fcab5400640 1 -- 172.21.15.69:0/432270178 learned_addr learned my addr 172.21.15.69:0/432270178 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:17:17.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.851+0000 7fcab6800640 1 --2- 172.21.15.69:0/432270178 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcab81071b0 0x7fcab819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:17.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.851+0000 7fcab5e00640 1 --2- 172.21.15.69:0/432270178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab8104890 0x7fcab81a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:17.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.851+0000 7fcab5400640 1 -- 172.21.15.69:0/432270178 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcab81071b0 msgr2=0x7fcab819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:17.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.851+0000 7fcab5400640 1 --2- 172.21.15.69:0/432270178 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcab81071b0 0x7fcab819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.851+0000 7fcab5400640 1 -- 172.21.15.69:0/432270178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab8104890 msgr2=0x7fcab81a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:17.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.851+0000 7fcab5400640 1 --2- 172.21.15.69:0/432270178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab8104890 0x7fcab81a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:17.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.851+0000 7fcab5400640 1 -- 172.21.15.69:0/432270178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab819f0e0 con 0x7fcab8106d40 2024-04-24T09:17:17.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.851+0000 7fcab5e00640 1 --2- 172.21.15.69:0/432270178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab8104890 0x7fcab81a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:17.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.851+0000 7fcab5400640 1 --2- 172.21.15.69:0/432270178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab8106d40 0x7fcab81a4ca0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4006010 tx=0x7fcaa4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:17.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.851+0000 7fcaaea00640 1 -- 172.21.15.69:0/432270178 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa403f030 con 0x7fcab8106d40 2024-04-24T09:17:17.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.851+0000 7fcaaea00640 1 -- 172.21.15.69:0/432270178 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcaa40035f0 con 0x7fcab8106d40 2024-04-24T09:17:17.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.851+0000 7fcab7200640 1 -- 172.21.15.69:0/432270178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab819f3d0 con 0x7fcab8106d40 2024-04-24T09:17:17.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.851+0000 7fcaaea00640 1 -- 172.21.15.69:0/432270178 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4035070 con 0x7fcab8106d40 2024-04-24T09:17:17.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.852+0000 7fcab7200640 1 -- 172.21.15.69:0/432270178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab81ab620 con 0x7fcab8106d40 2024-04-24T09:17:17.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.853+0000 7fcab7200640 1 -- 172.21.15.69:0/432270178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab8068d40 con 0x7fcab8106d40 2024-04-24T09:17:17.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.855+0000 7fcaaea00640 1 -- 172.21.15.69:0/432270178 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcaa4003b10 con 0x7fcab8106d40 2024-04-24T09:17:17.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.857+0000 7fcaaea00640 1 --2- 172.21.15.69:0/432270178 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fca9c0a3da0 0x7fca9c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:17.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.857+0000 7fcaaea00640 1 -- 172.21.15.69:0/432270178 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fcaa40f1780 con 0x7fcab8106d40 2024-04-24T09:17:17.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.858+0000 7fcab5e00640 1 --2- 172.21.15.69:0/432270178 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fca9c0a3da0 0x7fca9c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:17.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.859+0000 7fcab5e00640 1 --2- 172.21.15.69:0/432270178 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fca9c0a3da0 0x7fca9c0a6260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcab8101a90 tx=0x7fcaa8005d30 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:17.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:17.862+0000 7fcaaea00640 1 -- 172.21.15.69:0/432270178 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcaa40bda30 con 0x7fcab8106d40 2024-04-24T09:17:18.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:18.124+0000 7fcab7200640 1 -- 172.21.15.69:0/432270178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fcab81ab970 con 0x7fcab8106d40 2024-04-24T09:17:18.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:18.138+0000 7fcaaea00640 1 -- 172.21.15.69:0/432270178 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746063 (secure 0 0 0) 0x7fcaa40bd330 con 0x7fcab8106d40 2024-04-24T09:17:18.150 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:17:18.150 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:17:18.150 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-04-24T09:17:18.151 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.151 INFO:teuthology.orchestra.run.smithi069.stdout: "active_gid": 15465, 2024-04-24T09:17:18.151 INFO:teuthology.orchestra.run.smithi069.stdout: "active_name": "x", 2024-04-24T09:17:18.151 INFO:teuthology.orchestra.run.smithi069.stdout: "active_addrs": { 2024-04-24T09:17:18.151 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-04-24T09:17:18.151 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.151 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-04-24T09:17:18.151 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:6800", 2024-04-24T09:17:18.151 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 2822728880 2024-04-24T09:17:18.152 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.152 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.152 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-04-24T09:17:18.152 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:6801", 2024-04-24T09:17:18.152 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 2822728880 2024-04-24T09:17:18.152 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.152 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-24T09:17:18.152 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.152 INFO:teuthology.orchestra.run.smithi069.stdout: "active_addr": "172.21.15.69:6801/2822728880", 2024-04-24T09:17:18.152 INFO:teuthology.orchestra.run.smithi069.stdout: "active_change": "2024-04-24T09:07:03.864484+0000", 2024-04-24T09:17:18.153 INFO:teuthology.orchestra.run.smithi069.stdout: "active_mgr_features": 4540701547738038271, 2024-04-24T09:17:18.153 INFO:teuthology.orchestra.run.smithi069.stdout: "available": true, 2024-04-24T09:17:18.153 INFO:teuthology.orchestra.run.smithi069.stdout: "standbys": [ 2024-04-24T09:17:18.153 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.153 INFO:teuthology.orchestra.run.smithi069.stdout: "gid": 24955, 2024-04-24T09:17:18.153 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "z", 2024-04-24T09:17:18.153 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr_features": 4540701547738038271, 2024-04-24T09:17:18.153 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2024-04-24T09:17:18.153 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.153 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2024-04-24T09:17:18.154 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.154 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.154 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.154 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-24T09:17:18.154 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-24T09:17:18.154 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.154 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.154 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.155 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-24T09:17:18.155 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.155 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.155 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.155 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T09:17:18.155 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.155 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.155 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.155 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.156 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.156 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.156 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.156 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.156 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.156 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.156 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.156 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.156 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.157 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.157 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.157 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.157 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.157 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.157 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.157 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.157 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.157 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.157 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.158 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.158 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.158 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.158 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.158 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.158 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.158 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.158 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.158 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.158 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.159 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.159 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.159 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.159 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.159 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.159 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.159 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.159 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.159 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.160 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.160 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.160 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.160 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.160 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.160 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.160 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.160 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.160 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.161 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.161 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.161 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.161 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.161 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.161 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.161 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.161 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.161 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.161 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.162 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.162 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.162 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.162 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.162 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.162 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.162 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.162 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.162 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.163 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.163 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.163 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.163 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.163 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2024-04-24T09:17:18.163 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2024-04-24T09:17:18.163 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.163 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.163 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.164 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.164 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.164 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.164 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.164 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2024-04-24T09:17:18.164 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.164 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.164 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.164 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.165 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2024-04-24T09:17:18.165 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2024-04-24T09:17:18.165 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.165 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.165 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.165 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2024-04-24T09:17:18.165 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.165 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.165 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.166 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2024-04-24T09:17:18.166 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.166 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.166 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.166 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.166 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2024-04-24T09:17:18.166 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2024-04-24T09:17:18.166 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.166 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.167 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.167 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.167 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.167 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.167 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.167 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2024-04-24T09:17:18.167 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.167 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.167 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.168 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.168 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2024-04-24T09:17:18.168 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2024-04-24T09:17:18.168 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.168 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.168 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.168 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.168 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.168 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.169 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.169 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2024-04-24T09:17:18.169 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.169 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.169 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.169 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.169 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2024-04-24T09:17:18.169 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2024-04-24T09:17:18.169 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.169 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.170 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.170 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2024-04-24T09:17:18.170 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.170 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.170 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.170 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2024-04-24T09:17:18.170 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.170 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.171 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.171 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.171 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2024-04-24T09:17:18.171 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2024-04-24T09:17:18.171 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.171 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.171 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.171 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.171 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.171 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.172 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.172 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2024-04-24T09:17:18.172 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.172 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.172 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.172 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.172 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2024-04-24T09:17:18.172 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2024-04-24T09:17:18.172 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.173 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.173 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.173 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.173 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.173 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.173 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.173 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T09:17:18.173 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.173 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.174 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.174 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.174 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2024-04-24T09:17:18.174 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2024-04-24T09:17:18.174 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.174 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.174 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.174 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.174 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.174 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.175 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.175 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2024-04-24T09:17:18.175 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.175 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.175 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.175 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.175 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.175 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.175 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.175 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2024-04-24T09:17:18.176 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.176 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.176 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.176 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2024-04-24T09:17:18.176 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2024-04-24T09:17:18.176 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.176 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.176 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.176 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.177 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.177 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.177 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.177 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T09:17:18.177 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.177 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.177 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.177 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.177 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2024-04-24T09:17:18.177 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2024-04-24T09:17:18.178 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.178 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.178 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.178 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2024-04-24T09:17:18.178 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.178 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.178 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.178 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T09:17:18.179 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:17:18.179 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.179 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.179 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.179 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2024-04-24T09:17:18.179 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2024-04-24T09:17:18.179 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-24T09:17:18.179 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.179 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.179 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-24T09:17:18.180 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-04-24T09:17:18.180 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-04-24T09:17:18.180 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.180 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T09:17:18.180 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:17:18.180 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.180 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.180 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.180 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2024-04-24T09:17:18.181 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2024-04-24T09:17:18.181 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-24T09:17:18.181 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.181 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.181 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2024-04-24T09:17:18.181 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-24T09:17:18.181 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2024-04-24T09:17:18.181 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.181 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T09:17:18.181 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.182 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.182 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.182 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.182 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2024-04-24T09:17:18.182 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2024-04-24T09:17:18.182 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.182 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.182 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.182 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T09:17:18.183 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.183 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.183 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.183 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T09:17:18.183 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T09:17:18.183 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.183 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.183 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.183 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2024-04-24T09:17:18.183 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2024-04-24T09:17:18.184 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:18.184 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.184 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.184 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2024-04-24T09:17:18.184 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2024-04-24T09:17:18.184 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2024-04-24T09:17:18.184 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.184 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2024-04-24T09:17:18.185 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T09:17:18.185 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.185 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.185 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.185 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2024-04-24T09:17:18.185 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2024-04-24T09:17:18.185 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.185 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.185 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.186 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2024-04-24T09:17:18.186 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.186 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.186 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.186 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T09:17:18.186 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:17:18.186 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.186 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.186 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.187 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2024-04-24T09:17:18.187 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2024-04-24T09:17:18.187 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-24T09:17:18.187 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.187 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.187 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-24T09:17:18.187 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-04-24T09:17:18.187 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-04-24T09:17:18.188 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.188 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T09:17:18.188 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:17:18.188 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.188 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.188 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.188 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.188 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.188 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.189 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.189 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.189 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.189 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.189 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.189 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.189 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.189 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.189 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.189 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.190 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.190 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.190 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.190 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.190 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.190 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.190 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.190 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.190 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.191 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.191 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.191 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.191 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.191 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.191 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.191 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.191 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.191 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.192 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.192 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.192 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.192 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.192 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.192 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.192 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.192 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.192 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.192 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.193 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.193 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.193 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.193 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.193 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.193 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.193 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.193 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.193 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.193 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.194 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.194 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.194 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.194 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.194 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.194 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.194 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.194 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.194 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.195 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.195 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.195 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.195 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.195 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.195 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.195 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.195 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.195 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.196 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.196 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2024-04-24T09:17:18.196 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2024-04-24T09:17:18.196 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:18.196 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.196 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.196 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-24T09:17:18.196 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.196 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.197 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.197 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T09:17:18.197 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.197 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.197 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.197 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.197 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-04-24T09:17:18.197 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-04-24T09:17:18.197 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.197 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.198 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.198 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2024-04-24T09:17:18.198 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.198 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.198 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.198 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2024-04-24T09:17:18.198 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2024-04-24T09:17:18.198 INFO:teuthology.orchestra.run.smithi069.stdout: "read", 2024-04-24T09:17:18.198 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap", 2024-04-24T09:17:18.199 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap-read" 2024-04-24T09:17:18.199 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.199 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2024-04-24T09:17:18.199 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.199 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.199 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.199 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.199 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2024-04-24T09:17:18.199 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2024-04-24T09:17:18.199 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.200 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.200 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.200 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.200 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.200 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.200 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.200 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T09:17:18.200 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.200 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.201 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.201 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.201 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-24T09:17:18.201 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-24T09:17:18.201 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.201 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.201 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.201 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-24T09:17:18.201 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.202 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.202 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.202 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T09:17:18.202 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.202 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.202 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.202 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.202 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2024-04-24T09:17:18.202 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2024-04-24T09:17:18.203 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.203 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.203 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.203 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-24T09:17:18.203 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-24T09:17:18.203 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.203 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.203 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T09:17:18.203 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T09:17:18.203 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.204 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.204 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.204 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2024-04-24T09:17:18.204 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2024-04-24T09:17:18.204 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-24T09:17:18.204 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.204 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.204 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:18.204 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.205 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.205 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.205 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T09:17:18.205 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.205 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.205 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.205 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.205 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.205 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.205 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.206 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2024-04-24T09:17:18.206 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.206 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.206 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.206 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2024-04-24T09:17:18.206 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2024-04-24T09:17:18.206 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:18.206 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.206 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.207 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-04-24T09:17:18.207 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.207 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.207 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.207 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T09:17:18.207 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.207 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.207 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.207 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.207 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2024-04-24T09:17:18.208 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2024-04-24T09:17:18.208 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.208 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.208 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.208 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2024-04-24T09:17:18.208 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.208 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.208 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.208 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T09:17:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2024-04-24T09:17:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2024-04-24T09:17:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2024-04-24T09:17:18.209 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T09:17:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.210 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2024-04-24T09:17:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2024-04-24T09:17:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T09:17:18.211 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T09:17:18.212 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.212 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.212 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.212 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2024-04-24T09:17:18.212 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2024-04-24T09:17:18.212 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.212 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.212 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.212 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-24T09:17:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T09:17:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2024-04-24T09:17:18.213 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T09:17:18.214 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:18.214 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.214 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.214 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2024-04-24T09:17:18.214 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.214 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.214 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.214 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T09:17:18.214 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.215 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.215 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.215 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.215 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm_log_destination": { 2024-04-24T09:17:18.215 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm_log_destination", 2024-04-24T09:17:18.215 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.215 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.215 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.215 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.215 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.216 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.216 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.216 INFO:teuthology.orchestra.run.smithi069.stdout: "file", 2024-04-24T09:17:18.216 INFO:teuthology.orchestra.run.smithi069.stdout: "file,syslog", 2024-04-24T09:17:18.216 INFO:teuthology.orchestra.run.smithi069.stdout: "syslog" 2024-04-24T09:17:18.216 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.216 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T09:17:18.216 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.216 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.216 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.217 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.217 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2024-04-24T09:17:18.217 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2024-04-24T09:17:18.217 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.217 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.217 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.217 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.217 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.217 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.218 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.218 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T09:17:18.218 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.218 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.218 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.218 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.218 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2024-04-24T09:17:18.218 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2024-04-24T09:17:18.218 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.219 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.219 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.219 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.219 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.219 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.219 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.219 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T09:17:18.219 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.219 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.219 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.220 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.220 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2024-04-24T09:17:18.220 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2024-04-24T09:17:18.220 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.220 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.220 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.220 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.220 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.220 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.221 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.221 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T09:17:18.221 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.221 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.221 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.221 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.221 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2024-04-24T09:17:18.221 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2024-04-24T09:17:18.221 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.221 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.222 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.222 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T09:17:18.222 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.222 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.222 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.222 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-24T09:17:18.222 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.222 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.222 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.222 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.223 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2024-04-24T09:17:18.223 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2024-04-24T09:17:18.223 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.223 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.223 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.223 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T09:17:18.223 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.223 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.223 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.224 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2024-04-24T09:17:18.224 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.224 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.224 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.224 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.224 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2024-04-24T09:17:18.224 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2024-04-24T09:17:18.224 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.224 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.224 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.225 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T09:17:18.225 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.225 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.225 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.225 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2024-04-24T09:17:18.225 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.225 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.225 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.225 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.225 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2024-04-24T09:17:18.226 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2024-04-24T09:17:18.226 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.226 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.226 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.226 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T09:17:18.226 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.226 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.226 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.227 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-24T09:17:18.227 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.227 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.227 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.227 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.227 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2024-04-24T09:17:18.227 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2024-04-24T09:17:18.227 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.227 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.228 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.228 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T09:17:18.228 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.228 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.228 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.228 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2024-04-24T09:17:18.228 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.228 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.229 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.229 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.229 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2024-04-24T09:17:18.229 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2024-04-24T09:17:18.229 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.229 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.229 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.229 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T09:17:18.229 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.230 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.230 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.230 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2024-04-24T09:17:18.230 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.230 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.230 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.230 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.230 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2024-04-24T09:17:18.231 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2024-04-24T09:17:18.231 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.231 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.231 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.231 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T09:17:18.231 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.231 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.231 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.231 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2024-04-24T09:17:18.231 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.232 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.232 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.232 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.232 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2024-04-24T09:17:18.232 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2024-04-24T09:17:18.232 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.232 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.232 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.232 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T09:17:18.233 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.233 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.233 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.233 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2024-04-24T09:17:18.233 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.233 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.233 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.233 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.233 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2024-04-24T09:17:18.234 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2024-04-24T09:17:18.234 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.234 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.234 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.234 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T09:17:18.234 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.234 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.234 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.234 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2024-04-24T09:17:18.235 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.235 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.235 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.235 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.235 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2024-04-24T09:17:18.235 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2024-04-24T09:17:18.235 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.235 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.235 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.236 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T09:17:18.236 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.236 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.236 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.236 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2024-04-24T09:17:18.236 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.236 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.236 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.236 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.237 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2024-04-24T09:17:18.237 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2024-04-24T09:17:18.237 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.237 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.237 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.237 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T09:17:18.237 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.237 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.237 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.238 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-24T09:17:18.238 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.238 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.238 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.238 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.238 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_nvmeof": { 2024-04-24T09:17:18.238 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_nvmeof", 2024-04-24T09:17:18.238 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.238 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.239 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.239 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T09:17:18.239 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.239 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.239 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.239 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Nvme-of container image", 2024-04-24T09:17:18.239 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.239 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.239 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.240 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.240 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2024-04-24T09:17:18.240 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2024-04-24T09:17:18.240 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.240 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.240 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.240 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T09:17:18.240 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.240 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.240 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.241 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-24T09:17:18.241 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.241 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.241 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.241 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.241 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2024-04-24T09:17:18.241 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2024-04-24T09:17:18.241 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.241 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.241 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.242 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T09:17:18.242 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.242 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.242 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.242 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2024-04-24T09:17:18.242 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.242 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.242 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.242 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.243 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_samba": { 2024-04-24T09:17:18.243 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_samba", 2024-04-24T09:17:18.243 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.243 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.243 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.243 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T09:17:18.243 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.243 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.243 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.243 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Samba/SMB container image", 2024-04-24T09:17:18.244 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.244 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.244 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.244 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.244 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2024-04-24T09:17:18.244 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2024-04-24T09:17:18.244 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.244 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.244 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.245 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T09:17:18.245 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.245 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.245 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.245 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2024-04-24T09:17:18.245 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.245 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.245 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.245 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.245 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2024-04-24T09:17:18.246 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2024-04-24T09:17:18.246 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.246 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.246 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.246 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.246 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.246 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.246 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.246 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T09:17:18.247 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.247 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.247 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.247 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.247 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2024-04-24T09:17:18.247 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2024-04-24T09:17:18.247 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.247 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.247 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.248 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-24T09:17:18.248 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.248 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.248 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.248 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T09:17:18.248 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.248 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.248 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.248 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.248 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2024-04-24T09:17:18.249 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T09:17:18.249 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.249 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.249 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.249 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2024-04-24T09:17:18.249 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.249 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.249 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.249 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T09:17:18.250 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.250 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.250 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.250 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.250 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2024-04-24T09:17:18.250 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2024-04-24T09:17:18.250 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.250 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.250 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.251 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2024-04-24T09:17:18.251 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.251 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.251 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.251 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T09:17:18.251 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.251 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.251 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.251 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.251 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2024-04-24T09:17:18.252 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2024-04-24T09:17:18.252 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.252 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.252 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.252 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2024-04-24T09:17:18.252 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.252 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.252 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.252 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2024-04-24T09:17:18.252 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.253 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.253 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.253 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.253 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2024-04-24T09:17:18.253 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2024-04-24T09:17:18.253 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.253 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.253 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.253 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.254 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.254 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.254 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.254 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T09:17:18.254 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.254 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.254 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.254 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.254 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2024-04-24T09:17:18.255 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2024-04-24T09:17:18.255 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.255 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.255 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.255 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-24T09:17:18.255 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.255 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.255 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.255 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2024-04-24T09:17:18.255 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.256 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.256 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.256 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.256 INFO:teuthology.orchestra.run.smithi069.stdout: "grafana_dashboards_path": { 2024-04-24T09:17:18.256 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "grafana_dashboards_path", 2024-04-24T09:17:18.256 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.256 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.256 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.256 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T09:17:18.257 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.257 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.257 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.257 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T09:17:18.257 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.257 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.257 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.257 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.257 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2024-04-24T09:17:18.258 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2024-04-24T09:17:18.258 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.258 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.258 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.258 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-24T09:17:18.258 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.258 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.258 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.258 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2024-04-24T09:17:18.259 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.259 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.259 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.259 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.259 INFO:teuthology.orchestra.run.smithi069.stdout: "hw_monitoring": { 2024-04-24T09:17:18.259 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hw_monitoring", 2024-04-24T09:17:18.259 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.259 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.259 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.260 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.260 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.260 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.260 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.260 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T09:17:18.260 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.260 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.260 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.260 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.260 INFO:teuthology.orchestra.run.smithi069.stdout: "inventory_list_all": { 2024-04-24T09:17:18.261 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "inventory_list_all", 2024-04-24T09:17:18.261 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.261 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.261 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.261 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.261 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.261 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.261 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.261 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T09:17:18.261 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.262 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.262 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.262 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.262 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.262 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.262 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.262 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.262 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.262 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.262 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.263 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.263 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.263 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.263 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.263 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.263 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.263 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.263 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.263 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.264 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.264 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.264 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.264 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.264 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.264 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2024-04-24T09:17:18.264 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2024-04-24T09:17:18.264 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.264 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.264 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.265 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.265 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.265 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.265 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.265 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T09:17:18.265 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.265 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.265 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.265 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.266 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.266 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.266 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.266 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.266 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.266 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.266 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.266 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.266 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.267 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T09:17:18.267 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.267 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.267 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.267 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.267 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.267 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.267 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.267 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.267 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.268 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.268 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.268 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.268 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.268 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.268 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.268 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.268 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.268 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.269 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.269 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.269 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.269 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.269 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.269 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.269 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.269 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.269 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.269 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.270 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.270 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.270 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.270 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.270 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.270 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.270 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.270 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.271 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.271 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.271 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.271 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T09:17:18.271 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T09:17:18.271 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.271 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.271 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.271 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.271 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.272 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.272 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.272 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T09:17:18.272 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.272 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.272 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.272 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.272 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T09:17:18.272 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T09:17:18.272 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.273 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.273 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.273 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2024-04-24T09:17:18.273 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.273 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.273 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.273 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T09:17:18.273 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.273 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.274 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.274 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.274 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2024-04-24T09:17:18.274 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2024-04-24T09:17:18.274 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.274 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.274 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.274 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:18.275 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.275 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.275 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.275 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2024-04-24T09:17:18.275 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.275 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.275 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.275 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.275 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2024-04-24T09:17:18.276 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2024-04-24T09:17:18.276 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.276 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.276 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.276 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:18.276 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.276 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.276 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.276 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T09:17:18.277 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.277 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.277 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.277 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.277 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2024-04-24T09:17:18.277 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2024-04-24T09:17:18.277 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.277 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.277 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.277 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.278 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.278 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.278 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.278 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2024-04-24T09:17:18.278 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.278 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.278 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.278 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.279 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-04-24T09:17:18.279 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-04-24T09:17:18.279 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.279 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.279 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.279 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2024-04-24T09:17:18.279 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.279 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.279 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.280 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2024-04-24T09:17:18.280 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2024-04-24T09:17:18.280 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.280 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T09:17:18.280 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.280 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.280 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.280 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.280 INFO:teuthology.orchestra.run.smithi069.stdout: "oob_default_addr": { 2024-04-24T09:17:18.280 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "oob_default_addr", 2024-04-24T09:17:18.281 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.281 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.281 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.281 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "169.254.1.1", 2024-04-24T09:17:18.281 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.281 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.281 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.281 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T09:17:18.281 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.281 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.282 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.282 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.282 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2024-04-24T09:17:18.282 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2024-04-24T09:17:18.282 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.282 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.282 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.282 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T09:17:18.283 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.283 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.283 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.283 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T09:17:18.283 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.283 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.283 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.283 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.283 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2024-04-24T09:17:18.283 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2024-04-24T09:17:18.284 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.284 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.284 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.284 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.284 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.284 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.284 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.284 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T09:17:18.284 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.285 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.285 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.285 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.285 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2024-04-24T09:17:18.285 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2024-04-24T09:17:18.285 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.285 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.285 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.285 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.286 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.286 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.286 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.286 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T09:17:18.286 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.286 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.286 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.286 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.286 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2024-04-24T09:17:18.286 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2024-04-24T09:17:18.287 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.287 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.287 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.287 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.287 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.287 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.287 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.287 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T09:17:18.287 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.287 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.288 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.288 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.288 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2024-04-24T09:17:18.288 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2024-04-24T09:17:18.288 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.288 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.288 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.288 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.288 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.289 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.289 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.289 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T09:17:18.289 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.289 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.289 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.289 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.289 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2024-04-24T09:17:18.289 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2024-04-24T09:17:18.289 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.290 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.290 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.290 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.290 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.290 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.290 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.290 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T09:17:18.290 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.290 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.291 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.291 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.291 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2024-04-24T09:17:18.291 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2024-04-24T09:17:18.291 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.291 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.291 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.291 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2024-04-24T09:17:18.291 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.291 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.292 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.292 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2024-04-24T09:17:18.292 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.292 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.292 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.292 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.292 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2024-04-24T09:17:18.292 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2024-04-24T09:17:18.292 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.293 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.293 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.293 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.293 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.293 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.293 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.293 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T09:17:18.293 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.293 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.293 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.294 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.294 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2024-04-24T09:17:18.294 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2024-04-24T09:17:18.294 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.294 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.294 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.294 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.294 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.294 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.295 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.295 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T09:17:18.295 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.295 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.295 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.295 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.295 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2024-04-24T09:17:18.295 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2024-04-24T09:17:18.295 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.295 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.296 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.296 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.296 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.296 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.296 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.296 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T09:17:18.296 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.296 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.296 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.296 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.297 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2024-04-24T09:17:18.297 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2024-04-24T09:17:18.297 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.297 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.297 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.297 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.297 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.297 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.297 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.298 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T09:17:18.298 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.298 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.298 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.298 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.298 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2024-04-24T09:17:18.298 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2024-04-24T09:17:18.298 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.298 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.299 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.299 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.299 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.299 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.299 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.299 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T09:17:18.299 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.299 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.299 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.300 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.300 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2024-04-24T09:17:18.300 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2024-04-24T09:17:18.300 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.300 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.300 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.300 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.300 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.300 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.300 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.301 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T09:17:18.301 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.301 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.301 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.301 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.301 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.301 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.301 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.301 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2024-04-24T09:17:18.302 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.302 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.302 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.302 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.302 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.302 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.302 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.302 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.302 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.303 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.303 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.303 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.303 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.303 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.303 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.303 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.303 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.303 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.303 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.304 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.304 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.304 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.304 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.304 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.304 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.304 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.304 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.304 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.305 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.305 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.305 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.305 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.305 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.305 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.305 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.305 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.305 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.306 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.306 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.306 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.306 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.306 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.306 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.306 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.306 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.306 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.306 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.307 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.307 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.307 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.307 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.307 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.307 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.307 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.307 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.307 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.308 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.308 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.308 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.308 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.308 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.308 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.308 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.308 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.308 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.308 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.309 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.309 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.309 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.309 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.309 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.309 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.309 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.309 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2024-04-24T09:17:18.309 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2024-04-24T09:17:18.310 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.310 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.310 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.310 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2024-04-24T09:17:18.310 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.310 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.310 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.310 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T09:17:18.311 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.311 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.311 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.311 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.311 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2024-04-24T09:17:18.311 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2024-04-24T09:17:18.311 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.311 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.311 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.311 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2024-04-24T09:17:18.312 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.312 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.312 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.312 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T09:17:18.312 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.312 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.312 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.312 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.312 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.313 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.313 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.313 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2024-04-24T09:17:18.313 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.313 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.313 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.313 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T09:17:18.313 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T09:17:18.313 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.313 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.314 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.314 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:18.314 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.314 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.314 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.314 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.314 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.314 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.314 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.315 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.315 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T09:17:18.315 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T09:17:18.315 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.315 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.315 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.315 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.315 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.315 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.316 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.316 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.316 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.316 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.316 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.316 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.316 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T09:17:18.316 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T09:17:18.316 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.316 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.317 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.317 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.317 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.317 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.317 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.317 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.317 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.317 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.317 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.318 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.318 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2024-04-24T09:17:18.318 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T09:17:18.318 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.318 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.318 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.318 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.318 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.318 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.319 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.319 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.319 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.319 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.319 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.319 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.319 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T09:17:18.319 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T09:17:18.319 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.319 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.320 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.320 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.320 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.320 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.320 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.320 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.320 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.320 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.320 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.320 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T09:17:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T09:17:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T09:17:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T09:17:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.323 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.323 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.323 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.323 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.323 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.323 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.323 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.323 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.324 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.324 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.324 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T09:17:18.324 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T09:17:18.324 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.324 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.324 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.324 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.324 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.324 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.325 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.325 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.325 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.325 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.325 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.325 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.325 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T09:17:18.325 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T09:17:18.325 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.326 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.326 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.326 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.326 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.326 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.326 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.326 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.326 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.326 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.326 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.327 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.327 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T09:17:18.327 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T09:17:18.327 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.327 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.327 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.327 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.327 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.327 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.328 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.328 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.328 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.328 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.328 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.328 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.328 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T09:17:18.328 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T09:17:18.328 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.328 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.329 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.329 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.329 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.329 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.329 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.329 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.329 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.329 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.329 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.329 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.330 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T09:17:18.330 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T09:17:18.330 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.330 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.330 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.330 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.330 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.330 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.330 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.330 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.331 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.331 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.331 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.331 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.331 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T09:17:18.331 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T09:17:18.331 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.331 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.331 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.332 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.332 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.332 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.332 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.332 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.332 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.332 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.332 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.332 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.332 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T09:17:18.333 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T09:17:18.333 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.333 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.333 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.333 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.333 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.333 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.333 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.333 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.334 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.334 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.334 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.334 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.334 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T09:17:18.334 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T09:17:18.334 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.334 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.334 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.335 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-24T09:17:18.335 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.335 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.335 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.335 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.335 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.335 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.335 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.335 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.336 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T09:17:18.336 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T09:17:18.336 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.336 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.336 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.336 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.336 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.336 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.336 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.336 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.337 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.337 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.337 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.337 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.337 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2024-04-24T09:17:18.337 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2024-04-24T09:17:18.337 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.337 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.337 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.338 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.338 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.338 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.338 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.338 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.338 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.338 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.338 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.339 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.339 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T09:17:18.339 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T09:17:18.339 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.339 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.339 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.339 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-24T09:17:18.339 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.339 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.340 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.340 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.340 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.340 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.340 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.340 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.340 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T09:17:18.340 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T09:17:18.340 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.341 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.341 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.341 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.341 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.341 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.341 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.341 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.341 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.341 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.341 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.342 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.342 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T09:17:18.342 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T09:17:18.342 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.342 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.342 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.342 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.342 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.342 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.343 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.343 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.343 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.343 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.343 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.343 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.343 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T09:17:18.343 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T09:17:18.344 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.344 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.344 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.344 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.344 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.344 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.344 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.344 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.344 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.345 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.345 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.345 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.345 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T09:17:18.345 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T09:17:18.345 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.345 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.345 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.345 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.345 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.346 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.346 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.346 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.346 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.346 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.346 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.346 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.346 INFO:teuthology.orchestra.run.smithi069.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-24T09:17:18.346 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-24T09:17:18.346 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.347 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.347 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.347 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "[]", 2024-04-24T09:17:18.347 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.347 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.347 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.347 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.347 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.347 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.348 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.348 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.348 INFO:teuthology.orchestra.run.smithi069.stdout: "MULTICLUSTER_CONFIG": { 2024-04-24T09:17:18.348 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-24T09:17:18.348 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.348 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.348 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.348 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "{}", 2024-04-24T09:17:18.348 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.348 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.349 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.349 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.349 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.349 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.349 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.349 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.349 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T09:17:18.349 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T09:17:18.349 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.349 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.350 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.350 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.350 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.350 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.350 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.350 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.350 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.350 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.350 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.351 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.351 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T09:17:18.351 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T09:17:18.351 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.351 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.351 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.351 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.351 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.351 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.351 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.352 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.352 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.352 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.352 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.352 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.352 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T09:17:18.352 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T09:17:18.352 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.352 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.352 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.353 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.353 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.353 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.353 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.353 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.353 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.353 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.353 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.353 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.354 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T09:17:18.354 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T09:17:18.354 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.354 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.354 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.354 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.354 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.354 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.354 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.355 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.355 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.355 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.355 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.355 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.355 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T09:17:18.355 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T09:17:18.355 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.355 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.356 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.356 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.356 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.356 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.356 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.356 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.356 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.356 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.356 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.356 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.357 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T09:17:18.357 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T09:17:18.357 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.357 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.357 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.357 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.357 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.357 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.357 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.357 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.358 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.358 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.358 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.358 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.358 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T09:17:18.358 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T09:17:18.358 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.358 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.358 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.359 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.359 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.359 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.359 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.359 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.359 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.359 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.359 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.359 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.359 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T09:17:18.360 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T09:17:18.360 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.360 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.360 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.360 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.360 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.360 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.360 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.360 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.361 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.361 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.361 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.361 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.361 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T09:17:18.361 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T09:17:18.361 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.361 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.361 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.362 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.362 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.362 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.362 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.362 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.362 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.362 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.362 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.362 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.363 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T09:17:18.363 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T09:17:18.363 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.363 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.363 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.363 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.363 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.363 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.363 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.363 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.364 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.364 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.364 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.364 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.364 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T09:17:18.364 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T09:17:18.364 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.364 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.364 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.365 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T09:17:18.365 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.365 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.365 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.365 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.365 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.365 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.365 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.365 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.365 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T09:17:18.366 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T09:17:18.366 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.366 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.366 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.366 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:18.366 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.366 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.366 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.366 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.367 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.367 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.367 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.367 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.367 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T09:17:18.367 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T09:17:18.367 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.367 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.367 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.367 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2024-04-24T09:17:18.368 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.368 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.368 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.368 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.368 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.368 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.368 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.368 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.368 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T09:17:18.368 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T09:17:18.369 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.369 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.369 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.369 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2024-04-24T09:17:18.369 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.369 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.369 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.369 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.369 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.369 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.370 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.370 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.370 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T09:17:18.370 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T09:17:18.370 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.370 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.370 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.370 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.371 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.371 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.371 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.371 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.371 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.371 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.371 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.371 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.371 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T09:17:18.371 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T09:17:18.372 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.372 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.372 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.372 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-24T09:17:18.372 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.372 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.372 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.372 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.372 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.373 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.373 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.373 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.373 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T09:17:18.373 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T09:17:18.373 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.373 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.373 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.373 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.374 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.374 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.374 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.374 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.374 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.374 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.374 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.374 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.374 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T09:17:18.374 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T09:17:18.375 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.375 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.375 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.375 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.375 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.375 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.375 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.375 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.375 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.375 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.376 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.376 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.376 INFO:teuthology.orchestra.run.smithi069.stdout: "UNSAFE_TLS_v1_2": { 2024-04-24T09:17:18.376 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-24T09:17:18.376 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.376 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.376 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.376 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.376 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.376 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.377 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.377 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.377 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.377 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.377 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.377 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.377 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T09:17:18.377 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T09:17:18.377 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.377 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.378 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.378 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-24T09:17:18.378 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.378 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.378 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.378 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.378 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.378 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.378 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.379 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.379 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T09:17:18.379 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T09:17:18.379 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.379 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.379 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.379 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:18.379 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.379 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.379 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.380 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.380 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.380 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.380 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.380 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.380 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T09:17:18.380 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T09:17:18.380 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.380 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.381 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.381 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-24T09:17:18.381 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.381 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.381 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.381 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.381 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.381 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.381 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.381 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.382 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2024-04-24T09:17:18.382 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2024-04-24T09:17:18.382 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.382 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.382 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.382 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.382 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.382 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.382 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.383 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.383 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.383 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.383 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.383 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.383 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2024-04-24T09:17:18.383 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2024-04-24T09:17:18.383 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.383 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.384 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.384 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.384 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.384 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.384 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.384 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.384 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.384 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.384 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.384 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.385 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2024-04-24T09:17:18.385 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2024-04-24T09:17:18.385 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.385 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.385 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.385 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.385 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.385 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.385 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.385 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2024-04-24T09:17:18.386 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.386 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.386 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.386 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.386 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2024-04-24T09:17:18.386 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2024-04-24T09:17:18.386 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.386 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.387 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.387 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2024-04-24T09:17:18.387 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.387 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.387 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.387 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.387 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.387 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.387 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.388 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.388 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-04-24T09:17:18.388 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-04-24T09:17:18.388 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.388 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.388 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.388 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.388 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.388 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.389 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.389 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.389 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.389 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.389 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.389 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.389 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.389 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.389 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.389 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.390 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.390 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.390 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.390 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.390 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.390 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.390 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.390 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.390 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.391 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.391 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.391 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.391 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.391 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.391 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.391 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.391 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.391 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.391 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.392 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.392 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.392 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.392 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.392 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.392 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.392 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.392 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.392 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.392 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.393 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.393 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.393 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.393 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.393 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.393 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.393 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.393 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.393 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.394 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.394 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.394 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.394 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.394 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.394 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.394 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.394 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.394 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.394 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.395 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.395 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.395 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.395 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.395 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.395 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.395 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.395 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.395 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.396 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.396 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.396 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.396 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.396 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.396 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.396 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.396 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.396 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.396 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2024-04-24T09:17:18.397 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2024-04-24T09:17:18.397 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.397 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.397 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.397 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.397 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.397 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.397 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.397 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2024-04-24T09:17:18.398 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.398 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.398 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.398 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.398 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2024-04-24T09:17:18.398 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T09:17:18.398 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.398 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.398 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.398 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.399 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.399 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.399 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.399 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.399 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.399 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.399 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.399 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.399 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-24T09:17:18.399 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-24T09:17:18.400 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.400 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.400 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.400 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-04-24T09:17:18.400 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.400 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.400 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.400 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.400 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.401 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.401 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.401 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.401 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-24T09:17:18.401 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-24T09:17:18.401 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.401 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.401 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.401 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2024-04-24T09:17:18.402 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.402 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.402 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.402 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.402 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.402 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.402 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.402 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.402 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-04-24T09:17:18.403 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-04-24T09:17:18.403 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.403 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.403 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.403 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.403 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.403 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.403 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.403 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.403 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.404 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.404 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.404 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.404 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2024-04-24T09:17:18.404 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2024-04-24T09:17:18.404 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.404 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.404 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.404 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2024-04-24T09:17:18.405 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.405 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.405 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.405 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.405 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.405 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.405 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.405 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.405 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-04-24T09:17:18.405 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-04-24T09:17:18.406 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.406 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.406 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.406 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2024-04-24T09:17:18.406 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.406 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.406 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.406 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.406 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2024-04-24T09:17:18.407 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.407 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.407 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.407 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.407 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.407 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.407 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-04-24T09:17:18.407 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-04-24T09:17:18.407 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.407 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.408 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.408 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-04-24T09:17:18.408 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-04-24T09:17:18.408 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-04-24T09:17:18.408 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.408 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.408 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.408 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.408 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.408 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.409 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2024-04-24T09:17:18.409 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2024-04-24T09:17:18.409 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.409 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.409 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.409 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.409 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.409 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.409 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.409 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.410 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.410 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.410 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.410 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.410 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.410 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.410 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.410 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-04-24T09:17:18.411 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.411 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.411 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.411 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2024-04-24T09:17:18.411 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2024-04-24T09:17:18.411 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.411 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.411 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.411 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.411 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.412 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.412 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.412 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2024-04-24T09:17:18.412 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.412 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.412 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.412 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.412 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.412 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.413 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.413 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.413 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.413 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.413 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.413 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.413 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.413 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.413 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.413 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.414 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.414 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.414 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.414 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.414 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.414 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.414 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.414 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.415 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.415 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.415 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.415 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.415 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.415 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.415 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.415 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.415 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.415 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.416 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.416 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.416 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.416 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.416 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.416 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.416 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.416 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.416 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.417 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.417 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.417 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.417 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.417 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.417 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.417 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.417 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.417 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.418 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.418 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.418 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.418 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.418 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.418 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.418 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.418 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.418 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.419 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.419 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.419 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.419 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.419 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.419 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.419 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.419 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.419 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.419 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.420 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.420 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.420 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.420 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2024-04-24T09:17:18.420 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2024-04-24T09:17:18.420 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.420 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.420 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.420 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2024-04-24T09:17:18.421 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.421 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.421 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.421 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T09:17:18.421 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.421 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.421 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.421 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.422 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2024-04-24T09:17:18.422 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2024-04-24T09:17:18.422 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.422 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.422 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.422 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2024-04-24T09:17:18.422 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.422 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.422 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.423 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T09:17:18.423 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.423 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.423 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.423 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.423 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2024-04-24T09:17:18.423 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2024-04-24T09:17:18.423 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.423 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.424 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.424 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2024-04-24T09:17:18.424 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.424 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.424 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.424 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2024-04-24T09:17:18.424 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.424 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.425 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.425 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.425 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2024-04-24T09:17:18.425 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2024-04-24T09:17:18.425 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.425 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.425 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.425 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-04-24T09:17:18.425 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.426 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.426 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.426 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T09:17:18.426 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.426 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.426 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.426 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.426 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2024-04-24T09:17:18.427 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2024-04-24T09:17:18.427 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.427 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.427 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.427 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.427 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.427 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.427 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.428 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T09:17:18.428 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.428 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.428 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.428 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.428 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-24T09:17:18.428 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-24T09:17:18.428 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.428 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.429 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.429 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-24T09:17:18.429 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.429 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.429 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.429 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T09:17:18.429 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.429 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.429 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.430 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.430 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2024-04-24T09:17:18.430 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2024-04-24T09:17:18.430 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.430 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.430 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.430 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2024-04-24T09:17:18.430 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.431 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.431 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.431 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T09:17:18.431 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.431 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.431 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.431 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.431 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.432 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.432 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.432 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2024-04-24T09:17:18.432 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.432 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.432 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.432 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.432 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.432 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.433 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.433 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.433 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.433 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.433 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.433 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.433 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.433 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.433 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.434 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.434 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.434 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.434 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.434 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.434 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.434 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.434 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.435 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.435 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.435 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.435 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.435 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.435 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.435 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.435 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.436 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.436 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.436 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.436 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.436 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.436 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.436 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.436 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.436 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.437 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.437 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.437 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.437 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.437 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.437 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.437 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.437 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.437 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.437 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.438 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.438 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.438 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.438 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.438 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.438 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.438 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.438 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.439 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.439 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.439 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.439 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.439 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.439 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.439 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.439 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.439 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.440 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.440 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.440 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.440 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.440 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.440 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.440 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2024-04-24T09:17:18.440 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2024-04-24T09:17:18.440 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.440 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.441 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.441 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-04-24T09:17:18.441 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.441 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.441 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.441 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.441 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.441 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.441 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.442 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.442 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2024-04-24T09:17:18.442 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2024-04-24T09:17:18.442 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.442 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.442 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.442 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2024-04-24T09:17:18.442 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.443 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.443 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.443 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.443 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.443 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.443 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.443 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.443 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-24T09:17:18.443 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-24T09:17:18.444 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.444 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.444 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.444 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-24T09:17:18.444 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.444 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.444 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.444 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.444 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.444 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.445 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.445 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.445 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.445 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.445 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.445 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2024-04-24T09:17:18.445 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2024-04-24T09:17:18.445 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2024-04-24T09:17:18.446 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.446 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2024-04-24T09:17:18.446 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2024-04-24T09:17:18.446 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.446 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.446 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.446 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2024-04-24T09:17:18.446 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.446 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.447 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.447 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T09:17:18.447 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.447 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.447 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.447 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.447 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2024-04-24T09:17:18.447 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2024-04-24T09:17:18.447 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.447 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.448 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.448 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2024-04-24T09:17:18.448 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.448 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.448 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.448 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T09:17:18.448 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.448 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.448 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.449 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.449 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2024-04-24T09:17:18.449 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2024-04-24T09:17:18.449 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.449 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.449 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.449 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.449 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.450 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.450 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.450 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2024-04-24T09:17:18.450 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.450 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.450 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.450 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.450 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-24T09:17:18.450 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-24T09:17:18.451 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.451 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.451 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.451 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2024-04-24T09:17:18.451 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2024-04-24T09:17:18.451 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.451 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.451 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T09:17:18.451 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.452 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.452 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.452 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.452 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.452 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.452 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.452 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.452 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.452 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.452 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.453 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.453 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.453 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.453 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.453 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.453 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.453 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.453 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.453 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.453 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.454 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.454 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.454 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.454 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.454 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.454 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.454 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.454 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.455 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.455 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.455 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.455 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.455 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.455 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.455 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.455 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.455 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.456 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.456 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.456 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.456 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.456 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.456 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.456 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.456 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.456 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.457 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.457 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.457 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.457 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.457 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.457 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.457 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.457 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.457 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.457 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.458 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.458 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.458 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.458 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.458 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.458 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.458 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.458 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.458 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.459 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.459 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.459 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.459 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.459 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.459 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.459 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.459 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.459 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2024-04-24T09:17:18.459 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2024-04-24T09:17:18.460 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.460 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.460 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.460 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.460 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.460 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.460 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.460 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2024-04-24T09:17:18.460 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.461 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.461 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.461 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.461 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2024-04-24T09:17:18.461 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2024-04-24T09:17:18.461 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.461 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.461 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.462 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2024-04-24T09:17:18.462 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.462 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.462 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.462 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2024-04-24T09:17:18.462 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.462 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.462 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.463 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.463 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-04-24T09:17:18.463 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-04-24T09:17:18.463 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.463 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.463 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.463 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2024-04-24T09:17:18.463 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.463 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.464 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.464 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:17:18.464 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.464 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.464 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.464 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.464 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2024-04-24T09:17:18.464 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2024-04-24T09:17:18.464 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.465 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.465 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.465 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-24T09:17:18.465 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-24T09:17:18.465 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2024-04-24T09:17:18.465 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.465 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T09:17:18.465 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.465 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.465 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.466 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.466 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2024-04-24T09:17:18.466 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2024-04-24T09:17:18.466 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.466 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.466 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.466 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.466 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.466 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.467 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.467 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2024-04-24T09:17:18.467 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.467 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.467 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.467 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.467 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2024-04-24T09:17:18.467 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2024-04-24T09:17:18.467 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.468 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.468 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.468 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2024-04-24T09:17:18.468 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.468 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.468 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.468 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:17:18.468 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.468 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.468 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.469 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.469 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.469 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.469 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.469 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2024-04-24T09:17:18.469 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.469 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.469 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.469 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.469 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.470 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.470 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.470 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.470 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.470 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.470 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.470 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.470 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.470 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.471 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.471 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.471 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.471 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.471 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.471 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.471 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.471 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.471 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.472 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.472 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.472 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.472 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.472 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.472 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.472 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.472 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.472 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.473 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.473 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.473 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.473 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.473 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.473 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.473 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.473 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.473 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.473 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.474 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.474 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.474 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.474 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.474 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.474 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.474 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.474 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.474 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.475 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.475 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.475 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.475 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.475 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.475 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.475 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.475 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.475 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.475 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.476 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.476 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.476 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.476 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.476 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.476 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.476 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.476 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.476 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.477 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.477 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.477 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.477 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.477 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.477 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.477 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2024-04-24T09:17:18.477 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.477 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.477 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.478 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.478 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.478 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.478 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.478 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.478 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.478 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.478 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.478 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.479 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.479 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.479 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.479 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.479 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.479 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.479 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.479 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.479 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.479 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.480 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.480 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.480 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.480 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.480 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.480 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.480 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.480 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.480 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.480 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.481 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.481 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.481 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.481 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.481 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.481 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.481 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.481 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.481 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.481 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.482 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.482 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.482 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.482 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.482 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.482 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.482 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.482 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.482 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.483 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.483 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.483 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.483 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.483 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.483 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.483 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.483 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.483 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.484 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.484 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.484 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.484 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.484 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.484 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.484 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.484 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.484 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.485 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.485 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.485 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.485 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.485 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.485 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.485 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.485 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2024-04-24T09:17:18.485 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.485 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.486 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.486 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2024-04-24T09:17:18.486 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2024-04-24T09:17:18.486 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.486 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.486 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.486 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2024-04-24T09:17:18.486 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.487 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.487 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.487 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T09:17:18.487 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.487 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.487 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.487 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.487 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2024-04-24T09:17:18.488 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2024-04-24T09:17:18.488 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.488 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.488 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.488 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:18.488 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2024-04-24T09:17:18.488 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.488 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.488 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T09:17:18.489 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.489 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.489 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.489 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.489 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.489 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.489 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.489 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.489 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.490 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.490 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.490 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.490 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.490 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.490 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.490 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.490 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.491 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.491 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.491 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.491 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.491 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.491 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.491 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.491 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.491 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.491 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.492 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.492 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.492 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.492 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.492 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.492 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.492 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.492 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.492 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.493 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.493 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.493 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.493 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.493 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.493 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.493 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.493 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.493 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.493 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.494 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.494 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.494 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.494 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.494 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.494 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.494 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.494 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.494 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.495 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.495 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.495 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.495 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.495 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.495 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.495 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.495 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.495 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.495 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.496 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.496 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.496 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.496 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.496 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.496 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.496 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.496 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.496 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.496 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.497 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.497 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.497 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2024-04-24T09:17:18.497 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.497 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.497 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.497 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2024-04-24T09:17:18.497 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2024-04-24T09:17:18.497 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.498 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.498 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.498 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2024-04-24T09:17:18.498 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.498 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.498 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.498 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2024-04-24T09:17:18.498 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T09:17:18.498 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.499 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.499 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.499 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.499 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.499 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.499 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.499 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.499 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.500 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.500 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.500 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.500 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.500 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.500 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.500 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.500 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.500 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.501 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.501 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.501 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.501 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.501 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.501 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.501 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.501 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.501 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.502 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.502 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.502 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.502 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.502 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.502 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.502 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.502 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.503 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.503 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.503 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.503 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.503 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.503 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.503 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.503 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.503 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.503 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.504 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.504 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.504 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.504 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.504 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.504 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.504 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.504 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.504 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.504 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.505 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.505 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.505 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.505 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.505 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.505 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.505 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.505 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.505 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.506 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.506 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.506 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.506 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.506 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.506 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.506 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.506 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.506 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.507 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2024-04-24T09:17:18.507 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2024-04-24T09:17:18.507 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.507 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.507 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.507 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.507 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.507 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.507 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.508 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2024-04-24T09:17:18.508 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T09:17:18.508 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.508 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.508 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.508 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2024-04-24T09:17:18.508 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2024-04-24T09:17:18.508 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.508 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.508 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.509 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2024-04-24T09:17:18.509 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.509 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.509 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.509 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2024-04-24T09:17:18.509 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.509 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.509 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.509 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.510 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2024-04-24T09:17:18.510 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2024-04-24T09:17:18.510 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.510 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.510 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.510 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2024-04-24T09:17:18.510 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.510 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.510 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.511 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T09:17:18.511 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.511 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.511 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.511 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.511 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2024-04-24T09:17:18.511 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2024-04-24T09:17:18.511 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.511 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.512 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.512 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.512 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.512 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.512 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.512 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2024-04-24T09:17:18.512 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.512 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.512 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.512 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.513 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2024-04-24T09:17:18.513 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2024-04-24T09:17:18.513 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.513 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.513 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.513 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2024-04-24T09:17:18.513 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.513 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.513 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.513 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T09:17:18.514 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T09:17:18.514 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.514 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.514 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.514 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.514 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.514 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.514 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2024-04-24T09:17:18.514 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.515 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.515 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.515 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.515 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.515 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.515 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.515 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.515 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.515 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.516 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.516 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.516 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.516 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.516 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.516 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.516 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.516 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.516 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.517 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.517 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.517 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.517 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.517 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.517 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.517 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.517 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.517 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.517 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.518 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.518 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.518 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.518 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.518 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.518 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.518 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.518 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.518 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.519 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.519 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.519 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.519 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.519 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.519 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.519 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.519 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.519 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.520 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.520 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.520 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.520 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.520 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.520 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.520 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.520 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.520 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.520 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.521 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.521 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.521 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.521 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.521 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.521 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.521 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.521 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.521 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.521 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.522 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.522 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.522 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.522 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.522 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.522 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.522 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.522 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.522 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.523 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2024-04-24T09:17:18.523 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.523 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.523 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.523 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.523 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.523 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.523 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.523 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.524 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.524 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.524 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.524 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.524 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.524 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.524 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.524 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.524 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.524 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.525 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.525 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.525 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.525 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.525 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.525 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.525 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.525 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.525 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.526 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.526 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.526 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.526 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.526 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.526 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.526 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.526 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.527 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.527 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.527 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.527 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.527 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.527 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.527 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.527 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.527 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.527 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.528 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.528 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.528 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.528 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.528 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.528 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.528 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.528 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.528 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.529 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.529 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.529 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.529 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.529 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.529 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.529 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.529 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.529 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.530 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.530 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.530 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.530 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.530 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.530 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.530 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.530 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.530 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.531 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.531 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.531 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.531 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.531 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2024-04-24T09:17:18.531 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.531 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.531 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.531 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.531 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.532 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.532 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.532 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.532 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.532 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.532 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.532 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.532 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.532 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.533 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.533 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.533 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.533 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.533 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.533 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.533 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.533 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.533 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.533 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.534 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.534 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.534 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.534 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.534 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.534 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.534 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.534 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.534 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.535 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.535 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.535 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.535 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.535 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.535 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.535 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.535 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.536 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.536 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.536 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.536 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.536 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.536 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.536 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.536 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.536 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.536 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.537 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.537 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.537 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.537 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.537 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.537 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.537 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.537 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.537 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.537 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.538 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.538 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.538 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.538 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.538 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.538 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.538 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.538 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.538 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.539 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.539 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.539 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.539 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.539 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.539 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.539 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-04-24T09:17:18.539 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.539 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.540 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.540 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2024-04-24T09:17:18.540 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2024-04-24T09:17:18.540 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.540 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.540 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.540 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.540 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.540 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.541 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.541 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T09:17:18.541 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.541 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.541 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.541 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.541 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.541 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.541 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.541 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.542 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.542 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.542 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.542 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.542 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.542 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.542 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.542 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.542 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.543 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.543 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.543 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.543 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.543 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.543 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.543 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.543 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.543 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.543 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.544 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.544 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.544 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.544 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.544 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.544 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.544 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.544 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.544 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.544 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.545 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.545 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.545 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.545 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.545 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.545 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.545 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.545 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.545 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.545 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.546 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.546 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.546 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.546 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.546 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.546 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.546 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.546 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.546 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.547 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.547 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.547 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.547 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.547 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.547 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.547 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.547 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.547 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.548 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.548 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.548 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.548 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.548 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.548 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.548 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.548 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.548 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.548 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2024-04-24T09:17:18.549 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-04-24T09:17:18.549 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.549 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.549 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.549 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.549 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.549 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.549 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.549 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-04-24T09:17:18.549 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2024-04-24T09:17:18.550 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2024-04-24T09:17:18.550 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.550 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2024-04-24T09:17:18.550 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.550 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.550 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.550 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.550 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.550 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.551 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.551 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2024-04-24T09:17:18.551 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.551 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.551 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.551 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.551 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.551 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.551 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.552 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.552 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.552 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.552 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.552 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.552 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.552 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.552 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.552 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.553 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.553 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.553 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.553 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.553 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.553 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.553 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.553 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.553 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.553 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.554 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.554 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.554 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.554 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.554 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.554 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.554 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.554 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.554 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.555 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.555 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.555 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.555 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.555 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.555 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.555 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.555 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.556 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.556 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.556 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.556 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.556 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.556 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.556 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.556 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.556 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.556 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.557 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.557 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.557 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.557 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.557 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.557 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.557 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.557 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.557 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.558 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.558 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.558 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.558 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.558 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.558 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.558 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.558 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.558 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.559 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.559 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.559 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.559 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.559 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.559 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2024-04-24T09:17:18.559 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.559 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.559 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.560 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.560 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.560 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.560 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.560 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.560 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.560 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.560 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.560 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.560 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.561 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.561 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.561 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.561 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.561 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.561 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.561 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.561 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.561 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.561 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.562 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.562 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.562 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.562 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.562 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.562 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.562 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.562 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.562 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.563 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.563 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.563 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.563 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.563 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.563 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.563 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.563 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.563 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.564 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.564 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.564 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.564 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.564 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.564 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.564 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.564 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.564 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.565 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.565 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.565 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.565 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.565 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.565 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.565 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.565 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.565 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.566 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.566 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.566 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.566 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.566 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.566 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.566 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.566 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.566 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.567 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.567 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.567 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.567 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.567 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.567 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.567 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.567 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.567 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2024-04-24T09:17:18.568 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.568 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.568 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.568 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.568 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.568 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.568 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.568 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.568 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.568 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.569 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.569 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.569 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.569 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.569 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.569 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.569 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.569 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.569 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.569 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.570 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.570 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.570 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.570 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.570 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.570 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.570 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.570 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.570 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.571 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.571 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.571 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.571 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.571 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.571 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.571 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.571 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.571 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.572 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.572 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.572 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.572 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.572 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.572 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.572 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.572 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.572 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.572 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.573 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.573 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.573 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.573 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.573 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.573 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.573 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.573 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.573 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.573 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.574 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.574 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.574 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.574 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.574 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.574 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.574 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.574 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.574 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.575 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.575 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.575 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.575 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.575 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.575 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.575 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-24T09:17:18.575 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-24T09:17:18.575 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.575 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.576 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.576 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-24T09:17:18.576 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.576 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.576 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.576 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.576 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.576 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.576 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.576 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.577 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2024-04-24T09:17:18.577 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2024-04-24T09:17:18.577 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:18.577 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.577 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.577 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-04-24T09:17:18.577 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2024-04-24T09:17:18.577 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.577 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.577 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2024-04-24T09:17:18.578 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T09:17:18.578 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.578 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.578 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.578 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.578 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.578 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.578 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2024-04-24T09:17:18.578 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.579 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.579 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.579 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2024-04-24T09:17:18.579 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2024-04-24T09:17:18.579 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.579 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.579 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.579 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.579 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.579 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T09:17:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-04-24T09:17:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-04-24T09:17:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.580 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.581 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.582 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.583 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.584 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.585 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.586 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.587 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.588 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2024-04-24T09:17:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2024-04-24T09:17:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.589 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2024-04-24T09:17:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2024-04-24T09:17:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.590 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-24T09:17:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-24T09:17:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.591 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-24T09:17:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2024-04-24T09:17:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.592 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.593 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.593 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.593 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2024-04-24T09:17:18.593 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.593 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.593 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.593 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2024-04-24T09:17:18.593 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2024-04-24T09:17:18.593 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.593 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.594 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.594 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.594 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.594 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.594 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.594 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.594 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.594 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.594 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.595 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.595 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2024-04-24T09:17:18.595 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2024-04-24T09:17:18.595 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.595 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.595 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.595 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.595 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.595 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.595 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.596 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T09:17:18.596 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T09:17:18.596 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.596 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.596 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.596 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.596 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.596 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.596 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.597 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.597 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.597 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.597 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.597 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.597 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.597 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.597 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.597 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.597 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.598 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.598 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.598 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.598 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.598 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.598 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.598 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.598 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.598 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.599 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.599 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.599 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.599 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.599 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.599 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.599 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.599 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.599 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.599 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.600 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.600 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.600 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.600 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.600 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.600 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.600 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.600 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.600 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.600 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.601 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.601 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.601 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.601 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.601 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.601 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.601 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.601 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.601 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.602 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.602 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.602 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.602 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.602 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.602 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.602 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.602 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.602 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.603 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.603 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.603 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.603 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.603 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.603 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.603 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.603 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.603 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.603 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2024-04-24T09:17:18.604 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2024-04-24T09:17:18.604 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.604 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.604 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.604 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.604 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.604 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.604 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.604 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.604 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.605 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.605 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.605 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.605 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T09:17:18.605 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T09:17:18.605 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.605 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.605 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.605 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2024-04-24T09:17:18.605 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.606 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.606 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.606 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.606 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.606 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.606 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.606 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.606 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2024-04-24T09:17:18.606 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2024-04-24T09:17:18.607 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:18.607 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.607 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.607 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2024-04-24T09:17:18.607 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.607 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.607 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.607 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.608 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.608 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.608 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.608 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.608 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-24T09:17:18.608 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-24T09:17:18.608 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.608 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.608 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.609 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-04-24T09:17:18.609 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.609 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.609 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.609 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T09:17:18.609 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.609 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.609 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.609 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.609 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-24T09:17:18.610 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-24T09:17:18.610 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.610 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.610 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.610 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2024-04-24T09:17:18.610 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.610 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.610 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.611 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T09:17:18.611 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.611 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.611 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.611 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.611 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2024-04-24T09:17:18.611 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2024-04-24T09:17:18.611 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.611 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.611 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.612 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2024-04-24T09:17:18.612 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.612 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.612 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.612 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.612 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.612 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.612 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.613 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.613 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-04-24T09:17:18.613 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-04-24T09:17:18.613 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.613 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.613 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.613 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2024-04-24T09:17:18.613 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.613 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.613 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.614 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2024-04-24T09:17:18.614 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2024-04-24T09:17:18.614 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.614 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.614 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.614 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.614 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.614 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.614 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-04-24T09:17:18.615 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-04-24T09:17:18.615 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.615 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.615 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.615 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-04-24T09:17:18.615 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-04-24T09:17:18.615 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-04-24T09:17:18.615 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.615 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.615 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.616 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.616 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.616 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.616 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.616 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.616 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.616 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-04-24T09:17:18.616 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.616 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.616 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.617 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.617 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.617 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.617 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.617 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.617 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.617 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.617 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.617 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.618 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.618 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.618 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.618 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.618 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.618 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.618 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.618 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.618 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.619 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.619 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.619 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.619 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.619 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.619 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.619 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.619 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.619 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.620 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.620 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.620 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.620 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.620 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.620 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.620 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.620 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.621 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.621 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.621 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.621 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.621 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.621 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.621 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.621 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.621 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.621 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.622 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.622 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.622 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.622 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.622 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.622 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.622 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.622 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.622 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.623 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.623 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.623 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.623 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.623 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.623 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.623 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.623 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.623 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.623 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.624 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.624 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.624 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.624 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.624 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.624 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.624 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2024-04-24T09:17:18.624 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2024-04-24T09:17:18.624 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.625 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.625 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.625 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:18.625 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.625 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.625 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.625 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.625 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.625 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.626 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.626 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.626 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2024-04-24T09:17:18.626 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T09:17:18.626 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.626 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.626 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.626 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.626 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.626 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.627 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.627 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.627 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.627 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.627 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.627 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.627 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2024-04-24T09:17:18.628 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2024-04-24T09:17:18.628 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.628 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.628 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.628 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.628 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.628 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.628 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.628 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.629 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.629 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.629 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.629 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.629 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.629 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.629 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.629 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2024-04-24T09:17:18.629 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.630 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.630 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.630 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2024-04-24T09:17:18.630 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2024-04-24T09:17:18.630 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.630 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.630 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.630 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.630 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.631 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.631 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.631 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.631 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.631 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.631 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.631 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.631 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-04-24T09:17:18.631 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-04-24T09:17:18.632 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.632 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.632 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.632 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.632 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.632 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.632 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.632 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.632 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.632 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.633 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.633 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.633 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.633 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.633 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.633 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.633 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.633 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.633 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.633 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.634 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.634 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.634 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.634 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.634 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.634 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.634 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.634 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.634 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.635 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.635 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.635 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.635 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.635 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.635 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.635 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.635 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.635 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.635 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.636 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.636 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.636 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.636 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.636 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.636 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.636 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.636 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.636 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.637 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.637 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.637 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.637 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.637 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.637 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.637 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.637 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.637 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.637 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.638 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.638 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.638 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.638 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.638 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.638 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.638 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.638 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.638 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.639 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.639 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.639 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.639 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.639 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.639 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.639 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.639 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.639 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.640 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.640 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.640 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.640 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.640 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.640 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.640 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-24T09:17:18.640 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-24T09:17:18.640 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.641 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.641 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.641 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.641 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.641 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.641 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.641 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.641 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.641 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.642 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.642 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.642 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-24T09:17:18.642 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-24T09:17:18.642 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.642 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.642 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.642 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.642 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.643 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.643 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.643 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.643 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.643 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.643 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.643 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.643 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.643 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.644 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.644 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2024-04-24T09:17:18.644 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.644 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.644 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.644 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.644 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.644 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.644 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.644 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.645 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.645 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.645 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.645 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.645 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.645 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.645 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.645 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.645 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.645 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.646 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.646 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.646 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.646 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.646 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.646 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.646 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.646 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.646 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.647 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.647 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.647 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.647 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.647 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.647 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.647 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.647 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.647 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.647 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.648 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.648 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.648 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.648 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.648 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.648 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.648 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.648 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.648 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.648 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.649 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.649 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.649 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.649 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.649 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.649 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.649 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.649 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.649 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.650 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.650 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.650 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.650 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.650 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.650 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.650 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.650 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.650 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.651 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.651 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.651 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.651 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.651 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.651 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.651 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.651 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.651 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.652 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.652 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.652 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2024-04-24T09:17:18.652 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.652 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.652 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.652 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.652 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.652 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.653 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.653 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.653 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.653 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.653 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.653 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.653 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.653 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.653 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.653 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.654 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.654 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.654 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.654 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.654 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.654 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.654 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.654 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.654 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.654 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.655 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.655 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.655 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.655 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.655 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.655 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.655 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.655 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.655 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.656 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.656 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.656 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.656 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.656 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.656 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.656 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.656 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.657 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.657 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.657 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.657 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.657 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.657 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.657 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.657 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.657 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.658 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.658 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.658 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.658 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.658 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.658 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.658 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.658 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.659 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.659 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.659 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.659 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.659 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.659 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.659 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.659 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.659 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.659 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.660 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.660 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.660 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.660 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2024-04-24T09:17:18.660 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2024-04-24T09:17:18.660 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.660 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.660 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.660 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2024-04-24T09:17:18.660 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.661 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.661 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.661 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T09:17:18.661 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.661 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.661 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.661 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.661 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.661 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.661 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.662 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2024-04-24T09:17:18.662 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.662 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.662 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.662 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.662 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.662 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.662 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.662 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.663 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.663 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.663 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.663 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.663 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.663 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.663 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.663 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.664 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.664 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.664 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.664 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.664 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.664 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.664 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.664 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.664 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.664 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.665 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.665 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.665 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.665 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.665 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.665 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.665 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.665 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.666 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.666 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.666 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.666 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.666 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.666 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.666 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.666 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.666 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.667 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.667 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.667 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.667 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.667 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.667 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.667 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.667 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.667 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.668 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.668 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.668 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.668 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.668 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.668 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.668 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.668 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.668 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.669 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.669 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.669 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.669 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.669 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.669 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.669 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.669 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.670 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.670 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.670 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.670 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.670 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2024-04-24T09:17:18.670 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2024-04-24T09:17:18.670 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.670 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.671 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.671 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.671 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.671 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.671 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.671 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.671 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.671 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.671 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.672 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.672 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2024-04-24T09:17:18.672 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2024-04-24T09:17:18.672 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.672 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.672 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.672 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2024-04-24T09:17:18.672 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.672 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.672 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.673 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.673 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.673 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.673 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.673 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.673 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2024-04-24T09:17:18.673 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2024-04-24T09:17:18.673 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.673 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.673 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.674 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.674 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.674 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.674 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.674 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.674 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.674 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.674 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.675 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.675 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2024-04-24T09:17:18.675 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2024-04-24T09:17:18.675 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.675 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.675 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.675 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.675 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.675 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.675 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.676 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.676 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.676 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.676 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.676 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.676 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2024-04-24T09:17:18.676 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2024-04-24T09:17:18.676 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:18.676 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.676 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.677 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.677 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.677 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.677 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.677 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.677 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.677 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.677 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.677 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.677 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2024-04-24T09:17:18.678 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2024-04-24T09:17:18.678 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.678 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.678 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.678 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.678 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.678 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.678 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.678 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.679 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.679 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.679 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.679 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.679 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2024-04-24T09:17:18.679 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2024-04-24T09:17:18.679 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.679 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.679 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.679 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.680 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.680 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.680 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.680 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.680 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.680 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.680 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.680 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.680 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2024-04-24T09:17:18.680 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2024-04-24T09:17:18.681 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.681 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.681 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.681 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.681 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.681 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.681 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.681 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.681 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.681 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.682 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.682 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.682 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2024-04-24T09:17:18.682 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2024-04-24T09:17:18.682 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.682 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.682 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.682 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.682 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-24T09:17:18.683 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2024-04-24T09:17:18.683 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.683 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.683 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.683 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.683 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.683 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.683 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2024-04-24T09:17:18.683 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2024-04-24T09:17:18.683 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.684 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.684 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.684 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.684 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.684 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.684 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.684 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.684 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.684 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.685 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.685 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.685 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2024-04-24T09:17:18.685 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2024-04-24T09:17:18.685 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.685 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.685 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.685 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.685 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.685 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.686 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.686 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.686 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.686 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.686 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.686 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.686 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2024-04-24T09:17:18.686 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2024-04-24T09:17:18.686 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.687 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.687 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.687 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.687 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.687 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.687 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.687 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.687 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.687 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.687 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.688 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.688 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.688 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.688 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.688 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-04-24T09:17:18.688 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.688 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.688 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.688 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2024-04-24T09:17:18.688 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2024-04-24T09:17:18.689 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.689 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.689 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.689 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.689 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.689 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.689 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.689 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T09:17:18.689 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.690 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.690 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.690 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.690 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2024-04-24T09:17:18.690 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2024-04-24T09:17:18.690 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.690 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.690 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.691 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.691 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.691 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.691 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.691 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2024-04-24T09:17:18.691 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.691 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.691 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.691 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.692 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.692 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.692 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.692 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.692 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.692 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.692 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.692 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.692 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.693 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.693 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.693 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.693 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.693 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.693 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.693 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.693 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.693 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.694 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.694 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.694 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.694 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.694 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.694 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.694 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.694 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.695 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.695 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.695 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.695 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.695 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.695 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.695 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.695 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.695 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.696 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.696 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.696 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.696 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.696 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.696 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.696 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.696 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.696 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.697 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.697 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.697 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.697 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.697 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.697 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.697 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.697 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.697 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.698 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.698 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.698 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.698 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.698 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.698 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.698 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.698 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.698 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.699 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.699 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.699 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.699 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.699 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.699 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.699 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.699 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.699 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.699 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.700 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.700 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2024-04-24T09:17:18.700 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.700 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.700 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.700 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.700 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.700 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.700 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.700 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.701 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.701 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.701 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.701 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.701 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.701 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.701 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.701 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.701 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.702 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.702 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.702 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.702 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.702 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.702 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.702 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.702 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.702 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.703 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.703 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.703 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.703 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.703 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.703 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.703 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.703 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.703 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.704 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.704 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.704 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.704 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.704 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.704 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.704 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.704 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.704 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.704 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.705 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.705 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.705 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.705 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.705 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.705 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.705 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.705 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.705 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.705 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.706 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.706 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.706 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.706 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.706 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.706 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.706 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.706 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.706 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.707 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.707 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.707 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.707 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.707 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.707 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.707 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.707 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.707 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.707 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.708 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.708 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.708 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2024-04-24T09:17:18.708 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.708 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.708 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.708 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.708 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.708 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.708 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.709 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.709 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.709 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.709 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.709 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.709 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.709 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.709 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.709 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.709 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.710 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.710 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.710 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.710 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.710 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.710 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.710 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.710 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.711 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.711 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.711 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.711 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.711 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.711 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.711 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.711 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.711 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.712 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.712 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.712 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.712 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.712 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.712 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.712 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.712 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.712 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.713 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.713 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.713 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.713 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.713 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.713 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.713 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.713 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.713 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.714 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.714 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.714 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.714 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.714 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.714 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.714 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.714 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.714 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.714 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.715 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.715 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.715 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.715 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.715 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.715 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.715 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.715 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.715 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.716 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.716 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.716 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.716 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.716 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.716 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2024-04-24T09:17:18.716 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.716 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.716 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.717 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2024-04-24T09:17:18.717 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2024-04-24T09:17:18.717 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.717 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.717 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.717 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T09:17:18.717 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.717 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.717 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.718 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.718 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.718 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.718 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.718 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.718 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-24T09:17:18.718 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-24T09:17:18.718 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.718 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.718 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.719 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2024-04-24T09:17:18.719 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.719 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.719 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.719 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.719 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.719 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.719 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.719 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.720 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.720 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.720 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.720 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.720 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.720 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.720 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.720 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.720 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.720 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.721 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.721 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.721 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.721 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.721 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.721 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.721 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.721 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.721 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.722 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.722 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.722 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.722 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.722 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.722 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.722 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.722 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.722 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.722 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.723 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.723 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.723 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.723 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.723 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.723 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.723 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.723 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.723 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.723 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.724 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.724 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.724 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.724 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.724 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.724 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.724 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.724 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.724 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.724 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.725 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.725 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.725 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.725 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.725 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.725 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.725 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.725 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.725 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.726 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.726 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.726 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.726 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.726 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.726 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.726 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.726 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.726 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.727 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.727 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.727 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.727 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.727 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.727 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.727 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2024-04-24T09:17:18.727 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.727 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.727 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.728 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2024-04-24T09:17:18.728 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2024-04-24T09:17:18.728 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.728 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.728 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.728 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.728 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.728 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.728 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.729 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T09:17:18.729 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.729 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.729 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.729 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.729 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2024-04-24T09:17:18.729 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2024-04-24T09:17:18.729 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.729 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.729 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.730 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.730 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.730 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.730 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.730 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T09:17:18.730 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.730 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.730 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.730 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.730 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2024-04-24T09:17:18.731 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2024-04-24T09:17:18.731 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.731 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.731 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.731 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.731 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.731 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.731 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.731 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T09:17:18.732 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.732 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.732 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.732 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.732 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2024-04-24T09:17:18.732 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2024-04-24T09:17:18.732 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.732 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.732 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.733 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.733 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.733 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.733 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.733 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T09:17:18.733 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.733 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.733 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.733 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.733 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2024-04-24T09:17:18.734 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2024-04-24T09:17:18.734 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.734 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.734 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.734 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.734 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.734 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.734 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.735 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T09:17:18.735 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.735 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.735 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.735 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.735 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2024-04-24T09:17:18.735 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2024-04-24T09:17:18.735 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.735 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.735 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.736 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.736 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.736 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.736 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.736 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.736 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.736 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.736 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.736 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.737 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2024-04-24T09:17:18.737 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2024-04-24T09:17:18.737 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.737 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.737 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.737 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.737 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.737 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.737 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.738 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.738 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.738 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.738 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.738 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.738 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2024-04-24T09:17:18.738 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2024-04-24T09:17:18.738 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.738 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.739 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.739 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T09:17:18.739 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.739 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.739 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.739 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.739 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.739 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.739 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.739 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.740 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-04-24T09:17:18.740 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-04-24T09:17:18.740 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.740 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.740 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.740 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.740 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.740 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.740 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.740 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.741 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.741 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.741 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.741 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.741 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-24T09:17:18.741 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-24T09:17:18.741 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.741 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.741 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.742 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2024-04-24T09:17:18.742 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2024-04-24T09:17:18.742 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.742 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.742 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.742 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.742 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.742 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.742 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.742 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2024-04-24T09:17:18.743 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2024-04-24T09:17:18.743 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.743 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.743 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.743 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2024-04-24T09:17:18.743 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.743 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.743 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.743 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.743 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.744 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.744 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.744 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.744 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2024-04-24T09:17:18.744 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2024-04-24T09:17:18.744 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.744 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.744 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.744 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.744 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.745 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.745 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.745 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.745 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.745 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.745 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.745 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.745 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2024-04-24T09:17:18.745 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2024-04-24T09:17:18.746 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.746 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.746 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.746 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.746 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.746 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.746 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.746 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.746 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.747 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.747 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.747 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.747 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.747 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.747 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.747 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.747 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.747 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.747 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.748 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.748 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.748 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.748 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.748 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.748 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.748 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.748 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.748 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.748 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.749 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.749 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.749 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.749 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.749 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.749 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.749 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.749 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.749 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.750 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.750 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.750 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.750 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.750 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.750 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.750 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.750 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.750 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.751 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.751 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.751 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.751 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.751 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.751 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.751 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.751 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.751 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.751 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.752 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.752 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.752 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.752 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.752 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.752 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.752 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.752 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.752 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.753 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.753 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.753 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.753 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.753 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.753 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.753 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.753 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.753 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.753 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.754 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.754 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.754 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.754 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.754 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.754 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.754 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2024-04-24T09:17:18.754 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2024-04-24T09:17:18.754 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.755 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.755 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.755 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.755 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.755 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.755 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.755 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.755 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.755 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.755 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.756 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.756 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2024-04-24T09:17:18.756 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2024-04-24T09:17:18.756 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.756 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.756 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.756 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.756 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.756 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.756 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.757 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.757 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.757 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.757 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.757 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.757 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2024-04-24T09:17:18.757 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2024-04-24T09:17:18.757 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.757 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.757 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.758 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T09:17:18.758 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.758 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.758 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.758 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.758 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.758 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.758 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.759 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.759 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.759 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.759 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.759 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2024-04-24T09:17:18.759 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.759 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.759 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.759 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.759 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.760 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.760 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.760 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.760 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.760 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.760 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.760 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.760 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.760 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.760 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.761 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.761 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.761 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.761 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.761 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.761 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.761 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.761 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.761 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.762 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.762 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.762 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.762 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.762 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.762 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.762 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.762 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.762 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.763 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.763 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.763 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.763 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.763 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.763 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.763 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.763 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.764 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.764 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.764 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.764 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.764 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.764 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.764 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.764 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.764 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.764 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.765 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.765 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.765 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.765 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.765 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.765 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.765 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.765 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.765 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.766 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.766 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.766 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.766 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.766 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.766 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.766 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.766 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.766 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.767 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.767 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.767 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.767 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.767 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.767 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.767 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-04-24T09:17:18.767 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.767 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.768 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.768 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.768 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.768 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.768 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.768 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.768 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.768 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.768 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.768 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.769 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.769 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.769 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.769 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.769 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.769 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.769 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.769 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.769 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.770 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.770 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.770 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.770 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.770 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.770 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.770 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.770 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.770 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.771 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.771 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.771 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.771 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.771 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.771 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.771 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.771 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.772 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.772 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.772 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.772 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.772 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.772 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.772 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.772 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.772 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.773 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.773 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.773 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.773 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.773 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.773 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.773 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.773 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.773 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.774 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.774 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.774 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.774 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.774 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.774 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.774 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.774 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.775 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.775 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.775 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.775 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.775 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.775 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.775 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.775 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.775 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.775 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2024-04-24T09:17:18.776 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2024-04-24T09:17:18.776 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.776 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.776 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.776 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2024-04-24T09:17:18.776 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.776 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.776 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.776 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T09:17:18.777 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.777 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.777 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.777 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.777 INFO:teuthology.orchestra.run.smithi069.stdout: "periodic_async_work": { 2024-04-24T09:17:18.777 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "periodic_async_work", 2024-04-24T09:17:18.777 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.777 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.777 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.777 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.778 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.778 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.778 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.778 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Periodically check for async work", 2024-04-24T09:17:18.778 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.778 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.778 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.778 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.778 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2024-04-24T09:17:18.779 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2024-04-24T09:17:18.779 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.779 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.779 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.779 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-24T09:17:18.779 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.779 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.779 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.779 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T09:17:18.780 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.780 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.780 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.780 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.780 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_no_wait": { 2024-04-24T09:17:18.780 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T09:17:18.780 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.780 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.780 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.780 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.781 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.781 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.781 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.781 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T09:17:18.781 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.781 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.781 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.781 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.781 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.782 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.782 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.782 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2024-04-24T09:17:18.782 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.782 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.782 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.782 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2024-04-24T09:17:18.782 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2024-04-24T09:17:18.782 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-24T09:17:18.783 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.783 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.783 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2024-04-24T09:17:18.783 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.783 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.783 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.783 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.783 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.783 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.784 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.784 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.784 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2024-04-24T09:17:18.784 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2024-04-24T09:17:18.784 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.784 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.784 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.784 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.784 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.785 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.785 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.785 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.785 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.785 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.785 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.785 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.785 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-24T09:17:18.785 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-24T09:17:18.785 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.786 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.786 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.786 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-24T09:17:18.786 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.786 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.786 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.786 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.786 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.787 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.787 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.787 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.787 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.787 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.787 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.787 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.787 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.787 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.787 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.788 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.788 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.788 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.788 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.788 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.788 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.788 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.788 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.788 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.788 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.789 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.789 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.789 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.789 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.789 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.789 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.789 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.789 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.789 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.790 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.790 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.790 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.790 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.790 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.790 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.790 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.790 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.790 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.791 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.791 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.791 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.791 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.791 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.791 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.791 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.791 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.791 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.792 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.792 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.792 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.792 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.792 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.792 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.792 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.792 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.792 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.792 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.793 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.793 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.793 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.793 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.793 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.793 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.793 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.793 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.793 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.793 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.794 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.794 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.794 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.794 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.794 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.794 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.794 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2024-04-24T09:17:18.795 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2024-04-24T09:17:18.795 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.795 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.795 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.795 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.795 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.795 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.795 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.795 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.795 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.796 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.796 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.796 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.796 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2024-04-24T09:17:18.796 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2024-04-24T09:17:18.796 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.796 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.796 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.796 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2024-04-24T09:17:18.796 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.797 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.797 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.797 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.797 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.797 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.797 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.797 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.797 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2024-04-24T09:17:18.797 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2024-04-24T09:17:18.798 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.798 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.798 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.798 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T09:17:18.798 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.798 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.798 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.798 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.798 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.798 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.799 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.799 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.799 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.799 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.799 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-24T09:17:18.799 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.799 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.799 INFO:teuthology.orchestra.run.smithi069.stdout: "gid": 24958, 2024-04-24T09:17:18.799 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "y", 2024-04-24T09:17:18.799 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr_features": 4540701547738038271, 2024-04-24T09:17:18.800 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2024-04-24T09:17:18.800 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.800 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2024-04-24T09:17:18.800 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.800 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.800 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.800 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-24T09:17:18.800 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-24T09:17:18.800 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.801 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.801 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.801 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-24T09:17:18.801 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.801 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.801 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.801 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T09:17:18.802 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.802 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.802 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.802 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.802 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.802 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.802 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.802 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.802 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.803 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.803 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.803 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.803 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.803 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.803 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.803 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.803 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.803 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.803 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.804 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.804 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.804 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.804 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.804 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.804 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.804 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.804 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.804 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.804 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.805 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.805 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.805 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.805 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.805 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.805 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.805 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.805 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.805 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.805 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.806 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.806 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.806 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.806 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.806 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.806 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.806 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.806 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.806 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.807 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.807 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.807 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.807 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.807 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.807 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.807 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.807 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.807 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.808 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.808 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.808 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.808 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.808 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.808 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.808 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.808 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.808 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.808 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.809 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.809 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.809 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.809 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.809 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.809 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.809 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.809 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2024-04-24T09:17:18.809 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2024-04-24T09:17:18.809 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.810 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.810 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.810 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.810 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.810 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.810 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.810 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2024-04-24T09:17:18.810 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.810 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.811 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.811 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.811 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2024-04-24T09:17:18.811 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2024-04-24T09:17:18.811 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.811 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.811 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.811 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2024-04-24T09:17:18.811 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.812 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.812 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.812 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2024-04-24T09:17:18.812 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.812 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.812 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.812 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.812 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2024-04-24T09:17:18.812 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2024-04-24T09:17:18.812 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.813 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.813 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.813 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.813 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.813 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.813 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.813 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2024-04-24T09:17:18.813 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.813 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.814 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.814 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.814 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2024-04-24T09:17:18.814 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2024-04-24T09:17:18.814 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.814 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.814 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.814 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.814 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.814 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.815 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.815 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2024-04-24T09:17:18.815 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.815 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.815 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.815 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.815 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2024-04-24T09:17:18.815 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2024-04-24T09:17:18.815 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.816 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.816 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.816 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2024-04-24T09:17:18.816 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.816 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.816 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.816 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2024-04-24T09:17:18.816 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.816 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.816 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.817 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.817 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2024-04-24T09:17:18.817 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2024-04-24T09:17:18.817 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.817 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.817 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.817 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.817 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.817 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.818 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.818 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2024-04-24T09:17:18.818 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.818 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.818 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.818 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.818 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2024-04-24T09:17:18.818 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2024-04-24T09:17:18.818 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.819 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.819 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.819 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.819 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.819 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.819 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.819 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T09:17:18.819 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.819 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.820 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.820 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.820 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2024-04-24T09:17:18.820 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2024-04-24T09:17:18.820 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.820 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.820 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.820 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.820 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.821 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.821 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.821 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2024-04-24T09:17:18.821 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.821 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.821 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.821 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.821 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.821 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.822 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.822 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2024-04-24T09:17:18.822 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.822 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.822 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.822 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2024-04-24T09:17:18.822 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2024-04-24T09:17:18.822 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.822 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.823 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.823 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.823 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.823 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.823 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.823 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T09:17:18.823 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.823 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.823 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.824 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.824 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2024-04-24T09:17:18.824 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2024-04-24T09:17:18.824 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.824 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.824 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.824 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2024-04-24T09:17:18.824 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.824 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.824 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.825 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T09:17:18.825 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:17:18.825 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.825 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.825 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.825 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2024-04-24T09:17:18.825 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2024-04-24T09:17:18.825 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-24T09:17:18.825 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.826 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.826 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-24T09:17:18.826 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-04-24T09:17:18.826 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-04-24T09:17:18.826 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.826 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T09:17:18.826 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:17:18.826 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.826 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.826 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.827 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2024-04-24T09:17:18.827 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2024-04-24T09:17:18.827 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-24T09:17:18.827 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.827 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.827 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2024-04-24T09:17:18.827 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-24T09:17:18.827 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2024-04-24T09:17:18.827 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.828 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T09:17:18.828 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.828 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.828 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.828 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.828 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2024-04-24T09:17:18.828 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2024-04-24T09:17:18.828 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.828 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.829 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.829 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T09:17:18.829 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.829 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.829 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.829 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T09:17:18.829 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T09:17:18.829 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.829 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.829 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.830 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2024-04-24T09:17:18.830 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2024-04-24T09:17:18.830 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:18.830 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.830 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.830 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2024-04-24T09:17:18.830 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2024-04-24T09:17:18.830 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2024-04-24T09:17:18.831 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.831 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2024-04-24T09:17:18.831 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T09:17:18.831 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.831 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.831 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.831 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2024-04-24T09:17:18.831 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2024-04-24T09:17:18.831 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.831 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.832 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.832 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2024-04-24T09:17:18.832 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.832 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.832 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.832 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T09:17:18.832 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:17:18.832 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.832 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.832 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.833 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2024-04-24T09:17:18.833 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2024-04-24T09:17:18.833 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-24T09:17:18.833 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.833 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.833 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-24T09:17:18.833 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-04-24T09:17:18.833 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-04-24T09:17:18.833 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.833 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T09:17:18.834 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:17:18.834 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.834 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.834 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.834 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.834 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.834 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.834 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.835 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.835 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.835 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.835 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.835 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.835 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.835 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.835 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.835 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.835 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.836 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.836 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.836 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.836 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.836 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.836 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.836 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.836 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.836 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.837 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.837 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.837 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.837 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.837 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.837 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.837 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.837 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.837 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.837 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.838 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.838 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.838 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.838 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.838 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.838 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.838 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.838 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.839 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.839 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.839 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.839 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.839 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.839 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.839 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.839 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.839 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.840 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.840 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.840 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.840 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.840 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.840 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.840 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.840 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.840 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.840 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.841 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.841 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.841 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.841 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.841 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.841 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.841 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.841 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.841 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.841 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.842 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2024-04-24T09:17:18.842 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2024-04-24T09:17:18.842 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:18.842 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.842 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.842 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-24T09:17:18.842 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.842 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.843 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.843 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T09:17:18.843 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.843 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.843 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.843 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.843 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-04-24T09:17:18.843 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-04-24T09:17:18.843 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.843 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.844 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.844 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2024-04-24T09:17:18.844 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.844 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.844 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.844 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2024-04-24T09:17:18.844 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2024-04-24T09:17:18.844 INFO:teuthology.orchestra.run.smithi069.stdout: "read", 2024-04-24T09:17:18.844 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap", 2024-04-24T09:17:18.845 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap-read" 2024-04-24T09:17:18.845 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.845 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2024-04-24T09:17:18.845 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.845 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.845 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.845 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.845 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2024-04-24T09:17:18.845 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2024-04-24T09:17:18.845 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.846 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.846 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.846 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.846 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.846 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.846 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.846 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T09:17:18.846 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.846 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.847 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.847 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.847 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-24T09:17:18.847 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-24T09:17:18.847 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.847 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.847 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.847 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-24T09:17:18.847 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.847 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.848 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.848 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T09:17:18.848 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.848 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.848 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.848 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.848 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2024-04-24T09:17:18.848 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2024-04-24T09:17:18.848 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.848 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.849 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.849 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-24T09:17:18.849 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-24T09:17:18.849 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.849 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.849 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T09:17:18.849 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T09:17:18.849 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.849 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.849 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.850 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2024-04-24T09:17:18.850 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2024-04-24T09:17:18.850 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-24T09:17:18.850 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.850 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.850 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:18.850 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.850 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.850 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.851 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T09:17:18.851 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.851 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.851 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.851 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.851 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.851 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.851 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.851 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2024-04-24T09:17:18.851 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.852 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.852 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.852 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2024-04-24T09:17:18.852 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2024-04-24T09:17:18.852 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:18.852 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.852 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.852 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-04-24T09:17:18.852 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.852 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.853 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.853 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T09:17:18.853 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.853 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.853 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.853 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.853 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2024-04-24T09:17:18.853 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2024-04-24T09:17:18.853 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.854 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.854 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.854 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2024-04-24T09:17:18.854 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.854 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.854 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.854 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T09:17:18.854 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.854 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.855 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.855 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.855 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2024-04-24T09:17:18.855 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2024-04-24T09:17:18.855 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.855 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.855 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.855 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2024-04-24T09:17:18.855 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.855 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.856 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.856 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T09:17:18.856 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.856 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.856 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.856 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.856 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2024-04-24T09:17:18.856 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2024-04-24T09:17:18.857 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.857 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.857 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.857 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.857 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.857 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.857 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.857 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T09:17:18.857 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T09:17:18.858 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.858 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.858 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.858 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2024-04-24T09:17:18.858 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2024-04-24T09:17:18.858 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.858 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.859 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.859 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-24T09:17:18.859 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.859 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.859 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.860 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T09:17:18.860 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.860 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.860 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.860 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.860 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2024-04-24T09:17:18.861 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T09:17:18.861 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:18.861 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.861 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.861 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2024-04-24T09:17:18.862 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.862 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.862 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.862 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T09:17:18.862 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.862 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.862 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.862 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.863 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm_log_destination": { 2024-04-24T09:17:18.863 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm_log_destination", 2024-04-24T09:17:18.863 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.863 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.863 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.863 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.863 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.863 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.864 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.864 INFO:teuthology.orchestra.run.smithi069.stdout: "file", 2024-04-24T09:17:18.864 INFO:teuthology.orchestra.run.smithi069.stdout: "file,syslog", 2024-04-24T09:17:18.864 INFO:teuthology.orchestra.run.smithi069.stdout: "syslog" 2024-04-24T09:17:18.864 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.864 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T09:17:18.864 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.864 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.864 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.865 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.865 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2024-04-24T09:17:18.865 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2024-04-24T09:17:18.865 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.865 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.865 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.865 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.865 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.866 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.866 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.866 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T09:17:18.866 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.866 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.866 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.866 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.866 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2024-04-24T09:17:18.867 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2024-04-24T09:17:18.867 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.867 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.867 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.867 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.867 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.867 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.867 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.868 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T09:17:18.868 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.868 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.868 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.868 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.868 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2024-04-24T09:17:18.868 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2024-04-24T09:17:18.868 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.868 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.869 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.869 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.869 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.869 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.869 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.869 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T09:17:18.869 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.869 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.869 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.870 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.870 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2024-04-24T09:17:18.870 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2024-04-24T09:17:18.870 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.870 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.870 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.870 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T09:17:18.870 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.871 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.871 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.871 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-24T09:17:18.871 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.871 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.871 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.871 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.871 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2024-04-24T09:17:18.871 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2024-04-24T09:17:18.872 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.872 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.872 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.872 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T09:17:18.872 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.872 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.872 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.872 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2024-04-24T09:17:18.873 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.873 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.873 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.873 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.873 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2024-04-24T09:17:18.873 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2024-04-24T09:17:18.873 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.873 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.873 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.874 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T09:17:18.874 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.874 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.874 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.874 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2024-04-24T09:17:18.874 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.874 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.874 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.875 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.875 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2024-04-24T09:17:18.875 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2024-04-24T09:17:18.875 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.875 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.875 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.875 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T09:17:18.875 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.876 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.876 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.876 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-24T09:17:18.876 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.876 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.876 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.876 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.876 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2024-04-24T09:17:18.876 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2024-04-24T09:17:18.877 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.877 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.877 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.877 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T09:17:18.877 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.877 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.877 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.877 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2024-04-24T09:17:18.878 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.878 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.878 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.878 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.878 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2024-04-24T09:17:18.878 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2024-04-24T09:17:18.878 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.878 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.878 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.879 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T09:17:18.879 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.879 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.879 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.879 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2024-04-24T09:17:18.879 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.879 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.879 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.879 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.880 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2024-04-24T09:17:18.880 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2024-04-24T09:17:18.880 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.880 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.880 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.880 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T09:17:18.880 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.880 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.881 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.881 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2024-04-24T09:17:18.881 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.881 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.881 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.881 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.881 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2024-04-24T09:17:18.881 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2024-04-24T09:17:18.881 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.881 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.882 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.882 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T09:17:18.882 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.882 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.882 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.882 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2024-04-24T09:17:18.882 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.882 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.882 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.883 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.883 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2024-04-24T09:17:18.883 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2024-04-24T09:17:18.883 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.883 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.883 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.883 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T09:17:18.883 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.883 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.884 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.884 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2024-04-24T09:17:18.884 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.884 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.884 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.884 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.884 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2024-04-24T09:17:18.884 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2024-04-24T09:17:18.884 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.885 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.885 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.885 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T09:17:18.885 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.885 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.885 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.885 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2024-04-24T09:17:18.885 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.886 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.886 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.886 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.886 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2024-04-24T09:17:18.886 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2024-04-24T09:17:18.886 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.886 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.887 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.887 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T09:17:18.887 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.887 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.887 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.887 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-24T09:17:18.887 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.887 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.888 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.888 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.888 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_nvmeof": { 2024-04-24T09:17:18.888 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_nvmeof", 2024-04-24T09:17:18.888 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.888 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.888 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.888 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T09:17:18.888 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.889 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.889 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.889 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Nvme-of container image", 2024-04-24T09:17:18.889 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.889 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.889 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.889 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.889 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2024-04-24T09:17:18.890 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2024-04-24T09:17:18.890 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.890 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.890 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.890 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T09:17:18.890 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.890 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.890 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.890 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-24T09:17:18.891 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.891 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.891 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.891 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.891 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2024-04-24T09:17:18.891 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2024-04-24T09:17:18.891 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.891 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.892 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.892 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T09:17:18.892 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.892 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.892 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.892 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2024-04-24T09:17:18.892 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.892 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.892 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.893 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.893 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_samba": { 2024-04-24T09:17:18.893 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_samba", 2024-04-24T09:17:18.893 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.893 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.893 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.893 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T09:17:18.893 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.893 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.894 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.894 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Samba/SMB container image", 2024-04-24T09:17:18.894 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.894 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.894 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.894 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.894 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2024-04-24T09:17:18.894 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2024-04-24T09:17:18.894 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.894 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.895 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.895 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T09:17:18.895 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.895 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.895 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.895 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2024-04-24T09:17:18.895 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.895 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.895 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.896 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.896 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2024-04-24T09:17:18.896 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2024-04-24T09:17:18.896 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.896 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.896 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.896 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.896 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.896 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.897 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.897 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T09:17:18.897 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.897 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.897 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.897 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.897 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2024-04-24T09:17:18.897 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2024-04-24T09:17:18.898 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.898 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.898 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.898 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-24T09:17:18.898 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.898 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.898 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.898 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T09:17:18.898 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.899 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.899 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.899 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.899 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2024-04-24T09:17:18.899 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T09:17:18.899 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.899 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.899 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.899 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2024-04-24T09:17:18.900 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.900 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.900 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.900 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T09:17:18.900 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.900 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.900 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.900 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.900 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2024-04-24T09:17:18.901 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2024-04-24T09:17:18.901 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.901 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.901 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.901 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2024-04-24T09:17:18.901 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.901 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.901 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.901 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T09:17:18.902 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.902 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.902 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.902 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.902 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2024-04-24T09:17:18.902 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2024-04-24T09:17:18.902 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.902 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.903 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.903 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2024-04-24T09:17:18.903 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.903 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.903 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.903 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2024-04-24T09:17:18.903 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.903 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.903 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.904 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.904 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2024-04-24T09:17:18.904 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2024-04-24T09:17:18.904 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.904 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.904 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.904 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.904 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.904 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.904 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.905 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T09:17:18.905 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.905 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.905 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.905 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.905 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2024-04-24T09:17:18.905 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2024-04-24T09:17:18.905 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.905 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.905 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.906 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-24T09:17:18.906 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.906 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.906 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.906 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2024-04-24T09:17:18.906 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.906 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.906 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.906 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.907 INFO:teuthology.orchestra.run.smithi069.stdout: "grafana_dashboards_path": { 2024-04-24T09:17:18.907 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "grafana_dashboards_path", 2024-04-24T09:17:18.907 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.907 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.907 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.907 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T09:17:18.907 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.907 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.907 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.907 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T09:17:18.908 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.908 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.908 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.908 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.908 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2024-04-24T09:17:18.908 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2024-04-24T09:17:18.908 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.908 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.908 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.908 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-24T09:17:18.909 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.909 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.909 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.909 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2024-04-24T09:17:18.909 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.909 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.909 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.909 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.909 INFO:teuthology.orchestra.run.smithi069.stdout: "hw_monitoring": { 2024-04-24T09:17:18.910 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hw_monitoring", 2024-04-24T09:17:18.910 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.910 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.910 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.910 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.910 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.910 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.910 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.910 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T09:17:18.911 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.911 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.911 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.911 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.911 INFO:teuthology.orchestra.run.smithi069.stdout: "inventory_list_all": { 2024-04-24T09:17:18.911 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "inventory_list_all", 2024-04-24T09:17:18.911 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.911 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.911 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.912 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.912 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.912 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.912 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.912 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T09:17:18.912 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.912 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.912 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.913 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.913 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.913 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.913 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.913 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.913 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.913 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.913 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.913 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.914 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.914 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.914 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.914 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.914 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.914 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.914 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.914 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.914 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.914 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.915 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.915 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.915 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.915 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2024-04-24T09:17:18.915 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2024-04-24T09:17:18.915 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.915 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.915 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.915 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.916 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.916 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.916 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.916 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T09:17:18.916 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.916 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.916 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.916 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.916 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.916 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.917 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.917 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.917 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.917 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.917 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.917 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.917 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.917 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T09:17:18.917 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.917 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.918 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.918 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.918 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.918 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.918 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.918 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.918 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.918 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.918 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.919 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.919 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.919 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.919 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.919 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.919 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.919 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.919 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.919 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.920 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.920 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.920 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.920 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.920 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.920 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.920 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.920 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.920 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.921 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.921 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.921 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.921 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.921 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.921 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.921 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.921 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.921 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.922 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.922 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T09:17:18.922 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T09:17:18.922 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.922 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.922 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.922 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.922 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.923 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.923 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.923 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T09:17:18.923 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.923 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.923 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.923 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.923 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T09:17:18.924 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T09:17:18.924 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.924 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.924 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.924 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2024-04-24T09:17:18.924 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.924 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.924 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.925 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T09:17:18.925 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.925 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.925 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.925 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.925 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2024-04-24T09:17:18.925 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2024-04-24T09:17:18.925 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.925 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.926 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.926 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:18.926 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.926 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.926 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.926 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2024-04-24T09:17:18.926 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.926 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.927 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.927 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.927 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2024-04-24T09:17:18.927 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2024-04-24T09:17:18.927 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.927 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.927 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.927 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:18.928 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.928 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.928 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.928 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T09:17:18.928 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.928 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.928 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.928 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.928 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2024-04-24T09:17:18.929 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2024-04-24T09:17:18.929 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.929 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.929 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.929 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.929 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.929 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.929 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.929 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2024-04-24T09:17:18.929 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.930 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.930 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.930 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.930 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-04-24T09:17:18.930 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-04-24T09:17:18.930 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.930 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.930 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.930 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2024-04-24T09:17:18.931 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.931 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.931 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.931 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2024-04-24T09:17:18.931 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2024-04-24T09:17:18.931 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.931 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T09:17:18.931 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.931 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.931 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.932 INFO:teuthology.orchestra.run.smithi069.stdout: "oob_default_addr": { 2024-04-24T09:17:18.932 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "oob_default_addr", 2024-04-24T09:17:18.932 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.932 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.932 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.932 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "169.254.1.1", 2024-04-24T09:17:18.932 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.932 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.933 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.933 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T09:17:18.933 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.933 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.933 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.933 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.933 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2024-04-24T09:17:18.933 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2024-04-24T09:17:18.933 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.933 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.934 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.934 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T09:17:18.934 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.934 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.934 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.934 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T09:17:18.934 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.934 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.935 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.935 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2024-04-24T09:17:18.935 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2024-04-24T09:17:18.935 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.935 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.935 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.935 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.935 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.935 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.936 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.936 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T09:17:18.936 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.936 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.936 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.936 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.936 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2024-04-24T09:17:18.936 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2024-04-24T09:17:18.936 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.937 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.937 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.937 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.937 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.937 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.937 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.937 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T09:17:18.937 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.937 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.937 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2024-04-24T09:17:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2024-04-24T09:17:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T09:17:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2024-04-24T09:17:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2024-04-24T09:17:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.940 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.940 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.940 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.940 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.940 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.940 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T09:17:18.940 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.940 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.940 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.940 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.941 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2024-04-24T09:17:18.941 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2024-04-24T09:17:18.941 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.941 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.941 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.941 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.941 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.941 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.941 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.942 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T09:17:18.942 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.942 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.942 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.942 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.942 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2024-04-24T09:17:18.942 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2024-04-24T09:17:18.942 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.942 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.942 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.943 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2024-04-24T09:17:18.943 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.943 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.943 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.943 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2024-04-24T09:17:18.943 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.943 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.943 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.943 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.944 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2024-04-24T09:17:18.944 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2024-04-24T09:17:18.944 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.944 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.944 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.944 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.944 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.944 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.944 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.944 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T09:17:18.945 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.945 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.945 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.945 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.945 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2024-04-24T09:17:18.945 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2024-04-24T09:17:18.945 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.945 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.945 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.946 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.946 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.946 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.946 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.946 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T09:17:18.946 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.946 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.946 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.946 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.946 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2024-04-24T09:17:18.947 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2024-04-24T09:17:18.947 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.947 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.947 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.947 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.947 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.947 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.947 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.947 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T09:17:18.948 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.948 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.948 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.948 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.948 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2024-04-24T09:17:18.948 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2024-04-24T09:17:18.948 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.948 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.948 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.948 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.949 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.949 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.949 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.949 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T09:17:18.949 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.949 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.949 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.949 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.949 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2024-04-24T09:17:18.949 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2024-04-24T09:17:18.950 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.950 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.950 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.950 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.950 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.950 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.950 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.950 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T09:17:18.951 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.951 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.951 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.951 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.951 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2024-04-24T09:17:18.951 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2024-04-24T09:17:18.951 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.951 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.951 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.951 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.952 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.952 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.952 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.952 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T09:17:18.952 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.952 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.952 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.952 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.952 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.953 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.953 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.953 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2024-04-24T09:17:18.953 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.953 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.953 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.953 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:18.953 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:18.953 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.953 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.954 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.954 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.954 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.954 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.954 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.954 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.954 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.954 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.954 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.955 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.955 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.955 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.955 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.955 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.955 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.955 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.955 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.955 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:18.956 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:18.956 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.956 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.956 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.956 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.956 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.956 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.956 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.956 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.956 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.957 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.957 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.957 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.957 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:18.957 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:18.957 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.957 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.957 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.957 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:18.957 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.958 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.958 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:18.958 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:18.958 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:18.958 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:18.958 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:18.958 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:18.958 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:18.958 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:18.959 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.959 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.959 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.959 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.959 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.959 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:18.959 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:18.959 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.959 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.960 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.960 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.960 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.960 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.960 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.960 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.960 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.960 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.960 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.961 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.961 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2024-04-24T09:17:18.961 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2024-04-24T09:17:18.961 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.961 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.961 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.961 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2024-04-24T09:17:18.961 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.961 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.962 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.962 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T09:17:18.962 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.962 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.962 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.962 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.962 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2024-04-24T09:17:18.962 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2024-04-24T09:17:18.963 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:18.963 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.963 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:18.963 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2024-04-24T09:17:18.963 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.963 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.963 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.963 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T09:17:18.964 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.964 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.964 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.964 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.964 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:18.964 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.964 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:18.964 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2024-04-24T09:17:18.964 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:18.965 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:18.965 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:18.965 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T09:17:18.965 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T09:17:18.965 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:18.965 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.965 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.965 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:18.965 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.965 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.966 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.966 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.966 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.966 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.966 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.966 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.966 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T09:17:18.966 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T09:17:18.966 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.967 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.967 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.967 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.967 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.967 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.967 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.967 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.967 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.968 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.968 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.968 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.968 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T09:17:18.968 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T09:17:18.968 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.968 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.968 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.968 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.969 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.969 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.969 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.969 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.969 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.969 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.969 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.969 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.969 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2024-04-24T09:17:18.970 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T09:17:18.970 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.970 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.970 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.970 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.970 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.970 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.970 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.971 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.971 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.971 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.971 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.971 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.971 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T09:17:18.971 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T09:17:18.971 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.972 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.972 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.972 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.972 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.972 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.972 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.972 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.972 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.973 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.973 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.973 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.973 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T09:17:18.973 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T09:17:18.973 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.973 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.973 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.973 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.973 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.974 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.974 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.974 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.974 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.974 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.974 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.974 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.974 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T09:17:18.974 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T09:17:18.975 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.975 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.975 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.975 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.975 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.975 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.975 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.975 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.975 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.976 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.976 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.976 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.976 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T09:17:18.976 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T09:17:18.976 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.976 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.976 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.976 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.976 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.977 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.977 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.977 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.977 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.977 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.977 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.977 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.977 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T09:17:18.977 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T09:17:18.978 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.978 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.978 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.978 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.978 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.978 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.978 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.978 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.979 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.979 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.979 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.979 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.979 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T09:17:18.979 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T09:17:18.979 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.979 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.979 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.979 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.980 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.980 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.980 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.980 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.980 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.980 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.980 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.980 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.980 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T09:17:18.981 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T09:17:18.981 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.981 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.981 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.981 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.981 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.981 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.981 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.981 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.981 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.982 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.982 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.982 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.982 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T09:17:18.982 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T09:17:18.982 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.982 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.982 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.983 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.983 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.983 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.983 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.983 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.983 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.983 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.983 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.983 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.984 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T09:17:18.984 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T09:17:18.984 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.984 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.984 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.984 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.984 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.984 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.984 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.985 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.985 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.985 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.985 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.985 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.985 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T09:17:18.985 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T09:17:18.985 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.985 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.986 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.986 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.986 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.986 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.986 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.986 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.986 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.986 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.986 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.987 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.987 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T09:17:18.987 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T09:17:18.987 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.987 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.987 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.987 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-24T09:17:18.987 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.987 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.987 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.988 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.988 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.988 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.988 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.988 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.988 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T09:17:18.988 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T09:17:18.988 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.989 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.989 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.989 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.989 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.989 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.989 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.989 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.989 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.989 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.989 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.990 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.990 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2024-04-24T09:17:18.990 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2024-04-24T09:17:18.990 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.990 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.990 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.990 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.990 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.991 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.991 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.991 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.991 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.991 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.991 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.991 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.991 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T09:17:18.991 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T09:17:18.992 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.992 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.992 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.992 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-24T09:17:18.992 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.992 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.992 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.992 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.992 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.993 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.993 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.993 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.993 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T09:17:18.993 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T09:17:18.993 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.993 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.993 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.993 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.994 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.994 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.994 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.994 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.994 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.994 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.994 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.994 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.994 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T09:17:18.995 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T09:17:18.995 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.995 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.995 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.995 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:18.995 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.995 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.995 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.995 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.995 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.996 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.996 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.996 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.996 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T09:17:18.996 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T09:17:18.996 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:18.996 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.996 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.996 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:18.997 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.997 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.997 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.997 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.997 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.997 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.997 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.997 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.997 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T09:17:18.998 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T09:17:18.998 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.998 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.998 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.998 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:18.998 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:18.998 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:18.998 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:18.998 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:18.998 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:18.999 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:18.999 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:18.999 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:18.999 INFO:teuthology.orchestra.run.smithi069.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-24T09:17:18.999 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-24T09:17:18.999 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:18.999 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:18.999 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:18.999 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "[]", 2024-04-24T09:17:19.000 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.000 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.000 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.000 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.000 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.000 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.000 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.000 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.000 INFO:teuthology.orchestra.run.smithi069.stdout: "MULTICLUSTER_CONFIG": { 2024-04-24T09:17:19.000 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-24T09:17:19.001 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.001 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.001 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.001 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "{}", 2024-04-24T09:17:19.001 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.001 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.001 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.001 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.001 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.002 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.002 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.002 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.002 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T09:17:19.002 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T09:17:19.002 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.002 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.002 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.002 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.003 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.003 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.003 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.003 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.003 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.003 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.003 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.003 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.003 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T09:17:19.004 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T09:17:19.004 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.004 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.004 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.004 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.004 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.004 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.004 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.004 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.004 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.005 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.005 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.005 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.005 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T09:17:19.005 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T09:17:19.005 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.005 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.005 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.005 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.005 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.006 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.006 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.006 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.006 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.006 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.006 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.006 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.006 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T09:17:19.006 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T09:17:19.007 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.007 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.007 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.007 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.007 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.007 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.007 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.007 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.007 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.008 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.008 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.008 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.008 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T09:17:19.008 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T09:17:19.008 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.008 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.008 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.008 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.008 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.009 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.009 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.009 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.009 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.009 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.009 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.009 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.009 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T09:17:19.009 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T09:17:19.009 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.010 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.010 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.010 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.010 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.010 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.010 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.010 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.010 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.010 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.011 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.011 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.011 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T09:17:19.011 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T09:17:19.011 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.011 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.011 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.011 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.011 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.012 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.012 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.012 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.012 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.012 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.012 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.012 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.012 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T09:17:19.012 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T09:17:19.012 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.013 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.013 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.013 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.013 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.013 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.013 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.013 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.013 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.013 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.014 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.014 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.014 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T09:17:19.014 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T09:17:19.014 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.014 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.014 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.014 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.015 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.015 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.015 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.015 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.015 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.015 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.015 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.015 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.015 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T09:17:19.015 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T09:17:19.016 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.016 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.016 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.016 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.016 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.016 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.016 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.016 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.016 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.017 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.017 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.017 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.017 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T09:17:19.017 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T09:17:19.017 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.017 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.017 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.017 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T09:17:19.018 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.018 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.018 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.018 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.018 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.018 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.018 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.018 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.018 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T09:17:19.019 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T09:17:19.019 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.019 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.019 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.019 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:19.019 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.019 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.019 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.019 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.019 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.020 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.020 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.020 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.020 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T09:17:19.020 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T09:17:19.020 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.020 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.020 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.020 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2024-04-24T09:17:19.021 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.021 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.021 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.021 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.021 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.021 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.021 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.021 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.021 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T09:17:19.022 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T09:17:19.022 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.022 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.022 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.022 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2024-04-24T09:17:19.022 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.022 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.022 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.022 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.023 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.023 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.023 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.023 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.023 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T09:17:19.023 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T09:17:19.023 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.023 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.023 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.024 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.024 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.024 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.024 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.024 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.024 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.024 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.024 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.024 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.025 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T09:17:19.025 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T09:17:19.025 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.025 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.025 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.025 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-24T09:17:19.025 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.025 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.025 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.025 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.026 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.026 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.026 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.026 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.026 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T09:17:19.026 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T09:17:19.026 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.026 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.027 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.027 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.027 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.027 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.027 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.027 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.027 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.027 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.028 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.028 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.028 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T09:17:19.028 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T09:17:19.028 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.028 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.028 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.028 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.029 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.029 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.029 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.029 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.029 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.029 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.029 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.029 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.030 INFO:teuthology.orchestra.run.smithi069.stdout: "UNSAFE_TLS_v1_2": { 2024-04-24T09:17:19.030 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-24T09:17:19.030 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.030 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.030 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.030 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.030 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.030 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.031 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.031 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.031 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.031 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.031 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.031 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.031 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T09:17:19.031 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T09:17:19.032 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.032 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.032 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.032 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-24T09:17:19.032 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.032 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.032 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.032 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.032 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.033 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.033 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.033 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.033 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T09:17:19.033 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T09:17:19.033 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.033 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.033 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.034 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:19.034 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.034 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.034 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.034 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.034 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.034 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.034 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.034 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.034 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T09:17:19.035 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T09:17:19.035 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.035 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.035 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.035 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-24T09:17:19.035 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.035 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.035 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.035 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.036 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.036 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.036 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.036 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.036 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2024-04-24T09:17:19.036 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2024-04-24T09:17:19.036 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.036 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.037 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.037 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.037 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.037 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.037 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.037 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.037 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.037 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.037 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.038 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.038 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2024-04-24T09:17:19.038 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2024-04-24T09:17:19.038 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.038 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.038 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.038 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.038 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.039 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.039 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.039 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.039 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.039 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.039 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.039 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.039 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2024-04-24T09:17:19.039 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2024-04-24T09:17:19.040 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.040 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.040 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.040 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.040 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.040 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.040 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.040 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2024-04-24T09:17:19.040 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.041 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.041 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.041 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.041 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2024-04-24T09:17:19.041 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2024-04-24T09:17:19.041 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.041 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.041 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.041 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2024-04-24T09:17:19.041 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.042 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.042 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.042 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.042 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.042 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.042 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.042 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.042 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-04-24T09:17:19.042 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-04-24T09:17:19.043 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.043 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.043 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.043 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.043 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.043 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.043 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.043 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.043 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.044 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.044 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.044 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.044 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.044 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.044 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.044 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.044 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.044 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.044 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.045 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.045 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.045 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.045 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.045 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.045 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.045 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.045 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.045 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.046 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.046 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.046 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.046 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.046 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.046 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.046 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.046 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.046 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.047 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.047 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.047 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.047 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.047 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.047 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.047 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.047 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.047 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.048 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.048 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.048 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.048 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.048 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.048 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.048 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.048 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.048 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.049 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.049 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.049 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.049 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.049 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.049 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.049 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.049 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.049 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.049 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.050 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.050 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.050 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.050 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.050 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.050 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.050 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.050 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.050 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.051 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.051 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.051 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.051 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.051 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.051 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.051 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.051 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.052 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2024-04-24T09:17:19.052 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2024-04-24T09:17:19.052 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.052 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.052 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.052 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.052 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.052 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.052 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.053 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2024-04-24T09:17:19.053 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.053 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.053 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.053 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.053 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2024-04-24T09:17:19.053 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T09:17:19.053 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.053 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.054 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.054 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.054 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.054 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.054 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.054 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.054 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.054 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.054 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.055 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.055 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-24T09:17:19.055 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-24T09:17:19.055 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.055 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.055 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.055 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-04-24T09:17:19.055 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.055 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.055 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.056 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.056 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.056 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.056 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.056 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.056 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-24T09:17:19.056 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-24T09:17:19.056 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.057 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.057 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.057 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2024-04-24T09:17:19.057 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.057 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.057 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.057 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.057 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.057 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.058 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.058 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.058 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-04-24T09:17:19.058 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-04-24T09:17:19.058 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.058 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.058 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.058 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.058 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.059 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.059 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.059 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.059 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.059 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.059 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.059 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.059 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2024-04-24T09:17:19.059 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2024-04-24T09:17:19.060 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.060 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.060 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.060 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2024-04-24T09:17:19.060 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.060 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.060 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.060 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.061 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.061 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.061 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.061 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.061 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-04-24T09:17:19.061 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-04-24T09:17:19.061 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.061 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.061 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.062 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2024-04-24T09:17:19.062 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.062 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.062 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.062 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.062 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2024-04-24T09:17:19.062 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.062 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.062 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.063 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.063 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.063 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.063 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-04-24T09:17:19.063 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-04-24T09:17:19.063 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.063 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.063 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.064 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-04-24T09:17:19.064 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-04-24T09:17:19.064 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-04-24T09:17:19.064 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.064 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.064 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.064 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.064 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.064 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.065 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2024-04-24T09:17:19.065 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2024-04-24T09:17:19.065 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.065 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.065 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.065 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.065 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.065 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.066 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.066 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.066 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.066 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.066 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.066 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.066 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.066 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.066 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.067 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-04-24T09:17:19.067 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.067 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.067 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.067 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2024-04-24T09:17:19.067 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2024-04-24T09:17:19.067 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.067 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.067 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.068 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.068 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.068 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.068 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.068 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2024-04-24T09:17:19.068 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.068 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.068 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.068 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.069 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.069 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.069 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.069 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.069 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.069 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.069 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.069 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.069 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.070 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.070 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.070 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.070 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.070 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.070 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.070 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.070 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.070 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.071 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.071 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.071 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.071 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.071 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.071 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.071 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.071 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.071 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.071 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.072 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.072 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.072 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.072 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.072 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.072 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.072 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.072 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.072 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.073 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.073 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.073 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.073 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.073 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.073 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.073 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.073 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.073 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.074 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.074 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.074 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.074 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.074 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.074 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.074 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.074 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.074 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.075 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.075 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.075 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.075 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.075 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.075 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.075 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.075 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.075 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.075 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.076 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.076 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.076 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.076 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.076 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.076 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2024-04-24T09:17:19.076 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2024-04-24T09:17:19.076 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.076 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.077 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.077 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2024-04-24T09:17:19.077 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.077 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.077 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.077 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T09:17:19.077 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.077 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.077 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.077 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.078 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2024-04-24T09:17:19.078 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2024-04-24T09:17:19.078 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.078 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.078 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.078 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2024-04-24T09:17:19.078 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.078 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.078 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.079 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T09:17:19.079 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.079 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.079 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.079 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.079 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2024-04-24T09:17:19.079 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2024-04-24T09:17:19.079 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.080 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.080 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.080 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2024-04-24T09:17:19.080 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.080 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.080 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.080 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2024-04-24T09:17:19.080 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.080 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.080 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.081 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.081 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2024-04-24T09:17:19.081 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2024-04-24T09:17:19.081 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.081 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.081 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.081 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-04-24T09:17:19.081 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.081 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.081 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.082 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T09:17:19.082 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.082 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.082 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.082 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.082 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2024-04-24T09:17:19.082 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2024-04-24T09:17:19.082 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.083 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.083 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.083 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.083 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.083 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.083 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.083 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T09:17:19.083 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.083 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.083 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.084 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.084 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-24T09:17:19.084 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-24T09:17:19.084 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.084 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.084 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.084 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-24T09:17:19.084 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.084 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.085 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.085 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T09:17:19.085 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.085 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.085 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.085 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.085 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2024-04-24T09:17:19.085 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2024-04-24T09:17:19.086 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.086 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.086 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.086 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2024-04-24T09:17:19.086 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.086 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.086 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.086 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T09:17:19.086 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.087 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.087 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.087 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.087 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.087 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.087 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.087 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2024-04-24T09:17:19.087 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.088 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.088 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.088 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.088 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.088 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.088 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.088 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.088 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.089 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.089 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.089 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.089 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.089 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.089 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.089 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.089 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.089 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.089 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.090 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.090 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.090 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.090 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.090 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.090 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.090 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.090 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.091 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.091 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.091 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.091 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.091 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.091 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.091 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.091 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.092 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.092 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.092 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.092 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.092 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.092 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.092 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.092 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.092 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.093 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.093 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.093 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.093 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.093 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.093 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.093 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.093 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.093 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.093 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.094 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.094 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.094 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.094 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.094 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.094 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.094 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.094 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.094 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.095 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.095 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.095 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.095 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.095 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.095 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.095 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.095 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.095 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.096 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.096 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2024-04-24T09:17:19.096 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2024-04-24T09:17:19.096 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.096 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.096 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.096 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-04-24T09:17:19.096 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.096 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.097 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.097 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.097 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.097 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.097 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.097 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.097 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2024-04-24T09:17:19.097 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2024-04-24T09:17:19.097 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.098 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.098 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.098 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2024-04-24T09:17:19.098 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.098 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.098 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.098 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.098 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.098 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.099 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.099 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.099 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-24T09:17:19.099 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-24T09:17:19.099 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.099 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.099 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.099 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-24T09:17:19.100 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.100 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.100 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.100 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.100 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.100 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.100 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.100 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.100 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.101 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.101 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.101 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2024-04-24T09:17:19.101 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2024-04-24T09:17:19.101 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2024-04-24T09:17:19.101 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.101 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2024-04-24T09:17:19.101 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2024-04-24T09:17:19.102 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.102 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.102 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.102 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2024-04-24T09:17:19.102 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.102 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.102 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.102 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T09:17:19.103 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.103 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.103 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.103 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.103 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2024-04-24T09:17:19.103 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2024-04-24T09:17:19.103 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.103 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.103 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.104 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2024-04-24T09:17:19.104 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.104 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.104 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.104 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T09:17:19.104 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.104 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.104 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.104 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.104 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2024-04-24T09:17:19.105 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2024-04-24T09:17:19.105 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.105 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.105 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.105 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.105 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.105 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.105 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.105 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2024-04-24T09:17:19.105 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.106 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.106 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.106 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.106 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-24T09:17:19.106 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-24T09:17:19.106 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.106 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.106 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.106 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2024-04-24T09:17:19.107 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2024-04-24T09:17:19.107 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.107 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.107 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T09:17:19.107 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.107 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.107 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.107 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.107 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.107 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.108 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.108 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.108 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.108 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.108 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.108 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.108 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.108 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.108 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.108 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.109 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.109 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.109 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.109 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.109 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.109 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.109 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.109 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.109 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.110 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.110 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.110 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.110 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.110 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.110 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.110 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.110 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.110 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.111 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.111 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.111 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.111 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.111 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.111 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.111 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.111 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.111 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.111 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.112 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.112 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.112 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.112 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.112 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.112 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.112 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.112 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.112 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.113 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.113 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.113 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.113 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.113 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.113 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.113 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.113 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.114 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.114 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.114 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.114 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.114 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.114 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.114 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.114 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.114 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.115 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.115 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.115 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.115 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.115 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2024-04-24T09:17:19.115 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2024-04-24T09:17:19.115 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.115 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.115 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.116 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.116 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.116 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.116 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.116 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2024-04-24T09:17:19.116 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.116 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.116 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.116 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.117 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2024-04-24T09:17:19.117 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2024-04-24T09:17:19.117 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.117 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.117 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.117 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2024-04-24T09:17:19.117 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.117 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.117 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.117 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2024-04-24T09:17:19.118 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.118 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.118 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.118 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.118 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-04-24T09:17:19.118 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-04-24T09:17:19.118 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.118 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.119 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.119 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2024-04-24T09:17:19.119 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.119 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.119 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.119 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:17:19.119 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.119 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.119 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.120 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.120 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2024-04-24T09:17:19.120 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2024-04-24T09:17:19.120 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.120 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.120 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.120 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-24T09:17:19.120 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-24T09:17:19.121 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2024-04-24T09:17:19.121 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.121 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T09:17:19.121 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.121 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.121 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.121 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.121 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2024-04-24T09:17:19.121 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2024-04-24T09:17:19.122 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.122 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.122 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.122 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.122 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.122 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.122 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.122 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2024-04-24T09:17:19.122 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.123 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.123 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.123 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.123 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2024-04-24T09:17:19.123 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2024-04-24T09:17:19.123 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.123 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.123 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.123 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2024-04-24T09:17:19.123 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.124 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.124 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.124 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:17:19.124 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.124 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.124 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.124 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.124 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.124 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.125 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.125 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2024-04-24T09:17:19.125 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.125 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.125 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.125 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.125 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.125 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.125 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.125 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.126 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.126 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.126 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.126 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.126 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.126 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.126 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.126 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.127 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.127 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.127 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.127 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.127 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.127 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.127 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.127 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.127 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.128 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.128 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.128 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.128 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.128 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.128 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.128 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.128 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.128 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.128 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.129 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.129 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.129 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.129 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.129 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.129 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.129 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.129 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.129 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.129 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.130 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.130 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.130 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.130 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.130 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.130 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.130 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.130 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.131 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.131 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.131 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.131 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.131 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.131 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.131 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.131 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.131 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.132 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.132 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.132 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.132 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.132 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.132 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.132 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.132 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.132 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.132 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.133 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.133 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.133 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.133 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.133 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2024-04-24T09:17:19.133 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.133 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.133 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.133 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.133 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.134 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.134 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.134 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.134 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.134 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.134 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.134 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.134 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.135 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.135 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.135 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.135 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.135 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.135 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.135 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.135 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.136 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.136 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.136 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.136 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.136 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.136 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.136 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.136 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.136 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.137 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.137 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.137 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.137 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.137 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.137 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.137 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.137 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.137 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.138 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.138 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.138 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.138 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.138 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.138 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.138 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.138 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.139 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.139 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.139 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.139 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.139 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.139 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.139 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.139 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.139 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.140 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.140 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.140 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.140 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.140 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.140 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.140 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.140 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.140 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.140 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.141 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.141 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.141 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.141 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.141 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.141 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.141 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.141 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.141 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.142 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.142 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2024-04-24T09:17:19.142 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.142 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.142 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.142 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2024-04-24T09:17:19.142 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2024-04-24T09:17:19.142 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.142 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.143 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.143 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2024-04-24T09:17:19.143 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.143 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.143 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.143 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T09:17:19.143 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.143 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.143 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.143 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.144 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2024-04-24T09:17:19.144 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2024-04-24T09:17:19.144 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.144 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.144 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.144 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:19.144 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2024-04-24T09:17:19.144 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.145 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.145 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T09:17:19.145 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.145 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.145 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.145 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.145 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.145 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.145 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.145 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.146 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.146 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.146 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.146 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.146 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.146 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.146 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.146 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.146 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.147 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.147 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.147 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.147 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.147 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.147 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.147 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.147 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.147 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.148 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.148 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.148 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.148 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.148 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.148 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.148 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.148 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.148 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.149 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.149 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.149 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.149 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.149 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.149 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.149 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.149 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.149 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.149 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.150 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.150 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.150 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.150 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.150 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.150 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.150 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.150 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.150 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.150 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.151 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.151 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.151 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.151 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.151 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.151 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.151 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.151 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.152 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.152 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.152 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.152 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.152 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.152 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.152 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.152 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.152 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.153 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.153 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.153 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.153 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.153 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.153 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2024-04-24T09:17:19.153 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.153 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.153 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.154 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2024-04-24T09:17:19.154 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2024-04-24T09:17:19.154 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.154 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.154 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.154 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2024-04-24T09:17:19.154 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.154 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.154 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.155 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2024-04-24T09:17:19.155 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T09:17:19.155 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.155 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.155 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.155 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.155 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.155 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.155 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.155 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.156 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.156 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.156 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.156 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.156 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.156 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.156 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.156 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.156 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.157 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.157 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.157 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.157 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.157 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.157 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.157 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.158 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.158 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.158 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.158 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.158 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.158 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.158 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.158 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.158 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.159 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.159 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.159 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.159 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.159 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.159 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.159 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.159 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.159 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.159 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.160 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.160 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.160 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.160 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.160 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.160 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.160 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.160 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.160 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.161 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.161 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.161 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.161 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.161 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.161 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.161 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.161 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.162 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.162 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.162 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.162 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.162 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.162 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.162 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.162 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.163 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.163 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.163 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.163 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.163 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.163 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2024-04-24T09:17:19.163 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2024-04-24T09:17:19.163 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.163 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.164 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.164 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.164 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.164 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.164 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.164 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2024-04-24T09:17:19.164 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T09:17:19.164 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.164 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.165 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.165 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2024-04-24T09:17:19.165 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2024-04-24T09:17:19.165 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.165 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.165 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.165 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2024-04-24T09:17:19.165 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.165 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.165 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.166 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2024-04-24T09:17:19.166 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.166 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.166 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.166 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.166 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2024-04-24T09:17:19.166 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2024-04-24T09:17:19.167 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.167 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.167 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.167 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2024-04-24T09:17:19.167 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.167 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.167 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.167 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T09:17:19.167 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.168 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.168 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.168 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.168 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2024-04-24T09:17:19.168 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2024-04-24T09:17:19.168 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.168 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.168 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.168 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.168 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.169 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.169 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.169 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2024-04-24T09:17:19.169 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.169 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.169 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.169 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.169 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2024-04-24T09:17:19.169 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2024-04-24T09:17:19.170 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.170 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.170 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.170 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2024-04-24T09:17:19.170 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.170 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.170 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.170 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T09:17:19.170 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T09:17:19.171 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.171 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.171 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.171 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.171 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.171 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.171 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2024-04-24T09:17:19.171 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.171 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.171 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.172 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.172 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.172 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.172 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.172 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.172 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.172 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.172 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.172 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.173 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.173 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.173 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.173 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.173 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.173 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.173 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.173 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.173 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.174 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.174 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.174 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.174 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.174 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.174 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.174 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.174 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.174 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.175 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.175 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.175 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.175 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.175 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.175 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.175 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.175 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.175 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.176 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.176 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.176 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.176 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.176 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.176 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.176 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.176 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.176 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.176 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.177 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.177 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.177 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.177 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.177 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.177 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.177 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.177 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.177 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.177 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.178 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.178 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.178 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.178 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.178 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.178 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.178 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.178 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.179 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.179 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.179 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.179 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.179 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.179 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.179 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.179 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.179 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.179 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2024-04-24T09:17:19.180 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.180 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.180 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.180 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.180 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.180 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.180 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.180 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.180 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.181 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.181 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.181 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.181 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.181 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.181 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.181 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.181 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.181 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.181 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.182 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.182 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.182 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.182 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.182 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.182 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.182 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.182 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.183 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.183 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.183 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.183 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.183 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.183 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.183 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.183 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.183 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.184 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.184 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.184 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.184 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.184 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.184 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.184 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.184 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.184 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.185 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.185 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.185 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.185 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.185 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.185 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.185 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.185 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.185 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.186 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.186 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.186 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.186 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.186 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.186 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.186 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.186 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.186 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.187 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.187 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.187 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.187 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.187 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.187 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.187 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.187 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.187 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.187 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.188 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.188 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.188 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.188 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2024-04-24T09:17:19.188 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.188 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.188 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.188 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.189 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.189 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.189 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.189 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.189 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.189 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.189 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.189 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.190 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.190 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.190 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.190 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.190 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.190 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.190 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.190 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.191 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.191 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.191 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.191 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.191 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.191 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.191 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.191 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.191 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.192 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.192 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.192 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.192 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.192 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.192 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.192 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.192 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.192 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.193 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.193 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.193 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.193 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.193 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.193 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.193 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.193 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.193 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.194 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.194 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.194 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.194 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.194 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.194 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.194 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.194 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.194 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.195 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.195 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.195 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.195 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.195 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.195 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.195 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.195 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.195 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.195 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.196 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.196 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.196 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.196 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.196 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.196 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.196 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.196 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.196 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.196 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.197 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-04-24T09:17:19.197 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.197 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.197 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.197 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2024-04-24T09:17:19.197 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2024-04-24T09:17:19.197 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.197 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.197 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.198 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.198 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.198 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.198 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.198 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T09:17:19.198 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.198 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.198 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.198 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.199 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.199 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.199 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.199 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.199 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.199 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.199 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.199 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.199 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.200 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.200 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.200 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.200 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.200 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.200 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.200 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.200 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.201 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.201 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.201 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.201 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.201 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.201 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.201 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.201 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.201 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.202 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.202 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.202 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.202 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.202 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.202 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.202 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.202 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.203 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.203 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.203 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.203 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.203 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.203 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.203 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.203 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.203 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.203 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.204 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.204 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.204 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.204 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.204 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.204 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.204 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.204 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.204 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.205 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.205 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.205 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.205 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.205 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.205 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.205 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.205 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.205 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.206 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.206 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.206 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.206 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.206 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.206 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.206 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.206 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.207 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2024-04-24T09:17:19.207 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-04-24T09:17:19.207 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.207 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.207 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.207 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.207 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.207 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.207 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.207 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-04-24T09:17:19.208 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2024-04-24T09:17:19.208 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2024-04-24T09:17:19.208 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.208 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2024-04-24T09:17:19.208 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.208 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.208 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.208 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.208 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.208 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.209 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.209 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2024-04-24T09:17:19.209 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.209 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.209 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.209 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.209 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.209 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.209 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.210 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.210 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.210 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.210 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.210 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.210 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.210 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.210 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.210 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.211 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.211 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.211 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.211 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.211 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.211 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.211 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.211 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.211 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.212 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.212 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.212 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.212 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.212 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.212 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.212 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.212 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.212 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.212 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.213 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.213 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.213 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.213 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.213 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.213 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.213 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.213 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.213 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.214 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.214 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.214 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.214 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.214 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.214 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.214 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.214 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.214 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.215 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.215 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.215 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.215 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.215 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.215 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.215 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.215 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.215 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.215 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.216 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.216 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.216 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.216 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.216 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.216 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.216 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.216 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.216 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.217 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.217 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.217 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.217 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.217 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2024-04-24T09:17:19.217 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.217 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.217 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.217 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.218 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.218 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.218 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.218 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.218 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.218 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.218 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.218 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.218 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.219 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.219 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.219 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.219 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.219 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.219 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.219 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.219 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.219 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.220 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.220 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.220 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.220 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.220 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.220 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.220 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.220 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.220 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.221 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.221 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.221 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.221 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.221 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.221 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.221 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.221 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.221 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.221 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.222 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.222 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.222 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.222 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.222 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.222 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.222 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.222 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.223 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.223 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.223 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.223 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.223 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.223 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.223 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.223 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.223 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.224 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.224 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.224 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.224 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.224 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.224 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.224 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.224 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.224 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.224 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.225 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.225 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.225 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.225 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.225 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.225 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.225 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.225 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.225 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2024-04-24T09:17:19.226 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.226 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.226 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.226 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.226 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.226 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.226 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.226 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.227 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.227 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.227 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.227 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.227 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.227 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.227 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.227 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.227 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.227 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.228 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.228 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.228 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.228 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.228 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.228 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.228 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.228 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.228 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.229 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.229 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.229 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.229 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.229 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.229 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.229 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.229 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.229 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.230 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.230 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.230 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.230 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.230 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.230 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.230 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.230 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.230 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.231 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.231 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.231 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.231 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.231 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.231 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.231 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.231 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.231 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.232 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.232 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.232 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.232 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.232 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.232 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.232 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.232 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.232 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.233 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.233 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.233 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.233 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.233 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.233 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.233 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.233 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.233 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.233 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.234 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-24T09:17:19.234 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-24T09:17:19.234 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.234 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.234 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.234 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-24T09:17:19.234 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.234 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.235 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.235 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.235 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.235 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.235 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.235 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.235 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2024-04-24T09:17:19.235 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2024-04-24T09:17:19.235 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:19.235 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.236 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.236 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-04-24T09:17:19.236 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2024-04-24T09:17:19.236 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.236 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.236 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2024-04-24T09:17:19.236 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T09:17:19.236 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.236 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.236 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.237 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.237 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.237 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.237 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2024-04-24T09:17:19.237 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.237 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.237 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.237 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2024-04-24T09:17:19.237 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2024-04-24T09:17:19.238 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.238 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.238 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.238 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.238 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.238 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.238 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.238 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T09:17:19.239 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.239 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.239 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.239 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.239 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-04-24T09:17:19.239 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-04-24T09:17:19.239 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.239 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.239 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.239 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.240 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.240 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.240 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.240 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.240 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.240 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.240 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.240 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.240 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.240 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.241 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.241 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.241 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.241 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.241 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.241 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.241 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.241 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.241 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.241 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.242 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.242 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.242 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.242 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.242 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.242 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.242 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.242 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.243 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.243 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.243 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.243 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.243 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.243 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.243 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.243 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.243 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.243 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.244 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.244 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.244 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.244 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.244 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.244 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.244 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.244 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.244 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.245 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.245 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.245 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.245 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.245 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.245 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.245 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.245 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.245 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.246 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.246 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.246 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.246 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.246 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.246 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.246 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.246 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.246 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.246 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.247 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.247 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.247 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.247 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.247 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.247 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.247 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.247 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.248 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.248 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.248 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.248 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.248 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2024-04-24T09:17:19.248 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2024-04-24T09:17:19.248 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.248 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.248 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.249 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2024-04-24T09:17:19.249 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.249 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.249 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.249 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2024-04-24T09:17:19.249 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.249 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.249 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.249 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.249 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-24T09:17:19.250 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-24T09:17:19.250 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.250 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.250 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.250 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-24T09:17:19.250 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.250 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.250 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.251 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2024-04-24T09:17:19.251 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.251 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.251 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.251 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.251 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.251 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.251 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.251 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2024-04-24T09:17:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2024-04-24T09:17:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2024-04-24T09:17:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.252 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.253 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.253 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.253 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.253 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.253 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.253 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.253 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.253 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.253 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2024-04-24T09:17:19.254 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2024-04-24T09:17:19.254 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.254 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.254 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.254 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.254 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.254 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.254 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.254 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T09:17:19.255 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T09:17:19.255 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.255 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.255 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.255 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.255 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.255 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.255 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.255 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.256 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.256 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.256 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.256 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.256 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.256 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.256 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.256 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.256 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.257 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.257 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.257 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.257 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.257 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.257 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.257 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.257 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.257 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.258 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.258 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.258 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.258 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.258 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.258 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.258 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.258 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.258 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.258 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.259 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.259 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.259 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.259 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.259 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.259 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.259 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.259 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.259 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.259 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.260 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.260 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.260 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.260 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.260 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.260 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.260 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.260 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.260 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.260 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.261 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.261 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.261 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.261 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.261 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.261 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.261 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.261 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.261 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.262 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.262 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.262 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.262 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.262 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.262 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.262 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.262 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.262 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2024-04-24T09:17:19.263 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2024-04-24T09:17:19.263 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.263 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.263 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.263 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.263 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.263 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.263 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.263 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.264 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.264 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.264 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.264 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.264 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T09:17:19.264 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T09:17:19.264 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.264 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.264 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.265 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2024-04-24T09:17:19.265 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.265 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.265 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.265 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.265 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.265 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.265 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.265 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.266 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2024-04-24T09:17:19.266 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2024-04-24T09:17:19.266 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:19.266 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.266 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.266 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2024-04-24T09:17:19.266 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.266 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.266 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.267 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.267 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.267 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.267 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.267 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.267 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-24T09:17:19.267 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-24T09:17:19.267 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.267 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.267 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.268 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-04-24T09:17:19.268 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.268 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.268 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.268 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T09:17:19.268 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.268 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.268 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.268 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.269 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-24T09:17:19.269 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-24T09:17:19.269 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.269 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.269 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.269 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2024-04-24T09:17:19.269 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.269 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.269 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.269 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T09:17:19.270 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.270 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.270 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.270 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.270 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2024-04-24T09:17:19.270 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2024-04-24T09:17:19.270 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.270 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.271 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.271 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2024-04-24T09:17:19.271 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.271 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.271 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.271 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.271 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.271 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.271 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.271 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.272 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-04-24T09:17:19.272 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-04-24T09:17:19.272 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.272 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.272 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.272 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2024-04-24T09:17:19.272 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.272 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.272 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.273 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2024-04-24T09:17:19.273 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2024-04-24T09:17:19.273 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.273 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.273 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.273 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.273 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.273 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.273 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-04-24T09:17:19.274 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-04-24T09:17:19.274 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.274 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.274 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.274 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-04-24T09:17:19.274 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-04-24T09:17:19.274 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-04-24T09:17:19.274 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.274 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.275 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.275 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.275 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.275 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.275 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.275 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.275 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.275 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-04-24T09:17:19.275 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.276 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.276 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.276 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.276 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.276 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.276 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.276 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.276 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.277 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.277 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.277 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.277 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.277 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.277 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.277 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.277 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.277 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.277 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.278 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.278 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.278 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.278 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.278 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.278 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.278 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.278 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.278 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.278 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.279 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.279 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.279 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.279 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.279 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.279 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.279 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.279 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.280 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.280 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.280 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.280 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.280 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.280 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.280 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.280 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.280 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.281 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.281 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.281 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.281 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.281 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.281 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.281 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.281 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.281 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.281 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.282 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.282 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.282 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.282 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.282 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.282 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.282 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.282 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.282 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.282 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.283 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.283 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.283 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.283 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.283 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.283 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.283 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2024-04-24T09:17:19.283 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2024-04-24T09:17:19.283 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.284 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.284 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.284 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:19.284 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.284 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.284 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.284 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.284 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.284 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.285 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.285 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.285 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2024-04-24T09:17:19.285 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T09:17:19.285 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.285 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.285 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.285 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.286 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.286 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.286 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.286 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.286 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.286 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.286 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.286 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.287 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2024-04-24T09:17:19.287 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2024-04-24T09:17:19.287 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.287 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.287 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.287 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.287 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.287 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.287 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.288 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.288 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.288 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.288 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.288 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.288 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.288 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.288 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.288 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2024-04-24T09:17:19.289 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.289 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.289 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.289 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2024-04-24T09:17:19.289 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2024-04-24T09:17:19.289 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.289 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.289 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.289 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.289 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.290 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.290 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.290 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.290 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.290 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.290 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.290 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.290 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-04-24T09:17:19.291 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-04-24T09:17:19.291 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.291 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.291 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.291 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.291 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.291 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.291 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.291 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.292 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.292 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.292 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.292 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.292 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.292 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.292 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.292 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.292 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.293 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.293 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.293 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.293 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.293 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.293 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.293 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.293 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.294 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.294 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.294 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.294 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.294 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.294 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.294 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.294 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.295 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.295 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.295 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.295 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.295 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.295 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.295 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.295 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.295 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.296 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.296 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.296 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.296 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.296 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.296 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.296 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.296 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.296 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.296 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.297 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.297 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.297 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.297 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.297 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.297 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.297 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.297 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.297 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.297 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.298 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.298 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.298 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.298 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.298 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.298 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.298 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.298 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.299 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.299 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.299 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.299 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.299 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.299 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.299 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.299 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.299 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.300 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.300 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.300 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.300 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-24T09:17:19.300 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-24T09:17:19.300 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.300 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.300 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.300 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.300 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.301 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.301 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.301 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.301 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.301 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.301 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.301 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.301 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-24T09:17:19.301 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-24T09:17:19.302 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.302 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.302 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.302 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.302 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.302 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.302 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.302 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.303 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.303 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.303 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.303 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.303 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.303 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.303 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.303 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2024-04-24T09:17:19.303 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.303 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.304 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.304 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.304 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.304 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.304 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.304 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.304 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.304 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.304 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.305 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.305 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.305 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.305 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.305 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.305 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.305 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.305 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.305 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.305 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.306 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.306 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.306 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.306 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.306 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.306 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.306 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.306 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.306 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.307 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.307 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.307 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.307 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.307 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.307 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.307 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.307 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.307 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.307 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.308 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.308 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.308 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.308 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.308 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.308 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.308 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.308 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.308 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.309 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.309 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.309 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.309 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.309 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.309 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.309 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.309 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.309 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.310 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.310 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.310 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.310 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.310 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.310 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.310 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.310 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.310 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.311 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.311 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.311 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.311 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.311 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.311 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.311 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.311 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.311 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.312 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2024-04-24T09:17:19.312 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.312 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.312 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.312 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.312 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.312 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.312 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.312 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.313 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.313 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.313 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.313 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.313 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.313 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.313 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.313 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.313 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.313 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.314 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.314 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.314 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.314 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.314 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.314 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.314 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.314 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.314 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.315 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.315 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.315 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.315 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.315 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.315 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.315 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.315 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.315 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.315 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.316 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.316 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.316 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.316 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.316 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.316 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.316 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.316 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.316 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.317 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.317 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.317 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.317 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.317 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.317 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.317 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.317 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.317 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.317 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.318 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.318 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.318 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.318 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.318 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.318 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.318 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.318 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.318 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.319 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.319 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.319 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.319 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.319 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.319 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.319 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.319 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.319 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2024-04-24T09:17:19.319 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2024-04-24T09:17:19.320 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.320 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.320 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.320 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2024-04-24T09:17:19.320 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.320 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.320 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.320 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T09:17:19.320 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.321 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.321 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.321 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.321 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.321 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.321 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.321 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2024-04-24T09:17:19.321 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.321 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.322 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.322 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.322 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.322 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.322 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.322 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.322 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.322 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.322 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.322 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.323 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.323 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.323 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.323 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.323 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.323 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.323 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.323 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.323 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.324 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.324 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.324 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.324 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.324 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.324 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.324 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.324 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.324 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.324 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.325 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.325 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.325 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.325 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.325 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.325 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.325 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.325 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.325 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.325 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.326 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.326 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.326 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.326 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.326 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.326 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.326 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.326 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.326 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.327 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.327 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.327 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.327 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.327 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.327 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.327 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.327 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.327 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.328 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.328 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.328 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.328 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.328 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.328 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.328 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.328 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.328 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.329 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.329 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.329 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.329 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.329 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.329 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2024-04-24T09:17:19.329 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2024-04-24T09:17:19.329 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.329 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.330 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.330 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.330 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.330 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.330 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.330 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.330 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.330 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.330 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.331 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.331 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2024-04-24T09:17:19.331 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2024-04-24T09:17:19.331 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.331 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.331 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.331 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2024-04-24T09:17:19.331 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.332 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.332 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.332 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.332 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.332 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.332 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.332 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.332 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2024-04-24T09:17:19.332 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2024-04-24T09:17:19.333 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.333 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.333 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.333 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.333 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.333 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.333 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.333 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.333 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.334 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.334 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.334 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.334 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2024-04-24T09:17:19.334 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2024-04-24T09:17:19.334 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.334 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.334 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.334 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.335 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.335 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.335 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.335 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.335 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.335 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.335 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.335 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.335 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2024-04-24T09:17:19.336 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2024-04-24T09:17:19.336 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:19.336 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.336 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.336 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.336 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.336 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.336 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.336 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.337 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.337 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.337 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.337 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.337 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2024-04-24T09:17:19.337 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2024-04-24T09:17:19.337 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.337 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.337 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.338 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.338 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.338 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.338 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.338 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.338 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.338 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.338 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.338 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.338 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2024-04-24T09:17:19.339 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2024-04-24T09:17:19.339 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.339 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.339 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.339 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.339 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.339 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.339 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.340 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.340 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.340 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.340 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.340 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.340 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2024-04-24T09:17:19.340 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2024-04-24T09:17:19.340 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.340 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.340 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.341 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.341 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.341 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.341 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.341 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.341 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.341 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.341 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.341 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.342 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2024-04-24T09:17:19.342 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2024-04-24T09:17:19.342 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.342 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.342 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.342 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.342 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-24T09:17:19.342 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2024-04-24T09:17:19.342 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.343 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.343 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.343 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.343 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.343 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.343 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2024-04-24T09:17:19.343 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2024-04-24T09:17:19.343 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.344 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.344 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.344 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.344 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.344 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.344 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.344 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.344 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.344 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.344 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.345 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.345 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2024-04-24T09:17:19.345 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2024-04-24T09:17:19.345 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.345 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.345 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.345 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.345 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.345 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.346 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.346 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.346 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.346 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.346 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.346 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.346 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2024-04-24T09:17:19.346 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2024-04-24T09:17:19.346 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.347 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.347 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.347 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.347 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.347 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.347 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.347 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.347 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.347 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.348 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.348 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.348 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.348 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.348 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.348 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-04-24T09:17:19.348 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.348 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.349 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.349 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2024-04-24T09:17:19.349 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2024-04-24T09:17:19.349 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.349 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.349 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.349 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.349 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.349 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.349 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.350 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T09:17:19.350 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.350 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.350 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.350 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.350 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2024-04-24T09:17:19.350 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2024-04-24T09:17:19.350 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.351 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.351 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.351 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.351 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.351 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.351 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.351 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2024-04-24T09:17:19.351 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.351 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.352 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.352 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.352 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.352 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.352 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.352 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.352 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.352 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.353 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.353 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.353 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.353 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.353 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.353 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.353 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.353 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.354 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.354 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.354 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.354 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.354 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.354 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.354 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.354 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.354 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.355 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.355 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.355 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.355 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.355 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.355 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.355 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.355 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.355 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.356 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.356 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.356 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.356 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.356 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.356 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.356 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.356 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.357 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.357 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.357 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.357 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.357 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.357 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.357 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.357 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.357 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.358 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.358 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.358 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.358 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.358 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.358 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.358 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.358 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.358 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.358 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.359 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.359 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.359 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.359 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.359 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.359 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.359 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.359 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.359 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.360 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.360 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.360 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.360 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.360 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.360 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2024-04-24T09:17:19.360 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.360 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.360 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.361 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.361 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.361 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.361 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.361 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.361 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.361 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.361 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.362 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.362 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.362 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.362 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.362 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.362 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.362 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.362 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.362 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.362 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.363 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.363 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.363 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.363 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.363 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.363 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.363 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.363 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.364 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.364 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.364 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.364 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.364 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.364 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.364 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.364 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.364 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.365 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.365 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.365 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.365 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.365 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.365 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.365 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.365 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.365 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.366 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.366 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.366 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.366 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.366 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.366 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.366 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.366 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.366 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.367 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.367 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.367 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.367 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.367 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.367 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.367 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.367 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.367 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.368 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.368 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.368 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.368 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.368 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.368 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.368 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.368 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.368 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.369 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.369 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.369 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2024-04-24T09:17:19.369 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.369 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.369 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.369 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.369 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.369 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.370 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.370 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.370 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.370 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.370 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.370 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.370 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.371 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.371 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.371 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.371 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.371 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.371 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.371 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.371 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.371 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.372 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.372 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.372 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.372 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.372 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.372 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.372 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.372 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.373 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.373 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.373 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.373 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.373 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.373 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.373 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.373 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.374 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.374 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.374 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.374 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.374 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.374 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.374 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.374 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.374 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.375 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.375 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.375 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.375 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.375 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.375 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.375 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.375 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.375 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.376 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.376 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.376 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.376 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.376 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.376 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.376 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.376 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.376 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.377 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.377 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.377 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.377 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.377 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.377 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.377 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.377 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.378 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.378 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.378 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.378 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2024-04-24T09:17:19.378 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.378 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.378 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.378 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2024-04-24T09:17:19.378 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2024-04-24T09:17:19.379 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.379 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.379 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.379 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T09:17:19.379 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.379 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.379 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.379 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.380 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.380 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.380 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.380 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.380 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-24T09:17:19.380 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-24T09:17:19.380 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.380 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.380 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.381 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2024-04-24T09:17:19.381 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.381 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.381 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.381 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.381 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.381 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.381 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.381 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.382 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.383 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.383 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.383 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.383 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.383 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.383 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.383 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.383 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.383 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.384 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.384 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.384 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.384 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.384 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.384 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.384 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.384 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.385 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.385 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.385 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.385 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.385 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.385 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.385 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.385 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.385 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.386 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.386 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.386 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.386 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.386 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.386 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.386 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.386 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.387 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.387 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.387 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.387 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.387 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.387 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.387 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.387 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.387 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.388 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.388 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.388 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.388 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.388 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.388 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.388 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.388 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.388 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.389 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.389 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.389 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.389 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.389 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.389 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.389 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.389 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.389 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.390 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.390 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.390 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.390 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.390 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2024-04-24T09:17:19.390 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.390 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.390 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.390 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2024-04-24T09:17:19.391 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2024-04-24T09:17:19.391 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.391 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.391 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.391 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.391 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.391 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.391 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.392 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T09:17:19.392 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.392 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.392 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.392 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.392 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2024-04-24T09:17:19.392 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2024-04-24T09:17:19.392 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.392 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.393 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.393 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.393 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.393 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.393 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.393 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T09:17:19.393 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.393 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.394 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.394 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.394 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2024-04-24T09:17:19.394 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2024-04-24T09:17:19.394 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.394 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.394 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.394 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.395 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.395 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.395 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.395 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T09:17:19.395 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.395 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.395 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.395 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.395 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2024-04-24T09:17:19.396 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2024-04-24T09:17:19.396 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.396 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.396 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.396 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.396 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.396 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.396 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.396 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T09:17:19.397 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.397 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.397 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.397 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.397 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2024-04-24T09:17:19.397 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2024-04-24T09:17:19.397 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.397 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.397 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.398 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.398 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.398 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.398 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.398 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T09:17:19.398 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.398 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.398 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.399 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.399 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2024-04-24T09:17:19.399 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2024-04-24T09:17:19.399 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.399 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.399 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.399 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.399 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.399 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.399 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.400 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.400 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.400 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.400 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.400 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.400 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2024-04-24T09:17:19.400 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2024-04-24T09:17:19.400 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.400 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.401 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.401 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.401 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.401 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.401 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.401 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.401 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.401 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.402 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.402 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.402 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2024-04-24T09:17:19.402 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2024-04-24T09:17:19.402 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.402 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.402 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.402 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T09:17:19.403 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.403 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.403 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.403 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.403 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.403 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.403 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.403 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.403 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-04-24T09:17:19.404 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-04-24T09:17:19.404 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.404 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.404 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.404 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.404 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.404 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.404 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.405 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.405 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.405 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.405 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.405 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.405 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-24T09:17:19.405 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-24T09:17:19.405 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.405 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.406 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.406 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2024-04-24T09:17:19.406 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2024-04-24T09:17:19.406 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.406 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.406 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.406 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.406 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.406 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.407 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.407 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2024-04-24T09:17:19.407 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2024-04-24T09:17:19.407 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.407 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.407 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.407 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2024-04-24T09:17:19.407 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.407 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.408 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.408 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.408 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.408 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.408 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.408 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.408 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2024-04-24T09:17:19.408 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2024-04-24T09:17:19.408 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.409 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.409 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.409 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.409 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.409 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.409 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.409 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.409 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.409 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.409 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.410 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.410 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2024-04-24T09:17:19.410 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2024-04-24T09:17:19.410 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.410 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.410 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.410 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.410 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.411 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.411 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.411 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.411 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.411 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.411 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.411 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.411 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.411 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.412 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.412 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.412 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.412 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.412 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.412 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.412 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.412 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.412 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.413 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.413 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.413 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.413 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.413 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.413 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.413 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.413 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.413 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.414 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.414 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.414 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.414 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.414 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.414 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.414 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.414 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.415 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.415 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.415 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.415 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.415 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.415 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.415 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.415 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.416 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.416 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.416 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.416 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.416 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.416 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.416 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.416 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.416 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.416 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.417 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.417 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.417 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.417 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.417 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.417 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.417 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.417 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.417 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.417 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.418 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.418 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.418 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.418 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.418 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.418 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.418 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.418 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.419 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.419 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.419 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.419 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.419 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.419 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.419 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2024-04-24T09:17:19.419 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2024-04-24T09:17:19.419 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.419 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.420 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.420 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.420 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.420 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.420 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.420 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.420 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.420 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.420 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.420 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.421 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2024-04-24T09:17:19.421 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2024-04-24T09:17:19.421 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.421 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.421 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.421 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.421 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.421 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.421 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.422 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.422 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.422 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.422 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.422 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.422 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2024-04-24T09:17:19.422 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2024-04-24T09:17:19.422 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.423 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.423 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.423 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T09:17:19.423 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.423 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.423 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.423 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.423 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.424 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.424 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.424 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.424 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.424 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.424 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.424 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2024-04-24T09:17:19.424 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.424 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.425 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.425 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.425 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.425 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.425 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.425 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.425 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.425 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.425 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.426 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.426 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.426 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.426 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.426 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.426 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.426 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.426 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.427 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.427 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.427 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.427 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.427 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.427 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.427 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.427 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.427 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.428 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.428 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.428 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.428 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.428 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.428 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.428 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.428 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.429 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.429 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.429 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.429 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.429 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.429 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.429 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.429 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.429 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.430 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.430 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.430 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.430 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.430 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.430 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.430 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.430 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.431 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.431 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.431 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.431 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.431 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.431 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.431 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.431 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.431 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.432 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.432 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.432 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.432 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.432 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.432 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.432 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.432 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.433 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.433 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.433 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.433 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.433 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.433 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.433 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-04-24T09:17:19.433 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.433 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.434 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.434 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.434 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.434 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.434 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.434 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.434 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.434 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.435 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.435 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.435 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.435 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.435 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.435 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.435 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.435 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.435 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.436 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.436 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.436 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.436 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.436 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.436 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.436 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.436 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.437 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.437 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.437 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.437 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.437 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.437 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.437 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.437 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.438 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.438 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.438 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.438 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.438 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.438 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.438 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.438 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.439 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.439 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.439 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.439 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.439 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.439 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.439 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.439 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.440 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.440 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.440 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.440 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.440 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.440 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.440 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.440 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.440 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.440 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.441 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.441 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.441 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.441 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.441 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.441 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.441 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.441 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.441 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.442 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.442 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.442 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.442 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2024-04-24T09:17:19.442 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2024-04-24T09:17:19.442 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.442 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.442 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.443 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2024-04-24T09:17:19.443 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.443 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.443 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.443 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T09:17:19.443 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.443 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.443 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.444 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.444 INFO:teuthology.orchestra.run.smithi069.stdout: "periodic_async_work": { 2024-04-24T09:17:19.444 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "periodic_async_work", 2024-04-24T09:17:19.444 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.444 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.444 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.444 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.444 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.445 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.445 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.445 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Periodically check for async work", 2024-04-24T09:17:19.445 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.445 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.445 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.445 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.445 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2024-04-24T09:17:19.445 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2024-04-24T09:17:19.446 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.446 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.446 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.446 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-24T09:17:19.446 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.446 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.446 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.446 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T09:17:19.447 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.447 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.447 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.447 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.447 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_no_wait": { 2024-04-24T09:17:19.447 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T09:17:19.447 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.447 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.447 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.447 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.448 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.448 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.448 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.448 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T09:17:19.448 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.448 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.448 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.448 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.448 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.448 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.449 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.449 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2024-04-24T09:17:19.449 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.449 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.449 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.449 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2024-04-24T09:17:19.449 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2024-04-24T09:17:19.449 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-24T09:17:19.449 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.450 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.450 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2024-04-24T09:17:19.450 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.450 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.450 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.450 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.450 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.450 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.451 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.451 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.451 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2024-04-24T09:17:19.451 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2024-04-24T09:17:19.451 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.451 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.451 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.451 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.451 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.451 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.452 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.452 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.452 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.452 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.452 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.452 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.452 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-24T09:17:19.452 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-24T09:17:19.452 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.452 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.453 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.453 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-24T09:17:19.453 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.453 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.453 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.453 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.453 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.453 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.453 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.454 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.454 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.454 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.454 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.454 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.454 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.454 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.454 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.454 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.455 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.455 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.455 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.455 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.455 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.455 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.455 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.455 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.455 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.456 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.456 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.456 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.456 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.456 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.456 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.456 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.456 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.456 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.456 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.457 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.457 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.457 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.457 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.457 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.457 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.457 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.457 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.457 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.457 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.458 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.458 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.458 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.458 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.458 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.458 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.458 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.458 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.458 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.459 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.459 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.459 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.459 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.459 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.459 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.459 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.459 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.459 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.460 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.460 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.460 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.460 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.460 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.460 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.460 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.460 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.460 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.460 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.461 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.461 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.461 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.461 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.461 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.461 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2024-04-24T09:17:19.461 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2024-04-24T09:17:19.461 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.461 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.461 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.462 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.462 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.462 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.462 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.462 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.462 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.462 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.462 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.463 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.463 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2024-04-24T09:17:19.463 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2024-04-24T09:17:19.463 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.463 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.463 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.463 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2024-04-24T09:17:19.463 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.463 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.463 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.464 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.464 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.464 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.464 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.464 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.464 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2024-04-24T09:17:19.464 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2024-04-24T09:17:19.464 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.464 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.465 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.465 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T09:17:19.465 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.465 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.465 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.465 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.465 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.465 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.465 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.466 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.466 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.466 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.466 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-24T09:17:19.466 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.466 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.466 INFO:teuthology.orchestra.run.smithi069.stdout: "modules": [ 2024-04-24T09:17:19.466 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-04-24T09:17:19.467 INFO:teuthology.orchestra.run.smithi069.stdout: "dashboard", 2024-04-24T09:17:19.467 INFO:teuthology.orchestra.run.smithi069.stdout: "iostat", 2024-04-24T09:17:19.467 INFO:teuthology.orchestra.run.smithi069.stdout: "nfs", 2024-04-24T09:17:19.467 INFO:teuthology.orchestra.run.smithi069.stdout: "restful", 2024-04-24T09:17:19.467 INFO:teuthology.orchestra.run.smithi069.stdout: "snap_schedule" 2024-04-24T09:17:19.467 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.467 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2024-04-24T09:17:19.467 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.467 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2024-04-24T09:17:19.468 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.468 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.468 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.468 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-24T09:17:19.468 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-24T09:17:19.468 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.468 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.468 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.468 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-24T09:17:19.468 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.469 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.469 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.469 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T09:17:19.469 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.469 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.469 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.469 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.469 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.469 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.470 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.470 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.470 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.470 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.470 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.470 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.470 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.470 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.471 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.471 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.471 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.471 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.471 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.471 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.471 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.471 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.471 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.472 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.472 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.472 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.472 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.472 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.472 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.472 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.472 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.472 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.472 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.473 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.473 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.473 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.473 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.473 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.473 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.473 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.473 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.473 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.474 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.474 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.474 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.474 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.474 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.474 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.474 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.474 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.474 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.475 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.475 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.475 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.475 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.475 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.475 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.475 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.475 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.475 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.475 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.476 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.476 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.476 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.476 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.476 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.476 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.476 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.476 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.476 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.476 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.477 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.477 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.477 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.477 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2024-04-24T09:17:19.477 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2024-04-24T09:17:19.477 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.477 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.477 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.477 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.478 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.478 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.478 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.478 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2024-04-24T09:17:19.478 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.478 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.478 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.478 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.478 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2024-04-24T09:17:19.479 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2024-04-24T09:17:19.479 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.479 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.479 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.479 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2024-04-24T09:17:19.479 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.479 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.479 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.479 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2024-04-24T09:17:19.479 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.480 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.480 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.480 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.480 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2024-04-24T09:17:19.480 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2024-04-24T09:17:19.480 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.480 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.480 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.480 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.480 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.481 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.481 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.481 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2024-04-24T09:17:19.481 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.481 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.481 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.481 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.481 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2024-04-24T09:17:19.481 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2024-04-24T09:17:19.482 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.482 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.482 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.482 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.482 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.482 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.482 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.482 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2024-04-24T09:17:19.482 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.483 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.483 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.483 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.483 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2024-04-24T09:17:19.483 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2024-04-24T09:17:19.483 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.483 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.483 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.483 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2024-04-24T09:17:19.484 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.484 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.484 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.484 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2024-04-24T09:17:19.484 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.484 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.484 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.484 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.484 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2024-04-24T09:17:19.484 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2024-04-24T09:17:19.485 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.485 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.485 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.485 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.485 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.485 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.485 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.485 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2024-04-24T09:17:19.485 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.486 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.486 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.486 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.486 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2024-04-24T09:17:19.486 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2024-04-24T09:17:19.486 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.486 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.486 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.487 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.487 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.487 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.487 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.487 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T09:17:19.487 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.487 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.487 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.487 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.487 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2024-04-24T09:17:19.488 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2024-04-24T09:17:19.488 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.488 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.488 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.488 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.488 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.488 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.488 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.488 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2024-04-24T09:17:19.489 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.489 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.489 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.489 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.489 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.489 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.489 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.489 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2024-04-24T09:17:19.489 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.490 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.490 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.490 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2024-04-24T09:17:19.490 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2024-04-24T09:17:19.490 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.490 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.490 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.490 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.490 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.491 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.491 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.491 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T09:17:19.491 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.491 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.491 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.491 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.491 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2024-04-24T09:17:19.491 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2024-04-24T09:17:19.491 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.492 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.492 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.492 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2024-04-24T09:17:19.492 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.492 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.492 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.492 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T09:17:19.492 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:17:19.492 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.493 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.493 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.493 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2024-04-24T09:17:19.493 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2024-04-24T09:17:19.493 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-24T09:17:19.493 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.493 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.493 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-24T09:17:19.493 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-04-24T09:17:19.493 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-04-24T09:17:19.494 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.494 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T09:17:19.494 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:17:19.494 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.494 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.494 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.494 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2024-04-24T09:17:19.494 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2024-04-24T09:17:19.495 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-24T09:17:19.495 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.495 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.495 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2024-04-24T09:17:19.495 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-24T09:17:19.495 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2024-04-24T09:17:19.495 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.495 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T09:17:19.495 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.495 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.496 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.496 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.496 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2024-04-24T09:17:19.496 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2024-04-24T09:17:19.496 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.496 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.496 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.496 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T09:17:19.496 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.496 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.497 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.497 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T09:17:19.497 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T09:17:19.497 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.497 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.497 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.497 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2024-04-24T09:17:19.497 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2024-04-24T09:17:19.497 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:19.497 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.498 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.498 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2024-04-24T09:17:19.498 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2024-04-24T09:17:19.498 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2024-04-24T09:17:19.498 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.498 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2024-04-24T09:17:19.498 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T09:17:19.498 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.499 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.499 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.499 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2024-04-24T09:17:19.499 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2024-04-24T09:17:19.499 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.499 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.499 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.499 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2024-04-24T09:17:19.499 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.499 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.500 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.500 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T09:17:19.500 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:17:19.500 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.500 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.500 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.500 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2024-04-24T09:17:19.500 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2024-04-24T09:17:19.500 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-24T09:17:19.501 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.501 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.501 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-24T09:17:19.501 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-04-24T09:17:19.501 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-04-24T09:17:19.501 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.501 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T09:17:19.501 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:17:19.501 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.501 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.502 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.502 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.502 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.502 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.502 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.502 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.502 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.502 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.502 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.503 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.503 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.503 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.503 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.503 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.503 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.503 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.503 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.503 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.504 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.504 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.504 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.504 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.504 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.504 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.504 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.504 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.504 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.505 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.505 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.505 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.505 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.505 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.505 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.505 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.505 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.505 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.506 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.506 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.506 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.506 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.506 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.506 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.506 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.506 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.506 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.507 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.507 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.507 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.507 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.507 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.507 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.507 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.507 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.507 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.508 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.508 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.508 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.508 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.508 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.508 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.508 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.508 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.508 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.509 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.509 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.509 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.509 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.509 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.509 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.509 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.509 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.509 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2024-04-24T09:17:19.509 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2024-04-24T09:17:19.510 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:19.510 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.510 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.510 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-24T09:17:19.510 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.510 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.510 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.510 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T09:17:19.511 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.511 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.511 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.511 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.511 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-04-24T09:17:19.511 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-04-24T09:17:19.511 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.511 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.511 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.512 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2024-04-24T09:17:19.512 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.512 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.512 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.512 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2024-04-24T09:17:19.512 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2024-04-24T09:17:19.512 INFO:teuthology.orchestra.run.smithi069.stdout: "read", 2024-04-24T09:17:19.512 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap", 2024-04-24T09:17:19.512 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap-read" 2024-04-24T09:17:19.513 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.513 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2024-04-24T09:17:19.513 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.513 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.513 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.513 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.513 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2024-04-24T09:17:19.513 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2024-04-24T09:17:19.513 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.513 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.514 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.514 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.514 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.514 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.514 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.514 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T09:17:19.514 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.514 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.515 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.515 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.515 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-24T09:17:19.515 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-24T09:17:19.515 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.515 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.515 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.515 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-24T09:17:19.515 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.516 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.516 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.516 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T09:17:19.516 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.516 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.516 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.516 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.516 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2024-04-24T09:17:19.516 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2024-04-24T09:17:19.517 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.517 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.517 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.517 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-24T09:17:19.517 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-24T09:17:19.517 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.517 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.517 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T09:17:19.517 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T09:17:19.518 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.518 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.518 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.518 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2024-04-24T09:17:19.518 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2024-04-24T09:17:19.518 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-24T09:17:19.518 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.518 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.519 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:19.519 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.519 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.519 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.519 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T09:17:19.519 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.519 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.519 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.519 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.519 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.520 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.520 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.520 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2024-04-24T09:17:19.520 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.520 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.520 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.520 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2024-04-24T09:17:19.520 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2024-04-24T09:17:19.520 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:19.520 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.521 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.521 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-04-24T09:17:19.521 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.521 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.521 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.521 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T09:17:19.521 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.521 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.521 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.522 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.522 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2024-04-24T09:17:19.522 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2024-04-24T09:17:19.522 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.522 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.522 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.522 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2024-04-24T09:17:19.522 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.522 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.523 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.523 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T09:17:19.523 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.523 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.523 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.523 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.523 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2024-04-24T09:17:19.523 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2024-04-24T09:17:19.523 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.524 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.524 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.524 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2024-04-24T09:17:19.524 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.524 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.524 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.524 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T09:17:19.524 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.524 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.525 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.525 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.525 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2024-04-24T09:17:19.525 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2024-04-24T09:17:19.525 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.525 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.525 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.525 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.525 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.525 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.526 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.526 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T09:17:19.526 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T09:17:19.526 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.526 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.526 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.526 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2024-04-24T09:17:19.526 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2024-04-24T09:17:19.526 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.527 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.527 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.527 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-24T09:17:19.527 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.527 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.527 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.527 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T09:17:19.527 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.527 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.528 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.528 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.528 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2024-04-24T09:17:19.528 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T09:17:19.528 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:19.528 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.528 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.528 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2024-04-24T09:17:19.528 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.528 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.529 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.529 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T09:17:19.529 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.529 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.529 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.529 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.529 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm_log_destination": { 2024-04-24T09:17:19.529 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm_log_destination", 2024-04-24T09:17:19.529 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.530 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.530 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.530 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.530 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.530 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.530 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.530 INFO:teuthology.orchestra.run.smithi069.stdout: "file", 2024-04-24T09:17:19.530 INFO:teuthology.orchestra.run.smithi069.stdout: "file,syslog", 2024-04-24T09:17:19.530 INFO:teuthology.orchestra.run.smithi069.stdout: "syslog" 2024-04-24T09:17:19.531 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.531 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T09:17:19.531 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.531 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.531 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.531 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.531 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2024-04-24T09:17:19.531 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2024-04-24T09:17:19.531 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.532 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.532 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.532 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.532 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.532 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.532 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.532 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T09:17:19.532 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.532 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.532 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.533 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.533 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2024-04-24T09:17:19.533 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2024-04-24T09:17:19.533 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.533 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.533 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.533 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.533 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.533 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.533 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.534 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T09:17:19.534 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.534 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.534 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.534 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.534 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2024-04-24T09:17:19.534 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2024-04-24T09:17:19.534 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.534 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.535 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.535 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.535 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.535 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.535 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.535 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T09:17:19.535 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.535 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.535 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.536 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.536 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2024-04-24T09:17:19.536 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2024-04-24T09:17:19.536 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.536 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.536 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.536 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T09:17:19.536 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.536 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.537 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.537 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-24T09:17:19.537 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.537 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.537 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.537 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.537 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2024-04-24T09:17:19.537 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2024-04-24T09:17:19.537 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.537 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.538 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.538 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T09:17:19.538 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.538 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.538 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.538 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2024-04-24T09:17:19.538 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.538 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.539 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.539 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.539 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2024-04-24T09:17:19.539 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2024-04-24T09:17:19.539 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.539 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.539 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.539 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T09:17:19.539 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.539 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.540 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.540 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2024-04-24T09:17:19.540 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.540 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.540 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.540 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.540 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2024-04-24T09:17:19.540 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2024-04-24T09:17:19.540 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.541 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.541 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.541 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T09:17:19.541 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.541 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.541 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.541 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-24T09:17:19.541 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.541 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.541 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.542 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.542 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2024-04-24T09:17:19.542 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2024-04-24T09:17:19.542 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.542 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.542 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.542 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T09:17:19.542 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.543 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.543 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.543 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2024-04-24T09:17:19.543 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.543 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.543 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.543 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.543 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2024-04-24T09:17:19.543 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2024-04-24T09:17:19.543 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.544 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.544 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.544 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T09:17:19.544 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.544 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.544 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.544 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2024-04-24T09:17:19.544 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.544 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.545 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.545 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.545 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2024-04-24T09:17:19.545 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2024-04-24T09:17:19.545 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.545 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.545 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.545 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T09:17:19.545 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.545 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.546 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.546 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2024-04-24T09:17:19.546 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.546 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.546 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.546 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.546 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2024-04-24T09:17:19.546 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2024-04-24T09:17:19.546 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.547 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.547 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.547 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T09:17:19.547 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.547 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.547 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.547 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2024-04-24T09:17:19.547 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.547 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.547 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.548 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.548 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2024-04-24T09:17:19.548 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2024-04-24T09:17:19.548 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.548 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.548 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.548 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T09:17:19.548 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.548 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.549 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.549 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2024-04-24T09:17:19.549 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.549 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.549 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.549 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.549 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2024-04-24T09:17:19.549 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2024-04-24T09:17:19.549 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.550 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.550 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.550 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T09:17:19.550 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.550 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.550 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.550 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2024-04-24T09:17:19.550 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.551 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.551 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.551 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.551 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2024-04-24T09:17:19.551 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2024-04-24T09:17:19.551 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.551 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.551 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.552 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T09:17:19.552 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.552 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.552 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.552 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-24T09:17:19.552 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.552 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.552 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.552 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.553 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_nvmeof": { 2024-04-24T09:17:19.553 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_nvmeof", 2024-04-24T09:17:19.553 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.553 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.553 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.553 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T09:17:19.553 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.553 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.553 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.554 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Nvme-of container image", 2024-04-24T09:17:19.554 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.554 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.554 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.554 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.554 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2024-04-24T09:17:19.554 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2024-04-24T09:17:19.554 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.555 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.555 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.555 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T09:17:19.555 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.555 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.555 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.555 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-24T09:17:19.555 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.555 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.556 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.556 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.556 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2024-04-24T09:17:19.556 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2024-04-24T09:17:19.556 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.556 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.556 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.556 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T09:17:19.556 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.557 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.557 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.557 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2024-04-24T09:17:19.557 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.557 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.557 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.557 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.557 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_samba": { 2024-04-24T09:17:19.557 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_samba", 2024-04-24T09:17:19.558 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.558 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.558 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.558 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T09:17:19.558 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.558 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.558 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.558 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Samba/SMB container image", 2024-04-24T09:17:19.558 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.559 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.559 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.559 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.559 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2024-04-24T09:17:19.559 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2024-04-24T09:17:19.559 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.559 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.559 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.559 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T09:17:19.559 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.560 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.560 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.560 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2024-04-24T09:17:19.560 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.560 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.560 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.560 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.560 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2024-04-24T09:17:19.560 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2024-04-24T09:17:19.561 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.561 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.561 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.561 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.561 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.561 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.561 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.561 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T09:17:19.561 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.562 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.562 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.562 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.562 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2024-04-24T09:17:19.562 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2024-04-24T09:17:19.562 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.562 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.562 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.562 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-24T09:17:19.563 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.563 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.563 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.563 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T09:17:19.563 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.563 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.563 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.563 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.563 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2024-04-24T09:17:19.563 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T09:17:19.564 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.564 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.564 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.564 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2024-04-24T09:17:19.564 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.564 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.564 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.564 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T09:17:19.565 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.565 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.565 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.565 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.565 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2024-04-24T09:17:19.565 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2024-04-24T09:17:19.565 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.565 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.565 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.566 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2024-04-24T09:17:19.566 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.566 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.566 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.566 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T09:17:19.566 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.566 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.566 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.567 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.567 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2024-04-24T09:17:19.567 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2024-04-24T09:17:19.567 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.567 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.567 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.567 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2024-04-24T09:17:19.567 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.567 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.568 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.568 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2024-04-24T09:17:19.568 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.568 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.568 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.568 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.568 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2024-04-24T09:17:19.568 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2024-04-24T09:17:19.568 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.569 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.569 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.569 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.569 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.569 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.569 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.569 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T09:17:19.569 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.570 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.570 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.570 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.570 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2024-04-24T09:17:19.570 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2024-04-24T09:17:19.570 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.570 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.570 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.571 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-24T09:17:19.571 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.571 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.571 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.571 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2024-04-24T09:17:19.571 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.571 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.571 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.571 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.571 INFO:teuthology.orchestra.run.smithi069.stdout: "grafana_dashboards_path": { 2024-04-24T09:17:19.572 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "grafana_dashboards_path", 2024-04-24T09:17:19.572 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.572 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.572 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.572 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T09:17:19.572 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.572 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.572 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.573 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T09:17:19.573 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.573 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.573 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.573 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.573 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2024-04-24T09:17:19.573 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2024-04-24T09:17:19.574 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.574 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.574 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.574 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-24T09:17:19.574 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.574 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.574 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.574 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2024-04-24T09:17:19.575 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.575 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.575 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.575 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.575 INFO:teuthology.orchestra.run.smithi069.stdout: "hw_monitoring": { 2024-04-24T09:17:19.575 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hw_monitoring", 2024-04-24T09:17:19.575 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.575 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.575 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.576 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.576 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.576 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.576 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.576 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T09:17:19.576 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.576 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.576 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.577 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.577 INFO:teuthology.orchestra.run.smithi069.stdout: "inventory_list_all": { 2024-04-24T09:17:19.577 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "inventory_list_all", 2024-04-24T09:17:19.577 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.577 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.577 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.577 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.577 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.577 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.578 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.578 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T09:17:19.578 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.578 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.578 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.578 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.578 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.578 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.578 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.579 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.579 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.579 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.579 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.579 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.579 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.579 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.579 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.579 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.579 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.580 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.580 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.580 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.580 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.580 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.580 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.580 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.580 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.580 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2024-04-24T09:17:19.581 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2024-04-24T09:17:19.581 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.581 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.581 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.581 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.581 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.581 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.581 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.582 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T09:17:19.582 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.582 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.582 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.582 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.582 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.582 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.582 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.583 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.583 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.583 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.583 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.583 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.583 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.583 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T09:17:19.583 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.583 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.583 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.584 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.584 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.584 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.584 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.584 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.584 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.584 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.584 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.584 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.585 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.585 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.585 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.585 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.585 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.585 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.585 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.585 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.586 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.586 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.586 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.586 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.586 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.586 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.586 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.586 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.587 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.587 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.587 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.587 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.587 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.587 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.587 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.587 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.587 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.588 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.588 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.588 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T09:17:19.588 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T09:17:19.588 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.588 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.588 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.588 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.588 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.589 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.589 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.589 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T09:17:19.589 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.589 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.589 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.589 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.589 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T09:17:19.590 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T09:17:19.590 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.590 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.590 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.590 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2024-04-24T09:17:19.590 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.590 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.590 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.591 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T09:17:19.591 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.591 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.591 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.591 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.591 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2024-04-24T09:17:19.591 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2024-04-24T09:17:19.591 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.591 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.591 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.592 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:19.592 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.592 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.592 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.592 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2024-04-24T09:17:19.592 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.592 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.592 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.592 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.593 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2024-04-24T09:17:19.593 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2024-04-24T09:17:19.593 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.593 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.593 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.593 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:19.593 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.593 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.593 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.594 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T09:17:19.594 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.594 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.594 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.594 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.594 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2024-04-24T09:17:19.594 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2024-04-24T09:17:19.594 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.594 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.595 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.595 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.595 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.595 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.595 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.595 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2024-04-24T09:17:19.595 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.595 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.595 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.596 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.596 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-04-24T09:17:19.596 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-04-24T09:17:19.596 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.596 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.596 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.596 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2024-04-24T09:17:19.596 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.596 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.596 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.597 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2024-04-24T09:17:19.597 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2024-04-24T09:17:19.597 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.597 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T09:17:19.597 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.597 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.597 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.597 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.598 INFO:teuthology.orchestra.run.smithi069.stdout: "oob_default_addr": { 2024-04-24T09:17:19.598 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "oob_default_addr", 2024-04-24T09:17:19.598 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.598 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.598 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.598 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "169.254.1.1", 2024-04-24T09:17:19.598 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.598 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.599 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.599 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T09:17:19.599 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.599 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.599 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.599 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.599 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2024-04-24T09:17:19.599 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2024-04-24T09:17:19.599 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.600 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.600 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.600 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T09:17:19.600 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.600 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.600 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.600 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T09:17:19.600 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.600 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.601 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.601 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.601 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2024-04-24T09:17:19.601 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2024-04-24T09:17:19.601 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.601 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.601 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.601 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.602 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.602 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.602 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.602 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T09:17:19.602 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.602 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.602 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.602 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.602 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2024-04-24T09:17:19.603 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2024-04-24T09:17:19.603 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.603 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.603 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.603 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.603 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.603 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.603 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.603 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T09:17:19.603 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.604 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.604 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.604 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.604 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2024-04-24T09:17:19.604 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2024-04-24T09:17:19.604 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.604 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.604 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.604 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.605 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.605 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.605 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.605 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T09:17:19.605 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.605 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.605 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.605 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.605 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2024-04-24T09:17:19.605 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2024-04-24T09:17:19.606 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.606 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.606 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.606 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.606 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.606 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.606 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.606 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T09:17:19.606 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.607 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.607 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.607 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.607 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2024-04-24T09:17:19.607 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2024-04-24T09:17:19.607 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.607 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.607 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.607 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.607 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.608 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.608 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.608 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T09:17:19.608 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.608 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.608 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.608 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.608 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2024-04-24T09:17:19.608 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2024-04-24T09:17:19.609 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.609 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.609 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.609 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2024-04-24T09:17:19.609 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.609 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.609 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.609 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2024-04-24T09:17:19.609 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.609 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.610 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.610 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.610 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2024-04-24T09:17:19.610 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2024-04-24T09:17:19.610 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.610 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.610 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.610 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.611 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.611 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.611 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.611 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T09:17:19.611 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.611 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.611 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.611 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.611 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2024-04-24T09:17:19.611 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2024-04-24T09:17:19.612 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.612 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.612 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.612 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.612 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.612 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.612 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.612 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T09:17:19.612 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.612 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.613 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.613 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.613 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2024-04-24T09:17:19.613 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2024-04-24T09:17:19.613 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.613 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.613 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.613 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.613 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.613 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.614 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.614 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T09:17:19.614 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.614 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.614 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.614 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.614 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2024-04-24T09:17:19.614 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2024-04-24T09:17:19.615 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.615 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.615 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.615 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.615 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.615 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.615 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.615 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T09:17:19.615 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.616 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.616 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.616 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.616 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2024-04-24T09:17:19.616 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2024-04-24T09:17:19.616 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.616 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.616 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.616 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.617 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.617 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.617 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.617 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T09:17:19.617 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.617 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.617 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.617 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.617 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2024-04-24T09:17:19.618 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2024-04-24T09:17:19.618 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.618 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.618 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.618 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.618 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.618 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.618 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.619 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T09:17:19.619 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.619 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.619 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.619 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.619 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.619 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.619 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.619 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2024-04-24T09:17:19.619 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.620 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.620 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.620 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.620 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.620 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.620 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.620 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.620 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.621 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.621 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.621 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.621 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.621 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.621 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.621 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.621 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.621 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.621 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.622 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.622 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.622 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.622 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.622 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.622 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.622 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.622 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.622 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.623 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.623 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.623 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.623 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.623 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.623 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.623 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.623 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.623 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.624 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.624 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.624 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.624 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.624 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.624 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.624 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.624 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.625 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.625 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.625 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.625 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.625 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.625 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.625 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.625 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.625 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.625 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.626 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.626 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.626 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.626 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.626 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.626 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.626 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.626 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.627 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.627 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.627 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.627 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.627 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.627 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.627 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.627 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.627 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.627 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.628 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2024-04-24T09:17:19.628 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2024-04-24T09:17:19.628 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.628 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.628 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.628 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2024-04-24T09:17:19.628 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.628 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.628 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.629 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T09:17:19.629 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.629 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.629 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.629 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.629 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2024-04-24T09:17:19.629 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2024-04-24T09:17:19.629 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.629 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.629 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.630 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2024-04-24T09:17:19.630 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.630 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.630 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.630 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T09:17:19.630 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.630 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.630 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.631 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.631 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.631 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.631 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.631 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2024-04-24T09:17:19.631 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.631 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.631 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.631 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T09:17:19.632 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T09:17:19.632 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.632 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.632 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.632 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:19.632 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.632 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.632 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.632 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.633 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.633 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.633 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.633 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.633 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T09:17:19.633 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T09:17:19.633 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.633 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.633 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.633 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.634 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.634 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.634 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.634 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.634 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.634 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.634 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.634 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.635 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T09:17:19.635 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T09:17:19.635 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.635 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.635 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.635 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.635 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.635 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.635 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.635 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.636 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.636 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.636 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.636 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.636 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2024-04-24T09:17:19.636 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T09:17:19.636 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.636 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.636 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.637 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.637 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.637 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.637 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.637 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.637 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.637 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.637 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.637 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.638 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T09:17:19.638 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T09:17:19.638 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.638 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.638 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.638 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.638 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.638 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.638 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.639 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.639 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.639 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.639 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.639 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.639 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T09:17:19.639 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T09:17:19.639 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.639 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.640 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.640 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.640 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.640 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.640 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.640 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.640 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.640 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.640 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.641 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.641 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T09:17:19.641 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T09:17:19.641 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.641 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.641 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.641 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.641 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.641 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.641 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.642 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.642 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.642 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.642 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.642 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.642 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T09:17:19.642 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T09:17:19.642 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.643 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.643 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.643 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.643 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.643 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.643 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.643 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.643 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.643 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.643 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.644 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.644 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T09:17:19.644 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T09:17:19.644 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.644 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.644 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.644 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.644 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.644 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.644 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.645 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.645 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.645 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.645 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.645 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.645 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T09:17:19.645 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T09:17:19.645 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.645 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.646 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.646 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.646 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.646 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.646 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.646 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.646 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.646 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.647 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.647 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.647 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T09:17:19.647 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T09:17:19.647 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.647 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.647 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.647 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.647 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.648 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.648 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.648 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.648 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.648 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.648 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.648 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.648 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T09:17:19.648 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T09:17:19.649 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.649 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.649 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.649 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.649 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.649 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.649 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.649 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.649 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.650 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.650 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.650 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.650 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T09:17:19.650 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T09:17:19.650 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.650 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.650 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.651 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.651 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.651 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.651 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.651 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.651 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.651 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.651 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.651 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.651 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T09:17:19.652 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T09:17:19.652 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.652 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.652 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.652 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.652 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.652 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.652 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.653 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.653 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.653 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.653 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.653 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.653 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T09:17:19.653 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T09:17:19.653 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.654 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.654 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.654 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-24T09:17:19.654 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.654 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.654 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.654 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.654 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.654 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.655 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.655 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.655 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T09:17:19.655 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T09:17:19.655 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.655 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.655 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.655 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.655 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.656 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.656 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.656 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.656 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.656 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.656 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.656 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.656 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2024-04-24T09:17:19.656 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2024-04-24T09:17:19.657 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.657 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.657 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.657 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.657 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.657 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.657 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.657 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.657 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.658 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.658 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.658 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.658 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T09:17:19.658 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T09:17:19.658 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.658 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.658 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.659 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-24T09:17:19.659 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.659 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.659 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.659 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.659 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.659 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.659 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.659 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.659 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T09:17:19.660 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T09:17:19.660 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.660 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.660 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.660 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.660 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.660 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.660 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.660 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.661 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.661 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.661 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.661 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.661 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T09:17:19.661 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T09:17:19.661 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.661 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.661 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.662 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.662 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.662 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.662 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.662 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.662 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.662 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.662 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.663 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.663 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T09:17:19.663 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T09:17:19.663 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.663 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.663 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.663 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.663 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.664 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.664 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.664 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.664 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.664 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.664 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.664 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.664 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T09:17:19.665 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T09:17:19.665 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.665 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.665 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.665 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.665 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.665 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.665 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.665 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.666 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.666 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.666 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.666 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.666 INFO:teuthology.orchestra.run.smithi069.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-24T09:17:19.666 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-24T09:17:19.666 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.666 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.667 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.667 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "[]", 2024-04-24T09:17:19.667 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.667 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.667 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.667 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.667 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.667 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.668 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.668 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.668 INFO:teuthology.orchestra.run.smithi069.stdout: "MULTICLUSTER_CONFIG": { 2024-04-24T09:17:19.668 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-24T09:17:19.668 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.668 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.668 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.668 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "{}", 2024-04-24T09:17:19.668 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.668 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.669 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.669 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.669 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.669 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.669 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.669 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.669 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T09:17:19.669 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T09:17:19.669 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.670 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.670 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.670 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.670 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.670 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.670 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.670 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.670 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.670 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.671 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.671 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.671 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T09:17:19.671 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T09:17:19.671 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.671 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.671 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.671 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.671 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.671 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.672 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.672 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.672 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.672 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.672 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.672 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.672 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T09:17:19.672 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T09:17:19.672 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.673 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.673 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.673 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.673 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.673 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.673 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.673 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.673 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.673 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.673 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.674 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.674 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T09:17:19.674 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T09:17:19.674 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.674 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.674 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.674 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.674 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.675 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.675 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.675 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.675 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.675 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.675 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.675 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.675 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T09:17:19.675 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T09:17:19.676 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.676 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.676 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.676 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.676 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.676 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.676 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.676 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.677 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.677 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.677 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.677 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.677 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T09:17:19.677 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T09:17:19.677 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.677 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.677 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.677 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.678 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.678 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.678 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.678 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.678 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.678 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.678 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.678 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.679 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T09:17:19.679 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T09:17:19.679 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.679 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.679 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.679 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.679 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.679 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.679 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.680 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.680 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.680 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.680 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.680 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.680 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T09:17:19.680 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T09:17:19.680 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.680 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.680 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.681 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.681 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.681 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.681 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.681 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.681 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.681 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.681 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.682 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.682 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T09:17:19.682 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T09:17:19.682 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.682 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.682 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.682 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.682 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.682 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.683 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.683 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.683 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.683 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.683 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.683 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.683 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T09:17:19.683 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T09:17:19.683 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.684 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.684 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.684 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.684 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.684 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.684 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.684 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.684 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.684 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.684 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.685 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.685 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T09:17:19.685 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T09:17:19.685 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.685 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.685 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.685 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T09:17:19.685 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.685 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.686 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.686 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.686 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.686 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.686 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.686 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.686 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T09:17:19.686 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T09:17:19.687 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.687 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.687 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.687 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:19.687 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.687 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.687 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.687 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.687 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.687 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.688 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.688 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.688 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T09:17:19.688 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T09:17:19.688 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.688 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.688 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.688 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2024-04-24T09:17:19.688 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.689 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.689 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.689 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.689 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.689 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.689 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.689 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.689 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T09:17:19.689 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T09:17:19.690 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.690 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.690 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.690 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2024-04-24T09:17:19.690 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.690 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.690 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.690 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.691 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.691 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.691 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.691 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.691 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T09:17:19.691 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T09:17:19.691 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.691 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.692 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.692 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.692 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.692 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.692 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.692 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.692 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.692 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.692 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.693 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.693 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T09:17:19.693 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T09:17:19.693 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.693 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.693 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.693 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-24T09:17:19.693 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.694 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.694 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.694 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.694 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.694 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.694 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.694 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.694 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T09:17:19.694 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T09:17:19.695 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.695 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.695 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.695 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.695 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.695 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.695 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.695 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.695 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.695 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.696 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.696 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.696 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T09:17:19.696 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T09:17:19.696 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.696 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.696 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.696 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.697 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.697 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.697 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.697 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.697 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.697 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.697 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.697 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.697 INFO:teuthology.orchestra.run.smithi069.stdout: "UNSAFE_TLS_v1_2": { 2024-04-24T09:17:19.698 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-24T09:17:19.698 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.698 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.698 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.698 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.698 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.698 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.698 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.698 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.699 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.699 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.699 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.699 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.699 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T09:17:19.699 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T09:17:19.699 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.699 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.699 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.700 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-24T09:17:19.700 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.700 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.700 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.700 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.700 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.700 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.700 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.700 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.701 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T09:17:19.701 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T09:17:19.701 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.701 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.701 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.701 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:19.701 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.701 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.701 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.702 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.702 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.702 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.702 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.702 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.702 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T09:17:19.702 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T09:17:19.702 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.703 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.703 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.703 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-24T09:17:19.703 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.703 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.703 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.703 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.703 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.703 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.704 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.704 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.704 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2024-04-24T09:17:19.704 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2024-04-24T09:17:19.704 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.704 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.704 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.704 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.705 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.705 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.705 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.705 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.705 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.705 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.705 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.705 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.705 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2024-04-24T09:17:19.705 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2024-04-24T09:17:19.706 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.706 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.706 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.706 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.706 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.706 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.706 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.706 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.706 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.707 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.707 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.707 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.707 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2024-04-24T09:17:19.707 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2024-04-24T09:17:19.707 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.707 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.707 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.707 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.707 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.708 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.708 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.708 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2024-04-24T09:17:19.708 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.708 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.708 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.708 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.708 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2024-04-24T09:17:19.708 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2024-04-24T09:17:19.708 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.709 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.709 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.709 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2024-04-24T09:17:19.709 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.709 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.709 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.709 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.709 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.710 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.710 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.710 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.710 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-04-24T09:17:19.710 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-04-24T09:17:19.710 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.710 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.710 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.711 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.711 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.711 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.711 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.711 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.711 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.711 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.711 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.711 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.711 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.712 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.712 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.712 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.712 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.712 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.712 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.712 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.712 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.712 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.713 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.713 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.713 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.713 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.713 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.713 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.713 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.713 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.714 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.714 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.714 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.714 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.714 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.714 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.714 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.714 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.714 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.715 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.715 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.715 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.715 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.715 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.715 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.715 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.715 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.716 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.716 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.716 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.716 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.716 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.716 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.716 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.716 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.716 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.717 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.717 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.717 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.717 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.717 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.717 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.717 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.717 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.717 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.717 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.718 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.718 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.718 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.718 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.718 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.718 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.718 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.718 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.718 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.719 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.719 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.719 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.719 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.719 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.719 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.719 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.719 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2024-04-24T09:17:19.719 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2024-04-24T09:17:19.719 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.720 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.720 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.720 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.720 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.720 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.720 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.720 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2024-04-24T09:17:19.720 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.720 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.721 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.721 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.721 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2024-04-24T09:17:19.721 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T09:17:19.721 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.721 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.721 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.721 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.721 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.722 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.722 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.722 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.722 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.722 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.722 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.722 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.722 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-24T09:17:19.722 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-24T09:17:19.723 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.723 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.723 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.723 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-04-24T09:17:19.723 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.723 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.723 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.723 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.723 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.724 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.724 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.724 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.724 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-24T09:17:19.724 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-24T09:17:19.724 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.724 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.724 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.724 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2024-04-24T09:17:19.725 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.725 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.725 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.725 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.725 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.725 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.725 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.725 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.725 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-04-24T09:17:19.726 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-04-24T09:17:19.726 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.726 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.726 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.726 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.726 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.726 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.726 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.726 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.727 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.727 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.727 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.727 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.727 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2024-04-24T09:17:19.727 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2024-04-24T09:17:19.727 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.727 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.727 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.727 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2024-04-24T09:17:19.728 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.728 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.728 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.728 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.728 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.728 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.728 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.728 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.728 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-04-24T09:17:19.728 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-04-24T09:17:19.729 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.729 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.729 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.729 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2024-04-24T09:17:19.729 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.729 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.729 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.729 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.729 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2024-04-24T09:17:19.730 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.730 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.730 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.730 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.730 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.730 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.730 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-04-24T09:17:19.730 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-04-24T09:17:19.731 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.731 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.731 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.731 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-04-24T09:17:19.731 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-04-24T09:17:19.731 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-04-24T09:17:19.731 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.731 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.731 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.732 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.732 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.732 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.732 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2024-04-24T09:17:19.732 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2024-04-24T09:17:19.732 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.732 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.732 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.732 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.732 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.733 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.733 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.733 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.733 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.733 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.733 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.733 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.733 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.733 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.734 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.734 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-04-24T09:17:19.734 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.734 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.734 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.734 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2024-04-24T09:17:19.734 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2024-04-24T09:17:19.734 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.734 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.734 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.735 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.735 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.735 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.735 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.735 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2024-04-24T09:17:19.735 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.735 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.735 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.735 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.736 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.736 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.736 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.736 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.736 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.736 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.736 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.736 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.736 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.737 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.737 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.737 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.737 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.737 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.737 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.737 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.737 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.737 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.738 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.738 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.738 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.738 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.738 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.738 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.738 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.738 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.739 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.739 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.739 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.739 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.739 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.739 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.739 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.739 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.739 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.740 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.740 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.740 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.740 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.740 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.740 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.740 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.740 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.740 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.741 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.741 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.741 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.741 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.741 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.741 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.741 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.741 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.741 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.741 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.742 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.742 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.742 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.742 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.742 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.742 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.742 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.742 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.742 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.743 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.743 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.743 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.743 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.743 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.743 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.743 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.743 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2024-04-24T09:17:19.743 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2024-04-24T09:17:19.744 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.744 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.744 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.744 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2024-04-24T09:17:19.744 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.744 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.744 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.744 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T09:17:19.744 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.744 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.745 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.745 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.745 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2024-04-24T09:17:19.745 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2024-04-24T09:17:19.745 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.745 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.745 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.745 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2024-04-24T09:17:19.745 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.746 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.746 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.746 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T09:17:19.746 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.746 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.746 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.746 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.746 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2024-04-24T09:17:19.747 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2024-04-24T09:17:19.747 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.747 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.747 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.747 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2024-04-24T09:17:19.747 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.747 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.748 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.748 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2024-04-24T09:17:19.748 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.748 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.748 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.748 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.748 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2024-04-24T09:17:19.748 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2024-04-24T09:17:19.748 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.749 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.749 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.749 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-04-24T09:17:19.749 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.749 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.749 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.749 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T09:17:19.749 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.750 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.750 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.750 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.750 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2024-04-24T09:17:19.750 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2024-04-24T09:17:19.750 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.750 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.750 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.750 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.751 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.751 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.751 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.751 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T09:17:19.751 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.751 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.751 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.751 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.752 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-24T09:17:19.752 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-24T09:17:19.752 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.752 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.752 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.752 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-24T09:17:19.752 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.752 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.752 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.753 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T09:17:19.753 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.753 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.753 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.753 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.753 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2024-04-24T09:17:19.753 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2024-04-24T09:17:19.753 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.753 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.754 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.754 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2024-04-24T09:17:19.754 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.754 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.754 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.754 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T09:17:19.754 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.754 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.754 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.754 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.755 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.755 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.755 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.755 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2024-04-24T09:17:19.755 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.755 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.755 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.755 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.755 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.756 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.756 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.756 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.756 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.756 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.756 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.756 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.756 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.757 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.757 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.757 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.757 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.757 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.757 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.757 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.757 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.757 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.758 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.758 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.758 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.758 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.758 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.758 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.758 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.758 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.758 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.759 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.759 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.759 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.759 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.759 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.759 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.759 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.759 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.760 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.760 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.760 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.760 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.760 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.760 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.760 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.760 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.760 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.761 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.761 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.761 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.761 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.761 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.761 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.761 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.761 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.761 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.762 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.762 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.762 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.762 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.762 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.762 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.762 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.762 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.763 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.763 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.763 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.763 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.763 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.763 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.763 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.763 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.763 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2024-04-24T09:17:19.764 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2024-04-24T09:17:19.764 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.764 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.764 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.764 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-04-24T09:17:19.764 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.764 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.764 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.765 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.765 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.765 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.765 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.765 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.765 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2024-04-24T09:17:19.765 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2024-04-24T09:17:19.765 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.765 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.766 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.766 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2024-04-24T09:17:19.766 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.766 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.766 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.766 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.766 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.766 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.767 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.767 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.767 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-24T09:17:19.767 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-24T09:17:19.767 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.767 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.767 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.767 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-24T09:17:19.767 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.767 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.768 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.768 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.768 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.768 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.768 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.768 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.768 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.768 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.769 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.769 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2024-04-24T09:17:19.769 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2024-04-24T09:17:19.769 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2024-04-24T09:17:19.769 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.769 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2024-04-24T09:17:19.769 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2024-04-24T09:17:19.769 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.769 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.770 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.770 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2024-04-24T09:17:19.770 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.770 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.770 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.770 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T09:17:19.770 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.770 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.771 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.771 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.771 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2024-04-24T09:17:19.771 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2024-04-24T09:17:19.771 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.771 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.771 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.771 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2024-04-24T09:17:19.771 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.772 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.772 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.772 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T09:17:19.772 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.772 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.772 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.772 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.772 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2024-04-24T09:17:19.772 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2024-04-24T09:17:19.773 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.773 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.773 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.773 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.773 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.773 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.773 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.773 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2024-04-24T09:17:19.773 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.773 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.774 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.774 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.774 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-24T09:17:19.774 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-24T09:17:19.774 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.774 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.774 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.774 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2024-04-24T09:17:19.774 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2024-04-24T09:17:19.775 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.775 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.775 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T09:17:19.775 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.775 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.775 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.775 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.775 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.775 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.776 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.776 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.776 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.776 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.776 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.776 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.776 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.776 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.776 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.776 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.777 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.777 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.777 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.777 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.777 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.777 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.777 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.777 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.777 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.778 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.778 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.778 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.778 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.778 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.778 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.778 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.778 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.778 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.779 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.779 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.779 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.779 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.779 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.779 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.779 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.779 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.779 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.780 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.780 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.780 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.780 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.780 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.780 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.780 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.780 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.780 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.780 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.781 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.781 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.781 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.781 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.781 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.781 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.781 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.781 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.781 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.782 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.782 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.782 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.782 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.782 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.782 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.782 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.782 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.783 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.783 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.783 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.783 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.783 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2024-04-24T09:17:19.783 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2024-04-24T09:17:19.783 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.783 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.783 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.783 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.784 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.784 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.784 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.784 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2024-04-24T09:17:19.784 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.784 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.784 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.784 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.784 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2024-04-24T09:17:19.784 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2024-04-24T09:17:19.785 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.785 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.785 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.785 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2024-04-24T09:17:19.785 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.785 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.785 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.785 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2024-04-24T09:17:19.785 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.785 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.786 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.786 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.786 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-04-24T09:17:19.786 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-04-24T09:17:19.786 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.786 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.786 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.786 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2024-04-24T09:17:19.786 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.787 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.787 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.787 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:17:19.787 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.787 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.787 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.787 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.787 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2024-04-24T09:17:19.787 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2024-04-24T09:17:19.788 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.788 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.788 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.788 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-24T09:17:19.788 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-24T09:17:19.788 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2024-04-24T09:17:19.788 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.788 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T09:17:19.788 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.788 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.789 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.789 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.789 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2024-04-24T09:17:19.789 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2024-04-24T09:17:19.789 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.789 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.789 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.789 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.789 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.790 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.790 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.790 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2024-04-24T09:17:19.790 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.790 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.790 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.790 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.790 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2024-04-24T09:17:19.791 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2024-04-24T09:17:19.791 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.791 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.791 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.791 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2024-04-24T09:17:19.791 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.791 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.791 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.791 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:17:19.791 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.792 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.792 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.792 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.792 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.792 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.792 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.792 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2024-04-24T09:17:19.792 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.792 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.793 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.793 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.793 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.793 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.793 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.793 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.793 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.793 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.793 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.793 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.794 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.794 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.794 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.794 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.794 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.794 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.794 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.794 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.794 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.794 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.795 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.795 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.795 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.795 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.795 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.795 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.795 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.795 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.795 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.796 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.796 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.796 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.796 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.796 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.796 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.796 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.796 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.796 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.796 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.797 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.797 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.797 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.797 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.797 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.797 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.797 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.797 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.797 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.798 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.798 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.798 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.798 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.798 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.798 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.798 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.798 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.798 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.798 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.799 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.799 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.799 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.799 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.799 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.799 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.799 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.799 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.799 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.800 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.800 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.800 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.800 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.800 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.800 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.800 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.800 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2024-04-24T09:17:19.800 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.800 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.801 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.801 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.801 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.801 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.801 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.801 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.801 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.801 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.801 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.802 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.802 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.802 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.802 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.802 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.802 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.802 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.802 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.802 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.803 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.803 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.803 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.803 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.803 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.803 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.803 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.803 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.803 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.803 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.804 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.804 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.804 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.804 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.804 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.804 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.804 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.804 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.804 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.804 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.805 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.805 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.805 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.805 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.805 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.805 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.805 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.805 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.805 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.805 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.806 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.806 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.806 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.806 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.806 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.806 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.806 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.806 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.806 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.807 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.807 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.807 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.807 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.807 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.807 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.807 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.807 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.807 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.808 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.808 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.808 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.808 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.808 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.808 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.808 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.808 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.808 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2024-04-24T09:17:19.808 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.809 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.809 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.809 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2024-04-24T09:17:19.809 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2024-04-24T09:17:19.809 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.809 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.809 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.809 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2024-04-24T09:17:19.809 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.809 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.810 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.810 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T09:17:19.810 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.810 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.810 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.810 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.810 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2024-04-24T09:17:19.810 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2024-04-24T09:17:19.810 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.811 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.811 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.811 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:19.811 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2024-04-24T09:17:19.811 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.811 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.811 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T09:17:19.811 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.812 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.812 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.812 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.812 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.812 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.812 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.812 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.812 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.812 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.813 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.813 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.813 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.813 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.813 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.813 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.813 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.813 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.813 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.814 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.814 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.814 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.814 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.814 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.814 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.814 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.814 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.815 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.815 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.815 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.815 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.815 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.815 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.815 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.815 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.816 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.816 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.816 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.816 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.816 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.816 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.816 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.816 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.817 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.817 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.817 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.817 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.817 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.817 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.817 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.817 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.817 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.817 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.818 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.818 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.818 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.818 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.818 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.818 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.818 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.818 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.819 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.819 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.819 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.819 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.819 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.819 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.819 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.819 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.819 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.819 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.820 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.820 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.820 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.820 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.820 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.820 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.820 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2024-04-24T09:17:19.820 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.820 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.821 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.821 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2024-04-24T09:17:19.821 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2024-04-24T09:17:19.821 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.821 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.821 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.821 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2024-04-24T09:17:19.821 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.821 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.822 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.822 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2024-04-24T09:17:19.822 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T09:17:19.822 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.822 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.822 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.822 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.822 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.823 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.823 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.823 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.823 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.823 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.823 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.823 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.823 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.823 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.823 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.824 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.824 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.824 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.824 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.824 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.824 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.824 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.824 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.825 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.825 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.825 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.825 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.825 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.825 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.825 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.825 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.825 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.826 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.826 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.826 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.826 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.826 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.826 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.826 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.826 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.827 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.827 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.827 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.827 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.827 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.827 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.827 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.827 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.827 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.827 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.828 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.828 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.828 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.828 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.828 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.828 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.828 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.828 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.828 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.829 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.829 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.829 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.829 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.829 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.829 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.829 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.829 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.829 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.829 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.830 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.830 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.830 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.830 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.830 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2024-04-24T09:17:19.830 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2024-04-24T09:17:19.830 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.831 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.831 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.831 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.831 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.831 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.831 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.831 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2024-04-24T09:17:19.831 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T09:17:19.831 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.832 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.832 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.832 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2024-04-24T09:17:19.832 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2024-04-24T09:17:19.832 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.832 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.832 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.832 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2024-04-24T09:17:19.833 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.833 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.833 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.833 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2024-04-24T09:17:19.833 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.833 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.833 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.833 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.833 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2024-04-24T09:17:19.834 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2024-04-24T09:17:19.834 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.834 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.834 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.834 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2024-04-24T09:17:19.834 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.834 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.834 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.835 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T09:17:19.835 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.835 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.835 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.835 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.835 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2024-04-24T09:17:19.835 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2024-04-24T09:17:19.835 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.835 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.835 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.836 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.836 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.836 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.836 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.836 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2024-04-24T09:17:19.836 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.836 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.836 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.837 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.837 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2024-04-24T09:17:19.837 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2024-04-24T09:17:19.837 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.837 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.837 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.837 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2024-04-24T09:17:19.837 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.837 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.837 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.838 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T09:17:19.838 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T09:17:19.838 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.838 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.838 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.838 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.838 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.842 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.842 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2024-04-24T09:17:19.842 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.843 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.843 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.843 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.843 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.843 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.843 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.843 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.843 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.843 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.844 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.844 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.844 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.844 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.844 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.844 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.844 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.844 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.844 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.845 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.845 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.845 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.845 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.845 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.845 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.845 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.845 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.846 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.846 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.846 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.846 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.846 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.846 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.846 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.846 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.846 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.847 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.847 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.847 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.847 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.847 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.847 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.847 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.847 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.848 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.848 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.848 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.848 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.848 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.848 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.848 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.848 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.849 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.849 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.849 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.849 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.849 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.849 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.849 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.849 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.849 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.850 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.850 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.850 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.850 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.850 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.850 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.850 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.850 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.851 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.851 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.851 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.851 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.851 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.851 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.851 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.851 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2024-04-24T09:17:19.852 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.852 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.852 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.852 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.852 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.852 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.852 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.852 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.853 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.853 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.853 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.853 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.853 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.853 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.853 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.853 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.853 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.854 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.854 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.854 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.854 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.854 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.854 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.854 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.854 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.855 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.855 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.855 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.855 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.855 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.855 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.855 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.855 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.855 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.856 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.856 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.856 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.856 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.856 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.856 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.856 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.856 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.856 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.857 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.857 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.857 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.857 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.857 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.857 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.857 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.857 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.857 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.857 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.858 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.858 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.858 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.858 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.858 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.858 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.858 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.859 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.859 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.859 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.859 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.859 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.859 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.859 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.859 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.859 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.859 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.860 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.860 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.860 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.860 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.860 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.860 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.860 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2024-04-24T09:17:19.860 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.860 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.861 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.861 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.861 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.861 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.861 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.861 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.861 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.861 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.861 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.861 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.862 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.862 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.862 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.862 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.862 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.862 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.862 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.862 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.863 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.863 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.863 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.863 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.863 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.863 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.863 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.863 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.863 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.864 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.864 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.864 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.864 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.864 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.864 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.864 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.864 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.864 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.865 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.865 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.865 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.865 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.865 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.865 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.865 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.865 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.865 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.866 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.866 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.866 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.866 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.866 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.866 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.866 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.866 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.867 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.867 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.867 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.867 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.867 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.867 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.867 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.867 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.867 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.868 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.868 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.868 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.868 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.868 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.868 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.868 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.868 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.868 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.868 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.869 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.869 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.869 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-04-24T09:17:19.869 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.869 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.869 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.869 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2024-04-24T09:17:19.869 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2024-04-24T09:17:19.869 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.870 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.870 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.870 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.870 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.870 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.870 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.870 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T09:17:19.870 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.871 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.871 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.871 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.871 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.871 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.871 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.871 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.871 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.871 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.871 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.872 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.872 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.872 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.872 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.872 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.872 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.872 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.872 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.872 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.873 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.873 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.873 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.873 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.873 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.873 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.873 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.873 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.873 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.873 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.874 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.874 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.874 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.874 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.874 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.874 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.874 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.874 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.875 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.875 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.875 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.875 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.875 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.875 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.875 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.875 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.875 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.876 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.876 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.876 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.876 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.876 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.876 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.876 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.876 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.876 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.877 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.877 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.877 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.877 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.877 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.877 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.877 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.877 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.877 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.877 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.878 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.878 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.878 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.878 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.878 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.878 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.878 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.878 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.878 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2024-04-24T09:17:19.879 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-04-24T09:17:19.879 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.879 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.879 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.879 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.879 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.879 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.879 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.879 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-04-24T09:17:19.880 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2024-04-24T09:17:19.880 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2024-04-24T09:17:19.880 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.880 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2024-04-24T09:17:19.880 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.880 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.880 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.880 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.880 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.880 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.881 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.881 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2024-04-24T09:17:19.881 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.881 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.881 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.881 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.881 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.881 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.881 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.881 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.882 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.882 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.882 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.882 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.882 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.882 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.882 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.882 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.882 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.883 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.883 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.883 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.883 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.883 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.883 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.883 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.883 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.883 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.884 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.884 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.884 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.884 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.884 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.884 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.884 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.884 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.885 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.885 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.885 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.885 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.885 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.885 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.885 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.885 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.885 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.886 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.886 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.886 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.886 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.886 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.886 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.886 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.886 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.887 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.887 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.887 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.887 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.887 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.887 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.887 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.887 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.888 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.888 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.888 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.888 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.888 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.888 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.888 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.888 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.889 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.889 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.889 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.889 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.889 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.889 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.889 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.889 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.889 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.890 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2024-04-24T09:17:19.890 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.890 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.890 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.890 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.890 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.890 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.890 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.891 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.891 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.891 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.891 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.891 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.891 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.891 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.891 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.891 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.891 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.892 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.892 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.892 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.892 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.892 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.892 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.892 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.892 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.892 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.893 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.893 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.893 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.893 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.893 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.893 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.893 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.893 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.893 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.894 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.894 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.894 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.894 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.894 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.894 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.894 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.894 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.895 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.895 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.895 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.895 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.895 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.895 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.895 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.895 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.896 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.896 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.896 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.896 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.896 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.896 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.896 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.896 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.896 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.896 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.897 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.897 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.897 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.897 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.897 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.897 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.897 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.897 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.897 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.898 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.898 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.898 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.898 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.898 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.898 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.898 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2024-04-24T09:17:19.898 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.899 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.899 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.899 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.899 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.899 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.899 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.899 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.899 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.899 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.900 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.900 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.900 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.900 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.900 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.900 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.900 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.900 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.900 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.901 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.901 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.901 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.901 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.901 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.901 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.901 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.902 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.902 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.902 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.902 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.902 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.902 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.902 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.902 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.902 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.903 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.903 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.903 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.903 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.903 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.903 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.903 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.903 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.903 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.904 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.904 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.904 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.904 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.904 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.904 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.904 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.904 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.904 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.904 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.905 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.905 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.905 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.905 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.905 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.905 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.905 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.905 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.905 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.906 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.906 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.906 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.906 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.906 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.906 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.906 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.906 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.907 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.907 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-24T09:17:19.907 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-24T09:17:19.907 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.907 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.907 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.907 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-24T09:17:19.907 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.907 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.907 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.908 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.908 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.908 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.908 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.908 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.908 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2024-04-24T09:17:19.908 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2024-04-24T09:17:19.908 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:19.908 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.909 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.909 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-04-24T09:17:19.909 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2024-04-24T09:17:19.909 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.909 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.909 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2024-04-24T09:17:19.909 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T09:17:19.909 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.909 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.910 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.910 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.910 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.910 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.910 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2024-04-24T09:17:19.910 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.910 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.910 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.910 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2024-04-24T09:17:19.911 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2024-04-24T09:17:19.911 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.911 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.911 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.911 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.911 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.911 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.911 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.911 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T09:17:19.912 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.912 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.912 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.912 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.912 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-04-24T09:17:19.912 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-04-24T09:17:19.912 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.912 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.912 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.913 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.913 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.913 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.913 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.913 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.913 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.913 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.913 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.913 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.914 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.914 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.914 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.914 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.914 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.914 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.914 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.914 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.915 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.915 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.915 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.915 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.915 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.915 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.915 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.915 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.915 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.916 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.916 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.916 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.916 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.916 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.916 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.916 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.916 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.917 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.917 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.917 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.917 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.917 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.917 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.917 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.917 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.917 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.917 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.918 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.918 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.918 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.918 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.918 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.918 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.918 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.918 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.918 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.919 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.919 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.919 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.919 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.919 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.919 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.919 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.919 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.919 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.920 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.920 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.920 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.920 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.920 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.920 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.920 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.920 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.920 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.921 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.921 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.921 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.921 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.921 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.921 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.921 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.921 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.921 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2024-04-24T09:17:19.922 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2024-04-24T09:17:19.922 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.922 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.922 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.922 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2024-04-24T09:17:19.922 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.922 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.922 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.923 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2024-04-24T09:17:19.923 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.923 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.923 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.923 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.923 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-24T09:17:19.923 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-24T09:17:19.923 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:19.924 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.924 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.924 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-24T09:17:19.924 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.924 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.924 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.924 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2024-04-24T09:17:19.924 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.924 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.924 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2024-04-24T09:17:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2024-04-24T09:17:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2024-04-24T09:17:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2024-04-24T09:17:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2024-04-24T09:17:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T09:17:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T09:17:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2024-04-24T09:17:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2024-04-24T09:17:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.937 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.937 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.937 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.937 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.937 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.937 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.937 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.937 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.937 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.937 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T09:17:19.938 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T09:17:19.938 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.938 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.938 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.938 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2024-04-24T09:17:19.938 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.938 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.938 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.939 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.939 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.939 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.939 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.939 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.939 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2024-04-24T09:17:19.939 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2024-04-24T09:17:19.939 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:19.939 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.939 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.940 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2024-04-24T09:17:19.940 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.940 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.940 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.940 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.940 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.940 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.940 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.940 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.941 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-24T09:17:19.941 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-24T09:17:19.941 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.941 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.941 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.941 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-04-24T09:17:19.941 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.941 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.941 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.941 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T09:17:19.942 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.942 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.942 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.942 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.942 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-24T09:17:19.942 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-24T09:17:19.942 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.942 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.942 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.943 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2024-04-24T09:17:19.943 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.943 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.943 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.943 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T09:17:19.943 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.943 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.943 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.943 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.944 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2024-04-24T09:17:19.944 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2024-04-24T09:17:19.944 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.944 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.944 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.944 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2024-04-24T09:17:19.944 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.944 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.945 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.945 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.945 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.945 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.945 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.945 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.945 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-04-24T09:17:19.945 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-04-24T09:17:19.945 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.946 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.946 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.946 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2024-04-24T09:17:19.946 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.946 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.946 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.946 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2024-04-24T09:17:19.946 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2024-04-24T09:17:19.946 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.947 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.947 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.947 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.947 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.947 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.947 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-04-24T09:17:19.947 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-04-24T09:17:19.947 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.947 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.948 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.948 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-04-24T09:17:19.948 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-04-24T09:17:19.948 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-04-24T09:17:19.948 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.948 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.948 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.948 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.948 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.948 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.949 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.949 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.949 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.949 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-04-24T09:17:19.949 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.949 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.949 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.949 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.949 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.949 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.950 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.950 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.950 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.950 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.950 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.950 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.950 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.950 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.950 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.951 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.951 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.951 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.951 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.951 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.951 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.951 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.951 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.951 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.952 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.952 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.952 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.952 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.952 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.952 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.952 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.952 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.952 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.953 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.953 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.953 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.953 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.953 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.953 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.953 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.953 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.953 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.954 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.954 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.954 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.954 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.954 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.954 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.954 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.954 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.955 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.955 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.955 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.955 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.955 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.955 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.955 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.955 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.955 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.956 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.956 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.956 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.956 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.956 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.956 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.956 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.956 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.956 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.956 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.957 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.957 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.957 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.957 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.957 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2024-04-24T09:17:19.957 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2024-04-24T09:17:19.957 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:19.957 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.958 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.958 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-24T09:17:19.958 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.958 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.958 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.958 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.958 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.958 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.958 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.959 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.959 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2024-04-24T09:17:19.959 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T09:17:19.959 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.959 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.959 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.959 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.959 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.959 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.960 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.960 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.960 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.960 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.960 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.960 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.960 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2024-04-24T09:17:19.960 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2024-04-24T09:17:19.960 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.961 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.961 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.961 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.961 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.961 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.961 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.961 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.961 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.961 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.961 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.962 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.962 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.962 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.962 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.962 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2024-04-24T09:17:19.962 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.962 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.962 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.963 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2024-04-24T09:17:19.963 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2024-04-24T09:17:19.963 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.963 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.963 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.963 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:19.963 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.963 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.963 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.963 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.964 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.964 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.964 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.964 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.964 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-04-24T09:17:19.964 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-04-24T09:17:19.964 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.964 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.964 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.965 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.965 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.965 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.965 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.965 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.965 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.965 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.965 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.965 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.966 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.966 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.966 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.966 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.966 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.966 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.966 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.966 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.967 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.967 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.967 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.967 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.967 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.967 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.967 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.967 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.967 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.968 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.968 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.968 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.968 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.968 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.968 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.968 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.968 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.968 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.969 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.969 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.969 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.969 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.969 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.969 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.969 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.969 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.969 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.970 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.970 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.970 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.970 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.970 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.970 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.970 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.970 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.971 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.971 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.971 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.971 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.971 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.971 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.971 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.971 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.971 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.972 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.972 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.972 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.972 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.972 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.972 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.972 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.972 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.972 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.972 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.973 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.973 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.973 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.973 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.973 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.973 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.973 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.973 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.973 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-24T09:17:19.974 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-24T09:17:19.974 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.974 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.974 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.974 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.974 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.974 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.974 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.974 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.975 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.975 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.975 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.975 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.975 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-24T09:17:19.975 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-24T09:17:19.975 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.975 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.975 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.975 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.976 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.976 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.976 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.976 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.976 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.976 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.976 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.976 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.976 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.976 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.977 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.977 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2024-04-24T09:17:19.977 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.977 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.977 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.977 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.977 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.977 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.977 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.978 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.978 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.978 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.978 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.978 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.978 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.978 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.978 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.978 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.979 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.979 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.979 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.979 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.979 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.979 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.979 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.979 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.979 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.979 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.980 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.980 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.980 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.980 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.980 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.980 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.980 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.980 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.980 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.981 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.981 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.981 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.981 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.981 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.981 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.981 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.981 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.981 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.981 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.982 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.982 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.982 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.982 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.982 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.982 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.982 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.982 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.983 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.983 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.983 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.983 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.983 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.983 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.983 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.983 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.984 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.984 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.984 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.984 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.984 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.984 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.984 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.984 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.984 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.984 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.985 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.985 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.985 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.985 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.985 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.985 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2024-04-24T09:17:19.985 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.985 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.985 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.985 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.986 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.986 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.986 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.986 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.986 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.986 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.986 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.986 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.987 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.987 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.987 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.987 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.987 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.987 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.987 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.987 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.988 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.988 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.988 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.988 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.988 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.988 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.988 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.988 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.988 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.988 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.989 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.989 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.989 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.989 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.989 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.989 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.989 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.989 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.989 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:19.990 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:19.990 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.990 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.990 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.990 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:19.990 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.990 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.990 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.990 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.991 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.991 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.991 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.991 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.991 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.991 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.991 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.991 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.991 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.991 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.992 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.992 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:19.992 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:19.992 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.992 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.992 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.992 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.992 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.992 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.993 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.993 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.993 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.993 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.993 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.993 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.993 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2024-04-24T09:17:19.993 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2024-04-24T09:17:19.993 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.994 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.994 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:19.994 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2024-04-24T09:17:19.994 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.994 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.994 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.994 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T09:17:19.994 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.994 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.995 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.995 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.995 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:19.995 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.995 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:19.995 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2024-04-24T09:17:19.995 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:19.995 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:19.995 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:19.996 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:19.996 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:19.996 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:19.996 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.996 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.996 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:19.996 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.996 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.996 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:19.997 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:19.997 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:19.997 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:19.997 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:19.997 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:19.997 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:19.997 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:19.997 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.998 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.998 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.998 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.998 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.998 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:19.998 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:19.998 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:19.998 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:19.998 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:19.999 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:19.999 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:19.999 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:19.999 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:19.999 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:19.999 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:19.999 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:19.999 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:19.999 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:19.999 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:20.000 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:20.000 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.000 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.000 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.000 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:20.000 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.000 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.000 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.001 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.001 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.001 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.001 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.001 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.001 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.001 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.001 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.002 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.002 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.002 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.002 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.002 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:20.002 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:20.002 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.002 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.003 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.003 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.003 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.003 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.003 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.003 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.003 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.003 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.003 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.004 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.004 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2024-04-24T09:17:20.004 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2024-04-24T09:17:20.004 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.004 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.004 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.004 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.004 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.004 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.005 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.005 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.005 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.005 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.005 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.005 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.005 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2024-04-24T09:17:20.005 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2024-04-24T09:17:20.006 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.006 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.006 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.006 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2024-04-24T09:17:20.006 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.006 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.006 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.006 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.006 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.007 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.007 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.007 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.007 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2024-04-24T09:17:20.007 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2024-04-24T09:17:20.007 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.007 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.007 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.007 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.007 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.008 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.008 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.008 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.008 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.008 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.008 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.008 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.008 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2024-04-24T09:17:20.008 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2024-04-24T09:17:20.009 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:20.009 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.009 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.009 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.009 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.009 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.009 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.009 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.009 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.010 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.010 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.010 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.010 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2024-04-24T09:17:20.010 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2024-04-24T09:17:20.010 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-24T09:17:20.010 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.010 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.011 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.011 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.011 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.011 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.011 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.011 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.011 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.011 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.011 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.011 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2024-04-24T09:17:20.012 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2024-04-24T09:17:20.012 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.012 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.012 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.012 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.012 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.012 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.012 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.013 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.013 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.013 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.013 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.013 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.013 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2024-04-24T09:17:20.013 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2024-04-24T09:17:20.013 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.013 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.013 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.014 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.014 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.014 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.014 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.014 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.014 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.014 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.014 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.015 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.015 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2024-04-24T09:17:20.015 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2024-04-24T09:17:20.015 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.015 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.015 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.015 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:20.015 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.015 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.016 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.016 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.016 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.016 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.016 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.016 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.016 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2024-04-24T09:17:20.016 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2024-04-24T09:17:20.016 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:20.017 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.017 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.017 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.017 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-24T09:17:20.017 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2024-04-24T09:17:20.017 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.017 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.017 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.018 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.018 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.018 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.018 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2024-04-24T09:17:20.018 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2024-04-24T09:17:20.018 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.018 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.018 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.018 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.019 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.019 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.019 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.019 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.019 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.019 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.019 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.019 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.020 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2024-04-24T09:17:20.020 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2024-04-24T09:17:20.020 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.020 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.020 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.020 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.020 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.020 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.020 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.021 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.021 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.021 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.021 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.021 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.021 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2024-04-24T09:17:20.021 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2024-04-24T09:17:20.021 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.021 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.021 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.022 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.022 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.022 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.022 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.022 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.022 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.022 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.022 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.023 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.023 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.023 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.023 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.023 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-04-24T09:17:20.023 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:20.023 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:20.023 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:20.023 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2024-04-24T09:17:20.024 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2024-04-24T09:17:20.024 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.024 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.024 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.024 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.024 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.024 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.024 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.024 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T09:17:20.025 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.025 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.025 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.025 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.025 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2024-04-24T09:17:20.025 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2024-04-24T09:17:20.025 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.025 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.025 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.025 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.026 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.026 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.026 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.026 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2024-04-24T09:17:20.026 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.026 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.026 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.026 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.026 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:20.027 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:20.027 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.027 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.027 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.027 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.027 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.027 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.027 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.027 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.028 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.028 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.028 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.028 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.028 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.028 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.028 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.028 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.028 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.028 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.029 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.029 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:20.029 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:20.029 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.029 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.029 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.029 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.029 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.029 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.029 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.030 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.030 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.030 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.030 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.030 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.030 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:20.030 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:20.030 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.030 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.031 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.031 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:20.031 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.031 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.031 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.031 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.031 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.031 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.031 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.031 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.032 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.032 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.032 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.032 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.032 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.032 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.032 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.032 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:20.032 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:20.033 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.033 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.033 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.033 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.033 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.033 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.033 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.033 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.033 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.034 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.034 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.034 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.034 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.034 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.034 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.034 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2024-04-24T09:17:20.034 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:20.034 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:20.034 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:20.035 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:20.035 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:20.035 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.035 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.035 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.035 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.035 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.035 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.035 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.035 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.036 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.036 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.036 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.036 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.036 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.036 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.036 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.036 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.036 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.037 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.037 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.037 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:20.037 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:20.037 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.037 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.037 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.037 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.037 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.038 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.038 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.038 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.038 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.038 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.038 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.038 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.038 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:20.039 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:20.039 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.039 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.039 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.039 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:20.039 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.039 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.039 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.039 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.039 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.040 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.040 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.040 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.040 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.040 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.040 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.040 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.040 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.040 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.041 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.041 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:20.041 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:20.041 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.041 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.041 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.041 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.041 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.041 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.041 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.042 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.042 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.042 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.042 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.042 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.042 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.042 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.042 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.042 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2024-04-24T09:17:20.043 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:20.043 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:20.043 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:20.043 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:20.043 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:20.043 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.043 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.043 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.043 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.043 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.044 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.044 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.044 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.044 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.044 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.044 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.044 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.044 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.044 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.045 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.045 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.045 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.045 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.045 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.045 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:20.045 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:20.045 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.045 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.046 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.046 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.046 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.046 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.046 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.046 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.046 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.046 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.047 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.047 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.047 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:20.047 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:20.047 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.047 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.047 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.047 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:20.047 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.048 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.048 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.048 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.048 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.048 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.048 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.048 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.048 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.048 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.049 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.049 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.049 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.049 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.049 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.049 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:20.049 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:20.049 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.049 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.049 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.050 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.050 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.050 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.050 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.050 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.050 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.050 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.050 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.050 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.051 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.051 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.051 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.051 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2024-04-24T09:17:20.051 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:20.051 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:20.051 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:20.051 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2024-04-24T09:17:20.051 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2024-04-24T09:17:20.051 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.052 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.052 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.052 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T09:17:20.052 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.052 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.052 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.052 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.052 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.052 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.053 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.053 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.053 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-24T09:17:20.053 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-24T09:17:20.053 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:20.053 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.053 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.053 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2024-04-24T09:17:20.053 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.054 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.054 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.054 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.054 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.054 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.054 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.054 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.054 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:20.054 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:20.054 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.055 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.055 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.055 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.055 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.055 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.055 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.055 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.055 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.055 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.055 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.056 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.056 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.056 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.056 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.056 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.056 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.056 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.056 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.056 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:20.057 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:20.057 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.057 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.057 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.057 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.057 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.057 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.057 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.057 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.057 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.058 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.058 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.058 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.058 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:20.058 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:20.058 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.058 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.058 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.059 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:20.059 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.059 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.059 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.059 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.059 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.059 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.059 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.059 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.060 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.060 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.060 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.060 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.060 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.060 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.060 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.060 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:20.061 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:20.061 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.061 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.061 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.061 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.061 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.061 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.061 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.061 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.061 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.062 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.062 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.062 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.062 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.062 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.062 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.062 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2024-04-24T09:17:20.062 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:20.062 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:20.063 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:20.063 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2024-04-24T09:17:20.063 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2024-04-24T09:17:20.063 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.063 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.063 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.063 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:20.063 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.063 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.064 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.064 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T09:17:20.064 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.064 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.064 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.064 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.064 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2024-04-24T09:17:20.064 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2024-04-24T09:17:20.064 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.065 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.065 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.065 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:20.065 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.065 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.065 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.065 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T09:17:20.065 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.065 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.065 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.066 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.066 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2024-04-24T09:17:20.066 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2024-04-24T09:17:20.066 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.066 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.066 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.066 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:20.066 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.067 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.067 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.067 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T09:17:20.067 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.067 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.067 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.067 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.067 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2024-04-24T09:17:20.067 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2024-04-24T09:17:20.067 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.068 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.068 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.068 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.068 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.068 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.068 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.068 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T09:17:20.068 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.069 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.069 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.069 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.069 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2024-04-24T09:17:20.069 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2024-04-24T09:17:20.069 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.069 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.069 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.069 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.070 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.070 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.070 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.070 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T09:17:20.070 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.070 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.070 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.070 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.070 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2024-04-24T09:17:20.071 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2024-04-24T09:17:20.071 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.071 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.071 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.071 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.071 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.071 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.071 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.071 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.072 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.072 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.072 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.072 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.072 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2024-04-24T09:17:20.072 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2024-04-24T09:17:20.072 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.072 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.072 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.072 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.073 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.073 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.073 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.073 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.073 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.073 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.073 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.073 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.073 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2024-04-24T09:17:20.073 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2024-04-24T09:17:20.074 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.074 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.074 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.074 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T09:17:20.074 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.074 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.074 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.074 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.075 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.075 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.075 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.075 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.075 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-04-24T09:17:20.075 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-04-24T09:17:20.075 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.075 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.075 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.076 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.076 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.076 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.076 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.076 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.076 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.076 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.076 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.076 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.076 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-24T09:17:20.077 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-24T09:17:20.077 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:20.077 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.077 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.077 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2024-04-24T09:17:20.077 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2024-04-24T09:17:20.077 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.077 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.077 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.078 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.078 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.078 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.078 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.078 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2024-04-24T09:17:20.078 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2024-04-24T09:17:20.078 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:20.078 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.078 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.079 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2024-04-24T09:17:20.079 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.079 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.079 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.079 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.079 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.079 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.079 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.079 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.079 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2024-04-24T09:17:20.080 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2024-04-24T09:17:20.080 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.080 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.080 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.080 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.080 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.080 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.080 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.080 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.080 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.081 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.081 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.081 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.081 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2024-04-24T09:17:20.081 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2024-04-24T09:17:20.081 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.081 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.081 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.081 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.081 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.082 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.082 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.082 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.082 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.082 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.082 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.082 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.082 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:20.082 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:20.083 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.083 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.083 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.083 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.083 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.083 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.083 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.083 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.083 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.084 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.084 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.084 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.084 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.084 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.084 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.084 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.084 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.084 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.084 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.085 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:20.085 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:20.085 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.085 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.085 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.085 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.085 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.085 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.085 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.085 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.086 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.086 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.086 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.086 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.086 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:20.086 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:20.086 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.086 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.086 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.087 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:20.087 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.087 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.087 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.087 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.087 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.087 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.087 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.087 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.088 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.088 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.088 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.088 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.088 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.088 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.088 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.088 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:20.089 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:20.089 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.089 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.089 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.089 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.089 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.089 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.089 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.089 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.090 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.090 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.090 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.090 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.090 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2024-04-24T09:17:20.090 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2024-04-24T09:17:20.090 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.090 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.090 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.091 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.091 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.091 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.091 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.091 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.091 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.091 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.091 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.092 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.092 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2024-04-24T09:17:20.092 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2024-04-24T09:17:20.092 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.092 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.092 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.092 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.092 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.092 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.093 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.093 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.093 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.093 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.093 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.093 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.093 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2024-04-24T09:17:20.093 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2024-04-24T09:17:20.093 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.093 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.094 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.094 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T09:17:20.094 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.094 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.094 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.094 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.094 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.094 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.094 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.095 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.095 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.095 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.095 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.095 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2024-04-24T09:17:20.095 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:20.095 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:20.095 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:20.095 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:20.096 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:20.096 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.096 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.096 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.096 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.096 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.096 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.096 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.096 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.096 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.097 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.097 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.097 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.097 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.097 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.097 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.097 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.097 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.097 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.098 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.098 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:20.098 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:20.098 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.098 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.098 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.098 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.098 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.098 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.099 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.099 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.099 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.099 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.099 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.099 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.099 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:20.099 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:20.099 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.099 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.100 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.100 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:20.100 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.100 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.100 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.100 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.100 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.100 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.100 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.100 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.101 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.101 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.101 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.101 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.101 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.101 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.101 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.101 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:20.101 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:20.102 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.102 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.102 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.102 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.102 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.102 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.102 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.102 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.102 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.103 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.103 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.103 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.103 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.103 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.103 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.103 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-04-24T09:17:20.103 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:20.103 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:20.104 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:20.104 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:20.104 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:20.104 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.104 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.104 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.104 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.104 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.104 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.105 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.105 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.105 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.105 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.105 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.105 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.105 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.105 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.105 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.105 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.106 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.106 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.106 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.106 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:20.106 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:20.106 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.106 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.106 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.106 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.107 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.107 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.107 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.107 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.107 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.107 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.107 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.107 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.107 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:20.108 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:20.108 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.108 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.108 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.108 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:20.108 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.108 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.108 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.108 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.108 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.109 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.109 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.109 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.109 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.109 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.109 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.109 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.109 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.109 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.109 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.110 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:20.110 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:20.110 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.110 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.110 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.110 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.110 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.110 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.110 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.111 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.111 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.111 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.111 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.111 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.111 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2024-04-24T09:17:20.111 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2024-04-24T09:17:20.111 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:20.111 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.111 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.112 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2024-04-24T09:17:20.112 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.112 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.112 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.112 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T09:17:20.112 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.112 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.112 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.112 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.112 INFO:teuthology.orchestra.run.smithi069.stdout: "periodic_async_work": { 2024-04-24T09:17:20.113 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "periodic_async_work", 2024-04-24T09:17:20.113 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.113 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.113 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.113 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.113 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.113 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.113 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.113 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Periodically check for async work", 2024-04-24T09:17:20.114 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.114 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.114 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.114 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.114 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2024-04-24T09:17:20.114 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2024-04-24T09:17:20.114 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:20.114 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.114 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.115 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-24T09:17:20.115 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.115 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.115 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.115 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T09:17:20.115 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.115 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.115 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.115 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.116 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_no_wait": { 2024-04-24T09:17:20.116 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T09:17:20.116 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.116 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.116 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.116 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-24T09:17:20.116 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.116 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.116 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.117 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T09:17:20.117 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.117 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.117 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.117 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.117 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.117 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.117 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.117 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2024-04-24T09:17:20.117 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-24T09:17:20.118 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-24T09:17:20.118 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-24T09:17:20.118 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2024-04-24T09:17:20.118 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2024-04-24T09:17:20.118 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-24T09:17:20.118 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.118 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.118 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2024-04-24T09:17:20.118 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.119 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.119 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.119 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.119 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.119 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.119 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.119 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.119 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2024-04-24T09:17:20.119 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2024-04-24T09:17:20.120 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.120 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.120 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.120 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.120 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.120 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.120 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.120 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.120 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.120 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.121 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.121 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.121 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-24T09:17:20.121 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-24T09:17:20.121 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-24T09:17:20.121 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.121 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.121 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-24T09:17:20.121 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.121 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.122 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.122 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.122 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.122 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.122 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.122 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.122 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-24T09:17:20.122 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-24T09:17:20.123 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.123 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.123 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.123 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.123 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.123 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.123 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.123 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.123 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.123 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.124 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.124 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.124 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.124 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.124 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.124 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.124 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.124 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.124 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.125 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-24T09:17:20.125 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-24T09:17:20.125 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.125 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.125 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.125 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.125 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.125 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.125 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.126 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.126 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.126 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.126 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.126 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.126 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-24T09:17:20.126 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-24T09:17:20.126 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.127 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.127 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.127 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-24T09:17:20.127 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.127 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.127 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-24T09:17:20.127 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-24T09:17:20.127 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-24T09:17:20.127 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-24T09:17:20.127 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-24T09:17:20.128 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-24T09:17:20.128 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-24T09:17:20.128 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.128 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.128 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.128 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.128 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.128 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.128 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-24T09:17:20.129 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-24T09:17:20.129 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-24T09:17:20.129 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.129 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-24T09:17:20.129 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-24T09:17:20.129 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.129 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.129 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.129 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.130 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.130 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.130 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.130 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.130 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2024-04-24T09:17:20.130 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2024-04-24T09:17:20.130 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.130 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.130 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.130 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-24T09:17:20.131 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.131 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.131 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.131 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.131 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.131 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.131 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.131 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.131 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2024-04-24T09:17:20.132 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2024-04-24T09:17:20.132 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-24T09:17:20.132 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.132 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.132 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2024-04-24T09:17:20.132 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.132 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.132 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.132 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.133 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.133 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.133 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.133 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.133 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2024-04-24T09:17:20.133 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2024-04-24T09:17:20.133 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-24T09:17:20.133 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-24T09:17:20.133 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-24T09:17:20.133 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T09:17:20.134 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-24T09:17:20.134 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-24T09:17:20.134 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-24T09:17:20.134 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-24T09:17:20.134 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-24T09:17:20.134 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-24T09:17:20.134 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-24T09:17:20.134 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.135 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.135 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.135 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.135 INFO:teuthology.orchestra.run.smithi069.stdout: "services": { 2024-04-24T09:17:20.135 INFO:teuthology.orchestra.run.smithi069.stdout: "dashboard": "https://172.21.15.69:8443/" 2024-04-24T09:17:20.135 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.135 INFO:teuthology.orchestra.run.smithi069.stdout: "always_on_modules": { 2024-04-24T09:17:20.135 INFO:teuthology.orchestra.run.smithi069.stdout: "octopus": [ 2024-04-24T09:17:20.135 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-04-24T09:17:20.135 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-04-24T09:17:20.136 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-04-24T09:17:20.136 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-04-24T09:17:20.136 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-04-24T09:17:20.136 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-04-24T09:17:20.136 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-04-24T09:17:20.136 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-04-24T09:17:20.136 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-04-24T09:17:20.136 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-04-24T09:17:20.136 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.136 INFO:teuthology.orchestra.run.smithi069.stdout: "pacific": [ 2024-04-24T09:17:20.137 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-04-24T09:17:20.137 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-04-24T09:17:20.137 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-04-24T09:17:20.137 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-04-24T09:17:20.137 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-04-24T09:17:20.137 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-04-24T09:17:20.137 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-04-24T09:17:20.137 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-04-24T09:17:20.137 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-04-24T09:17:20.138 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-04-24T09:17:20.138 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.138 INFO:teuthology.orchestra.run.smithi069.stdout: "quincy": [ 2024-04-24T09:17:20.138 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-04-24T09:17:20.138 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-04-24T09:17:20.138 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-04-24T09:17:20.138 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-04-24T09:17:20.138 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-04-24T09:17:20.138 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-04-24T09:17:20.139 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-04-24T09:17:20.139 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-04-24T09:17:20.139 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-04-24T09:17:20.139 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-04-24T09:17:20.139 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.139 INFO:teuthology.orchestra.run.smithi069.stdout: "reef": [ 2024-04-24T09:17:20.139 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-04-24T09:17:20.139 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-04-24T09:17:20.139 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-04-24T09:17:20.140 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-04-24T09:17:20.140 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-04-24T09:17:20.140 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-04-24T09:17:20.140 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-04-24T09:17:20.140 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-04-24T09:17:20.140 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-04-24T09:17:20.140 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-04-24T09:17:20.140 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-24T09:17:20.140 INFO:teuthology.orchestra.run.smithi069.stdout: "squid": [ 2024-04-24T09:17:20.140 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-04-24T09:17:20.141 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-04-24T09:17:20.141 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-04-24T09:17:20.141 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-04-24T09:17:20.141 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-04-24T09:17:20.141 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-04-24T09:17:20.141 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-04-24T09:17:20.141 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-04-24T09:17:20.141 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-04-24T09:17:20.141 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-04-24T09:17:20.141 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-24T09:17:20.142 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.142 INFO:teuthology.orchestra.run.smithi069.stdout: "last_failure_osd_epoch": 93, 2024-04-24T09:17:20.142 INFO:teuthology.orchestra.run.smithi069.stdout: "active_clients": [ 2024-04-24T09:17:20.142 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.142 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-04-24T09:17:20.142 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-04-24T09:17:20.142 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.142 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-04-24T09:17:20.142 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-04-24T09:17:20.143 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 1239806257 2024-04-24T09:17:20.143 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.143 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-24T09:17:20.143 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.143 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.143 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "libcephsqlite", 2024-04-24T09:17:20.143 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-04-24T09:17:20.143 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.143 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-04-24T09:17:20.143 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-04-24T09:17:20.144 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 3232422425 2024-04-24T09:17:20.144 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.144 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-24T09:17:20.144 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.144 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.144 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-04-24T09:17:20.144 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-04-24T09:17:20.144 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.144 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-04-24T09:17:20.144 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-04-24T09:17:20.145 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 3271141573 2024-04-24T09:17:20.145 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.145 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-24T09:17:20.145 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.145 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.145 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-04-24T09:17:20.145 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-04-24T09:17:20.145 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.145 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-04-24T09:17:20.146 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-04-24T09:17:20.146 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 4025059608 2024-04-24T09:17:20.146 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.146 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-24T09:17:20.146 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.146 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.146 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-04-24T09:17:20.146 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-04-24T09:17:20.146 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.147 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-04-24T09:17:20.147 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-04-24T09:17:20.147 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 1603906372 2024-04-24T09:17:20.147 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.147 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-24T09:17:20.147 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:20.147 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.147 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-04-24T09:17:20.147 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-04-24T09:17:20.148 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-24T09:17:20.148 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-04-24T09:17:20.148 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-04-24T09:17:20.148 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 1001987171 2024-04-24T09:17:20.148 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.148 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-24T09:17:20.148 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:20.148 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-24T09:17:20.148 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:17:20.150 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:18 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/4036017596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:17:20.150 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:18 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/432270178' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T09:17:20.150 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:19 smithi130 ceph-mon[31553]: pgmap v318: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 209 B/s wr, 7 op/s 2024-04-24T09:17:20.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:18 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/4036017596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:17:20.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:18 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/432270178' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T09:17:20.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:19 smithi081 ceph-mon[30961]: pgmap v318: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 209 B/s wr, 7 op/s 2024-04-24T09:17:20.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:18.152+0000 7fcab7200640 1 -- 172.21.15.69:0/432270178 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fca9c0a3da0 msgr2=0x7fca9c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:20.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:18.152+0000 7fcab7200640 1 --2- 172.21.15.69:0/432270178 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fca9c0a3da0 0x7fca9c0a6260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcab8101a90 tx=0x7fcaa8005d30 comp rx=0 tx=0).stop 2024-04-24T09:17:20.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:18.152+0000 7fcab7200640 1 -- 172.21.15.69:0/432270178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab8106d40 msgr2=0x7fcab81a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:20.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:18.152+0000 7fcab7200640 1 --2- 172.21.15.69:0/432270178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab8106d40 0x7fcab81a4ca0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4006010 tx=0x7fcaa4004270 comp rx=0 tx=0).stop 2024-04-24T09:17:20.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:18.152+0000 7fcab7200640 1 -- 172.21.15.69:0/432270178 shutdown_connections 2024-04-24T09:17:20.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:18.152+0000 7fcab7200640 1 --2- 172.21.15.69:0/432270178 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fca9c0a3da0 0x7fca9c0a6260 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:20.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:18.152+0000 7fcab7200640 1 --2- 172.21.15.69:0/432270178 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcab81071b0 0x7fcab819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:20.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:18.152+0000 7fcab7200640 1 --2- 172.21.15.69:0/432270178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcab8106d40 0x7fcab81a4ca0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:20.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:18.152+0000 7fcab7200640 1 --2- 172.21.15.69:0/432270178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcab8104890 0x7fcab81a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:20.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:18.152+0000 7fcab7200640 1 -- 172.21.15.69:0/432270178 >> 172.21.15.69:0/432270178 conn(0x7fcab8076840 msgr2=0x7fcab80fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:20.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:18.152+0000 7fcab7200640 1 -- 172.21.15.69:0/432270178 shutdown_connections 2024-04-24T09:17:20.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:17:18.152+0000 7fcab7200640 1 -- 172.21.15.69:0/432270178 wait complete. 2024-04-24T09:17:20.160 DEBUG:tasks.check_counter:Getting stats from a 2024-04-24T09:17:20.160 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-24T09:17:20.161 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:18 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/4036017596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:17:20.161 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:18 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/432270178' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T09:17:20.161 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:19 smithi069 ceph-mon[28170]: pgmap v318: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 209 B/s wr, 7 op/s 2024-04-24T09:17:20.642 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:17:21.765 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:21 smithi069 ceph-mon[28170]: pgmap v319: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-24T09:17:21.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:21 smithi081 ceph-mon[30961]: pgmap v319: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-24T09:17:21.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:21 smithi130 ceph-mon[31553]: pgmap v319: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-24T09:17:23.120 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:17:23.120 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:17:23.120 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 232, 2024-04-24T09:17:23.120 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 219, 2024-04-24T09:17:23.121 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1594320, 2024-04-24T09:17:23.121 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 134112, 2024-04-24T09:17:23.121 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-04-24T09:17:23.121 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-04-24T09:17:23.121 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.098682785, 2024-04-24T09:17:23.121 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.034828224, 2024-04-24T09:17:23.121 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.064846844, 2024-04-24T09:17:23.122 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.009744679, 2024-04-24T09:17:23.122 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:23.122 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 219, 2024-04-24T09:17:23.122 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.022695675, 2024-04-24T09:17:23.122 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000103633 2024-04-24T09:17:23.122 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:23.122 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:23.122 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:23.122 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:23.123 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:23.123 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:23.123 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1594320, 2024-04-24T09:17:23.123 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 134112 2024-04-24T09:17:23.123 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:23.123 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:17:23.123 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 2, 2024-04-24T09:17:23.123 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 166, 2024-04-24T09:17:23.124 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 320, 2024-04-24T09:17:23.124 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 185392, 2024-04-24T09:17:23.124 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 5, 2024-04-24T09:17:23.124 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-04-24T09:17:23.124 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.033879896, 2024-04-24T09:17:23.124 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.024812830, 2024-04-24T09:17:23.124 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.003517635, 2024-04-24T09:17:23.124 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:17:23.125 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:23.125 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 163, 2024-04-24T09:17:23.125 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.030881843, 2024-04-24T09:17:23.125 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000189459 2024-04-24T09:17:23.125 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:23.125 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:23.125 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:23.125 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:23.126 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:23.126 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:23.126 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T09:17:23.126 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 185392 2024-04-24T09:17:23.126 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:23.126 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:17:23.126 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2024-04-24T09:17:23.126 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2024-04-24T09:17:23.126 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2024-04-24T09:17:23.126 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2024-04-24T09:17:23.127 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-04-24T09:17:23.127 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-04-24T09:17:23.127 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.000969181, 2024-04-24T09:17:23.127 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T09:17:23.127 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000610698, 2024-04-24T09:17:23.127 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:17:23.127 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:23.127 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:23.127 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:23.128 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:23.128 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:23.128 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:23.128 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:23.128 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:23.128 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:23.128 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:23.128 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:17:23.128 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:17:23.128 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:23.129 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-04-24T09:17:23.129 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-04-24T09:17:23.129 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-04-24T09:17:23.129 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:23.129 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-04-24T09:17:23.129 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:17:23.129 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-04-24T09:17:23.129 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:17:23.130 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:17:23.130 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:17:23.130 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:17:23.130 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:17:23.130 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:17:23.130 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:17:23.130 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:17:23.130 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:17:23.130 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:17:23.131 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:17:23.131 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:17:23.131 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:17:23.131 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-04-24T09:17:23.131 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:17:23.131 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-04-24T09:17:23.131 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:17:23.131 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:17:23.132 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:17:23.132 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:17:23.132 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:17:23.132 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:17:23.132 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:17:23.132 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-04-24T09:17:23.132 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:17:23.132 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:17:23.132 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:17:23.133 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-04-24T09:17:23.133 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-04-24T09:17:23.133 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-04-24T09:17:23.133 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:17:23.133 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:17:23.133 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:17:23.133 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:17:23.133 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148771, 2024-04-24T09:17:23.134 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 32, 2024-04-24T09:17:23.134 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:17:23.134 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-04-24T09:17:23.134 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-04-24T09:17:23.134 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-04-24T09:17:23.134 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:17:23.134 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-04-24T09:17:23.134 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:17:23.135 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-04-24T09:17:23.135 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-04-24T09:17:23.135 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-04-24T09:17:23.135 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:17:23.135 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:17:23.135 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-04-24T09:17:23.135 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-04-24T09:17:23.135 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-04-24T09:17:23.135 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-04-24T09:17:23.135 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-04-24T09:17:23.136 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-04-24T09:17:23.136 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-04-24T09:17:23.136 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-04-24T09:17:23.136 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:23.136 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:17:23.136 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-24T09:17:23.136 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-04-24T09:17:23.136 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-24T09:17:23.136 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 230, 2024-04-24T09:17:23.137 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1559184, 2024-04-24T09:17:23.137 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:23.137 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 230, 2024-04-24T09:17:23.137 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-24T09:17:23.137 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-24T09:17:23.137 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 230, 2024-04-24T09:17:23.137 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1559184, 2024-04-24T09:17:23.137 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-24T09:17:23.138 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:23.138 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:23.138 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:23.138 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:23.138 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:23.138 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:17:23.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:23 smithi081 ceph-mon[30961]: pgmap v320: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-24T09:17:23.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:23 smithi069 ceph-mon[28170]: pgmap v320: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-24T09:17:23.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:23 smithi130 ceph-mon[31553]: pgmap v320: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-24T09:17:24.018 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-24T09:17:24.019 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-24T09:17:24.019 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-24T09:17:24.019 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-24T09:17:24.019 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-24T09:17:24.019 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-24T09:17:24.019 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-24T09:17:24.019 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-24T09:17:24.019 DEBUG:tasks.check_counter:Getting stats from d 2024-04-24T09:17:24.020 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-24T09:17:24.483 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:17:25.690 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:25 smithi069 ceph-mon[28170]: pgmap v321: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 90 B/s wr, 8 op/s 2024-04-24T09:17:25.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:25 smithi081 ceph-mon[30961]: pgmap v321: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 90 B/s wr, 8 op/s 2024-04-24T09:17:25.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:25 smithi130 ceph-mon[31553]: pgmap v321: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 90 B/s wr, 8 op/s 2024-04-24T09:17:26.799 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:17:26.799 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:17:26.799 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2024-04-24T09:17:26.799 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2024-04-24T09:17:26.799 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2024-04-24T09:17:26.799 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2024-04-24T09:17:26.800 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-04-24T09:17:26.800 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-04-24T09:17:26.800 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.003587170, 2024-04-24T09:17:26.800 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T09:17:26.800 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000480605, 2024-04-24T09:17:26.800 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:17:26.800 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:26.800 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:26.800 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:26.801 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:26.801 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:26.801 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:26.801 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:26.801 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:26.801 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:26.801 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:26.801 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:17:26.801 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:17:26.802 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:26.802 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:17:26.802 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 236, 2024-04-24T09:17:26.802 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 225, 2024-04-24T09:17:26.802 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1593504, 2024-04-24T09:17:26.802 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 138784, 2024-04-24T09:17:26.802 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-04-24T09:17:26.802 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-04-24T09:17:26.803 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.099788855, 2024-04-24T09:17:26.803 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.035993084, 2024-04-24T09:17:26.803 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.071011817, 2024-04-24T09:17:26.803 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.011256827, 2024-04-24T09:17:26.803 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:26.803 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 224, 2024-04-24T09:17:26.803 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.023396049, 2024-04-24T09:17:26.803 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000104446 2024-04-24T09:17:26.803 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:26.804 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:26.804 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:26.804 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:26.804 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:26.804 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:26.804 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1593504, 2024-04-24T09:17:26.804 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 138784 2024-04-24T09:17:26.804 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:26.804 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:17:26.805 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 6, 2024-04-24T09:17:26.805 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 174, 2024-04-24T09:17:26.805 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 3312, 2024-04-24T09:17:26.805 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 186896, 2024-04-24T09:17:26.805 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 5, 2024-04-24T09:17:26.805 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-04-24T09:17:26.805 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.035297769, 2024-04-24T09:17:26.805 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.025918982, 2024-04-24T09:17:26.805 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.004376065, 2024-04-24T09:17:26.806 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:17:26.806 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:26.806 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 172, 2024-04-24T09:17:26.806 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.033438265, 2024-04-24T09:17:26.806 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000194408 2024-04-24T09:17:26.806 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:26.806 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:26.806 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:26.807 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:26.807 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:26.807 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:26.807 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-24T09:17:26.807 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 186896 2024-04-24T09:17:26.807 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:26.807 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-04-24T09:17:26.807 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-04-24T09:17:26.808 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-04-24T09:17:26.808 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:26.808 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-04-24T09:17:26.808 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:17:26.808 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-04-24T09:17:26.808 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:17:26.808 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:17:26.808 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:17:26.808 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:17:26.809 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:17:26.809 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:17:26.809 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:17:26.809 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:17:26.809 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:17:26.809 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:17:26.809 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:17:26.809 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:17:26.809 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:17:26.810 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-04-24T09:17:26.810 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:17:26.810 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-04-24T09:17:26.810 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:17:26.810 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:17:26.810 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:17:26.810 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:17:26.810 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:17:26.810 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:17:26.811 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:17:26.811 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-04-24T09:17:26.811 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:17:26.811 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:17:26.811 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:17:26.811 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-04-24T09:17:26.811 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-04-24T09:17:26.811 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-04-24T09:17:26.811 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:17:26.812 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:17:26.812 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:17:26.812 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:17:26.812 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148714, 2024-04-24T09:17:26.812 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 30, 2024-04-24T09:17:26.812 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:17:26.812 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-04-24T09:17:26.812 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-04-24T09:17:26.812 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-04-24T09:17:26.812 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:17:26.813 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-04-24T09:17:26.813 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:17:26.813 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-04-24T09:17:26.813 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-04-24T09:17:26.813 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-04-24T09:17:26.813 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:17:26.813 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:17:26.813 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-04-24T09:17:26.814 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-04-24T09:17:26.814 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-04-24T09:17:26.814 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-04-24T09:17:26.814 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-04-24T09:17:26.814 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-04-24T09:17:26.814 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-04-24T09:17:26.814 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-04-24T09:17:26.814 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:26.814 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:17:26.815 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-24T09:17:26.815 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-04-24T09:17:26.815 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-24T09:17:26.815 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 238, 2024-04-24T09:17:26.815 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1560224, 2024-04-24T09:17:26.815 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:26.815 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 238, 2024-04-24T09:17:26.815 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-24T09:17:26.815 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-24T09:17:26.815 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 238, 2024-04-24T09:17:26.816 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1560224, 2024-04-24T09:17:26.816 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-24T09:17:26.816 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:26.816 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:26.816 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:26.816 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:26.816 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:26.816 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:17:27.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:27 smithi069 ceph-mon[28170]: pgmap v322: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-24T09:17:27.704 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-24T09:17:27.704 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-24T09:17:27.704 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-24T09:17:27.704 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-24T09:17:27.705 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-24T09:17:27.705 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-24T09:17:27.705 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-24T09:17:27.705 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-24T09:17:27.705 DEBUG:tasks.check_counter:Getting stats from g 2024-04-24T09:17:27.705 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-24T09:17:27.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:27 smithi081 ceph-mon[30961]: pgmap v322: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-24T09:17:27.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:27 smithi130 ceph-mon[31553]: pgmap v322: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-24T09:17:28.157 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:17:29.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:29 smithi081 ceph-mon[30961]: pgmap v323: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-24T09:17:29.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:29 smithi069 ceph-mon[28170]: pgmap v323: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-24T09:17:29.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:29 smithi130 ceph-mon[31553]: pgmap v323: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-24T09:17:30.493 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:17:30.494 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:17:30.494 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 235, 2024-04-24T09:17:30.494 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 222, 2024-04-24T09:17:30.494 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1595136, 2024-04-24T09:17:30.494 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 136032, 2024-04-24T09:17:30.494 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-04-24T09:17:30.494 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-04-24T09:17:30.495 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.097466866, 2024-04-24T09:17:30.495 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.035534740, 2024-04-24T09:17:30.495 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.065575882, 2024-04-24T09:17:30.495 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.010124151, 2024-04-24T09:17:30.495 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:30.495 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 222, 2024-04-24T09:17:30.495 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.022067944, 2024-04-24T09:17:30.495 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000099405 2024-04-24T09:17:30.496 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:30.496 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:30.496 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:30.496 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:30.496 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:30.496 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:30.496 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1595136, 2024-04-24T09:17:30.496 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 136032 2024-04-24T09:17:30.497 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:30.497 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:17:30.497 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 2, 2024-04-24T09:17:30.497 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 169, 2024-04-24T09:17:30.497 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 320, 2024-04-24T09:17:30.497 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 185920, 2024-04-24T09:17:30.497 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 5, 2024-04-24T09:17:30.497 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-04-24T09:17:30.497 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.035347629, 2024-04-24T09:17:30.497 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.024838947, 2024-04-24T09:17:30.498 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.004970348, 2024-04-24T09:17:30.498 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:17:30.498 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:30.498 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 166, 2024-04-24T09:17:30.498 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.022713898, 2024-04-24T09:17:30.498 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000136830 2024-04-24T09:17:30.498 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:30.498 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:30.498 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:30.499 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:30.499 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:30.499 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:30.499 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T09:17:30.499 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 185920 2024-04-24T09:17:30.499 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:30.499 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:17:30.499 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2024-04-24T09:17:30.499 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2024-04-24T09:17:30.499 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2024-04-24T09:17:30.500 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2024-04-24T09:17:30.500 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-04-24T09:17:30.500 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-04-24T09:17:30.500 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.000562336, 2024-04-24T09:17:30.500 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T09:17:30.500 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000403390, 2024-04-24T09:17:30.500 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:17:30.501 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:30.501 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:30.501 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:30.501 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:30.501 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:30.501 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:30.501 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:30.501 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:30.502 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:30.502 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:30.502 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:17:30.502 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:17:30.502 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:30.502 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-04-24T09:17:30.502 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-04-24T09:17:30.502 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-04-24T09:17:30.503 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:30.503 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-04-24T09:17:30.503 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:17:30.503 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-04-24T09:17:30.503 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:17:30.503 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:17:30.503 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:17:30.503 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:17:30.504 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:17:30.504 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:17:30.504 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:17:30.504 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:17:30.504 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:17:30.504 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:17:30.504 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:17:30.505 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:17:30.505 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:17:30.505 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-04-24T09:17:30.505 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:17:30.505 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-04-24T09:17:30.505 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:17:30.505 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:17:30.505 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:17:30.505 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:17:30.506 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:17:30.506 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:17:30.506 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:17:30.506 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-04-24T09:17:30.506 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:17:30.506 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:17:30.506 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:17:30.506 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-04-24T09:17:30.507 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-04-24T09:17:30.507 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-04-24T09:17:30.507 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:17:30.507 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:17:30.507 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:17:30.507 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:17:30.507 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148794, 2024-04-24T09:17:30.507 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 32, 2024-04-24T09:17:30.508 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:17:30.508 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-04-24T09:17:30.508 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-04-24T09:17:30.508 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-04-24T09:17:30.508 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:17:30.508 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-04-24T09:17:30.508 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:17:30.508 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-04-24T09:17:30.508 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-04-24T09:17:30.508 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-04-24T09:17:30.509 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:17:30.509 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:17:30.509 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-04-24T09:17:30.509 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-04-24T09:17:30.509 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-04-24T09:17:30.509 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-04-24T09:17:30.509 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-04-24T09:17:30.509 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-04-24T09:17:30.509 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-04-24T09:17:30.510 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-04-24T09:17:30.510 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:30.510 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:17:30.510 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-24T09:17:30.510 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-04-24T09:17:30.510 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-24T09:17:30.510 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 233, 2024-04-24T09:17:30.510 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1559574, 2024-04-24T09:17:30.511 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:30.511 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 233, 2024-04-24T09:17:30.511 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-24T09:17:30.511 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-24T09:17:30.511 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 233, 2024-04-24T09:17:30.511 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1559574, 2024-04-24T09:17:30.511 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-24T09:17:30.512 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:30.512 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:30.512 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:30.512 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:30.512 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:30.512 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:17:31.313 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-24T09:17:31.313 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-24T09:17:31.313 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-24T09:17:31.313 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-24T09:17:31.314 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-24T09:17:31.314 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-24T09:17:31.314 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-24T09:17:31.314 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-24T09:17:31.314 DEBUG:tasks.check_counter:Getting stats from j 2024-04-24T09:17:31.314 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-24T09:17:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:31 smithi069 ceph-mon[28170]: pgmap v324: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-24T09:17:31.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:31 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:17:31.726 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:17:31.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:31 smithi081 ceph-mon[30961]: pgmap v324: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-24T09:17:31.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:31 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:17:31.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:31 smithi130 ceph-mon[31553]: pgmap v324: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-24T09:17:31.953 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:31 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:17:33.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:33 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:33.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:33 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:33.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:33 smithi081 ceph-mon[30961]: pgmap v325: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:33.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:33 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:33.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:33 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:33.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:33 smithi069 ceph-mon[28170]: pgmap v325: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:33.432 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:33 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:33.432 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:33 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:33.432 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:33 smithi130 ceph-mon[31553]: pgmap v325: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:34.205 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-24T09:17:34.205 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:17:34.206 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 239, 2024-04-24T09:17:34.206 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 228, 2024-04-24T09:17:34.206 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1594320, 2024-04-24T09:17:34.206 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 140704, 2024-04-24T09:17:34.206 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-04-24T09:17:34.206 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-04-24T09:17:34.206 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.101956414, 2024-04-24T09:17:34.206 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.035604245, 2024-04-24T09:17:34.207 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.068407461, 2024-04-24T09:17:34.207 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.010954460, 2024-04-24T09:17:34.207 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:34.207 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 227, 2024-04-24T09:17:34.207 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.022672867, 2024-04-24T09:17:34.207 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000099880 2024-04-24T09:17:34.207 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:34.207 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:34.207 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:34.207 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:34.208 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:34.208 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:34.208 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1594320, 2024-04-24T09:17:34.208 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 140704 2024-04-24T09:17:34.208 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:34.208 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:17:34.208 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 2, 2024-04-24T09:17:34.208 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 174, 2024-04-24T09:17:34.209 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 320, 2024-04-24T09:17:34.209 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 186800, 2024-04-24T09:17:34.209 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 5, 2024-04-24T09:17:34.209 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-04-24T09:17:34.209 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.036928721, 2024-04-24T09:17:34.209 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.026619222, 2024-04-24T09:17:34.209 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.004437619, 2024-04-24T09:17:34.209 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:17:34.209 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:34.210 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 171, 2024-04-24T09:17:34.210 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.018387117, 2024-04-24T09:17:34.210 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000107527 2024-04-24T09:17:34.210 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:34.210 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:34.210 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:34.210 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:34.210 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:34.210 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:34.211 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T09:17:34.211 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 186800 2024-04-24T09:17:34.211 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:34.211 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:17:34.211 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 4, 2024-04-24T09:17:34.211 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 4, 2024-04-24T09:17:34.211 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 2992, 2024-04-24T09:17:34.211 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 448, 2024-04-24T09:17:34.211 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-04-24T09:17:34.211 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-04-24T09:17:34.212 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.001773369, 2024-04-24T09:17:34.212 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000091125, 2024-04-24T09:17:34.212 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.001610421, 2024-04-24T09:17:34.212 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:17:34.212 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:34.212 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 3, 2024-04-24T09:17:34.212 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.002011465, 2024-04-24T09:17:34.212 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000670488 2024-04-24T09:17:34.212 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:34.213 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:34.213 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:34.213 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:34.213 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:34.213 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:34.213 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-24T09:17:34.213 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T09:17:34.213 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:34.213 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-04-24T09:17:34.214 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-04-24T09:17:34.214 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-04-24T09:17:34.214 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:34.214 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-04-24T09:17:34.214 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:17:34.214 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-04-24T09:17:34.214 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:17:34.214 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:17:34.214 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:17:34.214 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:17:34.215 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:17:34.215 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:17:34.215 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:17:34.215 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:17:34.215 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:17:34.215 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:17:34.215 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:17:34.215 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:17:34.215 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:17:34.216 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-04-24T09:17:34.216 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:17:34.216 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-04-24T09:17:34.216 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:17:34.216 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:17:34.216 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:17:34.216 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:17:34.216 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:17:34.216 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:17:34.217 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:17:34.217 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-04-24T09:17:34.217 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:17:34.217 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:17:34.217 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:17:34.217 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-04-24T09:17:34.217 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-04-24T09:17:34.217 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-04-24T09:17:34.218 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:17:34.218 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:17:34.218 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:17:34.218 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:17:34.218 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148714, 2024-04-24T09:17:34.218 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 30, 2024-04-24T09:17:34.218 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:17:34.218 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-04-24T09:17:34.218 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-04-24T09:17:34.219 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-04-24T09:17:34.219 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:17:34.219 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-04-24T09:17:34.219 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:17:34.219 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-04-24T09:17:34.219 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-04-24T09:17:34.219 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-04-24T09:17:34.219 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:17:34.219 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:17:34.219 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-04-24T09:17:34.220 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-04-24T09:17:34.220 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-04-24T09:17:34.220 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-04-24T09:17:34.220 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-04-24T09:17:34.220 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-04-24T09:17:34.220 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-04-24T09:17:34.220 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-04-24T09:17:34.220 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-24T09:17:34.220 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:17:34.221 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-24T09:17:34.221 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-04-24T09:17:34.221 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-24T09:17:34.221 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 241, 2024-04-24T09:17:34.221 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1560614, 2024-04-24T09:17:34.221 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:34.221 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 241, 2024-04-24T09:17:34.221 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-24T09:17:34.222 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-24T09:17:34.222 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 241, 2024-04-24T09:17:34.222 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1560614, 2024-04-24T09:17:34.222 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-24T09:17:34.222 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-24T09:17:34.222 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-24T09:17:34.222 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-24T09:17:34.222 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:34.222 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-24T09:17:34.222 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-24T09:17:35.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:35 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:35.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:35 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:35.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:35 smithi081 ceph-mon[30961]: pgmap v326: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:35.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:35 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:35.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:35 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:35.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:35 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:35.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:35 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:35.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:35 smithi069 ceph-mon[28170]: pgmap v326: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:35.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:35 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:35.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:35 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:35.447 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-24T09:17:35.448 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-24T09:17:35.448 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-24T09:17:35.448 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-24T09:17:35.448 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-24T09:17:35.448 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-24T09:17:35.448 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-24T09:17:35.448 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-24T09:17:35.448 DEBUG:tasks.check_counter:Getting stats from b 2024-04-24T09:17:35.449 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-24T09:17:35.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:35 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:35.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:35 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:35.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:35 smithi130 ceph-mon[31553]: pgmap v326: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:35.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:35 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:35.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:35 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:35.899 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:17:36.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:36 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:36.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:36 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:36.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:36 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:36.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:36 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:36.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:36 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi081 to 176.4M 2024-04-24T09:17:36.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:36 smithi081 ceph-mon[30961]: Unable to set osd_memory_target on smithi081 to 185061273: error parsing value: Value '185061273' is below minimum 939524096 2024-04-24T09:17:36.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:36 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:36.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:36 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:36.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:36 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:36.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:36 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:36.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:36 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi081 to 176.4M 2024-04-24T09:17:36.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:36 smithi069 ceph-mon[28170]: Unable to set osd_memory_target on smithi081 to 185061273: error parsing value: Value '185061273' is below minimum 939524096 2024-04-24T09:17:36.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:36 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:36.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:36 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:36.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:36 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:36.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:36 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:36.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:36 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi081 to 176.4M 2024-04-24T09:17:36.455 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:36 smithi130 ceph-mon[31553]: Unable to set osd_memory_target on smithi081 to 185061273: error parsing value: Value '185061273' is below minimum 939524096 2024-04-24T09:17:37.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:37 smithi130 ceph-mon[31553]: pgmap v327: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:37.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:37 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:37.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:37 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:37.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:37 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:37.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:37 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:37.205 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:37 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi130 to 176.4M 2024-04-24T09:17:37.205 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:37 smithi130 ceph-mon[31553]: Unable to set osd_memory_target on smithi130 to 185046937: error parsing value: Value '185046937' is below minimum 939524096 2024-04-24T09:17:37.205 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:37 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:37.205 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:37 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:37.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:37 smithi081 ceph-mon[30961]: pgmap v327: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:37.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:37 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:37.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:37 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:37.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:37 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:37.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:37 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:37.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:37 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi130 to 176.4M 2024-04-24T09:17:37.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:37 smithi081 ceph-mon[30961]: Unable to set osd_memory_target on smithi130 to 185046937: error parsing value: Value '185046937' is below minimum 939524096 2024-04-24T09:17:37.318 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:37 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:37.319 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:37 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:37.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:37 smithi069 ceph-mon[28170]: pgmap v327: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:37.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:37 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:37.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:37 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:37.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:37 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:37.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:37 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:37.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:37 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi130 to 176.4M 2024-04-24T09:17:37.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:37 smithi069 ceph-mon[28170]: Unable to set osd_memory_target on smithi130 to 185046937: error parsing value: Value '185046937' is below minimum 939524096 2024-04-24T09:17:37.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:37 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:37.342 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:37 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:38.285 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-04-24T09:17:38.285 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:17:38.286 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 3640, 2024-04-24T09:17:38.286 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 3377, 2024-04-24T09:17:38.286 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 2773995, 2024-04-24T09:17:38.286 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 13507003, 2024-04-24T09:17:38.287 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 107, 2024-04-24T09:17:38.287 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 6, 2024-04-24T09:17:38.287 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.793962870, 2024-04-24T09:17:38.287 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.273947770, 2024-04-24T09:17:38.287 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.308005878, 2024-04-24T09:17:38.287 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.167950412, 2024-04-24T09:17:38.287 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:38.288 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3373, 2024-04-24T09:17:38.288 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 1.564941969, 2024-04-24T09:17:38.288 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000463961 2024-04-24T09:17:38.288 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.288 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:38.288 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 478, 2024-04-24T09:17:38.288 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000137286, 2024-04-24T09:17:38.289 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000287 2024-04-24T09:17:38.289 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.289 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 1831888, 2024-04-24T09:17:38.289 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 186896 2024-04-24T09:17:38.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.290 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:17:38.290 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 28575, 2024-04-24T09:17:38.290 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 37042, 2024-04-24T09:17:38.290 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 9204566, 2024-04-24T09:17:38.290 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 34712620, 2024-04-24T09:17:38.290 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 10, 2024-04-24T09:17:38.290 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 6, 2024-04-24T09:17:38.291 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 6.633662722, 2024-04-24T09:17:38.291 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 2.970979037, 2024-04-24T09:17:38.291 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 3.978005103, 2024-04-24T09:17:38.291 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.279950983, 2024-04-24T09:17:38.291 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:38.291 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 37043, 2024-04-24T09:17:38.291 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 4.205775850, 2024-04-24T09:17:38.291 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000113537 2024-04-24T09:17:38.291 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.291 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:38.292 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 53758, 2024-04-24T09:17:38.292 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.026941521, 2024-04-24T09:17:38.292 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000501 2024-04-24T09:17:38.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.292 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T09:17:38.292 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 436816 2024-04-24T09:17:38.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.292 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:17:38.292 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 30272, 2024-04-24T09:17:38.293 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 26462, 2024-04-24T09:17:38.293 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 62431023, 2024-04-24T09:17:38.293 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 27872386, 2024-04-24T09:17:38.293 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 30, 2024-04-24T09:17:38.293 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 7, 2024-04-24T09:17:38.293 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 8.410593051, 2024-04-24T09:17:38.293 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 4.268807107, 2024-04-24T09:17:38.293 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 5.040904765, 2024-04-24T09:17:38.293 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.289047336, 2024-04-24T09:17:38.293 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:38.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 26460, 2024-04-24T09:17:38.294 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 5.161498383, 2024-04-24T09:17:38.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000195067 2024-04-24T09:17:38.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.294 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:38.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 47376, 2024-04-24T09:17:38.294 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.023428465, 2024-04-24T09:17:38.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000494 2024-04-24T09:17:38.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.295 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 303072, 2024-04-24T09:17:38.295 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 85600 2024-04-24T09:17:38.295 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.295 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2024-04-24T09:17:38.295 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 1, 2024-04-24T09:17:38.295 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2024-04-24T09:17:38.295 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.295 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-MDSRank": { 2024-04-24T09:17:38.295 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-04-24T09:17:38.296 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-04-24T09:17:38.296 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 4402, 2024-04-24T09:17:38.296 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 11.887772648, 2024-04-24T09:17:38.296 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.002700538 2024-04-24T09:17:38.296 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:38.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.296 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-PurgeQueue": { 2024-04-24T09:17:38.296 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-04-24T09:17:38.296 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-04-24T09:17:38.297 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1987, 2024-04-24T09:17:38.297 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.406638364, 2024-04-24T09:17:38.297 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000204649 2024-04-24T09:17:38.297 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:38.297 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.297 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-04-24T09:17:38.297 INFO:teuthology.orchestra.run.smithi081.stdout: "request": 16800, 2024-04-24T09:17:38.297 INFO:teuthology.orchestra.run.smithi081.stdout: "reply": 10990, 2024-04-24T09:17:38.297 INFO:teuthology.orchestra.run.smithi081.stdout: "reply_latency": { 2024-04-24T09:17:38.297 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 10990, 2024-04-24T09:17:38.298 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 58.525344612, 2024-04-24T09:17:38.298 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.005325327 2024-04-24T09:17:38.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.298 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_reply": 0, 2024-04-24T09:17:38.298 INFO:teuthology.orchestra.run.smithi081.stdout: "forward": 5810, 2024-04-24T09:17:38.298 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_complete": 17, 2024-04-24T09:17:38.298 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_keys": 1, 2024-04-24T09:17:38.298 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_commit": 676, 2024-04-24T09:17:38.299 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_split": 5, 2024-04-24T09:17:38.299 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_merge": 0, 2024-04-24T09:17:38.299 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes": 363, 2024-04-24T09:17:38.299 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_top": 72, 2024-04-24T09:17:38.299 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_bottom": 108, 2024-04-24T09:17:38.299 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pin_tail": 183, 2024-04-24T09:17:38.299 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pinned": 260, 2024-04-24T09:17:38.299 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_expired": 3662, 2024-04-24T09:17:38.299 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_with_caps": 247, 2024-04-24T09:17:38.299 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": 248, 2024-04-24T09:17:38.300 INFO:teuthology.orchestra.run.smithi081.stdout: "subtrees": 88, 2024-04-24T09:17:38.300 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse": 27081, 2024-04-24T09:17:38.300 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_hit": 11920, 2024-04-24T09:17:38.300 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_forward": 5786, 2024-04-24T09:17:38.300 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_discover": 73, 2024-04-24T09:17:38.300 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_dir_fetch": 2, 2024-04-24T09:17:38.300 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_remote_ino": 0, 2024-04-24T09:17:38.300 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_lock": 3, 2024-04-24T09:17:38.301 INFO:teuthology.orchestra.run.smithi081.stdout: "load_cent": 0, 2024-04-24T09:17:38.301 INFO:teuthology.orchestra.run.smithi081.stdout: "q": 0, 2024-04-24T09:17:38.301 INFO:teuthology.orchestra.run.smithi081.stdout: "exported": 99, 2024-04-24T09:17:38.301 INFO:teuthology.orchestra.run.smithi081.stdout: "exported_inodes": 3240, 2024-04-24T09:17:38.301 INFO:teuthology.orchestra.run.smithi081.stdout: "imported": 58, 2024-04-24T09:17:38.301 INFO:teuthology.orchestra.run.smithi081.stdout: "imported_inodes": 2010, 2024-04-24T09:17:38.301 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_dir_fetch": 0, 2024-04-24T09:17:38.301 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_backtrace_fetch": 392, 2024-04-24T09:17:38.301 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_peer_discover": 0, 2024-04-24T09:17:38.302 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rfiles": 3451, 2024-04-24T09:17:38.302 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rbytes": 26543607, 2024-04-24T09:17:38.302 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rsnaps": 6, 2024-04-24T09:17:38.302 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_fetch": 392, 2024-04-24T09:17:38.302 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_set_tag": 0, 2024-04-24T09:17:38.302 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:17:38.302 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:17:38.302 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_inodes": 97, 2024-04-24T09:17:38.302 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_base_inodes": 6, 2024-04-24T09:17:38.303 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dirfrag_rstats": 91, 2024-04-24T09:17:38.303 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_file_inodes": 295, 2024-04-24T09:17:38.303 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:17:38.303 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_revoke": 2286, 2024-04-24T09:17:38.303 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_grant": 2647, 2024-04-24T09:17:38.303 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_trunc": 8, 2024-04-24T09:17:38.303 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:17:38.303 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:17:38.304 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps": 4915, 2024-04-24T09:17:38.304 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps_dirty": 2057, 2024-04-24T09:17:38.304 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_cap_release": 121, 2024-04-24T09:17:38.304 INFO:teuthology.orchestra.run.smithi081.stdout: "process_request_cap_release": 3815 2024-04-24T09:17:38.304 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.304 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache": { 2024-04-24T09:17:38.304 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update": 92, 2024-04-24T09:17:38.304 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update_receipt": 27, 2024-04-24T09:17:38.304 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_try_discover": 16, 2024-04-24T09:17:38.304 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_send_discover": 116, 2024-04-24T09:17:38.305 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_handle_discover": 196, 2024-04-24T09:17:38.305 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays": 0, 2024-04-24T09:17:38.305 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_delayed": 0, 2024-04-24T09:17:38.305 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:17:38.305 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_created": 898, 2024-04-24T09:17:38.305 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_enqueued": 895, 2024-04-24T09:17:38.305 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_reintegrated": 3, 2024-04-24T09:17:38.305 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_migrated": 0, 2024-04-24T09:17:38.305 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_processing": 0, 2024-04-24T09:17:38.306 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:17:38.306 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:17:38.306 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_started": 0, 2024-04-24T09:17:38.306 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_completed": 0, 2024-04-24T09:17:38.306 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:17:38.306 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:17:38.306 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_enqueue_scrub": 6, 2024-04-24T09:17:38.306 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_exportdir": 113, 2024-04-24T09:17:38.307 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_flush": 0, 2024-04-24T09:17:38.307 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragmentdir": 5, 2024-04-24T09:17:38.307 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragstats": 0, 2024-04-24T09:17:38.307 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_inodestats": 0 2024-04-24T09:17:38.307 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.307 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log": { 2024-04-24T09:17:38.307 INFO:teuthology.orchestra.run.smithi081.stdout: "evlrg": 0, 2024-04-24T09:17:38.307 INFO:teuthology.orchestra.run.smithi081.stdout: "evadd": 31023, 2024-04-24T09:17:38.307 INFO:teuthology.orchestra.run.smithi081.stdout: "evex": 30962, 2024-04-24T09:17:38.308 INFO:teuthology.orchestra.run.smithi081.stdout: "evtrm": 30962, 2024-04-24T09:17:38.308 INFO:teuthology.orchestra.run.smithi081.stdout: "ev": 61, 2024-04-24T09:17:38.308 INFO:teuthology.orchestra.run.smithi081.stdout: "evexg": 0, 2024-04-24T09:17:38.308 INFO:teuthology.orchestra.run.smithi081.stdout: "evexd": 83, 2024-04-24T09:17:38.308 INFO:teuthology.orchestra.run.smithi081.stdout: "segadd": 60, 2024-04-24T09:17:38.308 INFO:teuthology.orchestra.run.smithi081.stdout: "segex": 59, 2024-04-24T09:17:38.308 INFO:teuthology.orchestra.run.smithi081.stdout: "segtrm": 59, 2024-04-24T09:17:38.308 INFO:teuthology.orchestra.run.smithi081.stdout: "seg": 1, 2024-04-24T09:17:38.308 INFO:teuthology.orchestra.run.smithi081.stdout: "segmjr": 1, 2024-04-24T09:17:38.308 INFO:teuthology.orchestra.run.smithi081.stdout: "segexg": 0, 2024-04-24T09:17:38.309 INFO:teuthology.orchestra.run.smithi081.stdout: "segexd": 1, 2024-04-24T09:17:38.309 INFO:teuthology.orchestra.run.smithi081.stdout: "expos": 42781044, 2024-04-24T09:17:38.309 INFO:teuthology.orchestra.run.smithi081.stdout: "wrpos": 43261064, 2024-04-24T09:17:38.309 INFO:teuthology.orchestra.run.smithi081.stdout: "rdpos": 0, 2024-04-24T09:17:38.309 INFO:teuthology.orchestra.run.smithi081.stdout: "jlat": { 2024-04-24T09:17:38.309 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1634, 2024-04-24T09:17:38.309 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 12.364115884, 2024-04-24T09:17:38.309 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.007566778 2024-04-24T09:17:38.309 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.310 INFO:teuthology.orchestra.run.smithi081.stdout: "replayed": 0 2024-04-24T09:17:38.310 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.310 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mem": { 2024-04-24T09:17:38.310 INFO:teuthology.orchestra.run.smithi081.stdout: "ino": 345, 2024-04-24T09:17:38.310 INFO:teuthology.orchestra.run.smithi081.stdout: "ino+": 4392, 2024-04-24T09:17:38.310 INFO:teuthology.orchestra.run.smithi081.stdout: "ino-": 4047, 2024-04-24T09:17:38.310 INFO:teuthology.orchestra.run.smithi081.stdout: "dir": 268, 2024-04-24T09:17:38.310 INFO:teuthology.orchestra.run.smithi081.stdout: "dir+": 317, 2024-04-24T09:17:38.311 INFO:teuthology.orchestra.run.smithi081.stdout: "dir-": 49, 2024-04-24T09:17:38.311 INFO:teuthology.orchestra.run.smithi081.stdout: "dn": 363, 2024-04-24T09:17:38.311 INFO:teuthology.orchestra.run.smithi081.stdout: "dn+": 5139, 2024-04-24T09:17:38.311 INFO:teuthology.orchestra.run.smithi081.stdout: "dn-": 4776, 2024-04-24T09:17:38.311 INFO:teuthology.orchestra.run.smithi081.stdout: "cap": 248, 2024-04-24T09:17:38.311 INFO:teuthology.orchestra.run.smithi081.stdout: "cap+": 3313, 2024-04-24T09:17:38.311 INFO:teuthology.orchestra.run.smithi081.stdout: "cap-": 3065, 2024-04-24T09:17:38.311 INFO:teuthology.orchestra.run.smithi081.stdout: "rss": 73348, 2024-04-24T09:17:38.311 INFO:teuthology.orchestra.run.smithi081.stdout: "heap": 231800 2024-04-24T09:17:38.311 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.312 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_server": { 2024-04-24T09:17:38.312 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_client_request": 24253, 2024-04-24T09:17:38.312 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_server_request": 163, 2024-04-24T09:17:38.312 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_request": 16800, 2024-04-24T09:17:38.312 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_session": 141, 2024-04-24T09:17:38.312 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_peer_request": 2600, 2024-04-24T09:17:38.312 INFO:teuthology.orchestra.run.smithi081.stdout: "req_create_latency": { 2024-04-24T09:17:38.312 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1745, 2024-04-24T09:17:38.312 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 9.198278084, 2024-04-24T09:17:38.313 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.005271219 2024-04-24T09:17:38.313 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.313 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getattr_latency": { 2024-04-24T09:17:38.313 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1132, 2024-04-24T09:17:38.313 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.972185402, 2024-04-24T09:17:38.313 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000858821 2024-04-24T09:17:38.313 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.313 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getfilelock_latency": { 2024-04-24T09:17:38.313 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.313 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.314 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.314 INFO:teuthology.orchestra.run.smithi081.stdout: "req_link_latency": { 2024-04-24T09:17:38.314 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3, 2024-04-24T09:17:38.314 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.016261334, 2024-04-24T09:17:38.314 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.005420444 2024-04-24T09:17:38.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.314 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookup_latency": { 2024-04-24T09:17:38.314 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 6826, 2024-04-24T09:17:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 38.308992723, 2024-04-24T09:17:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.005612216 2024-04-24T09:17:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookuphash_latency": { 2024-04-24T09:17:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupino_latency": { 2024-04-24T09:17:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupname_latency": { 2024-04-24T09:17:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupparent_latency": { 2024-04-24T09:17:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupsnap_latency": { 2024-04-24T09:17:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lssnap_latency": { 2024-04-24T09:17:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 10, 2024-04-24T09:17:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.007873406, 2024-04-24T09:17:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000787340 2024-04-24T09:17:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mkdir_latency": { 2024-04-24T09:17:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 70, 2024-04-24T09:17:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.217603064, 2024-04-24T09:17:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.003108615 2024-04-24T09:17:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mknod_latency": { 2024-04-24T09:17:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mksnap_latency": { 2024-04-24T09:17:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 10, 2024-04-24T09:17:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.439260783, 2024-04-24T09:17:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.043926078 2024-04-24T09:17:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "req_open_latency": { 2024-04-24T09:17:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 117, 2024-04-24T09:17:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.229156796, 2024-04-24T09:17:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001958605 2024-04-24T09:17:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: "req_readdir_latency": { 2024-04-24T09:17:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 108, 2024-04-24T09:17:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 1.548916143, 2024-04-24T09:17:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.014341816 2024-04-24T09:17:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rename_latency": { 2024-04-24T09:17:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 77, 2024-04-24T09:17:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.534231920, 2024-04-24T09:17:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.006938076 2024-04-24T09:17:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "req_renamesnap_latency": { 2024-04-24T09:17:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "req_snapdiff_latency": { 2024-04-24T09:17:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmdir_latency": { 2024-04-24T09:17:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 12, 2024-04-24T09:17:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.201421367, 2024-04-24T09:17:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.016785113 2024-04-24T09:17:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmsnap_latency": { 2024-04-24T09:17:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 4, 2024-04-24T09:17:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.051445363, 2024-04-24T09:17:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.012861340 2024-04-24T09:17:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmxattr_latency": { 2024-04-24T09:17:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setattr_latency": { 2024-04-24T09:17:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 8, 2024-04-24T09:17:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.028184187, 2024-04-24T09:17:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.003523023 2024-04-24T09:17:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setdirlayout_latency": { 2024-04-24T09:17:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setfilelock_latency": { 2024-04-24T09:17:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 8, 2024-04-24T09:17:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.009454490, 2024-04-24T09:17:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001181811 2024-04-24T09:17:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setlayout_latency": { 2024-04-24T09:17:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setxattr_latency": { 2024-04-24T09:17:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 8, 2024-04-24T09:17:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.056896350, 2024-04-24T09:17:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.007112043 2024-04-24T09:17:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "req_symlink_latency": { 2024-04-24T09:17:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 7, 2024-04-24T09:17:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.019827987, 2024-04-24T09:17:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.002832569 2024-04-24T09:17:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: "req_unlink_latency": { 2024-04-24T09:17:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 845, 2024-04-24T09:17:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 6.685355213, 2024-04-24T09:17:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.007911662 2024-04-24T09:17:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:17:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:17:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getvxattr_latency": { 2024-04-24T09:17:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessions": { 2024-04-24T09:17:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: "session_count": 2, 2024-04-24T09:17:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: "session_add": 6, 2024-04-24T09:17:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "session_remove": 4, 2024-04-24T09:17:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_open": 2, 2024-04-24T09:17:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_stale": 0, 2024-04-24T09:17:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "total_load": 588, 2024-04-24T09:17:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "average_load": 294, 2024-04-24T09:17:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "avg_session_uptime": 606, 2024-04-24T09:17:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "mdthresh_evicted": 0 2024-04-24T09:17:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2024-04-24T09:17:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 360, 2024-04-24T09:17:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 360, 2024-04-24T09:17:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:17:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:17:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:17:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:17:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:17:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:17:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:17:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:17:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:17:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:17:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:17:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:17:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:17:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2024-04-24T09:17:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:17:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2024-04-24T09:17:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:17:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:17:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:17:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:17:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:17:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:17:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:17:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2024-04-24T09:17:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:17:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:17:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:17:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2024-04-24T09:17:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2024-04-24T09:17:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2024-04-24T09:17:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:17:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:17:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:17:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:17:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 316073, 2024-04-24T09:17:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 126, 2024-04-24T09:17:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:17:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2024-04-24T09:17:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2024-04-24T09:17:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2024-04-24T09:17:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:17:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2024-04-24T09:17:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:17:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2024-04-24T09:17:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 9320, 2024-04-24T09:17:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 178, 2024-04-24T09:17:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:17:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:17:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2024-04-24T09:17:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2024-04-24T09:17:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 1561381, 2024-04-24T09:17:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 9959, 2024-04-24T09:17:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2024-04-24T09:17:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2024-04-24T09:17:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2024-04-24T09:17:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2024-04-24T09:17:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter": { 2024-04-24T09:17:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: "op_active": 0, 2024-04-24T09:17:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: "op_laggy": 0, 2024-04-24T09:17:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send": 11100, 2024-04-24T09:17:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send_bytes": 44200317, 2024-04-24T09:17:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: "op_resend": 32, 2024-04-24T09:17:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: "op_reply": 11100, 2024-04-24T09:17:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: "op_latency": { 2024-04-24T09:17:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 11100, 2024-04-24T09:17:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 496.847631589, 2024-04-24T09:17:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.044761047 2024-04-24T09:17:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "op_inflight": 0, 2024-04-24T09:17:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "oplen_avg": { 2024-04-24T09:17:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 11100, 2024-04-24T09:17:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 20429 2024-04-24T09:17:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "op": 11100, 2024-04-24T09:17:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "op_r": 664, 2024-04-24T09:17:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "op_w": 10436, 2024-04-24T09:17:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "op_rmw": 0, 2024-04-24T09:17:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "op_pg": 0, 2024-04-24T09:17:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_stat": 563, 2024-04-24T09:17:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_create": 5049, 2024-04-24T09:17:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_read": 211, 2024-04-24T09:17:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_write": 1839, 2024-04-24T09:17:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writefull": 257, 2024-04-24T09:17:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writesame": 0, 2024-04-24T09:17:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_append": 0, 2024-04-24T09:17:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_zero": 0, 2024-04-24T09:17:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_truncate": 2, 2024-04-24T09:17:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_delete": 2312, 2024-04-24T09:17:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_mapext": 0, 2024-04-24T09:17:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_sparse_read": 0, 2024-04-24T09:17:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_clonerange": 0, 2024-04-24T09:17:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_getxattr": 410, 2024-04-24T09:17:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_setxattr": 7658, 2024-04-24T09:17:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:17:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_rmxattr": 0, 2024-04-24T09:17:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:17:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_call": 0, 2024-04-24T09:17:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_watch": 0, 2024-04-24T09:17:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_notify": 0, 2024-04-24T09:17:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:17:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls": 0, 2024-04-24T09:17:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:17:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_other": 8, 2024-04-24T09:17:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_active": 0, 2024-04-24T09:17:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_send": 0, 2024-04-24T09:17:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_resend": 0, 2024-04-24T09:17:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_ping": 0, 2024-04-24T09:17:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_active": 0, 2024-04-24T09:17:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_send": 0, 2024-04-24T09:17:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_resend": 0, 2024-04-24T09:17:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_active": 0, 2024-04-24T09:17:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_send": 0, 2024-04-24T09:17:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_resend": 0, 2024-04-24T09:17:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_active": 0, 2024-04-24T09:17:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_send": 0, 2024-04-24T09:17:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_resend": 0, 2024-04-24T09:17:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "command_active": 0, 2024-04-24T09:17:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "command_send": 0, 2024-04-24T09:17:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "command_resend": 0, 2024-04-24T09:17:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "map_epoch": 101, 2024-04-24T09:17:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "map_full": 0, 2024-04-24T09:17:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "map_inc": 16, 2024-04-24T09:17:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_sessions": 12, 2024-04-24T09:17:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_open": 12, 2024-04-24T09:17:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_close": 0, 2024-04-24T09:17:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_laggy": 0, 2024-04-24T09:17:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_wr": 1736, 2024-04-24T09:17:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_rd": 73, 2024-04-24T09:17:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_del": 311 2024-04-24T09:17:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "oft": { 2024-04-24T09:17:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_objs": 1, 2024-04-24T09:17:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:17:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_updates": 0, 2024-04-24T09:17:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_removes": 0 2024-04-24T09:17:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: "osd-slow-ops": { 2024-04-24T09:17:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_ops_count": 0 2024-04-24T09:17:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: "purge_queue": { 2024-04-24T09:17:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops": 0, 2024-04-24T09:17:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops_high_water": 17, 2024-04-24T09:17:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing": 0, 2024-04-24T09:17:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_high_water": 10, 2024-04-24T09:17:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed_ops": 1779, 2024-04-24T09:17:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed": 895, 2024-04-24T09:17:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_item_in_journal": 0 2024-04-24T09:17:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:17:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-04-24T09:17:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 62483, 2024-04-24T09:17:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 68721139, 2024-04-24T09:17:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 62483, 2024-04-24T09:17:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-04-24T09:17:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-04-24T09:17:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 62483, 2024-04-24T09:17:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 68721139, 2024-04-24T09:17:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_bytes": { 2024-04-24T09:17:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-04-24T09:17:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-04-24T09:17:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-04-24T09:17:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-04-24T09:17:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 11100, 2024-04-24T09:17:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 44300871, 2024-04-24T09:17:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 11100, 2024-04-24T09:17:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 44300871, 2024-04-24T09:17:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_ops": { 2024-04-24T09:17:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 1024, 2024-04-24T09:17:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-04-24T09:17:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-04-24T09:17:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-04-24T09:17:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 11100, 2024-04-24T09:17:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 11100, 2024-04-24T09:17:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 11100, 2024-04-24T09:17:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 11100, 2024-04-24T09:17:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:17:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-04-24T09:17:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 895, 2024-04-24T09:17:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 102091, 2024-04-24T09:17:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 895, 2024-04-24T09:17:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-04-24T09:17:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-04-24T09:17:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 205, 2024-04-24T09:17:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 102091, 2024-04-24T09:17:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle-0x55d3c5886500": { 2024-04-24T09:17:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-04-24T09:17:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 31023, 2024-04-24T09:17:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 39066760, 2024-04-24T09:17:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 31023, 2024-04-24T09:17:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-04-24T09:17:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-04-24T09:17:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 1634, 2024-04-24T09:17:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 39066760, 2024-04-24T09:17:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:38.359 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-04-24T09:17:38.359 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:38 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:38.359 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:38 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:38.359 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:38 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:38.359 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:38 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:38.360 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:38 smithi081 ceph-mon[30961]: Adjusting osd_memory_target on smithi069 to 176.4M 2024-04-24T09:17:38.360 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:38 smithi081 ceph-mon[30961]: Unable to set osd_memory_target on smithi069 to 185003929: error parsing value: Value '185003929' is below minimum 939524096 2024-04-24T09:17:38.360 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:38 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:38.360 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:38 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:38.360 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:38 smithi081 ceph-mon[30961]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:38.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:38 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:38.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:38 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:38.361 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:38 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:38.361 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:38 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:38.361 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:38 smithi069 ceph-mon[28170]: Adjusting osd_memory_target on smithi069 to 176.4M 2024-04-24T09:17:38.361 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:38 smithi069 ceph-mon[28170]: Unable to set osd_memory_target on smithi069 to 185003929: error parsing value: Value '185003929' is below minimum 939524096 2024-04-24T09:17:38.361 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:38 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:38.361 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:38 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:38.361 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:38 smithi069 ceph-mon[28170]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:38.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:38 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:38.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:38 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:38.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:38 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:38.455 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:38 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:38.455 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:38 smithi130 ceph-mon[31553]: Adjusting osd_memory_target on smithi069 to 176.4M 2024-04-24T09:17:38.455 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:38 smithi130 ceph-mon[31553]: Unable to set osd_memory_target on smithi069 to 185003929: error parsing value: Value '185003929' is below minimum 939524096 2024-04-24T09:17:38.455 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:38 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:38.455 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:38 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:38.455 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:38 smithi130 ceph-mon[31553]: from='mgr.15465 172.21.15.69:0/3306601631' entity='mgr.x' 2024-04-24T09:17:39.219 INFO:tasks.check_counter:Daemon mds.b mds.exported=99 2024-04-24T09:17:39.220 INFO:tasks.check_counter:Daemon mds.b mds.imported=58 2024-04-24T09:17:39.220 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=92 2024-04-24T09:17:39.220 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=27 2024-04-24T09:17:39.220 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-24T09:17:39.220 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-24T09:17:39.220 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-24T09:17:39.220 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=5 2024-04-24T09:17:39.220 DEBUG:tasks.check_counter:Getting stats from e 2024-04-24T09:17:39.221 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-24T09:17:39.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:39 smithi069 ceph-mon[28170]: pgmap v328: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T09:17:39.453 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:39 smithi130 ceph-mon[31553]: pgmap v328: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T09:17:39.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:39 smithi081 ceph-mon[30961]: pgmap v328: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T09:17:39.639 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:17:41.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:41 smithi081 ceph-mon[30961]: pgmap v329: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:41.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:41 smithi069 ceph-mon[28170]: pgmap v329: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:41.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:41 smithi130 ceph-mon[31553]: pgmap v329: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:41.937 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-04-24T09:17:41.937 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:17:41.937 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 11502, 2024-04-24T09:17:41.937 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 15678, 2024-04-24T09:17:41.937 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 3584711, 2024-04-24T09:17:41.938 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 22603816, 2024-04-24T09:17:41.938 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 24, 2024-04-24T09:17:41.938 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 4, 2024-04-24T09:17:41.938 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 2.821112675, 2024-04-24T09:17:41.938 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 1.282211314, 2024-04-24T09:17:41.938 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 1.609546657, 2024-04-24T09:17:41.938 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.120577846, 2024-04-24T09:17:41.938 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:41.939 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 15677, 2024-04-24T09:17:41.939 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 1.321228948, 2024-04-24T09:17:41.939 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000084278 2024-04-24T09:17:41.939 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.939 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:41.939 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 21656, 2024-04-24T09:17:41.939 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.010742306, 2024-04-24T09:17:41.939 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000496 2024-04-24T09:17:41.939 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.940 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 4576, 2024-04-24T09:17:41.940 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 1888 2024-04-24T09:17:41.940 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.940 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:17:41.940 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 25200, 2024-04-24T09:17:41.940 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 22849, 2024-04-24T09:17:41.940 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 61470734, 2024-04-24T09:17:41.940 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 26898811, 2024-04-24T09:17:41.940 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 9, 2024-04-24T09:17:41.941 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 8, 2024-04-24T09:17:41.941 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 7.615069989, 2024-04-24T09:17:41.941 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 3.917601193, 2024-04-24T09:17:41.941 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 4.721028354, 2024-04-24T09:17:41.941 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.125509355, 2024-04-24T09:17:41.941 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:41.941 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 22848, 2024-04-24T09:17:41.941 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 2.046438093, 2024-04-24T09:17:41.941 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000089567 2024-04-24T09:17:41.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.942 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:41.942 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 43893, 2024-04-24T09:17:41.942 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.019275802, 2024-04-24T09:17:41.942 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000439 2024-04-24T09:17:41.942 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.942 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 1674496, 2024-04-24T09:17:41.942 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 199648 2024-04-24T09:17:41.942 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.942 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:17:41.943 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 4188, 2024-04-24T09:17:41.943 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 3798, 2024-04-24T09:17:41.943 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 1749692, 2024-04-24T09:17:41.943 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 5956113, 2024-04-24T09:17:41.943 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 13, 2024-04-24T09:17:41.943 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 5, 2024-04-24T09:17:41.943 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 1.128949984, 2024-04-24T09:17:41.943 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.580797019, 2024-04-24T09:17:41.943 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.540008326, 2024-04-24T09:17:41.944 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.088367310, 2024-04-24T09:17:41.944 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:41.944 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3795, 2024-04-24T09:17:41.944 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.464373949, 2024-04-24T09:17:41.944 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000122364 2024-04-24T09:17:41.944 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.944 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:41.944 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 5115, 2024-04-24T09:17:41.945 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.002254768, 2024-04-24T09:17:41.945 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000440 2024-04-24T09:17:41.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.945 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 3456, 2024-04-24T09:17:41.945 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 405872 2024-04-24T09:17:41.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.945 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2024-04-24T09:17:41.945 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 1, 2024-04-24T09:17:41.945 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2024-04-24T09:17:41.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.946 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-MDSRank": { 2024-04-24T09:17:41.946 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-04-24T09:17:41.946 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-04-24T09:17:41.946 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3952, 2024-04-24T09:17:41.946 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 8.257333568, 2024-04-24T09:17:41.946 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.002089406 2024-04-24T09:17:41.946 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:41.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.947 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-PurgeQueue": { 2024-04-24T09:17:41.947 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-04-24T09:17:41.947 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-04-24T09:17:41.947 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2282, 2024-04-24T09:17:41.947 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.511061524, 2024-04-24T09:17:41.947 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000223953 2024-04-24T09:17:41.947 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:41.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.948 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-04-24T09:17:41.948 INFO:teuthology.orchestra.run.smithi081.stdout: "request": 7111, 2024-04-24T09:17:41.948 INFO:teuthology.orchestra.run.smithi081.stdout: "reply": 6544, 2024-04-24T09:17:41.948 INFO:teuthology.orchestra.run.smithi081.stdout: "reply_latency": { 2024-04-24T09:17:41.948 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 6544, 2024-04-24T09:17:41.948 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 52.795909127, 2024-04-24T09:17:41.948 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.008067834 2024-04-24T09:17:41.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.948 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_reply": 0, 2024-04-24T09:17:41.949 INFO:teuthology.orchestra.run.smithi081.stdout: "forward": 567, 2024-04-24T09:17:41.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_complete": 0, 2024-04-24T09:17:41.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_keys": 0, 2024-04-24T09:17:41.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_commit": 43, 2024-04-24T09:17:41.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_split": 4, 2024-04-24T09:17:41.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_merge": 0, 2024-04-24T09:17:41.949 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes": 2784, 2024-04-24T09:17:41.950 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_top": 216, 2024-04-24T09:17:41.950 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_bottom": 2568, 2024-04-24T09:17:41.950 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pin_tail": 0, 2024-04-24T09:17:41.950 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pinned": 2475, 2024-04-24T09:17:41.950 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_expired": 8, 2024-04-24T09:17:41.950 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_with_caps": 1217, 2024-04-24T09:17:41.950 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": 1217, 2024-04-24T09:17:41.950 INFO:teuthology.orchestra.run.smithi081.stdout: "subtrees": 83, 2024-04-24T09:17:41.950 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse": 17113, 2024-04-24T09:17:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_hit": 8640, 2024-04-24T09:17:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_forward": 564, 2024-04-24T09:17:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_discover": 65, 2024-04-24T09:17:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:17:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_remote_ino": 0, 2024-04-24T09:17:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_lock": 0, 2024-04-24T09:17:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "load_cent": 0, 2024-04-24T09:17:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "q": 0, 2024-04-24T09:17:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "exported": 1, 2024-04-24T09:17:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "exported_inodes": 0, 2024-04-24T09:17:41.952 INFO:teuthology.orchestra.run.smithi081.stdout: "imported": 52, 2024-04-24T09:17:41.952 INFO:teuthology.orchestra.run.smithi081.stdout: "imported_inodes": 1623, 2024-04-24T09:17:41.952 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_dir_fetch": 0, 2024-04-24T09:17:41.952 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_backtrace_fetch": 147, 2024-04-24T09:17:41.952 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_peer_discover": 0, 2024-04-24T09:17:41.952 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rfiles": 3443, 2024-04-24T09:17:41.952 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rbytes": 26005079, 2024-04-24T09:17:41.952 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rsnaps": 6, 2024-04-24T09:17:41.952 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_fetch": 147, 2024-04-24T09:17:41.953 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_set_tag": 0, 2024-04-24T09:17:41.953 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:17:41.953 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:17:41.953 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_inodes": 18, 2024-04-24T09:17:41.953 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:17:41.953 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dirfrag_rstats": 18, 2024-04-24T09:17:41.953 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_file_inodes": 129, 2024-04-24T09:17:41.953 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:17:41.953 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_revoke": 1125, 2024-04-24T09:17:41.954 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_grant": 1656, 2024-04-24T09:17:41.954 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:17:41.954 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:17:41.954 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:17:41.954 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps": 2466, 2024-04-24T09:17:41.954 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps_dirty": 1095, 2024-04-24T09:17:41.954 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_cap_release": 147, 2024-04-24T09:17:41.954 INFO:teuthology.orchestra.run.smithi081.stdout: "process_request_cap_release": 2492 2024-04-24T09:17:41.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.955 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache": { 2024-04-24T09:17:41.955 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update": 25, 2024-04-24T09:17:41.955 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update_receipt": 321, 2024-04-24T09:17:41.955 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_try_discover": 169, 2024-04-24T09:17:41.955 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_send_discover": 289, 2024-04-24T09:17:41.955 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_handle_discover": 76, 2024-04-24T09:17:41.955 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays": 1, 2024-04-24T09:17:41.955 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_delayed": 0, 2024-04-24T09:17:41.955 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:17:41.956 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_created": 833, 2024-04-24T09:17:41.956 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_enqueued": 832, 2024-04-24T09:17:41.956 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_reintegrated": 0, 2024-04-24T09:17:41.956 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_migrated": 0, 2024-04-24T09:17:41.956 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_processing": 0, 2024-04-24T09:17:41.956 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:17:41.956 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:17:41.956 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_started": 0, 2024-04-24T09:17:41.956 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_completed": 0, 2024-04-24T09:17:41.956 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:17:41.957 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:17:41.957 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:17:41.957 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_exportdir": 1, 2024-04-24T09:17:41.957 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_flush": 0, 2024-04-24T09:17:41.957 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragmentdir": 4, 2024-04-24T09:17:41.957 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragstats": 0, 2024-04-24T09:17:41.957 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_inodestats": 0 2024-04-24T09:17:41.957 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.957 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log": { 2024-04-24T09:17:41.958 INFO:teuthology.orchestra.run.smithi081.stdout: "evlrg": 0, 2024-04-24T09:17:41.958 INFO:teuthology.orchestra.run.smithi081.stdout: "evadd": 44537, 2024-04-24T09:17:41.958 INFO:teuthology.orchestra.run.smithi081.stdout: "evex": 0, 2024-04-24T09:17:41.958 INFO:teuthology.orchestra.run.smithi081.stdout: "evtrm": 0, 2024-04-24T09:17:41.958 INFO:teuthology.orchestra.run.smithi081.stdout: "ev": 44537, 2024-04-24T09:17:41.958 INFO:teuthology.orchestra.run.smithi081.stdout: "evexg": 0, 2024-04-24T09:17:41.958 INFO:teuthology.orchestra.run.smithi081.stdout: "evexd": 0, 2024-04-24T09:17:41.958 INFO:teuthology.orchestra.run.smithi081.stdout: "segadd": 52, 2024-04-24T09:17:41.958 INFO:teuthology.orchestra.run.smithi081.stdout: "segex": 0, 2024-04-24T09:17:41.958 INFO:teuthology.orchestra.run.smithi081.stdout: "segtrm": 0, 2024-04-24T09:17:41.959 INFO:teuthology.orchestra.run.smithi081.stdout: "seg": 52, 2024-04-24T09:17:41.959 INFO:teuthology.orchestra.run.smithi081.stdout: "segmjr": 5, 2024-04-24T09:17:41.959 INFO:teuthology.orchestra.run.smithi081.stdout: "segexg": 0, 2024-04-24T09:17:41.959 INFO:teuthology.orchestra.run.smithi081.stdout: "segexd": 0, 2024-04-24T09:17:41.959 INFO:teuthology.orchestra.run.smithi081.stdout: "expos": 4194304, 2024-04-24T09:17:41.959 INFO:teuthology.orchestra.run.smithi081.stdout: "wrpos": 35415409, 2024-04-24T09:17:41.959 INFO:teuthology.orchestra.run.smithi081.stdout: "rdpos": 0, 2024-04-24T09:17:41.959 INFO:teuthology.orchestra.run.smithi081.stdout: "jlat": { 2024-04-24T09:17:41.959 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1784, 2024-04-24T09:17:41.959 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 9.601778478, 2024-04-24T09:17:41.960 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.005382162 2024-04-24T09:17:41.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.960 INFO:teuthology.orchestra.run.smithi081.stdout: "replayed": 0 2024-04-24T09:17:41.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.960 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mem": { 2024-04-24T09:17:41.960 INFO:teuthology.orchestra.run.smithi081.stdout: "ino": 2348, 2024-04-24T09:17:41.960 INFO:teuthology.orchestra.run.smithi081.stdout: "ino+": 3185, 2024-04-24T09:17:41.960 INFO:teuthology.orchestra.run.smithi081.stdout: "ino-": 837, 2024-04-24T09:17:41.960 INFO:teuthology.orchestra.run.smithi081.stdout: "dir": 257, 2024-04-24T09:17:41.961 INFO:teuthology.orchestra.run.smithi081.stdout: "dir+": 274, 2024-04-24T09:17:41.961 INFO:teuthology.orchestra.run.smithi081.stdout: "dir-": 17, 2024-04-24T09:17:41.961 INFO:teuthology.orchestra.run.smithi081.stdout: "dn": 2784, 2024-04-24T09:17:41.961 INFO:teuthology.orchestra.run.smithi081.stdout: "dn+": 3648, 2024-04-24T09:17:41.961 INFO:teuthology.orchestra.run.smithi081.stdout: "dn-": 864, 2024-04-24T09:17:41.961 INFO:teuthology.orchestra.run.smithi081.stdout: "cap": 1217, 2024-04-24T09:17:41.961 INFO:teuthology.orchestra.run.smithi081.stdout: "cap+": 2051, 2024-04-24T09:17:41.961 INFO:teuthology.orchestra.run.smithi081.stdout: "cap-": 834, 2024-04-24T09:17:41.961 INFO:teuthology.orchestra.run.smithi081.stdout: "rss": 69396, 2024-04-24T09:17:41.962 INFO:teuthology.orchestra.run.smithi081.stdout: "heap": 223608 2024-04-24T09:17:41.962 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.962 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_server": { 2024-04-24T09:17:41.962 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_client_request": 17032, 2024-04-24T09:17:41.962 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_server_request": 3, 2024-04-24T09:17:41.962 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_request": 7111, 2024-04-24T09:17:41.962 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_session": 21, 2024-04-24T09:17:41.962 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_peer_request": 3243, 2024-04-24T09:17:41.963 INFO:teuthology.orchestra.run.smithi081.stdout: "req_create_latency": { 2024-04-24T09:17:41.963 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 817, 2024-04-24T09:17:41.963 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 8.110657913, 2024-04-24T09:17:41.963 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.009927365 2024-04-24T09:17:41.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.963 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getattr_latency": { 2024-04-24T09:17:41.963 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 12, 2024-04-24T09:17:41.963 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.007757912, 2024-04-24T09:17:41.963 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000646492 2024-04-24T09:17:41.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.964 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getfilelock_latency": { 2024-04-24T09:17:41.964 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.964 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.964 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.964 INFO:teuthology.orchestra.run.smithi081.stdout: "req_link_latency": { 2024-04-24T09:17:41.964 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.964 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.964 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.965 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.965 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookup_latency": { 2024-04-24T09:17:41.965 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 4859, 2024-04-24T09:17:41.965 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 35.540650753, 2024-04-24T09:17:41.965 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.007314396 2024-04-24T09:17:41.965 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.965 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookuphash_latency": { 2024-04-24T09:17:41.965 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.965 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.965 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.966 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.966 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupino_latency": { 2024-04-24T09:17:41.966 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.966 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.966 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.966 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.966 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupname_latency": { 2024-04-24T09:17:41.966 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.966 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.967 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.967 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.967 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupparent_latency": { 2024-04-24T09:17:41.967 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.967 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.967 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.967 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.967 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupsnap_latency": { 2024-04-24T09:17:41.967 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.968 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.968 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.968 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.968 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lssnap_latency": { 2024-04-24T09:17:41.968 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.968 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.969 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.969 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.969 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mkdir_latency": { 2024-04-24T09:17:41.969 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1, 2024-04-24T09:17:41.969 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.007136684, 2024-04-24T09:17:41.969 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.007136684 2024-04-24T09:17:41.969 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.969 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mknod_latency": { 2024-04-24T09:17:41.969 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.970 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.970 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.970 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mksnap_latency": { 2024-04-24T09:17:41.970 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.970 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.970 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.970 INFO:teuthology.orchestra.run.smithi081.stdout: "req_open_latency": { 2024-04-24T09:17:41.971 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 4, 2024-04-24T09:17:41.971 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.005313746, 2024-04-24T09:17:41.971 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001328436 2024-04-24T09:17:41.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.971 INFO:teuthology.orchestra.run.smithi081.stdout: "req_readdir_latency": { 2024-04-24T09:17:41.971 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 17, 2024-04-24T09:17:41.971 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.037653601, 2024-04-24T09:17:41.971 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.002214917 2024-04-24T09:17:41.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.972 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rename_latency": { 2024-04-24T09:17:41.972 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3, 2024-04-24T09:17:41.972 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.030075507, 2024-04-24T09:17:41.972 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.010025169 2024-04-24T09:17:41.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.972 INFO:teuthology.orchestra.run.smithi081.stdout: "req_renamesnap_latency": { 2024-04-24T09:17:41.972 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.972 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.972 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.973 INFO:teuthology.orchestra.run.smithi081.stdout: "req_snapdiff_latency": { 2024-04-24T09:17:41.973 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.973 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.973 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.973 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmdir_latency": { 2024-04-24T09:17:41.973 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1, 2024-04-24T09:17:41.973 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.015250490, 2024-04-24T09:17:41.973 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.015250490 2024-04-24T09:17:41.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.974 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmsnap_latency": { 2024-04-24T09:17:41.974 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.974 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.974 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.974 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.974 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmxattr_latency": { 2024-04-24T09:17:41.974 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.974 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.974 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.975 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.975 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setattr_latency": { 2024-04-24T09:17:41.975 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.975 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.975 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.975 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.975 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setdirlayout_latency": { 2024-04-24T09:17:41.975 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.975 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.975 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.976 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.976 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setfilelock_latency": { 2024-04-24T09:17:41.976 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.976 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.976 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.976 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.976 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setlayout_latency": { 2024-04-24T09:17:41.976 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.976 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.977 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.977 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setxattr_latency": { 2024-04-24T09:17:41.977 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.977 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.977 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.977 INFO:teuthology.orchestra.run.smithi081.stdout: "req_symlink_latency": { 2024-04-24T09:17:41.977 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.977 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.978 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.978 INFO:teuthology.orchestra.run.smithi081.stdout: "req_unlink_latency": { 2024-04-24T09:17:41.978 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 830, 2024-04-24T09:17:41.978 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 9.041412521, 2024-04-24T09:17:41.978 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.010893268 2024-04-24T09:17:41.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.978 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:17:41.978 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:17:41.979 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getvxattr_latency": { 2024-04-24T09:17:41.979 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.979 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.979 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.979 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:41.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.979 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessions": { 2024-04-24T09:17:41.979 INFO:teuthology.orchestra.run.smithi081.stdout: "session_count": 2, 2024-04-24T09:17:41.980 INFO:teuthology.orchestra.run.smithi081.stdout: "session_add": 3, 2024-04-24T09:17:41.980 INFO:teuthology.orchestra.run.smithi081.stdout: "session_remove": 1, 2024-04-24T09:17:41.980 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_open": 2, 2024-04-24T09:17:41.980 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_stale": 0, 2024-04-24T09:17:41.980 INFO:teuthology.orchestra.run.smithi081.stdout: "total_load": 1547, 2024-04-24T09:17:41.980 INFO:teuthology.orchestra.run.smithi081.stdout: "average_load": 773, 2024-04-24T09:17:41.980 INFO:teuthology.orchestra.run.smithi081.stdout: "avg_session_uptime": 607, 2024-04-24T09:17:41.980 INFO:teuthology.orchestra.run.smithi081.stdout: "mdthresh_evicted": 0 2024-04-24T09:17:41.980 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.981 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2024-04-24T09:17:41.981 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:17:41.981 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 0, 2024-04-24T09:17:41.981 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:17:41.981 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:17:41.981 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:17:41.981 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:17:41.981 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:17:41.981 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:17:41.981 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:17:41.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:17:41.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:17:41.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:17:41.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:17:41.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:17:41.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:17:41.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2024-04-24T09:17:41.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:17:41.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2024-04-24T09:17:41.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:17:41.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:17:41.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:17:41.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:17:41.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:17:41.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:17:41.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:17:41.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2024-04-24T09:17:41.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:17:41.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:17:41.984 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:17:41.984 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2024-04-24T09:17:41.984 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2024-04-24T09:17:41.984 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2024-04-24T09:17:41.984 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:17:41.984 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:17:41.984 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:17:41.984 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:17:41.984 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 315874, 2024-04-24T09:17:41.984 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 126, 2024-04-24T09:17:41.985 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:17:41.985 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2024-04-24T09:17:41.985 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2024-04-24T09:17:41.985 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2024-04-24T09:17:41.985 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:17:41.985 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2024-04-24T09:17:41.985 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:17:41.985 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2024-04-24T09:17:41.985 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 9320, 2024-04-24T09:17:41.986 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 178, 2024-04-24T09:17:41.986 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:17:41.986 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:17:41.986 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2024-04-24T09:17:41.986 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2024-04-24T09:17:41.986 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 7918111, 2024-04-24T09:17:41.986 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 45819, 2024-04-24T09:17:41.986 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2024-04-24T09:17:41.986 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2024-04-24T09:17:41.987 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2024-04-24T09:17:41.987 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2024-04-24T09:17:41.987 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.987 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter": { 2024-04-24T09:17:41.987 INFO:teuthology.orchestra.run.smithi081.stdout: "op_active": 0, 2024-04-24T09:17:41.987 INFO:teuthology.orchestra.run.smithi081.stdout: "op_laggy": 0, 2024-04-24T09:17:41.987 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send": 4641, 2024-04-24T09:17:41.987 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send_bytes": 31688305, 2024-04-24T09:17:41.987 INFO:teuthology.orchestra.run.smithi081.stdout: "op_resend": 30, 2024-04-24T09:17:41.988 INFO:teuthology.orchestra.run.smithi081.stdout: "op_reply": 4641, 2024-04-24T09:17:41.988 INFO:teuthology.orchestra.run.smithi081.stdout: "op_latency": { 2024-04-24T09:17:41.988 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 4641, 2024-04-24T09:17:41.988 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 34.651346492, 2024-04-24T09:17:41.988 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.007466353 2024-04-24T09:17:41.988 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.988 INFO:teuthology.orchestra.run.smithi081.stdout: "op_inflight": 0, 2024-04-24T09:17:41.988 INFO:teuthology.orchestra.run.smithi081.stdout: "oplen_avg": { 2024-04-24T09:17:41.988 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 4641, 2024-04-24T09:17:41.988 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 5065 2024-04-24T09:17:41.989 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.989 INFO:teuthology.orchestra.run.smithi081.stdout: "op": 4641, 2024-04-24T09:17:41.989 INFO:teuthology.orchestra.run.smithi081.stdout: "op_r": 436, 2024-04-24T09:17:41.989 INFO:teuthology.orchestra.run.smithi081.stdout: "op_w": 4205, 2024-04-24T09:17:41.989 INFO:teuthology.orchestra.run.smithi081.stdout: "op_rmw": 0, 2024-04-24T09:17:41.989 INFO:teuthology.orchestra.run.smithi081.stdout: "op_pg": 0, 2024-04-24T09:17:41.989 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_stat": 0, 2024-04-24T09:17:41.989 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_create": 10, 2024-04-24T09:17:41.989 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_read": 289, 2024-04-24T09:17:41.990 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_write": 2073, 2024-04-24T09:17:41.990 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writefull": 177, 2024-04-24T09:17:41.990 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writesame": 0, 2024-04-24T09:17:41.990 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_append": 0, 2024-04-24T09:17:41.990 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_zero": 0, 2024-04-24T09:17:41.990 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_truncate": 0, 2024-04-24T09:17:41.990 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_delete": 1687, 2024-04-24T09:17:41.990 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_mapext": 0, 2024-04-24T09:17:41.991 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_sparse_read": 0, 2024-04-24T09:17:41.991 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_clonerange": 0, 2024-04-24T09:17:41.991 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_getxattr": 147, 2024-04-24T09:17:41.991 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_setxattr": 20, 2024-04-24T09:17:41.991 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:17:41.991 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_rmxattr": 0, 2024-04-24T09:17:41.991 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:17:41.991 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_call": 0, 2024-04-24T09:17:41.991 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_watch": 0, 2024-04-24T09:17:41.991 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_notify": 0, 2024-04-24T09:17:41.992 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:17:41.992 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls": 0, 2024-04-24T09:17:41.992 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:17:41.992 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_other": 0, 2024-04-24T09:17:41.992 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_active": 0, 2024-04-24T09:17:41.992 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_send": 0, 2024-04-24T09:17:41.992 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_resend": 0, 2024-04-24T09:17:41.992 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_ping": 0, 2024-04-24T09:17:41.992 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_active": 0, 2024-04-24T09:17:41.992 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_send": 0, 2024-04-24T09:17:41.993 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_resend": 0, 2024-04-24T09:17:41.993 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_active": 0, 2024-04-24T09:17:41.993 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_send": 0, 2024-04-24T09:17:41.993 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_resend": 0, 2024-04-24T09:17:41.993 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_active": 0, 2024-04-24T09:17:41.993 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_send": 0, 2024-04-24T09:17:41.993 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_resend": 0, 2024-04-24T09:17:41.993 INFO:teuthology.orchestra.run.smithi081.stdout: "command_active": 0, 2024-04-24T09:17:41.993 INFO:teuthology.orchestra.run.smithi081.stdout: "command_send": 0, 2024-04-24T09:17:41.994 INFO:teuthology.orchestra.run.smithi081.stdout: "command_resend": 0, 2024-04-24T09:17:41.994 INFO:teuthology.orchestra.run.smithi081.stdout: "map_epoch": 101, 2024-04-24T09:17:41.994 INFO:teuthology.orchestra.run.smithi081.stdout: "map_full": 0, 2024-04-24T09:17:41.994 INFO:teuthology.orchestra.run.smithi081.stdout: "map_inc": 9, 2024-04-24T09:17:41.994 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_sessions": 12, 2024-04-24T09:17:41.994 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_open": 12, 2024-04-24T09:17:41.994 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_close": 0, 2024-04-24T09:17:41.994 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_laggy": 0, 2024-04-24T09:17:41.994 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_wr": 473, 2024-04-24T09:17:41.995 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_rd": 0, 2024-04-24T09:17:41.995 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_del": 189 2024-04-24T09:17:41.995 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.995 INFO:teuthology.orchestra.run.smithi081.stdout: "oft": { 2024-04-24T09:17:41.995 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_objs": 1, 2024-04-24T09:17:41.995 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:17:41.995 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_updates": 0, 2024-04-24T09:17:41.995 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_removes": 0 2024-04-24T09:17:41.995 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.995 INFO:teuthology.orchestra.run.smithi081.stdout: "osd-slow-ops": { 2024-04-24T09:17:41.996 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_ops_count": 0 2024-04-24T09:17:41.996 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.996 INFO:teuthology.orchestra.run.smithi081.stdout: "purge_queue": { 2024-04-24T09:17:41.996 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops": 0, 2024-04-24T09:17:41.996 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops_high_water": 16, 2024-04-24T09:17:41.996 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing": 0, 2024-04-24T09:17:41.996 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_high_water": 8, 2024-04-24T09:17:41.996 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed_ops": 1664, 2024-04-24T09:17:41.996 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed": 832, 2024-04-24T09:17:41.997 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_item_in_journal": 0 2024-04-24T09:17:41.997 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.997 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:17:41.997 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:41.997 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-04-24T09:17:41.997 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:41.997 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 40886, 2024-04-24T09:17:41.997 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 63106580, 2024-04-24T09:17:41.997 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:41.997 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 40886, 2024-04-24T09:17:41.998 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-04-24T09:17:41.998 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-04-24T09:17:41.998 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 40886, 2024-04-24T09:17:41.998 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 63106580, 2024-04-24T09:17:41.998 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:41.998 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:41.998 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:41.998 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:41.999 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:41.999 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:41.999 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_bytes": { 2024-04-24T09:17:41.999 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:41.999 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-04-24T09:17:41.999 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:41.999 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-04-24T09:17:41.999 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-04-24T09:17:41.999 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:41.999 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-04-24T09:17:42.000 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 4641, 2024-04-24T09:17:42.000 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 31791889, 2024-04-24T09:17:42.000 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 4641, 2024-04-24T09:17:42.000 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 31791889, 2024-04-24T09:17:42.000 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:42.000 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:42.000 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:42.000 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:42.000 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:42.000 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:42.001 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_ops": { 2024-04-24T09:17:42.001 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:42.001 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 1024, 2024-04-24T09:17:42.001 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:42.001 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-04-24T09:17:42.001 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-04-24T09:17:42.001 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:42.001 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-04-24T09:17:42.001 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 4641, 2024-04-24T09:17:42.002 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 4641, 2024-04-24T09:17:42.002 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 4641, 2024-04-24T09:17:42.002 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 4641, 2024-04-24T09:17:42.002 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:42.002 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:42.002 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:42.002 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:42.002 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:42.003 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:42.003 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:17:42.003 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:42.003 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-04-24T09:17:42.003 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:42.003 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 832, 2024-04-24T09:17:42.003 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 103584, 2024-04-24T09:17:42.003 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:42.003 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 832, 2024-04-24T09:17:42.003 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-04-24T09:17:42.004 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-04-24T09:17:42.004 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 289, 2024-04-24T09:17:42.004 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 103584, 2024-04-24T09:17:42.004 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:42.004 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:42.004 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:42.004 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:42.004 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:42.004 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:42.004 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle-0x5589e9d2a500": { 2024-04-24T09:17:42.005 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:42.005 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-04-24T09:17:42.005 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:42.005 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 44537, 2024-04-24T09:17:42.005 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 31221105, 2024-04-24T09:17:42.005 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:42.005 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 44537, 2024-04-24T09:17:42.005 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-04-24T09:17:42.005 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-04-24T09:17:42.005 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 1784, 2024-04-24T09:17:42.006 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 31221105, 2024-04-24T09:17:42.006 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:42.006 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:42.006 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:42.006 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:42.006 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:42.006 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:42.006 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-04-24T09:17:42.834 INFO:tasks.check_counter:Daemon mds.e mds.exported=1 2024-04-24T09:17:42.835 INFO:tasks.check_counter:Daemon mds.e mds.imported=52 2024-04-24T09:17:42.835 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=25 2024-04-24T09:17:42.835 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=321 2024-04-24T09:17:42.835 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-04-24T09:17:42.835 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-24T09:17:42.835 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T09:17:42.835 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=4 2024-04-24T09:17:42.835 DEBUG:tasks.check_counter:Getting stats from h 2024-04-24T09:17:42.836 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-24T09:17:43.271 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:17:43.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:43 smithi081 ceph-mon[30961]: pgmap v330: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T09:17:43.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:43 smithi069 ceph-mon[28170]: pgmap v330: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T09:17:43.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:43 smithi130 ceph-mon[31553]: pgmap v330: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T09:17:45.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:45 smithi081 ceph-mon[30961]: pgmap v331: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T09:17:45.580 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-04-24T09:17:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:17:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 23796, 2024-04-24T09:17:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 38634, 2024-04-24T09:17:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 7517958, 2024-04-24T09:17:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 42112655, 2024-04-24T09:17:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 17, 2024-04-24T09:17:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 6, 2024-04-24T09:17:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 5.862168879, 2024-04-24T09:17:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 2.664051553, 2024-04-24T09:17:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 3.418404211, 2024-04-24T09:17:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.180256581, 2024-04-24T09:17:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 38628, 2024-04-24T09:17:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 2.854184803, 2024-04-24T09:17:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000073889 2024-04-24T09:17:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 50266, 2024-04-24T09:17:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.029409189, 2024-04-24T09:17:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000585 2024-04-24T09:17:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T09:17:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 319072 2024-04-24T09:17:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:17:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 24640, 2024-04-24T09:17:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 27887, 2024-04-24T09:17:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 13801137, 2024-04-24T09:17:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 74961563, 2024-04-24T09:17:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 12, 2024-04-24T09:17:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 8, 2024-04-24T09:17:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 7.966281333, 2024-04-24T09:17:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 4.191495202, 2024-04-24T09:17:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 4.509297721, 2024-04-24T09:17:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.146943796, 2024-04-24T09:17:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 27888, 2024-04-24T09:17:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 2.456723842, 2024-04-24T09:17:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000088092 2024-04-24T09:17:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 46863, 2024-04-24T09:17:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.023650728, 2024-04-24T09:17:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000504 2024-04-24T09:17:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 1679712, 2024-04-24T09:17:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 233920 2024-04-24T09:17:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:17:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 26802, 2024-04-24T09:17:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 29172, 2024-04-24T09:17:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 15047220, 2024-04-24T09:17:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 80193808, 2024-04-24T09:17:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 18, 2024-04-24T09:17:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 6, 2024-04-24T09:17:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 7.872872156, 2024-04-24T09:17:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 4.041476400, 2024-04-24T09:17:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 4.393767944, 2024-04-24T09:17:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.275515395, 2024-04-24T09:17:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 29172, 2024-04-24T09:17:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 2.450421286, 2024-04-24T09:17:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000083999 2024-04-24T09:17:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 44170, 2024-04-24T09:17:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.021895307, 2024-04-24T09:17:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000495 2024-04-24T09:17:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 6256, 2024-04-24T09:17:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 112176 2024-04-24T09:17:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2024-04-24T09:17:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 1, 2024-04-24T09:17:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2024-04-24T09:17:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-MDSRank": { 2024-04-24T09:17:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-04-24T09:17:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-04-24T09:17:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 12303, 2024-04-24T09:17:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 25.877432856, 2024-04-24T09:17:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.002103343 2024-04-24T09:17:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-PurgeQueue": { 2024-04-24T09:17:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-04-24T09:17:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-04-24T09:17:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3045, 2024-04-24T09:17:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.791004160, 2024-04-24T09:17:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000259771 2024-04-24T09:17:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-04-24T09:17:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: "request": 14311, 2024-04-24T09:17:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: "reply": 12588, 2024-04-24T09:17:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: "reply_latency": { 2024-04-24T09:17:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 12588, 2024-04-24T09:17:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 70.302967346, 2024-04-24T09:17:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.005584919 2024-04-24T09:17:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_reply": 0, 2024-04-24T09:17:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "forward": 1723, 2024-04-24T09:17:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_complete": 6, 2024-04-24T09:17:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_keys": 0, 2024-04-24T09:17:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_commit": 183, 2024-04-24T09:17:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_split": 21, 2024-04-24T09:17:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_merge": 0, 2024-04-24T09:17:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes": 4169, 2024-04-24T09:17:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_top": 494, 2024-04-24T09:17:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_bottom": 2454, 2024-04-24T09:17:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pin_tail": 1221, 2024-04-24T09:17:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pinned": 3428, 2024-04-24T09:17:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_expired": 3539, 2024-04-24T09:17:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_with_caps": 2095, 2024-04-24T09:17:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": 2095, 2024-04-24T09:17:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: "subtrees": 87, 2024-04-24T09:17:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse": 24066, 2024-04-24T09:17:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_hit": 13483, 2024-04-24T09:17:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_forward": 1723, 2024-04-24T09:17:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_discover": 47, 2024-04-24T09:17:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_dir_fetch": 5, 2024-04-24T09:17:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_remote_ino": 0, 2024-04-24T09:17:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_lock": 0, 2024-04-24T09:17:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "load_cent": 0, 2024-04-24T09:17:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "q": 0, 2024-04-24T09:17:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "exported": 96, 2024-04-24T09:17:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "exported_inodes": 3260, 2024-04-24T09:17:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "imported": 86, 2024-04-24T09:17:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "imported_inodes": 2867, 2024-04-24T09:17:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_dir_fetch": 0, 2024-04-24T09:17:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_backtrace_fetch": 2283, 2024-04-24T09:17:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_peer_discover": 0, 2024-04-24T09:17:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rfiles": 3443, 2024-04-24T09:17:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rbytes": 26005079, 2024-04-24T09:17:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rsnaps": 6, 2024-04-24T09:17:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_fetch": 2283, 2024-04-24T09:17:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_set_tag": 0, 2024-04-24T09:17:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:17:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:17:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_inodes": 6, 2024-04-24T09:17:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:17:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dirfrag_rstats": 6, 2024-04-24T09:17:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_file_inodes": 2277, 2024-04-24T09:17:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:17:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_revoke": 789, 2024-04-24T09:17:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_grant": 5638, 2024-04-24T09:17:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:17:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:17:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:17:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps": 5755, 2024-04-24T09:17:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps_dirty": 4192, 2024-04-24T09:17:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_cap_release": 146, 2024-04-24T09:17:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: "process_request_cap_release": 7254 2024-04-24T09:17:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache": { 2024-04-24T09:17:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update": 92, 2024-04-24T09:17:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update_receipt": 119, 2024-04-24T09:17:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_try_discover": 73, 2024-04-24T09:17:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_send_discover": 117, 2024-04-24T09:17:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_handle_discover": 252, 2024-04-24T09:17:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays": 3, 2024-04-24T09:17:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_delayed": 0, 2024-04-24T09:17:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:17:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_created": 1516, 2024-04-24T09:17:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_enqueued": 1513, 2024-04-24T09:17:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_reintegrated": 0, 2024-04-24T09:17:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_migrated": 0, 2024-04-24T09:17:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_processing": 0, 2024-04-24T09:17:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:17:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:17:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_started": 0, 2024-04-24T09:17:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_completed": 0, 2024-04-24T09:17:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:17:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:17:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:17:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_exportdir": 118, 2024-04-24T09:17:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_flush": 0, 2024-04-24T09:17:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragmentdir": 21, 2024-04-24T09:17:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragstats": 0, 2024-04-24T09:17:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_inodestats": 0 2024-04-24T09:17:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log": { 2024-04-24T09:17:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: "evlrg": 0, 2024-04-24T09:17:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "evadd": 14046, 2024-04-24T09:17:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "evex": 0, 2024-04-24T09:17:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "evtrm": 0, 2024-04-24T09:17:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "ev": 14046, 2024-04-24T09:17:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "evexg": 0, 2024-04-24T09:17:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "evexd": 0, 2024-04-24T09:17:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "segadd": 21, 2024-04-24T09:17:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "segex": 0, 2024-04-24T09:17:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: "segtrm": 0, 2024-04-24T09:17:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: "seg": 21, 2024-04-24T09:17:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: "segmjr": 3, 2024-04-24T09:17:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: "segexg": 0, 2024-04-24T09:17:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: "segexd": 0, 2024-04-24T09:17:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: "expos": 4194304, 2024-04-24T09:17:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: "wrpos": 59489353, 2024-04-24T09:17:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: "rdpos": 0, 2024-04-24T09:17:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: "jlat": { 2024-04-24T09:17:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 5172, 2024-04-24T09:17:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 27.388099093, 2024-04-24T09:17:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.005295456 2024-04-24T09:17:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: "replayed": 0 2024-04-24T09:17:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mem": { 2024-04-24T09:17:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: "ino": 3609, 2024-04-24T09:17:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: "ino+": 8210, 2024-04-24T09:17:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "ino-": 4601, 2024-04-24T09:17:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "dir": 265, 2024-04-24T09:17:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "dir+": 343, 2024-04-24T09:17:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "dir-": 78, 2024-04-24T09:17:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "dn": 4169, 2024-04-24T09:17:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "dn+": 9234, 2024-04-24T09:17:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "dn-": 5065, 2024-04-24T09:17:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "cap": 2095, 2024-04-24T09:17:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "cap+": 6050, 2024-04-24T09:17:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: "cap-": 3955, 2024-04-24T09:17:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: "rss": 81096, 2024-04-24T09:17:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: "heap": 223608 2024-04-24T09:17:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_server": { 2024-04-24T09:17:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_client_request": 19088, 2024-04-24T09:17:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_server_request": 5419, 2024-04-24T09:17:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_request": 14311, 2024-04-24T09:17:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_session": 120, 2024-04-24T09:17:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_peer_request": 10886, 2024-04-24T09:17:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: "req_create_latency": { 2024-04-24T09:17:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 4116, 2024-04-24T09:17:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 20.983698662, 2024-04-24T09:17:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.005098080 2024-04-24T09:17:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getattr_latency": { 2024-04-24T09:17:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 6, 2024-04-24T09:17:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.006028017, 2024-04-24T09:17:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001004669 2024-04-24T09:17:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getfilelock_latency": { 2024-04-24T09:17:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: "req_link_latency": { 2024-04-24T09:17:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookup_latency": { 2024-04-24T09:17:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 6918, 2024-04-24T09:17:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 32.090783325, 2024-04-24T09:17:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.004638737 2024-04-24T09:17:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookuphash_latency": { 2024-04-24T09:17:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupino_latency": { 2024-04-24T09:17:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.611 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupname_latency": { 2024-04-24T09:17:45.611 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.611 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.611 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.611 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.611 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupparent_latency": { 2024-04-24T09:17:45.611 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.611 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.611 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.612 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupsnap_latency": { 2024-04-24T09:17:45.612 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.612 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.613 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:45 smithi069 ceph-mon[28170]: pgmap v331: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T09:17:45.613 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.613 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lssnap_latency": { 2024-04-24T09:17:45.613 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.613 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.614 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.614 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mkdir_latency": { 2024-04-24T09:17:45.614 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 14, 2024-04-24T09:17:45.614 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.078576474, 2024-04-24T09:17:45.614 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.005612605 2024-04-24T09:17:45.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.614 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mknod_latency": { 2024-04-24T09:17:45.614 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.615 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.615 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.615 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mksnap_latency": { 2024-04-24T09:17:45.615 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.615 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.615 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.616 INFO:teuthology.orchestra.run.smithi081.stdout: "req_open_latency": { 2024-04-24T09:17:45.616 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1, 2024-04-24T09:17:45.616 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000922164, 2024-04-24T09:17:45.616 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000922164 2024-04-24T09:17:45.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.616 INFO:teuthology.orchestra.run.smithi081.stdout: "req_readdir_latency": { 2024-04-24T09:17:45.616 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 9, 2024-04-24T09:17:45.616 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.025522387, 2024-04-24T09:17:45.616 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.002835820 2024-04-24T09:17:45.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.617 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rename_latency": { 2024-04-24T09:17:45.617 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2, 2024-04-24T09:17:45.617 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 5.046993285, 2024-04-24T09:17:45.617 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 2.523496642 2024-04-24T09:17:45.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.617 INFO:teuthology.orchestra.run.smithi081.stdout: "req_renamesnap_latency": { 2024-04-24T09:17:45.617 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.617 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.617 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.618 INFO:teuthology.orchestra.run.smithi081.stdout: "req_snapdiff_latency": { 2024-04-24T09:17:45.618 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.618 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.618 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.618 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmdir_latency": { 2024-04-24T09:17:45.618 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 9, 2024-04-24T09:17:45.619 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.415700613, 2024-04-24T09:17:45.619 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.046188957 2024-04-24T09:17:45.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.619 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmsnap_latency": { 2024-04-24T09:17:45.619 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.619 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.619 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.619 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmxattr_latency": { 2024-04-24T09:17:45.620 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.620 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.620 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.620 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setattr_latency": { 2024-04-24T09:17:45.620 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.620 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.620 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.620 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setdirlayout_latency": { 2024-04-24T09:17:45.621 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.621 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.621 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.621 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setfilelock_latency": { 2024-04-24T09:17:45.621 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.621 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.621 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.621 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setlayout_latency": { 2024-04-24T09:17:45.622 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.622 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.622 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.622 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setxattr_latency": { 2024-04-24T09:17:45.622 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.622 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.622 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.623 INFO:teuthology.orchestra.run.smithi081.stdout: "req_symlink_latency": { 2024-04-24T09:17:45.623 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.623 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.623 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.623 INFO:teuthology.orchestra.run.smithi081.stdout: "req_unlink_latency": { 2024-04-24T09:17:45.623 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1513, 2024-04-24T09:17:45.623 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 11.654742419, 2024-04-24T09:17:45.623 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.007703068 2024-04-24T09:17:45.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.624 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:17:45.624 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:17:45.624 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getvxattr_latency": { 2024-04-24T09:17:45.624 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.624 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.624 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.624 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:45.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.624 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessions": { 2024-04-24T09:17:45.624 INFO:teuthology.orchestra.run.smithi081.stdout: "session_count": 2, 2024-04-24T09:17:45.625 INFO:teuthology.orchestra.run.smithi081.stdout: "session_add": 4, 2024-04-24T09:17:45.625 INFO:teuthology.orchestra.run.smithi081.stdout: "session_remove": 2, 2024-04-24T09:17:45.625 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_open": 2, 2024-04-24T09:17:45.625 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_stale": 0, 2024-04-24T09:17:45.625 INFO:teuthology.orchestra.run.smithi081.stdout: "total_load": 1682, 2024-04-24T09:17:45.625 INFO:teuthology.orchestra.run.smithi081.stdout: "average_load": 841, 2024-04-24T09:17:45.625 INFO:teuthology.orchestra.run.smithi081.stdout: "avg_session_uptime": 610, 2024-04-24T09:17:45.625 INFO:teuthology.orchestra.run.smithi081.stdout: "mdthresh_evicted": 0 2024-04-24T09:17:45.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.625 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2024-04-24T09:17:45.626 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:17:45.626 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 0, 2024-04-24T09:17:45.626 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:17:45.626 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:17:45.626 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:17:45.626 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:17:45.626 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:17:45.626 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:17:45.627 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:17:45.627 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:17:45.627 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:17:45.627 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:17:45.627 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:17:45.627 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:17:45.627 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:17:45.627 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2024-04-24T09:17:45.627 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:17:45.627 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2024-04-24T09:17:45.628 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:17:45.628 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:17:45.628 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:17:45.628 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:17:45.628 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:17:45.628 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:17:45.628 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:17:45.628 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2024-04-24T09:17:45.628 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:17:45.629 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:17:45.629 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:17:45.629 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2024-04-24T09:17:45.629 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2024-04-24T09:17:45.629 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2024-04-24T09:17:45.629 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:17:45.629 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:17:45.629 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:17:45.629 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:17:45.629 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 316664, 2024-04-24T09:17:45.630 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 128, 2024-04-24T09:17:45.630 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:17:45.630 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2024-04-24T09:17:45.630 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2024-04-24T09:17:45.630 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2024-04-24T09:17:45.630 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:17:45.630 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2024-04-24T09:17:45.630 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:17:45.630 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2024-04-24T09:17:45.631 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 9320, 2024-04-24T09:17:45.631 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 178, 2024-04-24T09:17:45.631 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:17:45.631 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:17:45.631 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2024-04-24T09:17:45.631 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2024-04-24T09:17:45.631 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 11932782, 2024-04-24T09:17:45.631 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 61419, 2024-04-24T09:17:45.631 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2024-04-24T09:17:45.632 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2024-04-24T09:17:45.632 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2024-04-24T09:17:45.632 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2024-04-24T09:17:45.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.632 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter": { 2024-04-24T09:17:45.632 INFO:teuthology.orchestra.run.smithi081.stdout: "op_active": 0, 2024-04-24T09:17:45.632 INFO:teuthology.orchestra.run.smithi081.stdout: "op_laggy": 0, 2024-04-24T09:17:45.632 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send": 11685, 2024-04-24T09:17:45.632 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send_bytes": 57622625, 2024-04-24T09:17:45.632 INFO:teuthology.orchestra.run.smithi081.stdout: "op_resend": 33, 2024-04-24T09:17:45.633 INFO:teuthology.orchestra.run.smithi081.stdout: "op_reply": 11685, 2024-04-24T09:17:45.633 INFO:teuthology.orchestra.run.smithi081.stdout: "op_latency": { 2024-04-24T09:17:45.633 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 11685, 2024-04-24T09:17:45.633 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 65.971755876, 2024-04-24T09:17:45.633 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.005645849 2024-04-24T09:17:45.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.633 INFO:teuthology.orchestra.run.smithi081.stdout: "op_inflight": 0, 2024-04-24T09:17:45.633 INFO:teuthology.orchestra.run.smithi081.stdout: "oplen_avg": { 2024-04-24T09:17:45.633 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 11685, 2024-04-24T09:17:45.634 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 12310 2024-04-24T09:17:45.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.634 INFO:teuthology.orchestra.run.smithi081.stdout: "op": 11685, 2024-04-24T09:17:45.634 INFO:teuthology.orchestra.run.smithi081.stdout: "op_r": 2572, 2024-04-24T09:17:45.634 INFO:teuthology.orchestra.run.smithi081.stdout: "op_w": 9113, 2024-04-24T09:17:45.634 INFO:teuthology.orchestra.run.smithi081.stdout: "op_rmw": 0, 2024-04-24T09:17:45.634 INFO:teuthology.orchestra.run.smithi081.stdout: "op_pg": 0, 2024-04-24T09:17:45.634 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_stat": 0, 2024-04-24T09:17:45.634 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_create": 10, 2024-04-24T09:17:45.635 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_read": 275, 2024-04-24T09:17:45.635 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_write": 5447, 2024-04-24T09:17:45.635 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writefull": 171, 2024-04-24T09:17:45.635 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writesame": 0, 2024-04-24T09:17:45.635 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_append": 0, 2024-04-24T09:17:45.635 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_zero": 0, 2024-04-24T09:17:45.635 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_truncate": 4, 2024-04-24T09:17:45.635 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_delete": 3068, 2024-04-24T09:17:45.635 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_mapext": 0, 2024-04-24T09:17:45.635 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_sparse_read": 0, 2024-04-24T09:17:45.636 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_clonerange": 0, 2024-04-24T09:17:45.636 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_getxattr": 2285, 2024-04-24T09:17:45.636 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_setxattr": 20, 2024-04-24T09:17:45.636 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:17:45.636 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_rmxattr": 0, 2024-04-24T09:17:45.636 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:17:45.636 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_call": 0, 2024-04-24T09:17:45.636 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_watch": 0, 2024-04-24T09:17:45.636 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_notify": 0, 2024-04-24T09:17:45.636 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:17:45.637 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls": 0, 2024-04-24T09:17:45.637 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:17:45.637 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_other": 0, 2024-04-24T09:17:45.637 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_active": 0, 2024-04-24T09:17:45.637 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_send": 0, 2024-04-24T09:17:45.637 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_resend": 0, 2024-04-24T09:17:45.637 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_ping": 0, 2024-04-24T09:17:45.637 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_active": 0, 2024-04-24T09:17:45.637 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_send": 0, 2024-04-24T09:17:45.638 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_resend": 0, 2024-04-24T09:17:45.638 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_active": 0, 2024-04-24T09:17:45.638 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_send": 0, 2024-04-24T09:17:45.638 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_resend": 0, 2024-04-24T09:17:45.638 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_active": 0, 2024-04-24T09:17:45.638 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_send": 0, 2024-04-24T09:17:45.638 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_resend": 0, 2024-04-24T09:17:45.638 INFO:teuthology.orchestra.run.smithi081.stdout: "command_active": 0, 2024-04-24T09:17:45.638 INFO:teuthology.orchestra.run.smithi081.stdout: "command_send": 0, 2024-04-24T09:17:45.638 INFO:teuthology.orchestra.run.smithi081.stdout: "command_resend": 0, 2024-04-24T09:17:45.639 INFO:teuthology.orchestra.run.smithi081.stdout: "map_epoch": 101, 2024-04-24T09:17:45.639 INFO:teuthology.orchestra.run.smithi081.stdout: "map_full": 0, 2024-04-24T09:17:45.639 INFO:teuthology.orchestra.run.smithi081.stdout: "map_inc": 9, 2024-04-24T09:17:45.639 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_sessions": 12, 2024-04-24T09:17:45.639 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_open": 12, 2024-04-24T09:17:45.639 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_close": 0, 2024-04-24T09:17:45.639 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_laggy": 0, 2024-04-24T09:17:45.639 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_wr": 796, 2024-04-24T09:17:45.640 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_rd": 20, 2024-04-24T09:17:45.640 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_del": 214 2024-04-24T09:17:45.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.640 INFO:teuthology.orchestra.run.smithi081.stdout: "oft": { 2024-04-24T09:17:45.640 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_objs": 1, 2024-04-24T09:17:45.640 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:17:45.640 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_updates": 0, 2024-04-24T09:17:45.640 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_removes": 0 2024-04-24T09:17:45.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.640 INFO:teuthology.orchestra.run.smithi081.stdout: "osd-slow-ops": { 2024-04-24T09:17:45.641 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_ops_count": 0 2024-04-24T09:17:45.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.641 INFO:teuthology.orchestra.run.smithi081.stdout: "purge_queue": { 2024-04-24T09:17:45.641 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops": 0, 2024-04-24T09:17:45.641 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops_high_water": 16, 2024-04-24T09:17:45.641 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing": 0, 2024-04-24T09:17:45.641 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_high_water": 8, 2024-04-24T09:17:45.641 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed_ops": 3026, 2024-04-24T09:17:45.641 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed": 1513, 2024-04-24T09:17:45.642 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_item_in_journal": 0 2024-04-24T09:17:45.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.642 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:17:45.642 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:45.642 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-04-24T09:17:45.642 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:45.642 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 75234, 2024-04-24T09:17:45.642 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 29576326, 2024-04-24T09:17:45.642 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:45.643 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 75234, 2024-04-24T09:17:45.643 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-04-24T09:17:45.643 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-04-24T09:17:45.643 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 75234, 2024-04-24T09:17:45.643 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 29576326, 2024-04-24T09:17:45.643 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:45.643 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.643 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.643 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.643 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:45.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.644 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_bytes": { 2024-04-24T09:17:45.644 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:45.644 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-04-24T09:17:45.644 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:45.644 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-04-24T09:17:45.644 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-04-24T09:17:45.644 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:45.644 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-04-24T09:17:45.645 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 11685, 2024-04-24T09:17:45.645 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 57810398, 2024-04-24T09:17:45.645 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 11685, 2024-04-24T09:17:45.645 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 57810398, 2024-04-24T09:17:45.645 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:45.645 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.645 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.645 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.645 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:45.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.646 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_ops": { 2024-04-24T09:17:45.646 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:45.646 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 1024, 2024-04-24T09:17:45.646 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:45.646 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-04-24T09:17:45.646 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-04-24T09:17:45.646 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:45.646 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-04-24T09:17:45.646 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 11685, 2024-04-24T09:17:45.646 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 11685, 2024-04-24T09:17:45.647 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 11685, 2024-04-24T09:17:45.647 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 11685, 2024-04-24T09:17:45.647 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:45.647 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.647 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.647 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.647 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:45.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.648 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:17:45.648 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:45.648 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-04-24T09:17:45.648 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:45.648 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 1513, 2024-04-24T09:17:45.648 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 188101, 2024-04-24T09:17:45.648 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:45.648 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 1513, 2024-04-24T09:17:45.648 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-04-24T09:17:45.649 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-04-24T09:17:45.649 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 275, 2024-04-24T09:17:45.649 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 188101, 2024-04-24T09:17:45.649 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:45.649 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.649 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.649 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.649 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:45.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:45.649 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle-0x55a84fb1e500": { 2024-04-24T09:17:45.650 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:45.650 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-04-24T09:17:45.650 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:45.650 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 14046, 2024-04-24T09:17:45.650 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 55295049, 2024-04-24T09:17:45.650 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:45.650 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 14046, 2024-04-24T09:17:45.650 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-04-24T09:17:45.650 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-04-24T09:17:45.651 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 5172, 2024-04-24T09:17:45.651 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 55295049, 2024-04-24T09:17:45.651 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:45.651 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:45.651 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:45.651 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:45.651 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:45.651 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:45.651 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-04-24T09:17:45.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:45 smithi130 ceph-mon[31553]: pgmap v331: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T09:17:46.543 INFO:tasks.check_counter:Daemon mds.h mds.exported=96 2024-04-24T09:17:46.543 INFO:tasks.check_counter:Daemon mds.h mds.imported=86 2024-04-24T09:17:46.543 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=92 2024-04-24T09:17:46.543 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=119 2024-04-24T09:17:46.543 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-24T09:17:46.543 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-24T09:17:46.544 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T09:17:46.544 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=21 2024-04-24T09:17:46.544 DEBUG:tasks.check_counter:Getting stats from k 2024-04-24T09:17:46.544 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-24T09:17:46.993 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:17:47.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:47 smithi130 ceph-mon[31553]: pgmap v332: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:47.516 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:47 smithi081 ceph-mon[30961]: pgmap v332: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:47.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:47 smithi069 ceph-mon[28170]: pgmap v332: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:49.253 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-04-24T09:17:49.253 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:17:49.253 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 1853, 2024-04-24T09:17:49.253 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 1853, 2024-04-24T09:17:49.253 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 14440855, 2024-04-24T09:17:49.253 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 588440, 2024-04-24T09:17:49.254 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 7, 2024-04-24T09:17:49.254 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 3, 2024-04-24T09:17:49.254 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.461374905, 2024-04-24T09:17:49.254 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.173020264, 2024-04-24T09:17:49.254 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.142409267, 2024-04-24T09:17:49.254 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.106780644, 2024-04-24T09:17:49.254 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:49.254 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1852, 2024-04-24T09:17:49.254 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.168196123, 2024-04-24T09:17:49.255 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000090818 2024-04-24T09:17:49.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.255 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:49.255 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.255 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.255 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.255 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-24T09:17:49.255 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T09:17:49.256 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.256 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:17:49.256 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 543, 2024-04-24T09:17:49.256 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 521, 2024-04-24T09:17:49.256 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 18512817, 2024-04-24T09:17:49.256 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 233202, 2024-04-24T09:17:49.256 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 4, 2024-04-24T09:17:49.256 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 3, 2024-04-24T09:17:49.256 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.231513198, 2024-04-24T09:17:49.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.053161587, 2024-04-24T09:17:49.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.175976056, 2024-04-24T09:17:49.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.029611659, 2024-04-24T09:17:49.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:49.257 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 520, 2024-04-24T09:17:49.257 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.055081083, 2024-04-24T09:17:49.257 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000105925 2024-04-24T09:17:49.257 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:49.258 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.258 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.258 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.258 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 1666224, 2024-04-24T09:17:49.258 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 146640 2024-04-24T09:17:49.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.258 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:17:49.258 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 539, 2024-04-24T09:17:49.259 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 715, 2024-04-24T09:17:49.259 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 8523243, 2024-04-24T09:17:49.259 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 566159, 2024-04-24T09:17:49.259 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 7, 2024-04-24T09:17:49.259 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 5, 2024-04-24T09:17:49.259 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.181256054, 2024-04-24T09:17:49.259 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.055516023, 2024-04-24T09:17:49.259 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.069330059, 2024-04-24T09:17:49.259 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.036544745, 2024-04-24T09:17:49.259 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:49.260 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 712, 2024-04-24T09:17:49.260 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.075575580, 2024-04-24T09:17:49.260 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000106145 2024-04-24T09:17:49.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.260 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:49.260 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.260 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.260 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.261 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T09:17:49.261 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 395392 2024-04-24T09:17:49.261 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.261 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2024-04-24T09:17:49.261 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 1, 2024-04-24T09:17:49.261 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2024-04-24T09:17:49.261 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.261 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-MDSRank": { 2024-04-24T09:17:49.261 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-04-24T09:17:49.262 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-04-24T09:17:49.262 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 4451, 2024-04-24T09:17:49.262 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.316171060, 2024-04-24T09:17:49.262 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000071033 2024-04-24T09:17:49.262 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:49.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.262 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-PurgeQueue": { 2024-04-24T09:17:49.262 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-04-24T09:17:49.262 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-04-24T09:17:49.263 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.263 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.263 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.263 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:49.263 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.263 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-04-24T09:17:49.263 INFO:teuthology.orchestra.run.smithi081.stdout: "request": 0, 2024-04-24T09:17:49.263 INFO:teuthology.orchestra.run.smithi081.stdout: "reply": 0, 2024-04-24T09:17:49.263 INFO:teuthology.orchestra.run.smithi081.stdout: "reply_latency": { 2024-04-24T09:17:49.264 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.264 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.264 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.264 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_reply": 0, 2024-04-24T09:17:49.264 INFO:teuthology.orchestra.run.smithi081.stdout: "forward": 0, 2024-04-24T09:17:49.264 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_complete": 0, 2024-04-24T09:17:49.264 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_keys": 0, 2024-04-24T09:17:49.264 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_commit": 0, 2024-04-24T09:17:49.264 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_split": 0, 2024-04-24T09:17:49.265 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_merge": 0, 2024-04-24T09:17:49.265 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes": 1235, 2024-04-24T09:17:49.265 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_top": 826, 2024-04-24T09:17:49.265 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_bottom": 409, 2024-04-24T09:17:49.265 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pin_tail": 0, 2024-04-24T09:17:49.265 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pinned": 54, 2024-04-24T09:17:49.265 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_expired": 0, 2024-04-24T09:17:49.265 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_with_caps": 0, 2024-04-24T09:17:49.265 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": 0, 2024-04-24T09:17:49.266 INFO:teuthology.orchestra.run.smithi081.stdout: "subtrees": 18, 2024-04-24T09:17:49.266 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse": 0, 2024-04-24T09:17:49.266 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_hit": 0, 2024-04-24T09:17:49.266 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_forward": 0, 2024-04-24T09:17:49.266 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_discover": 0, 2024-04-24T09:17:49.266 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:17:49.266 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_remote_ino": 0, 2024-04-24T09:17:49.266 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_lock": 0, 2024-04-24T09:17:49.266 INFO:teuthology.orchestra.run.smithi081.stdout: "load_cent": 0, 2024-04-24T09:17:49.267 INFO:teuthology.orchestra.run.smithi081.stdout: "q": 0, 2024-04-24T09:17:49.267 INFO:teuthology.orchestra.run.smithi081.stdout: "exported": 0, 2024-04-24T09:17:49.267 INFO:teuthology.orchestra.run.smithi081.stdout: "exported_inodes": 0, 2024-04-24T09:17:49.267 INFO:teuthology.orchestra.run.smithi081.stdout: "imported": 0, 2024-04-24T09:17:49.267 INFO:teuthology.orchestra.run.smithi081.stdout: "imported_inodes": 0, 2024-04-24T09:17:49.267 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_dir_fetch": 0, 2024-04-24T09:17:49.267 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:17:49.267 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_peer_discover": 0, 2024-04-24T09:17:49.267 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rfiles": 3451, 2024-04-24T09:17:49.267 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rbytes": 26543607, 2024-04-24T09:17:49.268 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rsnaps": 6, 2024-04-24T09:17:49.268 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:17:49.268 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_set_tag": 0, 2024-04-24T09:17:49.268 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:17:49.268 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:17:49.268 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:17:49.268 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:17:49.268 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:17:49.268 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_file_inodes": 0, 2024-04-24T09:17:49.269 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:17:49.269 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T09:17:49.269 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_grant": 0, 2024-04-24T09:17:49.269 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:17:49.269 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:17:49.269 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:17:49.269 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps": 0, 2024-04-24T09:17:49.269 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps_dirty": 0, 2024-04-24T09:17:49.269 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_cap_release": 0, 2024-04-24T09:17:49.269 INFO:teuthology.orchestra.run.smithi081.stdout: "process_request_cap_release": 0 2024-04-24T09:17:49.270 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.270 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache": { 2024-04-24T09:17:49.270 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update": 0, 2024-04-24T09:17:49.270 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update_receipt": 0, 2024-04-24T09:17:49.270 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_try_discover": 0, 2024-04-24T09:17:49.270 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_send_discover": 0, 2024-04-24T09:17:49.270 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_handle_discover": 0, 2024-04-24T09:17:49.270 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays": 0, 2024-04-24T09:17:49.270 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_delayed": 0, 2024-04-24T09:17:49.271 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:17:49.271 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_created": 0, 2024-04-24T09:17:49.271 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_enqueued": 0, 2024-04-24T09:17:49.271 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_reintegrated": 0, 2024-04-24T09:17:49.271 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_migrated": 0, 2024-04-24T09:17:49.271 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_processing": 0, 2024-04-24T09:17:49.271 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:17:49.271 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:17:49.271 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_started": 0, 2024-04-24T09:17:49.271 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_completed": 0, 2024-04-24T09:17:49.272 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:17:49.272 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:17:49.272 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:17:49.272 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_exportdir": 0, 2024-04-24T09:17:49.272 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_flush": 0, 2024-04-24T09:17:49.272 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:17:49.272 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragstats": 0, 2024-04-24T09:17:49.272 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_inodestats": 0 2024-04-24T09:17:49.273 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.273 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log": { 2024-04-24T09:17:49.273 INFO:teuthology.orchestra.run.smithi081.stdout: "evlrg": 0, 2024-04-24T09:17:49.273 INFO:teuthology.orchestra.run.smithi081.stdout: "evadd": 0, 2024-04-24T09:17:49.273 INFO:teuthology.orchestra.run.smithi081.stdout: "evex": 0, 2024-04-24T09:17:49.273 INFO:teuthology.orchestra.run.smithi081.stdout: "evtrm": 0, 2024-04-24T09:17:49.273 INFO:teuthology.orchestra.run.smithi081.stdout: "ev": 31023, 2024-04-24T09:17:49.273 INFO:teuthology.orchestra.run.smithi081.stdout: "evexg": 0, 2024-04-24T09:17:49.273 INFO:teuthology.orchestra.run.smithi081.stdout: "evexd": 0, 2024-04-24T09:17:49.274 INFO:teuthology.orchestra.run.smithi081.stdout: "segadd": 0, 2024-04-24T09:17:49.274 INFO:teuthology.orchestra.run.smithi081.stdout: "segex": 0, 2024-04-24T09:17:49.274 INFO:teuthology.orchestra.run.smithi081.stdout: "segtrm": 0, 2024-04-24T09:17:49.274 INFO:teuthology.orchestra.run.smithi081.stdout: "seg": 2, 2024-04-24T09:17:49.274 INFO:teuthology.orchestra.run.smithi081.stdout: "segmjr": 33, 2024-04-24T09:17:49.274 INFO:teuthology.orchestra.run.smithi081.stdout: "segexg": 0, 2024-04-24T09:17:49.274 INFO:teuthology.orchestra.run.smithi081.stdout: "segexd": 0, 2024-04-24T09:17:49.274 INFO:teuthology.orchestra.run.smithi081.stdout: "expos": 42781044, 2024-04-24T09:17:49.274 INFO:teuthology.orchestra.run.smithi081.stdout: "wrpos": 43261064, 2024-04-24T09:17:49.274 INFO:teuthology.orchestra.run.smithi081.stdout: "rdpos": 43260928, 2024-04-24T09:17:49.275 INFO:teuthology.orchestra.run.smithi081.stdout: "jlat": { 2024-04-24T09:17:49.275 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.275 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.275 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.275 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.275 INFO:teuthology.orchestra.run.smithi081.stdout: "replayed": 31023 2024-04-24T09:17:49.275 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.275 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mem": { 2024-04-24T09:17:49.275 INFO:teuthology.orchestra.run.smithi081.stdout: "ino": 298, 2024-04-24T09:17:49.275 INFO:teuthology.orchestra.run.smithi081.stdout: "ino+": 4182, 2024-04-24T09:17:49.276 INFO:teuthology.orchestra.run.smithi081.stdout: "ino-": 3884, 2024-04-24T09:17:49.276 INFO:teuthology.orchestra.run.smithi081.stdout: "dir": 60, 2024-04-24T09:17:49.276 INFO:teuthology.orchestra.run.smithi081.stdout: "dir+": 177, 2024-04-24T09:17:49.276 INFO:teuthology.orchestra.run.smithi081.stdout: "dir-": 117, 2024-04-24T09:17:49.276 INFO:teuthology.orchestra.run.smithi081.stdout: "dn": 1235, 2024-04-24T09:17:49.276 INFO:teuthology.orchestra.run.smithi081.stdout: "dn+": 4723, 2024-04-24T09:17:49.276 INFO:teuthology.orchestra.run.smithi081.stdout: "dn-": 3488, 2024-04-24T09:17:49.276 INFO:teuthology.orchestra.run.smithi081.stdout: "cap": 0, 2024-04-24T09:17:49.276 INFO:teuthology.orchestra.run.smithi081.stdout: "cap+": 0, 2024-04-24T09:17:49.277 INFO:teuthology.orchestra.run.smithi081.stdout: "cap-": 0, 2024-04-24T09:17:49.277 INFO:teuthology.orchestra.run.smithi081.stdout: "rss": 58248, 2024-04-24T09:17:49.277 INFO:teuthology.orchestra.run.smithi081.stdout: "heap": 215416 2024-04-24T09:17:49.277 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.277 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_server": { 2024-04-24T09:17:49.277 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_client_request": 0, 2024-04-24T09:17:49.277 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_server_request": 0, 2024-04-24T09:17:49.277 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_request": 0, 2024-04-24T09:17:49.277 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_session": 0, 2024-04-24T09:17:49.278 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_peer_request": 0, 2024-04-24T09:17:49.278 INFO:teuthology.orchestra.run.smithi081.stdout: "req_create_latency": { 2024-04-24T09:17:49.278 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.278 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.278 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.278 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.278 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getattr_latency": { 2024-04-24T09:17:49.278 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.278 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.278 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.279 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.279 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getfilelock_latency": { 2024-04-24T09:17:49.279 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.279 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.279 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.279 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.279 INFO:teuthology.orchestra.run.smithi081.stdout: "req_link_latency": { 2024-04-24T09:17:49.279 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.279 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.280 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.280 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookup_latency": { 2024-04-24T09:17:49.280 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.280 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.280 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.280 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookuphash_latency": { 2024-04-24T09:17:49.281 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.281 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.281 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.281 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.281 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupino_latency": { 2024-04-24T09:17:49.281 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.281 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.281 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.281 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.281 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupname_latency": { 2024-04-24T09:17:49.282 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.282 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.282 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.282 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.282 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupparent_latency": { 2024-04-24T09:17:49.282 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.282 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.282 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.282 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.283 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupsnap_latency": { 2024-04-24T09:17:49.283 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.283 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.283 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.283 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.283 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lssnap_latency": { 2024-04-24T09:17:49.283 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.283 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.283 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.283 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.284 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mkdir_latency": { 2024-04-24T09:17:49.284 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.284 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.284 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.284 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.285 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mknod_latency": { 2024-04-24T09:17:49.285 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.285 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.285 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.285 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.285 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mksnap_latency": { 2024-04-24T09:17:49.285 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.285 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.285 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.286 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.286 INFO:teuthology.orchestra.run.smithi081.stdout: "req_open_latency": { 2024-04-24T09:17:49.286 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.286 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.286 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.286 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.286 INFO:teuthology.orchestra.run.smithi081.stdout: "req_readdir_latency": { 2024-04-24T09:17:49.286 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.287 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.287 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.287 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.287 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rename_latency": { 2024-04-24T09:17:49.287 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.287 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.287 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.287 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.287 INFO:teuthology.orchestra.run.smithi081.stdout: "req_renamesnap_latency": { 2024-04-24T09:17:49.288 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.288 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.288 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.288 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.288 INFO:teuthology.orchestra.run.smithi081.stdout: "req_snapdiff_latency": { 2024-04-24T09:17:49.288 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.288 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.288 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.288 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.289 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmdir_latency": { 2024-04-24T09:17:49.289 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.289 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.289 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.289 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.289 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmsnap_latency": { 2024-04-24T09:17:49.289 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.289 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.289 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.290 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmxattr_latency": { 2024-04-24T09:17:49.290 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.290 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.290 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.290 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setattr_latency": { 2024-04-24T09:17:49.290 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.290 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.291 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.291 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.291 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setdirlayout_latency": { 2024-04-24T09:17:49.291 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.291 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.291 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.291 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.291 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setfilelock_latency": { 2024-04-24T09:17:49.292 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.292 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.292 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.292 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setlayout_latency": { 2024-04-24T09:17:49.292 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.292 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.292 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.292 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setxattr_latency": { 2024-04-24T09:17:49.293 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.293 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.293 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.293 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.293 INFO:teuthology.orchestra.run.smithi081.stdout: "req_symlink_latency": { 2024-04-24T09:17:49.293 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.293 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.293 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.293 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.294 INFO:teuthology.orchestra.run.smithi081.stdout: "req_unlink_latency": { 2024-04-24T09:17:49.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.294 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.294 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:17:49.294 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:17:49.294 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getvxattr_latency": { 2024-04-24T09:17:49.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.294 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.295 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.295 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:49.295 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.295 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessions": { 2024-04-24T09:17:49.295 INFO:teuthology.orchestra.run.smithi081.stdout: "session_count": 2, 2024-04-24T09:17:49.295 INFO:teuthology.orchestra.run.smithi081.stdout: "session_add": 6, 2024-04-24T09:17:49.295 INFO:teuthology.orchestra.run.smithi081.stdout: "session_remove": 4, 2024-04-24T09:17:49.295 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_open": 4, 2024-04-24T09:17:49.295 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_stale": 0, 2024-04-24T09:17:49.296 INFO:teuthology.orchestra.run.smithi081.stdout: "total_load": 0, 2024-04-24T09:17:49.296 INFO:teuthology.orchestra.run.smithi081.stdout: "average_load": 0, 2024-04-24T09:17:49.296 INFO:teuthology.orchestra.run.smithi081.stdout: "avg_session_uptime": 3491, 2024-04-24T09:17:49.296 INFO:teuthology.orchestra.run.smithi081.stdout: "mdthresh_evicted": 0 2024-04-24T09:17:49.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.296 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2024-04-24T09:17:49.296 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:17:49.296 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 0, 2024-04-24T09:17:49.296 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:17:49.296 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:17:49.297 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:17:49.297 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:17:49.297 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:17:49.297 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:17:49.297 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:17:49.297 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:17:49.297 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:17:49.297 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:17:49.297 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:17:49.297 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:17:49.298 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:17:49.298 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2024-04-24T09:17:49.298 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:17:49.298 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2024-04-24T09:17:49.298 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:17:49.298 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:17:49.298 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:17:49.298 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:17:49.298 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:17:49.299 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:17:49.299 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:17:49.299 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2024-04-24T09:17:49.299 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:17:49.299 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:17:49.299 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:17:49.299 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2024-04-24T09:17:49.299 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2024-04-24T09:17:49.299 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2024-04-24T09:17:49.300 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:17:49.300 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:17:49.300 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:17:49.300 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:17:49.300 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 207906, 2024-04-24T09:17:49.300 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 65, 2024-04-24T09:17:49.300 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:17:49.300 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2024-04-24T09:17:49.300 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2024-04-24T09:17:49.300 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2024-04-24T09:17:49.301 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:17:49.301 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2024-04-24T09:17:49.301 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:17:49.301 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2024-04-24T09:17:49.301 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 9320, 2024-04-24T09:17:49.301 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 178, 2024-04-24T09:17:49.301 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:17:49.301 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:17:49.301 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2024-04-24T09:17:49.302 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2024-04-24T09:17:49.302 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 1652226, 2024-04-24T09:17:49.302 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 7892, 2024-04-24T09:17:49.302 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2024-04-24T09:17:49.302 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2024-04-24T09:17:49.302 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2024-04-24T09:17:49.302 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2024-04-24T09:17:49.302 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.303 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter": { 2024-04-24T09:17:49.303 INFO:teuthology.orchestra.run.smithi081.stdout: "op_active": 0, 2024-04-24T09:17:49.303 INFO:teuthology.orchestra.run.smithi081.stdout: "op_laggy": 0, 2024-04-24T09:17:49.303 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send": 2658, 2024-04-24T09:17:49.303 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send_bytes": 26, 2024-04-24T09:17:49.303 INFO:teuthology.orchestra.run.smithi081.stdout: "op_resend": 4, 2024-04-24T09:17:49.303 INFO:teuthology.orchestra.run.smithi081.stdout: "op_reply": 2658, 2024-04-24T09:17:49.303 INFO:teuthology.orchestra.run.smithi081.stdout: "op_latency": { 2024-04-24T09:17:49.303 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2658, 2024-04-24T09:17:49.303 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 3.291937954, 2024-04-24T09:17:49.304 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001238501 2024-04-24T09:17:49.304 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.304 INFO:teuthology.orchestra.run.smithi081.stdout: "op_inflight": 0, 2024-04-24T09:17:49.304 INFO:teuthology.orchestra.run.smithi081.stdout: "oplen_avg": { 2024-04-24T09:17:49.304 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2658, 2024-04-24T09:17:49.304 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 2659 2024-04-24T09:17:49.304 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.304 INFO:teuthology.orchestra.run.smithi081.stdout: "op": 2658, 2024-04-24T09:17:49.304 INFO:teuthology.orchestra.run.smithi081.stdout: "op_r": 2658, 2024-04-24T09:17:49.305 INFO:teuthology.orchestra.run.smithi081.stdout: "op_w": 0, 2024-04-24T09:17:49.305 INFO:teuthology.orchestra.run.smithi081.stdout: "op_rmw": 0, 2024-04-24T09:17:49.305 INFO:teuthology.orchestra.run.smithi081.stdout: "op_pg": 0, 2024-04-24T09:17:49.305 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_stat": 1580, 2024-04-24T09:17:49.305 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_create": 0, 2024-04-24T09:17:49.305 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_read": 1075, 2024-04-24T09:17:49.305 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_write": 0, 2024-04-24T09:17:49.305 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writefull": 0, 2024-04-24T09:17:49.305 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writesame": 0, 2024-04-24T09:17:49.305 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_append": 0, 2024-04-24T09:17:49.306 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_zero": 0, 2024-04-24T09:17:49.306 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_truncate": 0, 2024-04-24T09:17:49.306 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_delete": 0, 2024-04-24T09:17:49.306 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_mapext": 0, 2024-04-24T09:17:49.306 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_sparse_read": 0, 2024-04-24T09:17:49.306 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_clonerange": 0, 2024-04-24T09:17:49.306 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_getxattr": 2, 2024-04-24T09:17:49.306 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_setxattr": 0, 2024-04-24T09:17:49.307 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:17:49.307 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_rmxattr": 0, 2024-04-24T09:17:49.307 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:17:49.307 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_call": 0, 2024-04-24T09:17:49.307 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_watch": 0, 2024-04-24T09:17:49.307 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_notify": 0, 2024-04-24T09:17:49.307 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:17:49.307 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls": 0, 2024-04-24T09:17:49.307 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:17:49.308 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_other": 0, 2024-04-24T09:17:49.308 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_active": 0, 2024-04-24T09:17:49.308 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_send": 0, 2024-04-24T09:17:49.308 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_resend": 0, 2024-04-24T09:17:49.308 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_ping": 0, 2024-04-24T09:17:49.308 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_active": 0, 2024-04-24T09:17:49.308 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_send": 0, 2024-04-24T09:17:49.308 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_resend": 0, 2024-04-24T09:17:49.308 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_active": 0, 2024-04-24T09:17:49.308 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_send": 0, 2024-04-24T09:17:49.309 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_resend": 0, 2024-04-24T09:17:49.309 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_active": 0, 2024-04-24T09:17:49.309 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_send": 0, 2024-04-24T09:17:49.309 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_resend": 0, 2024-04-24T09:17:49.309 INFO:teuthology.orchestra.run.smithi081.stdout: "command_active": 0, 2024-04-24T09:17:49.309 INFO:teuthology.orchestra.run.smithi081.stdout: "command_send": 0, 2024-04-24T09:17:49.309 INFO:teuthology.orchestra.run.smithi081.stdout: "command_resend": 0, 2024-04-24T09:17:49.309 INFO:teuthology.orchestra.run.smithi081.stdout: "map_epoch": 101, 2024-04-24T09:17:49.310 INFO:teuthology.orchestra.run.smithi081.stdout: "map_full": 0, 2024-04-24T09:17:49.310 INFO:teuthology.orchestra.run.smithi081.stdout: "map_inc": 9, 2024-04-24T09:17:49.310 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_sessions": 7, 2024-04-24T09:17:49.310 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_open": 7, 2024-04-24T09:17:49.310 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_close": 0, 2024-04-24T09:17:49.310 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_laggy": 0, 2024-04-24T09:17:49.310 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_wr": 0, 2024-04-24T09:17:49.310 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_rd": 2, 2024-04-24T09:17:49.311 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_del": 0 2024-04-24T09:17:49.311 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.311 INFO:teuthology.orchestra.run.smithi081.stdout: "oft": { 2024-04-24T09:17:49.311 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_objs": 0, 2024-04-24T09:17:49.311 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:17:49.311 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_updates": 0, 2024-04-24T09:17:49.311 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_removes": 0 2024-04-24T09:17:49.311 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.311 INFO:teuthology.orchestra.run.smithi081.stdout: "osd-slow-ops": { 2024-04-24T09:17:49.311 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_ops_count": 0 2024-04-24T09:17:49.312 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.312 INFO:teuthology.orchestra.run.smithi081.stdout: "purge_queue": { 2024-04-24T09:17:49.312 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops": 0, 2024-04-24T09:17:49.312 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T09:17:49.312 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing": 0, 2024-04-24T09:17:49.312 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_high_water": 0, 2024-04-24T09:17:49.312 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed_ops": 0, 2024-04-24T09:17:49.312 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed": 0, 2024-04-24T09:17:49.312 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_item_in_journal": 0 2024-04-24T09:17:49.312 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.313 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:17:49.313 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:49.313 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-04-24T09:17:49.313 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:49.313 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 2931, 2024-04-24T09:17:49.313 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 41196259, 2024-04-24T09:17:49.313 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:49.313 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 2931, 2024-04-24T09:17:49.313 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-04-24T09:17:49.314 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-04-24T09:17:49.314 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 2931, 2024-04-24T09:17:49.314 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 41196259, 2024-04-24T09:17:49.314 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:49.314 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.314 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.314 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.314 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:49.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.315 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_bytes": { 2024-04-24T09:17:49.315 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:49.315 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-04-24T09:17:49.315 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:49.315 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-04-24T09:17:49.315 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-04-24T09:17:49.315 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:49.315 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-04-24T09:17:49.315 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 2658, 2024-04-24T09:17:49.316 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 39067585, 2024-04-24T09:17:49.316 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 2658, 2024-04-24T09:17:49.316 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 39067585, 2024-04-24T09:17:49.316 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:49.316 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.316 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.316 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.316 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:49.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.317 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_ops": { 2024-04-24T09:17:49.317 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:49.317 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 1024, 2024-04-24T09:17:49.317 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:49.317 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-04-24T09:17:49.317 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-04-24T09:17:49.317 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:49.317 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-04-24T09:17:49.317 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 2658, 2024-04-24T09:17:49.317 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 2658, 2024-04-24T09:17:49.318 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 2658, 2024-04-24T09:17:49.318 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 2658, 2024-04-24T09:17:49.318 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:49.318 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.318 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.318 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.318 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:49.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.318 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:17:49.319 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:49.319 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-04-24T09:17:49.319 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:49.319 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-04-24T09:17:49.319 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-04-24T09:17:49.319 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:49.319 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-04-24T09:17:49.319 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-04-24T09:17:49.319 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-04-24T09:17:49.320 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 0, 2024-04-24T09:17:49.320 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 0, 2024-04-24T09:17:49.320 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:49.320 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.320 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.320 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.320 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:49.320 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-04-24T09:17:49.320 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle-0x559288416780": { 2024-04-24T09:17:49.320 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-04-24T09:17:49.321 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-04-24T09:17:49.321 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-04-24T09:17:49.321 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-04-24T09:17:49.321 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-04-24T09:17:49.321 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:49.321 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-04-24T09:17:49.321 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-04-24T09:17:49.321 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-04-24T09:17:49.321 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 0, 2024-04-24T09:17:49.321 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 0, 2024-04-24T09:17:49.322 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-04-24T09:17:49.322 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-04-24T09:17:49.322 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-04-24T09:17:49.322 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-04-24T09:17:49.322 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:49.322 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-04-24T09:17:49.322 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-04-24T09:17:49.323 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:49 smithi081 ceph-mon[30961]: pgmap v333: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:49.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:49 smithi069 ceph-mon[28170]: pgmap v333: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:49.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:49 smithi130 ceph-mon[31553]: pgmap v333: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:50.150 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-24T09:17:50.150 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-24T09:17:50.150 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-24T09:17:50.150 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-04-24T09:17:50.150 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-04-24T09:17:50.150 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-24T09:17:50.150 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T09:17:50.151 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-04-24T09:17:50.151 DEBUG:tasks.check_counter:Getting stats from c 2024-04-24T09:17:50.151 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-24T09:17:50.613 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:17:51.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:51 smithi130 ceph-mon[31553]: pgmap v334: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:51.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:51 smithi081 ceph-mon[30961]: pgmap v334: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:51.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:51 smithi069 ceph-mon[28170]: pgmap v334: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:53.041 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-04-24T09:17:53.041 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:17:53.042 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 572, 2024-04-24T09:17:53.042 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 572, 2024-04-24T09:17:53.042 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 12733007, 2024-04-24T09:17:53.042 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 181136, 2024-04-24T09:17:53.042 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 7, 2024-04-24T09:17:53.042 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 3, 2024-04-24T09:17:53.042 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.168396402, 2024-04-24T09:17:53.042 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.040688461, 2024-04-24T09:17:53.043 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.080561143, 2024-04-24T09:17:53.043 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.031534187, 2024-04-24T09:17:53.043 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:53.043 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 571, 2024-04-24T09:17:53.043 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.056354390, 2024-04-24T09:17:53.043 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000098694 2024-04-24T09:17:53.043 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.043 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:53.043 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.043 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.044 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.044 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.044 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-24T09:17:53.044 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T09:17:53.044 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.044 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:17:53.044 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 1024, 2024-04-24T09:17:53.044 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 1002, 2024-04-24T09:17:53.044 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 14435856, 2024-04-24T09:17:53.045 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 384647, 2024-04-24T09:17:53.045 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2024-04-24T09:17:53.045 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 3, 2024-04-24T09:17:53.045 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.322826908, 2024-04-24T09:17:53.045 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.102408904, 2024-04-24T09:17:53.045 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.168859857, 2024-04-24T09:17:53.045 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.062061333, 2024-04-24T09:17:53.045 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:53.045 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1001, 2024-04-24T09:17:53.046 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.096104414, 2024-04-24T09:17:53.046 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000096008 2024-04-24T09:17:53.046 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.046 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:53.046 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.046 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.046 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.046 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.047 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 1664704, 2024-04-24T09:17:53.047 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 143280 2024-04-24T09:17:53.047 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.047 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:17:53.047 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 1209, 2024-04-24T09:17:53.047 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 1382, 2024-04-24T09:17:53.047 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 6429189, 2024-04-24T09:17:53.047 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 780306, 2024-04-24T09:17:53.047 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 7, 2024-04-24T09:17:53.048 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 5, 2024-04-24T09:17:53.048 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.309182496, 2024-04-24T09:17:53.048 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.109390268, 2024-04-24T09:17:53.048 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.095225731, 2024-04-24T09:17:53.048 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.072229040, 2024-04-24T09:17:53.048 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:53.048 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1379, 2024-04-24T09:17:53.048 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.145501976, 2024-04-24T09:17:53.048 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000105512 2024-04-24T09:17:53.049 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.049 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:53.049 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.049 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.049 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.049 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.049 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T09:17:53.049 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 396480 2024-04-24T09:17:53.049 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.050 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-04-24T09:17:53.050 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 1, 2024-04-24T09:17:53.050 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-04-24T09:17:53.050 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.050 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-MDSRank": { 2024-04-24T09:17:53.050 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-04-24T09:17:53.050 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-04-24T09:17:53.050 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 4218, 2024-04-24T09:17:53.051 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.243105528, 2024-04-24T09:17:53.051 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000057635 2024-04-24T09:17:53.051 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:17:53.051 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.051 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-PurgeQueue": { 2024-04-24T09:17:53.051 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-04-24T09:17:53.051 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-04-24T09:17:53.051 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.052 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.052 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.052 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:17:53.052 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.052 INFO:teuthology.orchestra.run.smithi130.stdout: "mds": { 2024-04-24T09:17:53.052 INFO:teuthology.orchestra.run.smithi130.stdout: "request": 0, 2024-04-24T09:17:53.052 INFO:teuthology.orchestra.run.smithi130.stdout: "reply": 0, 2024-04-24T09:17:53.052 INFO:teuthology.orchestra.run.smithi130.stdout: "reply_latency": { 2024-04-24T09:17:53.052 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.052 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.053 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.053 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.053 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_reply": 0, 2024-04-24T09:17:53.053 INFO:teuthology.orchestra.run.smithi130.stdout: "forward": 0, 2024-04-24T09:17:53.053 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_complete": 0, 2024-04-24T09:17:53.053 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_keys": 0, 2024-04-24T09:17:53.053 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_commit": 0, 2024-04-24T09:17:53.053 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_split": 0, 2024-04-24T09:17:53.054 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_merge": 0, 2024-04-24T09:17:53.054 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes": 3567, 2024-04-24T09:17:53.054 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_top": 117, 2024-04-24T09:17:53.054 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_bottom": 3450, 2024-04-24T09:17:53.054 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pin_tail": 0, 2024-04-24T09:17:53.054 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pinned": 3400, 2024-04-24T09:17:53.054 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_expired": 0, 2024-04-24T09:17:53.054 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_with_caps": 0, 2024-04-24T09:17:53.054 INFO:teuthology.orchestra.run.smithi130.stdout: "caps": 0, 2024-04-24T09:17:53.055 INFO:teuthology.orchestra.run.smithi130.stdout: "subtrees": 77, 2024-04-24T09:17:53.055 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse": 0, 2024-04-24T09:17:53.055 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_hit": 0, 2024-04-24T09:17:53.055 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_forward": 0, 2024-04-24T09:17:53.055 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_discover": 0, 2024-04-24T09:17:53.055 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:17:53.055 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_remote_ino": 0, 2024-04-24T09:17:53.055 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_lock": 0, 2024-04-24T09:17:53.055 INFO:teuthology.orchestra.run.smithi130.stdout: "load_cent": 0, 2024-04-24T09:17:53.055 INFO:teuthology.orchestra.run.smithi130.stdout: "q": 0, 2024-04-24T09:17:53.056 INFO:teuthology.orchestra.run.smithi130.stdout: "exported": 0, 2024-04-24T09:17:53.056 INFO:teuthology.orchestra.run.smithi130.stdout: "exported_inodes": 0, 2024-04-24T09:17:53.056 INFO:teuthology.orchestra.run.smithi130.stdout: "imported": 0, 2024-04-24T09:17:53.056 INFO:teuthology.orchestra.run.smithi130.stdout: "imported_inodes": 0, 2024-04-24T09:17:53.056 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_dir_fetch": 0, 2024-04-24T09:17:53.056 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:17:53.056 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_peer_discover": 0, 2024-04-24T09:17:53.056 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rfiles": 0, 2024-04-24T09:17:53.056 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rbytes": 0, 2024-04-24T09:17:53.056 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rsnaps": 0, 2024-04-24T09:17:53.057 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:17:53.057 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_set_tag": 0, 2024-04-24T09:17:53.057 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:17:53.057 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:17:53.057 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:17:53.057 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:17:53.057 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:17:53.057 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_file_inodes": 0, 2024-04-24T09:17:53.058 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:17:53.058 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T09:17:53.058 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_grant": 0, 2024-04-24T09:17:53.058 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:17:53.058 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:17:53.058 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:17:53.058 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps": 0, 2024-04-24T09:17:53.058 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps_dirty": 0, 2024-04-24T09:17:53.058 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_cap_release": 0, 2024-04-24T09:17:53.059 INFO:teuthology.orchestra.run.smithi130.stdout: "process_request_cap_release": 0 2024-04-24T09:17:53.059 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.059 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_cache": { 2024-04-24T09:17:53.059 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update": 0, 2024-04-24T09:17:53.059 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update_receipt": 0, 2024-04-24T09:17:53.059 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_try_discover": 0, 2024-04-24T09:17:53.059 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_send_discover": 0, 2024-04-24T09:17:53.059 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_handle_discover": 0, 2024-04-24T09:17:53.059 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays": 0, 2024-04-24T09:17:53.060 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_delayed": 0, 2024-04-24T09:17:53.060 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:17:53.060 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_created": 0, 2024-04-24T09:17:53.060 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_enqueued": 0, 2024-04-24T09:17:53.060 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_reintegrated": 0, 2024-04-24T09:17:53.060 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_migrated": 0, 2024-04-24T09:17:53.060 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_processing": 0, 2024-04-24T09:17:53.060 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:17:53.061 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:17:53.061 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_started": 0, 2024-04-24T09:17:53.061 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_completed": 0, 2024-04-24T09:17:53.061 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:17:53.061 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:17:53.061 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:17:53.061 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_exportdir": 0, 2024-04-24T09:17:53.061 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_flush": 0, 2024-04-24T09:17:53.061 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:17:53.062 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragstats": 0, 2024-04-24T09:17:53.062 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_inodestats": 0 2024-04-24T09:17:53.062 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.062 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_log": { 2024-04-24T09:17:53.062 INFO:teuthology.orchestra.run.smithi130.stdout: "evlrg": 0, 2024-04-24T09:17:53.062 INFO:teuthology.orchestra.run.smithi130.stdout: "evadd": 0, 2024-04-24T09:17:53.062 INFO:teuthology.orchestra.run.smithi130.stdout: "evex": 0, 2024-04-24T09:17:53.062 INFO:teuthology.orchestra.run.smithi130.stdout: "evtrm": 0, 2024-04-24T09:17:53.063 INFO:teuthology.orchestra.run.smithi130.stdout: "ev": 44537, 2024-04-24T09:17:53.063 INFO:teuthology.orchestra.run.smithi130.stdout: "evexg": 0, 2024-04-24T09:17:53.063 INFO:teuthology.orchestra.run.smithi130.stdout: "evexd": 0, 2024-04-24T09:17:53.063 INFO:teuthology.orchestra.run.smithi130.stdout: "segadd": 0, 2024-04-24T09:17:53.063 INFO:teuthology.orchestra.run.smithi130.stdout: "segex": 0, 2024-04-24T09:17:53.063 INFO:teuthology.orchestra.run.smithi130.stdout: "segtrm": 0, 2024-04-24T09:17:53.063 INFO:teuthology.orchestra.run.smithi130.stdout: "seg": 52, 2024-04-24T09:17:53.063 INFO:teuthology.orchestra.run.smithi130.stdout: "segmjr": 5, 2024-04-24T09:17:53.063 INFO:teuthology.orchestra.run.smithi130.stdout: "segexg": 0, 2024-04-24T09:17:53.063 INFO:teuthology.orchestra.run.smithi130.stdout: "segexd": 0, 2024-04-24T09:17:53.064 INFO:teuthology.orchestra.run.smithi130.stdout: "expos": 4194304, 2024-04-24T09:17:53.064 INFO:teuthology.orchestra.run.smithi130.stdout: "wrpos": 35415409, 2024-04-24T09:17:53.064 INFO:teuthology.orchestra.run.smithi130.stdout: "rdpos": 35414835, 2024-04-24T09:17:53.064 INFO:teuthology.orchestra.run.smithi130.stdout: "jlat": { 2024-04-24T09:17:53.064 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.064 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.064 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.064 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.064 INFO:teuthology.orchestra.run.smithi130.stdout: "replayed": 44537 2024-04-24T09:17:53.064 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.065 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_mem": { 2024-04-24T09:17:53.065 INFO:teuthology.orchestra.run.smithi130.stdout: "ino": 2338, 2024-04-24T09:17:53.065 INFO:teuthology.orchestra.run.smithi130.stdout: "ino+": 3171, 2024-04-24T09:17:53.065 INFO:teuthology.orchestra.run.smithi130.stdout: "ino-": 833, 2024-04-24T09:17:53.065 INFO:teuthology.orchestra.run.smithi130.stdout: "dir": 116, 2024-04-24T09:17:53.065 INFO:teuthology.orchestra.run.smithi130.stdout: "dir+": 121, 2024-04-24T09:17:53.065 INFO:teuthology.orchestra.run.smithi130.stdout: "dir-": 5, 2024-04-24T09:17:53.065 INFO:teuthology.orchestra.run.smithi130.stdout: "dn": 3567, 2024-04-24T09:17:53.065 INFO:teuthology.orchestra.run.smithi130.stdout: "dn+": 3568, 2024-04-24T09:17:53.065 INFO:teuthology.orchestra.run.smithi130.stdout: "dn-": 1, 2024-04-24T09:17:53.066 INFO:teuthology.orchestra.run.smithi130.stdout: "cap": 0, 2024-04-24T09:17:53.066 INFO:teuthology.orchestra.run.smithi130.stdout: "cap+": 0, 2024-04-24T09:17:53.066 INFO:teuthology.orchestra.run.smithi130.stdout: "cap-": 0, 2024-04-24T09:17:53.066 INFO:teuthology.orchestra.run.smithi130.stdout: "rss": 60852, 2024-04-24T09:17:53.066 INFO:teuthology.orchestra.run.smithi130.stdout: "heap": 215416 2024-04-24T09:17:53.066 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.066 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_server": { 2024-04-24T09:17:53.066 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_client_request": 0, 2024-04-24T09:17:53.066 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_server_request": 0, 2024-04-24T09:17:53.067 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_request": 0, 2024-04-24T09:17:53.067 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_session": 0, 2024-04-24T09:17:53.067 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_peer_request": 0, 2024-04-24T09:17:53.067 INFO:teuthology.orchestra.run.smithi130.stdout: "req_create_latency": { 2024-04-24T09:17:53.067 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.067 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.067 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.067 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.067 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getattr_latency": { 2024-04-24T09:17:53.067 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.068 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.068 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.068 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.068 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getfilelock_latency": { 2024-04-24T09:17:53.068 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.068 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.068 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.068 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.068 INFO:teuthology.orchestra.run.smithi130.stdout: "req_link_latency": { 2024-04-24T09:17:53.069 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.069 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.069 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.069 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.069 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookup_latency": { 2024-04-24T09:17:53.069 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.069 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.069 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.069 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.070 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookuphash_latency": { 2024-04-24T09:17:53.070 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.070 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.070 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.070 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.070 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupino_latency": { 2024-04-24T09:17:53.070 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.070 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.071 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.071 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.071 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupname_latency": { 2024-04-24T09:17:53.071 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.071 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.071 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.071 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.071 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupparent_latency": { 2024-04-24T09:17:53.071 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.072 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.072 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.072 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.072 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupsnap_latency": { 2024-04-24T09:17:53.072 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.072 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.072 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.072 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.072 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lssnap_latency": { 2024-04-24T09:17:53.073 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.073 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.073 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.073 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.073 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mkdir_latency": { 2024-04-24T09:17:53.073 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.074 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.074 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.074 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.074 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mknod_latency": { 2024-04-24T09:17:53.074 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.074 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.074 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.074 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.075 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mksnap_latency": { 2024-04-24T09:17:53.075 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.075 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.075 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.075 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.075 INFO:teuthology.orchestra.run.smithi130.stdout: "req_open_latency": { 2024-04-24T09:17:53.075 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.075 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.075 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.076 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.076 INFO:teuthology.orchestra.run.smithi130.stdout: "req_readdir_latency": { 2024-04-24T09:17:53.076 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.076 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.076 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.076 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.076 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rename_latency": { 2024-04-24T09:17:53.076 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.076 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.077 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.077 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.077 INFO:teuthology.orchestra.run.smithi130.stdout: "req_renamesnap_latency": { 2024-04-24T09:17:53.077 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.077 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.077 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.077 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.077 INFO:teuthology.orchestra.run.smithi130.stdout: "req_snapdiff_latency": { 2024-04-24T09:17:53.077 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.078 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.078 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.078 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.078 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmdir_latency": { 2024-04-24T09:17:53.078 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.078 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.078 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.078 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.078 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmsnap_latency": { 2024-04-24T09:17:53.079 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.079 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.079 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.079 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.079 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmxattr_latency": { 2024-04-24T09:17:53.079 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.079 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.079 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.079 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.080 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setattr_latency": { 2024-04-24T09:17:53.080 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.080 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.080 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.080 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.080 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setdirlayout_latency": { 2024-04-24T09:17:53.080 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.080 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.081 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.081 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.081 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setfilelock_latency": { 2024-04-24T09:17:53.081 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.081 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.081 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.081 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.081 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setlayout_latency": { 2024-04-24T09:17:53.082 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.082 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.082 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.082 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.082 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setxattr_latency": { 2024-04-24T09:17:53.082 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.082 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.082 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.082 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.082 INFO:teuthology.orchestra.run.smithi130.stdout: "req_symlink_latency": { 2024-04-24T09:17:53.083 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.083 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.083 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.083 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.083 INFO:teuthology.orchestra.run.smithi130.stdout: "req_unlink_latency": { 2024-04-24T09:17:53.083 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.083 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.083 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.083 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.084 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:17:53.084 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:17:53.084 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getvxattr_latency": { 2024-04-24T09:17:53.084 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.084 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.084 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.084 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:17:53.084 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.084 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_sessions": { 2024-04-24T09:17:53.085 INFO:teuthology.orchestra.run.smithi130.stdout: "session_count": 2, 2024-04-24T09:17:53.085 INFO:teuthology.orchestra.run.smithi130.stdout: "session_add": 3, 2024-04-24T09:17:53.085 INFO:teuthology.orchestra.run.smithi130.stdout: "session_remove": 1, 2024-04-24T09:17:53.085 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_open": 2, 2024-04-24T09:17:53.085 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_stale": 0, 2024-04-24T09:17:53.085 INFO:teuthology.orchestra.run.smithi130.stdout: "total_load": 0, 2024-04-24T09:17:53.085 INFO:teuthology.orchestra.run.smithi130.stdout: "average_load": 0, 2024-04-24T09:17:53.085 INFO:teuthology.orchestra.run.smithi130.stdout: "avg_session_uptime": 2086, 2024-04-24T09:17:53.085 INFO:teuthology.orchestra.run.smithi130.stdout: "mdthresh_evicted": 0 2024-04-24T09:17:53.085 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.086 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-04-24T09:17:53.086 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:17:53.086 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-04-24T09:17:53.086 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:17:53.086 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:17:53.086 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:17:53.086 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:17:53.086 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:17:53.087 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:17:53.087 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:17:53.087 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:17:53.087 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:17:53.087 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:17:53.087 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:17:53.087 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:17:53.087 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:17:53.087 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-04-24T09:17:53.087 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:17:53.088 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-04-24T09:17:53.088 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:17:53.088 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:17:53.088 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:17:53.088 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:17:53.088 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:17:53.088 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:17:53.088 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:17:53.088 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-04-24T09:17:53.089 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:17:53.089 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:17:53.089 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:17:53.089 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-04-24T09:17:53.089 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-04-24T09:17:53.089 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-04-24T09:17:53.089 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:17:53.089 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:17:53.089 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:17:53.089 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:17:53.090 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 207185, 2024-04-24T09:17:53.090 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 64, 2024-04-24T09:17:53.090 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:17:53.090 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-04-24T09:17:53.090 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-04-24T09:17:53.090 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-04-24T09:17:53.090 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:17:53.090 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-04-24T09:17:53.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:17:53.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-04-24T09:17:53.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 9320, 2024-04-24T09:17:53.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 178, 2024-04-24T09:17:53.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:17:53.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:17:53.091 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-04-24T09:17:53.091 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-04-24T09:17:53.091 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 7642069, 2024-04-24T09:17:53.092 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 27800, 2024-04-24T09:17:53.092 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-04-24T09:17:53.092 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-04-24T09:17:53.092 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-04-24T09:17:53.092 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-04-24T09:17:53.092 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.092 INFO:teuthology.orchestra.run.smithi130.stdout: "objecter": { 2024-04-24T09:17:53.092 INFO:teuthology.orchestra.run.smithi130.stdout: "op_active": 0, 2024-04-24T09:17:53.092 INFO:teuthology.orchestra.run.smithi130.stdout: "op_laggy": 0, 2024-04-24T09:17:53.093 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send": 2534, 2024-04-24T09:17:53.093 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send_bytes": 21, 2024-04-24T09:17:53.093 INFO:teuthology.orchestra.run.smithi130.stdout: "op_resend": 3, 2024-04-24T09:17:53.093 INFO:teuthology.orchestra.run.smithi130.stdout: "op_reply": 2534, 2024-04-24T09:17:53.093 INFO:teuthology.orchestra.run.smithi130.stdout: "op_latency": { 2024-04-24T09:17:53.093 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2534, 2024-04-24T09:17:53.093 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 3.346937994, 2024-04-24T09:17:53.093 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.001320812 2024-04-24T09:17:53.093 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.093 INFO:teuthology.orchestra.run.smithi130.stdout: "op_inflight": 0, 2024-04-24T09:17:53.094 INFO:teuthology.orchestra.run.smithi130.stdout: "oplen_avg": { 2024-04-24T09:17:53.094 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2534, 2024-04-24T09:17:53.094 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 2535 2024-04-24T09:17:53.094 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.094 INFO:teuthology.orchestra.run.smithi130.stdout: "op": 2534, 2024-04-24T09:17:53.094 INFO:teuthology.orchestra.run.smithi130.stdout: "op_r": 2534, 2024-04-24T09:17:53.094 INFO:teuthology.orchestra.run.smithi130.stdout: "op_w": 0, 2024-04-24T09:17:53.094 INFO:teuthology.orchestra.run.smithi130.stdout: "op_rmw": 0, 2024-04-24T09:17:53.094 INFO:teuthology.orchestra.run.smithi130.stdout: "op_pg": 0, 2024-04-24T09:17:53.095 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_stat": 1507, 2024-04-24T09:17:53.095 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_create": 0, 2024-04-24T09:17:53.095 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_read": 1025, 2024-04-24T09:17:53.095 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_write": 0, 2024-04-24T09:17:53.095 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writefull": 0, 2024-04-24T09:17:53.095 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writesame": 0, 2024-04-24T09:17:53.095 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_append": 0, 2024-04-24T09:17:53.095 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_zero": 0, 2024-04-24T09:17:53.095 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_truncate": 0, 2024-04-24T09:17:53.095 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_delete": 0, 2024-04-24T09:17:53.096 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_mapext": 0, 2024-04-24T09:17:53.096 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_sparse_read": 0, 2024-04-24T09:17:53.096 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_clonerange": 0, 2024-04-24T09:17:53.096 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_getxattr": 1, 2024-04-24T09:17:53.096 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_setxattr": 0, 2024-04-24T09:17:53.096 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:17:53.096 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_rmxattr": 0, 2024-04-24T09:17:53.096 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:17:53.097 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_call": 0, 2024-04-24T09:17:53.097 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_watch": 0, 2024-04-24T09:17:53.097 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_notify": 0, 2024-04-24T09:17:53.097 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:17:53.097 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls": 0, 2024-04-24T09:17:53.097 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:17:53.097 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_other": 0, 2024-04-24T09:17:53.097 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_active": 0, 2024-04-24T09:17:53.097 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_send": 0, 2024-04-24T09:17:53.097 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_resend": 0, 2024-04-24T09:17:53.098 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_ping": 0, 2024-04-24T09:17:53.098 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_active": 0, 2024-04-24T09:17:53.098 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_send": 0, 2024-04-24T09:17:53.098 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_resend": 0, 2024-04-24T09:17:53.098 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_active": 0, 2024-04-24T09:17:53.098 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_send": 0, 2024-04-24T09:17:53.098 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_resend": 0, 2024-04-24T09:17:53.098 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_active": 0, 2024-04-24T09:17:53.098 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_send": 0, 2024-04-24T09:17:53.099 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_resend": 0, 2024-04-24T09:17:53.099 INFO:teuthology.orchestra.run.smithi130.stdout: "command_active": 0, 2024-04-24T09:17:53.099 INFO:teuthology.orchestra.run.smithi130.stdout: "command_send": 0, 2024-04-24T09:17:53.099 INFO:teuthology.orchestra.run.smithi130.stdout: "command_resend": 0, 2024-04-24T09:17:53.099 INFO:teuthology.orchestra.run.smithi130.stdout: "map_epoch": 101, 2024-04-24T09:17:53.099 INFO:teuthology.orchestra.run.smithi130.stdout: "map_full": 0, 2024-04-24T09:17:53.099 INFO:teuthology.orchestra.run.smithi130.stdout: "map_inc": 9, 2024-04-24T09:17:53.099 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_sessions": 7, 2024-04-24T09:17:53.099 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_open": 7, 2024-04-24T09:17:53.100 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_close": 0, 2024-04-24T09:17:53.100 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_laggy": 0, 2024-04-24T09:17:53.100 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_wr": 0, 2024-04-24T09:17:53.100 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_rd": 2, 2024-04-24T09:17:53.100 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_del": 0 2024-04-24T09:17:53.100 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.100 INFO:teuthology.orchestra.run.smithi130.stdout: "oft": { 2024-04-24T09:17:53.100 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_objs": 0, 2024-04-24T09:17:53.100 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:17:53.101 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_updates": 0, 2024-04-24T09:17:53.101 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_removes": 0 2024-04-24T09:17:53.101 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.101 INFO:teuthology.orchestra.run.smithi130.stdout: "osd-slow-ops": { 2024-04-24T09:17:53.101 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_ops_count": 0 2024-04-24T09:17:53.101 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.101 INFO:teuthology.orchestra.run.smithi130.stdout: "purge_queue": { 2024-04-24T09:17:53.101 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops": 0, 2024-04-24T09:17:53.101 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T09:17:53.102 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing": 0, 2024-04-24T09:17:53.102 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_high_water": 0, 2024-04-24T09:17:53.102 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed_ops": 0, 2024-04-24T09:17:53.102 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed": 0, 2024-04-24T09:17:53.102 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_item_in_journal": 0 2024-04-24T09:17:53.102 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.102 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:17:53.102 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-24T09:17:53.102 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-24T09:17:53.103 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-24T09:17:53.103 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 2801, 2024-04-24T09:17:53.103 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 33329402, 2024-04-24T09:17:53.103 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:53.103 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 2801, 2024-04-24T09:17:53.103 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-24T09:17:53.103 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-24T09:17:53.103 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2801, 2024-04-24T09:17:53.103 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 33329402, 2024-04-24T09:17:53.103 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-24T09:17:53.104 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.104 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.104 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.104 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:17:53.104 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.104 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_bytes": { 2024-04-24T09:17:53.104 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-24T09:17:53.104 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-24T09:17:53.104 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-24T09:17:53.104 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-24T09:17:53.105 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-24T09:17:53.105 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:53.105 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-24T09:17:53.105 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 2534, 2024-04-24T09:17:53.105 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 31221110, 2024-04-24T09:17:53.105 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2534, 2024-04-24T09:17:53.105 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 31221110, 2024-04-24T09:17:53.105 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-24T09:17:53.105 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.106 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.106 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.106 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:17:53.106 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.106 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_ops": { 2024-04-24T09:17:53.106 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-24T09:17:53.106 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 1024, 2024-04-24T09:17:53.106 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-24T09:17:53.107 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-24T09:17:53.107 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-24T09:17:53.107 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:53.107 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-24T09:17:53.107 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 2534, 2024-04-24T09:17:53.107 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 2534, 2024-04-24T09:17:53.107 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2534, 2024-04-24T09:17:53.107 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 2534, 2024-04-24T09:17:53.107 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-24T09:17:53.107 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.108 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.108 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.108 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:17:53.108 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.108 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:17:53.108 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-24T09:17:53.108 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-04-24T09:17:53.108 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-24T09:17:53.108 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-24T09:17:53.109 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-24T09:17:53.109 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:53.109 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-24T09:17:53.109 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-24T09:17:53.109 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-24T09:17:53.109 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-04-24T09:17:53.109 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-04-24T09:17:53.109 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-24T09:17:53.109 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.110 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.110 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.110 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:17:53.110 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:53.110 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle-0x5559eb3e0780": { 2024-04-24T09:17:53.110 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-24T09:17:53.110 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-04-24T09:17:53.110 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-24T09:17:53.110 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-24T09:17:53.111 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-24T09:17:53.111 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:53.111 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-24T09:17:53.111 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-24T09:17:53.111 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-24T09:17:53.111 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-04-24T09:17:53.111 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-04-24T09:17:53.111 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-24T09:17:53.111 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:53.112 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:53.112 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:53.112 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:17:53.112 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:17:53.112 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-04-24T09:17:53.552 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:53 smithi130 ceph-mon[31553]: pgmap v335: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:53.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:53 smithi081 ceph-mon[30961]: pgmap v335: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:53.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:53 smithi069 ceph-mon[28170]: pgmap v335: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:54.005 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-24T09:17:54.006 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-24T09:17:54.006 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-24T09:17:54.006 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-24T09:17:54.006 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-24T09:17:54.006 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-24T09:17:54.006 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T09:17:54.006 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-24T09:17:54.006 DEBUG:tasks.check_counter:Getting stats from f 2024-04-24T09:17:54.007 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-24T09:17:54.441 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:17:55.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:55 smithi081 ceph-mon[30961]: pgmap v336: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:55.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:55 smithi069 ceph-mon[28170]: pgmap v336: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:55.651 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:55 smithi130 ceph-mon[31553]: pgmap v336: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:56.791 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-04-24T09:17:56.792 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:17:56.792 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 242, 2024-04-24T09:17:56.792 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 231, 2024-04-24T09:17:56.792 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 1595200, 2024-04-24T09:17:56.792 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 142144, 2024-04-24T09:17:56.792 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2024-04-24T09:17:56.792 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 0, 2024-04-24T09:17:56.793 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.117168455, 2024-04-24T09:17:56.793 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.034908802, 2024-04-24T09:17:56.793 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.097729899, 2024-04-24T09:17:56.793 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.011503806, 2024-04-24T09:17:56.793 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:56.793 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 230, 2024-04-24T09:17:56.793 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.026751129, 2024-04-24T09:17:56.793 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000116309 2024-04-24T09:17:56.793 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:56.794 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:56.794 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:56.794 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:56.794 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:56.794 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:56.794 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 1595200, 2024-04-24T09:17:56.794 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 142144 2024-04-24T09:17:56.795 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:56.795 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:17:56.795 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 2, 2024-04-24T09:17:56.795 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 177, 2024-04-24T09:17:56.795 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 320, 2024-04-24T09:17:56.795 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 122944, 2024-04-24T09:17:56.795 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 5, 2024-04-24T09:17:56.795 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 3, 2024-04-24T09:17:56.796 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.026774870, 2024-04-24T09:17:56.796 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.015978708, 2024-04-24T09:17:56.796 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.005227882, 2024-04-24T09:17:56.796 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:17:56.796 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:56.796 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 174, 2024-04-24T09:17:56.796 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.026454288, 2024-04-24T09:17:56.796 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000152036 2024-04-24T09:17:56.796 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:56.797 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:56.797 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:56.797 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:56.797 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:56.797 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:56.797 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T09:17:56.797 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 122944 2024-04-24T09:17:56.797 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:56.797 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:17:56.798 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 4, 2024-04-24T09:17:56.798 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 4, 2024-04-24T09:17:56.798 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 2992, 2024-04-24T09:17:56.798 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 448, 2024-04-24T09:17:56.798 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 2, 2024-04-24T09:17:56.798 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 1, 2024-04-24T09:17:56.798 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.001342318, 2024-04-24T09:17:56.798 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.000092555, 2024-04-24T09:17:56.799 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.001191719, 2024-04-24T09:17:56.799 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:17:56.799 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:17:56.799 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 3, 2024-04-24T09:17:56.799 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.001550558, 2024-04-24T09:17:56.799 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000516852 2024-04-24T09:17:56.799 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:56.799 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:17:56.799 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:56.800 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:56.800 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:56.800 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:56.800 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-24T09:17:56.800 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T09:17:56.800 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:56.800 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-04-24T09:17:56.800 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 0, 2024-04-24T09:17:56.801 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-04-24T09:17:56.801 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:56.801 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-04-24T09:17:56.801 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:17:56.801 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-04-24T09:17:56.801 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:17:56.801 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:17:56.801 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:17:56.802 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:17:56.802 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:17:56.802 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:17:56.802 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:17:56.802 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:17:56.802 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:17:56.802 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:17:56.802 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:17:56.802 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:17:56.803 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:17:56.803 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-04-24T09:17:56.803 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:17:56.803 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-04-24T09:17:56.803 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:17:56.803 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:17:56.803 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:17:56.803 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:17:56.803 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:17:56.804 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:17:56.804 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:17:56.804 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-04-24T09:17:56.804 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:17:56.804 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:17:56.804 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:17:56.804 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-04-24T09:17:56.804 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-04-24T09:17:56.804 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-04-24T09:17:56.804 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:17:56.805 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:17:56.805 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:17:56.805 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:17:56.805 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 148714, 2024-04-24T09:17:56.805 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 30, 2024-04-24T09:17:56.805 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:17:56.805 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-04-24T09:17:56.805 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-04-24T09:17:56.805 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-04-24T09:17:56.805 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:17:56.806 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-04-24T09:17:56.806 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:17:56.806 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-04-24T09:17:56.806 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 0, 2024-04-24T09:17:56.806 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 0, 2024-04-24T09:17:56.806 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:17:56.806 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:17:56.806 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-04-24T09:17:56.806 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-04-24T09:17:56.807 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 928, 2024-04-24T09:17:56.807 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 2, 2024-04-24T09:17:56.807 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-04-24T09:17:56.807 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-04-24T09:17:56.807 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-04-24T09:17:56.807 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-04-24T09:17:56.807 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:17:56.807 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:17:56.808 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-24T09:17:56.808 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-24T09:17:56.808 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-24T09:17:56.808 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 244, 2024-04-24T09:17:56.808 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 1561068, 2024-04-24T09:17:56.808 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-24T09:17:56.808 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 244, 2024-04-24T09:17:56.809 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-24T09:17:56.809 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-24T09:17:56.809 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 244, 2024-04-24T09:17:56.809 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 1561068, 2024-04-24T09:17:56.809 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-24T09:17:56.809 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:17:56.809 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:17:56.809 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:17:56.810 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:17:56.810 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:17:56.810 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-04-24T09:17:57.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:57 smithi130 ceph-mon[31553]: pgmap v337: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:57.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:57 smithi081 ceph-mon[30961]: pgmap v337: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:57.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:57 smithi069 ceph-mon[28170]: pgmap v337: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:17:57.602 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-04-24T09:17:57.602 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-04-24T09:17:57.602 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.f 2024-04-24T09:17:57.603 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.f 2024-04-24T09:17:57.603 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-04-24T09:17:57.603 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-04-24T09:17:57.603 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-04-24T09:17:57.603 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-04-24T09:17:57.603 DEBUG:tasks.check_counter:Getting stats from i 2024-04-24T09:17:57.603 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-24T09:17:58.040 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:17:59.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:17:59 smithi130 ceph-mon[31553]: pgmap v338: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:59.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:17:59 smithi081 ceph-mon[30961]: pgmap v338: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:17:59.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:17:59 smithi069 ceph-mon[28170]: pgmap v338: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:18:00.366 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-04-24T09:18:00.366 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:18:00.366 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 1439, 2024-04-24T09:18:00.366 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 1417, 2024-04-24T09:18:00.366 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 10414810, 2024-04-24T09:18:00.367 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 516612, 2024-04-24T09:18:00.367 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 6, 2024-04-24T09:18:00.367 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 2, 2024-04-24T09:18:00.367 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.396178527, 2024-04-24T09:18:00.367 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.121562412, 2024-04-24T09:18:00.367 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.183495862, 2024-04-24T09:18:00.367 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.088145387, 2024-04-24T09:18:00.367 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:00.367 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1416, 2024-04-24T09:18:00.368 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.136907330, 2024-04-24T09:18:00.368 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000096685 2024-04-24T09:18:00.368 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.368 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:00.368 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.368 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.368 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.368 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.368 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 1664704, 2024-04-24T09:18:00.369 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 143280 2024-04-24T09:18:00.369 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.369 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:18:00.369 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 301, 2024-04-24T09:18:00.369 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 472, 2024-04-24T09:18:00.369 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 21049706, 2024-04-24T09:18:00.369 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 494032, 2024-04-24T09:18:00.369 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 8, 2024-04-24T09:18:00.369 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 6, 2024-04-24T09:18:00.370 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.161388147, 2024-04-24T09:18:00.370 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.037891129, 2024-04-24T09:18:00.370 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.093963581, 2024-04-24T09:18:00.370 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.017665662, 2024-04-24T09:18:00.370 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:00.370 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 470, 2024-04-24T09:18:00.370 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.056163511, 2024-04-24T09:18:00.371 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000119496 2024-04-24T09:18:00.371 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.371 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:00.371 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.371 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.371 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.371 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.371 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-24T09:18:00.371 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 400160 2024-04-24T09:18:00.371 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.372 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:18:00.372 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 1044, 2024-04-24T09:18:00.372 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 1046, 2024-04-24T09:18:00.372 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 26205606, 2024-04-24T09:18:00.372 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 331999, 2024-04-24T09:18:00.372 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 6, 2024-04-24T09:18:00.372 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 5, 2024-04-24T09:18:00.372 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.294910662, 2024-04-24T09:18:00.373 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.080066812, 2024-04-24T09:18:00.373 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.134368953, 2024-04-24T09:18:00.373 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.059976694, 2024-04-24T09:18:00.373 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:00.373 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1044, 2024-04-24T09:18:00.373 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.102353937, 2024-04-24T09:18:00.373 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000098040 2024-04-24T09:18:00.373 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.373 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:00.374 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.374 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.374 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.374 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.374 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:18:00.374 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:18:00.374 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.374 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-04-24T09:18:00.374 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 1, 2024-04-24T09:18:00.375 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-04-24T09:18:00.375 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.375 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-MDSRank": { 2024-04-24T09:18:00.375 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-04-24T09:18:00.375 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-04-24T09:18:00.375 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 4271, 2024-04-24T09:18:00.375 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.242504051, 2024-04-24T09:18:00.375 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000056779 2024-04-24T09:18:00.375 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:18:00.376 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.376 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-PurgeQueue": { 2024-04-24T09:18:00.376 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-04-24T09:18:00.376 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-04-24T09:18:00.376 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.376 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.376 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.376 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:18:00.376 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.377 INFO:teuthology.orchestra.run.smithi130.stdout: "mds": { 2024-04-24T09:18:00.377 INFO:teuthology.orchestra.run.smithi130.stdout: "request": 0, 2024-04-24T09:18:00.377 INFO:teuthology.orchestra.run.smithi130.stdout: "reply": 0, 2024-04-24T09:18:00.377 INFO:teuthology.orchestra.run.smithi130.stdout: "reply_latency": { 2024-04-24T09:18:00.377 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.377 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.377 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.377 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.377 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_reply": 0, 2024-04-24T09:18:00.377 INFO:teuthology.orchestra.run.smithi130.stdout: "forward": 0, 2024-04-24T09:18:00.378 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_complete": 0, 2024-04-24T09:18:00.378 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_keys": 0, 2024-04-24T09:18:00.378 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_commit": 0, 2024-04-24T09:18:00.378 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_split": 0, 2024-04-24T09:18:00.378 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_merge": 0, 2024-04-24T09:18:00.378 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes": 6024, 2024-04-24T09:18:00.378 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_top": 1196, 2024-04-24T09:18:00.378 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_bottom": 4828, 2024-04-24T09:18:00.379 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pin_tail": 0, 2024-04-24T09:18:00.379 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pinned": 4315, 2024-04-24T09:18:00.379 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_expired": 0, 2024-04-24T09:18:00.379 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_with_caps": 0, 2024-04-24T09:18:00.379 INFO:teuthology.orchestra.run.smithi130.stdout: "caps": 0, 2024-04-24T09:18:00.379 INFO:teuthology.orchestra.run.smithi130.stdout: "subtrees": 81, 2024-04-24T09:18:00.379 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse": 0, 2024-04-24T09:18:00.379 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_hit": 0, 2024-04-24T09:18:00.379 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_forward": 0, 2024-04-24T09:18:00.380 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_discover": 0, 2024-04-24T09:18:00.380 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:18:00.380 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_remote_ino": 0, 2024-04-24T09:18:00.380 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_lock": 0, 2024-04-24T09:18:00.380 INFO:teuthology.orchestra.run.smithi130.stdout: "load_cent": 0, 2024-04-24T09:18:00.380 INFO:teuthology.orchestra.run.smithi130.stdout: "q": 0, 2024-04-24T09:18:00.380 INFO:teuthology.orchestra.run.smithi130.stdout: "exported": 0, 2024-04-24T09:18:00.380 INFO:teuthology.orchestra.run.smithi130.stdout: "exported_inodes": 0, 2024-04-24T09:18:00.380 INFO:teuthology.orchestra.run.smithi130.stdout: "imported": 0, 2024-04-24T09:18:00.381 INFO:teuthology.orchestra.run.smithi130.stdout: "imported_inodes": 0, 2024-04-24T09:18:00.381 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_dir_fetch": 0, 2024-04-24T09:18:00.381 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:18:00.381 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_peer_discover": 0, 2024-04-24T09:18:00.381 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rfiles": 0, 2024-04-24T09:18:00.381 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rbytes": 0, 2024-04-24T09:18:00.382 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rsnaps": 0, 2024-04-24T09:18:00.382 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:18:00.382 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_set_tag": 0, 2024-04-24T09:18:00.382 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:18:00.382 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:18:00.383 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:18:00.383 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:18:00.383 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:18:00.383 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_file_inodes": 0, 2024-04-24T09:18:00.383 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:18:00.384 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T09:18:00.384 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_grant": 0, 2024-04-24T09:18:00.384 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:18:00.384 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:18:00.384 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:18:00.384 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps": 0, 2024-04-24T09:18:00.385 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps_dirty": 0, 2024-04-24T09:18:00.385 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_cap_release": 0, 2024-04-24T09:18:00.385 INFO:teuthology.orchestra.run.smithi130.stdout: "process_request_cap_release": 0 2024-04-24T09:18:00.385 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.385 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_cache": { 2024-04-24T09:18:00.385 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update": 0, 2024-04-24T09:18:00.385 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update_receipt": 0, 2024-04-24T09:18:00.386 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_try_discover": 0, 2024-04-24T09:18:00.386 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_send_discover": 0, 2024-04-24T09:18:00.386 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_handle_discover": 0, 2024-04-24T09:18:00.386 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays": 0, 2024-04-24T09:18:00.386 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_delayed": 0, 2024-04-24T09:18:00.386 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:18:00.386 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_created": 0, 2024-04-24T09:18:00.386 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_enqueued": 0, 2024-04-24T09:18:00.386 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_reintegrated": 0, 2024-04-24T09:18:00.387 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_migrated": 0, 2024-04-24T09:18:00.387 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_processing": 0, 2024-04-24T09:18:00.387 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:18:00.387 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:18:00.387 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_started": 0, 2024-04-24T09:18:00.387 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_completed": 0, 2024-04-24T09:18:00.387 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:18:00.387 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:18:00.387 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:18:00.388 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_exportdir": 0, 2024-04-24T09:18:00.388 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_flush": 0, 2024-04-24T09:18:00.388 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:18:00.388 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragstats": 0, 2024-04-24T09:18:00.388 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_inodestats": 0 2024-04-24T09:18:00.388 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.388 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_log": { 2024-04-24T09:18:00.388 INFO:teuthology.orchestra.run.smithi130.stdout: "evlrg": 0, 2024-04-24T09:18:00.388 INFO:teuthology.orchestra.run.smithi130.stdout: "evadd": 0, 2024-04-24T09:18:00.389 INFO:teuthology.orchestra.run.smithi130.stdout: "evex": 0, 2024-04-24T09:18:00.389 INFO:teuthology.orchestra.run.smithi130.stdout: "evtrm": 0, 2024-04-24T09:18:00.389 INFO:teuthology.orchestra.run.smithi130.stdout: "ev": 14046, 2024-04-24T09:18:00.389 INFO:teuthology.orchestra.run.smithi130.stdout: "evexg": 0, 2024-04-24T09:18:00.389 INFO:teuthology.orchestra.run.smithi130.stdout: "evexd": 0, 2024-04-24T09:18:00.389 INFO:teuthology.orchestra.run.smithi130.stdout: "segadd": 0, 2024-04-24T09:18:00.389 INFO:teuthology.orchestra.run.smithi130.stdout: "segex": 0, 2024-04-24T09:18:00.389 INFO:teuthology.orchestra.run.smithi130.stdout: "segtrm": 0, 2024-04-24T09:18:00.390 INFO:teuthology.orchestra.run.smithi130.stdout: "seg": 21, 2024-04-24T09:18:00.390 INFO:teuthology.orchestra.run.smithi130.stdout: "segmjr": 3, 2024-04-24T09:18:00.390 INFO:teuthology.orchestra.run.smithi130.stdout: "segexg": 0, 2024-04-24T09:18:00.390 INFO:teuthology.orchestra.run.smithi130.stdout: "segexd": 0, 2024-04-24T09:18:00.390 INFO:teuthology.orchestra.run.smithi130.stdout: "expos": 4194304, 2024-04-24T09:18:00.390 INFO:teuthology.orchestra.run.smithi130.stdout: "wrpos": 59489353, 2024-04-24T09:18:00.390 INFO:teuthology.orchestra.run.smithi130.stdout: "rdpos": 59488779, 2024-04-24T09:18:00.390 INFO:teuthology.orchestra.run.smithi130.stdout: "jlat": { 2024-04-24T09:18:00.391 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.391 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.391 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.391 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.391 INFO:teuthology.orchestra.run.smithi130.stdout: "replayed": 14046 2024-04-24T09:18:00.391 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.391 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_mem": { 2024-04-24T09:18:00.391 INFO:teuthology.orchestra.run.smithi130.stdout: "ino": 3823, 2024-04-24T09:18:00.391 INFO:teuthology.orchestra.run.smithi130.stdout: "ino+": 8318, 2024-04-24T09:18:00.392 INFO:teuthology.orchestra.run.smithi130.stdout: "ino-": 4495, 2024-04-24T09:18:00.392 INFO:teuthology.orchestra.run.smithi130.stdout: "dir": 249, 2024-04-24T09:18:00.392 INFO:teuthology.orchestra.run.smithi130.stdout: "dir+": 283, 2024-04-24T09:18:00.392 INFO:teuthology.orchestra.run.smithi130.stdout: "dir-": 34, 2024-04-24T09:18:00.392 INFO:teuthology.orchestra.run.smithi130.stdout: "dn": 6024, 2024-04-24T09:18:00.392 INFO:teuthology.orchestra.run.smithi130.stdout: "dn+": 9292, 2024-04-24T09:18:00.392 INFO:teuthology.orchestra.run.smithi130.stdout: "dn-": 3268, 2024-04-24T09:18:00.392 INFO:teuthology.orchestra.run.smithi130.stdout: "cap": 0, 2024-04-24T09:18:00.392 INFO:teuthology.orchestra.run.smithi130.stdout: "cap+": 0, 2024-04-24T09:18:00.393 INFO:teuthology.orchestra.run.smithi130.stdout: "cap-": 0, 2024-04-24T09:18:00.393 INFO:teuthology.orchestra.run.smithi130.stdout: "rss": 66964, 2024-04-24T09:18:00.393 INFO:teuthology.orchestra.run.smithi130.stdout: "heap": 215416 2024-04-24T09:18:00.393 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.393 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_server": { 2024-04-24T09:18:00.393 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_client_request": 0, 2024-04-24T09:18:00.393 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_server_request": 0, 2024-04-24T09:18:00.393 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_request": 0, 2024-04-24T09:18:00.393 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_session": 0, 2024-04-24T09:18:00.394 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_peer_request": 0, 2024-04-24T09:18:00.394 INFO:teuthology.orchestra.run.smithi130.stdout: "req_create_latency": { 2024-04-24T09:18:00.394 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.394 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.394 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.394 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.394 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getattr_latency": { 2024-04-24T09:18:00.394 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.395 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.395 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.395 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.395 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getfilelock_latency": { 2024-04-24T09:18:00.395 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.395 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.395 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.395 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.395 INFO:teuthology.orchestra.run.smithi130.stdout: "req_link_latency": { 2024-04-24T09:18:00.396 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.396 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.396 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.396 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.396 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookup_latency": { 2024-04-24T09:18:00.396 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.396 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.396 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.396 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.397 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookuphash_latency": { 2024-04-24T09:18:00.397 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.397 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.397 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.397 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.397 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupino_latency": { 2024-04-24T09:18:00.397 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.397 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.397 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.397 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.398 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupname_latency": { 2024-04-24T09:18:00.398 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.398 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.398 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.398 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.398 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupparent_latency": { 2024-04-24T09:18:00.398 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.399 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.399 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.399 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.399 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupsnap_latency": { 2024-04-24T09:18:00.399 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.399 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.399 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.399 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.399 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lssnap_latency": { 2024-04-24T09:18:00.400 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.400 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.400 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.400 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.401 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mkdir_latency": { 2024-04-24T09:18:00.401 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.401 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.401 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.401 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.401 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mknod_latency": { 2024-04-24T09:18:00.402 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.402 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.402 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.402 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.402 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mksnap_latency": { 2024-04-24T09:18:00.403 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.403 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.403 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.403 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.403 INFO:teuthology.orchestra.run.smithi130.stdout: "req_open_latency": { 2024-04-24T09:18:00.403 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.403 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.404 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.404 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.404 INFO:teuthology.orchestra.run.smithi130.stdout: "req_readdir_latency": { 2024-04-24T09:18:00.404 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.404 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.404 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.404 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.405 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rename_latency": { 2024-04-24T09:18:00.405 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.405 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.405 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.405 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.405 INFO:teuthology.orchestra.run.smithi130.stdout: "req_renamesnap_latency": { 2024-04-24T09:18:00.405 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.405 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.405 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.406 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.406 INFO:teuthology.orchestra.run.smithi130.stdout: "req_snapdiff_latency": { 2024-04-24T09:18:00.406 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.406 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.406 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.406 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.406 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmdir_latency": { 2024-04-24T09:18:00.406 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.407 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.407 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.407 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.407 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmsnap_latency": { 2024-04-24T09:18:00.407 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.407 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.407 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.407 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.407 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmxattr_latency": { 2024-04-24T09:18:00.408 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.408 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.408 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.408 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.408 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setattr_latency": { 2024-04-24T09:18:00.408 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.408 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.408 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.408 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.409 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setdirlayout_latency": { 2024-04-24T09:18:00.409 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.409 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.409 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.409 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.409 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setfilelock_latency": { 2024-04-24T09:18:00.409 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.409 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.410 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.410 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.410 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setlayout_latency": { 2024-04-24T09:18:00.410 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.410 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.410 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.410 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.410 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setxattr_latency": { 2024-04-24T09:18:00.410 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.411 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.411 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.411 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.411 INFO:teuthology.orchestra.run.smithi130.stdout: "req_symlink_latency": { 2024-04-24T09:18:00.411 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.411 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.411 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.411 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.411 INFO:teuthology.orchestra.run.smithi130.stdout: "req_unlink_latency": { 2024-04-24T09:18:00.412 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.412 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.412 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.412 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.412 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:18:00.412 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:18:00.412 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getvxattr_latency": { 2024-04-24T09:18:00.412 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.413 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.413 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.413 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:18:00.413 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.413 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_sessions": { 2024-04-24T09:18:00.413 INFO:teuthology.orchestra.run.smithi130.stdout: "session_count": 2, 2024-04-24T09:18:00.413 INFO:teuthology.orchestra.run.smithi130.stdout: "session_add": 4, 2024-04-24T09:18:00.413 INFO:teuthology.orchestra.run.smithi130.stdout: "session_remove": 2, 2024-04-24T09:18:00.413 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_open": 4, 2024-04-24T09:18:00.414 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_stale": 0, 2024-04-24T09:18:00.414 INFO:teuthology.orchestra.run.smithi130.stdout: "total_load": 0, 2024-04-24T09:18:00.414 INFO:teuthology.orchestra.run.smithi130.stdout: "average_load": 0, 2024-04-24T09:18:00.414 INFO:teuthology.orchestra.run.smithi130.stdout: "avg_session_uptime": 3533, 2024-04-24T09:18:00.414 INFO:teuthology.orchestra.run.smithi130.stdout: "mdthresh_evicted": 0 2024-04-24T09:18:00.414 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.414 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-04-24T09:18:00.414 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:18:00.414 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-04-24T09:18:00.415 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:18:00.415 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:18:00.415 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:18:00.415 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:18:00.415 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:18:00.415 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:18:00.415 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:18:00.415 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:18:00.415 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:18:00.416 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:18:00.416 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:18:00.416 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:18:00.416 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:18:00.416 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-04-24T09:18:00.416 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:18:00.416 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-04-24T09:18:00.416 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:18:00.416 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:18:00.417 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:18:00.417 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:18:00.417 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:18:00.417 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:18:00.417 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:18:00.417 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-04-24T09:18:00.418 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:18:00.418 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:18:00.418 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:18:00.418 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-04-24T09:18:00.418 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-04-24T09:18:00.419 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-04-24T09:18:00.419 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:18:00.419 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:18:00.419 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:18:00.419 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:18:00.419 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 224065, 2024-04-24T09:18:00.420 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 73, 2024-04-24T09:18:00.420 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:18:00.420 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-04-24T09:18:00.420 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-04-24T09:18:00.420 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-04-24T09:18:00.421 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:18:00.421 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-04-24T09:18:00.421 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:18:00.421 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-04-24T09:18:00.421 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 9320, 2024-04-24T09:18:00.421 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 178, 2024-04-24T09:18:00.422 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:18:00.422 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:18:00.422 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-04-24T09:18:00.422 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-04-24T09:18:00.422 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 12655358, 2024-04-24T09:18:00.422 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 39069, 2024-04-24T09:18:00.423 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-04-24T09:18:00.423 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-04-24T09:18:00.423 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-04-24T09:18:00.423 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-04-24T09:18:00.423 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.424 INFO:teuthology.orchestra.run.smithi130.stdout: "objecter": { 2024-04-24T09:18:00.424 INFO:teuthology.orchestra.run.smithi130.stdout: "op_active": 0, 2024-04-24T09:18:00.424 INFO:teuthology.orchestra.run.smithi130.stdout: "op_laggy": 0, 2024-04-24T09:18:00.424 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send": 2513, 2024-04-24T09:18:00.424 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send_bytes": 21, 2024-04-24T09:18:00.424 INFO:teuthology.orchestra.run.smithi130.stdout: "op_resend": 3, 2024-04-24T09:18:00.425 INFO:teuthology.orchestra.run.smithi130.stdout: "op_reply": 2513, 2024-04-24T09:18:00.425 INFO:teuthology.orchestra.run.smithi130.stdout: "op_latency": { 2024-04-24T09:18:00.425 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2513, 2024-04-24T09:18:00.425 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 3.566933959, 2024-04-24T09:18:00.425 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.001419392 2024-04-24T09:18:00.425 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.425 INFO:teuthology.orchestra.run.smithi130.stdout: "op_inflight": 0, 2024-04-24T09:18:00.425 INFO:teuthology.orchestra.run.smithi130.stdout: "oplen_avg": { 2024-04-24T09:18:00.425 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2513, 2024-04-24T09:18:00.426 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 2514 2024-04-24T09:18:00.426 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.426 INFO:teuthology.orchestra.run.smithi130.stdout: "op": 2513, 2024-04-24T09:18:00.426 INFO:teuthology.orchestra.run.smithi130.stdout: "op_r": 2513, 2024-04-24T09:18:00.426 INFO:teuthology.orchestra.run.smithi130.stdout: "op_w": 0, 2024-04-24T09:18:00.426 INFO:teuthology.orchestra.run.smithi130.stdout: "op_rmw": 0, 2024-04-24T09:18:00.426 INFO:teuthology.orchestra.run.smithi130.stdout: "op_pg": 0, 2024-04-24T09:18:00.426 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_stat": 1451, 2024-04-24T09:18:00.427 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_create": 0, 2024-04-24T09:18:00.427 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_read": 1060, 2024-04-24T09:18:00.427 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_write": 0, 2024-04-24T09:18:00.427 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writefull": 0, 2024-04-24T09:18:00.427 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writesame": 0, 2024-04-24T09:18:00.427 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_append": 0, 2024-04-24T09:18:00.427 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_zero": 0, 2024-04-24T09:18:00.427 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_truncate": 0, 2024-04-24T09:18:00.427 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_delete": 0, 2024-04-24T09:18:00.428 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_mapext": 0, 2024-04-24T09:18:00.428 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_sparse_read": 0, 2024-04-24T09:18:00.428 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_clonerange": 0, 2024-04-24T09:18:00.428 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_getxattr": 1, 2024-04-24T09:18:00.428 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_setxattr": 0, 2024-04-24T09:18:00.428 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:18:00.428 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_rmxattr": 0, 2024-04-24T09:18:00.428 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:18:00.428 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_call": 0, 2024-04-24T09:18:00.429 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_watch": 0, 2024-04-24T09:18:00.429 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_notify": 0, 2024-04-24T09:18:00.429 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:18:00.429 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls": 0, 2024-04-24T09:18:00.429 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:18:00.429 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_other": 0, 2024-04-24T09:18:00.429 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_active": 0, 2024-04-24T09:18:00.429 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_send": 0, 2024-04-24T09:18:00.429 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_resend": 0, 2024-04-24T09:18:00.430 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_ping": 0, 2024-04-24T09:18:00.430 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_active": 0, 2024-04-24T09:18:00.430 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_send": 0, 2024-04-24T09:18:00.430 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_resend": 0, 2024-04-24T09:18:00.430 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_active": 0, 2024-04-24T09:18:00.430 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_send": 0, 2024-04-24T09:18:00.430 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_resend": 0, 2024-04-24T09:18:00.430 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_active": 0, 2024-04-24T09:18:00.431 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_send": 0, 2024-04-24T09:18:00.431 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_resend": 0, 2024-04-24T09:18:00.431 INFO:teuthology.orchestra.run.smithi130.stdout: "command_active": 0, 2024-04-24T09:18:00.431 INFO:teuthology.orchestra.run.smithi130.stdout: "command_send": 0, 2024-04-24T09:18:00.431 INFO:teuthology.orchestra.run.smithi130.stdout: "command_resend": 0, 2024-04-24T09:18:00.431 INFO:teuthology.orchestra.run.smithi130.stdout: "map_epoch": 101, 2024-04-24T09:18:00.431 INFO:teuthology.orchestra.run.smithi130.stdout: "map_full": 0, 2024-04-24T09:18:00.431 INFO:teuthology.orchestra.run.smithi130.stdout: "map_inc": 9, 2024-04-24T09:18:00.431 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_sessions": 9, 2024-04-24T09:18:00.432 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_open": 9, 2024-04-24T09:18:00.432 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_close": 0, 2024-04-24T09:18:00.432 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_laggy": 0, 2024-04-24T09:18:00.432 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_wr": 0, 2024-04-24T09:18:00.432 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_rd": 2, 2024-04-24T09:18:00.432 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_del": 0 2024-04-24T09:18:00.432 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.432 INFO:teuthology.orchestra.run.smithi130.stdout: "oft": { 2024-04-24T09:18:00.432 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_objs": 0, 2024-04-24T09:18:00.433 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:18:00.433 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_updates": 0, 2024-04-24T09:18:00.433 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_removes": 0 2024-04-24T09:18:00.433 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.433 INFO:teuthology.orchestra.run.smithi130.stdout: "osd-slow-ops": { 2024-04-24T09:18:00.433 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_ops_count": 0 2024-04-24T09:18:00.433 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.433 INFO:teuthology.orchestra.run.smithi130.stdout: "purge_queue": { 2024-04-24T09:18:00.434 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops": 0, 2024-04-24T09:18:00.434 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T09:18:00.434 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing": 0, 2024-04-24T09:18:00.434 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_high_water": 0, 2024-04-24T09:18:00.434 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed_ops": 0, 2024-04-24T09:18:00.434 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed": 0, 2024-04-24T09:18:00.434 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_item_in_journal": 0 2024-04-24T09:18:00.434 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.435 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:18:00.435 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-24T09:18:00.435 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-24T09:18:00.435 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-24T09:18:00.435 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 2780, 2024-04-24T09:18:00.435 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 57403362, 2024-04-24T09:18:00.435 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:00.435 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 2780, 2024-04-24T09:18:00.435 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-24T09:18:00.435 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-24T09:18:00.436 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2780, 2024-04-24T09:18:00.436 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 57403362, 2024-04-24T09:18:00.436 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-24T09:18:00.436 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.436 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.436 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.436 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:18:00.436 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.437 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_bytes": { 2024-04-24T09:18:00.437 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-24T09:18:00.437 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-24T09:18:00.437 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-24T09:18:00.437 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-24T09:18:00.437 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-24T09:18:00.437 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:00.437 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:00.437 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 2513, 2024-04-24T09:18:00.437 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 55296894, 2024-04-24T09:18:00.438 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2513, 2024-04-24T09:18:00.438 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 55296894, 2024-04-24T09:18:00.438 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-24T09:18:00.438 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.438 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.438 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.438 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:18:00.439 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.439 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_ops": { 2024-04-24T09:18:00.439 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-24T09:18:00.439 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 1024, 2024-04-24T09:18:00.439 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-24T09:18:00.439 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-24T09:18:00.440 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-24T09:18:00.440 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:00.440 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:00.440 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 2513, 2024-04-24T09:18:00.440 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 2513, 2024-04-24T09:18:00.441 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2513, 2024-04-24T09:18:00.441 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 2513, 2024-04-24T09:18:00.441 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-24T09:18:00.441 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.441 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.442 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.442 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:18:00.442 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.442 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:18:00.442 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-24T09:18:00.442 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-04-24T09:18:00.442 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-24T09:18:00.442 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-24T09:18:00.443 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-24T09:18:00.443 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:00.443 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:00.443 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-24T09:18:00.443 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-24T09:18:00.443 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-04-24T09:18:00.443 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-04-24T09:18:00.443 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-24T09:18:00.443 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.444 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.444 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.444 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:18:00.444 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:00.444 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle-0x5643c643c780": { 2024-04-24T09:18:00.444 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-24T09:18:00.444 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-04-24T09:18:00.444 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-24T09:18:00.444 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-24T09:18:00.445 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-24T09:18:00.445 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:00.445 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:00.445 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-24T09:18:00.445 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-24T09:18:00.445 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-04-24T09:18:00.445 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-04-24T09:18:00.445 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-24T09:18:00.446 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:00.446 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:00.446 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:00.446 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:18:00.446 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:18:00.446 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-04-24T09:18:01.443 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-24T09:18:01.443 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-24T09:18:01.443 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-24T09:18:01.443 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-04-24T09:18:01.444 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-04-24T09:18:01.444 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-24T09:18:01.444 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T09:18:01.444 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-24T09:18:01.444 DEBUG:tasks.check_counter:Getting stats from l 2024-04-24T09:18:01.445 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-24T09:18:01.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:01 smithi069 ceph-mon[28170]: pgmap v339: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:18:01.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:01 smithi130 ceph-mon[31553]: pgmap v339: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:18:01.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:01 smithi081 ceph-mon[30961]: pgmap v339: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T09:18:01.893 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:18:03.584 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:03 smithi130 ceph-mon[31553]: pgmap v340: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:18:03.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:03 smithi069 ceph-mon[28170]: pgmap v340: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:18:03.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:03 smithi081 ceph-mon[30961]: pgmap v340: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 0, 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 2, 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 0, 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 0, 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 0, 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.003503367, 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.000762987, 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 6, 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 184, 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 3312, 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 124272, 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 5, 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 3, 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.028572458, 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.018340181, 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.004474587, 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 182, 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.032871152, 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000180610 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 124272 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 249, 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 238, 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 1597104, 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 146624, 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 2, 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 1, 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.115250752, 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.035683288, 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.097562324, 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.011749799, 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 237, 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.025660101, 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000108270 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 1597104, 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 146624 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 0, 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 148714, 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 30, 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 0, 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 0, 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 928, 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 2, 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 251, 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 1561978, 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 251, 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 251, 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 1561978, 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-04-24T09:18:05.048 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-24T09:18:05.048 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-24T09:18:05.048 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-24T09:18:05.048 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-24T09:18:05.049 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-24T09:18:05.049 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-24T09:18:05.049 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-24T09:18:05.049 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-24T09:18:05.049 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-24T09:18:05.061 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-24T09:18:05.061 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T09:18:05.062 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T09:18:05.120 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T09:18:05.120 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T09:18:05.171 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:18:05.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.381+0000 7ff6ccc00640 1 -- 172.21.15.69:0/3709574971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6c8069400 msgr2=0x7ff6c80697e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:05.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.381+0000 7ff6ccc00640 1 --2- 172.21.15.69:0/3709574971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6c8069400 0x7ff6c80697e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8009900 tx=0x7ff6b802a410 comp rx=0 tx=0).stop 2024-04-24T09:18:05.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.381+0000 7ff6ccc00640 1 -- 172.21.15.69:0/3709574971 shutdown_connections 2024-04-24T09:18:05.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.381+0000 7ff6ccc00640 1 --2- 172.21.15.69:0/3709574971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c81071a0 0x7ff6c810ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:05.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.381+0000 7ff6ccc00640 1 --2- 172.21.15.69:0/3709574971 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6c8069db0 0x7ff6c8106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:05.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.381+0000 7ff6ccc00640 1 --2- 172.21.15.69:0/3709574971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6c8069400 0x7ff6c80697e0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:05.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.381+0000 7ff6ccc00640 1 -- 172.21.15.69:0/3709574971 >> 172.21.15.69:0/3709574971 conn(0x7ff6c8076840 msgr2=0x7ff6c8076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:05.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.381+0000 7ff6ccc00640 1 -- 172.21.15.69:0/3709574971 shutdown_connections 2024-04-24T09:18:05.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.381+0000 7ff6ccc00640 1 -- 172.21.15.69:0/3709574971 wait complete. 2024-04-24T09:18:05.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.382+0000 7ff6ccc00640 1 Processor -- start 2024-04-24T09:18:05.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.382+0000 7ff6ccc00640 1 -- start start 2024-04-24T09:18:05.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.383+0000 7ff6ccc00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6c8069400 0x7ff6c8198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:05.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.383+0000 7ff6ccc00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c81071a0 0x7ff6c8198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:05.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.383+0000 7ff6ccc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6c819cef0 0x7ff6c819d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:05.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.383+0000 7ff6ccc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c81120d0 con 0x7ff6c819cef0 2024-04-24T09:18:05.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.383+0000 7ff6ccc00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c8111f50 con 0x7ff6c81071a0 2024-04-24T09:18:05.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.383+0000 7ff6ccc00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c8112250 con 0x7ff6c8069400 2024-04-24T09:18:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.383+0000 7ff6c7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6c819cef0 0x7ff6c819d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.383+0000 7ff6c7e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6c819cef0 0x7ff6c819d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55732/0 (socket says 172.21.15.69:55732) 2024-04-24T09:18:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.383+0000 7ff6c7e00640 1 -- 172.21.15.69:0/1793162189 learned_addr learned my addr 172.21.15.69:0/1793162189 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:18:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.383+0000 7ff6bfe00640 1 --2- 172.21.15.69:0/1793162189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c81071a0 0x7ff6c8198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.383+0000 7ff6c7400640 1 --2- 172.21.15.69:0/1793162189 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6c8069400 0x7ff6c8198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.384+0000 7ff6c7e00640 1 -- 172.21.15.69:0/1793162189 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6c8069400 msgr2=0x7ff6c8198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.384+0000 7ff6c7e00640 1 --2- 172.21.15.69:0/1793162189 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6c8069400 0x7ff6c8198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.384+0000 7ff6c7e00640 1 -- 172.21.15.69:0/1793162189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c81071a0 msgr2=0x7ff6c8198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:05.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.384+0000 7ff6c7e00640 1 --2- 172.21.15.69:0/1793162189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c81071a0 0x7ff6c8198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:05.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.384+0000 7ff6c7e00640 1 -- 172.21.15.69:0/1793162189 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c819d9e0 con 0x7ff6c819cef0 2024-04-24T09:18:05.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.384+0000 7ff6c7e00640 1 --2- 172.21.15.69:0/1793162189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6c819cef0 0x7ff6c819d3a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000eaa0 tx=0x7ff6c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:05.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.384+0000 7ff6c4c00640 1 -- 172.21.15.69:0/1793162189 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c000cdb0 con 0x7ff6c819cef0 2024-04-24T09:18:05.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.384+0000 7ff6c4c00640 1 -- 172.21.15.69:0/1793162189 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6c0005020 con 0x7ff6c819cef0 2024-04-24T09:18:05.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.384+0000 7ff6ccc00640 1 -- 172.21.15.69:0/1793162189 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c819dcd0 con 0x7ff6c819cef0 2024-04-24T09:18:05.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.384+0000 7ff6c4c00640 1 -- 172.21.15.69:0/1793162189 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c00116d0 con 0x7ff6c819cef0 2024-04-24T09:18:05.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.384+0000 7ff6ccc00640 1 -- 172.21.15.69:0/1793162189 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c81a5470 con 0x7ff6c819cef0 2024-04-24T09:18:05.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.385+0000 7ff6ccc00640 1 -- 172.21.15.69:0/1793162189 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c8068d40 con 0x7ff6c819cef0 2024-04-24T09:18:05.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.387+0000 7ff6c4c00640 1 -- 172.21.15.69:0/1793162189 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff6c00051d0 con 0x7ff6c819cef0 2024-04-24T09:18:05.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.389+0000 7ff6c4c00640 1 --2- 172.21.15.69:0/1793162189 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff6ac0a3d20 0x7ff6ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:05.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.389+0000 7ff6c4c00640 1 -- 172.21.15.69:0/1793162189 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7ff6c00d0e20 con 0x7ff6c819cef0 2024-04-24T09:18:05.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.389+0000 7ff6c7400640 1 --2- 172.21.15.69:0/1793162189 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff6ac0a3d20 0x7ff6ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:05.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.390+0000 7ff6c7400640 1 --2- 172.21.15.69:0/1793162189 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff6ac0a3d20 0x7ff6ac0a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8031440 tx=0x7ff6b80095f0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:05.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.393+0000 7ff6c4c00640 1 -- 172.21.15.69:0/1793162189 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6c0016030 con 0x7ff6c819cef0 2024-04-24T09:18:05.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:05 smithi069 ceph-mon[28170]: pgmap v341: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:18:05.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.598+0000 7ff6ccc00640 1 -- 172.21.15.69:0/1793162189 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6c8069db0 con 0x7ff6c819cef0 2024-04-24T09:18:05.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.602+0000 7ff6c4c00640 1 -- 172.21.15.69:0/1793162189 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+25127 (secure 0 0 0) 0x7ff6c009d0d0 con 0x7ff6c819cef0 2024-04-24T09:18:05.602 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:18:05.602 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":101,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:17:12.920525+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/660205566":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/1098677190":"2024-04-25T09:07:03.863940+0000","172.21.15.69:0/3145713060":"2024-04-25T09:07:03.863940+0000","172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/3008138297":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/2901345957":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/1098677190":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:18:05.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.606+0000 7ff6ccc00640 1 -- 172.21.15.69:0/1793162189 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff6ac0a3d20 msgr2=0x7ff6ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:05.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.606+0000 7ff6ccc00640 1 --2- 172.21.15.69:0/1793162189 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff6ac0a3d20 0x7ff6ac0a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8031440 tx=0x7ff6b80095f0 comp rx=0 tx=0).stop 2024-04-24T09:18:05.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.606+0000 7ff6ccc00640 1 -- 172.21.15.69:0/1793162189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6c819cef0 msgr2=0x7ff6c819d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:05.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.606+0000 7ff6ccc00640 1 --2- 172.21.15.69:0/1793162189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6c819cef0 0x7ff6c819d3a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000eaa0 tx=0x7ff6c000ef60 comp rx=0 tx=0).stop 2024-04-24T09:18:05.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.606+0000 7ff6ccc00640 1 -- 172.21.15.69:0/1793162189 shutdown_connections 2024-04-24T09:18:05.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.606+0000 7ff6ccc00640 1 --2- 172.21.15.69:0/1793162189 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff6ac0a3d20 0x7ff6ac0a61e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:05.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.606+0000 7ff6ccc00640 1 --2- 172.21.15.69:0/1793162189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6c819cef0 0x7ff6c819d3a0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:05.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.606+0000 7ff6ccc00640 1 --2- 172.21.15.69:0/1793162189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c81071a0 0x7ff6c8198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:05.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.606+0000 7ff6ccc00640 1 --2- 172.21.15.69:0/1793162189 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6c8069400 0x7ff6c8198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:05.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.606+0000 7ff6ccc00640 1 -- 172.21.15.69:0/1793162189 >> 172.21.15.69:0/1793162189 conn(0x7ff6c8076840 msgr2=0x7ff6c80ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:05.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.606+0000 7ff6ccc00640 1 -- 172.21.15.69:0/1793162189 shutdown_connections 2024-04-24T09:18:05.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.606+0000 7ff6ccc00640 1 -- 172.21.15.69:0/1793162189 wait complete. 2024-04-24T09:18:05.620 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:05 smithi081 ceph-mon[30961]: pgmap v341: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:18:05.626 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:57:37.377599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:04:00.299827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:04:00.958498+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T09:04:04.105314+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T09:18:05.626 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:18:05.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:05 smithi130 ceph-mon[31553]: pgmap v341: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T09:18:05.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.823+0000 7f4dabe00640 1 -- 172.21.15.69:0/1974063055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dac104cd0 msgr2=0x7f4dac108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:05.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.823+0000 7f4dabe00640 1 --2- 172.21.15.69:0/1974063055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dac104cd0 0x7f4dac108d10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009900 tx=0x7f4d9402a410 comp rx=0 tx=0).stop 2024-04-24T09:18:05.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.823+0000 7f4dabe00640 1 -- 172.21.15.69:0/1974063055 shutdown_connections 2024-04-24T09:18:05.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.823+0000 7f4dabe00640 1 --2- 172.21.15.69:0/1974063055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4dac1093f0 0x7f4dac110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:05.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.823+0000 7f4dabe00640 1 --2- 172.21.15.69:0/1974063055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dac104cd0 0x7f4dac108d10 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:05.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.823+0000 7f4dabe00640 1 --2- 172.21.15.69:0/1974063055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dac104320 0x7f4dac104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:05.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.823+0000 7f4dabe00640 1 -- 172.21.15.69:0/1974063055 >> 172.21.15.69:0/1974063055 conn(0x7f4dac0ffb40 msgr2=0x7f4dac101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:05.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.823+0000 7f4dabe00640 1 -- 172.21.15.69:0/1974063055 shutdown_connections 2024-04-24T09:18:05.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.823+0000 7f4dabe00640 1 -- 172.21.15.69:0/1974063055 wait complete. 2024-04-24T09:18:05.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.824+0000 7f4dabe00640 1 Processor -- start 2024-04-24T09:18:05.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.824+0000 7f4dabe00640 1 -- start start 2024-04-24T09:18:05.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.825+0000 7f4dabe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dac104320 0x7f4dac1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:05.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.825+0000 7f4dabe00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4dac1093f0 0x7f4dac1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:05.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.825+0000 7f4dabe00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dac19a750 0x7f4dac19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:05.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.825+0000 7f4dabe00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac114350 con 0x7f4dac19a750 2024-04-24T09:18:05.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.825+0000 7f4dabe00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac1141d0 con 0x7f4dac1093f0 2024-04-24T09:18:05.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.825+0000 7f4dabe00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac1144d0 con 0x7f4dac104320 2024-04-24T09:18:05.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.825+0000 7f4dab400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dac19a750 0x7f4dac19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:05.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.825+0000 7f4dab400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dac19a750 0x7f4dac19abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55752/0 (socket says 172.21.15.69:55752) 2024-04-24T09:18:05.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.825+0000 7f4dab400640 1 -- 172.21.15.69:0/944400253 learned_addr learned my addr 172.21.15.69:0/944400253 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:18:05.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.825+0000 7f4daa000640 1 --2- 172.21.15.69:0/944400253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4dac1093f0 0x7f4dac1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:05.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.825+0000 7f4daaa00640 1 --2- 172.21.15.69:0/944400253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dac104320 0x7f4dac1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:05.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.826+0000 7f4dab400640 1 -- 172.21.15.69:0/944400253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dac104320 msgr2=0x7f4dac1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:05.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.826+0000 7f4dab400640 1 --2- 172.21.15.69:0/944400253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dac104320 0x7f4dac1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:05.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.826+0000 7f4dab400640 1 -- 172.21.15.69:0/944400253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4dac1093f0 msgr2=0x7f4dac1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:05.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.826+0000 7f4dab400640 1 --2- 172.21.15.69:0/944400253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4dac1093f0 0x7f4dac1a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:05.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.826+0000 7f4dab400640 1 -- 172.21.15.69:0/944400253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dac19b350 con 0x7f4dac19a750 2024-04-24T09:18:05.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.826+0000 7f4daaa00640 1 --2- 172.21.15.69:0/944400253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dac104320 0x7f4dac1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:18:05.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.826+0000 7f4dab400640 1 --2- 172.21.15.69:0/944400253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dac19a750 0x7f4dac19abd0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4da400d750 tx=0x7f4da400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:05.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.826+0000 7f4da3400640 1 -- 172.21.15.69:0/944400253 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4017070 con 0x7f4dac19a750 2024-04-24T09:18:05.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.826+0000 7f4da3400640 1 -- 172.21.15.69:0/944400253 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4da40054d0 con 0x7f4dac19a750 2024-04-24T09:18:05.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.826+0000 7f4dabe00640 1 -- 172.21.15.69:0/944400253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dac1a7430 con 0x7f4dac19a750 2024-04-24T09:18:05.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.826+0000 7f4da3400640 1 -- 172.21.15.69:0/944400253 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4004c60 con 0x7f4dac19a750 2024-04-24T09:18:05.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.826+0000 7f4dabe00640 1 -- 172.21.15.69:0/944400253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dac1a77e0 con 0x7f4dac19a750 2024-04-24T09:18:05.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.827+0000 7f4dabe00640 1 -- 172.21.15.69:0/944400253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dac1057f0 con 0x7f4dac19a750 2024-04-24T09:18:05.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.829+0000 7f4da3400640 1 -- 172.21.15.69:0/944400253 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4da4004e00 con 0x7f4dac19a750 2024-04-24T09:18:05.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.831+0000 7f4da3400640 1 --2- 172.21.15.69:0/944400253 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4d900a3d20 0x7f4d900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:05.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.831+0000 7f4da3400640 1 -- 172.21.15.69:0/944400253 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f4da40d05c0 con 0x7f4dac19a750 2024-04-24T09:18:05.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.831+0000 7f4daaa00640 1 --2- 172.21.15.69:0/944400253 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4d900a3d20 0x7f4d900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:05.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.832+0000 7f4daaa00640 1 --2- 172.21.15.69:0/944400253 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4d900a3d20 0x7f4d900a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c005fd0 tx=0x7f4d9c0093e0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:05.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:05.835+0000 7f4da3400640 1 -- 172.21.15.69:0/944400253 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4da409c870 con 0x7f4dac19a750 2024-04-24T09:18:06.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.041+0000 7f4dabe00640 1 -- 172.21.15.69:0/944400253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4dac104cd0 con 0x7f4dac19a750 2024-04-24T09:18:06.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.041+0000 7f4da3400640 1 -- 172.21.15.69:0/944400253 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f4da409c170 con 0x7f4dac19a750 2024-04-24T09:18:06.042 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-24T09:18:06.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.044+0000 7f4dabe00640 1 -- 172.21.15.69:0/944400253 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4d900a3d20 msgr2=0x7f4d900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:06.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.044+0000 7f4dabe00640 1 --2- 172.21.15.69:0/944400253 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4d900a3d20 0x7f4d900a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c005fd0 tx=0x7f4d9c0093e0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.044+0000 7f4dabe00640 1 -- 172.21.15.69:0/944400253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dac19a750 msgr2=0x7f4dac19abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:06.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.044+0000 7f4dabe00640 1 --2- 172.21.15.69:0/944400253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dac19a750 0x7f4dac19abd0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4da400d750 tx=0x7f4da400dc10 comp rx=0 tx=0).stop 2024-04-24T09:18:06.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.045+0000 7f4dabe00640 1 -- 172.21.15.69:0/944400253 shutdown_connections 2024-04-24T09:18:06.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.045+0000 7f4dabe00640 1 --2- 172.21.15.69:0/944400253 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f4d900a3d20 0x7f4d900a61e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.045+0000 7f4dabe00640 1 --2- 172.21.15.69:0/944400253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dac19a750 0x7f4dac19abd0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.045+0000 7f4dabe00640 1 --2- 172.21.15.69:0/944400253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4dac1093f0 0x7f4dac1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.045+0000 7f4dabe00640 1 --2- 172.21.15.69:0/944400253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4dac104320 0x7f4dac1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.045+0000 7f4dabe00640 1 -- 172.21.15.69:0/944400253 >> 172.21.15.69:0/944400253 conn(0x7f4dac0ffb40 msgr2=0x7f4dac100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:06.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.045+0000 7f4dabe00640 1 -- 172.21.15.69:0/944400253 shutdown_connections 2024-04-24T09:18:06.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.045+0000 7f4dabe00640 1 -- 172.21.15.69:0/944400253 wait complete. 2024-04-24T09:18:06.064 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:18:06.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.281+0000 7fd6ec800640 1 -- 172.21.15.69:0/300608211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6e4107000 msgr2=0x7fd6e41073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:06.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.281+0000 7fd6ec800640 1 --2- 172.21.15.69:0/300608211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6e4107000 0x7fd6e41073e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009930 tx=0x7fd6d802a420 comp rx=0 tx=0).stop 2024-04-24T09:18:06.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.281+0000 7fd6ec800640 1 -- 172.21.15.69:0/300608211 shutdown_connections 2024-04-24T09:18:06.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.281+0000 7fd6ec800640 1 --2- 172.21.15.69:0/300608211 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6e4100f30 0x7fd6e410ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.281+0000 7fd6ec800640 1 --2- 172.21.15.69:0/300608211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6e4100570 0x7fd6e41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.281+0000 7fd6ec800640 1 --2- 172.21.15.69:0/300608211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6e4107000 0x7fd6e41073e0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.281+0000 7fd6ec800640 1 -- 172.21.15.69:0/300608211 >> 172.21.15.69:0/300608211 conn(0x7fd6e4077500 msgr2=0x7fd6e40fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:06.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.281+0000 7fd6ec800640 1 -- 172.21.15.69:0/300608211 shutdown_connections 2024-04-24T09:18:06.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.281+0000 7fd6ec800640 1 -- 172.21.15.69:0/300608211 wait complete. 2024-04-24T09:18:06.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.282+0000 7fd6ec800640 1 Processor -- start 2024-04-24T09:18:06.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.282+0000 7fd6ec800640 1 -- start start 2024-04-24T09:18:06.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.283+0000 7fd6ec800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6e4100570 0x7fd6e4101750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:06.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.283+0000 7fd6ec800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6e4100f30 0x7fd6e4101c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:06.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.283+0000 7fd6ec800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6e4107000 0x7fd6e41021d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:06.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.283+0000 7fd6ec800640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4112380 con 0x7fd6e4100570 2024-04-24T09:18:06.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.283+0000 7fd6ec800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4112200 con 0x7fd6e4100f30 2024-04-24T09:18:06.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.283+0000 7fd6ec800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4112500 con 0x7fd6e4107000 2024-04-24T09:18:06.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.283+0000 7fd6e9e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6e4100570 0x7fd6e4101750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:06.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.283+0000 7fd6e9e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6e4100570 0x7fd6e4101750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55770/0 (socket says 172.21.15.69:55770) 2024-04-24T09:18:06.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.283+0000 7fd6e9e00640 1 -- 172.21.15.69:0/701539173 learned_addr learned my addr 172.21.15.69:0/701539173 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:18:06.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.283+0000 7fd6e9400640 1 --2- 172.21.15.69:0/701539173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6e4100f30 0x7fd6e4101c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:06.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.284+0000 7fd6ea800640 1 --2- 172.21.15.69:0/701539173 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6e4107000 0x7fd6e41021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:06.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.284+0000 7fd6e9e00640 1 -- 172.21.15.69:0/701539173 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6e4107000 msgr2=0x7fd6e41021d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:06.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.284+0000 7fd6e9e00640 1 --2- 172.21.15.69:0/701539173 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6e4107000 0x7fd6e41021d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.284+0000 7fd6e9e00640 1 -- 172.21.15.69:0/701539173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6e4100f30 msgr2=0x7fd6e4101c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:06.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.284+0000 7fd6e9e00640 1 --2- 172.21.15.69:0/701539173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6e4100f30 0x7fd6e4101c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.284+0000 7fd6e9e00640 1 -- 172.21.15.69:0/701539173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e41a6d60 con 0x7fd6e4100570 2024-04-24T09:18:06.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.284+0000 7fd6e9e00640 1 --2- 172.21.15.69:0/701539173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6e4100570 0x7fd6e4101750 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd6d802a940 tx=0x7fd6d8003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:06.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.284+0000 7fd6dea00640 1 -- 172.21.15.69:0/701539173 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d803f070 con 0x7fd6e4100570 2024-04-24T09:18:06.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.284+0000 7fd6dea00640 1 -- 172.21.15.69:0/701539173 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6d803bde0 con 0x7fd6e4100570 2024-04-24T09:18:06.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.284+0000 7fd6ec800640 1 -- 172.21.15.69:0/701539173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e41a6ff0 con 0x7fd6e4100570 2024-04-24T09:18:06.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.284+0000 7fd6dea00640 1 -- 172.21.15.69:0/701539173 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8035030 con 0x7fd6e4100570 2024-04-24T09:18:06.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.285+0000 7fd6ec800640 1 -- 172.21.15.69:0/701539173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e41a7450 con 0x7fd6e4100570 2024-04-24T09:18:06.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.286+0000 7fd6ec800640 1 -- 172.21.15.69:0/701539173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e4069270 con 0x7fd6e4100570 2024-04-24T09:18:06.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.288+0000 7fd6dea00640 1 -- 172.21.15.69:0/701539173 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd6d8042070 con 0x7fd6e4100570 2024-04-24T09:18:06.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.289+0000 7fd6dea00640 1 --2- 172.21.15.69:0/701539173 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd6cc0a3d20 0x7fd6cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:06.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.289+0000 7fd6dea00640 1 -- 172.21.15.69:0/701539173 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fd6d80f1a20 con 0x7fd6e4100570 2024-04-24T09:18:06.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.290+0000 7fd6e9400640 1 --2- 172.21.15.69:0/701539173 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd6cc0a3d20 0x7fd6cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:06.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.291+0000 7fd6e9400640 1 --2- 172.21.15.69:0/701539173 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd6cc0a3d20 0x7fd6cc0a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4005e10 tx=0x7fd6d4005da0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:06.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.291+0000 7fd6dea00640 1 -- 172.21.15.69:0/701539173 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6d8031e20 con 0x7fd6e4100570 2024-04-24T09:18:06.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.492+0000 7fd6ec800640 1 -- 172.21.15.69:0/701539173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd6e402d620 con 0x7fd6e4100570 2024-04-24T09:18:06.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.493+0000 7fd6dea00640 1 -- 172.21.15.69:0/701539173 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7fd6d80bdcd0 con 0x7fd6e4100570 2024-04-24T09:18:06.493 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:18:06.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.496+0000 7fd6ec800640 1 -- 172.21.15.69:0/701539173 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd6cc0a3d20 msgr2=0x7fd6cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:06.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.496+0000 7fd6ec800640 1 --2- 172.21.15.69:0/701539173 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd6cc0a3d20 0x7fd6cc0a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4005e10 tx=0x7fd6d4005da0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.496+0000 7fd6ec800640 1 -- 172.21.15.69:0/701539173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6e4100570 msgr2=0x7fd6e4101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:06.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.496+0000 7fd6ec800640 1 --2- 172.21.15.69:0/701539173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6e4100570 0x7fd6e4101750 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd6d802a940 tx=0x7fd6d8003d60 comp rx=0 tx=0).stop 2024-04-24T09:18:06.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.496+0000 7fd6ec800640 1 -- 172.21.15.69:0/701539173 shutdown_connections 2024-04-24T09:18:06.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.496+0000 7fd6ec800640 1 --2- 172.21.15.69:0/701539173 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd6cc0a3d20 0x7fd6cc0a61e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.496+0000 7fd6ec800640 1 --2- 172.21.15.69:0/701539173 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6e4107000 0x7fd6e41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.496+0000 7fd6ec800640 1 --2- 172.21.15.69:0/701539173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6e4100f30 0x7fd6e4101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.496+0000 7fd6ec800640 1 --2- 172.21.15.69:0/701539173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6e4100570 0x7fd6e4101750 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.496+0000 7fd6ec800640 1 -- 172.21.15.69:0/701539173 >> 172.21.15.69:0/701539173 conn(0x7fd6e4077500 msgr2=0x7fd6e4105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:06.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.496+0000 7fd6ec800640 1 -- 172.21.15.69:0/701539173 shutdown_connections 2024-04-24T09:18:06.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.496+0000 7fd6ec800640 1 -- 172.21.15.69:0/701539173 wait complete. 2024-04-24T09:18:06.516 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:18:06.588 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:06 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1793162189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:06.588 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:06 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/944400253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:06.593 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:06 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1793162189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:06.593 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:06 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/944400253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:06.643 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:06 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1793162189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:06.643 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:06 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/944400253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:06.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.722+0000 7ff93ac00640 1 -- 172.21.15.69:0/528341435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff934103980 msgr2=0x7ff93410be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:06.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.722+0000 7ff93ac00640 1 --2- 172.21.15.69:0/528341435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff934103980 0x7ff93410be60 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7ff920009900 tx=0x7ff92002a480 comp rx=0 tx=0).stop 2024-04-24T09:18:06.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.722+0000 7ff93ac00640 1 -- 172.21.15.69:0/528341435 shutdown_connections 2024-04-24T09:18:06.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.722+0000 7ff93ac00640 1 --2- 172.21.15.69:0/528341435 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff93410c480 0x7ff93410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.722+0000 7ff93ac00640 1 --2- 172.21.15.69:0/528341435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff934103980 0x7ff93410be60 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.722+0000 7ff93ac00640 1 --2- 172.21.15.69:0/528341435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff934103060 0x7ff934103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.722+0000 7ff93ac00640 1 -- 172.21.15.69:0/528341435 >> 172.21.15.69:0/528341435 conn(0x7ff9340fc880 msgr2=0x7ff9340feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:06.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.722+0000 7ff93ac00640 1 -- 172.21.15.69:0/528341435 shutdown_connections 2024-04-24T09:18:06.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.722+0000 7ff93ac00640 1 -- 172.21.15.69:0/528341435 wait complete. 2024-04-24T09:18:06.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.723+0000 7ff93ac00640 1 Processor -- start 2024-04-24T09:18:06.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.724+0000 7ff93ac00640 1 -- start start 2024-04-24T09:18:06.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.724+0000 7ff93ac00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff934103060 0x7ff934198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:06.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.724+0000 7ff93ac00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff934103980 0x7ff934198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:06.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.724+0000 7ff93ac00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff93410c480 0x7ff93419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:06.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.724+0000 7ff93ac00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff9341120f0 con 0x7ff934103060 2024-04-24T09:18:06.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.724+0000 7ff93ac00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff934111f70 con 0x7ff93410c480 2024-04-24T09:18:06.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.724+0000 7ff93ac00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff934112270 con 0x7ff934103980 2024-04-24T09:18:06.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.724+0000 7ff939800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff934103060 0x7ff934198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:06.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.725+0000 7ff939800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff934103060 0x7ff934198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55802/0 (socket says 172.21.15.69:55802) 2024-04-24T09:18:06.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.725+0000 7ff939800640 1 -- 172.21.15.69:0/3626089011 learned_addr learned my addr 172.21.15.69:0/3626089011 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:18:06.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.725+0000 7ff938e00640 1 --2- 172.21.15.69:0/3626089011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff934103980 0x7ff934198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:06.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.725+0000 7ff93a200640 1 --2- 172.21.15.69:0/3626089011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff93410c480 0x7ff93419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:06.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.725+0000 7ff939800640 1 -- 172.21.15.69:0/3626089011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff934103980 msgr2=0x7ff934198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:06.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.725+0000 7ff939800640 1 --2- 172.21.15.69:0/3626089011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff934103980 0x7ff934198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.725+0000 7ff939800640 1 -- 172.21.15.69:0/3626089011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff93410c480 msgr2=0x7ff93419cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:06.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.725+0000 7ff939800640 1 --2- 172.21.15.69:0/3626089011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff93410c480 0x7ff93419cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.725+0000 7ff939800640 1 -- 172.21.15.69:0/3626089011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93419d680 con 0x7ff934103060 2024-04-24T09:18:06.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.725+0000 7ff939800640 1 --2- 172.21.15.69:0/3626089011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff934103060 0x7ff934198690 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff92400c920 tx=0x7ff92400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:06.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.726+0000 7ff932000640 1 -- 172.21.15.69:0/3626089011 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924007c70 con 0x7ff934103060 2024-04-24T09:18:06.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.726+0000 7ff932000640 1 -- 172.21.15.69:0/3626089011 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff924007e10 con 0x7ff934103060 2024-04-24T09:18:06.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.726+0000 7ff93ac00640 1 -- 172.21.15.69:0/3626089011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93419d970 con 0x7ff934103060 2024-04-24T09:18:06.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.726+0000 7ff932000640 1 -- 172.21.15.69:0/3626089011 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924018820 con 0x7ff934103060 2024-04-24T09:18:06.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.726+0000 7ff93ac00640 1 -- 172.21.15.69:0/3626089011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9341a5200 con 0x7ff934103060 2024-04-24T09:18:06.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.727+0000 7ff93ac00640 1 -- 172.21.15.69:0/3626089011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff934068d40 con 0x7ff934103060 2024-04-24T09:18:06.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.729+0000 7ff932000640 1 -- 172.21.15.69:0/3626089011 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff9240189c0 con 0x7ff934103060 2024-04-24T09:18:06.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.731+0000 7ff932000640 1 --2- 172.21.15.69:0/3626089011 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff9180a3da0 0x7ff9180a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:06.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.731+0000 7ff932000640 1 -- 172.21.15.69:0/3626089011 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7ff9240d0ca0 con 0x7ff934103060 2024-04-24T09:18:06.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.731+0000 7ff938e00640 1 --2- 172.21.15.69:0/3626089011 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff9180a3da0 0x7ff9180a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:06.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.732+0000 7ff938e00640 1 --2- 172.21.15.69:0/3626089011 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff9180a3da0 0x7ff9180a6260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff920031440 tx=0x7ff920005b50 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:06.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.735+0000 7ff932000640 1 -- 172.21.15.69:0/3626089011 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff924014030 con 0x7ff934103060 2024-04-24T09:18:06.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.937+0000 7ff93ac00640 1 -- 172.21.15.69:0/3626089011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff934107520 con 0x7ff934103060 2024-04-24T09:18:06.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.938+0000 7ff932000640 1 -- 172.21.15.69:0/3626089011 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7ff92409cf50 con 0x7ff934103060 2024-04-24T09:18:06.938 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-24T09:18:06.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.940+0000 7ff93ac00640 1 -- 172.21.15.69:0/3626089011 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff9180a3da0 msgr2=0x7ff9180a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:06.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.940+0000 7ff93ac00640 1 --2- 172.21.15.69:0/3626089011 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff9180a3da0 0x7ff9180a6260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff920031440 tx=0x7ff920005b50 comp rx=0 tx=0).stop 2024-04-24T09:18:06.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.940+0000 7ff93ac00640 1 -- 172.21.15.69:0/3626089011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff934103060 msgr2=0x7ff934198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:06.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.940+0000 7ff93ac00640 1 --2- 172.21.15.69:0/3626089011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff934103060 0x7ff934198690 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff92400c920 tx=0x7ff92400cde0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.940+0000 7ff93ac00640 1 -- 172.21.15.69:0/3626089011 shutdown_connections 2024-04-24T09:18:06.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.940+0000 7ff93ac00640 1 --2- 172.21.15.69:0/3626089011 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7ff9180a3da0 0x7ff9180a6260 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.940+0000 7ff93ac00640 1 --2- 172.21.15.69:0/3626089011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff93410c480 0x7ff93419cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.940+0000 7ff93ac00640 1 --2- 172.21.15.69:0/3626089011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff934103980 0x7ff934198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.940+0000 7ff93ac00640 1 --2- 172.21.15.69:0/3626089011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff934103060 0x7ff934198690 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:06.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.940+0000 7ff93ac00640 1 -- 172.21.15.69:0/3626089011 >> 172.21.15.69:0/3626089011 conn(0x7ff9340fc880 msgr2=0x7ff9340fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:06.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.940+0000 7ff93ac00640 1 -- 172.21.15.69:0/3626089011 shutdown_connections 2024-04-24T09:18:06.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:06.940+0000 7ff93ac00640 1 -- 172.21.15.69:0/3626089011 wait complete. 2024-04-24T09:18:06.960 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T09:18:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.173+0000 7f903a600640 1 -- 172.21.15.69:0/4150613070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9034104320 msgr2=0x7f9034104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.173+0000 7f903a600640 1 --2- 172.21.15.69:0/4150613070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9034104320 0x7f9034104700 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f9028009900 tx=0x7f902802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.173+0000 7f903a600640 1 -- 172.21.15.69:0/4150613070 shutdown_connections 2024-04-24T09:18:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.173+0000 7f903a600640 1 --2- 172.21.15.69:0/4150613070 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90341093f0 0x7f9034110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.173+0000 7f903a600640 1 --2- 172.21.15.69:0/4150613070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9034104cd0 0x7f9034108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.173+0000 7f903a600640 1 --2- 172.21.15.69:0/4150613070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9034104320 0x7f9034104700 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.173+0000 7f903a600640 1 -- 172.21.15.69:0/4150613070 >> 172.21.15.69:0/4150613070 conn(0x7f90340ffb40 msgr2=0x7f9034101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.173+0000 7f903a600640 1 -- 172.21.15.69:0/4150613070 shutdown_connections 2024-04-24T09:18:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.174+0000 7f903a600640 1 -- 172.21.15.69:0/4150613070 wait complete. 2024-04-24T09:18:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.174+0000 7f903a600640 1 Processor -- start 2024-04-24T09:18:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.175+0000 7f903a600640 1 -- start start 2024-04-24T09:18:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.175+0000 7f903a600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9034104320 0x7f90341a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.175+0000 7f903a600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90341093f0 0x7f90341a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.175+0000 7f903a600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f903419a710 0x7f903419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.175+0000 7f903a600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9034114330 con 0x7f9034104320 2024-04-24T09:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.175+0000 7f903a600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f90341141b0 con 0x7f903419a710 2024-04-24T09:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.175+0000 7f903a600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f90341144b0 con 0x7f90341093f0 2024-04-24T09:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.175+0000 7f9039200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9034104320 0x7f90341a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.175+0000 7f9039200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9034104320 0x7f90341a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55810/0 (socket says 172.21.15.69:55810) 2024-04-24T09:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.175+0000 7f9039200640 1 -- 172.21.15.69:0/3811845217 learned_addr learned my addr 172.21.15.69:0/3811845217 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.175+0000 7f9039c00640 1 --2- 172.21.15.69:0/3811845217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f903419a710 0x7f903419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.176+0000 7f9033e00640 1 --2- 172.21.15.69:0/3811845217 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90341093f0 0x7f90341a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.176+0000 7f9039200640 1 -- 172.21.15.69:0/3811845217 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90341093f0 msgr2=0x7f90341a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.176+0000 7f9039200640 1 --2- 172.21.15.69:0/3811845217 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90341093f0 0x7f90341a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.176+0000 7f9039200640 1 -- 172.21.15.69:0/3811845217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f903419a710 msgr2=0x7f903419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.176+0000 7f9039200640 1 --2- 172.21.15.69:0/3811845217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f903419a710 0x7f903419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.176+0000 7f9039200640 1 -- 172.21.15.69:0/3811845217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903419b310 con 0x7f9034104320 2024-04-24T09:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.176+0000 7f9039200640 1 --2- 172.21.15.69:0/3811845217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9034104320 0x7f90341a0600 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f9028005940 tx=0x7f90280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.177+0000 7f9031600640 1 -- 172.21.15.69:0/3811845217 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902803f030 con 0x7f9034104320 2024-04-24T09:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.177+0000 7f9031600640 1 -- 172.21.15.69:0/3811845217 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9028003cb0 con 0x7f9034104320 2024-04-24T09:18:07.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.177+0000 7f903a600640 1 -- 172.21.15.69:0/3811845217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903419b5a0 con 0x7f9034104320 2024-04-24T09:18:07.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.177+0000 7f9031600640 1 -- 172.21.15.69:0/3811845217 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028035070 con 0x7f9034104320 2024-04-24T09:18:07.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.177+0000 7f903a600640 1 -- 172.21.15.69:0/3811845217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90341a7690 con 0x7f9034104320 2024-04-24T09:18:07.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.178+0000 7f903a600640 1 -- 172.21.15.69:0/3811845217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90341057f0 con 0x7f9034104320 2024-04-24T09:18:07.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.180+0000 7f9031600640 1 -- 172.21.15.69:0/3811845217 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9028042030 con 0x7f9034104320 2024-04-24T09:18:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.181+0000 7f9031600640 1 --2- 172.21.15.69:0/3811845217 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f901c0a3d20 0x7f901c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.182+0000 7f9031600640 1 -- 172.21.15.69:0/3811845217 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f90280f13b0 con 0x7f9034104320 2024-04-24T09:18:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.182+0000 7f9033e00640 1 --2- 172.21.15.69:0/3811845217 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f901c0a3d20 0x7f901c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.182+0000 7f9033e00640 1 --2- 172.21.15.69:0/3811845217 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f901c0a3d20 0x7f901c0a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f903419bf10 tx=0x7f9020005f90 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:07.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.185+0000 7f9031600640 1 -- 172.21.15.69:0/3811845217 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90280bd660 con 0x7f9034104320 2024-04-24T09:18:07.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.386+0000 7f903a600640 1 -- 172.21.15.69:0/3811845217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f903410c620 con 0x7f9034104320 2024-04-24T09:18:07.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.387+0000 7f9031600640 1 -- 172.21.15.69:0/3811845217 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v101) ==== 103+0+11 (secure 0 0 0) 0x7f90280bcf60 con 0x7f9034104320 2024-04-24T09:18:07.388 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:18:07.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.390+0000 7f903a600640 1 -- 172.21.15.69:0/3811845217 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f901c0a3d20 msgr2=0x7f901c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:07.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.390+0000 7f903a600640 1 --2- 172.21.15.69:0/3811845217 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f901c0a3d20 0x7f901c0a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f903419bf10 tx=0x7f9020005f90 comp rx=0 tx=0).stop 2024-04-24T09:18:07.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.390+0000 7f903a600640 1 -- 172.21.15.69:0/3811845217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9034104320 msgr2=0x7f90341a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:07.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.390+0000 7f903a600640 1 --2- 172.21.15.69:0/3811845217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9034104320 0x7f90341a0600 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f9028005940 tx=0x7f90280042f0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.390+0000 7f903a600640 1 -- 172.21.15.69:0/3811845217 shutdown_connections 2024-04-24T09:18:07.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.390+0000 7f903a600640 1 --2- 172.21.15.69:0/3811845217 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f901c0a3d20 0x7f901c0a61e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.390+0000 7f903a600640 1 --2- 172.21.15.69:0/3811845217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f903419a710 0x7f903419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.390+0000 7f903a600640 1 --2- 172.21.15.69:0/3811845217 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90341093f0 0x7f90341a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.390+0000 7f903a600640 1 --2- 172.21.15.69:0/3811845217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9034104320 0x7f90341a0600 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.390+0000 7f903a600640 1 -- 172.21.15.69:0/3811845217 >> 172.21.15.69:0/3811845217 conn(0x7f90340ffb40 msgr2=0x7f9034100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:07.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.390+0000 7f903a600640 1 -- 172.21.15.69:0/3811845217 shutdown_connections 2024-04-24T09:18:07.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.390+0000 7f903a600640 1 -- 172.21.15.69:0/3811845217 wait complete. 2024-04-24T09:18:07.410 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T09:18:07.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.614+0000 7f8a1cc00640 1 -- 172.21.15.69:0/3619119316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a18101000 msgr2=0x7f8a181013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:07.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.614+0000 7f8a1cc00640 1 --2- 172.21.15.69:0/3619119316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a18101000 0x7f8a181013e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009900 tx=0x7f8a0802a410 comp rx=0 tx=0).stop 2024-04-24T09:18:07.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.614+0000 7f8a1cc00640 1 -- 172.21.15.69:0/3619119316 shutdown_connections 2024-04-24T09:18:07.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.614+0000 7f8a1cc00640 1 --2- 172.21.15.69:0/3619119316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a181060d0 0x7f8a1810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.614+0000 7f8a1cc00640 1 --2- 172.21.15.69:0/3619119316 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a181019b0 0x7f8a181059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.614+0000 7f8a1cc00640 1 --2- 172.21.15.69:0/3619119316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a18101000 0x7f8a181013e0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.614+0000 7f8a1cc00640 1 -- 172.21.15.69:0/3619119316 >> 172.21.15.69:0/3619119316 conn(0x7f8a180fc880 msgr2=0x7f8a180feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:07.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.614+0000 7f8a1cc00640 1 -- 172.21.15.69:0/3619119316 shutdown_connections 2024-04-24T09:18:07.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.614+0000 7f8a1cc00640 1 -- 172.21.15.69:0/3619119316 wait complete. 2024-04-24T09:18:07.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.615+0000 7f8a1cc00640 1 Processor -- start 2024-04-24T09:18:07.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.615+0000 7f8a1cc00640 1 -- start start 2024-04-24T09:18:07.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.616+0000 7f8a1cc00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a18101000 0x7f8a181a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:07.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.616+0000 7f8a1cc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a181060d0 0x7f8a181a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:07.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.616+0000 7f8a1cc00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a1819a790 0x7f8a1819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:07.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.616+0000 7f8a1cc00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18111030 con 0x7f8a181060d0 2024-04-24T09:18:07.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.616+0000 7f8a1cc00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18110eb0 con 0x7f8a1819a790 2024-04-24T09:18:07.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.616+0000 7f8a1cc00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8a181111b0 con 0x7f8a18101000 2024-04-24T09:18:07.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.616+0000 7f8a16a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a181060d0 0x7f8a181a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:07.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.616+0000 7f8a16a00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a181060d0 0x7f8a181a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55824/0 (socket says 172.21.15.69:55824) 2024-04-24T09:18:07.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.616+0000 7f8a16a00640 1 -- 172.21.15.69:0/1634631078 learned_addr learned my addr 172.21.15.69:0/1634631078 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:18:07.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.616+0000 7f8a17400640 1 --2- 172.21.15.69:0/1634631078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a18101000 0x7f8a181a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:07.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.616+0000 7f8a17e00640 1 --2- 172.21.15.69:0/1634631078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a1819a790 0x7f8a1819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:07.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.616+0000 7f8a16a00640 1 -- 172.21.15.69:0/1634631078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a18101000 msgr2=0x7f8a181a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:07.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.617+0000 7f8a16a00640 1 --2- 172.21.15.69:0/1634631078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a18101000 0x7f8a181a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.617+0000 7f8a16a00640 1 -- 172.21.15.69:0/1634631078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a1819a790 msgr2=0x7f8a1819ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:07.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.617+0000 7f8a16a00640 1 --2- 172.21.15.69:0/1634631078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a1819a790 0x7f8a1819ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.617+0000 7f8a16a00640 1 -- 172.21.15.69:0/1634631078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1819b320 con 0x7f8a181060d0 2024-04-24T09:18:07.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.617+0000 7f8a17e00640 1 --2- 172.21.15.69:0/1634631078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a1819a790 0x7f8a1819ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:18:07.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.617+0000 7f8a16a00640 1 --2- 172.21.15.69:0/1634631078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a181060d0 0x7f8a181a0b60 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000ea50 tx=0x7f8a0000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:07.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.617+0000 7f8a0fe00640 1 -- 172.21.15.69:0/1634631078 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0000cdc0 con 0x7f8a181060d0 2024-04-24T09:18:07.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.617+0000 7f8a0fe00640 1 -- 172.21.15.69:0/1634631078 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a00005020 con 0x7f8a181060d0 2024-04-24T09:18:07.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.617+0000 7f8a0fe00640 1 -- 172.21.15.69:0/1634631078 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a000116b0 con 0x7f8a181060d0 2024-04-24T09:18:07.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.617+0000 7f8a1cc00640 1 -- 172.21.15.69:0/1634631078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1819b610 con 0x7f8a181060d0 2024-04-24T09:18:07.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.617+0000 7f8a1cc00640 1 -- 172.21.15.69:0/1634631078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a181a77a0 con 0x7f8a181060d0 2024-04-24T09:18:07.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.618+0000 7f8a1cc00640 1 -- 172.21.15.69:0/1634631078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a181024d0 con 0x7f8a181060d0 2024-04-24T09:18:07.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.620+0000 7f8a0fe00640 1 -- 172.21.15.69:0/1634631078 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8a000051d0 con 0x7f8a181060d0 2024-04-24T09:18:07.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.622+0000 7f8a0fe00640 1 --2- 172.21.15.69:0/1634631078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f89f80a3da0 0x7f89f80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:07.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.622+0000 7f8a0fe00640 1 -- 172.21.15.69:0/1634631078 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f8a000d0f10 con 0x7f8a181060d0 2024-04-24T09:18:07.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.622+0000 7f8a17400640 1 --2- 172.21.15.69:0/1634631078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f89f80a3da0 0x7f89f80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:07.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.623+0000 7f8a17400640 1 --2- 172.21.15.69:0/1634631078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f89f80a3da0 0x7f89f80a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8a08031440 tx=0x7f8a080095f0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:07.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.626+0000 7f8a0fe00640 1 -- 172.21.15.69:0/1634631078 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a1819b610 con 0x7f8a181060d0 2024-04-24T09:18:07.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:07 smithi130 ceph-mon[31553]: pgmap v342: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-24T09:18:07.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:07 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/701539173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:18:07.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:07 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3626089011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:18:07.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:07 smithi081 ceph-mon[30961]: pgmap v342: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-24T09:18:07.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:07 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/701539173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:18:07.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:07 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3626089011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:18:07.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:07 smithi069 ceph-mon[28170]: pgmap v342: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-24T09:18:07.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:07 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/701539173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:18:07.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:07 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3626089011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:18:07.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.844+0000 7f8a1cc00640 1 -- 172.21.15.69:0/1634631078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8a181019b0 con 0x7f8a181060d0 2024-04-24T09:18:07.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.845+0000 7f8a0fe00640 1 -- 172.21.15.69:0/1634631078 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v101) ==== 81+0+945 (secure 0 0 0) 0x7f8a00016030 con 0x7f8a181060d0 2024-04-24T09:18:07.845 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/660205566 2024-04-25T09:07:03.863940+0000 2024-04-24T09:18:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/1098677190 2024-04-25T09:07:03.863940+0000 2024-04-24T09:18:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3145713060 2024-04-25T09:07:03.863940+0000 2024-04-24T09:18:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3826117580 2024-04-25T08:54:23.196411+0000 2024-04-24T09:18:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1006368686 2024-04-25T08:54:23.196411+0000 2024-04-24T09:18:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3008138297 2024-04-25T09:07:03.863940+0000 2024-04-24T09:18:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/3470783122 2024-04-25T08:54:23.196411+0000 2024-04-24T09:18:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2102503077 2024-04-25T08:53:48.948375+0000 2024-04-24T09:18:07.846 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2901345957 2024-04-25T09:07:03.863940+0000 2024-04-24T09:18:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/3470783122 2024-04-25T08:54:23.196411+0000 2024-04-24T09:18:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1041454348 2024-04-25T08:53:48.948375+0000 2024-04-24T09:18:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/3911821813 2024-04-25T08:53:48.948375+0000 2024-04-24T09:18:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/1098677190 2024-04-25T09:07:03.863940+0000 2024-04-24T09:18:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/3911821813 2024-04-25T08:53:48.948375+0000 2024-04-24T09:18:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/4072155336 2024-04-25T08:53:48.948375+0000 2024-04-24T09:18:07.847 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1278076203 2024-04-25T08:54:23.196411+0000 2024-04-24T09:18:07.848 INFO:teuthology.orchestra.run.smithi069.stderr:listed 16 entries 2024-04-24T09:18:07.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.847+0000 7f8a1cc00640 1 -- 172.21.15.69:0/1634631078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f89f80a3da0 msgr2=0x7f89f80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:07.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.847+0000 7f8a1cc00640 1 --2- 172.21.15.69:0/1634631078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f89f80a3da0 0x7f89f80a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8a08031440 tx=0x7f8a080095f0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.847+0000 7f8a1cc00640 1 -- 172.21.15.69:0/1634631078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a181060d0 msgr2=0x7f8a181a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:07.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.847+0000 7f8a1cc00640 1 --2- 172.21.15.69:0/1634631078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a181060d0 0x7f8a181a0b60 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000ea50 tx=0x7f8a0000ef10 comp rx=0 tx=0).stop 2024-04-24T09:18:07.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.848+0000 7f8a1cc00640 1 -- 172.21.15.69:0/1634631078 shutdown_connections 2024-04-24T09:18:07.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.848+0000 7f8a1cc00640 1 --2- 172.21.15.69:0/1634631078 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f89f80a3da0 0x7f89f80a6260 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.848+0000 7f8a1cc00640 1 --2- 172.21.15.69:0/1634631078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a1819a790 0x7f8a1819ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.848+0000 7f8a1cc00640 1 --2- 172.21.15.69:0/1634631078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a181060d0 0x7f8a181a0b60 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.848+0000 7f8a1cc00640 1 --2- 172.21.15.69:0/1634631078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a18101000 0x7f8a181a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.848+0000 7f8a1cc00640 1 -- 172.21.15.69:0/1634631078 >> 172.21.15.69:0/1634631078 conn(0x7f8a180fc880 msgr2=0x7f8a18069430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:07.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.848+0000 7f8a1cc00640 1 -- 172.21.15.69:0/1634631078 shutdown_connections 2024-04-24T09:18:07.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:07.848+0000 7f8a1cc00640 1 -- 172.21.15.69:0/1634631078 wait complete. 2024-04-24T09:18:07.867 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-24T09:18:07.867 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T09:18:07.917 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:18:08.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.114+0000 7f0afd600640 1 -- 172.21.15.69:0/1563223500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af8102050 msgr2=0x7f0af8102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:08.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.114+0000 7f0afd600640 1 --2- 172.21.15.69:0/1563223500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af8102050 0x7f0af8102430 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009900 tx=0x7f0ae802a400 comp rx=0 tx=0).stop 2024-04-24T09:18:08.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.115+0000 7f0afd600640 1 -- 172.21.15.69:0/1563223500 shutdown_connections 2024-04-24T09:18:08.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.115+0000 7f0afd600640 1 --2- 172.21.15.69:0/1563223500 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0af8107120 0x7f0af810ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:08.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.115+0000 7f0afd600640 1 --2- 172.21.15.69:0/1563223500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0af8102a00 0x7f0af8106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:08.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.115+0000 7f0afd600640 1 --2- 172.21.15.69:0/1563223500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af8102050 0x7f0af8102430 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:08.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.115+0000 7f0afd600640 1 -- 172.21.15.69:0/1563223500 >> 172.21.15.69:0/1563223500 conn(0x7f0af80fd930 msgr2=0x7f0af80ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:08.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.115+0000 7f0afd600640 1 -- 172.21.15.69:0/1563223500 shutdown_connections 2024-04-24T09:18:08.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.115+0000 7f0afd600640 1 -- 172.21.15.69:0/1563223500 wait complete. 2024-04-24T09:18:08.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.116+0000 7f0afd600640 1 Processor -- start 2024-04-24T09:18:08.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.116+0000 7f0afd600640 1 -- start start 2024-04-24T09:18:08.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.117+0000 7f0afd600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0af8102050 0x7f0af81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:08.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.117+0000 7f0afd600640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0af8107120 0x7f0af81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:08.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.117+0000 7f0afd600640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af819a760 0x7f0af819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:08.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.117+0000 7f0afd600640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0af81120e0 con 0x7f0af819a760 2024-04-24T09:18:08.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.117+0000 7f0afd600640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0af8111f60 con 0x7f0af8107120 2024-04-24T09:18:08.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.117+0000 7f0afd600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0af8112260 con 0x7f0af8102050 2024-04-24T09:18:08.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.117+0000 7f0afcc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af819a760 0x7f0af819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:08.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.117+0000 7f0afcc00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af819a760 0x7f0af819abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55844/0 (socket says 172.21.15.69:55844) 2024-04-24T09:18:08.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.117+0000 7f0afcc00640 1 -- 172.21.15.69:0/243807172 learned_addr learned my addr 172.21.15.69:0/243807172 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:18:08.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.117+0000 7f0aef400640 1 --2- 172.21.15.69:0/243807172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0af8107120 0x7f0af81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:08.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.117+0000 7f0af7e00640 1 --2- 172.21.15.69:0/243807172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0af8102050 0x7f0af81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:08.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.117+0000 7f0afcc00640 1 -- 172.21.15.69:0/243807172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0af8102050 msgr2=0x7f0af81a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:08.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.117+0000 7f0afcc00640 1 --2- 172.21.15.69:0/243807172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0af8102050 0x7f0af81a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:08.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.117+0000 7f0afcc00640 1 -- 172.21.15.69:0/243807172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0af8107120 msgr2=0x7f0af81a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:08.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.117+0000 7f0afcc00640 1 --2- 172.21.15.69:0/243807172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0af8107120 0x7f0af81a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:08.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.117+0000 7f0afcc00640 1 -- 172.21.15.69:0/243807172 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af819b360 con 0x7f0af819a760 2024-04-24T09:18:08.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.118+0000 7f0afcc00640 1 --2- 172.21.15.69:0/243807172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af819a760 0x7f0af819abe0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f0af000cad0 tx=0x7f0af000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:08.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.118+0000 7f0af5600640 1 -- 172.21.15.69:0/243807172 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af000ae20 con 0x7f0af819a760 2024-04-24T09:18:08.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.118+0000 7f0af5600640 1 -- 172.21.15.69:0/243807172 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0af0005020 con 0x7f0af819a760 2024-04-24T09:18:08.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.118+0000 7f0afd600640 1 -- 172.21.15.69:0/243807172 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af81a73f0 con 0x7f0af819a760 2024-04-24T09:18:08.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.118+0000 7f0af5600640 1 -- 172.21.15.69:0/243807172 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af000f6b0 con 0x7f0af819a760 2024-04-24T09:18:08.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.118+0000 7f0afd600640 1 -- 172.21.15.69:0/243807172 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af81a7850 con 0x7f0af819a760 2024-04-24T09:18:08.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.119+0000 7f0afd600640 1 -- 172.21.15.69:0/243807172 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af8103520 con 0x7f0af819a760 2024-04-24T09:18:08.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.121+0000 7f0af5600640 1 -- 172.21.15.69:0/243807172 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0af00051d0 con 0x7f0af819a760 2024-04-24T09:18:08.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.123+0000 7f0af5600640 1 --2- 172.21.15.69:0/243807172 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f0adc0a3da0 0x7f0adc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:08.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.123+0000 7f0af5600640 1 -- 172.21.15.69:0/243807172 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f0af00d0b20 con 0x7f0af819a760 2024-04-24T09:18:08.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.123+0000 7f0af7e00640 1 --2- 172.21.15.69:0/243807172 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f0adc0a3da0 0x7f0adc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:08.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.124+0000 7f0af7e00640 1 --2- 172.21.15.69:0/243807172 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f0adc0a3da0 0x7f0adc0a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8000c00 tx=0x7f0ae8031400 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:08.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.127+0000 7f0af5600640 1 -- 172.21.15.69:0/243807172 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0af0014030 con 0x7f0af819a760 2024-04-24T09:18:08.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.332+0000 7f0afd600640 1 -- 172.21.15.69:0/243807172 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0af8102ad0 con 0x7f0af819a760 2024-04-24T09:18:08.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.334+0000 7f0af5600640 1 -- 172.21.15.69:0/243807172 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+25127 (secure 0 0 0) 0x7f0af009cdd0 con 0x7f0af819a760 2024-04-24T09:18:08.335 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-24T09:18:08.335 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":101,"fsid":"ef1ec7e8-0217-11ef-bc93-c7b262605968","created":"2024-04-24T08:53:15.725881+0000","modified":"2024-04-24T09:17:12.920525+0000","last_up_change":"2024-04-24T09:02:53.494505+0000","last_in_change":"2024-04-24T09:02:28.640749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:57:37.377599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:04:00.299827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:04:00.958498+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T09:04:04.105314+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2c221fcb-ab5f-4f37-b5b4-40366081fbb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6803","nonce":340713595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6805","nonce":340713595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6809","nonce":340713595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":340713595},{"type":"v1","addr":"172.21.15.69:6807","nonce":340713595}]},"public_addr":"172.21.15.69:6803/340713595","cluster_addr":"172.21.15.69:6805/340713595","heartbeat_back_addr":"172.21.15.69:6809/340713595","heartbeat_front_addr":"172.21.15.69:6807/340713595","state":["exists","up"]},{"osd":1,"uuid":"dae4a019-875a-48ff-96d8-c19efa5987ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6801","nonce":3515249330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6803","nonce":3515249330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6807","nonce":3515249330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3515249330},{"type":"v1","addr":"172.21.15.81:6805","nonce":3515249330}]},"public_addr":"172.21.15.81:6801/3515249330","cluster_addr":"172.21.15.81:6803/3515249330","heartbeat_back_addr":"172.21.15.81:6807/3515249330","heartbeat_front_addr":"172.21.15.81:6805/3515249330","state":["exists","up"]},{"osd":2,"uuid":"62f0fd9a-64fe-4680-9107-ae8a97490dec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6801","nonce":3109352638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6803","nonce":3109352638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6807","nonce":3109352638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3109352638},{"type":"v1","addr":"172.21.15.130:6805","nonce":3109352638}]},"public_addr":"172.21.15.130:6801/3109352638","cluster_addr":"172.21.15.130:6803/3109352638","heartbeat_back_addr":"172.21.15.130:6807/3109352638","heartbeat_front_addr":"172.21.15.130:6805/3109352638","state":["exists","up"]},{"osd":3,"uuid":"09c5d15c-129e-45cf-9891-5b7a7f4a450f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6811","nonce":3549681529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6813","nonce":3549681529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6817","nonce":3549681529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3549681529},{"type":"v1","addr":"172.21.15.69:6815","nonce":3549681529}]},"public_addr":"172.21.15.69:6811/3549681529","cluster_addr":"172.21.15.69:6813/3549681529","heartbeat_back_addr":"172.21.15.69:6817/3549681529","heartbeat_front_addr":"172.21.15.69:6815/3549681529","state":["exists","up"]},{"osd":4,"uuid":"ae0c645d-b56e-4348-88c5-bf4f6738e0a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6809","nonce":575869990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6811","nonce":575869990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6815","nonce":575869990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":575869990},{"type":"v1","addr":"172.21.15.81:6813","nonce":575869990}]},"public_addr":"172.21.15.81:6809/575869990","cluster_addr":"172.21.15.81:6811/575869990","heartbeat_back_addr":"172.21.15.81:6815/575869990","heartbeat_front_addr":"172.21.15.81:6813/575869990","state":["exists","up"]},{"osd":5,"uuid":"dd082142-bbe7-40f2-85c3-94cf0cf96466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6809","nonce":1208873184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6811","nonce":1208873184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6815","nonce":1208873184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":1208873184},{"type":"v1","addr":"172.21.15.130:6813","nonce":1208873184}]},"public_addr":"172.21.15.130:6809/1208873184","cluster_addr":"172.21.15.130:6811/1208873184","heartbeat_back_addr":"172.21.15.130:6815/1208873184","heartbeat_front_addr":"172.21.15.130:6813/1208873184","state":["exists","up"]},{"osd":6,"uuid":"58e89c83-2d93-42a3-9fa4-e6b08810c776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6819","nonce":2606282387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6821","nonce":2606282387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6825","nonce":2606282387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2606282387},{"type":"v1","addr":"172.21.15.69:6823","nonce":2606282387}]},"public_addr":"172.21.15.69:6819/2606282387","cluster_addr":"172.21.15.69:6821/2606282387","heartbeat_back_addr":"172.21.15.69:6825/2606282387","heartbeat_front_addr":"172.21.15.69:6823/2606282387","state":["exists","up"]},{"osd":7,"uuid":"f1daca7a-630f-4a56-bd3d-713fc3f88ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6817","nonce":4105801153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6819","nonce":4105801153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6823","nonce":4105801153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":4105801153},{"type":"v1","addr":"172.21.15.81:6821","nonce":4105801153}]},"public_addr":"172.21.15.81:6817/4105801153","cluster_addr":"172.21.15.81:6819/4105801153","heartbeat_back_addr":"172.21.15.81:6823/4105801153","heartbeat_front_addr":"172.21.15.81:6821/4105801153","state":["exists","up"]},{"osd":8,"uuid":"151820a5-68cc-4d46-af26-eff7f37e3186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6817","nonce":3037039143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6819","nonce":3037039143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6823","nonce":3037039143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3037039143},{"type":"v1","addr":"172.21.15.130:6821","nonce":3037039143}]},"public_addr":"172.21.15.130:6817/3037039143","cluster_addr":"172.21.15.130:6819/3037039143","heartbeat_back_addr":"172.21.15.130:6823/3037039143","heartbeat_front_addr":"172.21.15.130:6821/3037039143","state":["exists","up"]},{"osd":9,"uuid":"9155e0e5-52f4-48d3-b024-cf346de31211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6827","nonce":4110734159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6829","nonce":4110734159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6833","nonce":4110734159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4110734159},{"type":"v1","addr":"172.21.15.69:6831","nonce":4110734159}]},"public_addr":"172.21.15.69:6827/4110734159","cluster_addr":"172.21.15.69:6829/4110734159","heartbeat_back_addr":"172.21.15.69:6833/4110734159","heartbeat_front_addr":"172.21.15.69:6831/4110734159","state":["exists","up"]},{"osd":10,"uuid":"a2f8028d-7966-4ceb-9e12-a728d89665ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6825","nonce":3563037549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6827","nonce":3563037549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6831","nonce":3563037549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3563037549},{"type":"v1","addr":"172.21.15.81:6829","nonce":3563037549}]},"public_addr":"172.21.15.81:6825/3563037549","cluster_addr":"172.21.15.81:6827/3563037549","heartbeat_back_addr":"172.21.15.81:6831/3563037549","heartbeat_front_addr":"172.21.15.81:6829/3563037549","state":["exists","up"]},{"osd":11,"uuid":"ebe21375-9751-4541-9dbf-0d5e0c4bbd5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6825","nonce":1208587980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6827","nonce":1208587980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6831","nonce":1208587980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":1208587980},{"type":"v1","addr":"172.21.15.130:6829","nonce":1208587980}]},"public_addr":"172.21.15.130:6825/1208587980","cluster_addr":"172.21.15.130:6827/1208587980","heartbeat_back_addr":"172.21.15.130:6831/1208587980","heartbeat_front_addr":"172.21.15.130:6829/1208587980","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:56:28.879890+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:02.015126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:57:35.438700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:09.532247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:58:43.410833+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:18.080912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:59:53.606276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:00:28.516908+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:04.168528+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:01:40.683454+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:15.984699+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:02:52.210478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/660205566":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/1098677190":"2024-04-25T09:07:03.863940+0000","172.21.15.69:0/3145713060":"2024-04-25T09:07:03.863940+0000","172.21.15.69:0/3826117580":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1006368686":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/3008138297":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6800/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/2102503077":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/2901345957":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/3470783122":"2024-04-25T08:54:23.196411+0000","172.21.15.69:0/1041454348":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:6800/1098677190":"2024-04-25T09:07:03.863940+0000","172.21.15.69:6801/3911821813":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/4072155336":"2024-04-25T08:53:48.948375+0000","172.21.15.69:0/1278076203":"2024-04-25T08:54:23.196411+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:18:08.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.337+0000 7f0afd600640 1 -- 172.21.15.69:0/243807172 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f0adc0a3da0 msgr2=0x7f0adc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:08.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.337+0000 7f0afd600640 1 --2- 172.21.15.69:0/243807172 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f0adc0a3da0 0x7f0adc0a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8000c00 tx=0x7f0ae8031400 comp rx=0 tx=0).stop 2024-04-24T09:18:08.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.337+0000 7f0afd600640 1 -- 172.21.15.69:0/243807172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af819a760 msgr2=0x7f0af819abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:08.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.337+0000 7f0afd600640 1 --2- 172.21.15.69:0/243807172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af819a760 0x7f0af819abe0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f0af000cad0 tx=0x7f0af000cf90 comp rx=0 tx=0).stop 2024-04-24T09:18:08.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.337+0000 7f0afd600640 1 -- 172.21.15.69:0/243807172 shutdown_connections 2024-04-24T09:18:08.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.337+0000 7f0afd600640 1 --2- 172.21.15.69:0/243807172 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f0adc0a3da0 0x7f0adc0a6260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:08.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.337+0000 7f0afd600640 1 --2- 172.21.15.69:0/243807172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af819a760 0x7f0af819abe0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:08.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.337+0000 7f0afd600640 1 --2- 172.21.15.69:0/243807172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0af8107120 0x7f0af81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:08.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.337+0000 7f0afd600640 1 --2- 172.21.15.69:0/243807172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0af8102050 0x7f0af81a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:08.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.337+0000 7f0afd600640 1 -- 172.21.15.69:0/243807172 >> 172.21.15.69:0/243807172 conn(0x7f0af80fd930 msgr2=0x7f0af80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:08.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.337+0000 7f0afd600640 1 -- 172.21.15.69:0/243807172 shutdown_connections 2024-04-24T09:18:08.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.337+0000 7f0afd600640 1 -- 172.21.15.69:0/243807172 wait complete. 2024-04-24T09:18:08.357 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:57:37.377599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:04:00.299827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:04:00.958498+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T09:04:04.105314+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T09:18:08.358 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:18:08.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:08 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/3811845217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:18:08.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:08 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1634631078' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:18:08.567 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:08 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/243807172' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:08.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.673+0000 7fda0c200640 1 -- 172.21.15.69:0/3246772708 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda0410dd30 msgr2=0x7fda04115900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.673+0000 7fda0c200640 1 --2- 172.21.15.69:0/3246772708 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda0410dd30 0x7fda04115900 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc005fd0 tx=0x7fd9fc02b3f0 comp rx=0 tx=0).stop 2024-04-24T09:18:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.674+0000 7fda0c200640 1 -- 172.21.15.69:0/3246772708 shutdown_connections 2024-04-24T09:18:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.674+0000 7fda0c200640 1 --2- 172.21.15.69:0/3246772708 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda0410dd30 0x7fda04115900 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.674+0000 7fda0c200640 1 --2- 172.21.15.69:0/3246772708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda04109c20 0x7fda0410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.674+0000 7fda0c200640 1 --2- 172.21.15.69:0/3246772708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda04109270 0x7fda04109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:08.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.674+0000 7fda0c200640 1 -- 172.21.15.69:0/3246772708 >> 172.21.15.69:0/3246772708 conn(0x7fda0406d110 msgr2=0x7fda0406d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:08.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.674+0000 7fda0c200640 1 -- 172.21.15.69:0/3246772708 shutdown_connections 2024-04-24T09:18:08.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.674+0000 7fda0c200640 1 -- 172.21.15.69:0/3246772708 wait complete. 2024-04-24T09:18:08.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.676+0000 7fda0c200640 1 Processor -- start 2024-04-24T09:18:08.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.676+0000 7fda0c200640 1 -- start start 2024-04-24T09:18:08.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.677+0000 7fda0c200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda04109270 0x7fda041a3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:08.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.677+0000 7fda0c200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda0410dd30 0x7fda041a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:08.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.677+0000 7fda0c200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda041a7b50 0x7fda041a8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:08.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.677+0000 7fda0c200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fda041180c0 con 0x7fda041a7b50 2024-04-24T09:18:08.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.677+0000 7fda0c200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fda04117f40 con 0x7fda04109270 2024-04-24T09:18:08.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.677+0000 7fda0c200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fda04118240 con 0x7fda0410dd30 2024-04-24T09:18:08.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.678+0000 7fda0a200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda041a7b50 0x7fda041a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:08.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.678+0000 7fda0a200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda041a7b50 0x7fda041a8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55854/0 (socket says 172.21.15.69:55854) 2024-04-24T09:18:08.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.678+0000 7fda0a200640 1 -- 172.21.15.69:0/1264327998 learned_addr learned my addr 172.21.15.69:0/1264327998 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:18:08.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.678+0000 7fda09800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda04109270 0x7fda041a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:08.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.678+0000 7fda08e00640 1 --2- 172.21.15.69:0/1264327998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda0410dd30 0x7fda041a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:08.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.679+0000 7fda0a200640 1 -- 172.21.15.69:0/1264327998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda0410dd30 msgr2=0x7fda041a37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:08.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.679+0000 7fda0a200640 1 --2- 172.21.15.69:0/1264327998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda0410dd30 0x7fda041a37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:08.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.679+0000 7fda0a200640 1 -- 172.21.15.69:0/1264327998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda04109270 msgr2=0x7fda041a3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:08.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.680+0000 7fda0a200640 1 --2- 172.21.15.69:0/1264327998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda04109270 0x7fda041a3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:08.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.680+0000 7fda0a200640 1 -- 172.21.15.69:0/1264327998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda041a8640 con 0x7fda041a7b50 2024-04-24T09:18:08.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.680+0000 7fda09800640 1 --2- 172.21.15.69:0/1264327998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda04109270 0x7fda041a3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:18:08.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.680+0000 7fda0a200640 1 --2- 172.21.15.69:0/1264327998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda041a7b50 0x7fda041a8000 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00d090 tx=0x7fd9fc033f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:08.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.681+0000 7fda02000640 1 -- 172.21.15.69:0/1264327998 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc037040 con 0x7fda041a7b50 2024-04-24T09:18:08.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.681+0000 7fda0c200640 1 -- 172.21.15.69:0/1264327998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda041a88d0 con 0x7fda041a7b50 2024-04-24T09:18:08.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.682+0000 7fda02000640 1 -- 172.21.15.69:0/1264327998 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9fc007c50 con 0x7fda041a7b50 2024-04-24T09:18:08.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.682+0000 7fda0c200640 1 -- 172.21.15.69:0/1264327998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda041b0120 con 0x7fda041a7b50 2024-04-24T09:18:08.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.685+0000 7fda02000640 1 -- 172.21.15.69:0/1264327998 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc00a580 con 0x7fda041a7b50 2024-04-24T09:18:08.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.686+0000 7fda0c200640 1 -- 172.21.15.69:0/1264327998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda04068e20 con 0x7fda041a7b50 2024-04-24T09:18:08.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.688+0000 7fda02000640 1 -- 172.21.15.69:0/1264327998 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd9fc032780 con 0x7fda041a7b50 2024-04-24T09:18:08.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.691+0000 7fda02000640 1 --2- 172.21.15.69:0/1264327998 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd9ec0a3ec0 0x7fd9ec0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:08.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.692+0000 7fda02000640 1 -- 172.21.15.69:0/1264327998 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7fd9fc0f1160 con 0x7fda041a7b50 2024-04-24T09:18:08.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.693+0000 7fda09800640 1 --2- 172.21.15.69:0/1264327998 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd9ec0a3ec0 0x7fd9ec0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:08.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.697+0000 7fda09800640 1 --2- 172.21.15.69:0/1264327998 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd9ec0a3ec0 0x7fd9ec0a6380 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009d60 tx=0x7fd9f4009110 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:08.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:08.700+0000 7fda02000640 1 -- 172.21.15.69:0/1264327998 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9fc0bd410 con 0x7fda041a7b50 2024-04-24T09:18:08.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:08 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/3811845217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:18:08.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:08 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1634631078' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:18:08.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:08 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/243807172' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:08 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/3811845217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:18:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:08 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1634631078' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:18:08.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:08 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/243807172' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:09.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.043+0000 7fda0c200640 1 -- 172.21.15.69:0/1264327998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda04109c20 con 0x7fda041a7b50 2024-04-24T09:18:09.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.044+0000 7fda02000640 1 -- 172.21.15.69:0/1264327998 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) ==== 93+0+10 (secure 0 0 0) 0x7fd9fc0bcd10 con 0x7fda041a7b50 2024-04-24T09:18:09.044 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-24T09:18:09.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.046+0000 7fda0c200640 1 -- 172.21.15.69:0/1264327998 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd9ec0a3ec0 msgr2=0x7fd9ec0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:09.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.046+0000 7fda0c200640 1 --2- 172.21.15.69:0/1264327998 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd9ec0a3ec0 0x7fd9ec0a6380 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009d60 tx=0x7fd9f4009110 comp rx=0 tx=0).stop 2024-04-24T09:18:09.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.046+0000 7fda0c200640 1 -- 172.21.15.69:0/1264327998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda041a7b50 msgr2=0x7fda041a8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:09.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.046+0000 7fda0c200640 1 --2- 172.21.15.69:0/1264327998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda041a7b50 0x7fda041a8000 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00d090 tx=0x7fd9fc033f90 comp rx=0 tx=0).stop 2024-04-24T09:18:09.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.046+0000 7fda0c200640 1 -- 172.21.15.69:0/1264327998 shutdown_connections 2024-04-24T09:18:09.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.046+0000 7fda0c200640 1 --2- 172.21.15.69:0/1264327998 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7fd9ec0a3ec0 0x7fd9ec0a6380 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.046+0000 7fda0c200640 1 --2- 172.21.15.69:0/1264327998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda041a7b50 0x7fda041a8000 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.046+0000 7fda0c200640 1 --2- 172.21.15.69:0/1264327998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda0410dd30 0x7fda041a37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.046+0000 7fda0c200640 1 --2- 172.21.15.69:0/1264327998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda04109270 0x7fda041a3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.046+0000 7fda0c200640 1 -- 172.21.15.69:0/1264327998 >> 172.21.15.69:0/1264327998 conn(0x7fda0406d110 msgr2=0x7fda04071f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:09.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.046+0000 7fda0c200640 1 -- 172.21.15.69:0/1264327998 shutdown_connections 2024-04-24T09:18:09.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.046+0000 7fda0c200640 1 -- 172.21.15.69:0/1264327998 wait complete. 2024-04-24T09:18:09.063 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:18:09.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.234+0000 7f66eac00640 1 -- 172.21.15.69:0/3929816644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66e410d500 msgr2=0x7f66e410d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:09.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.234+0000 7f66eac00640 1 --2- 172.21.15.69:0/3929816644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66e410d500 0x7f66e410d8e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009930 tx=0x7f66d802a420 comp rx=0 tx=0).stop 2024-04-24T09:18:09.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.235+0000 7f66eac00640 1 -- 172.21.15.69:0/3929816644 shutdown_connections 2024-04-24T09:18:09.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.235+0000 7f66eac00640 1 --2- 172.21.15.69:0/3929816644 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f66e4108f40 0x7f66e41153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.235+0000 7f66eac00640 1 --2- 172.21.15.69:0/3929816644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66e4108580 0x7f66e4108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.235+0000 7f66eac00640 1 --2- 172.21.15.69:0/3929816644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66e410d500 0x7f66e410d8e0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.235+0000 7f66eac00640 1 -- 172.21.15.69:0/3929816644 >> 172.21.15.69:0/3929816644 conn(0x7f66e4077e20 msgr2=0x7f66e4106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:09.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.235+0000 7f66eac00640 1 -- 172.21.15.69:0/3929816644 shutdown_connections 2024-04-24T09:18:09.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.235+0000 7f66eac00640 1 -- 172.21.15.69:0/3929816644 wait complete. 2024-04-24T09:18:09.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.236+0000 7f66eac00640 1 Processor -- start 2024-04-24T09:18:09.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.236+0000 7f66eac00640 1 -- start start 2024-04-24T09:18:09.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.236+0000 7f66eac00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66e4108580 0x7f66e41a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:09.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.236+0000 7f66eac00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66e4108f40 0x7f66e41a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:09.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.236+0000 7f66eac00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f66e410d500 0x7f66e419ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:09.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.236+0000 7f66eac00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f66e4118770 con 0x7f66e4108580 2024-04-24T09:18:09.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.236+0000 7f66eac00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f66e41185f0 con 0x7f66e4108f40 2024-04-24T09:18:09.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.236+0000 7f66eac00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f66e41188f0 con 0x7f66e410d500 2024-04-24T09:18:09.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.236+0000 7f66e9800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66e4108580 0x7f66e41a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:09.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.236+0000 7f66e9800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66e4108580 0x7f66e41a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55880/0 (socket says 172.21.15.69:55880) 2024-04-24T09:18:09.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.236+0000 7f66e9800640 1 -- 172.21.15.69:0/2643276276 learned_addr learned my addr 172.21.15.69:0/2643276276 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:18:09.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.237+0000 7f66e8e00640 1 --2- 172.21.15.69:0/2643276276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66e4108f40 0x7f66e41a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:09.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.237+0000 7f66ea200640 1 --2- 172.21.15.69:0/2643276276 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f66e410d500 0x7f66e419ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:09.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.237+0000 7f66e9800640 1 -- 172.21.15.69:0/2643276276 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f66e410d500 msgr2=0x7f66e419ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:09.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.237+0000 7f66e9800640 1 --2- 172.21.15.69:0/2643276276 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f66e410d500 0x7f66e419ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.237+0000 7f66e9800640 1 -- 172.21.15.69:0/2643276276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66e4108f40 msgr2=0x7f66e41a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:09.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.237+0000 7f66e9800640 1 --2- 172.21.15.69:0/2643276276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66e4108f40 0x7f66e41a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.237+0000 7f66e9800640 1 -- 172.21.15.69:0/2643276276 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e419f1b0 con 0x7f66e4108580 2024-04-24T09:18:09.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.237+0000 7f66ea200640 1 --2- 172.21.15.69:0/2643276276 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f66e410d500 0x7f66e419ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:18:09.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.237+0000 7f66e8e00640 1 --2- 172.21.15.69:0/2643276276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66e4108f40 0x7f66e41a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:18:09.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.237+0000 7f66e9800640 1 --2- 172.21.15.69:0/2643276276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66e4108580 0x7f66e41a4880 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f66d8031440 tx=0x7f66d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:09.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.237+0000 7f66e2000640 1 -- 172.21.15.69:0/2643276276 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d803f030 con 0x7f66e4108580 2024-04-24T09:18:09.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.237+0000 7f66e2000640 1 -- 172.21.15.69:0/2643276276 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66d8003aa0 con 0x7f66e4108580 2024-04-24T09:18:09.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.237+0000 7f66eac00640 1 -- 172.21.15.69:0/2643276276 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e419f440 con 0x7f66e4108580 2024-04-24T09:18:09.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.237+0000 7f66e2000640 1 -- 172.21.15.69:0/2643276276 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8035070 con 0x7f66e4108580 2024-04-24T09:18:09.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.237+0000 7f66eac00640 1 -- 172.21.15.69:0/2643276276 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e41ab740 con 0x7f66e4108580 2024-04-24T09:18:09.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.239+0000 7f66eac00640 1 -- 172.21.15.69:0/2643276276 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e4069190 con 0x7f66e4108580 2024-04-24T09:18:09.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.241+0000 7f66e2000640 1 -- 172.21.15.69:0/2643276276 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f66d8042030 con 0x7f66e4108580 2024-04-24T09:18:09.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.242+0000 7f66e2000640 1 --2- 172.21.15.69:0/2643276276 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f66cc0a3d20 0x7f66cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:09.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.242+0000 7f66e2000640 1 -- 172.21.15.69:0/2643276276 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f66d80f13c0 con 0x7f66e4108580 2024-04-24T09:18:09.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.242+0000 7f66e8e00640 1 --2- 172.21.15.69:0/2643276276 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f66cc0a3d20 0x7f66cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:09.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.243+0000 7f66e8e00640 1 --2- 172.21.15.69:0/2643276276 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f66cc0a3d20 0x7f66cc0a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f66e4107fb0 tx=0x7f66d0005d90 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:09.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.246+0000 7f66e2000640 1 -- 172.21.15.69:0/2643276276 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66d80bd610 con 0x7f66e4108580 2024-04-24T09:18:09.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.436+0000 7f66eac00640 1 -- 172.21.15.69:0/2643276276 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f66e4110a50 con 0x7f66e4108580 2024-04-24T09:18:09.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.437+0000 7f66e2000640 1 -- 172.21.15.69:0/2643276276 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) ==== 104+0+11 (secure 0 0 0) 0x7f66d80bcf10 con 0x7f66e4108580 2024-04-24T09:18:09.437 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:18:09.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.439+0000 7f66eac00640 1 -- 172.21.15.69:0/2643276276 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f66cc0a3d20 msgr2=0x7f66cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:09.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.439+0000 7f66eac00640 1 --2- 172.21.15.69:0/2643276276 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f66cc0a3d20 0x7f66cc0a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f66e4107fb0 tx=0x7f66d0005d90 comp rx=0 tx=0).stop 2024-04-24T09:18:09.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.439+0000 7f66eac00640 1 -- 172.21.15.69:0/2643276276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66e4108580 msgr2=0x7f66e41a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:09.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.439+0000 7f66eac00640 1 --2- 172.21.15.69:0/2643276276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66e4108580 0x7f66e41a4880 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f66d8031440 tx=0x7f66d8004270 comp rx=0 tx=0).stop 2024-04-24T09:18:09.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.439+0000 7f66eac00640 1 -- 172.21.15.69:0/2643276276 shutdown_connections 2024-04-24T09:18:09.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.439+0000 7f66eac00640 1 --2- 172.21.15.69:0/2643276276 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f66cc0a3d20 0x7f66cc0a61e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.439+0000 7f66eac00640 1 --2- 172.21.15.69:0/2643276276 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f66e410d500 0x7f66e419ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.439+0000 7f66eac00640 1 --2- 172.21.15.69:0/2643276276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66e4108f40 0x7f66e41a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.439+0000 7f66eac00640 1 --2- 172.21.15.69:0/2643276276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66e4108580 0x7f66e41a4880 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.439+0000 7f66eac00640 1 -- 172.21.15.69:0/2643276276 >> 172.21.15.69:0/2643276276 conn(0x7f66e4077e20 msgr2=0x7f66e4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:09.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.439+0000 7f66eac00640 1 -- 172.21.15.69:0/2643276276 shutdown_connections 2024-04-24T09:18:09.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.440+0000 7f66eac00640 1 -- 172.21.15.69:0/2643276276 wait complete. 2024-04-24T09:18:09.457 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:18:09.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.658+0000 7f9b98c00640 1 -- 172.21.15.69:0/3717339573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b94108580 msgr2=0x7f9b94108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:09.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.658+0000 7f9b98c00640 1 --2- 172.21.15.69:0/3717339573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b94108580 0x7f9b94108a00 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009930 tx=0x7f9b7c02a420 comp rx=0 tx=0).stop 2024-04-24T09:18:09.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.658+0000 7f9b98c00640 1 -- 172.21.15.69:0/3717339573 shutdown_connections 2024-04-24T09:18:09.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.658+0000 7f9b98c00640 1 --2- 172.21.15.69:0/3717339573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b94108f40 0x7f9b941153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.658+0000 7f9b98c00640 1 --2- 172.21.15.69:0/3717339573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b94108580 0x7f9b94108a00 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.658+0000 7f9b98c00640 1 --2- 172.21.15.69:0/3717339573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b9410d500 0x7f9b9410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.658+0000 7f9b98c00640 1 -- 172.21.15.69:0/3717339573 >> 172.21.15.69:0/3717339573 conn(0x7f9b94077e20 msgr2=0x7f9b94106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:09.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.658+0000 7f9b98c00640 1 -- 172.21.15.69:0/3717339573 shutdown_connections 2024-04-24T09:18:09.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.658+0000 7f9b98c00640 1 -- 172.21.15.69:0/3717339573 wait complete. 2024-04-24T09:18:09.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.659+0000 7f9b98c00640 1 Processor -- start 2024-04-24T09:18:09.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.659+0000 7f9b98c00640 1 -- start start 2024-04-24T09:18:09.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.660+0000 7f9b98c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b94108580 0x7f9b9419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:09.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.660+0000 7f9b98c00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b94108f40 0x7f9b9419f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:09.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.660+0000 7f9b98c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b9410d500 0x7f9b941a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:09.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.660+0000 7f9b98c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94118870 con 0x7f9b9410d500 2024-04-24T09:18:09.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.660+0000 7f9b98c00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941186f0 con 0x7f9b94108f40 2024-04-24T09:18:09.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.660+0000 7f9b98c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941189f0 con 0x7f9b94108580 2024-04-24T09:18:09.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.660+0000 7f9b93e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b9410d500 0x7f9b941a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:09.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.660+0000 7f9b93e00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b9410d500 0x7f9b941a36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55904/0 (socket says 172.21.15.69:55904) 2024-04-24T09:18:09.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.660+0000 7f9b93e00640 1 -- 172.21.15.69:0/973569653 learned_addr learned my addr 172.21.15.69:0/973569653 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:18:09.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.660+0000 7f9b92a00640 1 --2- 172.21.15.69:0/973569653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b94108f40 0x7f9b9419f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.660+0000 7f9b93400640 1 --2- 172.21.15.69:0/973569653 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b94108580 0x7f9b9419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.660+0000 7f9b93e00640 1 -- 172.21.15.69:0/973569653 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b94108580 msgr2=0x7f9b9419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.660+0000 7f9b93e00640 1 --2- 172.21.15.69:0/973569653 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b94108580 0x7f9b9419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.660+0000 7f9b93e00640 1 -- 172.21.15.69:0/973569653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b94108f40 msgr2=0x7f9b9419f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.660+0000 7f9b93e00640 1 --2- 172.21.15.69:0/973569653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b94108f40 0x7f9b9419f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.661+0000 7f9b93e00640 1 -- 172.21.15.69:0/973569653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b941a3e00 con 0x7f9b9410d500 2024-04-24T09:18:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.661+0000 7f9b93400640 1 --2- 172.21.15.69:0/973569653 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b94108580 0x7f9b9419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:18:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.661+0000 7f9b93e00640 1 --2- 172.21.15.69:0/973569653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b9410d500 0x7f9b941a36d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00b3e0 tx=0x7f9b8c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:09.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.661+0000 7f9b8be00640 1 -- 172.21.15.69:0/973569653 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c015020 con 0x7f9b9410d500 2024-04-24T09:18:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.661+0000 7f9b8be00640 1 -- 172.21.15.69:0/973569653 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b8c011ce0 con 0x7f9b9410d500 2024-04-24T09:18:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.661+0000 7f9b98c00640 1 -- 172.21.15.69:0/973569653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b941a40f0 con 0x7f9b9410d500 2024-04-24T09:18:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.661+0000 7f9b8be00640 1 -- 172.21.15.69:0/973569653 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c010de0 con 0x7f9b9410d500 2024-04-24T09:18:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.661+0000 7f9b98c00640 1 -- 172.21.15.69:0/973569653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b941ab980 con 0x7f9b9410d500 2024-04-24T09:18:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.662+0000 7f9b98c00640 1 -- 172.21.15.69:0/973569653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b94069190 con 0x7f9b9410d500 2024-04-24T09:18:09.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.664+0000 7f9b8be00640 1 -- 172.21.15.69:0/973569653 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9b8c018020 con 0x7f9b9410d500 2024-04-24T09:18:09.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.666+0000 7f9b8be00640 1 --2- 172.21.15.69:0/973569653 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9b780a3d20 0x7f9b780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.667+0000 7f9b8be00640 1 -- 172.21.15.69:0/973569653 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f9b8c0d0430 con 0x7f9b9410d500 2024-04-24T09:18:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.667+0000 7f9b93400640 1 --2- 172.21.15.69:0/973569653 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9b780a3d20 0x7f9b780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:09.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.667+0000 7f9b93400640 1 --2- 172.21.15.69:0/973569653 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9b780a3d20 0x7f9b780a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9b9406b340 tx=0x7f9b840073c0 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:09.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.671+0000 7f9b8be00640 1 -- 172.21.15.69:0/973569653 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b8c01f020 con 0x7f9b9410d500 2024-04-24T09:18:09.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:09 smithi130 ceph-mon[31553]: pgmap v343: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-24T09:18:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:09 smithi130 ceph-mon[31553]: osdmap e102: 12 total, 12 up, 12 in 2024-04-24T09:18:09.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:09 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1264327998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:09.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:09 smithi081 ceph-mon[30961]: pgmap v343: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-24T09:18:09.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:09 smithi081 ceph-mon[30961]: osdmap e102: 12 total, 12 up, 12 in 2024-04-24T09:18:09.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:09 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1264327998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:09.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:09 smithi069 ceph-mon[28170]: pgmap v343: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-24T09:18:09.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:09 smithi069 ceph-mon[28170]: osdmap e102: 12 total, 12 up, 12 in 2024-04-24T09:18:09.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:09 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1264327998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:09.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.860+0000 7f9b98c00640 1 -- 172.21.15.69:0/973569653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9b94110a50 con 0x7f9b9410d500 2024-04-24T09:18:09.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.862+0000 7f9b8be00640 1 -- 172.21.15.69:0/973569653 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) ==== 100+0+11 (secure 0 0 0) 0x7f9b8c09c140 con 0x7f9b9410d500 2024-04-24T09:18:09.862 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-24T09:18:09.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.864+0000 7f9b98c00640 1 -- 172.21.15.69:0/973569653 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9b780a3d20 msgr2=0x7f9b780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:09.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.864+0000 7f9b98c00640 1 --2- 172.21.15.69:0/973569653 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9b780a3d20 0x7f9b780a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9b9406b340 tx=0x7f9b840073c0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.864+0000 7f9b98c00640 1 -- 172.21.15.69:0/973569653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b9410d500 msgr2=0x7f9b941a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:09.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.864+0000 7f9b98c00640 1 --2- 172.21.15.69:0/973569653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b9410d500 0x7f9b941a36d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00b3e0 tx=0x7f9b8c009f90 comp rx=0 tx=0).stop 2024-04-24T09:18:09.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.864+0000 7f9b98c00640 1 -- 172.21.15.69:0/973569653 shutdown_connections 2024-04-24T09:18:09.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.864+0000 7f9b98c00640 1 --2- 172.21.15.69:0/973569653 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f9b780a3d20 0x7f9b780a61e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.864+0000 7f9b98c00640 1 --2- 172.21.15.69:0/973569653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b9410d500 0x7f9b941a36d0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.864+0000 7f9b98c00640 1 --2- 172.21.15.69:0/973569653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b94108f40 0x7f9b9419f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.864+0000 7f9b98c00640 1 --2- 172.21.15.69:0/973569653 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b94108580 0x7f9b9419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:09.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.864+0000 7f9b98c00640 1 -- 172.21.15.69:0/973569653 >> 172.21.15.69:0/973569653 conn(0x7f9b94077e20 msgr2=0x7f9b940784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:09.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.864+0000 7f9b98c00640 1 -- 172.21.15.69:0/973569653 shutdown_connections 2024-04-24T09:18:09.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:09.864+0000 7f9b98c00640 1 -- 172.21.15.69:0/973569653 wait complete. 2024-04-24T09:18:09.882 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T09:18:10.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.079+0000 7f7353200640 1 -- 172.21.15.69:0/2919589055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f734c108580 msgr2=0x7f734c108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:10.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.079+0000 7f7353200640 1 --2- 172.21.15.69:0/2919589055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f734c108580 0x7f734c108a00 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f733c009930 tx=0x7f733c02a420 comp rx=0 tx=0).stop 2024-04-24T09:18:10.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.079+0000 7f7353200640 1 -- 172.21.15.69:0/2919589055 shutdown_connections 2024-04-24T09:18:10.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.079+0000 7f7353200640 1 --2- 172.21.15.69:0/2919589055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f734c108f40 0x7f734c1153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.079+0000 7f7353200640 1 --2- 172.21.15.69:0/2919589055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f734c108580 0x7f734c108a00 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.079+0000 7f7353200640 1 --2- 172.21.15.69:0/2919589055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f734c10d500 0x7f734c10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.079+0000 7f7353200640 1 -- 172.21.15.69:0/2919589055 >> 172.21.15.69:0/2919589055 conn(0x7f734c077e20 msgr2=0x7f734c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:10.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.079+0000 7f7353200640 1 -- 172.21.15.69:0/2919589055 shutdown_connections 2024-04-24T09:18:10.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.079+0000 7f7353200640 1 -- 172.21.15.69:0/2919589055 wait complete. 2024-04-24T09:18:10.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.080+0000 7f7353200640 1 Processor -- start 2024-04-24T09:18:10.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.080+0000 7f7353200640 1 -- start start 2024-04-24T09:18:10.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.080+0000 7f7353200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f734c108580 0x7f734c1a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:10.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.080+0000 7f7353200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f734c108f40 0x7f734c1a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:10.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.080+0000 7f7353200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f734c10d500 0x7f734c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:10.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.080+0000 7f7353200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f734c118760 con 0x7f734c108f40 2024-04-24T09:18:10.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.080+0000 7f7353200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1185e0 con 0x7f734c10d500 2024-04-24T09:18:10.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.080+0000 7f7353200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1188e0 con 0x7f734c108580 2024-04-24T09:18:10.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.081+0000 7f7351400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f734c108f40 0x7f734c1a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:10.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.081+0000 7f7351400640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f734c108f40 0x7f734c1a4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55920/0 (socket says 172.21.15.69:55920) 2024-04-24T09:18:10.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.081+0000 7f7351400640 1 -- 172.21.15.69:0/148513414 learned_addr learned my addr 172.21.15.69:0/148513414 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:18:10.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.081+0000 7f7352800640 1 --2- 172.21.15.69:0/148513414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f734c10d500 0x7f734c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:10.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.081+0000 7f7351e00640 1 --2- 172.21.15.69:0/148513414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f734c108580 0x7f734c1a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:10.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.081+0000 7f7351400640 1 -- 172.21.15.69:0/148513414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f734c108580 msgr2=0x7f734c1a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:10.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.081+0000 7f7351400640 1 --2- 172.21.15.69:0/148513414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f734c108580 0x7f734c1a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.081+0000 7f7351400640 1 -- 172.21.15.69:0/148513414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f734c10d500 msgr2=0x7f734c19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:10.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.081+0000 7f7351400640 1 --2- 172.21.15.69:0/148513414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f734c10d500 0x7f734c19eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.081+0000 7f7351400640 1 -- 172.21.15.69:0/148513414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c19f260 con 0x7f734c108f40 2024-04-24T09:18:10.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.081+0000 7f7351e00640 1 --2- 172.21.15.69:0/148513414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f734c108580 0x7f734c1a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:18:10.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.081+0000 7f7351400640 1 --2- 172.21.15.69:0/148513414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f734c108f40 0x7f734c1a4e20 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f733c005940 tx=0x7f733c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:10.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.082+0000 7f7346a00640 1 -- 172.21.15.69:0/148513414 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c03f030 con 0x7f734c108f40 2024-04-24T09:18:10.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.082+0000 7f7353200640 1 -- 172.21.15.69:0/148513414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734c19f4f0 con 0x7f734c108f40 2024-04-24T09:18:10.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.082+0000 7f7346a00640 1 -- 172.21.15.69:0/148513414 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f733c003cb0 con 0x7f734c108f40 2024-04-24T09:18:10.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.082+0000 7f7353200640 1 -- 172.21.15.69:0/148513414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c1ab7a0 con 0x7f734c108f40 2024-04-24T09:18:10.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.082+0000 7f7346a00640 1 -- 172.21.15.69:0/148513414 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c049060 con 0x7f734c108f40 2024-04-24T09:18:10.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.083+0000 7f7353200640 1 -- 172.21.15.69:0/148513414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f734c069190 con 0x7f734c108f40 2024-04-24T09:18:10.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.085+0000 7f7346a00640 1 -- 172.21.15.69:0/148513414 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f733c042030 con 0x7f734c108f40 2024-04-24T09:18:10.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.087+0000 7f7346a00640 1 --2- 172.21.15.69:0/148513414 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f73340a3d20 0x7f73340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:10.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.087+0000 7f7346a00640 1 -- 172.21.15.69:0/148513414 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f733c0f1590 con 0x7f734c108f40 2024-04-24T09:18:10.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.087+0000 7f7351e00640 1 --2- 172.21.15.69:0/148513414 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f73340a3d20 0x7f73340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:10.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.088+0000 7f7351e00640 1 --2- 172.21.15.69:0/148513414 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f73340a3d20 0x7f73340a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f734c07a010 tx=0x7f7340005d50 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:10.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.091+0000 7f7346a00640 1 -- 172.21.15.69:0/148513414 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f733c0bd840 con 0x7f734c108f40 2024-04-24T09:18:10.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.289+0000 7f7353200640 1 -- 172.21.15.69:0/148513414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f734c02d620 con 0x7f734c108f40 2024-04-24T09:18:10.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.289+0000 7f7346a00640 1 -- 172.21.15.69:0/148513414 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v102) ==== 103+0+11 (secure 0 0 0) 0x7f733c0bd140 con 0x7f734c108f40 2024-04-24T09:18:10.289 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-24T09:18:10.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.292+0000 7f7353200640 1 -- 172.21.15.69:0/148513414 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f73340a3d20 msgr2=0x7f73340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:10.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.292+0000 7f7353200640 1 --2- 172.21.15.69:0/148513414 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f73340a3d20 0x7f73340a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f734c07a010 tx=0x7f7340005d50 comp rx=0 tx=0).stop 2024-04-24T09:18:10.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.292+0000 7f7353200640 1 -- 172.21.15.69:0/148513414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f734c108f40 msgr2=0x7f734c1a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:10.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.292+0000 7f7353200640 1 --2- 172.21.15.69:0/148513414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f734c108f40 0x7f734c1a4e20 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f733c005940 tx=0x7f733c0042f0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.292+0000 7f7353200640 1 -- 172.21.15.69:0/148513414 shutdown_connections 2024-04-24T09:18:10.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.292+0000 7f7353200640 1 --2- 172.21.15.69:0/148513414 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f73340a3d20 0x7f73340a61e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.292+0000 7f7353200640 1 --2- 172.21.15.69:0/148513414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f734c10d500 0x7f734c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.292+0000 7f7353200640 1 --2- 172.21.15.69:0/148513414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f734c108f40 0x7f734c1a4e20 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.292+0000 7f7353200640 1 --2- 172.21.15.69:0/148513414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f734c108580 0x7f734c1a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.292+0000 7f7353200640 1 -- 172.21.15.69:0/148513414 >> 172.21.15.69:0/148513414 conn(0x7f734c077e20 msgr2=0x7f734c078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:10.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.292+0000 7f7353200640 1 -- 172.21.15.69:0/148513414 shutdown_connections 2024-04-24T09:18:10.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.292+0000 7f7353200640 1 -- 172.21.15.69:0/148513414 wait complete. 2024-04-24T09:18:10.310 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T09:18:10.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.505+0000 7f46a8200640 1 -- 172.21.15.69:0/3253546080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a0109580 msgr2=0x7f46a0111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:10.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.505+0000 7f46a8200640 1 --2- 172.21.15.69:0/3253546080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a0109580 0x7f46a0111150 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f4698009ca0 tx=0x7f469802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.506+0000 7f46a8200640 1 -- 172.21.15.69:0/3253546080 shutdown_connections 2024-04-24T09:18:10.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.506+0000 7f46a8200640 1 --2- 172.21.15.69:0/3253546080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a0109580 0x7f46a0111150 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.506+0000 7f46a8200640 1 --2- 172.21.15.69:0/3253546080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46a0104e60 0x7f46a0108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.506+0000 7f46a8200640 1 --2- 172.21.15.69:0/3253546080 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a01044b0 0x7f46a0104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.506+0000 7f46a8200640 1 -- 172.21.15.69:0/3253546080 >> 172.21.15.69:0/3253546080 conn(0x7f46a00ffd90 msgr2=0x7f46a01021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:10.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.506+0000 7f46a8200640 1 -- 172.21.15.69:0/3253546080 shutdown_connections 2024-04-24T09:18:10.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.506+0000 7f46a8200640 1 -- 172.21.15.69:0/3253546080 wait complete. 2024-04-24T09:18:10.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.506+0000 7f46a8200640 1 Processor -- start 2024-04-24T09:18:10.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.507+0000 7f46a8200640 1 -- start start 2024-04-24T09:18:10.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.507+0000 7f46a8200640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a01044b0 0x7f46a01a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:10.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.507+0000 7f46a8200640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a0109580 0x7f46a01a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:10.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.507+0000 7f46a8200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46a019a7e0 0x7f46a019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:10.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.507+0000 7f46a8200640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0114540 con 0x7f46a01044b0 2024-04-24T09:18:10.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.507+0000 7f46a8200640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01143c0 con 0x7f46a0109580 2024-04-24T09:18:10.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.507+0000 7f46a8200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01146c0 con 0x7f46a019a7e0 2024-04-24T09:18:10.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.507+0000 7f46a5800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a01044b0 0x7f46a01a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:10.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.507+0000 7f46a5800640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a01044b0 0x7f46a01a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55946/0 (socket says 172.21.15.69:55946) 2024-04-24T09:18:10.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.507+0000 7f46a5800640 1 -- 172.21.15.69:0/1229766426 learned_addr learned my addr 172.21.15.69:0/1229766426 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-24T09:18:10.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.508+0000 7f46a5800640 1 -- 172.21.15.69:0/1229766426 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46a019a7e0 msgr2=0x7f46a019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:10.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.508+0000 7f46a5800640 1 --2- 172.21.15.69:0/1229766426 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46a019a7e0 0x7f46a019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.508+0000 7f46a5800640 1 -- 172.21.15.69:0/1229766426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a0109580 msgr2=0x7f46a01a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:10.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.508+0000 7f46a5800640 1 --2- 172.21.15.69:0/1229766426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a0109580 0x7f46a01a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.508+0000 7f46a5800640 1 -- 172.21.15.69:0/1229766426 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a019b370 con 0x7f46a01044b0 2024-04-24T09:18:10.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.508+0000 7f46a5800640 1 --2- 172.21.15.69:0/1229766426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a01044b0 0x7f46a01a0670 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f469400c920 tx=0x7f469400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:10.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.508+0000 7f469e000640 1 -- 172.21.15.69:0/1229766426 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694007c70 con 0x7f46a01044b0 2024-04-24T09:18:10.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.508+0000 7f469e000640 1 -- 172.21.15.69:0/1229766426 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4694007e10 con 0x7f46a01044b0 2024-04-24T09:18:10.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.508+0000 7f46a8200640 1 -- 172.21.15.69:0/1229766426 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a019b660 con 0x7f46a01044b0 2024-04-24T09:18:10.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.508+0000 7f469e000640 1 -- 172.21.15.69:0/1229766426 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694018820 con 0x7f46a01044b0 2024-04-24T09:18:10.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.508+0000 7f46a8200640 1 -- 172.21.15.69:0/1229766426 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a01a7840 con 0x7f46a01044b0 2024-04-24T09:18:10.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.509+0000 7f46a8200640 1 -- 172.21.15.69:0/1229766426 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a0105980 con 0x7f46a01044b0 2024-04-24T09:18:10.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.511+0000 7f469e000640 1 -- 172.21.15.69:0/1229766426 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f46940189c0 con 0x7f46a01044b0 2024-04-24T09:18:10.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.512+0000 7f469e000640 1 --2- 172.21.15.69:0/1229766426 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f46880a3d20 0x7f46880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:10.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.513+0000 7f469e000640 1 -- 172.21.15.69:0/1229766426 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f46940d1100 con 0x7f46a01044b0 2024-04-24T09:18:10.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.513+0000 7f46a4e00640 1 --2- 172.21.15.69:0/1229766426 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f46880a3d20 0x7f46880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:10.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.513+0000 7f46a4e00640 1 --2- 172.21.15.69:0/1229766426 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f46880a3d20 0x7f46880a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f468c005d80 tx=0x7f468c005c90 comp rx=0 tx=0).ready entity=mgr.15465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:10.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.515+0000 7f469e000640 1 -- 172.21.15.69:0/1229766426 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46a019b660 con 0x7f46a01044b0 2024-04-24T09:18:10.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:10 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/2643276276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:18:10.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:10 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/973569653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:18:10.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:10 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/148513414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:18:10.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:10 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/2643276276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:18:10.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:10 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/973569653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:18:10.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:10 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/148513414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:18:10.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.716+0000 7f46a8200640 1 -- 172.21.15.69:0/1229766426 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f46a0069270 con 0x7f46a01044b0 2024-04-24T09:18:10.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.717+0000 7f469e000640 1 -- 172.21.15.69:0/1229766426 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v102) ==== 81+0+945 (secure 0 0 0) 0x7f4694014030 con 0x7f46a01044b0 2024-04-24T09:18:10.717 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/660205566 2024-04-25T09:07:03.863940+0000 2024-04-24T09:18:10.717 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/1098677190 2024-04-25T09:07:03.863940+0000 2024-04-24T09:18:10.717 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3145713060 2024-04-25T09:07:03.863940+0000 2024-04-24T09:18:10.717 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3826117580 2024-04-25T08:54:23.196411+0000 2024-04-24T09:18:10.717 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1006368686 2024-04-25T08:54:23.196411+0000 2024-04-24T09:18:10.718 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3008138297 2024-04-25T09:07:03.863940+0000 2024-04-24T09:18:10.718 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/3470783122 2024-04-25T08:54:23.196411+0000 2024-04-24T09:18:10.718 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2102503077 2024-04-25T08:53:48.948375+0000 2024-04-24T09:18:10.718 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2901345957 2024-04-25T09:07:03.863940+0000 2024-04-24T09:18:10.718 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/3470783122 2024-04-25T08:54:23.196411+0000 2024-04-24T09:18:10.718 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1041454348 2024-04-25T08:53:48.948375+0000 2024-04-24T09:18:10.718 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/3911821813 2024-04-25T08:53:48.948375+0000 2024-04-24T09:18:10.718 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/1098677190 2024-04-25T09:07:03.863940+0000 2024-04-24T09:18:10.718 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/3911821813 2024-04-25T08:53:48.948375+0000 2024-04-24T09:18:10.719 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/4072155336 2024-04-25T08:53:48.948375+0000 2024-04-24T09:18:10.719 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1278076203 2024-04-25T08:54:23.196411+0000 2024-04-24T09:18:10.720 INFO:teuthology.orchestra.run.smithi069.stderr:listed 16 entries 2024-04-24T09:18:10.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.719+0000 7f46a8200640 1 -- 172.21.15.69:0/1229766426 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f46880a3d20 msgr2=0x7f46880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:10.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.719+0000 7f46a8200640 1 --2- 172.21.15.69:0/1229766426 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f46880a3d20 0x7f46880a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f468c005d80 tx=0x7f468c005c90 comp rx=0 tx=0).stop 2024-04-24T09:18:10.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.719+0000 7f46a8200640 1 -- 172.21.15.69:0/1229766426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a01044b0 msgr2=0x7f46a01a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:10.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.719+0000 7f46a8200640 1 --2- 172.21.15.69:0/1229766426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a01044b0 0x7f46a01a0670 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f469400c920 tx=0x7f469400cde0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.719+0000 7f46a8200640 1 -- 172.21.15.69:0/1229766426 shutdown_connections 2024-04-24T09:18:10.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.719+0000 7f46a8200640 1 --2- 172.21.15.69:0/1229766426 >> [v2:172.21.15.69:6800/2822728880,v1:172.21.15.69:6801/2822728880] conn(0x7f46880a3d20 0x7f46880a61e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.719+0000 7f46a8200640 1 --2- 172.21.15.69:0/1229766426 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46a019a7e0 0x7f46a019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.719+0000 7f46a8200640 1 --2- 172.21.15.69:0/1229766426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a0109580 0x7f46a01a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.719+0000 7f46a8200640 1 --2- 172.21.15.69:0/1229766426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a01044b0 0x7f46a01a0670 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:10.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.719+0000 7f46a8200640 1 -- 172.21.15.69:0/1229766426 >> 172.21.15.69:0/1229766426 conn(0x7f46a00ffd90 msgr2=0x7f46a01020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:10.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.720+0000 7f46a8200640 1 -- 172.21.15.69:0/1229766426 shutdown_connections 2024-04-24T09:18:10.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-24T09:18:10.720+0000 7f46a8200640 1 -- 172.21.15.69:0/1229766426 wait complete. 2024-04-24T09:18:10.737 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi069.front.sepia.ceph.com... 2024-04-24T09:18:10.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:18:10.738 DEBUG:teuthology.orchestra.run.smithi069:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T09:18:10.812 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi069.stderr:ceph-fuse[79744]: fuse finished with error 0 and tester_r 0 2024-04-24T09:18:10.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:10 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/2643276276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:18:10.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:10 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/973569653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:18:10.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:10 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/148513414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:18:10.857 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T09:18:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:11 smithi130 ceph-mon[31553]: pgmap v345: 129 pgs: 129 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-24T09:18:11.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:11 smithi130 ceph-mon[31553]: from='client.? 172.21.15.69:0/1229766426' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:18:11.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:11 smithi081 ceph-mon[30961]: pgmap v345: 129 pgs: 129 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-24T09:18:11.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:11 smithi081 ceph-mon[30961]: from='client.? 172.21.15.69:0/1229766426' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:18:11.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:11 smithi069 ceph-mon[28170]: pgmap v345: 129 pgs: 129 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-24T09:18:11.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:11 smithi069 ceph-mon[28170]: from='client.? 172.21.15.69:0/1229766426' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:18:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:13 smithi130 ceph-mon[31553]: pgmap v346: 129 pgs: 129 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-24T09:18:13.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:13 smithi130 ceph-mon[31553]: osdmap e103: 12 total, 12 up, 12 in 2024-04-24T09:18:13.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:13 smithi081 ceph-mon[30961]: pgmap v346: 129 pgs: 129 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-24T09:18:13.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:13 smithi081 ceph-mon[30961]: osdmap e103: 12 total, 12 up, 12 in 2024-04-24T09:18:13.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:13 smithi069 ceph-mon[28170]: pgmap v346: 129 pgs: 129 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-24T09:18:13.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:13 smithi069 ceph-mon[28170]: osdmap e103: 12 total, 12 up, 12 in 2024-04-24T09:18:15.704 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:15 smithi130 ceph-mon[31553]: pgmap v348: 129 pgs: 129 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T09:18:15.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:15 smithi081 ceph-mon[30961]: pgmap v348: 129 pgs: 129 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T09:18:15.841 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:15 smithi069 ceph-mon[28170]: pgmap v348: 129 pgs: 129 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T09:18:16.959 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi069.front.sepia.ceph.com 2024-04-24T09:18:16.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:18:16.959 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T09:18:17.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:18:17.019 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-04-24T09:18:17.071 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T09:18:17.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:18:17.072 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T09:18:17.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:18:17.158 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-04-24T09:18:17.272 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-24T09:18:17.283 INFO:tasks.cephadm:Teardown begin 2024-04-24T09:18:17.283 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/cephadm.py", line 2246, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-04-24T09:18:17.286 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:18:17.348 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:18:17.436 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:18:17.530 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-24T09:18:17.531 DEBUG:teuthology.orchestra.run.smithi069:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T09:18:17.586 DEBUG:teuthology.orchestra.run.smithi081:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T09:18:17.646 DEBUG:teuthology.orchestra.run.smithi130:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T09:18:17.694 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:17 smithi081 ceph-mon[30961]: pgmap v349: 129 pgs: 129 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-24T09:18:17.702 INFO:tasks.cephadm:Stopping all daemons... 2024-04-24T09:18:17.703 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-24T09:18:17.703 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mon.a 2024-04-24T09:18:17.703 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:17 smithi130 ceph-mon[31553]: pgmap v349: 129 pgs: 129 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-24T09:18:17.781 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:17 smithi069 ceph-mon[28170]: pgmap v349: 129 pgs: 129 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-24T09:18:18.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:17 smithi069 systemd[1]: Stopping Ceph mon.a for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:18:18.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:18 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a[28166]: 2024-04-24T09:18:18.234+0000 7fe78e600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:18:18.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:18 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a[28166]: 2024-04-24T09:18:18.234+0000 7fe78e600700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-24T09:18:18.592 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:18 smithi069 podman[96150]: 2024-04-24 09:18:18.347478117 +0000 UTC m=+0.253743832 container died 9d225ee88aa4ed83985c0a2a1dea31c2fcea164d865bdd996e6f8b6033070096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-24T09:18:19.087 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:18 smithi069 podman[96150]: 2024-04-24 09:18:18.810922594 +0000 UTC m=+0.717188319 container cleanup 9d225ee88aa4ed83985c0a2a1dea31c2fcea164d865bdd996e6f8b6033070096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-24T09:18:19.087 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:18 smithi069 bash[96150]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a 2024-04-24T09:18:19.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 24 09:18:19 smithi069 podman[96164]: 2024-04-24 09:18:19.087452874 +0000 UTC m=+0.722615596 container remove 9d225ee88aa4ed83985c0a2a1dea31c2fcea164d865bdd996e6f8b6033070096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:18:19.640 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mon.a.service' 2024-04-24T09:18:19.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:18:19.752 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-24T09:18:19.752 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-24T09:18:19.752 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mon.b 2024-04-24T09:18:20.317 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:19 smithi081 systemd[1]: Stopping Ceph mon.b for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:18:20.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:20 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-b[30957]: 2024-04-24T09:18:20.366+0000 7efd3aa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:18:20.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:20 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-b[30957]: 2024-04-24T09:18:20.366+0000 7efd3aa00700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-24T09:18:20.818 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:20 smithi081 podman[67129]: 2024-04-24 09:18:20.501738919 +0000 UTC m=+0.305671764 container died 500bbac1214c29cf70c55ca09dac6ecba873f4511b29c675563a13624cb57d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:18:21.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:21 smithi081 podman[67129]: 2024-04-24 09:18:21.052419832 +0000 UTC m=+0.856352670 container cleanup 500bbac1214c29cf70c55ca09dac6ecba873f4511b29c675563a13624cb57d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:18:21.067 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:21 smithi081 bash[67129]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-b 2024-04-24T09:18:21.817 INFO:journalctl@ceph.mon.b.smithi081.stdout:Apr 24 09:18:21 smithi081 podman[67143]: 2024-04-24 09:18:21.469136994 +0000 UTC m=+0.950034046 container remove 500bbac1214c29cf70c55ca09dac6ecba873f4511b29c675563a13624cb57d64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:18:21.942 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mon.b.service' 2024-04-24T09:18:22.057 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:18:22.058 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-24T09:18:22.058 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-24T09:18:22.058 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mon.c 2024-04-24T09:18:22.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:22 smithi130 systemd[1]: Stopping Ceph mon.c for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:18:22.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:22 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-c[31536]: 2024-04-24T09:18:22.694+0000 7f5154a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:18:22.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:22 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-c[31536]: 2024-04-24T09:18:22.694+0000 7f5154a00700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-24T09:18:22.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:22 smithi130 podman[68729]: 2024-04-24 09:18:22.778655547 +0000 UTC m=+0.268187132 container died dc34ddfa715916bad147fd274e09d1172a4e655672bc44baae5397ee8d51e69e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-24T09:18:23.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:23 smithi130 podman[68729]: 2024-04-24 09:18:23.246626376 +0000 UTC m=+0.736157970 container cleanup dc34ddfa715916bad147fd274e09d1172a4e655672bc44baae5397ee8d51e69e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-24T09:18:23.454 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:23 smithi130 bash[68729]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-c 2024-04-24T09:18:23.954 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 24 09:18:23 smithi130 podman[68743]: 2024-04-24 09:18:23.503864876 +0000 UTC m=+0.714498709 container remove dc34ddfa715916bad147fd274e09d1172a4e655672bc44baae5397ee8d51e69e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:18:24.018 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mon.c.service' 2024-04-24T09:18:24.141 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:18:24.141 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-24T09:18:24.141 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-24T09:18:24.142 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mgr.x 2024-04-24T09:18:24.591 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:18:24 smithi069 systemd[1]: Stopping Ceph mgr.x for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:18:25.092 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:18:24 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:18:24.697+0000 7fe45b800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:18:25.092 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:18:24 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x[28411]: 2024-04-24T09:18:24.697+0000 7fe45b800700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-24T09:18:25.092 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:18:24 smithi069 podman[96284]: 2024-04-24 09:18:24.800537097 +0000 UTC m=+0.278668280 container died 6e4c09281b32c4f9ae5b210f95e3421b58941d23a09aaf2f14dbea502b1e9a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-24T09:18:25.560 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:18:25 smithi069 podman[96284]: 2024-04-24 09:18:25.325015122 +0000 UTC m=+0.803146333 container cleanup 6e4c09281b32c4f9ae5b210f95e3421b58941d23a09aaf2f14dbea502b1e9a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-24T09:18:25.560 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:18:25 smithi069 bash[96284]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x 2024-04-24T09:18:25.841 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 24 09:18:25 smithi069 podman[96298]: 2024-04-24 09:18:25.56033825 +0000 UTC m=+0.748044057 container remove 6e4c09281b32c4f9ae5b210f95e3421b58941d23a09aaf2f14dbea502b1e9a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-x, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-24T09:18:26.006 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-24T09:18:26.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:18:26.116 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-24T09:18:26.116 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-24T09:18:26.116 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mgr.y 2024-04-24T09:18:26.567 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:18:26 smithi081 systemd[1]: Stopping Ceph mgr.y for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:18:27.029 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:18:26 smithi081 podman[67250]: 2024-04-24 09:18:26.745887345 +0000 UTC m=+0.224717115 container died 86e9cccf0aaa04a29533f7030aee547b1f4202d3e2d05c571da4936c7ab54895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-24T09:18:27.317 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:18:27 smithi081 podman[67250]: 2024-04-24 09:18:27.200289259 +0000 UTC m=+0.679119028 container cleanup 86e9cccf0aaa04a29533f7030aee547b1f4202d3e2d05c571da4936c7ab54895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-24T09:18:27.317 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:18:27 smithi081 bash[67250]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y 2024-04-24T09:18:27.817 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:18:27 smithi081 podman[67264]: 2024-04-24 09:18:27.435070851 +0000 UTC m=+0.678119084 container remove 86e9cccf0aaa04a29533f7030aee547b1f4202d3e2d05c571da4936c7ab54895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-24T09:18:27.817 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Apr 24 09:18:27 smithi081 systemd[1]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-24T09:18:27.979 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-24T09:18:28.083 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:18:28.083 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-24T09:18:28.083 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-24T09:18:28.083 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mgr.z 2024-04-24T09:18:28.454 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:18:28 smithi130 systemd[1]: Stopping Ceph mgr.z for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:18:29.074 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:18:28 smithi130 podman[68864]: 2024-04-24 09:18:28.716597725 +0000 UTC m=+0.242929984 container died 9a364bef97f34c0ee337a09136e005d09555d6b6be117c884cc8ecd9f7e53658 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-24T09:18:29.454 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:18:29 smithi130 podman[68864]: 2024-04-24 09:18:29.236074452 +0000 UTC m=+0.762406730 container cleanup 9a364bef97f34c0ee337a09136e005d09555d6b6be117c884cc8ecd9f7e53658 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:18:29.454 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:18:29 smithi130 bash[68864]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z 2024-04-24T09:18:29.954 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:18:29 smithi130 podman[68878]: 2024-04-24 09:18:29.504431938 +0000 UTC m=+0.772299983 container remove 9a364bef97f34c0ee337a09136e005d09555d6b6be117c884cc8ecd9f7e53658 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-24T09:18:29.954 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 24 09:18:29 smithi130 systemd[1]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-24T09:18:30.091 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-24T09:18:30.196 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:18:30.196 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-24T09:18:30.196 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-24T09:18:30.196 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.0 2024-04-24T09:18:30.591 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 09:18:30 smithi069 systemd[1]: Stopping Ceph osd.0 for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:18:31.092 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 09:18:30 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-0[40254]: 2024-04-24T09:18:30.778+0000 7f3e71400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:18:31.092 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 09:18:30 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-0[40254]: 2024-04-24T09:18:30.778+0000 7f3e71400700 -1 osd.0 103 *** Got signal Terminated *** 2024-04-24T09:18:31.092 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 09:18:30 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-0[40254]: 2024-04-24T09:18:30.778+0000 7f3e71400700 -1 osd.0 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:18:36.571 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 09:18:36 smithi069 podman[96404]: 2024-04-24 09:18:36.24034014 +0000 UTC m=+5.605093093 container died e67b884e8a6d4a197e8af9429798d55342db17c0977094e883c8910efacc0696 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424) 2024-04-24T09:18:36.841 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 09:18:36 smithi069 podman[96404]: 2024-04-24 09:18:36.705020025 +0000 UTC m=+6.069773003 container cleanup e67b884e8a6d4a197e8af9429798d55342db17c0977094e883c8910efacc0696 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:18:36.841 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 09:18:36 smithi069 bash[96404]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-0 2024-04-24T09:18:37.341 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 09:18:36 smithi069 podman[96427]: 2024-04-24 09:18:36.922998039 +0000 UTC m=+0.670708867 container remove e67b884e8a6d4a197e8af9429798d55342db17c0977094e883c8910efacc0696 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-24T09:18:38.078 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 09:18:37 smithi069 podman[96514]: 2024-04-24 09:18:37.635045581 +0000 UTC m=+0.032516203 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:18:38.078 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 09:18:37 smithi069 podman[96514]: 2024-04-24 09:18:37.765766173 +0000 UTC m=+0.163236830 container create 09955c9366be280b96cc927f53bd2d2b43a4614980d1cf964d8a7894e4a7bfe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-24T09:18:38.078 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 09:18:38 smithi069 podman[96514]: 2024-04-24 09:18:38.078170339 +0000 UTC m=+0.475641083 container init 09955c9366be280b96cc927f53bd2d2b43a4614980d1cf964d8a7894e4a7bfe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:18:38.341 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 09:18:38 smithi069 podman[96514]: 2024-04-24 09:18:38.089282877 +0000 UTC m=+0.486753520 container start 09955c9366be280b96cc927f53bd2d2b43a4614980d1cf964d8a7894e4a7bfe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-04-24T09:18:38.342 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 09:18:38 smithi069 podman[96514]: 2024-04-24 09:18:38.210517569 +0000 UTC m=+0.607988232 container attach 09955c9366be280b96cc927f53bd2d2b43a4614980d1cf964d8a7894e4a7bfe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:18:38.841 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 24 09:18:38 smithi069 podman[96514]: 2024-04-24 09:18:38.549809761 +0000 UTC m=+0.947280458 container died 09955c9366be280b96cc927f53bd2d2b43a4614980d1cf964d8a7894e4a7bfe2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:18:39.376 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.0.service' 2024-04-24T09:18:39.493 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:18:39.493 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-24T09:18:39.493 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-24T09:18:39.493 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.1 2024-04-24T09:18:40.067 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 09:18:39 smithi081 systemd[1]: Stopping Ceph osd.1 for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:18:40.568 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 09:18:40 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-1[35536]: 2024-04-24T09:18:40.087+0000 7fe466a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:18:40.568 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 09:18:40 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-1[35536]: 2024-04-24T09:18:40.087+0000 7fe466a00700 -1 osd.1 103 *** Got signal Terminated *** 2024-04-24T09:18:40.568 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 09:18:40 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-1[35536]: 2024-04-24T09:18:40.087+0000 7fe466a00700 -1 osd.1 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:18:45.877 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 09:18:45 smithi081 podman[67388]: 2024-04-24 09:18:45.573838017 +0000 UTC m=+5.632197477 container died 189486e7cdeb76e164d5e6a1b97f08ab0b31c95426c51f2bf716511e2f8df305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T09:18:46.248 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 09:18:46 smithi081 podman[67388]: 2024-04-24 09:18:46.005589335 +0000 UTC m=+6.063948836 container cleanup 189486e7cdeb76e164d5e6a1b97f08ab0b31c95426c51f2bf716511e2f8df305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-04-24T09:18:46.249 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 09:18:46 smithi081 bash[67388]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-1 2024-04-24T09:18:46.557 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 09:18:46 smithi081 podman[67406]: 2024-04-24 09:18:46.248609527 +0000 UTC m=+0.662009128 container remove 189486e7cdeb76e164d5e6a1b97f08ab0b31c95426c51f2bf716511e2f8df305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:18:47.453 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 09:18:47 smithi081 podman[67497]: 2024-04-24 09:18:47.018394214 +0000 UTC m=+0.039594993 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:18:47.453 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 09:18:47 smithi081 podman[67497]: 2024-04-24 09:18:47.167352701 +0000 UTC m=+0.188553490 container create b16d018c1cda03e932272470dc2e43493b274765c6e21efc1afd9653395ec0e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:18:47.817 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 09:18:47 smithi081 podman[67497]: 2024-04-24 09:18:47.453794929 +0000 UTC m=+0.474995739 container init b16d018c1cda03e932272470dc2e43493b274765c6e21efc1afd9653395ec0e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:18:47.817 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 09:18:47 smithi081 podman[67497]: 2024-04-24 09:18:47.465649223 +0000 UTC m=+0.486850001 container start b16d018c1cda03e932272470dc2e43493b274765c6e21efc1afd9653395ec0e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-24T09:18:47.818 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 09:18:47 smithi081 podman[67497]: 2024-04-24 09:18:47.560996711 +0000 UTC m=+0.582197549 container attach b16d018c1cda03e932272470dc2e43493b274765c6e21efc1afd9653395ec0e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-24T09:18:48.272 INFO:journalctl@ceph.osd.1.smithi081.stdout:Apr 24 09:18:47 smithi081 podman[67497]: 2024-04-24 09:18:47.9199966 +0000 UTC m=+0.941197410 container died b16d018c1cda03e932272470dc2e43493b274765c6e21efc1afd9653395ec0e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-24T09:18:48.663 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.1.service' 2024-04-24T09:18:48.775 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:18:48.776 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-24T09:18:48.776 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-24T09:18:48.776 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.2 2024-04-24T09:18:49.204 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 09:18:48 smithi130 systemd[1]: Stopping Ceph osd.2 for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:18:49.704 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 09:18:49 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-2[36838]: 2024-04-24T09:18:49.346+0000 7fa627600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:18:49.704 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 09:18:49 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-2[36838]: 2024-04-24T09:18:49.346+0000 7fa627600700 -1 osd.2 103 *** Got signal Terminated *** 2024-04-24T09:18:49.704 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 09:18:49 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-2[36838]: 2024-04-24T09:18:49.346+0000 7fa627600700 -1 osd.2 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:18:55.204 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 09:18:54 smithi130 podman[68988]: 2024-04-24 09:18:54.925720115 +0000 UTC m=+5.725236522 container died 5ce7313f55d6fdbba9f976b8d0094d898d7f769b017353ff10e0bcd433057d65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-24T09:18:55.660 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 09:18:55 smithi130 podman[68988]: 2024-04-24 09:18:55.406829947 +0000 UTC m=+6.206346402 container cleanup 5ce7313f55d6fdbba9f976b8d0094d898d7f769b017353ff10e0bcd433057d65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-04-24T09:18:55.660 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 09:18:55 smithi130 bash[68988]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-2 2024-04-24T09:18:55.954 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 09:18:55 smithi130 podman[69007]: 2024-04-24 09:18:55.660425474 +0000 UTC m=+0.727346228 container remove 5ce7313f55d6fdbba9f976b8d0094d898d7f769b017353ff10e0bcd433057d65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:18:56.892 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 09:18:56 smithi130 podman[69097]: 2024-04-24 09:18:56.428168299 +0000 UTC m=+0.037441803 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:18:56.893 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 09:18:56 smithi130 podman[69097]: 2024-04-24 09:18:56.584483631 +0000 UTC m=+0.193757162 container create d7e2e8e1575eed2c269b8ddf9342a5d58ed3757e565a8abb12b3bcd4522a31bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-24T09:18:57.204 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 09:18:56 smithi130 podman[69097]: 2024-04-24 09:18:56.892674869 +0000 UTC m=+0.501948407 container init d7e2e8e1575eed2c269b8ddf9342a5d58ed3757e565a8abb12b3bcd4522a31bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0) 2024-04-24T09:18:57.204 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 09:18:56 smithi130 podman[69097]: 2024-04-24 09:18:56.903778678 +0000 UTC m=+0.513052219 container start d7e2e8e1575eed2c269b8ddf9342a5d58ed3757e565a8abb12b3bcd4522a31bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:18:57.205 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 09:18:57 smithi130 podman[69097]: 2024-04-24 09:18:57.021351332 +0000 UTC m=+0.630624916 container attach d7e2e8e1575eed2c269b8ddf9342a5d58ed3757e565a8abb12b3bcd4522a31bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:18:57.704 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 24 09:18:57 smithi130 podman[69097]: 2024-04-24 09:18:57.358263768 +0000 UTC m=+0.967537289 container died d7e2e8e1575eed2c269b8ddf9342a5d58ed3757e565a8abb12b3bcd4522a31bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-24T09:18:58.109 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:18:57 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:18:57.960+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:18:58.151 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.2.service' 2024-04-24T09:18:58.253 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:18:58.253 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-24T09:18:58.253 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-24T09:18:58.253 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.3 2024-04-24T09:18:58.591 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:18:58 smithi069 systemd[1]: Stopping Ceph osd.3 for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:18:59.204 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:18:58 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:18:58.937+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:18:59.204 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:18:59 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:18:59.021+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:18:59.341 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:18:58 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3[46907]: 2024-04-24T09:18:58.868+0000 7f1a4b800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:18:59.341 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:18:58 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3[46907]: 2024-04-24T09:18:58.868+0000 7f1a4b800700 -1 osd.3 103 *** Got signal Terminated *** 2024-04-24T09:18:59.342 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:18:58 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3[46907]: 2024-04-24T09:18:58.868+0000 7f1a4b800700 -1 osd.3 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:19:00.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:18:59 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:18:59.628+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:00.453 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:18:59 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:18:59.968+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:00.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:00 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:00.068+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:00.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:00 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:00.197+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:00.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:00 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:00.642+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:01.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:00 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:00.602+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:01.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:01.109+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:01.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:00 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:00.955+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:01.591 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:19:01 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3[46907]: 2024-04-24T09:19:01.207+0000 7f1a42000700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.091316+0000 front 2024-04-24T09:18:35.090788+0000 (oldest deadline 2024-04-24T09:19:00.990694+0000) 2024-04-24T09:19:01.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:01 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:01.212+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:01.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:01 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:01.186+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:01.927 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:01.665+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:02.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:01 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:01.591+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:02.204 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:01.926+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:02.204 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:02 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:02.127+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:02.591 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:19:02 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3[46907]: 2024-04-24T09:19:02.220+0000 7f1a42000700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.091316+0000 front 2024-04-24T09:18:35.090788+0000 (oldest deadline 2024-04-24T09:19:00.990694+0000) 2024-04-24T09:19:02.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:02 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:02.228+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:02.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:02 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:02.241+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:02.817 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:02 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:02.566+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:02.953 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:02 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:02.633+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:03.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:02 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:02.970+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:03.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:03 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:03.164+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:03.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:03 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:03.269+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:03.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:03 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:03.285+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:03.592 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:19:03 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3[46907]: 2024-04-24T09:19:03.191+0000 7f1a42000700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.091316+0000 front 2024-04-24T09:18:35.090788+0000 (oldest deadline 2024-04-24T09:19:00.990694+0000) 2024-04-24T09:19:03.817 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:03 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:03.560+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:03.945 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:03 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:03.587+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:04.203 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:04 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:04.147+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:04.204 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:03 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:03.944+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:04.591 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:19:04 smithi069 podman[96619]: 2024-04-24 09:19:04.492887974 +0000 UTC m=+5.789346541 container died b87ab8c4e0381852ce3f88c867285d3d28fd56ade5418861fd20701bef67ec89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-24T09:19:04.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:04 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:04.280+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:04.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:04 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:04.254+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:04.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:04 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:04.571+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:05.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:04 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:04.571+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:05.091 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:19:04 smithi069 podman[96619]: 2024-04-24 09:19:04.98274807 +0000 UTC m=+6.279206675 container cleanup b87ab8c4e0381852ce3f88c867285d3d28fd56ade5418861fd20701bef67ec89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-24T09:19:05.091 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:19:04 smithi069 bash[96619]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3 2024-04-24T09:19:05.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:04 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:04.974+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:05.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:05 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:05.155+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:05.591 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:19:05 smithi069 podman[96649]: 2024-04-24 09:19:05.300902095 +0000 UTC m=+0.798339209 container remove b87ab8c4e0381852ce3f88c867285d3d28fd56ade5418861fd20701bef67ec89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:19:05.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:05 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:05.294+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:05.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:05 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:05.251+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:05.817 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:05 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:05.553+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:05.840 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:05 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:05.569+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:06.156 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:05 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:05.937+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:06.173 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:19:05 smithi069 podman[96733]: 2024-04-24 09:19:05.826242865 +0000 UTC m=+0.039070651 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:19:06.173 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:19:05 smithi069 podman[96733]: 2024-04-24 09:19:05.976993365 +0000 UTC m=+0.189821176 container create dafcf75df5708480b7144daa00d00e5a95530593fe95910240f750833f761e90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:19:06.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:06 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:06.155+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:06.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:06 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:06.276+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:06.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:06 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:06.256+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:06.592 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:19:06 smithi069 podman[96733]: 2024-04-24 09:19:06.280870132 +0000 UTC m=+0.493697967 container init dafcf75df5708480b7144daa00d00e5a95530593fe95910240f750833f761e90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:19:06.592 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:19:06 smithi069 podman[96733]: 2024-04-24 09:19:06.292278276 +0000 UTC m=+0.505106117 container start dafcf75df5708480b7144daa00d00e5a95530593fe95910240f750833f761e90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T09:19:06.592 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:19:06 smithi069 podman[96733]: 2024-04-24 09:19:06.388202859 +0000 UTC m=+0.601030720 container attach dafcf75df5708480b7144daa00d00e5a95530593fe95910240f750833f761e90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T09:19:06.817 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:06 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:06.578+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:06.892 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:06 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:06.593+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:07.091 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 24 09:19:06 smithi069 podman[96733]: 2024-04-24 09:19:06.746961733 +0000 UTC m=+0.959789548 container died dafcf75df5708480b7144daa00d00e5a95530593fe95910240f750833f761e90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:19:07.168 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:06 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:06.891+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:07.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:07 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:07.168+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:07.558 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:07 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:07.212+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:07.559 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:07 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:07.318+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:07.610 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.3.service' 2024-04-24T09:19:07.727 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:19:07.727 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-24T09:19:07.727 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-24T09:19:07.727 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.4 2024-04-24T09:19:07.806 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:07 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:07.602+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:07.806 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:07 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:07.617+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:07.806 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:07 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:07.617+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:07.807 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:07 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4[42171]: 2024-04-24T09:19:07.455+0000 7f5b11a00700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.629414+0000 front 2024-04-24T09:18:40.629493+0000 (oldest deadline 2024-04-24T09:19:06.529633+0000) 2024-04-24T09:19:07.939 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:07 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:07.588+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:07.939 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:07 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:07.588+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:08.067 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:07 smithi081 systemd[1]: Stopping Ceph osd.4 for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:19:08.204 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:07 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:07.938+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:08.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:08 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:08.251+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:08.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:08 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:08.251+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:08.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:08 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:08.278+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:08.610 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:08 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:08.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:08.622 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4[42171]: 2024-04-24T09:19:08.324+0000 7f5b16200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:19:08.622 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4[42171]: 2024-04-24T09:19:08.324+0000 7f5b16200700 -1 osd.4 103 *** Got signal Terminated *** 2024-04-24T09:19:08.622 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4[42171]: 2024-04-24T09:19:08.324+0000 7f5b16200700 -1 osd.4 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:19:08.622 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4[42171]: 2024-04-24T09:19:08.466+0000 7f5b11a00700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.629414+0000 front 2024-04-24T09:18:40.629493+0000 (oldest deadline 2024-04-24T09:19:06.529633+0000) 2024-04-24T09:19:08.623 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:08.563+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:08.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:08 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:08.610+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:08.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:08 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:08.610+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:09.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:08.622+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:09.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:08.622+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:09.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:08 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:08.974+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:09.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:09 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:09.177+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:09.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:09 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:09.177+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:09.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:09 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:09.215+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:09.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:09 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:09.215+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:09.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:09 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:09.323+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:09.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:09 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:09.323+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:09.817 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:09 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4[42171]: 2024-04-24T09:19:09.451+0000 7f5b11a00700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.629414+0000 front 2024-04-24T09:18:40.629493+0000 (oldest deadline 2024-04-24T09:19:06.529633+0000) 2024-04-24T09:19:09.817 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:09 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:09.608+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:09.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:09 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:09.608+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:09.818 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:09 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:09.587+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:09.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:09 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:09.597+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:09.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:09 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:09.597+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:10.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:09 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:09.992+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:10.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:09 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:09.992+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.844249+0000 front 2024-04-24T09:18:44.844185+0000 (oldest deadline 2024-04-24T09:19:09.544404+0000) 2024-04-24T09:19:10.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:10 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:10.140+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:10.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:10 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:10.140+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:10.578 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:10 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:10.254+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:10.579 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:10 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:10.254+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:10.579 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:10 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:10.278+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:10.579 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:10 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:10.278+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:10.817 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:10 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4[42171]: 2024-04-24T09:19:10.444+0000 7f5b11a00700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.629414+0000 front 2024-04-24T09:18:40.629493+0000 (oldest deadline 2024-04-24T09:19:06.529633+0000) 2024-04-24T09:19:10.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:10 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:10.572+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:10.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:10 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:10.644+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:10.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:10 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:10.644+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:10.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:10 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:10.563+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:10.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:10 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:10.563+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:11.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:10 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:10.986+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:11.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:10 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:10.986+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.844249+0000 front 2024-04-24T09:18:44.844185+0000 (oldest deadline 2024-04-24T09:19:09.544404+0000) 2024-04-24T09:19:11.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:11 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:11.154+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:11.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:11 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:11.154+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:11.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:11 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:11.234+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:11.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:11 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:11.234+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:11.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:11 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:11.263+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:11.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:11 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:11.263+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:11.817 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:11 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4[42171]: 2024-04-24T09:19:11.441+0000 7f5b11a00700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.629414+0000 front 2024-04-24T09:18:40.629493+0000 (oldest deadline 2024-04-24T09:19:06.529633+0000) 2024-04-24T09:19:11.817 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:11 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:11.686+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:11.817 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:11 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:11.686+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:11.818 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:11 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:11.615+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:11.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:11 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:11.552+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:11.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:11 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:11.552+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:12.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:12 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:12.014+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:12.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:12 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:12.014+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.844249+0000 front 2024-04-24T09:18:44.844185+0000 (oldest deadline 2024-04-24T09:19:09.544404+0000) 2024-04-24T09:19:12.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:12 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:12.157+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:12.455 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:12 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:12.157+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:12.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:12 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:12.194+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:12.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:12 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:12.194+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:12.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:12 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:12.297+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:12.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:12 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:12.297+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:12.817 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:12 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4[42171]: 2024-04-24T09:19:12.431+0000 7f5b11a00700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.629414+0000 front 2024-04-24T09:18:40.629493+0000 (oldest deadline 2024-04-24T09:19:06.529633+0000) 2024-04-24T09:19:12.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:12 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:12.574+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:12.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:12 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:12.671+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:12.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:12 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:12.671+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:12.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:12 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:12.557+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:12.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:12 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:12.557+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:13.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:12 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:12.986+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:13.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:12 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:12.986+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.844249+0000 front 2024-04-24T09:18:44.844185+0000 (oldest deadline 2024-04-24T09:19:09.544404+0000) 2024-04-24T09:19:13.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:13.203+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:13.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:13.203+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:13.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:13 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:13.175+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:13.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:13 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:13.175+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:13.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:13 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:13.297+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:13.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:13 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:13.297+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:13.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:13.507+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:13.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:13.507+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:14.067 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:13 smithi081 podman[67614]: 2024-04-24 09:19:13.853532015 +0000 UTC m=+5.664259294 container died 6a42c299965c0f7640a54ee1c708b028049254d62fc3cdfb1bb6a1062e2239d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-24T09:19:14.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:13 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:13.627+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:14.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:13 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:13.627+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:14.068 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:13 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:13.616+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:14.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:13.975+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:14.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:13.975+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.844249+0000 front 2024-04-24T09:18:44.844185+0000 (oldest deadline 2024-04-24T09:19:09.544404+0000) 2024-04-24T09:19:14.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:14 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:14.190+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:14.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:14 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:14.190+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:14.567 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:14 smithi081 podman[67614]: 2024-04-24 09:19:14.252192807 +0000 UTC m=+6.062920099 container cleanup 6a42c299965c0f7640a54ee1c708b028049254d62fc3cdfb1bb6a1062e2239d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-24T09:19:14.567 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:14 smithi081 bash[67614]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4 2024-04-24T09:19:14.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:14 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:14.141+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:14.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:14 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:14.141+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:14.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:14 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:14.346+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:14.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:14 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:14.346+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:14.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:14 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:14.546+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:14.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:14 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:14.546+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:15.067 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:14 smithi081 podman[67633]: 2024-04-24 09:19:14.579687243 +0000 UTC m=+0.715925394 container remove 6a42c299965c0f7640a54ee1c708b028049254d62fc3cdfb1bb6a1062e2239d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-24T09:19:15.067 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:14 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:14.568+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:15.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:14 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:14.623+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:15.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:14 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:14.623+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:15.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:14 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:14.959+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:15.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:14 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:14.959+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.844249+0000 front 2024-04-24T09:18:44.844185+0000 (oldest deadline 2024-04-24T09:19:09.544404+0000) 2024-04-24T09:19:15.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:15 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:15.177+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:15.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:15 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:15.177+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:15.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:15 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:15.189+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:15.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:15 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:15.189+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:15.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:15 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:15.388+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:15.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:15 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:15.388+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:15.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:15 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:15.564+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:15.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:15 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:15.634+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:15.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:15 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:15.634+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:15.818 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:15 smithi081 podman[67721]: 2024-04-24 09:19:15.39089521 +0000 UTC m=+0.034182432 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:19:15.818 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:15 smithi081 podman[67721]: 2024-04-24 09:19:15.537962038 +0000 UTC m=+0.181249285 container create fb30e35337043f344f8ed794bf1ba67797d3d299c151b7895f6754b8b33743e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:19:15.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:15 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:15.583+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:15.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:15 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:15.583+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:16.295 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:15 smithi081 podman[67721]: 2024-04-24 09:19:15.825165862 +0000 UTC m=+0.468453189 container init fb30e35337043f344f8ed794bf1ba67797d3d299c151b7895f6754b8b33743e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux ) 2024-04-24T09:19:16.295 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:15 smithi081 podman[67721]: 2024-04-24 09:19:15.835663682 +0000 UTC m=+0.478950933 container start fb30e35337043f344f8ed794bf1ba67797d3d299c151b7895f6754b8b33743e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:19:16.295 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:15 smithi081 podman[67721]: 2024-04-24 09:19:15.924190868 +0000 UTC m=+0.567478122 container attach fb30e35337043f344f8ed794bf1ba67797d3d299c151b7895f6754b8b33743e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-24T09:19:16.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:15 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:15.954+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:16.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:15 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:15.954+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.844249+0000 front 2024-04-24T09:18:44.844185+0000 (oldest deadline 2024-04-24T09:19:09.544404+0000) 2024-04-24T09:19:16.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:16 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:16.178+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:16.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:16 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:16.178+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:16.455 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:16 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:16.178+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:16.567 INFO:journalctl@ceph.osd.4.smithi081.stdout:Apr 24 09:19:16 smithi081 podman[67747]: 2024-04-24 09:19:16.357829595 +0000 UTC m=+0.045207392 container died fb30e35337043f344f8ed794bf1ba67797d3d299c151b7895f6754b8b33743e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:19:16.567 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:16 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:16.531+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:16.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:16 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:16.206+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:16.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:16 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:16.206+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:16.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:16 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:16.206+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.845243+0000 front 2024-04-24T09:18:52.844934+0000 (oldest deadline 2024-04-24T09:19:15.744993+0000) 2024-04-24T09:19:16.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:16 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:16.381+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:16.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:16 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:16.381+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:16.917 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:16 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:16.541+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:16.917 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:16 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:16.541+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:16.917 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:16 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:16.916+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:16.917 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:16 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:16.916+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.844249+0000 front 2024-04-24T09:18:44.844185+0000 (oldest deadline 2024-04-24T09:19:09.544404+0000) 2024-04-24T09:19:17.020 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:16 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:16.625+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:17.020 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:16 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:16.625+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:17.067 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.4.service' 2024-04-24T09:19:17.172 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:19:17.172 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-24T09:19:17.172 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-24T09:19:17.172 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.5 2024-04-24T09:19:17.204 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:16 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:16.916+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.947465+0000 front 2024-04-24T09:18:52.947517+0000 (oldest deadline 2024-04-24T09:19:16.447556+0000) 2024-04-24T09:19:17.204 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:17.159+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:17.204 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:17.159+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:17.204 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:17.159+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:17.572 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:17 smithi130 systemd[1]: Stopping Ceph osd.5 for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:19:17.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:17 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:17.374+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:17.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:17 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:17.374+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:17.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:17 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:17.256+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:17.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:17 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:17.256+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:17.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:17 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:17.256+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.845243+0000 front 2024-04-24T09:18:52.844934+0000 (oldest deadline 2024-04-24T09:19:15.744993+0000) 2024-04-24T09:19:17.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:17 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:17.507+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:17.817 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:17 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:17.660+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:17.817 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:17 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:17.660+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:17.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:17.572+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:17.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:17.572+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:17.955 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:17.793+0000 7fd708200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:19:17.955 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:17.793+0000 7fd708200700 -1 osd.5 103 *** Got signal Terminated *** 2024-04-24T09:19:17.955 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:17.793+0000 7fd708200700 -1 osd.5 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:19:18.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:18.163+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:18.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:18.163+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:18.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:18.163+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:18.455 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:17.954+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:18.455 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:17.954+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.844249+0000 front 2024-04-24T09:18:44.844185+0000 (oldest deadline 2024-04-24T09:19:09.544404+0000) 2024-04-24T09:19:18.455 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:17.954+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.947465+0000 front 2024-04-24T09:18:52.947517+0000 (oldest deadline 2024-04-24T09:19:16.447556+0000) 2024-04-24T09:19:18.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:18 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:18.293+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:18.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:18 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:18.293+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:18.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:18 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:18.293+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.845243+0000 front 2024-04-24T09:18:52.844934+0000 (oldest deadline 2024-04-24T09:19:15.744993+0000) 2024-04-24T09:19:18.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:18 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:18.397+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:18.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:18 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:18.397+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:18.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:18 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:18.397+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:18.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:18 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:18.543+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:18.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:18 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:18.543+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:18.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:18 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:18.670+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:18.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:18 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:18.670+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:18.928 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:18.538+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:18.928 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:18.538+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:19.204 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:18.927+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:19.204 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:18.927+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.844249+0000 front 2024-04-24T09:18:44.844185+0000 (oldest deadline 2024-04-24T09:19:09.544404+0000) 2024-04-24T09:19:19.204 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:18.927+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.947465+0000 front 2024-04-24T09:18:52.947517+0000 (oldest deadline 2024-04-24T09:19:16.447556+0000) 2024-04-24T09:19:19.204 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:19 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:19.136+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:19.205 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:19 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:19.136+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:19.205 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:19 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:19.136+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:19.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:19 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:19.327+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:19.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:19 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:19.327+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:19.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:19 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:19.327+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.845243+0000 front 2024-04-24T09:18:52.844934+0000 (oldest deadline 2024-04-24T09:19:15.744993+0000) 2024-04-24T09:19:19.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:19 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:19.377+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:19.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:19 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:19.377+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:19.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:19 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:19.377+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:19.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:19 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:19.562+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:19.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:19 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:19.562+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:19.817 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:19 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:19.652+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:19.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:19 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:19.652+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:19.942 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:19 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:19.552+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:19.942 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:19 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:19.552+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:20.204 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:19 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:19.941+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:20.204 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:19 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:19.941+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.844249+0000 front 2024-04-24T09:18:44.844185+0000 (oldest deadline 2024-04-24T09:19:09.544404+0000) 2024-04-24T09:19:20.204 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:19 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:19.941+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.947465+0000 front 2024-04-24T09:18:52.947517+0000 (oldest deadline 2024-04-24T09:19:16.447556+0000) 2024-04-24T09:19:20.204 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:20 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:20.101+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:20.205 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:20 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:20.101+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:20.205 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:20 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:20.101+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:20.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:20 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:20.417+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:20.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:20 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:20.417+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:20.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:20 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:20.417+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:20.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:20 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:20.278+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:20.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:20 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:20.278+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:20.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:20 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:20.278+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.845243+0000 front 2024-04-24T09:18:52.844934+0000 (oldest deadline 2024-04-24T09:19:15.744993+0000) 2024-04-24T09:19:20.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:20 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:20.544+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:20.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:20 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:20.544+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:20.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:20 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:20.686+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:20.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:20 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:20.686+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:20.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:20 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:20.686+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:20.945 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:20 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:20.524+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:20.945 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:20 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:20.524+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:20.946 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:20 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:20.524+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:21.204 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:21 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:21.135+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:21.204 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:21 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:21.135+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:21.204 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:21 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:21.135+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:21.204 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:20 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:20.945+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:21.204 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:20 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:20.945+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.844249+0000 front 2024-04-24T09:18:44.844185+0000 (oldest deadline 2024-04-24T09:19:09.544404+0000) 2024-04-24T09:19:21.205 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:20 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:20.945+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.947465+0000 front 2024-04-24T09:18:52.947517+0000 (oldest deadline 2024-04-24T09:19:16.447556+0000) 2024-04-24T09:19:21.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:21 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:21.269+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:21.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:21 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:21.269+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:21.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:21 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:21.269+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.845243+0000 front 2024-04-24T09:18:52.844934+0000 (oldest deadline 2024-04-24T09:19:15.744993+0000) 2024-04-24T09:19:21.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:21 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:21.411+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:21.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:21 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:21.411+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:21.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:21 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:21.411+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:21.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:21 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:21.509+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:21.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:21 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:21.509+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:21.817 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:21 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:21.693+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:21.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:21 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:21.693+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:21.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:21 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:21.693+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:21.952 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:21 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:21.559+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:21.952 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:21 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:21.559+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:21.952 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:21 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:21.559+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:22.204 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:21 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:21.951+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.640865+0000 front 2024-04-24T09:18:35.641063+0000 (oldest deadline 2024-04-24T09:18:57.940133+0000) 2024-04-24T09:19:22.204 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:21 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:21.951+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.844249+0000 front 2024-04-24T09:18:44.844185+0000 (oldest deadline 2024-04-24T09:19:09.544404+0000) 2024-04-24T09:19:22.204 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:21 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5[43572]: 2024-04-24T09:19:21.951+0000 7fd6fea00700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.947465+0000 front 2024-04-24T09:18:52.947517+0000 (oldest deadline 2024-04-24T09:19:16.447556+0000) 2024-04-24T09:19:22.204 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:22 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:22.176+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:22.205 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:22 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:22.176+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:22.205 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:22 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:22.176+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:22.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:22 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:22.295+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:22.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:22 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:22.295+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:22.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:22 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:22.295+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.845243+0000 front 2024-04-24T09:18:52.844934+0000 (oldest deadline 2024-04-24T09:19:15.744993+0000) 2024-04-24T09:19:22.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:22 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:22.407+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:22.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:22 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:22.407+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:22.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:22 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:22.407+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:22.736 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:22 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:22.476+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:22.736 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:22 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:22.476+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:22.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:22 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:22.540+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:22.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:22 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:22.540+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:22.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:22 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:22.540+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:23.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:22 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:22.736+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:23.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:22 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:22.736+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:23.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:22 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:22.736+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:23.454 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:23 smithi130 podman[69214]: 2024-04-24 09:19:23.206104358 +0000 UTC m=+5.602763971 container died 52b36dcfbb8680cc6c8f180f1f2397263d93a724367ccf571f4eadc5efe5f638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:19:23.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:23 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:23.166+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:23.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:23 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:23.166+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:23.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:23 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:23.166+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:23.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:23 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:23.254+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:23.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:23 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:23.254+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:23.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:23 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:23.254+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.845243+0000 front 2024-04-24T09:18:52.844934+0000 (oldest deadline 2024-04-24T09:19:15.744993+0000) 2024-04-24T09:19:23.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:23 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:23.437+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:23.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:23 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:23.437+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:23.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:23 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:23.437+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:23.693 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:23 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:23.443+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:23.693 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:23 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:23.443+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:23.954 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:23 smithi130 podman[69214]: 2024-04-24 09:19:23.703834729 +0000 UTC m=+6.100494363 container cleanup 52b36dcfbb8680cc6c8f180f1f2397263d93a724367ccf571f4eadc5efe5f638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T09:19:23.954 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:23 smithi130 bash[69214]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5 2024-04-24T09:19:23.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:23 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:23.552+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:23.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:23 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:23.552+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:23.955 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:23 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:23.552+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:24.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:23 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:23.693+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:24.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:23 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:23.693+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:24.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:23 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:23.693+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:24.453 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:24 smithi130 podman[69230]: 2024-04-24 09:19:24.005491327 +0000 UTC m=+0.785715271 container remove 52b36dcfbb8680cc6c8f180f1f2397263d93a724367ccf571f4eadc5efe5f638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-24T09:19:24.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:24 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:24.188+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:24.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:24 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:24.188+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:24.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:24 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:24.188+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:24.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:24 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:24.214+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:24.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:24 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:24.214+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:24.592 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:24 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:24.214+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.845243+0000 front 2024-04-24T09:18:52.844934+0000 (oldest deadline 2024-04-24T09:19:15.744993+0000) 2024-04-24T09:19:24.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:24 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:24.415+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:24.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:24 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:24.415+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:24.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:24 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:24.415+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:24.712 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:24 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:24.412+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:24.712 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:24 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:24.412+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:24.783 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:24 smithi130 podman[69320]: 2024-04-24 09:19:24.478693202 +0000 UTC m=+0.033493720 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:19:24.783 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:24 smithi130 podman[69320]: 2024-04-24 09:19:24.598394539 +0000 UTC m=+0.153195061 container create 4168713d5136af0b5a80d89d4fda39c97e3590542372f7afe5385a01c5023f2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-24T09:19:24.783 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:24 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:24.522+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:24.783 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:24 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:24.522+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:24.783 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:24 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:24.522+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:25.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:24 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:24.712+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:25.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:24 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:24.712+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:25.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:24 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:24.712+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:25.422 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:25 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:25.159+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:25.422 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:25 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:25.159+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:25.422 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:25 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:25.159+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:25.422 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:24 smithi130 podman[69320]: 2024-04-24 09:19:24.954635114 +0000 UTC m=+0.509435669 container init 4168713d5136af0b5a80d89d4fda39c97e3590542372f7afe5385a01c5023f2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T09:19:25.423 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:24 smithi130 podman[69320]: 2024-04-24 09:19:24.965630314 +0000 UTC m=+0.520430832 container start 4168713d5136af0b5a80d89d4fda39c97e3590542372f7afe5385a01c5023f2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-24T09:19:25.423 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:25 smithi130 podman[69320]: 2024-04-24 09:19:25.067989782 +0000 UTC m=+0.622790337 container attach 4168713d5136af0b5a80d89d4fda39c97e3590542372f7afe5385a01c5023f2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:19:25.423 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 24 09:19:25 smithi130 podman[69320]: 2024-04-24 09:19:25.422154866 +0000 UTC m=+0.976955380 container died 4168713d5136af0b5a80d89d4fda39c97e3590542372f7afe5385a01c5023f2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:19:25.465 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:25 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:25.190+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:25.465 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:25 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:25.190+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:25.466 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:25 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:25.190+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.845243+0000 front 2024-04-24T09:18:52.844934+0000 (oldest deadline 2024-04-24T09:19:15.744993+0000) 2024-04-24T09:19:25.703 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:25 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:25.516+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:25.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:25 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:25.516+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:25.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:25 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:25.516+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:25.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:25 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:25.454+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:25.747 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:25 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:25.454+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:25.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:25 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:25.465+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:25.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:25 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:25.465+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:25.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:25 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:25.465+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:26.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:25 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:25.747+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:26.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:25 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:25.747+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:26.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:25 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:25.747+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:26.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:25 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:25.747+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:26.274 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.5.service' 2024-04-24T09:19:26.421 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:26 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:26.162+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:26.421 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:26 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:26.162+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:26.421 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:26 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:26.162+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.845243+0000 front 2024-04-24T09:18:52.844934+0000 (oldest deadline 2024-04-24T09:19:15.744993+0000) 2024-04-24T09:19:26.422 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:26 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:26.162+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:18:59.748816+0000 front 2024-04-24T09:18:59.748880+0000 (oldest deadline 2024-04-24T09:19:25.649072+0000) 2024-04-24T09:19:26.430 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:19:26.430 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-24T09:19:26.431 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-24T09:19:26.431 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.6 2024-04-24T09:19:26.475 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:26 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:26.207+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:26.476 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:26 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:26.207+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:26.476 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:26 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:26.207+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:26.773 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:26 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:26.493+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:26.773 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:26 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:26.493+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:26.841 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:26 smithi069 systemd[1]: Stopping Ceph osd.6 for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:19:26.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:26 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:26.420+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:26.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:26 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:26.420+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:26.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:26 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:26.420+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:26.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:26 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:26.420+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:26.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:26 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:26.475+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:26.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:26 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:26.475+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:26.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:26 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:26.475+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:27.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:26 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:26.773+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:27.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:26 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:26.773+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:27.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:26 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:26.773+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:27.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:26 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:26.773+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:27.342 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:26 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:26.974+0000 7f8954800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:19:27.342 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:26 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:26.974+0000 7f8954800700 -1 osd.6 103 *** Got signal Terminated *** 2024-04-24T09:19:27.342 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:26 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:26.974+0000 7f8954800700 -1 osd.6 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:19:27.342 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:27 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:27.157+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:27.342 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:27 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:27.157+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:27.343 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:27 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:27.157+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.845243+0000 front 2024-04-24T09:18:52.844934+0000 (oldest deadline 2024-04-24T09:19:15.744993+0000) 2024-04-24T09:19:27.343 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:27 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:27.157+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:18:59.748816+0000 front 2024-04-24T09:18:59.748880+0000 (oldest deadline 2024-04-24T09:19:25.649072+0000) 2024-04-24T09:19:27.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:27 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:27.257+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:27.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:27 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:27.257+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:27.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:27 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:27.257+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:27.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:27 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:27.464+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:27.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:27 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:27.464+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:27.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:27 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:27.464+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:27.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:27 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:27.464+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:27.810 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:27 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:27.525+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:27.810 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:27 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:27.525+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:27.811 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:27 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:27.525+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.828743+0000 front 2024-04-24T09:19:01.828808+0000 (oldest deadline 2024-04-24T09:19:27.128907+0000) 2024-04-24T09:19:27.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:27 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:27.374+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:27.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:27 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:27.374+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:27.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:27 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:27.374+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:27.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:27 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:27.374+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:28.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:27 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:27.809+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:28.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:27 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:27.809+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:28.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:27 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:27.809+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:28.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:27 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:27.809+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:28.387 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:28 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:28.134+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:28.387 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:28 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:28.134+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:28.387 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:28 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:28.134+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.845243+0000 front 2024-04-24T09:18:52.844934+0000 (oldest deadline 2024-04-24T09:19:15.744993+0000) 2024-04-24T09:19:28.387 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:28 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:28.134+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:18:59.748816+0000 front 2024-04-24T09:18:59.748880+0000 (oldest deadline 2024-04-24T09:19:25.649072+0000) 2024-04-24T09:19:28.417 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:28 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:28.297+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:28.417 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:28 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:28.297+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:28.417 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:28 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:28.297+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:28.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:28 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:28.417+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:28.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:28 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:28.417+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:28.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:28 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:28.417+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:28.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:28 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:28.417+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:28.784 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:28.501+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:28.784 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:28.501+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:28.784 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:28.501+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.828743+0000 front 2024-04-24T09:19:01.828808+0000 (oldest deadline 2024-04-24T09:19:27.128907+0000) 2024-04-24T09:19:28.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:28 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:28.386+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:28.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:28 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:28.386+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:28.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:28 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:28.386+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:28.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:28 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:28.386+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:29.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:28.783+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:29.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:28.783+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:29.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:28.783+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:29.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:28 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:28.783+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:29.422 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:29 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:29.159+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:29.422 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:29 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:29.159+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:29.422 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:29 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:29.159+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.845243+0000 front 2024-04-24T09:18:52.844934+0000 (oldest deadline 2024-04-24T09:19:15.744993+0000) 2024-04-24T09:19:29.422 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:29 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:29.159+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:18:59.748816+0000 front 2024-04-24T09:18:59.748880+0000 (oldest deadline 2024-04-24T09:19:25.649072+0000) 2024-04-24T09:19:29.423 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:29 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:29.362+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:29.423 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:29 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:29.362+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:29.423 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:29 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:29.362+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:29.423 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:29 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:29.362+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:29.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:29 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:29.292+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:29.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:29 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:29.292+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:29.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:29 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:29.292+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:29.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:29 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:29.459+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:29.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:29 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:29.459+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:29.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:29 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:29.459+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:29.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:29 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:29.459+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:29.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:29.527+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:29.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:29.527+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:29.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:29.527+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.828743+0000 front 2024-04-24T09:19:01.828808+0000 (oldest deadline 2024-04-24T09:19:27.128907+0000) 2024-04-24T09:19:29.817 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:29.761+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:29.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:29.761+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:29.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:29.761+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:29.818 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:29 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:29.761+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:30.342 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:30 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:30.144+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:30.342 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:30 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:30.144+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:30.342 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:30 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:30.144+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.845243+0000 front 2024-04-24T09:18:52.844934+0000 (oldest deadline 2024-04-24T09:19:15.744993+0000) 2024-04-24T09:19:30.342 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:30 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:30.144+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:18:59.748816+0000 front 2024-04-24T09:18:59.748880+0000 (oldest deadline 2024-04-24T09:19:25.649072+0000) 2024-04-24T09:19:30.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:30 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:30.340+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:30.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:30 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:30.340+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:30.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:30 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:30.340+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:30.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:30 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:30.485+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:30.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:30 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:30.485+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:30.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:30 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:30.485+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:30.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:30 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:30.485+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:30.806 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:30 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:30.514+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:30.806 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:30 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:30.514+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:30.806 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:30 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:30.514+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.828743+0000 front 2024-04-24T09:19:01.828808+0000 (oldest deadline 2024-04-24T09:19:27.128907+0000) 2024-04-24T09:19:30.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:30 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:30.350+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:30.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:30 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:30.350+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:30.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:30 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:30.350+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:30.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:30 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:30.350+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:31.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:30 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:30.805+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:31.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:30 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:30.805+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:31.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:30 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:30.805+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:31.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:30 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:30.805+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:31.342 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:31 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:31.103+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:35.436973+0000 front 2024-04-24T09:18:35.437983+0000 (oldest deadline 2024-04-24T09:19:00.638777+0000) 2024-04-24T09:19:31.342 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:31 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:31.103+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:40.640205+0000 front 2024-04-24T09:18:40.640029+0000 (oldest deadline 2024-04-24T09:19:07.641118+0000) 2024-04-24T09:19:31.342 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:31 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:31.103+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.845243+0000 front 2024-04-24T09:18:52.844934+0000 (oldest deadline 2024-04-24T09:19:15.744993+0000) 2024-04-24T09:19:31.342 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:31 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6[53978]: 2024-04-24T09:19:31.103+0000 7f8950000700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:18:59.748816+0000 front 2024-04-24T09:18:59.748880+0000 (oldest deadline 2024-04-24T09:19:25.649072+0000) 2024-04-24T09:19:31.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:31 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:31.363+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:31.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:31 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:31.363+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:31.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:31 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:31.363+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:31.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:31 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:31.485+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:31.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:31 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:31.485+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:31.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:31 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:31.485+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:31.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:31 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:31.485+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:31.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:31 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:31.484+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:31.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:31 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:31.484+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:31.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:31 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:31.484+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.828743+0000 front 2024-04-24T09:19:01.828808+0000 (oldest deadline 2024-04-24T09:19:27.128907+0000) 2024-04-24T09:19:31.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:31 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:31.359+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:31.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:31 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:31.359+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:31.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:31 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:31.359+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:31.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:31 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:31.359+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:32.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:31 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:31.846+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:32.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:31 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:31.846+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:32.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:31 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:31.846+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:32.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:31 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:31.846+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:32.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:32 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:32.366+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:32.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:32 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:32.366+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:32.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:32 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:32.366+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:32.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:32 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:32.512+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:32.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:32 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:32.512+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:32.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:32 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:32.512+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:32.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:32 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:32.512+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:32.742 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:32 smithi069 podman[96840]: 2024-04-24 09:19:32.406289002 +0000 UTC m=+5.569105139 container died 3eff8f82b658c7816e86f165e32d0e7deb019039c11d73680184433ae550aea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-24T09:19:32.743 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:32 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:32.396+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:32.743 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:32 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:32.396+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:32.743 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:32 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:32.396+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:32.743 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:32 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:32.396+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:32.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:32 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:32.437+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:32.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:32 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:32.437+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:32.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:32 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:32.437+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.828743+0000 front 2024-04-24T09:19:01.828808+0000 (oldest deadline 2024-04-24T09:19:27.128907+0000) 2024-04-24T09:19:33.060 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:32 smithi069 podman[96840]: 2024-04-24 09:19:32.843885913 +0000 UTC m=+6.006702109 container cleanup 3eff8f82b658c7816e86f165e32d0e7deb019039c11d73680184433ae550aea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-24T09:19:33.061 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:32 smithi069 bash[96840]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6 2024-04-24T09:19:33.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:32 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:32.848+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:33.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:32 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:32.848+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:33.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:32 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:32.848+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:33.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:32 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:32.848+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:33.341 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:33 smithi069 podman[96872]: 2024-04-24 09:19:33.061379936 +0000 UTC m=+0.641476672 container remove 3eff8f82b658c7816e86f165e32d0e7deb019039c11d73680184433ae550aea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T09:19:33.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:33 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:33.367+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:33.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:33 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:33.367+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:33.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:33 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:33.367+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:33.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:33 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:33.530+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:33.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:33 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:33.530+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:33.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:33 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:33.530+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:33.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:33 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:33.530+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:33.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:33 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:33.458+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:33.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:33 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:33.458+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:33.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:33 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:33.458+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.828743+0000 front 2024-04-24T09:19:01.828808+0000 (oldest deadline 2024-04-24T09:19:27.128907+0000) 2024-04-24T09:19:33.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:33 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:33.346+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:33.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:33 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:33.346+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:33.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:33 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:33.346+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:33.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:33 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:33.346+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:34.282 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:33 smithi069 podman[96957]: 2024-04-24 09:19:33.823718971 +0000 UTC m=+0.038049603 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:19:34.282 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:33 smithi069 podman[96957]: 2024-04-24 09:19:33.94585423 +0000 UTC m=+0.160184900 container create 3c173211a81014d7843a80c112b5ba7ddb80547114fc4e38f4ac45c2f2da50a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-24T09:19:34.282 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:34 smithi069 podman[96957]: 2024-04-24 09:19:34.282222712 +0000 UTC m=+0.496553385 container init 3c173211a81014d7843a80c112b5ba7ddb80547114fc4e38f4ac45c2f2da50a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:19:34.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:33 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:33.893+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:34.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:33 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:33.893+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:34.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:33 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:33.893+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:34.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:33 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:33.893+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:34.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:34 smithi069 podman[96957]: 2024-04-24 09:19:34.294532819 +0000 UTC m=+0.508863493 container start 3c173211a81014d7843a80c112b5ba7ddb80547114fc4e38f4ac45c2f2da50a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-24T09:19:34.591 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:34 smithi069 podman[96957]: 2024-04-24 09:19:34.398398575 +0000 UTC m=+0.612729249 container attach 3c173211a81014d7843a80c112b5ba7ddb80547114fc4e38f4ac45c2f2da50a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True) 2024-04-24T09:19:34.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:34 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:34.359+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:34.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:34 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:34.359+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:34.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:34 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:34.359+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:34.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:34 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:34.359+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:34.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:34 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:34.375+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:34.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:34 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:34.375+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:34.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:34 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:34.375+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:34.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:34 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:34.494+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:34.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:34 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:34.494+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:34.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:34 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:34.494+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:34.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:34 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:34.494+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:34.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:34 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:34.461+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:34.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:34 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:34.461+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:34.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:34 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:34.461+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.828743+0000 front 2024-04-24T09:19:01.828808+0000 (oldest deadline 2024-04-24T09:19:27.128907+0000) 2024-04-24T09:19:35.091 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 24 09:19:34 smithi069 podman[96957]: 2024-04-24 09:19:34.743617326 +0000 UTC m=+0.957947989 container died 3c173211a81014d7843a80c112b5ba7ddb80547114fc4e38f4ac45c2f2da50a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-24T09:19:35.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:34 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:34.885+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:35.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:34 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:34.885+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:35.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:34 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:34.885+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:35.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:34 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:34.885+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:35.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:34 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:34.885+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:35.356 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:35 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:35.355+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:35.356 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:35 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:35.355+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:35.356 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:35 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:35.355+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:35.356 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:35 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:35.355+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:35.525 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.6.service' 2024-04-24T09:19:35.606 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:35 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:35.542+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:35.606 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:35 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:35.542+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:35.606 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:35 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:35.542+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:35.606 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:35 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:35.542+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:35.607 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:35 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:35.330+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:35.607 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:35 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:35.330+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:35.607 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:35 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:35.331+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:35.634 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:19:35.634 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-24T09:19:35.635 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-24T09:19:35.635 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.7 2024-04-24T09:19:35.688 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:35 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:35.428+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:35.688 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:35 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:35.428+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:35.689 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:35 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:35.428+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.828743+0000 front 2024-04-24T09:19:01.828808+0000 (oldest deadline 2024-04-24T09:19:27.128907+0000) 2024-04-24T09:19:36.006 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:35 smithi081 systemd[1]: Stopping Ceph osd.7 for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:19:36.006 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:35 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:35.861+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:36.006 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:35 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:35.861+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:36.006 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:35 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:35.861+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:36.006 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:35 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:35.861+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:36.007 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:35 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:35.861+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:36.317 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:36 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:36.145+0000 7f544fa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:19:36.317 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:36 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:36.145+0000 7f544fa00700 -1 osd.7 103 *** Got signal Terminated *** 2024-04-24T09:19:36.318 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:36 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:36.145+0000 7f544fa00700 -1 osd.7 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:19:36.655 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:36 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:36.362+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:36.655 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:36 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:36.362+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:36.656 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:36 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:36.362+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:36.656 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:36 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:36.362+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:36.656 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:36 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:36.362+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:36.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:36 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:36.308+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:36.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:36 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:36.308+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:36.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:36 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:36.308+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:36.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:36 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:36.520+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:36.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:36 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:36.520+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:36.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:36 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:36.520+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:36.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:36 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:36.520+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:36.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:36 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:36.520+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:11.583298+0000 front 2024-04-24T09:19:11.583263+0000 (oldest deadline 2024-04-24T09:19:36.283362+0000) 2024-04-24T09:19:36.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:36 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:36.388+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:36.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:36 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:36.388+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:36.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:36 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:36.388+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.828743+0000 front 2024-04-24T09:19:01.828808+0000 (oldest deadline 2024-04-24T09:19:27.128907+0000) 2024-04-24T09:19:37.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:36 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:36.842+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:37.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:36 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:36.842+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:37.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:36 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:36.842+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:37.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:36 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:36.842+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:37.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:36 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:36.842+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:37.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:37 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:37.323+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:37.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:37 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:37.323+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:37.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:37 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:37.323+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:37.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:37 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:37.323+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:37.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:37 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:37.323+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:37.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:37 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:37.316+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:37.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:37 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:37.316+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:37.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:37 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:37.316+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:37.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:37 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:37.521+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:37.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:37 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:37.521+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:37.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:37 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:37.521+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:37.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:37 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:37.521+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:37.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:37 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:37.521+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:11.583298+0000 front 2024-04-24T09:19:11.583263+0000 (oldest deadline 2024-04-24T09:19:36.283362+0000) 2024-04-24T09:19:37.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:37 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:37.360+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:37.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:37 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:37.360+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:37.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:37 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:37.360+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.828743+0000 front 2024-04-24T09:19:01.828808+0000 (oldest deadline 2024-04-24T09:19:27.128907+0000) 2024-04-24T09:19:38.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:37 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:37.847+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:38.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:37 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:37.847+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:38.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:37 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:37.847+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:38.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:37 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:37.847+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:38.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:37 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:37.847+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:38.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:38 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:38.335+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:38.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:38 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:38.335+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:38.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:38 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:38.335+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:38.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:38 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:38.335+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:38.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:38 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:38.335+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:38.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:38 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:38.299+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:38.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:38 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:38.299+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:38.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:38 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:38.299+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:38.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:38 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:38.299+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:38.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:38 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:38.547+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:38.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:38 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:38.547+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:38.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:38 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:38.547+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:38.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:38 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:38.547+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:38.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:38 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:38.547+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:11.583298+0000 front 2024-04-24T09:19:11.583263+0000 (oldest deadline 2024-04-24T09:19:36.283362+0000) 2024-04-24T09:19:38.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:38 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:38.318+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:38.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:38 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:38.318+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:38.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:38 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:38.318+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.828743+0000 front 2024-04-24T09:19:01.828808+0000 (oldest deadline 2024-04-24T09:19:27.128907+0000) 2024-04-24T09:19:38.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:38 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:38.318+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.230530+0000 front 2024-04-24T09:19:08.230265+0000 (oldest deadline 2024-04-24T09:19:38.231407+0000) 2024-04-24T09:19:39.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:38 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:38.886+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:39.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:38 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:38.886+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:39.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:38 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:38.886+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:39.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:38 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:38.886+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:39.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:38 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:38.886+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:39.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:39 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:39.267+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:39.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:39 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:39.267+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:39.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:39 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:39.267+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:39.455 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:39 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:39.267+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:39.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:39 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:39.306+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:39.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:39 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:39.306+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:39.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:39 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:39.306+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:39.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:39 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:39.306+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:39.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:39 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:39.306+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:39.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:39 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:39.359+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:39.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:39 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:39.359+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:39.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:39 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:39.359+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.828743+0000 front 2024-04-24T09:19:01.828808+0000 (oldest deadline 2024-04-24T09:19:27.128907+0000) 2024-04-24T09:19:39.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:39 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:39.359+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.230530+0000 front 2024-04-24T09:19:08.230265+0000 (oldest deadline 2024-04-24T09:19:38.231407+0000) 2024-04-24T09:19:39.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:39 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:39.532+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:39.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:39 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:39.532+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:39.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:39 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:39.532+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:39.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:39 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:39.532+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:39.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:39 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:39.532+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:11.583298+0000 front 2024-04-24T09:19:11.583263+0000 (oldest deadline 2024-04-24T09:19:36.283362+0000) 2024-04-24T09:19:40.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:39 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:39.842+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:40.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:39 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:39.842+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:40.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:39 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:39.842+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:40.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:39 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:39.842+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:40.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:39 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:39.842+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:40.558 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:40 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:40.224+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:40.559 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:40 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:40.224+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:40.559 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:40 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:40.224+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:40.559 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:40 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:40.224+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:40.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:40 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:40.284+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:40.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:40 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:40.284+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:40.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:40 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:40.284+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:40.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:40 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:40.284+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:40.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:40 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:40.284+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:40.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:40 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:40.384+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.822124+0000 front 2024-04-24T09:18:43.822000+0000 (oldest deadline 2024-04-24T09:19:06.722293+0000) 2024-04-24T09:19:40.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:40 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:40.384+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:52.526651+0000 front 2024-04-24T09:18:52.526093+0000 (oldest deadline 2024-04-24T09:19:17.826013+0000) 2024-04-24T09:19:40.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:40 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:40.384+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.828743+0000 front 2024-04-24T09:19:01.828808+0000 (oldest deadline 2024-04-24T09:19:27.128907+0000) 2024-04-24T09:19:40.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:40 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7[48792]: 2024-04-24T09:19:40.384+0000 7f5446200700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.230530+0000 front 2024-04-24T09:19:08.230265+0000 (oldest deadline 2024-04-24T09:19:38.231407+0000) 2024-04-24T09:19:40.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:40 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:40.558+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:40.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:40 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:40.558+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:40.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:40 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:40.558+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:40.955 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:40 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:40.558+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:40.955 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:40 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:40.558+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:11.583298+0000 front 2024-04-24T09:19:11.583263+0000 (oldest deadline 2024-04-24T09:19:36.283362+0000) 2024-04-24T09:19:41.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:40 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:40.838+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:41.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:40 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:40.838+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:41.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:40 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:40.838+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:41.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:40 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:40.838+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:41.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:40 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:40.838+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:41.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:41 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:41.187+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:41.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:41 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:41.187+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:41.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:41 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:41.187+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:41.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:41 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:41.187+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:41.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:41 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:41.319+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:41.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:41 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:41.319+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:41.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:41 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:41.319+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:41.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:41 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:41.319+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:41.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:41 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:41.319+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:41.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:41 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:41.557+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:41.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:41 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:41.557+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:41.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:41 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:41.557+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:41.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:41 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:41.557+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:41.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:41 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:41.557+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:11.583298+0000 front 2024-04-24T09:19:11.583263+0000 (oldest deadline 2024-04-24T09:19:36.283362+0000) 2024-04-24T09:19:41.995 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:41 smithi081 podman[67834]: 2024-04-24 09:19:41.671889253 +0000 UTC m=+5.668422259 container died 8397aef235ea3ed234e68b7e9ef26d5e093a9543fc863b14bdcd34fda9462db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-24T09:19:41.995 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:41 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:41.884+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:41.995 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:41 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:41.884+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:41.995 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:41 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:41.884+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:41.995 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:41 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:41.884+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:41.996 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:41 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:41.884+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:42.317 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:42 smithi081 podman[67834]: 2024-04-24 09:19:42.123776257 +0000 UTC m=+6.120309269 container cleanup 8397aef235ea3ed234e68b7e9ef26d5e093a9543fc863b14bdcd34fda9462db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-24T09:19:42.317 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:42 smithi081 bash[67834]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7 2024-04-24T09:19:42.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:42 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:42.167+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:42.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:42 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:42.167+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:42.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:42 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:42.167+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:42.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:42 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:42.167+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:42.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:42 smithi081 podman[67857]: 2024-04-24 09:19:42.400037547 +0000 UTC m=+0.708906008 container remove 8397aef235ea3ed234e68b7e9ef26d5e093a9543fc863b14bdcd34fda9462db2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T09:19:42.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:42 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:42.347+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:42.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:42 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:42.347+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:42.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:42 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:42.347+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:42.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:42 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:42.347+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:42.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:42 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:42.347+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:42.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:42 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:42.539+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:42.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:42 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:42.539+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:42.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:42 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:42.539+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:42.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:42 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:42.539+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:42.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:42 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:42.539+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:11.583298+0000 front 2024-04-24T09:19:11.583263+0000 (oldest deadline 2024-04-24T09:19:36.283362+0000) 2024-04-24T09:19:43.211 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:42 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:42.866+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:43.211 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:42 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:42.866+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:43.211 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:42 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:42.866+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:43.211 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:42 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:42.866+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:43.211 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:42 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:42.866+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:43.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:43 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:43.187+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:43.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:43 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:43.187+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:43.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:43 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:43.187+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:43.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:43 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:43.187+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:43.539 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:43 smithi081 podman[67943]: 2024-04-24 09:19:43.117941803 +0000 UTC m=+0.036254510 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:19:43.539 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:43 smithi081 podman[67943]: 2024-04-24 09:19:43.24281951 +0000 UTC m=+0.161132194 container create 6c6fcc14280b0e1c1a22078df7d81db0972e11dc4ba66bec501988d1968ade82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS) 2024-04-24T09:19:43.539 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:43 smithi081 podman[67943]: 2024-04-24 09:19:43.538980206 +0000 UTC m=+0.457292970 container init 6c6fcc14280b0e1c1a22078df7d81db0972e11dc4ba66bec501988d1968ade82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:19:43.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:43 smithi081 podman[67943]: 2024-04-24 09:19:43.550377373 +0000 UTC m=+0.468690066 container start 6c6fcc14280b0e1c1a22078df7d81db0972e11dc4ba66bec501988d1968ade82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:19:43.817 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:43 smithi081 podman[67943]: 2024-04-24 09:19:43.639216217 +0000 UTC m=+0.557528944 container attach 6c6fcc14280b0e1c1a22078df7d81db0972e11dc4ba66bec501988d1968ade82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-24T09:19:43.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:43 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:43.353+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:43.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:43 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:43.353+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:43.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:43 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:43.353+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:43.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:43 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:43.353+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:43.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:43 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:43.353+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:43.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:43 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:43.526+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:43.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:43 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:43.526+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:43.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:43 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:43.526+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:43.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:43 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:43.526+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:43.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:43 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:43.526+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:11.583298+0000 front 2024-04-24T09:19:11.583263+0000 (oldest deadline 2024-04-24T09:19:36.283362+0000) 2024-04-24T09:19:44.304 INFO:journalctl@ceph.osd.7.smithi081.stdout:Apr 24 09:19:44 smithi081 podman[67943]: 2024-04-24 09:19:44.010785343 +0000 UTC m=+0.929098027 container died 6c6fcc14280b0e1c1a22078df7d81db0972e11dc4ba66bec501988d1968ade82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=) 2024-04-24T09:19:44.304 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:43 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:43.896+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:44.304 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:43 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:43.896+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:44.304 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:43 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:43.896+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:44.304 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:43 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:43.896+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:44.304 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:43 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:43.897+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:44.526 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:44 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:44.212+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:44.526 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:44 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:44.212+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:44.526 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:44 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:44.212+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:44.526 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:44 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:44.212+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:44.675 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.7.service' 2024-04-24T09:19:44.784 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:19:44.785 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-24T09:19:44.785 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-24T09:19:44.785 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.8 2024-04-24T09:19:44.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:44 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:44.398+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:44.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:44 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:44.398+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:44.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:44 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:44.398+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:44.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:44 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:44.398+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:44.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:44 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:44.398+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:44.866 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:44 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:44.526+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:44.866 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:44 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:44.526+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:44.866 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:44 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:44.526+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:44.867 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:44 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:44.526+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:44.867 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:44 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:44.526+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:11.583298+0000 front 2024-04-24T09:19:11.583263+0000 (oldest deadline 2024-04-24T09:19:36.283362+0000) 2024-04-24T09:19:45.197 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:44 smithi130 systemd[1]: Stopping Ceph osd.8 for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:19:45.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:44 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:44.890+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:45.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:44 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:44.890+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:45.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:44 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:44.890+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:45.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:44 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:44.890+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:45.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:44 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:44.890+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:45.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:44 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:44.890+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:19:45.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:45 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:45.197+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:45.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:45 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:45.197+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:45.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:45 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:45.197+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:45.455 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:45 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:45.197+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:45.455 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:45 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:45.391+0000 7fd83f800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:19:45.455 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:45 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:45.391+0000 7fd83f800700 -1 osd.8 103 *** Got signal Terminated *** 2024-04-24T09:19:45.455 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:45 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:45.391+0000 7fd83f800700 -1 osd.8 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:19:45.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:45 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:45.432+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:45.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:45 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:45.432+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:45.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:45 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:45.432+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:45.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:45 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:45.432+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:45.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:45 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:45.432+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:45.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:45 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:45.499+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:45.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:45 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:45.499+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:45.955 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:45 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:45.499+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:45.955 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:45 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:45.499+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:45.955 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:45 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:45.499+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:11.583298+0000 front 2024-04-24T09:19:11.583263+0000 (oldest deadline 2024-04-24T09:19:36.283362+0000) 2024-04-24T09:19:45.955 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:45 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:45.499+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.086298+0000 front 2024-04-24T09:19:22.086645+0000 (oldest deadline 2024-04-24T09:19:44.986193+0000) 2024-04-24T09:19:46.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:45 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:45.881+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:46.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:45 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:45.881+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:46.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:45 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:45.881+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:46.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:45 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:45.881+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:46.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:45 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:45.881+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:46.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:45 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:45.881+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:19:46.488 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:46 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:46.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:46.488 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:46 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:46.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:46.488 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:46 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:46.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:46.488 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:46 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:46.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:46.488 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:46 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:46.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:19:46.712 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:46 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:46.415+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:46.712 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:46 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:46.415+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:46.712 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:46 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:46.415+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:46.712 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:46 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:46.415+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:46.713 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:46 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:46.415+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:46.713 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:46 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:46.415+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:21.903009+0000 front 2024-04-24T09:19:21.903324+0000 (oldest deadline 2024-04-24T09:19:46.002792+0000) 2024-04-24T09:19:46.954 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:46 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:46.487+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:46.955 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:46 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:46.487+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:46.955 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:46 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:46.487+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:46.955 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:46 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:46.487+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:46.955 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:46 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:46.487+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:11.583298+0000 front 2024-04-24T09:19:11.583263+0000 (oldest deadline 2024-04-24T09:19:36.283362+0000) 2024-04-24T09:19:46.955 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:46 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:46.487+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.086298+0000 front 2024-04-24T09:19:22.086645+0000 (oldest deadline 2024-04-24T09:19:44.986193+0000) 2024-04-24T09:19:47.309 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:46 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:46.920+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:47.310 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:46 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:46.920+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:47.310 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:46 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:46.920+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:47.310 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:46 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:46.920+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:47.310 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:46 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:46.920+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:47.310 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:46 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:46.920+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:19:47.446 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:47 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:47.180+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:47.447 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:47 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:47.180+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:47.447 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:47 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:47.180+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:47.447 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:47 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:47.180+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:47.447 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:47 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:47.180+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:19:47.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:47 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:47.446+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:47.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:47 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:47.446+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:47.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:47 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:47.446+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:47.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:47 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:47.446+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:47.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:47 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:47.446+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:11.583298+0000 front 2024-04-24T09:19:11.583263+0000 (oldest deadline 2024-04-24T09:19:36.283362+0000) 2024-04-24T09:19:47.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:47 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:47.446+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.086298+0000 front 2024-04-24T09:19:22.086645+0000 (oldest deadline 2024-04-24T09:19:44.986193+0000) 2024-04-24T09:19:47.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:47 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:47.448+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:47.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:47 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:47.448+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:47.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:47 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:47.448+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:47.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:47 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:47.448+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:47.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:47 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:47.448+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:47.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:47 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:47.448+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:21.903009+0000 front 2024-04-24T09:19:21.903324+0000 (oldest deadline 2024-04-24T09:19:46.002792+0000) 2024-04-24T09:19:48.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:47 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:47.922+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:48.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:47 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:47.922+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:48.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:47 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:47.922+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:48.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:47 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:47.922+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:48.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:47 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:47.922+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:48.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:47 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:47.922+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:19:48.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:48 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:48.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:48.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:48 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:48.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:48.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:48 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:48.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:48.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:48 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:48.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:48.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:48 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:48.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:19:48.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:48 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:48.407+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:48.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:48 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:48.407+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:48.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:48 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:48.407+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:48.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:48 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:48.408+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:48.706 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:48 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:48.408+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:11.583298+0000 front 2024-04-24T09:19:11.583263+0000 (oldest deadline 2024-04-24T09:19:36.283362+0000) 2024-04-24T09:19:48.706 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:48 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:48.408+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.086298+0000 front 2024-04-24T09:19:22.086645+0000 (oldest deadline 2024-04-24T09:19:44.986193+0000) 2024-04-24T09:19:48.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:48 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:48.492+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:48.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:48 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:48.492+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:48.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:48 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:48.492+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:48.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:48 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:48.492+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:48.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:48 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:48.492+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:48.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:48 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:48.492+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:21.903009+0000 front 2024-04-24T09:19:21.903324+0000 (oldest deadline 2024-04-24T09:19:46.002792+0000) 2024-04-24T09:19:49.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:48 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:48.918+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:49.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:48 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:48.918+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:49.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:48 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:48.918+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:49.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:48 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:48.918+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:49.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:48 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:48.918+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:49.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:48 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:48.918+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:19:49.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:49 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:49.261+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:49.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:49 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:49.261+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:49.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:49 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:49.261+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:49.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:49 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:49.261+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:49.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:49 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:49.261+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:19:49.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:49 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:49.396+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.869616+0000 front 2024-04-24T09:18:33.869108+0000 (oldest deadline 2024-04-24T09:18:59.769289+0000) 2024-04-24T09:19:49.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:49 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:49.396+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.071344+0000 front 2024-04-24T09:18:42.071456+0000 (oldest deadline 2024-04-24T09:19:07.371542+0000) 2024-04-24T09:19:49.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:49 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:49.396+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:53.774675+0000 front 2024-04-24T09:18:53.774744+0000 (oldest deadline 2024-04-24T09:19:19.674915+0000) 2024-04-24T09:19:49.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:49 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:49.396+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.177104+0000 front 2024-04-24T09:19:03.177242+0000 (oldest deadline 2024-04-24T09:19:26.677040+0000) 2024-04-24T09:19:49.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:49 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:49.396+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:11.583298+0000 front 2024-04-24T09:19:11.583263+0000 (oldest deadline 2024-04-24T09:19:36.283362+0000) 2024-04-24T09:19:49.705 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:49 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8[50399]: 2024-04-24T09:19:49.396+0000 7fd83b000700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.086298+0000 front 2024-04-24T09:19:22.086645+0000 (oldest deadline 2024-04-24T09:19:44.986193+0000) 2024-04-24T09:19:49.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:49 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:49.485+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:49.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:49 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:49.485+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:49.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:49 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:49.485+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:49.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:49 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:49.485+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:49.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:49 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:49.485+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:49.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:49 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:49.485+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:21.903009+0000 front 2024-04-24T09:19:21.903324+0000 (oldest deadline 2024-04-24T09:19:46.002792+0000) 2024-04-24T09:19:50.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:49 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:49.941+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:50.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:49 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:49.941+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:50.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:49 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:49.941+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:50.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:49 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:49.941+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:50.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:49 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:49.941+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:50.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:49 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:49.941+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:19:50.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:50 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:50.248+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:50.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:50 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:50.248+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:50.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:50 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:50.248+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:50.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:50 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:50.248+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:50.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:50 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:50.248+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:19:50.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:50 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:50.457+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:50.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:50 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:50.457+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:50.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:50 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:50.457+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:50.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:50 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:50.457+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:50.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:50 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:50.457+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:50.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:50 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:50.457+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:21.903009+0000 front 2024-04-24T09:19:21.903324+0000 (oldest deadline 2024-04-24T09:19:46.002792+0000) 2024-04-24T09:19:51.204 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:50 smithi130 podman[69435]: 2024-04-24 09:19:50.944090891 +0000 UTC m=+5.727277514 container died 3837203adaa9856f17b0471d976da13b1be5433987cb0baa0f1f95189bbb3f46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-24T09:19:51.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:50 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:50.922+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:51.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:50 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:50.922+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:51.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:50 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:50.922+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:51.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:50 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:50.922+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:51.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:50 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:50.922+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:51.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:50 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:50.922+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:19:51.679 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:51 smithi130 podman[69435]: 2024-04-24 09:19:51.417730366 +0000 UTC m=+6.200917011 container cleanup 3837203adaa9856f17b0471d976da13b1be5433987cb0baa0f1f95189bbb3f46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD) 2024-04-24T09:19:51.679 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:51 smithi130 bash[69435]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8 2024-04-24T09:19:51.680 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:51 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:51.232+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:51.680 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:51 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:51.232+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:51.680 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:51 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:51.232+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:51.680 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:51 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:51.232+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:51.680 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:51 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:51.232+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:19:51.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:51 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:51.428+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:51.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:51 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:51.428+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:51.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:51 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:51.428+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:51.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:51 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:51.428+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:51.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:51 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:51.428+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:51.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:51 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:51.428+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:21.903009+0000 front 2024-04-24T09:19:21.903324+0000 (oldest deadline 2024-04-24T09:19:46.002792+0000) 2024-04-24T09:19:51.953 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:51 smithi130 podman[69453]: 2024-04-24 09:19:51.677767581 +0000 UTC m=+0.720871368 container remove 3837203adaa9856f17b0471d976da13b1be5433987cb0baa0f1f95189bbb3f46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:19:52.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:51 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:51.883+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:52.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:51 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:51.883+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:52.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:51 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:51.883+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:52.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:51 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:51.883+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:52.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:51 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:51.883+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:52.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:51 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:51.883+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:19:52.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:52 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:52.190+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:52.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:52 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:52.190+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:52.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:52 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:52.190+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:52.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:52 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:52.190+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:52.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:52 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:52.190+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:19:52.828 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:52 smithi130 podman[69545]: 2024-04-24 09:19:52.365840013 +0000 UTC m=+0.036642608 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:19:52.828 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:52 smithi130 podman[69545]: 2024-04-24 09:19:52.51222104 +0000 UTC m=+0.183023669 container create bb20d03b12d0a161ec1ceac492d029a47c07e5a80fb99a6ac92fefb72270dca0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-24T09:19:52.828 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:52 smithi130 podman[69545]: 2024-04-24 09:19:52.827252064 +0000 UTC m=+0.498054700 container init bb20d03b12d0a161ec1ceac492d029a47c07e5a80fb99a6ac92fefb72270dca0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-24T09:19:52.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:52 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:52.382+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:52.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:52 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:52.382+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:52.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:52 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:52.382+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:52.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:52 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:52.382+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:52.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:52 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:52.382+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:52.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:52 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:52.382+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:21.903009+0000 front 2024-04-24T09:19:21.903324+0000 (oldest deadline 2024-04-24T09:19:46.002792+0000) 2024-04-24T09:19:53.204 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:52 smithi130 podman[69545]: 2024-04-24 09:19:52.838368186 +0000 UTC m=+0.509170821 container start bb20d03b12d0a161ec1ceac492d029a47c07e5a80fb99a6ac92fefb72270dca0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-24T09:19:53.204 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:52 smithi130 podman[69545]: 2024-04-24 09:19:52.948385462 +0000 UTC m=+0.619188151 container attach bb20d03b12d0a161ec1ceac492d029a47c07e5a80fb99a6ac92fefb72270dca0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-24T09:19:53.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:52 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:52.842+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:53.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:52 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:52.842+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:53.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:52 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:52.842+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:53.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:52 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:52.842+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:53.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:52 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:52.842+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:53.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:52 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:52.842+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:19:53.677 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 24 09:19:53 smithi130 podman[69545]: 2024-04-24 09:19:53.281599409 +0000 UTC m=+0.952402044 container died bb20d03b12d0a161ec1ceac492d029a47c07e5a80fb99a6ac92fefb72270dca0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-24T09:19:53.677 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:53 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:53.215+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:53.678 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:53 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:53.216+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:53.678 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:53 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:53.216+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:53.678 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:53 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:53.216+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:53.678 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:53 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:53.216+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:19:53.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:53 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:53.394+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:53.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:53 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:53.394+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:53.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:53 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:53.394+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:53.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:53 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:53.394+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:53.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:53 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:53.394+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:53.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:53 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:53.394+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:21.903009+0000 front 2024-04-24T09:19:21.903324+0000 (oldest deadline 2024-04-24T09:19:46.002792+0000) 2024-04-24T09:19:54.099 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.8.service' 2024-04-24T09:19:54.208 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:19:54.209 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-24T09:19:54.209 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-24T09:19:54.209 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.9 2024-04-24T09:19:54.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:53 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:53.824+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:54.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:53 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:53.824+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:54.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:53 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:53.824+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:54.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:53 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:53.824+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:54.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:53 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:53.824+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:54.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:53 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:53.824+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:19:54.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:54 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:54.183+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:54.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:54 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:54.183+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:54.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:54 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:54.183+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:54.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:54 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:54.183+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:54.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:54 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:54.183+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:19:54.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:54 smithi069 systemd[1]: Stopping Ceph osd.9 for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:19:54.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:54 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:54.403+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:54.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:54 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:54.403+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:54.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:54 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:54.403+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:54.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:54 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:54.403+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:54.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:54 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:54.403+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:54.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:54 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:54.403+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:21.903009+0000 front 2024-04-24T09:19:21.903324+0000 (oldest deadline 2024-04-24T09:19:46.002792+0000) 2024-04-24T09:19:55.092 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:54 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:54.742+0000 7f3eb3c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:19:55.092 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:54 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:54.742+0000 7f3eb3c00700 -1 osd.9 103 *** Got signal Terminated *** 2024-04-24T09:19:55.092 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:54 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:54.742+0000 7f3eb3c00700 -1 osd.9 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:19:55.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:54 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:54.848+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:55.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:54 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:54.848+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:55.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:54 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:54.848+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:55.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:54 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:54.848+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:55.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:54 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:54.848+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:55.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:54 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:54.848+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:19:55.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:55 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:55.212+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:55.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:55 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:55.212+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:55.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:55 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:55.212+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:55.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:55 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:55.212+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:55.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:55 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:55.212+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:19:55.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:55 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:55.212+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:19:55.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:55.433+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:55.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:55.433+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:55.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:55.433+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:55.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:55.433+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:55.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:55.433+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:55.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:55 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:55.433+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:21.903009+0000 front 2024-04-24T09:19:21.903324+0000 (oldest deadline 2024-04-24T09:19:46.002792+0000) 2024-04-24T09:19:56.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:55 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:55.823+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:56.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:55 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:55.823+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:56.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:55 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:55.823+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:56.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:55 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:55.823+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:56.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:55 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:55.823+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:56.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:55 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:55.823+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:19:56.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:56 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:56.236+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:56.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:56 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:56.236+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:56.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:56 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:56.236+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:56.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:56 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:56.236+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:56.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:56 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:56.236+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:19:56.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:56 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:56.236+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:19:56.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:56 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:56.402+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:56.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:56 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:56.402+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:56.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:56 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:56.402+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:56.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:56 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:56.402+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:56.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:56 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:56.402+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:56.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:56 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:56.402+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:21.903009+0000 front 2024-04-24T09:19:21.903324+0000 (oldest deadline 2024-04-24T09:19:46.002792+0000) 2024-04-24T09:19:57.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:56 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:56.827+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:57.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:56 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:56.827+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:57.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:56 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:56.827+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:57.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:56 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:56.827+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:57.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:56 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:56.827+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:57.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:56 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:56.827+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:19:57.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:57 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:57.258+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:57.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:57 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:57.258+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:57.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:57 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:57.258+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:57.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:57 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:57.258+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:57.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:57 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:57.258+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:19:57.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:57 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:57.258+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:19:57.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:57 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:57.433+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:57.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:57 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:57.433+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:57.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:57 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:57.433+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:57.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:57 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:57.433+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:57.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:57 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:57.433+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:57.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:57 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:57.433+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:21.903009+0000 front 2024-04-24T09:19:21.903324+0000 (oldest deadline 2024-04-24T09:19:46.002792+0000) 2024-04-24T09:19:58.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:57 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:57.819+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:58.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:57 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:57.819+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:58.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:57 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:57.819+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:58.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:57 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:57.819+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:58.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:57 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:57.819+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:58.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:57 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:57.819+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:19:58.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:57 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:57.819+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.529347+0000 front 2024-04-24T09:19:31.529329+0000 (oldest deadline 2024-04-24T09:19:57.428633+0000) 2024-04-24T09:19:58.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:58 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:58.251+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:58.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:58 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:58.251+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:58.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:58 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:58.251+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:58.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:58 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:58.251+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:58.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:58 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:58.251+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:19:58.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:58 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:58.251+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:19:58.743 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:58 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:58.481+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:58.743 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:58 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:58.481+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:58.743 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:58 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:58.481+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:58.744 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:58 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:58.481+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:58.744 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:58 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:58.481+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:58.744 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:58 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:58.481+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:21.903009+0000 front 2024-04-24T09:19:21.903324+0000 (oldest deadline 2024-04-24T09:19:46.002792+0000) 2024-04-24T09:19:59.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:58 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:58.860+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:19:59.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:58 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:58.860+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:19:59.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:58 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:58.860+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:19:59.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:58 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:58.860+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:19:59.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:58 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:58.860+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:19:59.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:58 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:58.860+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:19:59.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:58 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:58.860+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.529347+0000 front 2024-04-24T09:19:31.529329+0000 (oldest deadline 2024-04-24T09:19:57.428633+0000) 2024-04-24T09:19:59.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:59 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:59.271+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:19:59.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:59 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:59.271+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:19:59.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:59 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:59.271+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:19:59.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:59 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:59.272+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:19:59.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:59 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:59.272+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:19:59.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:19:59 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:19:59.272+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:19:59.841 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:59 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:59.501+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:31.985317+0000 front 2024-04-24T09:18:31.985399+0000 (oldest deadline 2024-04-24T09:18:59.586876+0000) 2024-04-24T09:19:59.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:59 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:59.501+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:44.789675+0000 front 2024-04-24T09:18:44.789955+0000 (oldest deadline 2024-04-24T09:19:08.889733+0000) 2024-04-24T09:19:59.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:59 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:59.501+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:51.191643+0000 front 2024-04-24T09:18:51.191983+0000 (oldest deadline 2024-04-24T09:19:17.592868+0000) 2024-04-24T09:19:59.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:59 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:59.501+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:03.395561+0000 front 2024-04-24T09:19:03.395447+0000 (oldest deadline 2024-04-24T09:19:26.295611+0000) 2024-04-24T09:19:59.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:59 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:59.501+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:13.298579+0000 front 2024-04-24T09:19:13.299012+0000 (oldest deadline 2024-04-24T09:19:35.598677+0000) 2024-04-24T09:19:59.842 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:19:59 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9[60764]: 2024-04-24T09:19:59.501+0000 7f3eaf400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:21.903009+0000 front 2024-04-24T09:19:21.903324+0000 (oldest deadline 2024-04-24T09:19:46.002792+0000) 2024-04-24T09:20:00.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:59 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:59.860+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:20:00.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:59 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:59.860+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:20:00.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:59 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:59.860+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:20:00.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:59 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:59.860+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:20:00.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:59 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:59.860+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:20:00.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:59 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:59.860+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:20:00.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:19:59 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:19:59.860+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.529347+0000 front 2024-04-24T09:19:31.529329+0000 (oldest deadline 2024-04-24T09:19:57.428633+0000) 2024-04-24T09:20:00.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:00 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:00.265+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:00.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:00 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:00.265+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:00.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:00 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:00.265+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:00.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:00 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:00.265+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:00.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:00 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:00.265+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:00.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:00 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:00.265+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:00.710 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:20:00 smithi069 podman[97062]: 2024-04-24 09:20:00.399326681 +0000 UTC m=+5.788267298 container died 08a6b188a4d46a9e5bef3f7ba47aacb2c6e72fd0433c70cbca8cb465b712272b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-24T09:20:01.091 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:20:00 smithi069 podman[97062]: 2024-04-24 09:20:00.819872886 +0000 UTC m=+6.208813519 container cleanup 08a6b188a4d46a9e5bef3f7ba47aacb2c6e72fd0433c70cbca8cb465b712272b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-24T09:20:01.091 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:20:00 smithi069 bash[97062]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9 2024-04-24T09:20:01.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:00 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:00.885+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:20:01.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:00 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:00.885+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:20:01.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:00 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:00.885+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:20:01.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:00 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:00.885+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:20:01.319 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:00 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:00.885+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:20:01.319 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:00 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:00.885+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:20:01.319 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:00 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:00.885+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.529347+0000 front 2024-04-24T09:19:31.529329+0000 (oldest deadline 2024-04-24T09:19:57.428633+0000) 2024-04-24T09:20:01.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:20:01 smithi069 podman[97079]: 2024-04-24 09:20:01.147282404 +0000 UTC m=+0.727625067 container remove 08a6b188a4d46a9e5bef3f7ba47aacb2c6e72fd0433c70cbca8cb465b712272b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-24T09:20:01.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:01.267+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:01.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:01.267+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:01.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:01.267+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:01.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:01.267+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:01.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:01.267+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:01.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:01 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:01.267+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:02.294 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:20:01 smithi069 podman[97170]: 2024-04-24 09:20:01.888606615 +0000 UTC m=+0.034564928 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:20:02.294 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:20:02 smithi069 podman[97170]: 2024-04-24 09:20:02.04720685 +0000 UTC m=+0.193165200 container create 4067cf01b509cec051cdad99d7f79bffff2b27c20e13ee8677b8f1b89c5b7fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True) 2024-04-24T09:20:02.294 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:20:02 smithi069 podman[97170]: 2024-04-24 09:20:02.294474947 +0000 UTC m=+0.440433289 container init 4067cf01b509cec051cdad99d7f79bffff2b27c20e13ee8677b8f1b89c5b7fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-24T09:20:02.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:01 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:01.856+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:20:02.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:01 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:01.856+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:20:02.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:01 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:01.856+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:20:02.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:01 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:01.856+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:20:02.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:01 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:01.856+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:20:02.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:01 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:01.856+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:20:02.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:01 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:01.856+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.529347+0000 front 2024-04-24T09:19:31.529329+0000 (oldest deadline 2024-04-24T09:19:57.428633+0000) 2024-04-24T09:20:02.591 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:20:02 smithi069 podman[97170]: 2024-04-24 09:20:02.305175062 +0000 UTC m=+0.451133392 container start 4067cf01b509cec051cdad99d7f79bffff2b27c20e13ee8677b8f1b89c5b7fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T09:20:02.592 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:20:02 smithi069 podman[97170]: 2024-04-24 09:20:02.399405725 +0000 UTC m=+0.545364112 container attach 4067cf01b509cec051cdad99d7f79bffff2b27c20e13ee8677b8f1b89c5b7fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-24T09:20:02.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:02 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:02.228+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:02.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:02 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:02.228+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:02.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:02 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:02.228+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:02.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:02 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:02.228+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:02.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:02 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:02.228+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:02.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:02 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:02.228+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:03.092 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 24 09:20:02 smithi069 podman[97170]: 2024-04-24 09:20:02.756977569 +0000 UTC m=+0.902935979 container died 4067cf01b509cec051cdad99d7f79bffff2b27c20e13ee8677b8f1b89c5b7fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-24T09:20:03.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:02 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:02.831+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:20:03.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:02 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:02.831+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:20:03.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:02 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:02.831+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:20:03.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:02 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:02.831+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:20:03.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:02 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:02.831+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:20:03.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:02 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:02.831+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:20:03.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:02 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:02.831+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.529347+0000 front 2024-04-24T09:19:31.529329+0000 (oldest deadline 2024-04-24T09:19:57.428633+0000) 2024-04-24T09:20:03.545 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.9.service' 2024-04-24T09:20:03.650 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:20:03.651 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-24T09:20:03.651 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-24T09:20:03.651 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.10 2024-04-24T09:20:03.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:03 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:03.249+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:03.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:03 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:03.249+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:03.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:03 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:03.249+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:03.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:03 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:03.249+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:03.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:03 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:03.249+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:03.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:03 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:03.249+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:04.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:03 smithi081 systemd[1]: Stopping Ceph osd.10 for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:20:04.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:03 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:03.810+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:20:04.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:03 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:03.810+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:20:04.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:03 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:03.810+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:20:04.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:03 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:03.810+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:20:04.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:03 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:03.810+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:20:04.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:03 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:03.810+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:20:04.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:03 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:03.810+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.529347+0000 front 2024-04-24T09:19:31.529329+0000 (oldest deadline 2024-04-24T09:19:57.428633+0000) 2024-04-24T09:20:04.567 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:04 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:04.235+0000 7f012b600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:20:04.567 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:04 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:04.235+0000 7f012b600700 -1 osd.10 103 *** Got signal Terminated *** 2024-04-24T09:20:04.567 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:04 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:04.235+0000 7f012b600700 -1 osd.10 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:20:04.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:04 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:04.297+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:04.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:04 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:04.297+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:04.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:04 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:04.297+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:04.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:04 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:04.297+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:04.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:04 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:04.297+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:04.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:04 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:04.297+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:05.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:04 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:04.779+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:20:05.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:04 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:04.779+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:20:05.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:04 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:04.779+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:20:05.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:04 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:04.779+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:20:05.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:04 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:04.779+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:20:05.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:04 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:04.779+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:20:05.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:04 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:04.779+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.529347+0000 front 2024-04-24T09:19:31.529329+0000 (oldest deadline 2024-04-24T09:19:57.428633+0000) 2024-04-24T09:20:05.606 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:05 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:05.300+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:05.606 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:05 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:05.300+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:05.607 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:05 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:05.300+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:05.607 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:05 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:05.300+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:05.607 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:05 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:05.300+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:05.607 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:05 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:05.300+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:05.607 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:05 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:05.300+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-04-24T09:19:39.868199+0000 front 2024-04-24T09:19:39.868166+0000 (oldest deadline 2024-04-24T09:20:05.167430+0000) 2024-04-24T09:20:06.101 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:05 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:05.821+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:20:06.101 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:05 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:05.821+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:20:06.101 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:05 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:05.821+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:20:06.101 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:05 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:05.821+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:20:06.102 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:05 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:05.821+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:20:06.102 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:05 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:05.821+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:20:06.102 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:05 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:05.821+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.529347+0000 front 2024-04-24T09:19:31.529329+0000 (oldest deadline 2024-04-24T09:19:57.428633+0000) 2024-04-24T09:20:06.635 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:06 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:06.274+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:06.635 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:06 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:06.274+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:06.635 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:06 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:06.274+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:06.635 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:06 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:06.274+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:06.635 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:06 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:06.274+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:06.636 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:06 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:06.274+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:06.636 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:06 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:06.274+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-04-24T09:19:39.868199+0000 front 2024-04-24T09:19:39.868166+0000 (oldest deadline 2024-04-24T09:20:05.167430+0000) 2024-04-24T09:20:07.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:06 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:06.806+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:20:07.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:06 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:06.806+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:20:07.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:06 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:06.806+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:20:07.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:06 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:06.806+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:20:07.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:06 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:06.806+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:20:07.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:06 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:06.806+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:20:07.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:06 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:06.806+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.529347+0000 front 2024-04-24T09:19:31.529329+0000 (oldest deadline 2024-04-24T09:19:57.428633+0000) 2024-04-24T09:20:07.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:07 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:07.294+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:07.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:07 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:07.294+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:07.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:07 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:07.294+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:07.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:07 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:07.294+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:07.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:07 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:07.294+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:07.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:07 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:07.294+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:07.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:07 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:07.294+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-04-24T09:19:39.868199+0000 front 2024-04-24T09:19:39.868166+0000 (oldest deadline 2024-04-24T09:20:05.167430+0000) 2024-04-24T09:20:08.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:07 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:07.784+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:20:08.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:07 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:07.784+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:20:08.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:07 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:07.784+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:20:08.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:07 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:07.784+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:20:08.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:07 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:07.784+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:20:08.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:07 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:07.784+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:20:08.068 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:07 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:07.784+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.529347+0000 front 2024-04-24T09:19:31.529329+0000 (oldest deadline 2024-04-24T09:19:57.428633+0000) 2024-04-24T09:20:08.453 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:08 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:08.245+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:08.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:08 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:08.245+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:08.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:08 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:08.245+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:08.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:08 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:08.245+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:08.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:08 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:08.245+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:08.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:08 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:08.245+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:08.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:08 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:08.245+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-04-24T09:19:39.868199+0000 front 2024-04-24T09:19:39.868166+0000 (oldest deadline 2024-04-24T09:20:05.167430+0000) 2024-04-24T09:20:09.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:08.834+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.713451+0000 front 2024-04-24T09:18:33.713257+0000 (oldest deadline 2024-04-24T09:18:59.613075+0000) 2024-04-24T09:20:09.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:08.834+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:42.515098+0000 front 2024-04-24T09:18:42.515269+0000 (oldest deadline 2024-04-24T09:19:06.615340+0000) 2024-04-24T09:20:09.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:08.834+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:50.117200+0000 front 2024-04-24T09:18:50.117324+0000 (oldest deadline 2024-04-24T09:19:20.118292+0000) 2024-04-24T09:20:09.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:08.834+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-24T09:19:01.220562+0000 front 2024-04-24T09:19:01.220518+0000 (oldest deadline 2024-04-24T09:19:25.221345+0000) 2024-04-24T09:20:09.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:08.834+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:08.123394+0000 front 2024-04-24T09:19:08.123303+0000 (oldest deadline 2024-04-24T09:19:34.023290+0000) 2024-04-24T09:20:09.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:08.835+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:18.725343+0000 front 2024-04-24T09:19:18.725569+0000 (oldest deadline 2024-04-24T09:19:44.025172+0000) 2024-04-24T09:20:09.318 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:08 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10[55811]: 2024-04-24T09:20:08.835+0000 7f0121e00700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.529347+0000 front 2024-04-24T09:19:31.529329+0000 (oldest deadline 2024-04-24T09:19:57.428633+0000) 2024-04-24T09:20:09.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:09 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:09.277+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:09.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:09 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:09.277+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:09.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:09 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:09.277+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:09.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:09 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:09.277+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:09.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:09 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:09.277+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:09.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:09 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:09.277+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:09.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:09 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:09.277+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-04-24T09:19:39.868199+0000 front 2024-04-24T09:19:39.868166+0000 (oldest deadline 2024-04-24T09:20:05.167430+0000) 2024-04-24T09:20:10.067 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:09 smithi081 podman[68054]: 2024-04-24 09:20:09.782677701 +0000 UTC m=+5.688494685 container died 79a5e3de6fa4d18137ef2af97fe101539d767b5d8d59974bde476d90614e4e63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-24T09:20:10.431 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:10 smithi081 podman[68054]: 2024-04-24 09:20:10.213851031 +0000 UTC m=+6.119668008 container cleanup 79a5e3de6fa4d18137ef2af97fe101539d767b5d8d59974bde476d90614e4e63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-24T09:20:10.432 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:10 smithi081 bash[68054]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10 2024-04-24T09:20:10.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:10 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:10.291+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:10.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:10 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:10.291+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:10.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:10 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:10.291+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:10.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:10 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:10.291+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:10.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:10 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:10.291+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:10.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:10 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:10.291+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:10.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:10 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:10.291+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-04-24T09:19:39.868199+0000 front 2024-04-24T09:19:39.868166+0000 (oldest deadline 2024-04-24T09:20:05.167430+0000) 2024-04-24T09:20:10.817 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:10 smithi081 podman[68085]: 2024-04-24 09:20:10.431300711 +0000 UTC m=+0.634894793 container remove 79a5e3de6fa4d18137ef2af97fe101539d767b5d8d59974bde476d90614e4e63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-04-24T09:20:11.564 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:11 smithi081 podman[68174]: 2024-04-24 09:20:11.152912339 +0000 UTC m=+0.035385338 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:20:11.564 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:11 smithi081 podman[68174]: 2024-04-24 09:20:11.274017583 +0000 UTC m=+0.156490605 container create 93bc5da90bcd926ddee181968f66749622726e5991fc66ed79da82ea26f8045b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424) 2024-04-24T09:20:11.564 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:11 smithi081 podman[68174]: 2024-04-24 09:20:11.562638807 +0000 UTC m=+0.445111859 container init 93bc5da90bcd926ddee181968f66749622726e5991fc66ed79da82ea26f8045b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-04-24T09:20:11.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:11 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:11.319+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:11.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:11 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:11.319+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:11.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:11 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:11.319+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:11.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:11 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:11.319+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:11.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:11 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:11.319+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:11.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:11 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:11.319+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:11.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:11 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:11.319+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-04-24T09:19:39.868199+0000 front 2024-04-24T09:19:39.868166+0000 (oldest deadline 2024-04-24T09:20:05.167430+0000) 2024-04-24T09:20:11.817 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:11 smithi081 podman[68174]: 2024-04-24 09:20:11.575474156 +0000 UTC m=+0.457947155 container start 93bc5da90bcd926ddee181968f66749622726e5991fc66ed79da82ea26f8045b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T09:20:11.817 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:11 smithi081 podman[68174]: 2024-04-24 09:20:11.66054834 +0000 UTC m=+0.543021402 container attach 93bc5da90bcd926ddee181968f66749622726e5991fc66ed79da82ea26f8045b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:20:12.317 INFO:journalctl@ceph.osd.10.smithi081.stdout:Apr 24 09:20:12 smithi081 podman[68174]: 2024-04-24 09:20:12.020308655 +0000 UTC m=+0.902781681 container died 93bc5da90bcd926ddee181968f66749622726e5991fc66ed79da82ea26f8045b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True) 2024-04-24T09:20:12.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:12 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:12.290+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:12.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:12 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:12.290+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:12.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:12 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:12.290+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:12.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:12 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:12.290+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:12.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:12 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:12.290+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:12.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:12 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:12.290+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:12.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:12 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:12.290+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-04-24T09:19:39.868199+0000 front 2024-04-24T09:19:39.868166+0000 (oldest deadline 2024-04-24T09:20:05.167430+0000) 2024-04-24T09:20:12.735 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.10.service' 2024-04-24T09:20:12.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:20:12.842 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-24T09:20:12.842 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-24T09:20:12.842 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.11 2024-04-24T09:20:13.204 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:12 smithi130 systemd[1]: Stopping Ceph osd.11 for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:20:13.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:13.338+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:13.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:13.338+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:13.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:13.338+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:13.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:13.338+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:13.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:13.338+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:13.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:13.338+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:13.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:13.338+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-04-24T09:19:39.868199+0000 front 2024-04-24T09:19:39.868166+0000 (oldest deadline 2024-04-24T09:20:05.167430+0000) 2024-04-24T09:20:13.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:13.338+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6820 osd.8 since back 2024-04-24T09:19:49.169626+0000 front 2024-04-24T09:19:49.169727+0000 (oldest deadline 2024-04-24T09:20:12.669579+0000) 2024-04-24T09:20:13.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:13.462+0000 7f19e8400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:20:13.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:13.462+0000 7f19e8400700 -1 osd.11 103 *** Got signal Terminated *** 2024-04-24T09:20:13.706 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:13 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:13.462+0000 7f19e8400700 -1 osd.11 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:20:14.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:14 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:14.293+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:14.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:14 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:14.293+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:14.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:14 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:14.293+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:14.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:14 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:14.293+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:14.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:14 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:14.293+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:14.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:14 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:14.293+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:14.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:14 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:14.293+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-04-24T09:19:39.868199+0000 front 2024-04-24T09:19:39.868166+0000 (oldest deadline 2024-04-24T09:20:05.167430+0000) 2024-04-24T09:20:14.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:14 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:14.293+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6820 osd.8 since back 2024-04-24T09:19:49.169626+0000 front 2024-04-24T09:19:49.169727+0000 (oldest deadline 2024-04-24T09:20:12.669579+0000) 2024-04-24T09:20:15.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:15 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:15.254+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:15.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:15 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:15.254+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:15.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:15 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:15.254+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:15.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:15 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:15.254+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:15.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:15 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:15.254+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:15.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:15 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:15.254+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:15.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:15 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:15.254+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-04-24T09:19:39.868199+0000 front 2024-04-24T09:19:39.868166+0000 (oldest deadline 2024-04-24T09:20:05.167430+0000) 2024-04-24T09:20:15.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:15 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:15.254+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6820 osd.8 since back 2024-04-24T09:19:49.169626+0000 front 2024-04-24T09:19:49.169727+0000 (oldest deadline 2024-04-24T09:20:12.669579+0000) 2024-04-24T09:20:16.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:16 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:16.225+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:16.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:16 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:16.225+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:16.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:16 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:16.225+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:16.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:16 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:16.225+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:16.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:16 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:16.225+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:16.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:16 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:16.225+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:16.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:16 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:16.225+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-04-24T09:19:39.868199+0000 front 2024-04-24T09:19:39.868166+0000 (oldest deadline 2024-04-24T09:20:05.167430+0000) 2024-04-24T09:20:16.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:16 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:16.225+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6820 osd.8 since back 2024-04-24T09:19:49.169626+0000 front 2024-04-24T09:19:49.169727+0000 (oldest deadline 2024-04-24T09:20:12.669579+0000) 2024-04-24T09:20:17.646 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:17.223+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:17.647 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:17.223+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:17.647 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:17.223+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:17.647 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:17.223+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:17.647 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:17.223+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:17.647 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:17.223+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:17.647 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:17.223+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-04-24T09:19:39.868199+0000 front 2024-04-24T09:19:39.868166+0000 (oldest deadline 2024-04-24T09:20:05.167430+0000) 2024-04-24T09:20:17.648 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:17 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:17.223+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6820 osd.8 since back 2024-04-24T09:19:49.169626+0000 front 2024-04-24T09:19:49.169727+0000 (oldest deadline 2024-04-24T09:20:12.669579+0000) 2024-04-24T09:20:18.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:18.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-24T09:18:33.654813+0000 front 2024-04-24T09:18:33.654555+0000 (oldest deadline 2024-04-24T09:18:58.353848+0000) 2024-04-24T09:20:18.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:18.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2024-04-24T09:18:43.056593+0000 front 2024-04-24T09:18:43.056147+0000 (oldest deadline 2024-04-24T09:19:08.955856+0000) 2024-04-24T09:20:18.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:18.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-24T09:18:48.956340+0000 front 2024-04-24T09:18:48.956416+0000 (oldest deadline 2024-04-24T09:19:15.957072+0000) 2024-04-24T09:20:18.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:18.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6812 osd.4 since back 2024-04-24T09:19:09.363209+0000 front 2024-04-24T09:19:09.362789+0000 (oldest deadline 2024-04-24T09:19:38.163153+0000) 2024-04-24T09:20:18.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:18.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-24T09:19:22.264884+0000 front 2024-04-24T09:19:22.265093+0000 (oldest deadline 2024-04-24T09:19:45.764717+0000) 2024-04-24T09:20:18.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:18.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-24T09:19:31.066647+0000 front 2024-04-24T09:19:31.066611+0000 (oldest deadline 2024-04-24T09:19:54.566158+0000) 2024-04-24T09:20:18.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:18.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.81:6820 osd.7 since back 2024-04-24T09:19:39.868199+0000 front 2024-04-24T09:19:39.868166+0000 (oldest deadline 2024-04-24T09:20:05.167430+0000) 2024-04-24T09:20:18.705 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11[57499]: 2024-04-24T09:20:18.218+0000 7f19e3c00700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.130:6820 osd.8 since back 2024-04-24T09:19:49.169626+0000 front 2024-04-24T09:19:49.169727+0000 (oldest deadline 2024-04-24T09:20:12.669579+0000) 2024-04-24T09:20:19.370 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:19 smithi130 podman[69662]: 2024-04-24 09:20:19.093548324 +0000 UTC m=+5.776569177 container died e681b68ca3059c2964a9510a8c02551dbf7dc6a436c799a11d1ca23f867977c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-24T09:20:19.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:19 smithi130 podman[69662]: 2024-04-24 09:20:19.548672684 +0000 UTC m=+6.231693569 container cleanup e681b68ca3059c2964a9510a8c02551dbf7dc6a436c799a11d1ca23f867977c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-24T09:20:19.704 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:19 smithi130 bash[69662]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11 2024-04-24T09:20:20.204 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:19 smithi130 podman[69683]: 2024-04-24 09:20:19.815345486 +0000 UTC m=+0.712173111 container remove e681b68ca3059c2964a9510a8c02551dbf7dc6a436c799a11d1ca23f867977c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:20:20.960 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:20 smithi130 podman[69772]: 2024-04-24 09:20:20.616543463 +0000 UTC m=+0.034142006 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:20:20.960 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:20 smithi130 podman[69772]: 2024-04-24 09:20:20.724393097 +0000 UTC m=+0.141991676 container create 1e1722ef1c1da0637bdf2155453c6c134c67cefc5b359fd1de4fe71962d05ea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, ceph=True) 2024-04-24T09:20:21.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:21 smithi130 podman[69772]: 2024-04-24 09:20:21.070140131 +0000 UTC m=+0.487738720 container init 1e1722ef1c1da0637bdf2155453c6c134c67cefc5b359fd1de4fe71962d05ea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:20:21.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:21 smithi130 podman[69772]: 2024-04-24 09:20:21.082515982 +0000 UTC m=+0.500114567 container start 1e1722ef1c1da0637bdf2155453c6c134c67cefc5b359fd1de4fe71962d05ea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-24T09:20:21.454 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:21 smithi130 podman[69772]: 2024-04-24 09:20:21.185553156 +0000 UTC m=+0.603151785 container attach 1e1722ef1c1da0637bdf2155453c6c134c67cefc5b359fd1de4fe71962d05ea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:20:21.875 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 24 09:20:21 smithi130 podman[69772]: 2024-04-24 09:20:21.529241378 +0000 UTC m=+0.946840010 container died 1e1722ef1c1da0637bdf2155453c6c134c67cefc5b359fd1de4fe71962d05ea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-24T09:20:22.322 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@osd.11.service' 2024-04-24T09:20:22.436 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:20:22.436 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-24T09:20:22.437 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-24T09:20:22.437 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.a 2024-04-24T09:20:22.841 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 24 09:20:22 smithi069 systemd[1]: Stopping Ceph mds.a for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:20:23.341 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 24 09:20:23 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-a[67376]: 2024-04-24T09:20:23.009+0000 7ff5d5400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:20:23.342 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 24 09:20:23 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-a[67376]: 2024-04-24T09:20:23.009+0000 7ff5d5400700 -1 mds.a *** got signal Terminated *** 2024-04-24T09:20:25.815 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 24 09:20:25 smithi069 podman[97275]: 2024-04-24 09:20:25.484712303 +0000 UTC m=+2.601430959 container died 3eefa30160f81c90203c386b030c3018aa59acf91b444aece3d10c1814c45d6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T09:20:26.091 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 24 09:20:25 smithi069 podman[97275]: 2024-04-24 09:20:25.927833783 +0000 UTC m=+3.044552415 container cleanup 3eefa30160f81c90203c386b030c3018aa59acf91b444aece3d10c1814c45d6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True) 2024-04-24T09:20:26.092 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 24 09:20:25 smithi069 bash[97275]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-a 2024-04-24T09:20:26.591 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 24 09:20:26 smithi069 podman[97289]: 2024-04-24 09:20:26.120490307 +0000 UTC m=+0.612587533 container remove 3eefa30160f81c90203c386b030c3018aa59acf91b444aece3d10c1814c45d6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-a, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-24T09:20:26.685 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.a.service' 2024-04-24T09:20:26.786 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:20:26.787 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-24T09:20:26.787 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-24T09:20:26.787 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.d 2024-04-24T09:20:27.341 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:20:26 smithi069 systemd[1]: Stopping Ceph mds.d for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:20:27.841 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:20:27 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-d[66277]: 2024-04-24T09:20:27.347+0000 7f9361a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:20:27.841 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:20:27 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-d[66277]: 2024-04-24T09:20:27.347+0000 7f9361a00700 -1 mds.d *** got signal Terminated *** 2024-04-24T09:20:28.952 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:20:28 smithi069 podman[97395]: 2024-04-24 09:20:28.662916972 +0000 UTC m=+1.439417999 container died 4e35c7e7412250d7c61d0961a4d9d857bfc58ae10f726e4405ad5d48cafff212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T09:20:29.341 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:20:29 smithi069 podman[97395]: 2024-04-24 09:20:29.064863371 +0000 UTC m=+1.841364430 container cleanup 4e35c7e7412250d7c61d0961a4d9d857bfc58ae10f726e4405ad5d48cafff212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T09:20:29.341 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:20:29 smithi069 bash[97395]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-d 2024-04-24T09:20:29.632 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.d.service' 2024-04-24T09:20:29.700 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:20:29 smithi069 podman[97418]: 2024-04-24 09:20:29.399329274 +0000 UTC m=+0.721807432 container remove 4e35c7e7412250d7c61d0961a4d9d857bfc58ae10f726e4405ad5d48cafff212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T09:20:29.701 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:20:29 smithi069 systemd[1]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.d.service: Deactivated successfully. 2024-04-24T09:20:29.701 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:20:29 smithi069 systemd[1]: Stopped Ceph mds.d for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T09:20:29.701 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 24 09:20:29 smithi069 systemd[1]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.d.service: Consumed 2.236s CPU time. 2024-04-24T09:20:29.735 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:20:29.735 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-24T09:20:29.735 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-24T09:20:29.736 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.g 2024-04-24T09:20:30.091 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 24 09:20:29 smithi069 systemd[1]: Stopping Ceph mds.g for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:20:30.841 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 24 09:20:30 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-g[67013]: 2024-04-24T09:20:30.357+0000 7fcac8800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:20:30.841 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 24 09:20:30 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-g[67013]: 2024-04-24T09:20:30.357+0000 7fcac8800700 -1 mds.g *** got signal Terminated *** 2024-04-24T09:20:33.583 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 24 09:20:33 smithi069 podman[97526]: 2024-04-24 09:20:33.287847932 +0000 UTC m=+3.065519048 container died bf470faf116c17f1f68e738f8609ee41c69c870575137f43a71417e7e1bfe460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-24T09:20:33.841 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 24 09:20:33 smithi069 podman[97526]: 2024-04-24 09:20:33.693620986 +0000 UTC m=+3.471292076 container cleanup bf470faf116c17f1f68e738f8609ee41c69c870575137f43a71417e7e1bfe460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-g, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:20:33.842 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 24 09:20:33 smithi069 bash[97526]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-g 2024-04-24T09:20:34.341 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 24 09:20:33 smithi069 podman[97540]: 2024-04-24 09:20:33.953011057 +0000 UTC m=+0.646287565 container remove bf470faf116c17f1f68e738f8609ee41c69c870575137f43a71417e7e1bfe460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-g, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-24T09:20:34.545 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.g.service' 2024-04-24T09:20:34.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:20:34.651 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-24T09:20:34.651 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-24T09:20:34.651 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.j 2024-04-24T09:20:35.091 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 24 09:20:34 smithi069 systemd[1]: Stopping Ceph mds.j for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:20:35.591 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 24 09:20:35 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-j[65894]: 2024-04-24T09:20:35.219+0000 7fb9e3a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:20:35.592 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 24 09:20:35 smithi069 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-j[65894]: 2024-04-24T09:20:35.219+0000 7fb9e3a00700 -1 mds.j *** got signal Terminated *** 2024-04-24T09:20:36.841 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 24 09:20:36 smithi069 podman[97646]: 2024-04-24 09:20:36.548740138 +0000 UTC m=+1.440540817 container died 85b5983e6d728bd153c1e096cc747f11a0ff7135a070fea97d53e879ca500756 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-24T09:20:37.139 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 24 09:20:36 smithi069 podman[97646]: 2024-04-24 09:20:36.963875852 +0000 UTC m=+1.855676527 container cleanup 85b5983e6d728bd153c1e096cc747f11a0ff7135a070fea97d53e879ca500756 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-24T09:20:37.139 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 24 09:20:36 smithi069 bash[97646]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-j 2024-04-24T09:20:37.591 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 24 09:20:37 smithi069 podman[97660]: 2024-04-24 09:20:37.139647813 +0000 UTC m=+0.575111729 container remove 85b5983e6d728bd153c1e096cc747f11a0ff7135a070fea97d53e879ca500756 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-24T09:20:37.644 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.j.service' 2024-04-24T09:20:37.722 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:20:37.723 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-24T09:20:37.723 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-24T09:20:37.723 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.b 2024-04-24T09:20:38.067 INFO:journalctl@ceph.mds.b.smithi081.stdout:Apr 24 09:20:37 smithi081 systemd[1]: Stopping Ceph mds.b for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:20:38.567 INFO:journalctl@ceph.mds.b.smithi081.stdout:Apr 24 09:20:38 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-b[61445]: 2024-04-24T09:20:38.305+0000 7f7a18000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:20:38.568 INFO:journalctl@ceph.mds.b.smithi081.stdout:Apr 24 09:20:38 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-b[61445]: 2024-04-24T09:20:38.305+0000 7f7a18000700 -1 mds.b *** got signal Terminated *** 2024-04-24T09:20:48.171 INFO:journalctl@ceph.mds.b.smithi081.stdout:Apr 24 09:20:47 smithi081 podman[68279]: 2024-04-24 09:20:47.83393962 +0000 UTC m=+9.672883939 container stop 3c6bdbb22db83e2799fee2d0cbdae4cd3df623626ef2363d45977f355b764d34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-04-24T09:20:48.172 INFO:journalctl@ceph.mds.b.smithi081.stdout:Apr 24 09:20:47 smithi081 podman[68279]: 2024-04-24 09:20:47.860624923 +0000 UTC m=+9.699569227 container died 3c6bdbb22db83e2799fee2d0cbdae4cd3df623626ef2363d45977f355b764d34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-b, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-24T09:20:48.533 INFO:journalctl@ceph.mds.b.smithi081.stdout:Apr 24 09:20:48 smithi081 podman[68279]: 2024-04-24 09:20:48.299084007 +0000 UTC m=+10.138028345 container cleanup 3c6bdbb22db83e2799fee2d0cbdae4cd3df623626ef2363d45977f355b764d34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=) 2024-04-24T09:20:48.533 INFO:journalctl@ceph.mds.b.smithi081.stdout:Apr 24 09:20:48 smithi081 bash[68279]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-b 2024-04-24T09:20:48.817 INFO:journalctl@ceph.mds.b.smithi081.stdout:Apr 24 09:20:48 smithi081 podman[68305]: 2024-04-24 09:20:48.533639284 +0000 UTC m=+0.679845082 container remove 3c6bdbb22db83e2799fee2d0cbdae4cd3df623626ef2363d45977f355b764d34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-b, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-24T09:20:49.015 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.b.service' 2024-04-24T09:20:49.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:20:49.128 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-24T09:20:49.128 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-24T09:20:49.129 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.e 2024-04-24T09:20:49.566 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:20:49 smithi081 systemd[1]: Stopping Ceph mds.e for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:20:50.067 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:20:49 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:20:49.709+0000 7f259d000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:20:50.067 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:20:49 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e[60819]: 2024-04-24T09:20:49.709+0000 7f259d000700 -1 mds.e *** got signal Terminated *** 2024-04-24T09:20:55.232 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:20:54 smithi081 podman[68412]: 2024-04-24 09:20:54.942372224 +0000 UTC m=+5.345612204 container died ad16eaa7ced9008ba4040f4b19af68fe8490df01758e533d01028d8e05ead308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-24T09:20:55.567 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:20:55 smithi081 podman[68412]: 2024-04-24 09:20:55.344852076 +0000 UTC m=+5.748092016 container cleanup ad16eaa7ced9008ba4040f4b19af68fe8490df01758e533d01028d8e05ead308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T09:20:55.567 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:20:55 smithi081 bash[68412]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e 2024-04-24T09:20:55.873 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:20:55 smithi081 podman[68426]: 2024-04-24 09:20:55.621084039 +0000 UTC m=+0.655160895 container remove ad16eaa7ced9008ba4040f4b19af68fe8490df01758e533d01028d8e05ead308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True) 2024-04-24T09:20:55.873 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:20:55 smithi081 systemd[1]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.e.service: Deactivated successfully. 2024-04-24T09:20:55.873 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:20:55 smithi081 systemd[1]: Stopped Ceph mds.e for ef1ec7e8-0217-11ef-bc93-c7b262605968. 2024-04-24T09:20:55.874 INFO:journalctl@ceph.mds.e.smithi081.stdout:Apr 24 09:20:55 smithi081 systemd[1]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.e.service: Consumed 2min 992ms CPU time. 2024-04-24T09:20:55.876 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.e.service' 2024-04-24T09:20:55.985 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:20:55.985 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-24T09:20:55.985 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-24T09:20:55.986 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.h 2024-04-24T09:20:56.567 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:20:56 smithi081 systemd[1]: Stopping Ceph mds.h for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:20:57.067 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:20:56 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:20:56.606+0000 7f84af800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:20:57.067 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:20:56 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h[61133]: 2024-04-24T09:20:56.606+0000 7f84af800700 -1 mds.h *** got signal Terminated *** 2024-04-24T09:21:02.146 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:21:01 smithi081 podman[68532]: 2024-04-24 09:21:01.851900305 +0000 UTC m=+5.423734420 container died e2dbdb3a3aad4d33099296c2ed15f4b6f555f128ec8c027f8f1d7986a48243e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-24T09:21:02.475 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:21:02 smithi081 podman[68532]: 2024-04-24 09:21:02.257591511 +0000 UTC m=+5.829425620 container cleanup e2dbdb3a3aad4d33099296c2ed15f4b6f555f128ec8c027f8f1d7986a48243e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T09:21:02.475 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:21:02 smithi081 bash[68532]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h 2024-04-24T09:21:02.817 INFO:journalctl@ceph.mds.h.smithi081.stdout:Apr 24 09:21:02 smithi081 podman[68546]: 2024-04-24 09:21:02.475656524 +0000 UTC m=+0.609204961 container remove e2dbdb3a3aad4d33099296c2ed15f4b6f555f128ec8c027f8f1d7986a48243e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-24T09:21:02.942 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.h.service' 2024-04-24T09:21:03.041 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:03.041 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-24T09:21:03.041 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-24T09:21:03.041 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.k 2024-04-24T09:21:03.317 INFO:journalctl@ceph.mds.k.smithi081.stdout:Apr 24 09:21:03 smithi081 systemd[1]: Stopping Ceph mds.k for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:21:04.067 INFO:journalctl@ceph.mds.k.smithi081.stdout:Apr 24 09:21:03 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-k[60516]: 2024-04-24T09:21:03.577+0000 7f0168a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:21:04.067 INFO:journalctl@ceph.mds.k.smithi081.stdout:Apr 24 09:21:03 smithi081 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-k[60516]: 2024-04-24T09:21:03.577+0000 7f0168a00700 -1 mds.k *** got signal Terminated *** 2024-04-24T09:21:07.982 INFO:journalctl@ceph.mds.k.smithi081.stdout:Apr 24 09:21:07 smithi081 podman[68651]: 2024-04-24 09:21:07.704207141 +0000 UTC m=+4.258177447 container died ec97d78eb5b7728d88713380c054eb5ba85f3bea6dd6e3e742f0cbccb9cceb92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-24T09:21:08.304 INFO:journalctl@ceph.mds.k.smithi081.stdout:Apr 24 09:21:08 smithi081 podman[68651]: 2024-04-24 09:21:08.094826377 +0000 UTC m=+4.648796640 container cleanup ec97d78eb5b7728d88713380c054eb5ba85f3bea6dd6e3e742f0cbccb9cceb92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-24T09:21:08.304 INFO:journalctl@ceph.mds.k.smithi081.stdout:Apr 24 09:21:08 smithi081 bash[68651]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-k 2024-04-24T09:21:08.567 INFO:journalctl@ceph.mds.k.smithi081.stdout:Apr 24 09:21:08 smithi081 podman[68665]: 2024-04-24 09:21:08.304453354 +0000 UTC m=+0.579885075 container remove ec97d78eb5b7728d88713380c054eb5ba85f3bea6dd6e3e742f0cbccb9cceb92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T09:21:08.851 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.k.service' 2024-04-24T09:21:08.948 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:08.948 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-24T09:21:08.948 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-24T09:21:08.948 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.c 2024-04-24T09:21:09.454 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 24 09:21:09 smithi130 systemd[1]: Stopping Ceph mds.c for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:21:09.954 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 24 09:21:09 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-c[62519]: 2024-04-24T09:21:09.477+0000 7fdf6f800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:21:09.954 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 24 09:21:09 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-c[62519]: 2024-04-24T09:21:09.477+0000 7fdf6f800700 -1 mds.c *** got signal Terminated *** 2024-04-24T09:21:13.664 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 24 09:21:13 smithi130 podman[69890]: 2024-04-24 09:21:13.348713648 +0000 UTC m=+4.006316299 container died 2a62d12c03c39f9e455ef2fa87d5c882afc6472f036e8bbd361440e74c64e08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T09:21:13.953 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 24 09:21:13 smithi130 podman[69890]: 2024-04-24 09:21:13.853713093 +0000 UTC m=+4.511315767 container cleanup 2a62d12c03c39f9e455ef2fa87d5c882afc6472f036e8bbd361440e74c64e08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:21:13.953 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 24 09:21:13 smithi130 bash[69890]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-c 2024-04-24T09:21:14.479 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 24 09:21:14 smithi130 podman[69904]: 2024-04-24 09:21:14.216005524 +0000 UTC m=+0.845589507 container remove 2a62d12c03c39f9e455ef2fa87d5c882afc6472f036e8bbd361440e74c64e08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2) 2024-04-24T09:21:14.507 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.c.service' 2024-04-24T09:21:14.616 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:14.616 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-24T09:21:14.616 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-24T09:21:14.617 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.f 2024-04-24T09:21:15.182 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 24 09:21:14 smithi130 systemd[1]: Stopping Ceph mds.f for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:21:15.454 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 24 09:21:15 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-f[62215]: 2024-04-24T09:21:15.182+0000 7f66cfa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:21:15.454 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 24 09:21:15 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-f[62215]: 2024-04-24T09:21:15.182+0000 7f66cfa00700 -1 mds.f *** got signal Terminated *** 2024-04-24T09:21:17.186 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 24 09:21:16 smithi130 podman[70010]: 2024-04-24 09:21:16.82211166 +0000 UTC m=+1.778330807 container stop 47c7d8f5516209fbc94a98e44461210b3316300ae4b096198fce351786399039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-24T09:21:17.186 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 24 09:21:16 smithi130 podman[70010]: 2024-04-24 09:21:16.852139363 +0000 UTC m=+1.808358454 container died 47c7d8f5516209fbc94a98e44461210b3316300ae4b096198fce351786399039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:21:17.454 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 24 09:21:17 smithi130 podman[70010]: 2024-04-24 09:21:17.298764627 +0000 UTC m=+2.254983715 container cleanup 47c7d8f5516209fbc94a98e44461210b3316300ae4b096198fce351786399039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-24T09:21:17.455 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 24 09:21:17 smithi130 bash[70010]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-f 2024-04-24T09:21:17.954 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 24 09:21:17 smithi130 podman[70034]: 2024-04-24 09:21:17.525544598 +0000 UTC m=+0.685903478 container remove 47c7d8f5516209fbc94a98e44461210b3316300ae4b096198fce351786399039 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-24T09:21:18.013 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.f.service' 2024-04-24T09:21:18.122 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:18.122 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-24T09:21:18.122 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-24T09:21:18.123 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.i 2024-04-24T09:21:18.704 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 24 09:21:18 smithi130 systemd[1]: Stopping Ceph mds.i for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:21:19.204 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 24 09:21:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-i[62826]: 2024-04-24T09:21:18.783+0000 7f60aba00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:21:19.204 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 24 09:21:18 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-i[62826]: 2024-04-24T09:21:18.783+0000 7f60aba00700 -1 mds.i *** got signal Terminated *** 2024-04-24T09:21:21.601 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 24 09:21:21 smithi130 podman[70140]: 2024-04-24 09:21:21.29231993 +0000 UTC m=+2.648097377 container died 4a37e99aaa24d888329c929fa96a5c880b7c48359968a64ff1d17de34496abec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-24T09:21:21.954 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 24 09:21:21 smithi130 podman[70140]: 2024-04-24 09:21:21.750475228 +0000 UTC m=+3.106252679 container cleanup 4a37e99aaa24d888329c929fa96a5c880b7c48359968a64ff1d17de34496abec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-24T09:21:21.954 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 24 09:21:21 smithi130 bash[70140]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-i 2024-04-24T09:21:22.453 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 24 09:21:21 smithi130 podman[70154]: 2024-04-24 09:21:21.986653869 +0000 UTC m=+0.677621997 container remove 4a37e99aaa24d888329c929fa96a5c880b7c48359968a64ff1d17de34496abec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:21:22.496 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.i.service' 2024-04-24T09:21:22.601 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:22.601 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-24T09:21:22.601 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-24T09:21:22.601 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.l 2024-04-24T09:21:23.203 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 24 09:21:22 smithi130 systemd[1]: Stopping Ceph mds.l for ef1ec7e8-0217-11ef-bc93-c7b262605968... 2024-04-24T09:21:23.704 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 24 09:21:23 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-l[61902]: 2024-04-24T09:21:23.212+0000 7f54dfc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:21:23.704 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 24 09:21:23 smithi130 ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-l[61902]: 2024-04-24T09:21:23.212+0000 7f54dfc00700 -1 mds.l *** got signal Terminated *** 2024-04-24T09:21:24.601 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 24 09:21:24 smithi130 podman[70262]: 2024-04-24 09:21:24.325510312 +0000 UTC m=+1.243786435 container died 0ea7a1311dc2e20018728a8402ece7102758cb0ca2a9d747982b6176816926ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-24T09:21:24.948 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 24 09:21:24 smithi130 podman[70262]: 2024-04-24 09:21:24.747315713 +0000 UTC m=+1.665591829 container cleanup 0ea7a1311dc2e20018728a8402ece7102758cb0ca2a9d747982b6176816926ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-24T09:21:24.948 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 24 09:21:24 smithi130 bash[70262]: ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-l 2024-04-24T09:21:25.203 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 24 09:21:24 smithi130 podman[70276]: 2024-04-24 09:21:24.947975533 +0000 UTC m=+0.603860224 container remove 0ea7a1311dc2e20018728a8402ece7102758cb0ca2a9d747982b6176816926ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:21:25.429 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ef1ec7e8-0217-11ef-bc93-c7b262605968@mds.l.service' 2024-04-24T09:21:25.533 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:25.533 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-24T09:21:25.534 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T09:21:25.849 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T09:21:30.676 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T09:21:30.994 INFO:teuthology.orchestra.run.smithi081.stdout:Deleting cluster with fsid: ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T09:21:35.749 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T09:21:36.068 INFO:teuthology.orchestra.run.smithi130.stdout:Deleting cluster with fsid: ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T09:21:40.665 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:21:40.757 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:21:40.844 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:21:40.934 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-24T09:21:40.937 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219/remote/smithi069/crash 2024-04-24T09:21:40.938 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/crash -- . 2024-04-24T09:21:41.074 INFO:teuthology.orchestra.run.smithi069.stderr:tar: /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T09:21:41.074 INFO:teuthology.orchestra.run.smithi069.stderr:tar: Error is not recoverable: exiting now 2024-04-24T09:21:41.079 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219/remote/smithi081/crash 2024-04-24T09:21:41.080 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/crash -- . 2024-04-24T09:21:41.171 INFO:teuthology.orchestra.run.smithi081.stderr:tar: /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T09:21:41.171 INFO:teuthology.orchestra.run.smithi081.stderr:tar: Error is not recoverable: exiting now 2024-04-24T09:21:41.175 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219/remote/smithi130/crash 2024-04-24T09:21:41.176 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/crash -- . 2024-04-24T09:21:41.271 INFO:teuthology.orchestra.run.smithi130.stderr:tar: /var/lib/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T09:21:41.272 INFO:teuthology.orchestra.run.smithi130.stderr:tar: Error is not recoverable: exiting now 2024-04-24T09:21:41.276 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-24T09:21:41.276 DEBUG:teuthology.orchestra.run.smithi069:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-24T09:21:41.382 INFO:teuthology.orchestra.run.smithi069.stdout:2024-04-24T09:09:28.574296+0000 mds.e (mds.2) 2 : cluster [WRN] Scrub error on inode 0x10000000334 (/volumes/qa/sv_0/65ebee0b-37c1-4c5d-bbf7-ad639a509f68/client.0/tmp/xfstests-dev/src/md5.c) see mds.e log and `damage ls` output for details 2024-04-24T09:21:41.383 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-24T09:21:41.383 INFO:tasks.cephadm:Compressing logs... 2024-04-24T09:21:41.383 DEBUG:teuthology.orchestra.run.smithi069:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T09:21:41.388 DEBUG:teuthology.orchestra.run.smithi081:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T09:21:41.396 DEBUG:teuthology.orchestra.run.smithi130:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T09:21:41.449 INFO:teuthology.orchestra.run.smithi069.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.log 2024-04-24T09:21:41.449 INFO:teuthology.orchestra.run.smithi069.stderr:: No such file or directory 2024-04-24T09:21:41.449 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T09:21:41.450 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-24T09:21:41.451 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-24T09:21:41.452 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-24T09:21:41.454 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-24T09:21:41.455 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.3.log: 89.2% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T09:21:41.455 INFO:teuthology.orchestra.run.smithi069.stderr: 90.1% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T09:21:41.455 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-24T09:21:41.457 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-24T09:21:41.457 INFO:teuthology.orchestra.run.smithi069.stderr: 94.6% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-24T09:21:41.457 INFO:teuthology.orchestra.run.smithi069.stderr: 94.5% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-24T09:21:41.458 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-24T09:21:41.458 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T09:21:41.460 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-24T09:21:41.462 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T09:21:41.462 INFO:teuthology.orchestra.run.smithi069.stderr: 84.0% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T09:21:41.463 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-24T09:21:41.463 INFO:teuthology.orchestra.run.smithi081.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.log 2024-04-24T09:21:41.463 INFO:teuthology.orchestra.run.smithi081.stderr:: No such file or directory 2024-04-24T09:21:41.464 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T09:21:41.465 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-24T09:21:41.466 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-24T09:21:41.467 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-24T09:21:41.468 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-24T09:21:41.469 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-24T09:21:41.469 INFO:teuthology.orchestra.run.smithi069.stderr: 94.5% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-24T09:21:41.470 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.b.log: 90.3% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T09:21:41.470 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-24T09:21:41.470 INFO:teuthology.orchestra.run.smithi081.stderr: 89.3% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T09:21:41.470 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-24T09:21:41.471 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T09:21:41.471 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-24T09:21:41.473 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-24T09:21:41.474 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-24T09:21:41.476 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.h.log: /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T09:21:41.478 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-24T09:21:41.479 INFO:teuthology.orchestra.run.smithi130.stderr:find: ‘/var/log/rbd-target-api’gzip: No such file or directory 2024-04-24T09:21:41.479 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.log 2024-04-24T09:21:41.481 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T09:21:41.481 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-24T09:21:41.481 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.log: 90.9% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T09:21:41.481 INFO:teuthology.orchestra.run.smithi069.stderr: 94.4% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-24T09:21:41.483 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-24T09:21:41.483 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-24T09:21:41.483 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T09:21:41.484 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-24T09:21:41.484 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-volume.log: 92.0% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-24T09:21:41.484 INFO:teuthology.orchestra.run.smithi081.stderr: 83.7% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T09:21:41.485 INFO:teuthology.orchestra.run.smithi130.stderr: 89.2% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T09:21:41.485 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-24T09:21:41.486 INFO:teuthology.orchestra.run.smithi069.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-24T09:21:41.486 INFO:teuthology.orchestra.run.smithi069.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T09:21:41.486 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-24T09:21:41.487 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-24T09:21:41.488 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.c.log: 90.3% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T09:21:41.488 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-24T09:21:41.488 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mon.b.log: /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T09:21:41.489 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-24T09:21:41.490 INFO:teuthology.orchestra.run.smithi130.stderr: 94.5% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-24T09:21:41.491 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T09:21:41.492 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-24T09:21:41.494 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T09:21:41.494 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-24T09:21:41.497 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.11.log: /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T09:21:41.497 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/cephadm.log: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T09:21:41.502 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-24T09:21:41.504 INFO:teuthology.orchestra.run.smithi130.stderr: 92.0% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-24T09:21:41.504 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T09:21:41.513 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T09:21:41.515 INFO:teuthology.orchestra.run.smithi081.stderr: 90.9% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T09:21:41.521 INFO:teuthology.orchestra.run.smithi130.stderr: 94.6% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-24T09:21:41.523 INFO:teuthology.orchestra.run.smithi130.stderr: 90.9% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T09:21:41.876 INFO:teuthology.orchestra.run.smithi069.stderr: 90.1% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-24T09:21:41.965 INFO:teuthology.orchestra.run.smithi081.stderr: 92.1% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-24T09:21:41.973 INFO:teuthology.orchestra.run.smithi130.stderr: 91.9% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-24T09:21:42.892 INFO:teuthology.orchestra.run.smithi069.stderr: 90.5% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-24T09:21:43.152 INFO:teuthology.orchestra.run.smithi130.stderr: 95.0% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-24T09:21:43.224 INFO:teuthology.orchestra.run.smithi081.stderr: 94.8% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-24T09:21:44.139 INFO:teuthology.orchestra.run.smithi130.stderr: 94.9% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-24T09:21:45.523 INFO:teuthology.orchestra.run.smithi130.stderr: 91.4% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-24T09:21:45.841 INFO:teuthology.orchestra.run.smithi069.stderr: 91.6% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-24T09:21:46.124 INFO:teuthology.orchestra.run.smithi130.stderr: 91.5% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-24T09:21:46.495 INFO:teuthology.orchestra.run.smithi130.stderr: 91.6% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-24T09:21:47.453 INFO:teuthology.orchestra.run.smithi069.stderr: 91.7% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-24T09:21:47.603 INFO:teuthology.orchestra.run.smithi069.stderr: 91.4% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-24T09:21:47.824 INFO:teuthology.orchestra.run.smithi081.stderr: 91.5% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-24T09:21:48.144 INFO:teuthology.orchestra.run.smithi069.stderr: 91.3% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-24T09:21:48.372 INFO:teuthology.orchestra.run.smithi081.stderr: 91.6% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-24T09:21:48.733 INFO:teuthology.orchestra.run.smithi130.stderr: 91.6% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-24T09:21:48.737 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-04-24T09:21:48.738 INFO:teuthology.orchestra.run.smithi130.stderr:real 0m7.289s 2024-04-24T09:21:48.738 INFO:teuthology.orchestra.run.smithi130.stderr:user 0m23.967s 2024-04-24T09:21:48.738 INFO:teuthology.orchestra.run.smithi130.stderr:sys 0m1.885s 2024-04-24T09:21:48.900 INFO:teuthology.orchestra.run.smithi081.stderr: 91.8% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-24T09:21:50.140 INFO:teuthology.orchestra.run.smithi081.stderr: 91.3% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-24T09:21:51.517 INFO:teuthology.orchestra.run.smithi069.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-24T09:21:51.521 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-04-24T09:21:51.521 INFO:teuthology.orchestra.run.smithi069.stderr:real 0m10.098s 2024-04-24T09:21:51.521 INFO:teuthology.orchestra.run.smithi069.stderr:user 0m32.717s 2024-04-24T09:21:51.522 INFO:teuthology.orchestra.run.smithi069.stderr:sys 0m2.514s 2024-04-24T09:21:54.986 INFO:teuthology.orchestra.run.smithi081.stderr: 93.1% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-24T09:21:56.957 INFO:teuthology.orchestra.run.smithi081.stderr: 93.5% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-24T09:22:06.439 INFO:teuthology.orchestra.run.smithi081.stderr: 92.8% -- replaced with /var/log/ceph/ef1ec7e8-0217-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-24T09:22:06.444 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-04-24T09:22:06.444 INFO:teuthology.orchestra.run.smithi081.stderr:real 0m25.007s 2024-04-24T09:22:06.444 INFO:teuthology.orchestra.run.smithi081.stderr:user 1m17.793s 2024-04-24T09:22:06.444 INFO:teuthology.orchestra.run.smithi081.stderr:sys 0m5.596s 2024-04-24T09:22:06.445 INFO:tasks.cephadm:Archiving logs... 2024-04-24T09:22:06.446 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219/remote/smithi069/log 2024-04-24T09:22:06.447 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T09:22:11.224 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219/remote/smithi081/log 2024-04-24T09:22:11.225 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T09:22:21.284 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219/remote/smithi130/log 2024-04-24T09:22:21.285 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T09:22:24.702 INFO:tasks.cephadm:Removing cluster... 2024-04-24T09:22:24.703 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 --force 2024-04-24T09:22:25.018 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T09:22:25.631 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 --force 2024-04-24T09:22:25.952 INFO:teuthology.orchestra.run.smithi081.stdout:Deleting cluster with fsid: ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T09:22:26.767 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ef1ec7e8-0217-11ef-bc93-c7b262605968 --force 2024-04-24T09:22:27.074 INFO:teuthology.orchestra.run.smithi130.stdout:Deleting cluster with fsid: ef1ec7e8-0217-11ef-bc93-c7b262605968 2024-04-24T09:22:27.757 INFO:tasks.cephadm:Removing cephadm ... 2024-04-24T09:22:27.758 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T09:22:27.814 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T09:22:27.871 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T09:22:27.929 INFO:tasks.cephadm:Teardown complete 2024-04-24T09:22:27.929 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T09:22:27.942 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-04-24T09:22:27.944 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T09:22:27.944 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T09:22:27.952 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T09:22:27.959 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T09:22:28.124 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T09:22:28.135 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T09:22:28.136 DEBUG:teuthology.orchestra.run.smithi069:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T09:22:28.144 DEBUG:teuthology.orchestra.run.smithi081:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T09:22:28.152 DEBUG:teuthology.orchestra.run.smithi130:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T09:22:28.189 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-04-24T09:22:28.198 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-04-24T09:22:28.198 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T09:22:28.198 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-04-24T09:22:28.199 INFO:teuthology.orchestra.run.smithi069.stdout:^- hv01.front.sepia.ceph.com 2 6 377 1 +1648us[+1648us] +/- 568ms 2024-04-24T09:22:28.199 INFO:teuthology.orchestra.run.smithi069.stdout:^* hv02.front.sepia.ceph.com 3 6 377 61 -340us[ -272us] +/- 78ms 2024-04-24T09:22:28.199 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +305us[ +373us] +/- 60ms 2024-04-24T09:22:28.199 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:22:28.207 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T09:22:28.207 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-04-24T09:22:28.207 INFO:teuthology.orchestra.run.smithi081.stdout:^- hv01.front.sepia.ceph.com 2 6 377 64 +1700us[+1700us] +/- 599ms 2024-04-24T09:22:28.207 INFO:teuthology.orchestra.run.smithi081.stdout:^* hv02.front.sepia.ceph.com 3 7 377 64 -227us[ -226us] +/- 78ms 2024-04-24T09:22:28.207 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 121 +415us[ +417us] +/- 92ms 2024-04-24T09:22:28.208 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:22:28.209 INFO:teuthology.orchestra.run.smithi130.stderr:bash: line 1: ntpq: command not found 2024-04-24T09:22:28.218 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T09:22:28.218 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2024-04-24T09:22:28.218 INFO:teuthology.orchestra.run.smithi130.stdout:^- hv01.front.sepia.ceph.com 2 6 377 62 +1647us[+1700us] +/- 599ms 2024-04-24T09:22:28.218 INFO:teuthology.orchestra.run.smithi130.stdout:^* hv02.front.sepia.ceph.com 3 6 377 62 -297us[ -245us] +/- 78ms 2024-04-24T09:22:28.218 INFO:teuthology.orchestra.run.smithi130.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 +315us[ +368us] +/- 60ms 2024-04-24T09:22:28.218 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:22:28.220 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T09:22:28.230 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T09:22:28.231 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T09:22:28.268 DEBUG:teuthology.orchestra.run.smithi069:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T09:22:28.454 DEBUG:teuthology.orchestra.run.smithi081:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T09:22:28.645 DEBUG:teuthology.orchestra.run.smithi130:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T09:22:28.839 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T09:22:28.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:22:28.935 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T09:22:29.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:22:29.028 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T09:22:29.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:22:29.123 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T09:22:29.137 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T09:22:29.148 INFO:teuthology.task.internal:Duration was 2320.015328 seconds 2024-04-24T09:22:29.148 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T09:22:29.159 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T09:22:29.159 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T09:22:29.167 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T09:22:29.175 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T09:22:29.289 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T09:22:29.305 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T09:22:29.313 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T09:22:29.684 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T09:22:29.685 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi069.front.sepia.ceph.com 2024-04-24T09:22:29.685 DEBUG:teuthology.orchestra.run.smithi069:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T09:22:29.750 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi081.front.sepia.ceph.com 2024-04-24T09:22:29.751 DEBUG:teuthology.orchestra.run.smithi081:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T09:22:29.822 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi130.front.sepia.ceph.com 2024-04-24T09:22:29.822 DEBUG:teuthology.orchestra.run.smithi130:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T09:22:29.896 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T09:22:29.896 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T09:22:29.904 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T09:22:29.911 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T09:22:29.993 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T09:22:29.993 DEBUG:teuthology.orchestra.run.smithi069:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T09:22:29.998 DEBUG:teuthology.orchestra.run.smithi081:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T09:22:30.004 DEBUG:teuthology.orchestra.run.smithi130:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T09:22:30.506 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T09:22:30.518 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T09:22:30.518 DEBUG:teuthology.orchestra.run.smithi069:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T09:22:30.596 DEBUG:teuthology.orchestra.run.smithi081:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T09:22:30.683 DEBUG:teuthology.orchestra.run.smithi130:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T09:22:30.775 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T09:22:30.787 DEBUG:teuthology.orchestra.run.smithi069:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:22:30.795 DEBUG:teuthology.orchestra.run.smithi081:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:22:30.803 DEBUG:teuthology.orchestra.run.smithi130:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:22:30.886 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = core 2024-04-24T09:22:30.893 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = core 2024-04-24T09:22:30.900 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = core 2024-04-24T09:22:30.960 DEBUG:teuthology.orchestra.run.smithi069:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:22:31.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:22:31.011 DEBUG:teuthology.orchestra.run.smithi081:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:22:31.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:22:31.063 DEBUG:teuthology.orchestra.run.smithi130:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:22:31.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:22:31.116 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T09:22:31.128 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T09:22:31.128 DEBUG:teuthology.misc:Transferring archived files from smithi069:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219/remote/smithi069 2024-04-24T09:22:31.129 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T09:22:31.232 DEBUG:teuthology.misc:Transferring archived files from smithi081:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219/remote/smithi081 2024-04-24T09:22:31.232 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T09:22:31.330 DEBUG:teuthology.misc:Transferring archived files from smithi130:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219/remote/smithi130 2024-04-24T09:22:31.331 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T09:22:31.432 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T09:22:31.433 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T09:22:31.441 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T09:22:31.448 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T09:22:31.505 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T09:22:31.519 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T09:22:31.520 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T09:22:31.532 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T09:22:31.532 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T09:22:31.540 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T09:22:31.547 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T09:22:31.590 INFO:teuthology.orchestra.run.smithi069.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 09:22 /home/ubuntu/cephtest 2024-04-24T09:22:31.595 INFO:teuthology.orchestra.run.smithi081.stdout: 393430 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 09:22 /home/ubuntu/cephtest 2024-04-24T09:22:31.604 INFO:teuthology.orchestra.run.smithi130.stdout: 393949 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 09:22 /home/ubuntu/cephtest 2024-04-24T09:22:31.608 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T09:22:31.620 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T09:22:31.633 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-04-24T09:22:31.633 INFO:teuthology.nuke:Checking targets against current locks 2024-04-24T09:22:31.660 DEBUG:teuthology.nuke:shortname: smithi069 2024-04-24T09:22:31.660 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T09:22:31.681 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:35:17.219894', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMTekBTAJqh8wjIKHR5OeiLPqHY0gpHPKpuaKOns8SdwT/9OKNKbSaAlp+a2yYZff1hW/3YAqpIbFrL2RxJEsDA='} 2024-04-24T09:22:31.686 DEBUG:teuthology.nuke:shortname: smithi081 2024-04-24T09:22:31.686 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T09:22:31.707 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:35:17.221574', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHPYstxFn2yzqm8aijm4VdNHZ2Zm1zGkPkXcj8JTJUD+UotBcYC47p9JDaP4+eW0AsbJRVUlc24TUeqCXe5PYuY='} 2024-04-24T09:22:31.716 DEBUG:teuthology.nuke:shortname: smithi130 2024-04-24T09:22:31.716 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T09:22:31.728 INFO:teuthology.orchestra.console.smithi069:Power off 2024-04-24T09:22:31.728 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T09:22:31.739 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671219', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:35:17.223487', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLDmN90dI9z7y6K6jl3Pa8evR6qPIeIb6Qvf2Zf8iOAQQOdC+qA0X8neB1mDK31WocyDvMaoXme0yEGVs5cUL5w='} 2024-04-24T09:22:31.751 DEBUG:teuthology.orchestra.console.smithi069:power off output: Chassis Power Control: Down/Off 2024-04-24T09:22:31.751 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:22:31.772 INFO:teuthology.orchestra.console.smithi081:Power off 2024-04-24T09:22:31.772 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T09:22:31.780 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-04-24T09:22:31.787 INFO:teuthology.orchestra.console.smithi130:Power off 2024-04-24T09:22:31.787 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T09:22:31.797 DEBUG:teuthology.orchestra.console.smithi081:power off output: Chassis Power Control: Down/Off 2024-04-24T09:22:31.797 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:22:31.809 DEBUG:teuthology.orchestra.console.smithi130:power off output: Chassis Power Control: Down/Off 2024-04-24T09:22:31.809 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:22:31.819 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on 2024-04-24T09:22:31.831 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2024-04-24T09:22:35.781 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:22:35.820 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:22:35.832 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:22:35.892 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-04-24T09:22:35.929 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on 2024-04-24T09:22:35.942 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2024-04-24T09:22:39.897 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:22:39.930 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:22:39.944 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:22:40.007 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off 2024-04-24T09:22:40.038 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on 2024-04-24T09:22:40.052 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2024-04-24T09:22:40.108 INFO:teuthology.orchestra.console.smithi069:Power off completed 2024-04-24T09:22:44.043 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:22:44.053 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:22:44.154 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is off 2024-04-24T09:22:44.165 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is off 2024-04-24T09:22:44.256 INFO:teuthology.orchestra.console.smithi081:Power off completed 2024-04-24T09:22:44.266 INFO:teuthology.orchestra.console.smithi130:Power off completed 2024-04-24T09:22:44.375 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} duration: 2320.0153279304504 failure_reason: 'Command failed (workunit test suites/fsx.sh) on smithi069 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1f81fb397ae98da7563d451a78c61574c8f4e6e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh''' flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.c-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=b41adb3f1d194a7e85a4978d6ee2bb6e status: fail success: false 2024-04-24T09:22:44.375 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T09:22:44.456 INFO:teuthology.run:FAIL